Adds a configurable Hugging Face model download cache location to Unsloth Studio, selectable from Settings, with per-cache download manifests, scoped deletion, and read-only inventory of previously selected caches.
* Studio: reject whitespace-only passwords
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: reject any whitespace in passwords
* Studio: surface whitespace error in setup form, isolate auth test import
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Studio: clear composer draft on send
* Studio: verify the Cloudflare link is reachable before printing it
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: wait for tunnel DNS propagation before verifying the public URL
* Studio: bound tunnel DNS wait and health probe by one deadline
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: keep composer draft when overlay send validation fails
* Studio: retry transient DoH failures while waiting for tunnel DNS
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: mask AMD GPU pins via ROCR so an unsupported iGPU can't crash llama-server
On a mixed AMD host (e.g. a discrete gfx1102 GPU next to a gfx1103 iGPU)
the bundled rocm-gfx110X llama.cpp build segfaults during HSA device
enumeration on the unsupported iGPU -- before llama-server prints a line,
so every model load fails with a bare signal and empty logs.
The GPU-subset pin masked visibility with HIP_VISIBLE_DEVICES, but HIP
filtering runs only after the HSA runtime has already enumerated (and
crashed on) every agent. Mask the subset via ROCR_VISIBLE_DEVICES (the
ROCr/HSA layer) instead, so a deselected/unsupported GPU is never
enumerated. Exactly one layer is masked (HIP cleared) to avoid the
double-mask reindex that would otherwise drop the child to CPU. The
whole-set tensor-split path and the CPU-only sentinel keep their existing
HIP behavior.
Also stop misreporting the resulting startup segfault as a vision
projector incompatibility: when the text-only mmproj retry also hard-
crashes with a signal, surface a GPU/driver init crash (with the ROCR
hint) instead of blaming the projector.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten _emit_child_gpu_visibility comments for #7272
Comment and docstring only: condense the ROCR-vs-HIP masking rationale from ~22 to ~14 lines and the call-site note from 5 to 3, keeping every technical point (HSA enumeration segfault, physical ids, the -1 sentinel). Logic is unchanged, verified by an AST compare with docstrings stripped and by exercising _emit_child_gpu_visibility against a torch/HIP stub.
* Detect AMD SDK ROCm wheels (hip=None) in _emit_child_gpu_visibility (Codex P2)
The ROCm branch gated only on torch.version.hip, but AMD SDK wheels leave that unset while encoding 'rocm' in __version__ (detect_hardware handles this the same way). On such a wheel the masking was skipped entirely, leaving only CUDA_VISIBLE_DEVICES, so on a mixed AMD box the unsupported deselected iGPU still enumerated and could crash llama-server. Now the branch also treats 'rocm' in torch.__version__ as ROCm, mirroring detect_hardware. Adds tests for the hip=None SDK wheel (ROCR + default paths) and a CUDA guard so the version-string check can't false-positive.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Remap CUDA_VISIBLE_DEVICES to post-ROCR ordinals on prefer_rocr for PR #7272 (Codex P1)
On the prefer_rocr path _emit_child_gpu_visibility set ROCR_VISIBLE_DEVICES to the
physical id and cleared HIP_VISIBLE_DEVICES, but left CUDA_VISIBLE_DEVICES at the
physical id. ROCR re-indexes the visible agents from 0 and, with HIP cleared, HIP
honours CUDA_VISIBLE_DEVICES -- so a non-zero pick (e.g. GPU 1) pointed out of
range, HIP saw 0 devices, and the child fell back to CPU, defeating GPU-picker
selections other than physical GPU 0. Remap CUDA to the post-ROCR ordinals
(0..N-1); GPU 0 is unchanged, the default (HIP) path and the CPU sentinel are
untouched, and non-AMD wheels never enter this branch.
* Detect AMD SDK wheels in _resolve_visible_physical_ids for PR #7272 (Codex P2)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Keep the HIP mask on Windows ROCm in prefer_rocr for PR #7272 (Codex P2)
* Ignore ROCR_VISIBLE_DEVICES in _resolve_visible_physical_ids on Windows for PR #7272 (Codex P2)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Preserve inherited ROCR masks in the tensor-split pin for PR #7272 (Codex P2)
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Leo Borcherding <borchborchmail@gmail.com>
* studio: classify embedding models from the HF cache and honor offline mode
is_embedding_model() went straight to huggingface_hub.model_info() for any repo
id, so in offline mode (no DNS, or HF_HUB_OFFLINE set) selecting an
already-downloaded model hung on network retries that could never succeed and
training/export never started (#6817).
Check the local HF cache first: a sentence-transformers repo carries
modules.json in its snapshot (the same marker used for local paths), so a cached
model is classified with no network call. When HF_HUB_OFFLINE / TRANSFORMERS_OFFLINE
is set, anything not positively an embedding model returns False without a
network call instead of retrying a doomed request. Online, uncached lookups still
fall through to model_info(), so tag-only embedding models (feature-extraction)
are unaffected.
Adds _embedding_marker_in_hf_cache() over the existing _iter_hf_cache_snapshots.
* studio: judge the active cached revision, harden the cache probe, stop stub leaks
Three review fixes on the cache-first embedding detection:
1. Prefer the revision refs/main resolves to. The HF cache keeps snapshots of
older revisions, so an any-snapshot scan could classify a repo by a stale
revision -- e.g. a repo that used to be a sentence-transformers model would
short-circuit even the online lookup. When refs/main is recorded, only its
snapshot is consulted; the newest-first scan remains the fallback for caches
with no ref.
2. Keep the cache probe inside the detection error boundary. The snapshot
iterator stat()s entries and could raise if a cached model is deleted
concurrently, propagating a 500 out of the config/check-embedding routes.
_embedding_marker_in_hf_cache now catches everything and reads as
not-cached, so callers keep their normal Hub/offline fallback.
3. Stub loggers/structlog in the test only when the real modules are absent
(try-import, mirroring test_windows_gpu_detection_mock), so collecting this
file first can no longer shadow the real packages for later tests in the
same pytest process.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: treat a missing active-ref snapshot as a cache miss, don't cache offline misses
Two review fixes on the cache-first embedding detection:
1. When refs/main is recorded but points at a commit whose snapshot dir is
absent (partial download / cache pruning), the recorded ref is still
authoritative: return None (cache miss) instead of falling through to scan
older snapshots, which could report a stale historical revision's
modules.json as the active one -- the same stale-cache class this helper
avoids.
2. Do not cache the offline negative. When HF_HUB_OFFLINE/TRANSFORMERS_OFFLINE
is set and the repo is not positively an ST model from modules.json,
is_embedding_model stored False under the (model_name, hf_token) key shared
with online lookups; after the env var cleared in the same process, a
tag-only (feature-extraction) embedder returned the cached False and never
reached model_info(). The offline negative is now returned without caching.
* studio: defer online embedding detection to the Hub, re-probe offline
The local modules.json marker short-circuited is_embedding_model() even
online, so a repo that dropped (or added) the marker since it was cached
was judged by its stale local revision instead of the current remote one.
Online now treats model_info() as authoritative and uses the cache marker
only as an uncached fallback when the Hub is unreachable, so a transient
failure never poisons the memo. Offline re-probes the marker on every call
without consulting or populating the memo, so a model downloaded later in
the session (or a cached online negative that predates the download) is
detected. _embedding_marker_in_hf_cache() now treats an unreadable refs/main
(a non-FileNotFoundError OSError) as a cache miss rather than scanning stale
history -- only a genuinely missing ref enables the fallback scan.
* studio: harden offline embedding detection against empty refs, offline flips, and cache casing
- _embedding_marker_in_hf_cache: an existing-but-empty/whitespace refs/main
(a partial write or in-progress truncate-and-rewrite) now reads as a cache
miss (None) instead of falling through to scan stale snapshots; only a
genuinely missing ref enables the historical scan.
- is_embedding_model: while offline, retain a positive already confirmed online
this session (model_info only ever memoizes Hub-derived results), so
_hf_offline_if_dns_dead() flipping the process to offline mid-load can't
downgrade a verified tag-only embedder to False. Cached negatives are still
bypassed and re-probed.
- resolve_cached_repo_casing + settings route: persist the embedding model in
the casing its local HF cache dir uses. Validation accepts a case-insensitive
cache hit, but an offline SentenceTransformer load resolves the cache by exact
case, so storing the requested spelling (baai/bge-m3 vs models--BAAI--bge-m3)
made the model fail to load on a case-sensitive filesystem.
* studio: reuse the exact-match-first case resolver and preserve the default
Replace the ad-hoc resolve_cached_repo_casing with the existing
resolve_cached_repo_id_case, which already prefers the exact-case cache dir
before any case variant and tie-breaks variants deterministically -- so an
exact requested id is never rewritten to a differently cased directory just
because iterdir() happened to yield it first.
Skip the normalization entirely when the submitted model equals the default:
rewriting its casing would make set_rag_embedding_model()'s exact-string
default comparison treat it as a custom override, pinning it so later changes
to the configured default stop taking effect.
* studio: don't let a stale cache marker mask a permanent Hub error
is_embedding_model's Hub-failure fallback consulted the local modules.json
marker for ANY model_info() exception, so a permanent error -- a deleted repo,
a gated repo without credentials, or a typo that matches stale cache casing --
could pass online validation on a stale marker instead of returning the
documented 409, and the persisted model could then fail when the loader
refreshes from the Hub. Classify permanent Hub errors (RepositoryNotFound,
GatedRepo, RevisionNotFound, EntryNotFound) as False, matching the nearby
GGUF/vision detectors, and reserve the cache fallback for transient/5xx failures.
* studio: honor TRANSFORMERS_OFFLINE in the embedding preflight, skip casing for local paths
- The embedding-model save reached the offline-aware is_embedding_model() only
after two preflight helpers made direct huggingface_hub calls that honor just
HF_HUB_OFFLINE: _st_module_subdirs() downloads modules.json and the security
scan fetches Hub metadata twice. In a TRANSFORMERS_OFFLINE-only session those
blocked on network timeouts before the offline return, so saving an already
cached model stalled. Both now consult a canonical hf_env_offline() helper --
the download passes local_files_only, and the metadata-only security scan
short-circuits to its documented fail-open instead of burning both timeouts.
- Skip cache-casing normalization for local paths: a relative directory such as
"org/model" is loaded from disk, so rewriting it to a case-insensitive HF
cache collision ("Org/model") would stop resolving to that directory and be
read as a Hub repo id instead.
* studio: never skip the security scan on TRANSFORMERS_OFFLINE alone
The previous commit skipped the Hub security scan whenever either offline flag
was set, but huggingface_hub honors only HF_HUB_OFFLINE: under a
TRANSFORMERS_OFFLINE-only session the later SentenceTransformer load still
reaches the network, so the scan was being skipped while the repo's pickle could
still be downloaded and deserialized -- waving through exactly what
_guard_model_security exists to block.
Split the flags: hf_hub_offline() (HF_HUB_OFFLINE, the only one that actually
prevents a fetch) gates the security short-circuit, while hf_env_offline()
(either flag, the user's intent) is used only where local-only behavior is
forced explicitly. The SentenceTransformer load now passes local_files_only from
that intent, so TRANSFORMERS_OFFLINE genuinely stops the loader fetching instead
of merely being assumed to.
* studio: short-circuit the security preflight under either offline flag
With the loader now pinned to the local cache by local_files_only =
hf_env_offline(), a TRANSFORMERS_OFFLINE-only session can no longer fetch
anything -- yet the preflight still fell through to two model_info() attempts on
10s and 20s timeouts, stalling every save and load of an already-cached embedder
for half a minute before failing open anyway.
Skip the metadata-only scan whenever either flag is set. The scan's job is to
stop a poisoned pickle being downloaded and deserialized, and nothing can be
downloaded under that predicate; the residual case -- a model cached BEFORE it
was flagged -- is the same fail-open this function has always documented for an
unavailable scan, and is exactly what HF_HUB_OFFLINE already did.
That safety argument depends on every loader behind the gate honoring the same
predicate, so it is pinned as a test invariant instead of a comment: removing
local_files_only from the SentenceTransformer construction now fails the suite.
Drops the short-lived hf_hub_offline() helper, which no longer has a caller.
* studio: scope the offline scan bypass to callers that load local-only
The previous commit put the offline short-circuit inside _fetch_security_status,
which is the malware gate shared by every loader -- so TRANSFORMERS_OFFLINE=1
disabled it for all of them, while only the RAG embedder had been changed to
pass local_files_only. MLX inference (core/inference/worker.py -> FastMLXModel
.from_pretrained), training and export call from_pretrained with no local-only
argument, and huggingface_hub ignores that flag, so those paths could still
fetch and deserialize an unscanned model with the gate switched off.
The bypass is now an explicit local_only_load argument, defaulting to False, and
only the two RAG embedding callers -- whose loader is pinned to the local cache
by the same predicate -- opt in. Tests pin both halves: the shared gate must
still scan under either offline flag by default, and no other caller may pass
local_only_load without constraining its loader.
* studio: capture offline state once, and probe the ST cache root
Two holes in the offline embedding path:
- _get() read hf_env_offline() twice: once inside _guard_model_security and
again for local_files_only. _hf_offline_if_dns_dead() mutates the process-wide
offline vars and restores them on exit, so a concurrent load could see True in
the guard -- skipping the Hub malware scan -- and False by the time the
constructor ran, fetching and deserializing the unscanned repo and breaking
the very invariant that licenses the bypass. The value is now read once in
_get() and passed to both; _guard_model_security takes it as an argument
instead of re-deriving it.
- The cache probe searched only HF_HUB_CACHE. SentenceTransformer downloads into
SENTENCE_TRANSFORMERS_HOME when that is set, using the same
models--org--name/snapshots layout under a different root, so a model fully
present there looked uncached and was rejected with a 409 offline even though
the local-only loader could load it. Snapshot lookup now covers both roots.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: probe the cache the ST loader actually uses, and require it be loadable
Adding SENTENCE_TRANSFORMERS_HOME to the shared snapshot iterator was too broad
in one direction and too narrow in another:
- _get() builds SentenceTransformer with no cache_folder, so with ST_HOME set it
searches THAT root only, never the Hub cache. Probing the union let offline
validation pass on a repo cached only in the Hub cache, after which the loader
looked in ST_HOME and failed. The Sentence-Transformers probe now resolves to
exactly one root: ST_HOME when set, the Hub cache otherwise.
- The shared iterator is also used by the GGUF detectors, whose downloads go
through hf_hub_download with no cache_dir and therefore really do use the Hub
cache. It is back to Hub-cache-only so detection cannot pick a snapshot the
GGUF load will not find.
- Casing normalization ran through resolve_cached_repo_id_case, which scans the
Hub cache, so with ST_HOME set the requested spelling was persisted unchanged
and the exact-case offline load missed the differently cased directory that
detection had just accepted. It now resolves against the same roots detection
uses, exact match first.
- A snapshot carrying only modules.json no longer counts as cached: the online
security preflight downloads that single file itself, and a partial download
leaves it behind, so validation passed for a snapshot with no weights and the
first RAG load then failed. A hit now requires the marker plus a config and at
least one weight file.
* studio: thread the captured offline state into the module probe, fix the gate shard
- _st_module_subdirs() re-read the process env for its local_files_only. With
_hf_offline_if_dns_dead() flipping those vars from another thread, a load that
captured local_only=False could still force this probe local-only, get () back
because modules.json is not cached, and leave the scan with NO module load
roots -- a Hub-flagged pickle under 0_Transformer/ would then pass as an
unreferenced nested artifact while the loader fetched and deserialized it. It
now takes the captured predicate as an argument, and the settings route reads
the state once and uses that single value for both the probe and the scan.
- Skip ST-cache casing on the llama-server backend. Nothing there loads through
SentenceTransformer: the embedder derives a GGUF companion from the saved
spelling and fetches it from the HUB cache, so normalizing to an ST_HOME
spelling would point it at a repo _hf_gguf_backend_error() never validated
(BAAI/bge-m3-GGUF instead of the checked baai/bge-m3-GGUF).
- Fix the security-gate shard, which the signature change had broken: the direct
_guard_model_security / _st_module_subdirs callers now pass the new argument
(they were raising TypeError before reaching any assertion), and the casing
tests patch utils.models.resolve_st_cached_repo_id_case, which the route
actually calls, instead of the Hub-only resolver it no longer uses -- those
patches were being silently ignored.
* studio: accept only torch-loadable weights in the offline ST probe; fix re-export lint
_snapshot_is_loadable_st_model accepted a cached snapshot whose only weights
were .onnx (or .pt), but the RAG loader builds SentenceTransformer with the
default torch backend, so such a snapshot passed offline validation and then
failed on the first load, the exact validate-then-fail this helper exists to
prevent. Restrict _ST_WEIGHT_SUFFIXES to .safetensors and .bin and add a
regression test for an ONNX-only snapshot.
Also teach scripts/verify_import_hoist.py that names listed in a module-level
__all__ are uses, so the legitimately added resolve_st_cached_repo_id_case
re-export in utils/models/__init__.py no longer trips HOISTED-IMPORT-UNUSED.
Covered by two new self-test cases.
* studio: probe the exact repo dir and revision an offline load resolves
The cache probe modelled the cache loosely rather than modelling what
SentenceTransformer actually does with local_files_only=True:
- It merged snapshots across every case-variant repo dir and then read refs/main
from whichever held the newest one. With both models--baai--bge-m3 and
models--BAAI--bge-m3 present, a complete embedding snapshot in the directory
the loader opens could be judged by a newer partial snapshot in the other,
failing validation for a usable model. It now selects the ONE directory the
loader opens, by the same exact-case-first rule resolve_st_cached_repo_id_case
uses to choose the spelling that gets persisted.
- It fell back to scanning historical snapshots when refs/main was absent. With
local_files_only the default revision is resolved THROUGH that ref, so a
snapshot directory alone is not discoverable: the settings request succeeded
and the loader then failed at first indexing. A missing, empty or unreadable
ref is now a cache miss, and the historical scan is gone.
The tests exercise the real lookup against a built cache tree instead of
patching the snapshot iterator, so they now cover the directory selection and
ref resolution the loader depends on.
* studio: record refs/main in the ONNX-only probe test
The ONNX-only regression test predates the refs/main requirement, so after that
change it returned None (a cache miss for want of a ref) before ever reaching
the weight-format check it exists to make. Recording the ref restores its
intent: the snapshot resolves, and the answer is False because an ONNX export is
not loadable by the RAG loader's default Torch backend.
* studio: recognize base-model weight files and gate the offline positive on a materialized snapshot
_snapshot_is_loadable_st_model matched any .safetensors/.bin by suffix, so a
partial cache carrying only a commonly published non-weight bin such as
training_args.bin (or an adapter-only artifact) passed offline validation and
then failed the local_files_only load at first indexing. Match recognized Torch
base-model weight filenames (model / pytorch_model, including sharded) by name.
is_embedding_model retained an online-confirmed positive offline even when no
files were cached, so a metadata-only /check-embedding result let an uncached
repo be saved and then fail at first indexing. Retain the positive only when the
active revision is materialized locally, which still covers a downloaded tag-only
embedder whose snapshot carries no modules.json.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: require a complete weight set offline and persist embedder verdicts across restarts
Two follow-ups to the offline embedding-model classifier:
- _snapshot_is_loadable_st_model now requires a COMPLETE Torch base-model
weight set in one snapshot directory, not just any single recognized weight
file. A partially downloaded sharded model (model-00001-of-00002 without its
sibling) no longer passes offline validation and then fails at first indexing
under local_files_only. Weight files are grouped by directory and a directory
counts only when it holds a single model.safetensors / pytorch_model.bin or a
full shard set whose indices cover 1..total.
- Online-confirmed embedder verdicts are now recorded under the resolved Studio
home (embedding_verdicts.json). The session memo is lost on exit, so a
downloaded tag-only feature-extraction embedder (snapshot present but no
modules.json) was misclassified as non-embedding the first offline call after
a restart. The offline branch consults this durable allowlist in addition to
the memo, still gated on the active revision being materialized on disk, so an
uncached repo is never trusted. Writes are best-effort and only positive
verdicts are stored.
* studio: require complete weights (with shard index) and resolve default casing offline
Follow-ups to the offline embedding-model classifier from the latest review:
- Trust a recorded embedder verdict (session memo or persisted allowlist) offline
only when the active snapshot carries a COMPLETE, loadable weight set, not merely
that it is materialized. A partial download (config present, weights missing or an
incomplete shard set) makes _embedding_marker_in_hf_cache read False rather than
None, so the previous marker-is-not-None gate wrongly returned True and the
local_files_only load then failed. Split out _snapshot_has_complete_weights (config
plus complete weights, modules.json aside) and _active_snapshot_dir, and gate the
known-embedder positive on the weight set.
- Require a sharded checkpoint's index map (model.safetensors.index.json /
pytorch_model.bin.index.json) in addition to every shard before accepting it:
transformers discovers and wires shards through that index, so a complete shard set
without it fails the local-only load.
- Resolve the embedding model name to its exact cache casing in the RAG loader before
constructing SentenceTransformer. The settings route persists that spelling for a
custom override but deliberately leaves the configured default verbatim, so a
default whose casing differs from the cache dir would miss it and fail offline.
Resolving at load time covers the default too; a no-op for a local path or when
nothing case-matching is cached, and idempotent for an already-normalized override.
Adds regression tests for the partial-snapshot verdict, the missing shard index, and
the loader casing resolution; updates the offline-invariant source assertion to the
resolved-name variable.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: require a tokenizer, case-fold verdict ids, and serialize verdict writes
Three follow-ups to the offline embedding-model classifier from the latest review:
- _snapshot_has_complete_weights now also requires a tokenizer asset. A
SentenceTransformer Transformer module builds an AutoTokenizer, so a snapshot with
a complete weight set but no tokenizer.json / tokenizer_config.json / vocab still
fails the local_files_only load. The check is a permissive union over the common
fast-tokenizer, config, and WordPiece/BPE/SentencePiece assets, so an unusual but
valid layout is not rejected -- only a genuinely tokenizer-less partial download.
- The persisted embedder allowlist is now keyed case-insensitively. model_info() is
queried under the requested casing while the settings route saves the cache-resolved
casing, so an exact-string lookup missed the persisted positive after a restart
(baai/model recorded, BAAI/model looked up) and a loadable tag-only embedder was
rejected. Both persist and lookup case-fold the id.
- _persist_embedder serializes its read-modify-write under a lock and writes through a
per-thread temp file, so concurrent confirmations of different embedders no longer
drop each other's entry or collide on the temp path. Cross-process writers stay
best-effort (os.replace is atomic; a dropped verdict is only an optimization miss a
later online re-confirmation heals).
Adds regression tests for the missing-tokenizer reject, alternate tokenizer assets,
cross-casing verdict match, and concurrent verdict writes; updates the snapshot test
helpers to materialize a tokenizer alongside config and weights.
* studio: tighten comments in the offline embedding-model classifier
Comment-only pass over the PR's changed files. Collapse the long block
comments and docstrings around is_embedding_model, the cache-snapshot and
weight-completeness helpers, the embedder-verdict persistence, the offline
security gate, and the offline/casing tests to short one- or two-line forms.
Preserve the rationale (issue #6817, the local_files_only invariant, the
casing and weight-gate reasons) in far fewer words. No code changes.
* studio: drop redundant comments in the offline embedding-model classifier
Second comment-reduction pass over the offline embedding-model cache work:
delete comments and trailing notes that restate the adjacent code or an
assertion, and trim the remaining docstrings and rationale comments to their
load-bearing invariants. Comments and docstrings only; no code changes.
* studio: pin embedder verdicts to a revision, canonicalize default aliases
- A persisted verdict recorded that the Hub tagged ONE revision an embedder, but
was stored per repo. Once refs/main advanced to a complete but non-embedding
Transformer snapshot, the offline path still returned True: the settings route
accepted the updated model without force and RAG could silently load it as an
embedder. Verdicts now carry the commit they were confirmed at and are trusted
only while the active revision matches. One confirmed before the repo was
cached has no revision to compare, so the first revision observed afterwards is
pinned then -- which is what lets a later advance be caught. The persisted file
gains a {id: commit} form and still reads the previous list format.
- tokenizer_config.json no longer counts as a tokenizer asset. It only DESCRIBES
a tokenizer, so a snapshot with config, weights and just that file passed
validation and then failed AutoTokenizer.from_pretrained(local_files_only=True)
at first indexing for common BERT/GPT-style models.
- A casing-only alias of the default is canonicalized to the default up front.
Repo ids are case-insensitive but every gate here compares exact strings, so
saving "Unsloth/bge-m3" against a default of "unsloth/bge-m3" ran the
verification and scan for a custom model and then persisted an override --
after which later changes to the configured default stopped applying.
- verify_import_hoist.py replays __all__ assignments in order instead of unioning
them. Only the final value exports anything, so a later plain "=" that drops a
name must leave its import counted as unused; "+=" still extends, and an
unreadable rebind keeps the earlier names rather than flagging real re-exports.
* studio: validate the real ST load root, and pin verdicts to the Hub revision
Four ways the offline probe still disagreed with what the loader does:
- Verdicts were pinned to the LOCAL refs/main, but model_info() describes the
current HUB revision. With a stale cache the two differ, so an older snapshot
nobody verified was allowlisted. The pin is now info.sha, taken from the
ModelInfo that produced the positive. A verdict carrying no revision (a legacy
entry) is no longer trusted at all -- trusting it meant pinning whatever
happened to be cached, which is the same bug; the next online check re-records
it properly.
- config, tokenizer and weights had to exist somewhere in the snapshot, not
together. modules.json can send SentenceTransformer at 0_Transformer/, which is
loaded FROM that directory, so a cache with the config at the root and only
0_Transformer/model.safetensors passed and then failed the local-only load.
Each directory is now checked as a complete load root, which covers both the
plain HF layout and the ST module layout.
- vocab.json and merges.txt counted independently, but BPE needs the pair unless
a serialized tokenizer.json is present, so half a pair validated and then
failed AutoTokenizer.from_pretrained(local_files_only=True).
- A slashless short name like all-MiniLM-L6-v2 is a supported ST alias that the
loader resolves through the sentence-transformers/ organization, so its
snapshot is cached under that full id. Probing only the bare name reported a
miss and 409'd a model that was cached and loadable; the bare id is still tried
first, matching the loader's own order.
* studio: fail closed for an offline security scan instead of failing open
A local_only (offline) load cannot fetch Hugging Face's malware scan, and the previous
behaviour skipped the scan and failed OPEN, so a cached repo with a poisoned pickle weight
could deserialize under SentenceTransformer(local_files_only=True). Evaluate it fail-CLOSED
against the cached files instead: block a base-model pickle weight the load would deserialize
(pytorch_model.bin and its shards, in a directory with no safetensors alternative) and allow a
pickle-free (safetensors / gguf are inert) cache. A cached pickle model must be reloaded online
once to be scanned, or shipped as safetensors. Nothing cached is not a security event.
_fetch_security_status no longer needs the local_only_load skip (the offline branch is handled
in evaluate_file_security). Adds a regression test covering the safetensors-allow and
pickle-block paths with no Hub call.
* studio: only suppress an offline pickle when a loadable safetensors weight exists
The offline security gate treated any .safetensors in a directory as covering a
pickle weight, so a cache with pytorch_model.bin beside a bare adapter_model.safetensors
(or an orphan shard with no index) passed the fail-closed check even though
from_pretrained still selects and deserializes the pickle. Require a genuinely loadable
safetensors weight -- an unsharded base file or a complete indexed shard set -- before
treating the pickle as covered.
Also make the import-hoist analyzer preserve uncertainty when __all__ is extended by a
value it cannot read statically (__all__ += dynamic()), matching how it already handles
an unreadable rebind, so a dynamically-supplied re-export is not flagged HOISTED-IMPORT-UNUSED.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: scope the offline pickle scan to load paths; reset __all__ opacity on rebind
Address three review follow-ups on the offline security gate and the import-hoist analyzer:
- The offline pickle scan walked the whole snapshot, so a stray pickle in a non-load
subdirectory (archive/, nemo/) that SentenceTransformer never deserializes was blocked.
Scope it to real from_pretrained load roots -- the snapshot root, or a subdir that holds
its own config.json -- matching the online scan's load-path scoping.
- _collect_dunder_all kept a sticky opaque flag: a readable replacing assignment after an
unreadable extend (__all__ += dynamic(); __all__ = []) still credited every import, so a
genuinely unused hoist went unreported. A replacing assignment now resets opacity.
- A bare __all__: list[str] annotation has no runtime value; it was treated as an unreadable
assignment and marked the export set opaque. Skip annotation-only declarations.
* studio: recase slashless ST aliases and accept a pinned embedder after a transient failure
Two offline-detection gaps on well-formed input:
- resolve_st_cached_repo_id_case bailed on every slashless name, so a differently-cased
short alias (all-minilm-l6-v2) validated case-insensitively but was loaded verbatim; the
SentenceTransformer loader rewrites it to sentence-transformers/all-minilm-l6-v2 and looks
it up case-sensitively, missing the canonical sentence-transformers/all-MiniLM-L6-v2 cache
dir. Resolve through _st_cache_repo_dir, which follows the same org alias, and hand back the
on-disk casing.
- On a transient (non-permanent) Hub failure, is_embedding_model only accepted a cached
modules.json marker, so a downloaded tag-only embedder (no modules.json) with a verdict
pinned to the active revision was rejected even though the offline branch accepts the
identical cache. Mirror the offline branch's pinned-verdict acceptance.
* studio: scan modules.json-declared module roots in the offline pickle gate
The offline pickle scan treated only the snapshot root and config.json-bearing subdirs as
load roots, so a pickle in a non-Transformer SentenceTransformer module directory that has no
config.json (e.g. a 0_WordEmbeddings/ module: wordembedding_config.json + pytorch_model.bin)
was skipped even though the loader deserializes it. Parse modules.json (and thread through
load_subdirs) to treat every declared module directory as a load root, so such a pickle is
scanned and fail-closed offline.
* studio: classify cached non-Transformer SentenceTransformer models offline
_snapshot_has_complete_weights recognized only a Transformer-shaped load root (config +
tokenizer + weights co-located), so a fully-cached model built from a non-Transformer module
(0_WordEmbeddings uses wordembedding_config.json + embedding weights and its own tokenizer, no
HF config.json; BoW keeps its vocab in config.json) was classified non-embedding offline and
the settings endpoint returned 409.
Add _snapshot_modules_all_loadable, which parses modules.json and accepts a snapshot when every
declared module's path directory carries the files that module class's own load() reads (a
Transformer/root module still needs the full HF load root; a WordEmbeddings module needs its
config plus a complete weight set; other modules need their *_config.json), and at least one
embedding-producing module is present. It is OR-ed after the Transformer check, so it only ever
accepts more and cannot regress the existing path or reject a pruned cache.
* studio: scan PEFT adapter pickle weights in the offline security gate
from_pretrained auto-detects an adapter_config.json in the load root and deserializes the
adapter weights on top of the base model, so adapter_model.bin is a separate pickle RCE vector
that a safetensors base weight does not cover. The offline scan matched only base-model pickle
names, so an offline local-only load with safetensors base weights plus a cached
adapter_model.bin was allowed despite the live adapter pickle. Scan adapter pickles too, scoped
to a load root where adapter_config.json is present and no adapter_model.safetensors exists.
* studio: require weights for Dense/CNN/LSTM SentenceTransformer modules offline
_module_dir_is_loadable accepted a Dense, CNN, or LSTM module dir with only its config, but
those modules' load() hard-load model.safetensors else pytorch_model.bin (verified against
sentence-transformers source: no fallback, raises if neither exists) -- exactly like
WordEmbeddings. A cache with such a module's config but no weights would validate and then
fail the local_files_only load. Require a complete weight set for every weighted module, not
just WordEmbeddings.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: scan root-index subdir pickle shards offline; handle __all__.append/.extend
- The offline pickle scan followed only load-root directories, so a shard mapped by a root
pytorch_model.bin.index.json into a non-root subdirectory was skipped even though
from_pretrained follows the index weight_map and deserializes it (a layout an attacker can
craft to evade the scanner). Read the local index and scan its referenced pickle shards,
covered by a loadable base safetensors at the index root -- mirroring the online scan.
- The import-hoist analyzer ignored __all__.append("X") / __all__.extend([...]) runtime
re-export mutators, so an import added solely for one tripped HOISTED-IMPORT-UNUSED. Read
their string args like +=, and treat any other __all__ method call as opaque.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: classify StaticEmbedding offline, require WordEmbeddings tokenizer, bound model_info
- A StaticEmbedding module (e.g. sentence-transformers/static-retrieval-mrl-en-v1's
0_StaticEmbedding/) holds tokenizer.json + weights and NO config, so the config-gated
non-Transformer path 409'd it offline. Recognize it by what StaticEmbedding.load() reads: a
tokenizer.json plus a complete Torch weight set.
- WordEmbeddings.load() rebuilds its tokenizer via the configured tokenizer_class.load() from the
module dir, so a WordEmbeddings module now also requires a tokenizer artifact
(whitespacetokenizer_config.json / phrasetokenizer_config.json, or a shared HF tokenizer asset),
not just its config + weights.
- With neither offline env var set, an unbounded model_info() could hang on connect/DNS retries
for networkless users (the #6817 symptom). Bound it with a 15s timeout so a dead network fails
fast and the existing transient-failure cache fallback resolves a cached model, while a
reachable Hub still wins. (Documented caveat: a stalled DNS getaddrinfo may exceed this.)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Resolve indexed safetensors shards relative to their index
_safetensors_index_complete compared shard basenames against the flat
set of files in the index directory, so an index whose weight_map names
shards in a subdirectory was treated as incomplete whenever a legacy
pytorch_model.bin sat beside it. That falsely blocked a snapshot whose
pickle weights are fully covered by a complete, loadable safetensors
shard set. Resolve each shard path relative to the index directory
instead, and add a regression test for the subdir-mapped shard case.
* Restrict offline weight-completeness check to declared load roots
_snapshot_has_complete_weights scanned every directory in a snapshot and
accepted it when ANY directory was a complete Transformer load root. When
modules.json is present a SentenceTransformer load only opens the declared
module paths, so a snapshot whose declared modules are incomplete but which
happens to contain an unrelated complete directory was accepted offline and
then failed at the first local_files_only load. Restrict the candidate
directories to the roots a load actually opens: the snapshot root plus each
modules.json module path. For a well-formed snapshot the verdict is
unchanged; only a complete directory at an undeclared path no longer vouches
for an otherwise-incomplete snapshot.
* Scan SentenceTransformer Router child module weights offline
A Router (legacy Asym) snapshot declares its child sub-modules only in
router_config.json, not the top-level modules.json, and Router.load()
deserializes each child's weights from its own subdir. A config.json-less
child such as query_0_WordEmbeddings (wordembedding_config.json plus a
pickle pytorch_model.bin loaded via torch.load) was therefore neither a
modules.json-declared load root nor a config.json-bearing dir, so the
offline gate skipped its pickle even though the loader deserializes it.
Parse router_config.json at each load root and treat every declared child
subdir as a load root (bounded BFS, so nested routers are covered), so
those child pickles are scanned. Add Router regression tests: a pickle
child blocks, a safetensors child is allowed, and a Router in a declared
subfolder is followed.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Do not treat an unreferenced config subdir as an offline load root
The offline pickle gate skipped a directory only when it was neither a
declared load root nor held a config.json. Because _st_load_roots already
resolves every real load root (snapshot root, modules.json / load_subdirs
dirs, Router children), the config.json fallback only ever promoted an
UNREFERENCED subdir -- a nested checkpoint-500/ or archive/ that ships its
own config.json + pytorch_model.bin -- to a load root. from_pretrained
never descends into such a subdir and the online scan ignores the same
unindexed pickle, so offline mode wrongly blocked a model the loader reads
from a clean safetensors root. Scope the pickle to directory in roots
only, and add a regression test (a stray checkpoint-500/ no longer blocks;
a modules.json-declared module dir still does).
* Classify a root Router (Asym) model as loadable offline
_module_dir_is_loadable applied Transformer root requirements (config +
tokenizer + weights) to every root module, so a Router saved at the
snapshot root -- which carries only modules.json + router_config.json and
loads its weights from child subdirs -- was classified not loadable
offline, and is_embedding_model missed a cached Router embedder. Dispatch
on the module class before the root Transformer fallback: a Router/Asym
dir is loadable when router_config.json parses and every declared child
subdir is loadable (validated recursively through _module_dir_is_loadable,
so nested routers and every child type are covered) with at least one
embedding-producing child. This also tightens a non-root Router, which
previously validated on the mere presence of router_config.json without
checking its children. Add Router regression tests (root and declared
subfolder, complete and incomplete-child).
* Require every declared module before accepting an offline cache
_snapshot_is_loadable_st_model returned has_complete_weights OR
modules_all_loadable, so a complete 0_Transformer short-circuited the or
and vouched for the whole snapshot even when a declared sibling module was
missing its serialized weights; SentenceTransformer builds every module in
modules.json, so that snapshot passed offline validation and then failed
the local-only load. When modules.json declares a non-empty list it is now
authoritative (modules_all_loadable validates every declared module);
has_complete_weights stays the fallback only for an empty/non-list
modules.json (the plain from_pretrained root). Also add the weight-bearing
modules whose load() hard-loads via load_torch_weights and previously fell
to the config-only path -- LayerNorm, WeightedLayerPooling, SparseAutoEncoder
-- to _ST_WEIGHTED_MODULE_NAMES, with source citations and the deliberate
exclusions (Pooling/Normalize/BoW/WordWeights read no weights on load).
Add parametrized regression tests over LayerNorm/WeightedLayerPooling/Dense
(a weightless sibling rejects, a complete sibling accepts).
* Reject self-referential Router children instead of recursing forever
_router_dir_is_loadable validates each router_config.json child through
_module_dir_is_loadable, which re-enters _router_dir_is_loadable for a
Router child. A malformed types entry naming the router's own directory
(a key of ".", which normalizes to the same dir) made that recursion
never descend, so it looped until RecursionError -- breaking the
documented never-raises contract and turning a crafted/corrupted cached
model into a 500 from is_embedding_model instead of a graceful
unverifiable result. A real child reference is a subdir and always
resolves deeper, so reject any child whose resolved path is the router
dir itself. Add a regression test (a router_config naming "." as a
Router child returns False without raising).
* Treat a destructuring __all__ assignment as opaque
_collect_dunder_all detected __all__ only as a direct ast.Name assignment
target, so a binding through a destructuring target (__all__, meta = [...],
v -> an ast.Tuple) was skipped entirely, leaving an empty, non-opaque
export set. A newly hoisted import re-exported only through that assignment
was then falsely flagged HOISTED-IMPORT-UNUSED. Its value cannot be mapped
statically, so mark the export set opaque when __all__ is reached only
through a destructuring / item / attr target, matching how the collector
already handles other unreadable __all__ forms. Add a self-test case.
* Canonicalize declared module paths before scoping the offline pickle gate
A repo could declare a traversing module path such as 0/../evil in
modules.json (or a router_config child), which SentenceTransformer resolves
to evil/ and deserializes evil/pytorch_model.bin. _st_load_roots recorded
the raw snap/"0/../evil", which never equals the snap/evil that rglob
yields, so the offline pickle gate skipped that directory and a malicious
repo slipped a pickle past the newly added gate. Add _canonical_load_dir
to collapse ./ and ../ components lexically and reject an upward escape,
and route the modules.json paths, load_subdirs and router children through
it so the gate scopes the same normalized directory the loader opens. Add
regression tests for a traversing modules.json path and router child.
* Close offline embedding-classification completeness gaps
Five real offline misclassifications, each a false negative (the #6817 hang
recurs) or false positive (accepted then 409s at the local_files_only load).
Dispatch _module_dir_is_loadable on the module class before the root
Transformer fallback. A module with save_in_root=True (every InputModule:
WordEmbeddings, StaticEmbedding, SparseStaticEmbedding, Transformer, Router)
is saved at the snapshot root, so a root WordEmbeddings was wrongly held to
Transformer requirements (an HF tokenizer it never writes) and classified not
loadable.
CLIPModel is Transformer-shaped: CLIPModel.load() reads AutoModel weights plus
AutoProcessor, so a config-only CLIP dir must not validate.
SparseStaticEmbedding needs a tokenizer plus either idf.json or a complete
torch weight set (conditionally weight-bearing); a config alone is not enough.
A present but empty or malformed modules.json is not loadable and does not fall
back to a root Transformer: with modules.json present the loader never takes
the plain-Transformer path (base/model.py _load_config_modules). The tag-only
no-modules.json embedder is classified separately via
_snapshot_has_complete_weights.
Validate a sharded weight index against its weight_map (every mapped shard
present, resolved relative to the index dir) instead of trusting the index
file's mere existence, mirroring the security-side check.
Add regression tests for all five.
* Close case-folding and online-traversal holes in the offline pickle gate
Two gate bypasses where the security scan credited or scoped a path
differently from what the loader actually resolves:
The safetensors credit was case-folded. _cached_pickle_weight_files lowercases
every filename, and the loadable-safetensors and adapter checks tested those
folded keys against the exact-lowercase names. On a case-sensitive filesystem
(Linux, the Studio default) a crafted repo shipping Model.SafeTensors plus a
malicious pytorch_model.bin makes transformers and sentence-transformers miss
the exact-name model.safetensors and deserialize the pickle, while the gate
credited an inert safetensors and did not block. Credit safetensors
case-sensitively against real filenames, and drop pytorch_model.safetensors
from the credit set (transformers loads only model.safetensors, never that
name). Pickle matching stays case-insensitive (over-blocking a mis-cased
pickle the loader would not load is the safe direction).
The online scan did not canonicalize traversing paths while the offline gate
did. A repo-controlled modules.json path (threaded into the online scan via
the RAG guard) or a weight_map shard entry like 0/../evil / ../evil was
compared verbatim, so a flagged evil/pytorch_model.bin never matched and
evaded the online scan though the loader resolves and deserializes it.
Canonicalize the repo-controlled load-subdir prefixes and weight_map shards
the same way the offline gate does, so offline and online agree.
Add regression tests for both bypasses.
* Treat a conditional __all__ mutation as opaque in the import-hoist linter
_collect_dunder_all replayed only top-level module statements, so an __all__
assignment or mutation inside a module-level if / try / for / while / with /
match (or a deeper scope) was ignored, leaving the export set understated. A
newly hoisted import re-exported only through such a conditional __all__ was
then falsely flagged HOISTED-IMPORT-UNUSED, blocking a valid change. A
conditional value cannot be replayed statically, so mark the export set opaque
when __all__ is bound or mutated anywhere other than a top-level statement.
Add a self-test case.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Scope Router child sub-modules as load roots in the online embedding scan
The RAG embedding security guard unions the SentenceTransformer module dirs
from modules.json into the load roots it scopes for the Hub scan, so a flagged
pickle directly under a Transformer module blocks. A Router (legacy Asym)
module declares its child sub-modules only in router_config.json, not in
modules.json, and Router.load() deserializes each child from its own subdir.
The online scan therefore dropped a flagged child pickle (for example
query_0_WordEmbeddings/pytorch_model.bin) as an unreferenced nested shard while
the loader still deserialized it, the counterpart to the offline gate which
already expands router children via _router_child_dirs.
_st_module_subdirs now reads router_config.json for any Router-typed module and
adds each declared child (joined onto the module path, canonicalized so a
traversing entry is dropped) to the load roots. The config is read only for a
Router-typed module, so a plain embedder pays no extra fetch, and every failure
path still returns () so the guard never bricks the embedder.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Allow a recorded-clean pickle embedder to load offline
The offline embedding security gate is fail-closed: with no network to reach
Hugging Face's scan, a cached pickle weight cannot be verified, so it is blocked
and a model the user already downloaded and used online will not load offline.
This adds a persistent cache of clean Hub verdicts so that exact content can load
offline, without weakening the gate for an unknown or never-scanned pickle.
When an embedding repo is loaded online and HF's scan returns a completed clean
verdict, the load roots are hashed and recorded under the scanned commit as an
exact map of snapshot-relative pickle name to sha256, in a per-user JSON store at
studio_root()/security/embedding_scan_verdicts.json (atomic write, 0600, thread
and cross-process locked, 30-day TTL). Offline, a cached pickle model loads only
when the active cached commit and every load-root pickle's sha256 match the
recorded verdict; a missing record, moved commit, changed or added pickle,
expired record, or any error keeps blocking. Online loads always re-query the
Hub and an authoritative unsafe verdict deletes any stale record, so a
now-flagged commit cannot keep loading on an old clean record.
The store binds repo id, full commit, and a per-file sha256 map so a locally
swapped pickle at the same commit, a branch advance, or an added load-relevant
pickle is detected. A same-user attacker who can rewrite the model cache or the
store is outside the enforceable boundary and this is documented; the sha256 is
computed just before load, so a narrow verify-to-load window remains, and a Hub
scanner false negative is recorded faithfully (safetensors stays the stronger
defense).
Recording is triggered post-load in the RAG embedder because the settings route
only validates and the pre-load guard runs before the constructor downloads;
recording is skipped when the loaded commit differs from the scanned commit. The
blocked-pickle enumerator now returns snapshot-relative Paths so two module dirs
that ship the same pickle basename are hashed and reported distinctly.
* Harden the embedding verdict cache against review findings
Tighten the offline verdict cache and its enumeration so every uncertain or
malformed input fails closed and the recorded hashes always match the files the
loader reads:
- Hash every case-colliding pickle in a load root, not one representative. On a
case-sensitive filesystem pytorch_model.bin and PYTORCH_MODEL.BIN are distinct
files; keying by lowered name dropped one and could hash a decoy instead of the
loader's target. The enumerator now returns every variant Path.
- Only persist a clean verdict for a COMPLETED, entirely-benign scan. Require
scansDone to be the boolean True (not a truthy string), filesWithIssues to be a
well-formed list, and every flagged file to be a definitively-safe level; a
pending, error, unknown, or malformed entry no longer records as clean. The
online block decision is unchanged.
- Fail closed when the offline cache cannot be inspected: an rglob error now
propagates and blocks instead of reading as pickle-free, and a snapshot that
errors on resolution (vs a clean not-cached) blocks. The offline guard also
raises instead of returning when its own inspection throws, so the constructor
never deserializes an unverified cached pickle.
- Expand online Router children recursively (bounded BFS with a seen set),
mirroring the offline load-root expansion, so a flagged grandchild pickle is
scoped online and cannot be recorded clean.
- Reject absolute and drive/UNC declared paths in the load-root canonicalizers;
the loader would resolve them outside the snapshot, so collapsing them to an
in-snapshot relative dir scoped the wrong place.
- Pin verdict recording to the scanned commit's snapshot and take the offline
verify commit from the snapshot directory name, removing a second refs/main read
and the skew it allowed.
- Drop the now-unused pickle-name wrapper.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten offline embedding classification and the pickle gate
Close a set of offline edge cases where validation accepted a cache the
local_files_only load then rejects, and one gate bypass:
- Credit a sharded model.safetensors.index.json for a pickle sibling only at a
from_pretrained root. A non-Transformer SentenceTransformer module (Dense,
WordEmbeddings, StaticEmbedding) loads via Module.load_torch_weights, which
reads model.safetensors then pytorch_model.bin and never the index, so a sharded
safetensors index in such a module dir must not vouch for its pytorch_model.bin.
- Stop counting pytorch_model.safetensors as loadable in the offline classifier:
the loader probes model.safetensors (then its index) or pytorch_model.bin, never
pytorch_model.safetensors, matching the gate that already treats it as a decoy.
- Treat a present but unreadable weight index as incomplete: transformers opens
and parses any present index, so a malformed one or one without a weight_map
fails the load rather than falling back to filename-numbered shards.
- Require the CLIP image-processor config (preprocessor_config.json) for a CLIP
module: CLIPModel.load builds a CLIPProcessor that needs it, so a tokenizer
alone is not enough.
- Require a SparseStaticEmbedding config to actually select idf.json (a path
ending .json) or ship loadable weights; a bare idf.json the config does not name
falls through to load_torch_weights and raises.
- Do not use the tag-only recorded-verdict fallback when modules.json is present:
with the file present the loader takes the modules.json path, so a present but
empty or malformed manifest must not be validated as a plain root Transformer.
- Import-hoist linter: only a module-level conditional mutation or a function that
declares global __all__ makes the export set opaque; a __all__ bound as a local
in a nested function or class no longer masks a genuinely unused hoisted import.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Scope Router-child pickles to their deepest load root and gate the ST offline kwarg
The online scan stripped the first matching load-subdir prefix from a flagged file, so a
nested Router child pickle (0_Router/query_0_WordEmbeddings/pytorch_model.bin) matched the
parent 0_Router root, looked like an unreferenced nested shard, and slipped the gate even
though Router.load() deserializes that child directly. Match the deepest (longest) load
subdir instead, so the child becomes root-level under its own load root and blocks.
pyproject sets no lower bound on sentence-transformers and the local_files_only constructor
arg is absent on older releases, so always forwarding it broke every embedder warm on those
installs. Pass it only for an offline load; an online warm never forwards it and works as
before, while the offline capability still requires a version that supports it.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Reject snapshot-escaping shard paths and credit Transformer submodule safetensors
The offline pickle enumerator joined a weight-index weight_map value straight to the load
root and followed it, so a repo-controlled index mapping "../.." into a sibling snapshot
made an offline from_pretrained deserialize an out-of-snapshot pickle, and an online load
would then hash and record that external file as the scanned commit's clean content. Reject
any shard path that escapes the snapshot root and fail closed, mirroring the canonical-root
check the online shard scan already applies.
A complete model.safetensors.index.json was credited over a sibling pickle only at the
snapshot root, but a Transformer module subdirectory (0_Transformer/) is loaded via
AutoModel.from_pretrained, which honors that shard set and never reads the pickle. Credit the
sharded index for Transformer-typed modules declared in modules.json so a cached model that
ships both a sharded safetensors checkpoint and an unused PyTorch checkpoint is no longer
falsely blocked offline. Non-Transformer modules (Dense, WordEmbeddings, StaticEmbedding) read
a flat weight with no index and keep their pickle blocked.
Limit the import-hoist verifier's global __all__ scan to the declaring function's own scope so
a nested inner-scope local __all__ no longer marks the module export set opaque and mask an
unused hoisted import.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Scope Router children against the snapshot and mirror the ST alias rewrite
Router.load resolves each child at Path(subfolder, model_id) relative to the Router dir, so a
nested 1_Router with a "../evil" child points at evil/ inside the snapshot and the loader
deserializes evil/pytorch_model.bin. The offline enumerator canonicalized the child against
the Router dir alone and dropped anything with "..", so that pickle was never scanned and the
gate reported the cache pickle-free. Canonicalize router children against the snapshot,
retaining in-snapshot siblings as load roots and failing closed on a child that escapes the
snapshot itself, matching the online scan which already joins the prefix before normalizing.
The security gate resolved a slashless model id by probing the bare cache dir first, but the
SentenceTransformer constructor rewrites a non-basic slashless name to sentence-transformers/
<name> and loads THAT snapshot (only the basic ORIGINAL_TRANSFORMER_MODELS load bare). With
both models--<name> and models--sentence-transformers--<name> cached, the gate inspected the
bare dir while the loader read the namespaced one, so a pickle there bypassed the local-only
gate. Mirror the constructor: try the namespaced candidate first for non-basic slashless names.
Add the same not (snapshot / modules.json).is_file() guard to the transient-Hub-failure
tag-only fallback that the offline branch already carries, so a cache whose present manifest is
empty or malformed is no longer reported as a loadable embedder.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten root shard credit, module-path escapes, and weight-set probe order
Credit the sharded safetensors index at the snapshot ROOT only when the root is actually loaded
through an AutoModel/from_pretrained path. A modules.json root module of a non-Transformer type
(StaticEmbedding / WordEmbeddings / Dense) loads via load_torch_weights, which reads
pytorch_model.bin and ignores the index, so crediting a root shard index there suppressed a live
root pickle and let the offline gate report the cache pickle-free.
Recognize the Transformer subclasses CLIPModel and MLMTransformer as index-honoring load roots
(they load via from_pretrained), so a sharded-safetensors CLIP/MLM submodule with a legacy
pytorch_model.bin sibling is no longer falsely blocked offline. Mirrors the classifier dispatch.
Fail closed on an absolute or snapshot-escaping modules.json module path (or load_subdirs entry)
instead of silently dropping it: SentenceTransformer resolves such a path outside the snapshot and
would deserialize an external pytorch_model.bin the gate cannot scan.
On the classifier side, walk the weight set in the exact from_pretrained probe order
(model.safetensors, its index, pytorch_model.bin, its index) so a pickle behind a malformed
safetensors index is no longer accepted as complete, and restrict shard names to the loader-probed
stem/ext pairs so a decoy model-*.bin / pytorch_model-*.safetensors set is not treated as loadable.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Restore scripts/verify_import_hoist.py to main
The offline embedding cache fix does not depend on the __all__ scope
handling that had accumulated in this linter, so revert the file to its
main version and keep the PR focused on the feature. The feature modules
still pass the existing import hoist check unchanged.
* Reuse a shared HF cache skeleton in the offline classification tests
Extract _mk_repo and _activate helpers for the repeated snapshot cache
setup that every per-type builder duplicated, and fold the two
StaticEmbedding missing-asset cases into one parametrized test. Same 125
collected items, all still passing.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Reclassify embedding models from the cache on every offline call
is_embedding_model consulted its process memo before the offline branch, so an
online lookup that memoized True from tags (without caching any weights) was returned
unchanged once the session went offline -- the studio flips HF_HUB_OFFLINE in-process
on a dead DNS, and the ungated check-embedding route can populate the memo. Settings
would then accept a repo the offline loader cannot open. Run the offline
cache-marker reclassification ahead of the memo and never record it, so an offline
verdict always reflects the local cache and a later cache materialization is not
masked by a stale negative. Add regression tests.
* Tighten comments on the offline embedding path
Condense the offline-embedding helper docstrings and inline comments added in
this PR to fewer, clearer lines, keeping the non-obvious security and offline
rationale. Comments and docstrings only; no code change.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <unslothai@gmail.com>
* studio: show system-wide VRAM in the multi-GPU System tab view on ROCm
The System tab's per-GPU list comes from get_visible_gpu_utilization. When
amd-smi is unavailable (always on Windows, minimal Linux installs) it fell back
to torch, whose readings are process-local: on Windows WDDM hands each process
its own budget, so a model held by the separate llama-server process read as
~0 VRAM used even with the GPU full (#7072). The primary-GPU endpoint already
compensates with system-wide sources -- Windows Performance Counters (Task
Manager's source) and Linux DRM sysfs -- but the multi-device endpoint never
got those fallbacks.
Add per-GPU variants of both sources and overlay them onto the torch fallback:
_rocm_windows_perf_counter_vram_per_adapter_gb() attributes Dedicated Usage per
physical adapter (phys_<N> in the counter instance name), and
_rocm_linux_sysfs_vram_per_card_gb() reads mem_info_vram_{used,total} per DRM
card. _overlay_system_wide_vram() applies them to the device list, ROCm-only,
best-effort: unmatched adapters and ambiguous card counts keep the torch
figures, and NVIDIA paths are untouched.
Fixes#7072
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: match VRAM overlay sources by device, honor unified memory, unblock the loop
Five review fixes on the multi-GPU system-wide VRAM overlay:
1. Linux: match DRM cards to devices by PHYSICAL index instead of a positional
zip, so a reordering visibility mask (HIP_VISIBLE_DEVICES=1,0) no longer
swaps each card's figures onto the other GPU (which would mislead
auto_select_gpu_ids and the coexistence checks). An index with no matching
card keeps its torch figures.
2. Linux: skip the overlay for a device whose sysfs total is below torch's --
on unified-memory APUs (Strix Halo) mem_info_vram_total is only the small
dedicated slice while torch sees the GTT-backed pool, and
_apply_unified_memory_correction already defines larger-total-wins.
3. Windows: group counter instances by adapter LUID, not the phys_<N> suffix --
separate adapters each read phys_0, which collapsed every GPU into key 0.
LUIDs are mapped to 0-based positions by ascending value as the closest
stand-in for device order.
4. Windows: pair the system-wide usage with the physical capacity from
get_device_properties (as the primary-GPU fallback does) -- under WDDM
mem_get_info's "total" is the process budget, which misreported capacity
and pushed utilization to 100%.
5. Run get_visible_gpu_utilization off the event loop in the /hardware/visible
route (asyncio.to_thread, the repo's convention): the ROCm fallbacks can
shell out to PowerShell with a 5s timeout, which would stall every other
request while the System view polls.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: skip the system-wide VRAM overlay for relative GPU indices
The overlay matches its per-GPU sources (Windows perf counters, Linux sysfs) by
physical device index, but under a UUID/MIG visibility mask the torch fallback
enumerates ordinals and reports index_kind == "relative", where `index` is a
visible ordinal, not a physical id. Applying the overlay there let card/adapter
0's system-wide VRAM overwrite the torch reading of a process that actually
exposes physical GPU 1, misleading auto_select_gpu_ids and the coexistence
checks. Gate the overlay on index_kind == "physical"; relative-index paths keep
the torch fallback.
* studio: drop the unreliable Windows VRAM overlay, keep the Linux one
The multi-GPU system-wide VRAM overlay is now Linux-only. The Windows
per-adapter Performance Counter path could not be made correct: the wildcard
Get-Counter query also returns non-ROCm/iGPU adapters and LUID order is not the
ROCm device order, so an adapter's usage could be overlaid onto the wrong GPU;
and it read only Dedicated Usage, missing WDDM shared memory on unified-memory
GPUs (Strix Halo), overstating free VRAM. Rather than misattribute VRAM and
skew placement decisions, Windows keeps the process-local torch fallback (no
regression vs before this PR); Linux DRM sysfs -- matched by physical index --
still fixes#7072 for the reporter's native-Linux ROCm case.
Removes _rocm_windows_perf_counter_vram_per_adapter_gb and _torch_props_total_gb.
* studio: key sysfs VRAM by DRM card number so filtering can't renumber cards
_rocm_linux_sysfs_vram_per_card_gb dropped cards with a zero total or unreadable
files and then the overlay enumerated the compacted list, so if card0 was
dropped, card1's usage was assigned to physical GPU index 0 (equal-capacity GPUs
slip past the unified-memory total guard). Return {card_number: (used, total)}
and match a device to its card number directly: a hole stays a hole -- device 0
keeps its torch figures when card0 is absent, and card1 maps to device 1.
* studio: key system-wide VRAM by ROCm ordinal, not raw DRM card number
When a non-amdgpu adapter (Intel iGPU, a display-only card) owns an earlier
DRM slot, DRM card numbers stop equalling ROCm device ordinals -- Intel card0
plus AMD card1/card2 gives ROCm devices 0/1, so keying the sysfs overlay by
card number handed ROCm device 1 card1's data (AMD device 0) and left device 0
on stale torch figures, corrupting free-VRAM placement on equal-capacity GPUs.
Only amdgpu cards expose mem_info_vram_*, so the glob already excludes foreign
adapters; order the surviving cards by their PCI address (ROCm/HIP's default
device order, read from each card's device symlink) and key by that position --
the ROCm physical ordinal, which is what the overlay matches against dev index.
An unreadable / zero-total amdgpu card still consumes its ordinal so a later
card is never renumbered onto its slot.
* studio: skip the VRAM overlay under layered HIP-over-ROCR masks
ROCR_VISIBLE_DEVICES filters physical GPUs at the HSA/ROCr layer, and a
HIP_VISIBLE_DEVICES set on top selects WITHIN that already-filtered set
(apply_gpu_ids sets HIP while leaving an inherited ROCR mask in place). When
both are active _get_parent_visible_gpu_spec() prefers the HIP value, so the
reported device index is a ROCR-relative ordinal, not a physical GPU id --
overlaying DRM-sysfs figures by that index would pull another GPU's usage
(e.g. ROCR=2,3 + HIP=1 is physical GPU 3, but the overlay would read card 1),
and equal-capacity cards bypass the total-size safeguard. Detect layered masks
and keep torch's process-local figures there rather than risk misattribution;
a single mask still leaves the index physical and is overlaid as before.
* studio: only overlay whole-card VRAM onto 1:1 ROCm devices
The overlay guard only skipped the case where sysfs total < torch total
(unified-memory APUs), so a partitioned ROCm device (MI300 in CPX mode) --
where HIP exposes several logical devices per physical card but sysfs reports
the whole card's aggregate -- passed the guard: the card total exceeds a
partition's torch total, and the overlay overwrote the partition with
whole-card usage and capacity, letting downstream selection think a partition
had the entire card free. Require the sysfs card total to match the torch
device total (within ~10%) so a mismatch in either direction -- unified memory
(sysfs smaller) or partitioning (sysfs larger) -- keeps torch's figures.
* studio: treat CUDA-over-ROCR as layered, enumerate AMD cards by driver
Two remaining mismatches between the reported device index and the DRM card the
overlay reads:
- On ROCm the HIP layer honors CUDA_VISIBLE_DEVICES as well as
HIP_VISIBLE_DEVICES, so a CUDA mask composed over ROCR layers identically:
ROCR=2,3 with CUDA=1 is physical GPU 3, yet the spec reports the ROCR value
[2,3] and the device was labeled index 2, overlaying card 2's usage onto GPU 3.
The layered check now treats ROCR combined with either HIP or CUDA as layered.
- The ROCm device set is now enumerated by bound driver (device/driver resolves
to amdgpu) instead of by the presence of mem_info_vram_*. An AMD device with
incomplete sysfs support (some APUs expose no VRAM files at all) was omitted
by the glob entirely and shifted every later card down one ordinal, letting a
similar-capacity GPU pass the total guard with another device's usage. Such a
card now consumes its ordinal and simply yields no entry.
* studio: honor GPU_DEVICE_ORDINAL and require an unambiguous card mapping
Two remaining ways the reported device index could be matched to the wrong DRM
card:
- GPU_DEVICE_ORDINAL is a supported ROCm visibility variable that
_get_parent_visible_gpu_spec() never consults, so GPU_DEVICE_ORDINAL=1
surfaces physical GPU 1 as torch ordinal 0 and it was mislabeled index 0,
overlaying card 0's usage onto GPU 1. The mask check now covers it, and is
renamed _rocm_device_index_unreliable() to say what it actually decides.
- driver == amdgpu is only a SUPERSET of the ROCm-visible set: an amdgpu-bound
adapter HIP cannot enumerate (an unsupported older AMD GPU beside a supported
one) still took an ordinal and shifted every real compute device. There is no
torch-side PCI identity to match against, so the overlay now requires the
amdgpu card count to equal the device count -- exactly the condition under
which position-in-PCI-order is a sound 1:1 mapping. Any disagreement keeps
torch's process-local figures: less informative, never misattributed.
* studio: keep the VRAM overlay working for masked GPU subsets
The card-count guard compared the amdgpu card list against the VISIBLE device
list, so any visibility mask disabled the overlay outright: HIP_VISIBLE_DEVICES=1,3
on a four-GPU host gives two devices against four cards. Those masked GPUs then
kept reporting process-local torch usage, hiding VRAM held by llama-server and
letting the training/chat placement checks overestimate free memory -- the exact
problem the overlay exists to fix.
The count check now applies only when no visibility mask is active, which is the
case where the reported devices really are the whole host and a mismatch means an
amdgpu adapter ROCm cannot enumerate is shifting the ordinals. Under a mask the
subset is expected, so each device's physical index is validated individually
instead: the per-card lookup bounds-checks it and the total-size guard rejects a
card whose capacity does not match the device's.
* studio: match GPUs to DRM cards by PCI identity, not by position
Every mapping bug on this PR came from the same root cause: there was no
authoritative link between a reported device index and a DRM card, so the
overlay kept inferring one positionally and each heuristic broke on a new host
shape -- foreign adapters on earlier DRM slots, cards with no VRAM sysfs, and
most recently amdgpu-bound adapters HIP cannot enumerate, which the count guard
could only catch on an unmasked host and therefore missed under any mask.
Use the link ROCm itself enumerates from. KFD topology
(/sys/class/kfd/kfd/topology/nodes/<N>/properties) lists exactly the GPUs HIP
exposes -- GPU nodes in node-id order are HIP's device order -- and each carries
its PCI location, so index N there IS physical device N with a stable identity.
DRM sysfs now supplies system-wide VRAM keyed by that same PCI address, and the
overlay is a join on it.
Every previous skew becomes a failed join rather than a misattribution: an
unenumerable adapter has no KFD node so it never takes an ordinal, a foreign
adapter contributes no entry, and a masked subset resolves each physical index
directly. That removes the count heuristic and its mask exception entirely. With
no KFD topology there is no identity to join on, so the overlay is skipped rather
than guessing positionally.
* studio: require verified host visibility and AMD-only KFD nodes
Three ways the identity map could still be built on a false premise:
- The NVIDIA open kernel module registers KFD topology nodes with a positive
SIMD count, so an earlier NVIDIA node shifted every AMD ordinal and ROCm
device 1 resolved to AMD GPU 0. GPU nodes now require vendor_id 4098 (0x1002),
the same filter install.sh already applies for this exact reason.
- A GPU node with an unreadable properties file or no location_id was skipped,
which silently shifted every later ordinal. Both now fail the whole map
closed, so the overlay is disabled rather than misattributing.
- A container exposing only some render devices through device cgroups sets no
visibility variable, yet torch compacts what it can see to ordinals from zero
while the host-mounted KFD and DRM trees still list every GPU. Nothing in the
reported payload distinguishes that from a full host, and torch exposes no PCI
id to check against, so the overlay now runs only when host visibility is
positively verified: no visibility mask AND device count equal to the host GPU
count. That also subsumes the previous layered-mask and GPU_DEVICE_ORDINAL
checks, so _rocm_device_index_unreliable() is gone.
This trades coverage for correctness: masked subsets and filtered containers now
keep torch's process-local figures instead of a mapping that cannot be verified.
* Fix the multi-GPU VRAM overlay docstring for PR #7216
The docstring claimed a reordering mask keeps each card on the right GPU,
but the overlay skips any active visibility mask and keeps torch's figures.
State the actual gating instead.
* Tighten comments in the multi-GPU VRAM overlay and its tests
Collapse the verbose docstrings and inline explanations added for the Linux
ROCm system-wide VRAM overlay to succinct one-liners, keeping the non-obvious
rationale (fail-closed KFD mapping, PCI-identity join, mask gating, the 10%
whole-card guard). Comments only, no behavior change.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
* Improve unsloth start runtime lifecycle
* Remove speculative Gemma prompt override
* Polish model download progress output
* Refine unsloth start status output
* Clarify unsloth readiness banner
* Clarify model reuse and switching output
* Queue model switches behind active inference
* Tighten unsloth start model switching
* Reduce model switch bookkeeping
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix Studio re-exec compatibility
* Recheck sidecar reservation after inference drain
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Pass start marker through child environment
* Fix key redaction, switch-waiter ordering, and stop/messaging gaps for PR #7313
- Redact minted sk-unsloth keys from the startup-failure log tail: the early
key marker lands in the server log before the model load finishes, so a
load-phase crash printed a live key to the terminal
- Deregister a finished switch waiter before releasing the swap gate so a
swap on another event loop cannot count it as still queued and unload the
model the finished request is about to generate against
- Warn on same-repo quant switches: an explicit variant replaces the resident
weights for every attached session, but the repo ids match so no switch
warning was printed
- Note the agent exit code when it is nonzero so the server keep-alive
message does not read as a successful session
- Use taskkill /T in unsloth studio stop so llama-server children stop too
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten comments in start, studio, and inference changes
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Studio: reuse MLX prompt cache across turns instead of re-prefilling
* clean up
* key prompt cache on what the KV covers
* skip windowed KV caches past their window
* verify prefix coverage before caching KV
* Recognize Radeon 8065S (Gorgon Halo / Ryzen AI Max 400) as gfx1151
* Classify Radeon 8065S (Gorgon Halo) as unified memory in ROCm OOM guard
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix resume training crash recovery and MLX checkpoints
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: preserve interrupted stop-and-save output_dir, verify MLX checkpoint
- finish_run: add clear_output_dir flag; preserve output_dir for stopped/error
unless cancel explicitly clears it (fixes pump finalization wiping persisted path).
- training pump: pass interrupted stop-and-save context into finalize_run_in_db.
- MLX stop-and-save: verify resumable checkpoint exists before sending complete;
return bool from _write_mlx_stop_checkpoint and add regression tests.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address Codex review: MLX current-step checkpoint and cancel error finalize
- Only skip MLX stop checkpoint write when checkpoint-{current_step} exists;
stale periodic checkpoints no longer mask missing stop saves.
- Pass clear_output_dir through error-event finalization so Stop-without-save
cannot leave a persisted output_dir that still offers Resume.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review
* Address more reviews
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* more reviews
* clear in-memory output_dir on interrupted cancel
* allow resuming errored runs at the final step
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Clear persisted output_dir in cancel watchdog path
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Write MLX stop checkpoint in stop path, keep output_dir on crash finalize
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio): harden resumable run finalization
* fix(studio): defer safetensors checkpoint import
* fix(studio): reject stale training cancellation
* fix(studio): replay null resume targets
* fix(studio): serialize terminal cancellation
* Harden resume checkpoint validation and fix stop-save cleanup
- Reject unrecognized shard formats and keep indexed shard paths inside the checkpoint dir
- Require a non-empty tensor record when validating .pt/.bin optimizer and model state
- Always finalize TensorBoard and W&B on stop-save-failure exits
- Refuse writing an MLX stop checkpoint through a symlinked directory
- Clarify the resume rejection message to cover errored runs
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten resume/checkpoint comments
* Recover resumability when a valid stop checkpoint landed
- Re-validate the current-step checkpoint in the dead-worker and error finalization paths so a stop-and-save that actually wrote a valid checkpoint is not wrongly marked error/resume_blocked
- Accept a valid tensor-free optimizer state (e.g. SGD without momentum); the model-state check still requires real tensors
- Include errored runs in the frontend resume rejection message
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lyxot <longyixing331@gmail.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
* refactor(studio): move chat model picker into features/model-picker
Relocate model-selector + its support files from components/assistant-ui
into a self-contained features/model-picker feature (own barrel), mirroring
the modular Hub layout. Pure move + import repoint; no behaviour change.
* feat(model-picker): add per-model config persistence layer
Superset PerModelConfig (customContextLength, kvCacheDtype, speculativeType,
specDraftNMax, tensorParallel, chatTemplateOverride, trustRemoteCode) persisted
to localStorage (unsloth_model_configs) with schema versioning + LRU budget.
KV-dtype and speculative value sets match main's sidebar (no q4_0/ngram-simple).
Reuses features/hub/lib/model-identity for normalization; adds storage-key layer
and applyPerModelConfigToRuntime (sets tensorParallel, which the old PR omitted).
* feat(picker): modular backend for chat-template validate + default fetch
New studio/backend/picker package (schemas/service/routes) mounted at /api/picker:
- POST /api/picker/validate-chat-template (Jinja syntax validation, no false positives)
- GET /api/picker/chat-template/{model_name} (default template from tokenizer_config.json,
reusing get_cache_path/resolve_cached_repo_id_case; graceful null, no model-code exec)
Frontend api/templates.ts client + hooks/use-model-defaults lazy cache. No backend
changes to the existing inference load route (per-model load fields already supported).
* feat(model-picker): bind picker on-device list to shared hub inventory
Picker now sources cached + local models from useHubInventory (the Hub's shared
store) via a thin adapter, replacing its own /api/models/* fetchers + module
caches. Hub, download manager, and picker now share one source of truth, so
completed downloads reflect in the picker automatically. Partial/live-download
rows are filtered from the cached lists (unchanged rendering). Local naming/search
preserved via additive LocalInventoryRow modelId/displayName. Variant expander,
scan-folder management, recommended-fit, search, external providers untouched.
Known minor: cached 'Downloaded date' sort tiebreak degrades to alphabetical
(hub cached rows carry no mtime); default 'recent' (load-time) sort preserved.
* feat(model-picker): per-model config step inside the picker
Picking a (non-external) model now opens an in-picker config view built from
main's current load controls (context length, KV cache dtype, speculative
decoding, draft tokens, tensor parallel) plus a chat-template editor backed by
the picker validate/default endpoints. 'Remember for this model' persists the
config per model+variant; Run forwards the config to the existing load flow via
meta.config. External models bypass the step. Two-view orchestration lives in
model-selector (single interception point); pickers.tsx call sites untouched.
trustRemoteCode dropped from PerModelConfig to preserve main's per-load consent.
* feat(chat): apply/persist per-model config through the load flow
handleCheckpointChange threads meta.config into the selection; stageOrLoad and
the autoload/Hub-run paths now apply the picker config (explicit pick or saved
remembered config) via applyPerModelConfigToRuntime before staging/loading, with
keepSpeculative set so a remembered speculative mode survives the model switch.
Replaces the old remembered-load-settings seeding (resolveInitialConfig now the
single source). SelectedModelInput carries config.
* refactor(chat): remove per-model load config from the right sidebar
The load knobs (context, KV cache, speculative, draft tokens, tensor parallel)
and the chat-template editor now live only in the picker config step. The sheet's
Model section keeps the staged Load/Cancel flow (config is applied at pick time);
sampling params, system prompt, and RAG are unchanged. Deletes the superseded
remembered-load-settings module + the store's applyRememberedLoadSettings action,
removes the now-dead sheet state/imports, and points the settings reset at
unsloth_model_configs. Delete-cleanup deferred (stale config is LRU-capped).
* fix(model-picker): remove leftover sidebar-staging cogwheel + empty Model section
The downloaded-variant gear (ModelLoadSettingsAction) staged a model straight
into the right-sidebar Run-settings flow -- the old 'configure before load' path
now fully replaced by the in-picker config step. Removed the gear + its component.
Also gate the sheet's 'Model' section to staged picks only (pendingSelection):
after the load-knob strip its content is staged-only, so it was rendering an
empty section header whenever a model was merely loaded.
* chore(chat): remove dead per-model-config setters + modelControlsDisabled
After the load-config UI moved into the picker, the store's per-model setters
(setKvCacheDtype/setSpeculativeType/setSpecDraftNMax/setTensorParallel/
setCustomContextLength/setChatTemplateOverride) had zero callers
(applyPerModelConfigToRuntime writes via setState), and the sheet's
modelControlsDisabled was unreferenced. Verified dead across the whole tree.
* fix(chat): config-step Load actually loads (ignore Load-on-selection)
Root cause: with Settings > Chat > 'Load on selection' turned OFF, the config
step's load went down the deferred-staging path -- opening the right sidebar with
'<model> is staged, not loaded yet / Choose Load model'. The in-picker config step
IS the deliberate load action, so its Load now loads immediately (or downloads +
auto-loads when not cached) regardless of the toggle. Renamed the button
'Run model' -> 'Load model' to match. Native/dropped picks still honor the toggle.
* refactor(chat,hub): retire 'Load on selection' — config step is the only load flow
The in-picker config step (and the Hub Run button) now fully supersede the old
stage-to-sidebar flow, so the Load-on-selection toggle is removed everywhere:
- chat stageOrLoad: every pick loads immediately, or downloads + auto-loads when
not cached (the previous default behaviour, now universal).
- hub Run: drops the stage branch; downloaded GGUFs load directly with their saved
per-model config (no collision with the chat config step — both end at selectModel).
- store: removed loadOnSelection field/setter/key/default; Settings>Chat toggle and
its settings-reset entry removed.
- staged sidebar section is now a download-progress view (auto-loads on completion).
No manual staging remains; stageModel is used only for background auto-load downloads.
* feat(model-picker): default chat template from GGUF + thread variant through config flow
Read the embedded tokenizer.chat_template from GGUF files (read_gguf_chat_template
in gguf_metadata) and use it as the per-model default. Plumb gguf_variant through
the picker service, /api/picker/chat-template route, frontend templates API, and
use-model-defaults so the right variant's template is fetched.
Also refine the picker config-page/model-selector wiring, drop the dead
ggufNativeContextLength runtime path, and add the per-model-config storage keys to
the settings prefs export.
* feat(model-picker): read safetensors chat template + hide editor where it has no effect
Resolve the default chat template for safetensors models: prefer the modern
chat_template.jinja, fall back to the tokenizer_config.json chat_template field,
then chat_template.json (multimodal processor), then the GGUF embedded template.
Applied to local dirs, the HF cache snapshot scan, and the HF remote fetch.
Hide the chat-template editor in the picker for safetensors models — the override
is only applied at load by the GGUF/llama.cpp backend, so editing it on safetensors
currently has no effect. GGUF keeps the editor. Nothing removed; the dialog stays
for when the safetensors apply path is wired up in a later branch.
* fix(model-picker): set legacy-migration flag only after the write succeeds
Set unsloth_model_configs_migrated only once writeMap confirms the migrated
map persisted, so a quota/storage failure no longer marks migration done and
silently drops the user's pre-existing remembered settings — the next load retries.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* MVP model picker fixes
* MVP picker config fix
* MVP safetensors config
* MVP max seq config
* MVP max seq fix
* Fix static max tokens cap ignoring model context
* Fix picker GGUF scan parity
* fix(studio): harden model picker config loading
Apply remembered per-model configs consistently from picker and Hub loads, keep default configs from overriding standing speculative settings, add config access for direct local GGUF files, and support saving or forgetting active model settings without a reload.
* Fix model picker config flow
* Fix model picker config loads
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Avoid recursive per-model config migration reads
* Apply the displayed context length when loading a GGUF
* Fix template validation, cached template lookup, and failed load rollback
- Validate chat templates with the loopcontrols extension so templates
that use break or continue tags pass the picker validator, matching the
inference renderer that already accepts them.
- Read the default chat template from the newest cache snapshot rather than
an arbitrary iterdir order, so an older cached revision no longer prefills
a stale template.
- Capture the runtime per-model config before a load and reapply it when the
load fails, so a failed switch leaves the active model context, KV cache,
template, and speculative settings as they were.
* Make chat template view only for safetensors models
Custom chat template overrides are applied at inference only for GGUF
models, which pass the template to llama-server. The safetensors backend
renders with the model built-in template and ignores the override, so
editing it would save a value that never loads. For safetensors the
config page now opens the template as a read-only preview with a note
that editing is not available yet. This can become editable once
inference support for custom safetensors templates lands in main.
* Fix model picker config edge cases
- Restore prior runtime config when a load no-ops for the active model
- Cap the picker validator request body via the protected prefixes
- Keep the GGUF context slider max above the loaded context
- Fetch subfolder chat templates for uncached Hub repos
- Show the compare side config when reopening the picker
* Keep saved GGUF context above the fallback ceiling
* Show the model config in the run settings sidebar
* Fix model config sidebar reset and context slider
- Stack the remember toggle and action buttons in the sidebar
- Reset the config to defaults instead of the loaded values
- Fetch the native context so the slider max is not the loaded value
* Fix model picker config and download regressions
- Run picker chat template routes off the event loop
- Depth and root guard local template directory scans
- Restore download manager flow for uncached hub picks
- Apply per model context length on reload
- Import model picker symbols from the feature barrel
* Fix model picker config and cached download sorting
- Restore load settings when a Hub run is rejected mid load
- Reuse one NumericValueInput instead of a duplicate copy
- Fix double decode of the model name in the template route
- Remove the unused reset-to-loaded settings action
- Fix cached model download sorting
* Fix model picker per-model config edge cases
Honor a saved or typed max seq length above the model's native context so
RoPE extended values are no longer clamped and silently overwritten. Allow
typing past native while the slider keeps native as a soft ceiling.
Guard the fetch success paths in use-model-defaults against an aborted
signal, and refetch when the HF token changes.
Hash the chat template content in the sidebar remount key instead of its
length. Enable reset for a GGUF whose native context is unknown, and floor
the context slider max so it can never fall below the min.
* Fix GGUF context auto-fit and gated model config token
Stop forcing a 32768 context when a GGUF native context is unknown so the backend auto-fits to VRAM again, while still honoring an explicit context edit.
Send the HF token as a query param so gated safetensors models resolve their max position embeddings.
Derive model default state during render to drop the set-state-in-effect calls.
* Fix native GGUF context ceiling and guard picker template reads
Restore the native context store field so the sidebar slider keeps the
full ceiling for drag and drop GGUFs. Limit local chat template reads to
the browse allowlist, skip malformed repo ids, and drop unused model
picker exports.
* Fix model picker lint boundaries
* Fix model picker review findings
Chat template editor never seeded its draft. Radix only calls onOpenChange
from internal events, so the seed in the nextOpen branch was dead and a model
with a saved override opened empty. Saving then cleared the override. Drop the
dead branch, treat draft as an untouched sentinel, and reset it on every close.
Uncached Hub picks could auto load a model after the user left the chat. Main
detached the staged pick on route exit and on chat context change. Carry the
context key on the pending pick and skip the load when it no longer matches.
Also clear configTarget when the picker closes, restore the onUpdated ref so
variant rows stop resubscribing on every parent render, skip the LRU write when
the entry is already most recent, import NumericValueInput relatively, and drop
the unused ModelUpdateAction barrel export.
* Preserve GGUF context on active reload
* Fix model picker per-model config regressions
- Stop reloading the already loaded model on re-pick
- Hide infra models from the chat picker
- Detect vision support on cached GGUF repos
- Honor saved maxSeqLength on auto load
- Restore default chat template for local GGUFs
- Warn on save failure and revert config on cancel
- Refetch picker inventory on open
- Persist read only per model config safely
* Fix stale model auto load
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix model picker numeric input sizing and constraints
Size value inputs to their content so long context lengths are not clipped,
restrict them to numeric characters, and stop the speculative decoding label
from truncating in the sidebar.
* Fix picker CI tests and harden chat template resolution for PR #6647
- tests: point the descender guard at the moved model-selector.tsx path
- tests: exclude the disabled Reload model button from the regenerate locator so .first targets the real Regenerate
- picker/service.py: reject symlinked template/gguf leaves that resolve outside the browse allowlist (HF cache reads unchanged)
- compare mode: resolve each pane's own remembered chat template instead of inheriting the other pane's from the store
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Protect future-schema per-model configs from deletion for PR #6647
savePerModelConfig already refuses to overwrite a stored config whose schema version is newer than this client understands, but deletePerModelConfig did not. Unchecking Remember on an older client therefore silently destroyed a newer client's saved config. Apply the same guard on delete and surface the blocked case through the existing saveFailed toast.
* Protect future-schema per-model configs from quota eviction for PR #6647
The save and delete guards already refuse to touch a stored config whose schema version is newer than this client understands, but the quota-eviction path did not, so a full store on an older client could still evict a newer client's config. Skip future-schema entries when evicting and fail the save if the budget cannot be met without them.
* Fix GGUF context persistence, compare context, and rollback settings for PR #6647
Persist a GGUF context override from the user's intent instead of collapsing it against the loaded context, which reintroduced the context-reset (f4838782cb reverted the native-baseline fix). model-config-page now collapses the saved value against native, and use-chat-model-runtime and chat-adapter retain the requested context on load so re-saving another setting keeps the override; a null request stays null so a VRAM auto-fit never becomes a stored override.
shared-composer: a compare pane with no explicit GGUF context now loads at native (0) like single-view, not the session maxSeqLength that silently shrank the shown context.
use-chat-model-runtime: restore the previous model's KV cache dtype and chat template on a failed-load rollback so it runs as it was, not with backend defaults.
* Preserve native path token when reloading the active model for PR #6647
handleReloadActiveModel rebuilt the selection without the store's activeNativePathToken, so reloading a file-picked GGUF after a settings change validated the display label as a repo/path and failed. Thread the active native token through the reload selection so native-loaded models reopen correctly.
* Make picker template validation resilient and accept HF generation tags for PR #6647
Import Jinja lazily inside validate_chat_template so a backend without the optional jinja2 package (GGUF-only installs) still starts instead of raising ModuleNotFoundError at import time. Register a no-op extension for the Transformers {% generation %} assistant-mask tag so pasting a valid HF chat template validates, matching the renderer, rather than being rejected as an unknown tag.
* Honor remembered compare config and parse processor chat_template.json for PR #6647
* Fix failed-load rollback context and processor template map fallback for PR #6647
* Restore speculative decoding config on failed-switch rollback
When a model switch fails after the previous model was unloaded, the
rollback reload restored tensor_parallel, KV cache dtype and the chat
template override, but omitted speculative_type and spec_draft_n_max and
cleared their loaded shadows to null. The previous model therefore came
back running at backend defaults (speculation off) while the UI still
showed it enabled, and the status resync confirmed the off state. Resend
the previous model's speculative settings in the rollback load and keep
the store's active and loaded speculative fields in sync with them.
* Reset max sequence length when a model has no saved config
applyPerModelConfigToRuntime reset every per-model field except
maxSeqLength, which it only wrote when the incoming config had one.
maxSeqLength is the sole field carried on store.params, so selecting a
model with no remembered config left the previous model's value in place
and later loaded the new model at that leaked length. Fall back to the
standing default so an unremembered model loads at its own default.
* Surface a message when a variant update cannot start
startManagedUpdate handled the conflict and error start outcomes but let
busy fall through as if the update began, so the confirm dialog closed
with no job created and the cached variant stayed stale. Show an info
message when the repo is busy with a sibling transfer so the click is
not silently dropped.
* Keep per-model speculative choices out of the global default
A staged load with a per-model or one-off config sets keepSpeculative,
which already skips reading the global speculative preference. The
matching save still ran unconditionally, so the model-specific choice was
written to the global unsloth_chat_speculative_type and a later model with
no saved config started from it instead of Auto. Skip saveSpeculativeType
when keepSpeculative so the per-model choice stays isolated.
* Seed non-active model settings from the app default max length
The Run settings page captured initialMaxSeqLength from the loaded
model's runtime params and fell back to it for a model with no saved
config. Opening settings for a different, unloaded model and clicking
Load then sent the active model's context (for example 64k) instead of
the 4096 default, risking validation failures or OOMs. Seed the default
for non-active models and keep the runtime value only for the active one.
* Prefer sidecar tokenizer chat template over the GGUF copy for variants
_chat_template_from_dir returned the embedded GGUF template first when a
variant was selected, reversing the tokenizer-first precedence of the
no-variant path. A model whose chat_template.jinja or tokenizer_config.json
supersedes a stale embedded template then got the wrong template on
variant selection. Keep tokenizer files first regardless of variant; the
variant only picks which GGUF is the fallback. Adds regression tests for
both the tokenizer-wins and gguf-fallback cases.
* Keep per-model speculative choices load-local in autoload and compare
The interactive load path treats a per-model speculative choice as
load-local and skips writing it to the global default. Autoload and
generalized compare still called saveSpeculativeType unconditionally, so a
remembered off or ngram setting leaked into unsloth_chat_speculative_type
and later models with no saved config inherited it. Persist the global
preference only when the value came from the global settings.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: record the compare pane's loaded context in runtime state so the active model's settings and any reload or save use it, not the previous context
* Studio: notify the user when a Hub autoload can't start because another download for the model is already running, instead of silently dropping it
* Studio: drop the merge's orphaned staged-model store helpers and unused alert imports
The main merge left isPendingGguf and pendingSelectionMatches referencing the
removed PendingModelSelection type, and the alert-dialog/alert imports unused
after the permission-mode dropdown replaced the bypass dialog, so tsc -b failed.
* Studio: cache a null default chat template so the viewer stops re-fetching it
A model with no sidecar or embedded template resolves to a terminal null, but
that result was never cached, so reopening the template viewer re-ran the
backend and Hugging Face lookup every time.
* Studio: detect direct-file GGUFs in run settings so Max Tokens uses their context
A GGUF loaded from a local file or custom folder has no variant label, so the
run-settings panel treated it as non-GGUF and clamped Max Tokens to the session
max_seq_length instead of the loaded GGUF context. Detect it via the reported
GGUF context and the .gguf checkpoint suffix, matching the chat page.
* Studio: prompt to re-select a local model file when its lease expired before reload
A file-picked GGUF is reachable only through a native path token that the
desktop host prunes after a TTL. Reloading reused that token blindly, so a
reload long after the initial load failed with an opaque error. Track the
token's expiry and, when it has passed, ask the user to re-select the file
instead of attempting a doomed reload.
* Fix descender-clipping test to tolerate sidebar layout utilities
The sidebar account-block div carries layout utilities (min-w-0, flex-1)
between 'flex' and 'flex-col', so the descender-clipping guard's regex,
which required 'flex' immediately followed by 'flex-col', no longer matched
and the test failed to locate the account-block div. Generalize the prefix
to allow intervening flex utilities while still capturing the leading-*
class before the collapsible visibility utility and asserting leading-tight,
so the guard against clipped glyph descenders is fully preserved.
* Harden picker chat-template resolution
Enforce the 64 KiB chat-template contract at the validate endpoint's request
model so a direct caller cannot submit a template far larger than the frontend
allows (MaxBodyMiddleware only bounds the whole request body, not this field);
oversized templates now return a clean 422.
Apply sidecar-over-GGUF template precedence globally across cached snapshots
instead of per snapshot. A repo with multiple cached revisions previously
returned the first snapshot's template, so a newer GGUF-only revision could
win over an older revision's maintained chat_template.jinja sidecar, which
contradicted the documented intent that sidecars supersede the embedded copy.
* Guard per-model config against future-schema and lossy migration
Two forward-compatibility gaps in the versioned per-model config store:
- The load/apply path returned and normalized a stored record without checking
its schema version, so a record written by a newer client was reinterpreted
under the current schema and applied to a live model load, even though save,
delete and eviction all refuse to touch future-schema records. Reject
future-schema records on load too.
- The one-time legacy migration enforced the storage budget without protecting
the entries it had just migrated and set the completion flag unconditionally.
When storage was already full of future-schema records (which are unevictable
by an older client), the migrated entries were the only evictable ones and
could be dropped while migration was still marked complete. Protect the
migrated keys during eviction and only mark migration complete when they
survive, so it retries once space frees up.
* Discard chat-template validation results after the dialog closes
Server-side template validation is async, but closing or cancelling the editor
did not abort it, so a late-arriving valid response still called onSave and
applied a template the user had already dismissed. Track a validation token
that is bumped on close and ignore any validation result whose token is stale.
* Record native lease expiry when loading a picked GGUF from the chip
The pending-native-model chip loaded via stageOrLoad directly, bypassing
loadNativeModelIntent, so activeNativePathExpiresAtMs was never recorded for a
chip-loaded file. A later reload then either skipped the lease-expiry guard
entirely (expiry left null) or compared against a previously loaded file's
stale expiry, so reload could reuse an already-pruned token or wrongly block a
still-valid one. Route the chip through loadNativeModelIntent, which builds the
same selection and records the expiry.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Prefer sidecar template for a directly selected local GGUF file
A direct .gguf file path read its embedded chat template without checking the
parent directory for a maintained sidecar (chat_template.jinja /
tokenizer_config.json), while directory and variant selections already prefer
the sidecar. That let the config editor preview or save a stale embedded
template for the same model depending on how it was selected. Check the parent
directory sidecars first, then fall back to the embedded copy, and cover both
paths with tests.
* Resolve cached chat template per revision, newest first
The earlier change searched every cached snapshot for a sidecar before
considering any snapshot's embedded GGUF template, which let an obsolete sidecar
from an older revision override the newest revision's template. Restore
per-snapshot resolution (newest first): a revision's sidecar still supersedes
its own embedded GGUF copy, but a newer revision is no longer overridden by an
older revision's sidecar.
* Preserve autoload transport conflicts and surface background busy downloads
- When a Hub autoload hits a transport conflict, keep pendingHubAutoLoad bound
instead of clearing it. Clearing it re-keyed the download surface and its
cleanup cancelled the conflict the toast tells the user to resolve, so the
Hub resume affordance was gone the moment it appeared. Return early on
conflict, mirroring the started branch, so resolving it from the Hub still
auto-loads on completion.
- The background-download branch handled started and conflict but silently
dropped a busy outcome, leaving the user with no feedback when a peer variant
of the same repo was already downloading. Surface the same busy toast the
autoload path uses.
* Fix context length, GGUF template, fetch state and lease expiry bugs
Keep explicit context length values instead of collapsing to null at
native. The collapse made the slider jump back at the native maximum
and made Reload load the previous context instead of the chosen one.
Prefer the first split when resolving a GGUF without a variant. Later
splits carry no chat template metadata, so picking the largest file
could return no template for a sharded model.
Clear stale fetch state when template and metadata lookups retry, so
a previous terminal error is not shown while a new fetch is running.
Record native path lease expiry together with the token when a load
commits. The expiry was written by only one load path and even when
the load did not start, so a reload could be blocked with an expired
file message for a still valid token.
* fix(model-picker): resolve review findings across config, inventory, and templates
- Apply remembered per-model config in the training-compare chat handoff so a
prior model's customContextLength no longer leaks into the next load
- Match GGUF variant labels with the inventory extractor too, so cached
no-quant-token files resolve their default chat template
- Show "Auto" instead of a fabricated 32768 when native context is unknown
- Reuse the identical staged auto-load object on same-pick so a re-pick during
download pre-flight no longer disarms auto-load via "busy"
- Union supports_vision when deduping cross-cache inventory rows
- Serve hidden-model needles from a new GET /api/hub/hidden-models endpoint and
merge them client-side, covering runtime-configured RAG embedders
- Clamp GET chat templates to MAX_CHAT_TEMPLATE_BYTES (route + jinja sidecar),
matching the validate endpoint's contract
- Lower-clamp stored customContextLength to shared CONTEXT_LENGTH_MIN
- Wipe unsloth_chat_load_on_selection in Settings "Reset all"
- Drop stale pendingHasContext comment describing deleted staging machinery
* Fix stale defaults cache, token in query string and rounded up context ceiling
Refresh cached chat template and max position data when a model update
completes. Send the HF token for model config requests in the dedicated
header instead of the URL. Snap the native sequence length ceiling down
to the nearest step so the slider cannot exceed the declared maximum.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix compare pane reverting active checkpoint on non-GGUF load
Re-read runtime params after setCheckpoint so the fresh checkpoint is
kept instead of being overwritten by the pre-setCheckpoint snapshot.
* Send the HF token via header for the vision and embedding checks
checkVisionModel and checkEmbeddingModel still passed the HuggingFace
token as a ?hf_token= query parameter, so it landed in server access
logs, proxy logs, and browser history. Move them to the
X-Unsloth-HF-Token header like getModelConfig already does, and accept
the header on the check-vision and check-embedding routes with the
existing query parameter kept as a fallback for older clients.
* Cap the chat template on the model load path
The load endpoint accepted an unbounded chat_template_override, so a
direct caller could hand llama.cpp an arbitrarily large Jinja template
even though the frontend, the validate endpoint, and the read paths all
enforce the 64 KiB limit. Reuse MAX_CHAT_TEMPLATE_BYTES in the
LoadRequest validator, rejecting oversized templates with a fast
character-count check before the exact UTF-8 byte check.
* Protect existing per-model configs during legacy migration
When the one-time legacy import pushes the store over budget, eviction
now protects the entries the user already has and drops only the
just-migrated legacy entries, so importing old load settings can never
discard a newer per-model config.
* Reset clears the context override instead of pinning the native value
Reset wrote the discovered native context into customContextLength for
GGUF models, but isDefaultConfig treats any non-null customContextLength
as an explicit pin, so Reset with Remember enabled persisted a fixed
context and future loads stopped using the native auto context. Reset
now restores the full default (customContextLength null); the native
value is still shown through the existing display fallback.
* Bound chat-template sidecar reads to a size limit
The chat_template.json, tokenizer_config.json, and Hub-downloaded sidecar
readers decoded and json-parsed the whole file before the extracted
template hit the 64 KiB response cap, so an oversized metadata file could
exhaust memory. Read them through a bounded reader (4 MiB envelope) that
returns None when the file is larger, matching the existing chat_template.jinja
size guard. Adds tests for oversized tokenizer_config.json and chat_template.json.
* Keep the native-path token and lease expiry in sync
Rollback after a failed reload restored the previous token but left the
failed load's expiry in the store, so a later reload could be falsely
blocked as expired (token A paired with load B's lease). Restore the
previous lease alongside the token, and clear the expiry wherever the
token is cleared on a non-GGUF transition, so the two never diverge.
* Clear the native file lease on compare-pane loads
* Studio: add regression tests for the model-picker per-model-config
Guard the specific regressions that reverted the predecessor change:
- backend pytest (studio/backend/tests/test_model_picker_regression.py):
infra-model hiding, HF token via header with query fallback, and the
chat-template byte caps.
- source contracts (tests/studio/test_model_picker_contracts.py): the token
stays out of the URL, the context ceiling is floored, the native lease is
cleared on compare-load and restored on rollback, the default caches key on
the inventory version, and the hidden needles stay present.
- Playwright E2E (tests/studio/playwright_model_config.py) wired into
studio-ui-smoke.yml on port 18898: Context Length persists across a reload,
Reset clears the stored override, and infra models are absent from the picker.
- optional GPU-gated inference smoke (tests/studio/test_gpu_inference_smoke.py)
that auto-skips on GPU-less CI and stays short on a GPU.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: model pinning, row menus, hub inference settings, and inventory filters
Pinning
- Add a pinned models store (localStorage) with repo and per-quant pins
- Pinned section in the model selector's On Device list and the hub inventory,
with newest pins first so Pin to top lands on top
- Deleting a repo drops its pins
Row menus
- Replace loose row icons with a shared 3-dots menu (pin, reveal in file
manager, copy identifier, copy path, delete) on picker rows, hub quant rows,
the hub run bar, and on-device inventory rows
- Menus only render for models actually on disk; platform-aware reveal labels
- Backend: cached-model-path and reveal-cached-model endpoints resolving
managed HF-cache repos only
Hub inference settings
- Gear in the GGUF run bar opens an Inference settings dialog reusing the chat
page's controls: model config (context length, KV cache, speculative
decoding, chat template), system prompt, reasoning, sampling, tools and
retrieval
Inventory
- Model-type filter (text, vision, embedding, STT, TTS, diffusion) beside the
sort pill, both with a sort icon, capped widths and truncation so the
On device heading never wraps
- Unsloth-owned repos without an upstream provider logo fall back to the
Unsloth mascot avatar
- Discover / On Device tabs widened; hub search bar narrowed to match
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: revert the Unsloth mascot avatar fallback
Unsloth-owned repos without an upstream provider match go back to the
colored-initial tile, and unslothai is no longer a relabeled owner.
* Studio: run-bar options on single models, and aligned type/capability filters
- Give single-model (non-GGUF) run bars the same 3-dots options menu and
settings gear as GGUF, at repo level
- Drop Pin to top from the run-bar menus; pinning stays in the On Device list
- Add an Image to text (diffusion) capability with detection, and surface it
in both the hub Discover capability filter and the On Device type filter
- Align the On Device type filter with the Discover capability options and
share the same detection so both dropdowns match
* Studio: apply hub inference config on reload, eject action, and run-bar polish
- Fix inference settings not applying: the hub dialog now writes the config to
the runtime before reload, matching the chat page (selectModel reads runtime
state, not the selection)
- Order the settings gear before the 3-dots menu in the run bars
- Replace the loaded-model run-bar action (New Chat) with Eject, wired through
the inspector to the hub's ejectModel
- Truncate the results heading so a long search query clips instead of
overlapping the header pills in split view
- Use a plain magnifying-glass icon for the no-results empty state
* Studio: fix GPU settings loss, load guards, pins, filters, and cached paths
Reloading a model from the chat sidebar or the hub gear dialog rebuilt the
per-model config without the GPU memory fields, so manual GPU layers, MoE
placement, and the GPU pick were reset on every reload and could be saved
over a remembered config. The active config now comes from a shared
useActiveModelConfig hook that carries the GPU fields for GGUF models, and
the sidebar remount signature tracks them through a shared gpuFieldsSignature
helper.
The in-flight load guard lived in a ref inside each useChatModelRuntime
instance, so the chat page, hub page, and gear dialog could not see each
other's loads. A load started from the gear dialog left the hub page free to
eject the model mid-reload or start a second concurrent load. The runtime
store now records the loading pick, selectModel checks it across instances,
and ejectModel refuses with a toast while any load is in flight.
The cached-model-path endpoint matched GGUF files by basename and excluded
only mmproj, so Copy path and Reveal could return an MTP drafter for a quant
and returned 404 for directory layouts like BF16/model-00001.gguf. Variant
files are now resolved from snapshot-relative paths with the same drafter,
mmproj, and big-endian exclusions as the load path, shared through a new
_main_variant_gguf_label helper.
Hub and picker fixes:
- rename the diffusion capability label from "Image to text" to
"Image generation", since it detects image generators
- validate pinned quants through the cached variant listing, keep the last
verified set while revalidating, and drop deleted quants immediately
- pass a measured scroll margin to the on-device virtual list so rows past
the overscan stay visible below the pinned block
- keep the delete menu for stopped partial safetensors downloads
- give the inventory type filter a reset in Clear filters, a truthful empty
state with a Show all types action, and hide it on the datasets view
- order picker pinned rows by pin recency, include pinned matches in the
empty-state check, and sync pins across browser tabs
- count only the visible rows in the On device list header
Tests: contract checks for each fix in test_model_picker_contracts.py and a
backend test for the variant label selection.
* Studio: reveal cached models in Windows Explorer under WSL
The reveal endpoint only branched on macOS, Windows, and generic Linux.
Under WSL the Linux branch spawned xdg-open, which is missing on a stock
distro without a Linux desktop, so the request failed with a 500 and the
UI showed a failed to open file manager error.
WSL is now detected with the existing helper and the path is converted
with wslpath before opening explorer.exe, selecting the file the same
way native Windows does. Directories open directly. When interop is
unavailable the old xdg-open fallback still runs. The macOS, native
Windows, and native Linux branches are unchanged, and the Tauri app is
covered since its hub reveal calls this same local endpoint.
Tests: platform guards for the WSL reveal, the interop fallback, and
the unchanged native Linux behavior in tests/studio/test_reveal_file_manager.py.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Adjust model picker row spacing and cogwheel hover consistency
* Studio: exact hidden model ids and newest revision cached paths
A custom RAG embedder repo was published to the frontend as a basename
substring needle, so a generic name like org/model could hide unrelated
models in the pickers. The hidden-models endpoint now sends full repo ids
that are matched exactly.
Copy path and Reveal picked a GGUF variant from an arbitrary cache
revision when the same file existed in more than one. The newest revision
now wins, matching the whole repo lookup.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix model picker GPU config, metadata, and cache selection
Load each compare model with its saved GPU memory mode, GPU layers, CPU MoE layers, and selected GPU IDs. Reconcile saved GPU IDs with the current hardware. Include the active native GGUF path token in metadata checks. Search all Hugging Face cache roots when resolving cached models and select the largest visible cache entry. Remove obsolete barrel exports and the staging-only GPU memory helper.
* Studio: hide hub inference settings gear for now
The cogwheel in the hub download cards is out of scope for this PR. The
dialog component stays in place and a TODO marks where the button
returns in a future PR.
* Refresh hidden model matchers
* Fix GGUF detection, compare context pin, and picker delete staleness
Treat any pick with a GGUF variant as GGUF in selectModel so the first
load after downloading an uncached quant validates and sizes with the
right GPU settings instead of unloading the current model on a wrong
preflight. Variant picks now also set isGguf on their selection meta.
Stop compare panes from inheriting the active model's context pin when
their own saved config says Auto. Null context in a remembered config
now means no pin, matching how the pane settings are shown.
Route picker deletes through the hub inventory client, which
invalidates the HF cache scan and the variants cache. The legacy
delete route left the scan cache warm, so deleted models reappeared
in the picker until the TTL expired. Removed the now unused legacy
delete client and updated the contract test to match.
* Studio: fix stale GGUF load-marker ordering test
The load-in-flight marker still precedes the hub-download guard and the
unload, but the llama_extra_args inheritance that used to sit between the
marker and the guard now runs ahead of the GGUF branch, so it is no
longer a landmark inside the sliced source. Drop it from the ordering
assertion and keep the marker -> guard -> unload invariant.
* Studio: fix per-model config edge cases in compare loads and saved defaults
- chat-settings-sheet: gate the MTP fallback note and context/VRAM warning on
the broader isGguf (variant, loaded gguf context, or .gguf suffix) instead of
isLoadedGguf, so direct-file and custom-folder GGUF loads still surface
those diagnostics.
- shared-composer: a compare pane's context now comes from its own config only
(a saved pin, else null for Auto/native). It no longer inherits the active
model's shared snapshot, which resolveFitMaxSeqLength treated as an explicit
pin and could load a pane at another model's context (VRAM/OOM), matching the
single-model load path.
- model-config-page: when an auto-fit GGUF is saved with fixed GPU layers
(Manual) and Remember, pin the displayed fitted context so a later fresh load
keeps the placement instead of sending native/0 and recreating the OOM.
- per-model-config: treat Auto GPU memory mode and Auto/default speculative type
as follow-global defaults; do not persist them as per-model overrides so later
global preference changes keep applying.
* Studio: gate vision capability on GGUF projectors and bound remote template downloads
- cache_inventory: only mark a cached repo vision-capable when it holds an actual
GGUF mmproj projector, not any file whose name merely contains "mmproj" (e.g.
mmproj_config.json), matching the runtime's GGUF-only projector detection.
- picker/service: pre-check the remote file size before downloading an uncached
repo's chat template / tokenizer config, so a maliciously large sidecar is
skipped instead of fetched and retained in full, mirroring the size gate the
local-file path already applies.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: add source-contract guards for the per-model-config edge-case fixes
Guard the four per-model-config fixes against silent regression in CI:
- local GGUF diagnostics gate on the broad isGguf, not the variant-only isLoadedGguf
- fixed-layer GGUF saves pin the displayed context
- Auto GPU mode and Auto/default speculative are not persisted as per-model overrides
- a compare pane's context comes from its own config, not the active model's snapshot
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: clear manual GPU knobs on Default and resolve local embedders before repo-id
- model-config-page: switching GPU Memory back to Default now clears the Manual-only
knobs (gpuLayers/nCpuMoe/selectedGpuIds); otherwise a remembered config kept stale
pins that a later load re-applied when the global GPU preference was Manual, despite
the page showing Default.
- routes/models hidden_model_matchers: resolve an existing local path before the repo-id
regex, mirroring is_hidden_model, so a local embedder shaped like "models/embedder" is
hidden by exact path instead of leaking as a chat model.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: add _is_mtp_drafter to the model_config stub in the export-paths test
routes/models.py imports _is_mtp_drafter from utils.models.model_config at module
load, but the lightweight stub in test_export_absolute_paths.py did not provide it,
so loading the module under the stub raised ImportError on Backend CI. Add the stub.
* Studio: read a picked GGUF's chat template through the native path lease
The picker chat-template GET has no native-path-lease plumbing, so a
desktop-picked (drag-drop) GGUF could not show its default chat template
in Run Settings until the model was loaded: the endpoint only receives
the display label, not the leased file path.
Read the embedded template through the existing lease-aware
/api/inference/validate probe instead. A new include_chat_template flag
resolves the granted canonical path and returns the GGUF's own embedded
template, never a sibling sidecar (the grant authorizes just that one
file); it skips the training guard like include_context_length and is
bounded by MAX_CHAT_TEMPLATE_BYTES. The frontend fetch mints a one-shot
validate-model lease when a native token is present and keeps the plain
GET path for HF and allowlisted local models.
Adds backend and source-contract regression tests.
* Studio: call worker.direct_wheel_url in the ROCm wheel-url test
The ROCm Mamba/SSM test referenced worker.py's private _direct_wheel_url,
but the worker imports the wheel helper under its public name
direct_wheel_url (utils.wheel_utils). When the worker module loads (its
imports resolve in CI), worker_mod._direct_wheel_url raised AttributeError;
the test only masked it by skipping when the worker could not be imported.
Call the name that actually exists so the assertion runs; it still returns
None for an empty cuda_major (ROCm).
* Studio: reset max sequence length to the app default, not the loaded value
For a non-GGUF active model, the per-model config seeds maxSeqLength from
the loaded runtime value so the panel opens showing the running context.
Reset set config.maxSeqLength to null, but the null fallback resolved back
to that captured runtime value, so the field kept showing the old custom
length and the config saved/reloaded it again. A remembered or active
max-length override therefore could not be cleared from Run settings.
Fall the null/default case back to the app default (clamped to the model's
native ceiling) instead of the active runtime snapshot, so Reset actually
clears the override. The initial view is unaffected: an active model's
config.maxSeqLength is already non-null, so it still shows the loaded value.
Adds a source-contract regression guard.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: persist default max length, refresh deleted quants, hide non-chat locals
Three follow-up fixes from review of the per-model-config picker:
- Max sequence length: the persisted per-model record now keeps config's
maxSeqLength (null after Reset) so isDefaultConfig can clear a remembered
override; the resolved app-default is substituted only into the load
request, never the saved record. Previously Reset saved the concrete
default and left the model pinned/remembered.
- GGUF variant expander: deleting a downloaded quant from a repo that still
has other cached quants now bumps the expander refresh key, so the removed
quant stops showing as downloaded and clickable (which would try to reload
the deleted file) until the repo is collapsed and reopened.
- Local picker rows: require capabilities.canChat before listing a local
models-folder / LM Studio row. A weightless folder (only config.json) is
classified non-chat, and toLocalModelInfo drops capabilities, so selecting
such a row would try to load a path the inventory already marked non-chat.
Adds source-contract regression guards for all three.
* Fix compare-pane and Reset context defaults in model picker
Two related per-model-config default regressions:
- A non-GGUF compare pane with no saved maxSeqLength fell back to the
active model's shared runtime snapshot, so comparing a saved 128K model
against an unconfigured pane loaded the latter at 128K and could OOM. It
now falls back to the shared app default (DEFAULT_MAX_SEQ_LENGTH), the
same fallback the single-model config path uses.
- contextAtDefault treated an explicit customContextLength equal to the
native ceiling as a default, which wedged the Reset button disabled for
a deliberate pin-to-native. It now counts as default only when there is
no override at all.
DEFAULT_MAX_SEQ_LENGTH becomes a single exported constant in
per-model-config.ts so the single-model config and the compare path share
one source of truth. Adds source-contract guards for both fixes.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Skip over-cap remote Jinja templates so the tokenizer template wins
The remote chat-template resolver bounded raw chat_template.jinja downloads
only by MAX_TEMPLATE_METADATA_BYTES (4 MiB), then returned the first
non-empty Jinja unconditionally. The picker route drops any template larger
than MAX_CHAT_TEMPLATE_BYTES (64 KiB), so an uncached repo whose
chat_template.jinja sits between 64 KiB and 4 MiB returned no template at
all, even when a valid smaller tokenizer_config.json template existed. The
local path already skips oversized .jinja files and falls through.
Gate the extracted Jinja on MAX_CHAT_TEMPLATE_BYTES and continue searching
when it exceeds the cap, matching _chat_template_from_jinja_file. The 4 MiB
download bound stays for JSON files that merely embed a small template. Adds
a regression test that a big Jinja plus a valid tokenizer config resolves to
the tokenizer template.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Guard legacy per-model-config migration idempotency
The v1->v2 localStorage migration (unsloth_load_settings ->
unsloth_model_configs) runs on every store read, so it must migrate exactly
once and never re-run, duplicate, or clobber a newer per-model config on a
reload or restart. That was covered only by a manual proof, so add durable
guards:
- Source-contract test pinning the three idempotency layers (the in-memory
legacyMigrationChecked guard, the persistent unsloth_model_configs_migrated
flag set in every terminal branch, and the non-overwriting Object.hasOwn
merge-skip) plus the readMap invocation. Reddens if any layer is dropped.
- Playwright model-config E2E: promote the legacy-migration step to a gating
check (soft_fail, which gates under the CI STUDIO_UI_STRICT=1) that the
migrated value is preserved and the flag is set, then reload again with a
fresh legacy seed present and assert the stored key set is unchanged, so a
second reload cannot re-migrate, duplicate, or clobber.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Note the migration E2E now gates idempotency under STUDIO_UI_STRICT
* Tighten model-picker per-model-config code comments
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
Co-authored-by: shimmyshimmer <shimmyshimmer@users.noreply.github.com>
Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local>
Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
* Studio: make repeated tab switches feel immediate
* Keep cached Studio navigation data fresh
* Make first Studio tab visits responsive
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Serve range requests uncompressed for immutable assets (PR #7271)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: test <test@test.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Studio (Windows): keep prompt caching on full GPU offload (#5692 follow-up)
The #5692 full-offload tuning also added --no-cache-prompt, which disables
in-VRAM prompt-prefix reuse. That is unrelated to the host-RAM KV checkpoints
#5692 fixed (--cache-ram 0 / --ctx-checkpoints 0): a fully offloaded model keeps
its KV cache in VRAM, so reusing a common prefix does not copy to system RAM and
does not cause the PCI-E overhead. --no-cache-prompt only forces every request to
re-prefill the whole prompt, which is small for short chats but severe for large
stable system prompts reused across calls (coding agents, long multi-turn chats).
Remove --no-cache-prompt; keep the checkpoint disables and the thread/OMP tuning.
_prompt_cache_disabled stays False (its default), so slot save/restore is intact.
Verified on a fully offloaded gemma GGUF: an identical repeated prompt reprefills
1 token instead of 2220.
* Guard against re-adding --no-cache-prompt to any llama-server command
Add a backend-wide test that AST-scans studio/backend and fails if
--no-cache-prompt is appended/extended/+= into a command. This locks in
the #7260 fix across every code path, not just load_model. Detecting the
flag or honouring a user-supplied one stays allowed.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: validate Hugging Face tokens before use
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: keep token validation failures non-blocking
* Studio: harden Hugging Face token preflight
* Studio: make token validation effect lint-safe
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
* Studio: make Stop and stall deadlines interrupt a wedged stream portably
The cancel watcher unblocks a stalled read by shutting the socket down from
another thread, which works on POSIX but not reliably on native Windows, where
Winsock does not dependably wake a recv() already in progress on another thread.
Wrap the httpcore network stream so the reader loops each read in short slices
and polls the cancel event itself. Stop and the stall deadlines now interrupt a
wedged mid-stream read without any cross-thread socket teardown, and a slow but
still-alive stream is never torn down. The POSIX shutdown path is preserved.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Honor the post-first-token stall timeout in the cancel-aware read
httpcore snapshots request.extensions timeout read once when the body
starts, so lowering it to the stall timeout after the first token never
reached the socket read and a one-token-then-silent server hung for the
full prefill window. Re-read the live extensions timeout per call and
bound each read by it, falling back to the httpcore-passed timeout when
absent so prefill and normal completion are unchanged.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: tighten comments in the llama.cpp stall timeout path
* Tighten comments in the stream stall cancel path
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <unslothshared@gmail.com>
* Studio: fix per-GPU VRAM reporting on Windows ROCm
On Windows ROCm without a HIP SDK, amd-smi is disabled and the System tab fell
back to torch mem_get_info, which reports free==total there (ROCm/ROCm#1909), so
used VRAM showed as 0. The perf-counter fallback also summed every adapter into a
single device with only GPU 0's total, hiding the second GPU.
Read per-adapter Dedicated Usage (LUID-instanced) for used and take each GPU's
total from torch properties, and treat the free==total case as unknown rather
than 0, so every GPU shows real usage. NVIDIA, Linux ROCm, Apple and CPU paths
are unchanged. Final validation needs a real Windows AMD box.
Fixes#7072
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: report unknown VRAM instead of fabricating or zeroing it
Two gaps in the Windows ROCm VRAM path. When more adapters are actively using
VRAM than are visible to the process (a GPU outside the visibility mask), the
per-adapter attribution paired usage by size and fabricated a per-GPU value;
report unknown for every device in that case rather than mis-assign. And the
System API turned an unknown (None) used value into 0 with ``or 0``, then
reported the full card as free, re-hiding the exact case this change surfaces;
keep None so the UI shows unknown.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Render unknown VRAM as Unknown instead of zero in the System tab
The backend reports null usage when it is unknown (e.g. the Windows ROCm
perf counter is unavailable or localized), but the System tab coerced
null to 0 and derived free from it, fabricating a 0-used/full-free total.
Preserve null and render the translated Unknown for per-device used, free
and utilization, and mark the aggregate VRAM tile unknown when any device
is unknown.
* Render unknown VRAM as Unknown in the floating monitor and the util tile
The floating VRAM monitor and the aggregate utilization ring both still
coerced a null usage to 0, showing a fabricated 0.00 GiB / full free / 0%
on the same Windows ROCm no-counter case the resources tab already
handles. Guard both on whether every device reports a finite usage and
render Unknown (value and percent) instead of a concrete 0.
* Attribute per-adapter VRAM usage only when capacity forces the mapping
On Windows/ROCm there is no shared key between LUID performance-counter
instances and torch ordinals, so usage was paired to devices purely by capacity
ranking. That pairing is only trustworthy when capacity forces it (a usage
larger than every smaller device can sit on one card). When a smaller-capacity
device could equally hold a strictly larger usage (for example an 8 GiB card
near full beside a lightly used 48 GiB card), the two values are swappable
without violating any capacity, so the ranking is a guess with no key to break
the tie. A wrong guess both mislabels the System tab and feeds
routes/training_vram.py a wrong per-index free value, driving a wrong
keep-resident decision.
Report unknown for every device when the assignment is ambiguous, keeping the
attribution only for the capacity-forced case. Returning None is the
conservative direction: training_vram treats a missing index as zero free, so it
never keeps a chat model into an OOM. Add regression tests for the
not-capacity-ordered, same-capacity, single-fits-both, and capacity-forced
cases.
* Report unknown VRAM usage when a hidden adapter survives the noise filter
When HIP_VISIBLE_DEVICES exposes a subset of the physical adapters, the LUID
usage counters cover cards outside the visibility mask too. The sub-64 MiB noise
filter could drop a genuinely-idle visible card's real usage while keeping a
hidden larger card's high usage, which was then clamped onto the smaller visible
device and reported as fully used (for example a hidden 48 GiB card at 40 GiB
shown as a visible 8 GiB card fully used, with its true 10 MiB usage filtered
out). That fabricated reading also feeds routes/training_vram.py a wrong
per-index free value.
Flag extra adapters on the raw counter count (before the noise filter, since an
idle visible card can itself fall below the floor) and, when a kept usage exceeds
its ranked visible capacity, report unknown rather than clamp a hidden card's
usage onto a visible device. The genuinely-idle-noise and capacity-forced
single-model cases are unchanged. Add a regression test for the hidden
high-use-adapter case in both counter orders.
* Report unknown when only a placeholder adapter counter survives the noise filter
When more raw counters than visible devices are present but every counter sits
below the 64 MiB noise floor (an idle real GPU alongside a Windows Basic Render
Driver placeholder), the non_trivial-or-raw fallback resurrected the raw
magnitude-sorted counters and could attribute the placeholder to a real GPU while
dropping a real card's reading. With a single visible device the swap-ambiguity
check cannot catch it (it needs at least two ranks), so the fabricated value
reached the System tab and automatic GPU selection.
Return unknown for every device in that case instead of falling back to raw
counters. With the earlier guards this completes the invariant: a concrete
per-GPU usage is emitted only when the assignment is capacity-forced, and every
ambiguous, extra-adapter, placeholder-fallback, or count-mismatch path reports
unknown. Add a regression test for the placeholder fallback in both counter
orders and the two-idle-GPU case.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: attribute Windows/ROCm VRAM only when capacity forces a clean bijection
With more raw adapter counters than visible devices, a survivor that merely
fits a visible card was pinned to it by magnitude ranking, fabricating a hidden
GPU's usage onto an idle visible card whose true reading was dropped by the
sub-threshold noise filter (two visible 48/8 GiB cards using 40 GiB / 10 MiB
beside a hidden 6 GiB adapter returned [40, 6]). Emit a concrete per-device
value only when the supra-threshold counters number exactly the visible devices
(every visible card has one real reading, the extras were sub-threshold
placeholders) AND the ranked usage strictly exceeds every smaller visible card's
capacity. When a visible card is idle (fewer supra-threshold counters than
devices) a survivor could be the hidden GPU's usage, so every device reports
unknown; more active counters than visible cards, the smallest card, and any
merely-fitting usage stay unknown too. The reporter's loaded-card display is
preserved (40 GiB / 0.5 GiB across 48/8 GiB -> [40, None]). Adds a regression
test for the reported case plus an exhaustive capacity-forced/bijection matrix.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: keep the unified-memory total when Windows-ROCm used is unknown
_apply_unified_memory_correction gated both the total and the used update on
torch_used_gb being known, so on a unified-memory APU (Strix Halo) where torch
reports used=None (the Windows-ROCm free==total sentinel) but an authoritative
full-GTT total, the device kept amd-smi's small dedicated carve-out and
underreported its capacity on the System tab. Adopt torch's larger total
independently of used; overwrite used only when torch's is known (otherwise keep
amd-smi's dedicated-usage figure) and recompute utilization against the
corrected total. Adds regression tests.
* Tighten comments in the ROCm/Windows VRAM reporting path
* Tighten comments further in the ROCm/Windows VRAM reporting path
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <unslothshared@gmail.com>
* Studio: Data settings tab, uploaded files manager, quant pinning, image preview fix
Settings
- New Data tab in the settings sidebar, under Connections. Chat data
management (archived chats, confirm before deleting, exports, import,
clear all) moved there from the Chat tab.
- New Archive all chats action with confirmation. Archives every chat in
Recents and Projects; compare pairs count as one chat.
- New Uploaded files manager listing RAG documents (chats, projects,
knowledge bases) and chat message attachments with location, size and
date. Files can be opened in a new tab or deleted. Deleting a chat
attachment keeps the message text.
Backend
- GET /api/rag/documents lists all uploaded RAG documents with file size
plus KB and project names.
- GET /api/chat/attachments lists chat message attachments; per
attachment file and delete endpoints included.
Model selector
- Downloaded GGUF quants can be pinned from the quant row (next to the
settings and delete actions). Pinned quants show at the top of On
Device under a Pinned heading as model name plus a grey quant chip and
load directly with one click. Non GGUF cached repos pin as a whole.
- Toned down the green of the downloaded label.
Fix
- Clicking an image attachment in chat now opens the preview overlay.
The tooltip trigger wrapper called preventDefault before composed
handlers ran, which made Radix DialogTrigger skip opening.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: image previews and file type chips in uploaded files list
Image attachments now show a small thumbnail (lazy loaded from the
stored bytes, object URL revoked on unmount) and every row shows a grey
uppercase type chip derived from the extension or content type. Non
image rows keep a file icon. Name cell floors its width and clips
overflow so narrow dialogs stay aligned.
* Harden attachment serving, add tests, and polish pinned rows and previews
- Strict base64 decoding for attachment files: corrupt payloads now return
422 instead of silently serving empty or garbled bytes; whitespace,
missing padding, the URL-safe alphabet, and RFC 2397 percent-encoded
data URLs are all handled
- New backend test suite covering attachment listing, size accounting,
malformed rows, deletion semantics, and every file-serving edge case
- Pinned quant rows show a Loaded tag when that exact quant is active,
and reveal unpin, settings, and delete actions on hover
- Uploaded files dialog is wider and chat locations link straight to the
thread the attachment belongs to
- Chat image preview is now a chrome-free lightbox: dimmed backdrop,
rounded image, corner close button, click outside to dismiss
- File opens go through a synchronous window.open so Safari and Firefox
popup blockers do not eat them
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Uploaded files: click a file to jump to its chat, square thumbs, new Data icon
- Clicking a file row (thumbnail or name) now goes straight to the chat it
belongs to; files without a chat open directly as before
- File thumbnails pin a small 7px radius: the theme scales rounded-md up
to a near circle at this size
- Settings Data tab now uses the database-setting icon
* Uploaded files is now a Data tab subpage instead of a popup
- Manage swaps the tab body for an inline Uploaded files page with a back
header, matching the rest of settings navigation
- Size column header and values are left aligned like the other columns
- Column widths tightened so the table fits the settings panel
* Lightbox polish and Data tab row order
- Image preview close button is transparent until hovered
- Preview image no longer rounds its corners
- Import chats now sits below Clear all chats in the Data tab
* Data tab: export chats as fine-tuning data and open them in Recipes
- New Fine-tuning section in Settings > Data converts every chat into a
JSONL dataset in the OpenAI messages format, one conversation per line
with string-only system/user/assistant turns
- The Train tab detects this file as chatml natively: no column mapping
and no standardization pass, and it works with train on completions
since every assistant turn sits behind the chat template response marker
- Consecutive same-role turns merge, trailing turns without an assistant
reply drop, and reasoning, tool calls, and images are excluded so chat
templates format the data cleanly
- Open in Recipes stages the JSONL as a local seed upload, creates a new
Data Recipe with the seed block preconfigured, and jumps to the editor
* Data tab: load chats straight into the Train tab, row moved to the top
- New Load in Train tab button uploads the fine-tuning JSONL through the
training dataset endpoint, selects it in the training config store, and
opens the Train tab with the dataset loaded and format-checked
- Use chats as training data now sits at the very top of the Data tab
- The Chats subheading is gone; chat rows flow directly under it
* Address review findings on the uploads manager and quant pins
- Deleting the last attachment stores '[]' instead of NULL: a NULL reads
back as a missing field and triggers the legacy IndexedDB backfill,
which resurrected the deleted attachment on the next chat load
- The attachment file endpoint now serves audio: adapter parts store
{data, format} raw base64 and compare chats store a bare base64 string;
media type comes from the attachment contentType or the format
- Compare-chat uploads live in message content parts, not attachments;
the uploads list now includes those blobs via synthetic content-part
ids that the same get and delete routes resolve
- Deleting a quant from the expanded repo row also unpins it so a pinned
row cannot try to load a file that no longer exists
- Thumbnails in the uploads list fetch their blob only once the row is
visible, so a long screenshot history does not download everything
- Nine new backend tests cover audio serving, content-part listing,
serving, deletion, and the empty-list delete behavior
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Data tab: single action dropdown with format choices for chat training data
- The three fine-tune buttons collapse into one dropdown plus a run
button; pick Load in Train tab, Open in Recipes, or Export JSONL,
then click the arrow to run it
- The dropdown's Format section adds ShareGPT and Alpaca alongside the
default OpenAI messages format, ticked like a checklist; all three
shapes are auto-detected by the Train tab's format check
- Alpaca is single-turn, so each user to assistant pair becomes its own
record with the system prompt and earlier turns carried in the input
column
- Shorter description on the training data row
- Uploaded files rows show the size under the file name instead of a
separate column, matching the tighter layout
* Polish the training data action control
- Run button is a true circle (icon-sm plus rounded-full) with a
heavier arrow stroke
- Dropdown trigger uses the shared standard chevron and a fixed width
so switching actions no longer resizes the control
* Shorten the training data row description
* Use the standard chevron for the run button and enlarge the ticks
- Run button uses the shared standard right chevron so it matches the
dropdown chevron instead of the hugeicons arrow
- Dropdown ticks bumped up a size for legibility
* Reword the training data row description
* Shorten Data Recipes to Recipes in the training data description
* List Export JSONL first and rename the default format to Chat Completions
* Handle legacy string content in fine-tune exports and gate Train on chat-only hosts
- messageToPlainText now accepts plain-string message content, the shape
legacy and imported histories store, so those conversations export
instead of being skipped as having no exchange
- The Load in Train tab action is disabled on chat-only hosts the same
way the sidebar gates Train; the default action falls back to Export
JSONL there so the run button never uploads a dataset that /studio
would immediately redirect away from
* Narrow the training data action dropdown slightly
* Drop the format picker from the training data dropdown
Chat Completions (OpenAI messages) is the only export format we ship, so
the ShareGPT and Alpaca options and the Format section are removed. The
export always uses the OpenAI messages shape.
* Address the second round of review findings
Security
- Chat attachment data URLs no longer echo their embedded media type:
anything that is not a plain raster image serves as octet-stream, so
imported text/html or SVG payloads cannot render under the app origin
- Uploaded .html/.htm RAG documents serve as text/plain for the same
reason; the preview sheet only uses the file URL for PDFs
Uploads manager
- Remote image URLs in imported chats are no longer listed as stored
uploads (nothing to serve, and delete would strip the chat reference);
the delete guard mirrors the same data:-only rule
- Deleting a content-part upload refetches the list since the remaining
parts re-index, keeping sibling row ids current
- Deleting a project document from the Data tab invalidates the project
sources cache like the sources panel does
- Data-tab deletions now patch the loaded thread's in-memory copy via a
small event, so a later repo sync cannot write the attachment back
Fine-tune export
- Branch siblings from retries stay out of the exported conversation;
only the selected chain converts (full exports still keep everything)
- Assistant turns before the first user turn drop, preserving leading
system prompts, so no unconditioned assistant targets are emitted
Four new backend tests cover the media type clamp and remote-URL rows;
two existing tests updated for the clamped types
* Fix uploaded file lifecycle and model state
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Make archived chats a Data settings subpage
* Studio: fix attachment route tests and pinned quant edge cases
- test_chat_attachments: drop asyncio.run around the synchronous
/attachments routes (list/get/delete are plain def, so asyncio.run
raised 'a coroutine was expected' and failed the Repo tests CI job).
- test_chat_attachments: align compare-chat content-part assertions with
the stable content-hash id scheme (content-part-sha256-...) instead of
the removed array-index ids; resolve ids from the listing.
- pickers: pass disabled={deleteDisabled} to the pinned-quant delete
action so a quant cannot be deleted mid model-load, matching the
expanded variant rows.
- pickers: build the pinned-quant existence set from the query-unfiltered
cached GGUF repos (format filter still applied) so a pinned quant stays
findable when the search term matches only its quant name.
* Fix Studio review regressions
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Guard fine-tune export content blocks
* Add Export button for archived chats
Adds an Export action to the Archived chats view in Settings > Data that
downloads only the archived chats as a JSON backup (their threads, messages
and projects). The button sits in the archived header row and appears only
when archived chats exist.
* Refactor archived export into pure, testable units
Split the archived-chats export into a dependency-free filter
(archived-chat-export.ts) and a shared JSON download helper
(download-json.ts). Skip the download when nothing is archived so a
stray call never drops an empty file. No behavior change to the button.
---------
Co-authored-by: shimmyshimmer <shimmyshimmer@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Etherll <61019402+Etherll@users.noreply.github.com>
Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
* test(studio): add e2e test for cpu-fallback overriding vulkan
* feat(studio): add UNSLOTH_LLAMA_CPP_BACKEND env var
* feat(studio): add UNSLOTH_LLAMA_CPP_BACKEND env var
* Preserve UNSLOTH_LLAMA_CPP_BACKEND=cpu across llama.cpp updates for PR #7228
The in-app updater rebuilt the installer command without --cpu-fallback and
only re-asserted Vulkan, so accepting a llama.cpp update after forcing CPU on
an Intel iGPU host re-ran host detection and routed back to the crashing Vulkan
bundle (#7213). Record install_kind in the prebuilt marker and re-assert
--cpu-fallback on update when the installed bundle is CPU.
Also make setup.sh's UNSLOTH_LLAMA_CPP_BACKEND check case-insensitive to match
setup.ps1, and add tests for the updater CPU preservation and the setup.sh flag
plumbing.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Trim and validate UNSLOTH_LLAMA_CPP_BACKEND, warn on unknown values for PR #7228
Trim surrounding whitespace and lowercase the value in both setup.sh and
setup.ps1, so values like ' cpu ' or 'CPU' still force the CPU-only prebuilt.
An unrecognized value (e.g. 'gpu') now prints a warning instead of silently
falling back to auto. Extend test_setup_llama_cpp_backend.py to cover both
scripts, including trimmed, empty and unknown values.
* Preserve arm64 CPU installs on update and honor CPU override in Windows prune for PR #7228
The update-path CPU preservation only matched install_kind ending in -cpu, so
arm64 CPU bundles (linux-arm64, windows-arm64) were re-routed to a GPU or source
build on update. Match the full set of CPU-only kinds instead.
Persisting install_kind also activated the previously inert Windows
mismatch-prune in setup.ps1: on a GPU host with UNSLOTH_LLAMA_CPP_BACKEND=cpu it
saw the windows-cpu marker as mismatched and deleted it every rerun. Normalize
the override once and make CPU expected so a deliberate CPU install is kept.
Extend the tests to cover both.
* Document legacy llama.cpp markers keep heal-to-GPU on update for PR #7228
Legacy prebuilt markers written before install_kind was persisted intentionally
do not force --cpu-fallback on update: the in-app updater lets them re-resolve
(heal to a GPU bundle) per the existing behavior from #6097, and only markers
that explicitly record a CPU install_kind are pinned to CPU. Add a comment and a
regression case documenting the boundary.
* Tighten llama.cpp CPU-fallback comments for PR #7228
* Fix Windows install-prune to keep valid Intel/fallback bundles for PR #7228
Persisting install_kind activated the setup.ps1 mismatch-prune, whose
expectedKinds was incomplete: the non-NVIDIA/non-AMD branch omitted
windows-vulkan (the Intel auto-route) and the GPU branches omitted the
windows-cpu/windows-arm64 fallback the installer uses when a GPU prebuilt is
missing. That made every setup rerun delete and re-download a valid Intel Vulkan
(or CPU-fallback) install. List all kinds the installer can produce per host so
only a bundle the host cannot run is pruned. Cover the full matrix in tests.
* Persist force_cpu marker flag so only forced CPU installs re-assert on update for PR #7228
* Add --force-cpu for deliberate CPU installs and warn on macOS for PR #7228
* Record force_cpu when reusing a matching CPU bundle for PR #7228
* Accept force_cpu keyword in installer test validator fakes for PR #7228
---------
Co-authored-by: danielhanchen <unslothai@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Studio: persist llama.cpp KV cache across idle auto-unload (slot save/restore)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: address KV persistence review feedback
* Studio: guard KV restore on launch config
* Studio: fix KV resume purge race, fingerprint requested ctx, purge on disable
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: re-check idle/keep-KV settings after slot save, ns file identity
* Studio: shard-aware KV guard, honor user --no-cache-prompt, early save cap
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: honor LLAMA_ARG_CACHE_PROMPT env in slot-save guard
* Studio: derive prompt-cache state from final argv for slot saves
* Studio: stat LoRA/control-vector sidecars in KV restore fingerprint
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: parse csv and FNAME:SCALE sidecar syntax in KV fingerprint
* Studio: address codex review on idle-unload KV resume
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: harden slot-save cleanup, cap accounting, stale-KV guard, save timeout
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: treat unavailable KV estimate as full-cap for slot-save disk check
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
#6414 moved the llama_extra_args inheritance out of the GGUF branch in
_load_model_impl into _guard_chat_load_against_training, which runs before the
branch, so 'if request.llama_extra_args is None' is no longer inside the
gguf_branch slice that test_load_marker_precedes_hub_guard_and_unload checks.
The assertion failed on that now-missing landmark even though the guarantee it
protects (the gguf_load_in_flight marker is entered before the hub-download
guard and the unload) is intact. Drop the relocated landmark from the ordering
so the test matches the current structure.
Co-authored-by: danielhanchen <unslothshared@gmail.com>
* Studio: allow torch 2.11.x on the CUDA install path
The CUDA torch repair path (_ensure_cuda_torch) installs torch/torchvision/
torchaudio from an exclusive --index-url, so _CUDA_TORCH_PKG_SPEC decides
exactly which torch the Studio venv gets. It was capped at torch<2.11.0, so on
a cu128/cu130 host the venv resolved torch 2.10.x even though the CUDA indexes
now publish torch 2.11.0. That left the Studio venv a torch minor behind the
torch 2.11.0 Docker base image, so the CUDA dedup step would relink base libs
under a mismatched torch.
Raise the upper bound to <2.12.0 (torchvision <0.27.0, torchaudio <2.12.0) so
the CUDA install path lands on torch 2.11.x, matching the rocm7.2 spec and the
base image. The torchao selector already maps torch 2.11 -> torchao 0.17.0, and
_ensure_flash_attn degrades gracefully when no prebuilt wheel matches (Blackwell
skips it outright; non-Blackwell prints a warning and continues), so no other
pin needs to move.
Add test_cuda_torch_spec.py to lock the bound (torch 2.11.x in, 2.12.x out) and
assert the CUDA and rocm7.2 upper bounds stay in lockstep.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* test: use zip(strict=True) so a spec length mismatch fails loudly
* install.sh: widen the CUDA torch ceiling to <2.12.0 so a fresh install matches the base
Raising _CUDA_TORCH_PKG_SPEC alone was not enough: that spec only feeds
_ensure_cuda_torch(), the ROCm-poisoning repair path that early-returns on a
normal NVIDIA host. A fresh CUDA install (including the studio Docker build,
which runs `bash install.sh --local`) takes its torch from install.sh's
TORCH_CONSTRAINT, which was still capped at torch>=2.4,<2.11.0, so cu12x/cu13x
resolved torch 2.10.x and the venv landed a minor behind the torch 2.11.0 base
image.
Extend the existing `case "$TORCH_INDEX_URL"` block (which already relaxes
rocm7.2) with a `*/cu[0-9]*` branch that widens the ceiling to <2.12.0, keeping
the >=2.4 floor so an older CUDA index (e.g. cu118) that tops out below 2.11
still resolves. The CPU wheel and older ROCm tags stay on <2.11.0 (the glob
does not match /cpu). torchvision/torchaudio are bare on this install line and
resolve their compatible companions via wheel metadata, matching the rocm7.2
pattern.
Add behavioral tests (Python + shell) exercising the case block: cu118/124/126/
128/130 widen to <2.12.0, rocm7.2 stays 2.11.x, and /cpu plus older ROCm keep
the default <2.11.0.
* install.sh: key the CUDA torch widening off the index leaf, not the full URL
The `*/cu[0-9]*` glob matched a `cu<digit>` segment anywhere in TORCH_INDEX_URL,
so a custom UNSLOTH_PYTORCH_MIRROR whose base path contains e.g. cu128 but whose
final leaf is cpu or an older ROCm tag would still widen TORCH_CONSTRAINT to
<2.12.0, contradicting the block's own comment and letting a CPU / older-ROCm
mirror resolve torch 2.11.x. Match on _torch_index_leaf (the final path segment
the backend classification just above already computes) so only a real cu*/
rocm7.2 leaf is affected; cpu and older ROCm keep the default <2.11.0. Update
the Python + shell tests to mirror the leaf-anchored case and add regression
cases for a mirror base that contains cu128 but resolves to a cpu / rocm7.1 leaf.
* install: freeze the torch trio during the with-deps unsloth installs
Released unsloth wheels can pin an older torch than Step 1 installed
(unsloth 2026.7.2 declares torch<2.11.0), so the with-deps resolve from
PyPI silently downgrades the pinned +cuXXX torch trio to PyPI's default
wheel. The flavor guard cannot catch every such swap: PyPI's torch 2.10
default is itself cu128-flavored, so the cuXXX tag comparison still
matches while the version silently drops. Freeze the just-installed trio
with uv --overrides (overrides replace dependency requirements during
resolution), keeping torch 2.11.0+cuXXX in place while unsloth's other
dependencies resolve normally. Verified on the cu128 path: without the
override torch drops 2.11.0+cu128 -> 2.10.0; with it the trio survives
and unsloth 2026.7.2 + unsloth-zoo install cleanly.
* install: fold UV_OVERRIDE env files into the torch-trio overrides file
The CLI --overrides flag is the command-line form of UV_OVERRIDE, so
passing it replaced any overrides file already exported for the process;
macOS arm64 exports UV_OVERRIDE=overrides-darwin-arm64.txt for the same
generic install path and would have lost those pins. Concatenate any
UV_OVERRIDE files into the temp trio file so both keep applying.
* install: extend the torch-trio overrides guard to migrated installs
Four follow-ups to the Step-2 --overrides guard, all empirically verified:
1. The migrated-environment with-deps unsloth install resolved
unsloth>=2026.7.2 (which pins torch<2.11.0) without the overrides file,
so a migrated CUDA venv on torch 2.11 was silently downgraded -- the
exact bug this branch fixes on the fresh path. The overrides build is
now a function (_build_unsloth_torch_overrides, reading the trio
installed at call time) invoked by both with-deps paths; the migrated
no-torch path installs --no-deps and stays unguarded.
2. The overrides temp file is now cleaned by the EXIT trap (same pattern
as _UV_OVERRIDE_TMPDIR, pre-initialized empty so an inherited value can
never reach the trap's rm); previously any Step-2 failure leaked it.
3. Folding UV_OVERRIDE files used cat, which joins the last requirement of
a file lacking a trailing newline onto the next file's first requirement
(reproduced: idna==3.10certifi==2025.1.31 makes uv fail parsing).
4. Inherited torch/torchvision/torchaudio override lines are now filtered
out when folding: uv intersects duplicate overrides rather than
last-wins (verified on uv 0.10.12: direct conflict is unsatisfiable,
transitive conflict silently backtracks), so a conflicting inherited
trio pin would break the resolve the generated exact pins protect.
Both 3 and 4 are handled by a single newline-terminating awk filter
that preserves non-trio overrides (torchmetrics, torchao, ...).
test_unsloth_torch_override.sh extended: migrated-path coverage, trap
assertion, and a functional fold test (14 checks).
* installer: tighten comments
* install: keep the existing torch release when re-running the installer
Re-running `curl -fsSL https://unsloth.ai/install.sh | sh` over an existing
install rebuilds the venv for clean state, which silently moved users to the
newest torch in range (2.10 -> 2.11 once the constraint widened). A torch the
user already validated must survive an unsloth update.
Before the old venv is moved aside for rollback, its torch version is probed
(last stdout line only, so sitecustomize noise cannot corrupt it). After the
index leaf is chosen, _previous_torch_pin turns that version into a
torch==X.Y.Z pin, but only when it cannot do harm:
- cu*/cpu leaves only; rocm leaves keep their floors (rocm7.2 must land 2.11
for the Strix _grouped_mm fix) and the Radeon wheel-matching path is
untouched.
- The wheel's flavor tag must match the freshly chosen leaf, so a flavor
change (cpu -> cuda, cu126 -> cu130) still installs the correct new build.
- The base must look like a release, so probe noise never becomes a pin.
- UNSLOTH_TORCH_UPGRADE=1 opts out and restores the old always-newest
behavior; the substep line advertises it.
The supported range is kept in _PREV_FALLBACK_CONSTRAINT: if the exact
release is not resolvable from the chosen index (custom mirrors prune old
wheels), the install warns and falls back to the newest supported release
instead of failing the whole run. The later flavor-mismatch repair reuses
TORCH_CONSTRAINT, so a mid-install clobber is repaired back to the kept
release rather than the newest one.
Verified end to end: a venv seeded with torch 2.10.0+cu130 re-run through the
full installer finishes with torch 2.10.0+cu130 (previously 2.11.0+cu130).
Tests: tests/sh/test_previous_torch_pin.sh covers keep/flavor-change/rocm/
noise/opt-out plus wiring (probe ordering before venv replacement, fallback
present, SKIP_TORCH gate).
* install: constrain kept torch pins to the supported window
Review caught that _previous_torch_pin pinned the previous venv's torch on
flavor match alone, so a release outside the installer's active range (a
2.3.x manual install below the >=2.4 floor, or a 2.12.x manual upgrade above
the ceiling) replaced the bounds computed just above it and a rerun kept a
torch the installer otherwise deliberately excludes.
New _torch_release_in_window checks the probed base against the active
TORCH_CONSTRAINT ("torch>=A.B[,<C.D.F]") at major.minor granularity, which
is exact for the windows this script uses (ceilings are always X.Y.0; a
non-.0 ceiling would only make it conservative). Anything unparseable
answers no, so probe noise or a malformed window fails toward the supported
range instead of becoming a pin. _previous_torch_pin takes the active
constraint as a third argument and refuses out-of-window releases; the
in-window keep behavior is unchanged.
Tests: out-of-window rows (2.3.x floor, 2.12.x ceiling, boundary keeps, cpu
and macOS windows, malformed/empty windows) plus direct
_torch_release_in_window coverage.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: GPU memory dropdown — llama.cpp --fit on and manual gpu-layers/cpu-moe
* Studio: simplify GPU memory changes (reuse ParamSlider, GPU_LAYERS_ALL, loadedGpuMemoryFields helper)
* Studio: GPU picker — choose which GPUs a GGUF model loads on (gpu_ids)
* Studio: simplify GPU picker (share /api/system fetch, validate gpu_ids)
* Studio: GPU picker review fixes (gate relative indices, no cross-model leak, validate, types)
* Studio: group GPU controls under a collapsible GPU section
* Studio: GPU feature review fixes (fix fit-ctx test, behavior-test the floor, comment accuracy)
* Studio: make GPU a top-level settings section (not nested under Model)
* Studio: flatten GPU controls into the Model section, group by GPU/context/generation
* Studio: move GPU Memory to the bottom of Model with its dependent controls beneath it
* Studio: move GPU Memory below Tensor Parallelism and GPUs below GPU Memory
* Studio: tighten GPU Memory and GPU Layers tooltip copy
* Studio: fix fit-mode context slider track-click, restore GPU Memory tooltip, shorten fit dropdown label
* Studio: GPU Memory tooltip one mode per line, briefer
* Studio: note HIP_VISIBLE_DEVICES (ROCm) in the GPUs picker tooltip
* Studio: narrow the GPU Memory dropdown to fit the shortened label
* Studio: use 'llama.cpp --fit' in the GPU Memory tooltip for consistency
* Studio: allow Tensor Parallelism in Manual GPU mode
* Studio: graduated MoE-on-CPU offload (--n-cpu-moe) replacing the all-or-nothing toggle
* Studio: size the MoE-offload slider for staged (deferred-load) models
* Studio: share one GGUF header walk for the context-length and MoE-count readers
* Studio: size the GPU Layers slider for staged models (one staged-header read)
* Studio: move Tensor Parallelism below the GPUs picker
* Studio: GPU split (--tensor-split) per-GPU model share in Manual mode
* Studio: tolerate whitespace in GPU split input, move it below GPU Layers
* Studio: rename the GPU split control to "Split ratio"
* Studio: Split ratio sends explicit even input; fix blank=free-VRAM (not even) copy
* Studio: tighten llama.cpp --fit VRAM margin with --fit-target 512
* Studio: GPU memory review fixes (rollback re-baseline, single-GPU TP gate, accurate copy)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: move Split ratio below MoE Layers on CPU
* Studio: address PR review (fix GPU-info hydration race, share fit context-length across load paths)
* Studio: address codex review (manual single-GPU TP guard, GPU-aware spec defaults in fit/manual, GGUF-only context/preference)
* Studio: address codex review round 2 (gpu_present seed, single-GPU tensor-split guard, staged manual-knob reset, strip inherited offload flags)
* Studio: address codex review round 3 (strip inherited --n-cpu-moe, CPU-fallback warning in Manual mode)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: address codex review round 4 (preserve pinned fit context across a later Apply)
* Studio: address codex review round 5 (honor GPU picker for diffusion GGUFs, clear fit pin on cross-model switch)
* Studio: preserve the pending GPU Memory mode when staging a model
* Studio: pin diffusion GPU device order and reset GPU-memory state for diffusion loads
* Studio: address codex review round 6 (fit-Auto rollback context, preserve manual non-tensor split modes, persist GPU mode on load not select)
* Studio: persist the applied GPU Memory mode, not the requested one (skip diffusion loads)
* Studio: replace Manual-mode split-ratio field with per-GPU layer sliders
* Studio: clarify per-GPU layer split hint for tensor-parallel mode
* Studio: address codex review round 7 (allow GGUF gpu_ids past the legacy guard, replay GPU-memory fields on respawn)
* Studio: address codex review round 8 (size the validate preflight like the load in fit mode, across both load paths)
* Studio: skip the training-OOM guard for llama.cpp --fit GGUF loads (they spill to RAM)
* Studio: drop the now-redundant compare-path validate sizing (the --fit guard skip makes it moot)
* Studio: address codex review round 9 (keep the training guard for fit loads, forward gpu_ids to validate, strip inherited manual tensor-split)
* Studio: address codex review round 10 (gate GPU-memory adoption on is_gguf, record manual knobs only in Manual mode)
* Studio: handle diffusion GGUFs symmetrically in the GPU Memory controls (preserve the standing mode preference, hide the inapplicable mode/TP controls)
* Studio: remember the GPU Memory settings per model
* Studio: consolidate --fit mode and Manual mode into a single Manual mode
* Studio: preserve the per-GPU layer split across GPU Layers changes
* Studio: trim overly long GPU Memory comments
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* address GPU memory config review comments
* trim redundant GPU memory tests
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Reconcile manual-mode TP drops with the #6659 drop-site invariants
* Preserve quantized KV in manual --fit, charge GGUF companions in full, reconcile GPU pick on load
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Clear stale GPU baseline on non-GGUF loads so it can't read as dirty
* Fix no-context-shift test for the conditional -c flag
* Credit manual GPU-layer offload for cached HF GGUFs
* Reset per-model load knobs on GGUF quant switch
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Strip inherited tensor-split when manual ratio is cleared
* Match auto-load validation to safetensors placement
* Reset editable manual knobs after Auto GGUF loads
* Record a single device for diffusion GPU picks
* Reset per-model GPU knobs before applying saved settings
* Address review comments
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Guard manual tensor splits and keep remembered context on auto-load
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Snapshot compare knobs, seed splits from free VRAM, flag zero-offload loads
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Exempt CPU-only loads from the guard floor and harden compare and reseed paths
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Reach full offload from the layers slider and charge extras drafters in the guard
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Warm the GPU device cache before pick reconciles and disable staged GPU controls
* Align the training guard with inherited extras, spec mode, and compare targets
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Hide GPUs from companion-less zero-offload loads
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Size diffusion picks per device, own manual offload flags, reject XPU picks
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Drop tensor flags at zero layers and exempt CPU-pinned drafters
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Allowlist the zero-layer tensor parallel drop site
* Keep validate and load guards on the same extras and refresh stale baselines
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Drop mismatched manual tensor splits before launch
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Gate XPU picks on the real backend field and harden split and hydration paths
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Weight full GPUs as zero, clamp split shares, and refine the zero-layer mask gate
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Carry fit context across mode changes and align drafter and picker gates
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Catch variant switches, uncached diffusion repos, and text-only mmproj skips
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Check companions on the first device and size native and remote zero-layer loads
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Replace the training guard's precise VRAM modeling with a conservative bound
* Baseline context pins on non-GGUF hydration and reprobe list-seeded staged GGUFs
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Size manual splits by their largest share and preserve resolved context from Default
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Default-deny unsized required companions and price KV at the effective cache dtype
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Reserve MTP draft KV and MLA target-copy in the training guard
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Size tensor-parallel loads per device and show GPU controls for native GGUFs
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Reserve MTP overhead for uncached remote GGUFs and the mmproj runtime factor
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Drop the training-coexistence VRAM estimation this PR added
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Gate remembered load settings to GGUF picks
* Lock the remaining load-time controls during a staged load
* Clear the stale native-path token on compare loads
* Drop a stale guard reference from the zero-offload masking comment
* Seed GPU baselines from the rollback response and drop never-emitted offload flags
* Match validate's training guard to load and keep the native reload token
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Trim verbose GPU-memory comments
* Thread the variants header walk off the event loop, honor device pins on zero-offload, and hold staged GPU edits
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Honor manual placement and classify pinned zero-offload loads
* Close diffusion admission and status hydration gaps
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Check the actual diffusion GPU during training
* Align staged baselines and manual reload dedupe
* Fix GGUF placement and rollback state
* Harden manual GGUF placement boundaries
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Remove unused resolve_tensor_parallel import in llama_cpp.py
The name is used only in llama_server_args.py, routes/inference.py, and tests,
not in llama_cpp.py; the unused hoisted import trips the import-hoist verifier
in the source-lint CI job.
* Fix diffusion GPU dedup and training guard for non-numeric device tokens
The diffusion runner drives only its single lowest device and the backend
records that one device (self._gpu_ids = [sorted(gpu_ids)[0]]), but the reload
dedupe compared it against the full requested list, so a multi-GPU pick that
resolves to the same device forced a needless reload. Normalize the request the
same way for a loaded diffusion model in both _already_in_target_state and the
route _request_matches_loaded_settings.
The chat-during-training coexistence guard called int() on the single-device
token and hard-rejected when it could not parse. A non-numeric token (a CUDA
UUID / MIG handle) now sizes against the whole visible pool like the GGUF guard
instead of falsely blocking the load, and an empty token (a CPU-only runner such
as a CPU diffusion GGUF) is allowed outright since it uses no GPU VRAM.
* Tighten comments added by the GPU memory config changes
* Harden GGUF placement from independent review: VRAM sizing, diffusion TP reset, tensor_split validation
- Training coexistence guard: a single-device runner pinned through an
unresolvable UUID/MIG token was sized against the aggregate visible-VRAM pool,
so a load could pass on capacity it cannot use and then OOM active training.
Size against the worst-case visible device (min free) instead, keeping the
guard's documented default-deny contract. The empty-token (CPU-only runner)
allow path is unchanged.
- Diffusion startup: _start_diffusion_server now resets self._tensor_parallel to
False alongside the other placement resets. A prior tensor-parallel chat load
(process killed but not fully unload-reset) otherwise left /status misreporting
tensor parallelism and made an identical diffusion re-Apply reload against the
stale state.
- tensor_split: reject negative / non-finite / all-zero splits up front. They
were dropped at launch but still compared raw in the reload dedupe, so an
identical Apply reloaded indefinitely.
- Tests: the shared httpx stub was incomplete and, installed via setdefault
before real httpx loaded, broke a combined pytest run (collection errors on
httpx.Response). Import the real installed httpx instead.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <unslothshared@gmail.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
* studio: show the active run's saved config in the Training Progress popover
The Training Config popover on the live Training Progress page read the
editable form store (useTrainingConfigStore), so it showed stale/static values
whenever the form changed after the run started; only the History view read the
run's saved config snapshot, which is why re-opening the same run from Recents
showed the correct values (#6853).
Wire the live view to the same authoritative source History already uses:
- Extract History's field mapping into sections/run-config-override.ts
(mapRunConfigToOverride) so both views share one mapper over
GET /api/train/runs/{id} config.
- LiveTrainingView fetches the run record as soon as the job id is known and
passes the mapped override to ProgressSection; the fetched config is keyed
by job id, and until it loads (or if the fetch fails) the form store remains
the fallback. The run record is created at job start, so it is available
while the run is live.
- ProgressSection prefers configOverride whenever one is present instead of
only when isHistorical, so the live override takes effect.
Adds a source-level regression test pinning the wiring and the mapper's
backend config keys.
Fixes#6853
* studio: retry the run-config fetch after the first step, carry the saved method
Two review fixes on the live Training Config popover source:
1. The backend creates the run row only on the first progress event, so the
fetch issued as soon as the job id appeared commonly 404'd during
model/dataset preparation and never retried -- leaving the popover on the
form store for the whole run. The effect is now also keyed on
firstStepReceived (and skips once resolved for the job), so it re-fetches
exactly when the row is guaranteed to exist.
2. The popover's method label and LoRA-row visibility came from
viewData.trainingMethod, still read from the editable form store; changing
the form (e.g. LoRA -> Full) after starting a run relabeled it and hid its
saved LoRA rows. The run-config mapper now derives trainingMethod from the
snapshot's training_type/load_in_4bit (via parseBackendTrainingMethod, now
exported from the feature index) and the live view prefers it.
* studio: fetch the run config on a terminal phase too, not just the first step
The live config-popover fetch was keyed on firstStepReceived, which the runtime
store sets only when step > 0. A run that fails or completes during preparation
(before step 1) creates and finalizes its row from the terminal error/complete
event, but neither the job id nor firstStepReceived changed, so the fetch never
ran and the popover stayed on the editable form store -- showing the wrong
config/method if the form was edited afterward (Configure re-enables on failure).
Gate the fetch on a runRowReady signal = firstStepReceived OR a terminal phase
(completed/error/stopped), the states in which the backend guarantees the row
exists. This also stops the earlier fetch-then-404 churn during preparation and
lets the effect depend only on values it reads (no lint suppression needed).
* studio: retry the run-config lookup and accept a hydrated step as row-ready
Two ways the popover could stay stuck on the editable form store for a whole
run:
- The backend publishes the progress event that reveals the run before
create_run commits, so the first lookup can lose that race and 404. The catch
changed neither runRowReady nor fetchedRunConfig, leaving every effect
dependency identical, so no further attempt was ever made for that job. The
failure path now schedules an explicit retry, bounded and keyed by job id, so
a genuinely absent row falls back to the form store instead of polling.
- A run recovered through status/metrics polling (SSE unavailable or blocked)
has currentStep restored by applyStatus/applyMetrics but never
firstStepReceived, and the phase stays training, so the row was treated as
not ready even at step > 0. currentStep > 0 is now a readiness signal of its
own.
* studio: fetch the saved run config as soon as the job id exists
start_training() inserts the run row before the pump can consume any event --
deliberately, so the run appears in history during model loading -- and /status
exposes the job id throughout the pre-step phases. Gating the lookup on a first
step or a terminal phase therefore held the popover on the editable form store
for the whole configuring/loading/downloading window, which on a long model or
dataset load is minutes, and indefinitely for a run adopted from another client.
The job id is now the entire readiness condition; the existing bounded retry
still covers the instant before the insert commits.
* Fix Training Config popover fallback for history runs without a saved config; tighten popover comments
---------
Co-authored-by: danielhanchen <unslothai@gmail.com>
* Studio: hide infra models from the hub cached inventory
The hub inventory scans behind /api/hub/cached-gguf and /api/hub/cached-models
returned the llama.cpp install validation probe (ggml-org/models) and the RAG
embedder (unsloth/bge-small-en-v1.5[-GGUF]) as on-device models. Share the
hidden-model check from routes/models.py via utils/models/hidden_models.py and
apply it in both scans. A GGUF infra repo stays visible when the user
explicitly downloaded a variant through the Hub, since variant manifests only
exist for user-initiated downloads.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: make On Device trust the hub inventory, match repo ids exactly, lighten the hidden-model import
Follow-up on the hub cached-inventory hidden-model change, addressing the review.
On Device now trusts the Hub inventory API for cached rows. The backend already
hides the RAG embedder and the llama.cpp probe and re-includes a GGUF infra repo
once the user downloads a variant through the Hub, but the frontend was
re-hiding it by repo id, so the user-downloaded variant never appeared in the On
Device list or the count. isVisibleInventoryRow now short-circuits cached rows
(kind === "cache") to visible and keeps client-side needle hiding only for local
filesystem rows and Discover.
is_hidden_model matches Hub repo ids exactly (case-insensitive) against the probe
plus the effective embedder and its GGUF companion, instead of substring
matching the configured-embedder basename. A custom embedder with a generic
basename like org/model no longer hides unrelated cached repos such as
user/model-chat or org/model-instruct. The probe filename and local-path
embedders keep exact matching.
The helper moves to utils/hidden_models.py and is imported at module scope in the
hub cache scanner, so it no longer pulls in utils/models/__init__ (the eager
model-config/checkpoint stack) and a broken import fails at startup instead of
being swallowed per-repo and silently emptying the inventory. routes.models
keeps the _is_hidden_model and _safe_resolve aliases and drops the unused
_HF_REPO_ID_RE re-export that was failing source lint.
Tests: exact repo-id matching with a custom embedder, the cached-models scan
keeping an unrelated repo, and a clean-interpreter check that the helper imports
without the model-config stack.
* Studio: match the llama.cpp probe filename on both path separators
The hidden-model check compared the probe's on-disk filename with
Path(value).name, which on a POSIX interpreter does not split a Windows-style
path ("...\stories260K.gguf") and would let the probe through. Split on both
separators so the probe is matched regardless of which OS produced the path,
matching the tolerance of the previous substring check. Adds a Windows-path
assertion to the probe test.
* Studio: harden hidden infra model handling
* Fix hidden cache row confirmation
* Fix hidden local rows and confirmed hint merges
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Handle snapshot-configured hidden models
* Hide basename-only default embedders
* Fix dynamic embedder inventory filtering
* Studio: hide the configured RAG embedder from Discover and feed rows
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <unslothshared@gmail.com>
Co-authored-by: Daniel Han <23090290+danielhanchen@users.noreply.github.com>
* Replace standalone Studio wording with Unsloth
Replace the single word Studio with Unsloth wherever it is used as
shorthand for Unsloth Studio in docs, CLI output, UI strings, i18n
locales, workflow display names, comments and docstrings.
Kept unchanged: the full name Unsloth Studio, third party product
names (LM Studio, Visual Studio, Mac Studio), feature names
(Recipe Studio, Fine-tuning Studio and its translations), and all
identifiers such as env vars, commands, paths and filenames.
* Address review feedback on the Studio wording rename
Use "an" before Unsloth where the rename left the article as "a".
Restore the split brand where Unsloth and Studio render as two halves
of the full product name: the onboarding sidebar subtitle and the
IPv6 localhost warning. Scope two messages to the full name Unsloth
Studio where plain Unsloth was misleading: the AMD README bullet and
the CLI studio setup error.
* Studio: enforce 60s minimum on idle auto-unload TTL (0 stays off)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Drop decorative section separator from idle TTL floor tests
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: danielhanchen <unslothshared@gmail.com>
* fix(studio): add MLX adapter state control
* fix(studio): honor MLX adapter comparison state
* fix(studio): keep enabled MLX adapters permissive
* Studio: preserve public error message on MLX compare-mode adapter failures
generate_with_adapter_control raised a plain RuntimeError, which the compare
route handled with the generic handler that drops the operational message.
Raise GenStreamErrorRaised(public=chunk.public) instead and catch it in the
streaming and non-streaming consumers, matching the safetensors tool loop, so
errors like 'model is being unloaded' surface their real message.
* Studio: re-emit VLM think prefill inside the adapter context
The compare-mode merge dropped _generate_vlm's upfront yield of the prefilled
<think> block. Restore it as the first snapshot inside the lock+adapter context
(matching _generate_text) so the UI renders the thinking block during prefill
and a cancel/error before the first token does not drop it. Adds a regression
test asserting the prefill is emitted first, after entering the adapter context.
---------
Co-authored-by: danielhanchen <unslothshared@gmail.com>
* Stabilize Studio regression tests
Rebuild on current main. Restore the set-membership sidebar account-block matcher
(#6647, which fixed the same order-sensitive regex, was reverted on main, so the
guard is failing on main again) and keep the watchdog replacement-race fix, whose
blocked-watchdog stub now waits without a timeout so a superseded watchdog stays
alive until cleanup regardless of scheduler load.
* Tighten the blocked-watchdog stub comment
---------
Co-authored-by: Daniel Han <unslothshared@gmail.com>
* feat(studio): expose opt-in MCP control plane
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Harden Studio MCP tools: byte-safe auth, page clamping, forward export/checkpoint fields
Follow-up hardening on the opt-in MCP control plane. All changes are additive
and backwards compatible.
- BearerTokenMiddleware now compares the Authorization header on raw bytes.
A non-ASCII bearer value previously reached str-based hmac.compare_digest,
which raises TypeError and surfaced as a 500 instead of a clean 401. The
constructor also rejects an empty or whitespace-only token so an empty token
can never match an empty "Bearer " header.
- MCP tools call the route functions directly, which skips FastAPI Query
validation. list_training_runs and get_recipe_job_dataset now clamp limit and
offset to the same bounds the HTTP routes enforce (a negative SQLite LIMIT
otherwise means "no limit").
- export_gguf forwards hf_token (the backend rejects a Hub upload without it),
accepts a list of quantization methods, and exposes imatrix / imatrix_path so
the IQ low-bit quants are reachable.
- load_checkpoint forwards hf_token and approved_remote_code_fingerprint so
gated checkpoints and the remote-code approval retry work. Its docstring is
corrected: the export backend coexists with training and inference rather than
freeing GPU work.
- start_training passes via_api_key=False explicitly instead of relying on the
unfilled Depends default.
Tests: add coverage for non-ASCII and empty-token auth, the correct-token pass
through, non-http scope pass through, pagination clamping, and the forwarded
export/checkpoint fields.
* Harden Studio MCP: cap /mcp request bodies, reject unusable tokens, fix docs
Follow-up hardening from a full review pass. All changes are additive and
backwards compatible.
- Add "/mcp" to _BODY_PROTECTED_PREFIXES so MaxBodyMiddleware enforces the same
request-body cap it already applies to every other write endpoint (/api/train,
/api/export, /api/data-recipe, ...). The MCP endpoint accepts authenticated
POST tool-call bodies; without this an authenticated client could send an
unbounded body. The middleware only buffers the request body (not the SSE
response), so streaming is unaffected, and the 500MB default cap never affects
a real JSON-RPC tool call (verified live).
- Reject a non-ASCII UNSLOTH_STUDIO_MCP_TOKEN at construction. HTTP header values
are ASCII, so a non-ASCII token cannot be sent by a standard client and would
silently lock out the endpoint; fail fast instead.
- MCP.md: document the canonical /mcp/ endpoint and note that /mcp redirects to
it, so clients that do not follow redirected POSTs still connect.
Tests: add non-ASCII token rejection coverage.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten MCP server comments
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <unslothshared@gmail.com>
* refactor(studio): move chat model picker into features/model-picker
Relocate model-selector + its support files from components/assistant-ui
into a self-contained features/model-picker feature (own barrel), mirroring
the modular Hub layout. Pure move + import repoint; no behaviour change.
* feat(model-picker): add per-model config persistence layer
Superset PerModelConfig (customContextLength, kvCacheDtype, speculativeType,
specDraftNMax, tensorParallel, chatTemplateOverride, trustRemoteCode) persisted
to localStorage (unsloth_model_configs) with schema versioning + LRU budget.
KV-dtype and speculative value sets match main's sidebar (no q4_0/ngram-simple).
Reuses features/hub/lib/model-identity for normalization; adds storage-key layer
and applyPerModelConfigToRuntime (sets tensorParallel, which the old PR omitted).
* feat(picker): modular backend for chat-template validate + default fetch
New studio/backend/picker package (schemas/service/routes) mounted at /api/picker:
- POST /api/picker/validate-chat-template (Jinja syntax validation, no false positives)
- GET /api/picker/chat-template/{model_name} (default template from tokenizer_config.json,
reusing get_cache_path/resolve_cached_repo_id_case; graceful null, no model-code exec)
Frontend api/templates.ts client + hooks/use-model-defaults lazy cache. No backend
changes to the existing inference load route (per-model load fields already supported).
* feat(model-picker): bind picker on-device list to shared hub inventory
Picker now sources cached + local models from useHubInventory (the Hub's shared
store) via a thin adapter, replacing its own /api/models/* fetchers + module
caches. Hub, download manager, and picker now share one source of truth, so
completed downloads reflect in the picker automatically. Partial/live-download
rows are filtered from the cached lists (unchanged rendering). Local naming/search
preserved via additive LocalInventoryRow modelId/displayName. Variant expander,
scan-folder management, recommended-fit, search, external providers untouched.
Known minor: cached 'Downloaded date' sort tiebreak degrades to alphabetical
(hub cached rows carry no mtime); default 'recent' (load-time) sort preserved.
* feat(model-picker): per-model config step inside the picker
Picking a (non-external) model now opens an in-picker config view built from
main's current load controls (context length, KV cache dtype, speculative
decoding, draft tokens, tensor parallel) plus a chat-template editor backed by
the picker validate/default endpoints. 'Remember for this model' persists the
config per model+variant; Run forwards the config to the existing load flow via
meta.config. External models bypass the step. Two-view orchestration lives in
model-selector (single interception point); pickers.tsx call sites untouched.
trustRemoteCode dropped from PerModelConfig to preserve main's per-load consent.
* feat(chat): apply/persist per-model config through the load flow
handleCheckpointChange threads meta.config into the selection; stageOrLoad and
the autoload/Hub-run paths now apply the picker config (explicit pick or saved
remembered config) via applyPerModelConfigToRuntime before staging/loading, with
keepSpeculative set so a remembered speculative mode survives the model switch.
Replaces the old remembered-load-settings seeding (resolveInitialConfig now the
single source). SelectedModelInput carries config.
* refactor(chat): remove per-model load config from the right sidebar
The load knobs (context, KV cache, speculative, draft tokens, tensor parallel)
and the chat-template editor now live only in the picker config step. The sheet's
Model section keeps the staged Load/Cancel flow (config is applied at pick time);
sampling params, system prompt, and RAG are unchanged. Deletes the superseded
remembered-load-settings module + the store's applyRememberedLoadSettings action,
removes the now-dead sheet state/imports, and points the settings reset at
unsloth_model_configs. Delete-cleanup deferred (stale config is LRU-capped).
* fix(model-picker): remove leftover sidebar-staging cogwheel + empty Model section
The downloaded-variant gear (ModelLoadSettingsAction) staged a model straight
into the right-sidebar Run-settings flow -- the old 'configure before load' path
now fully replaced by the in-picker config step. Removed the gear + its component.
Also gate the sheet's 'Model' section to staged picks only (pendingSelection):
after the load-knob strip its content is staged-only, so it was rendering an
empty section header whenever a model was merely loaded.
* chore(chat): remove dead per-model-config setters + modelControlsDisabled
After the load-config UI moved into the picker, the store's per-model setters
(setKvCacheDtype/setSpeculativeType/setSpecDraftNMax/setTensorParallel/
setCustomContextLength/setChatTemplateOverride) had zero callers
(applyPerModelConfigToRuntime writes via setState), and the sheet's
modelControlsDisabled was unreferenced. Verified dead across the whole tree.
* fix(chat): config-step Load actually loads (ignore Load-on-selection)
Root cause: with Settings > Chat > 'Load on selection' turned OFF, the config
step's load went down the deferred-staging path -- opening the right sidebar with
'<model> is staged, not loaded yet / Choose Load model'. The in-picker config step
IS the deliberate load action, so its Load now loads immediately (or downloads +
auto-loads when not cached) regardless of the toggle. Renamed the button
'Run model' -> 'Load model' to match. Native/dropped picks still honor the toggle.
* refactor(chat,hub): retire 'Load on selection' — config step is the only load flow
The in-picker config step (and the Hub Run button) now fully supersede the old
stage-to-sidebar flow, so the Load-on-selection toggle is removed everywhere:
- chat stageOrLoad: every pick loads immediately, or downloads + auto-loads when
not cached (the previous default behaviour, now universal).
- hub Run: drops the stage branch; downloaded GGUFs load directly with their saved
per-model config (no collision with the chat config step — both end at selectModel).
- store: removed loadOnSelection field/setter/key/default; Settings>Chat toggle and
its settings-reset entry removed.
- staged sidebar section is now a download-progress view (auto-loads on completion).
No manual staging remains; stageModel is used only for background auto-load downloads.
* feat(model-picker): default chat template from GGUF + thread variant through config flow
Read the embedded tokenizer.chat_template from GGUF files (read_gguf_chat_template
in gguf_metadata) and use it as the per-model default. Plumb gguf_variant through
the picker service, /api/picker/chat-template route, frontend templates API, and
use-model-defaults so the right variant's template is fetched.
Also refine the picker config-page/model-selector wiring, drop the dead
ggufNativeContextLength runtime path, and add the per-model-config storage keys to
the settings prefs export.
* feat(model-picker): read safetensors chat template + hide editor where it has no effect
Resolve the default chat template for safetensors models: prefer the modern
chat_template.jinja, fall back to the tokenizer_config.json chat_template field,
then chat_template.json (multimodal processor), then the GGUF embedded template.
Applied to local dirs, the HF cache snapshot scan, and the HF remote fetch.
Hide the chat-template editor in the picker for safetensors models — the override
is only applied at load by the GGUF/llama.cpp backend, so editing it on safetensors
currently has no effect. GGUF keeps the editor. Nothing removed; the dialog stays
for when the safetensors apply path is wired up in a later branch.
* fix(model-picker): set legacy-migration flag only after the write succeeds
Set unsloth_model_configs_migrated only once writeMap confirms the migrated
map persisted, so a quota/storage failure no longer marks migration done and
silently drops the user's pre-existing remembered settings — the next load retries.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* MVP model picker fixes
* MVP picker config fix
* MVP safetensors config
* MVP max seq config
* MVP max seq fix
* Fix static max tokens cap ignoring model context
* Fix picker GGUF scan parity
* fix(studio): harden model picker config loading
Apply remembered per-model configs consistently from picker and Hub loads, keep default configs from overriding standing speculative settings, add config access for direct local GGUF files, and support saving or forgetting active model settings without a reload.
* Fix model picker config flow
* Fix model picker config loads
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Avoid recursive per-model config migration reads
* Apply the displayed context length when loading a GGUF
* Fix template validation, cached template lookup, and failed load rollback
- Validate chat templates with the loopcontrols extension so templates
that use break or continue tags pass the picker validator, matching the
inference renderer that already accepts them.
- Read the default chat template from the newest cache snapshot rather than
an arbitrary iterdir order, so an older cached revision no longer prefills
a stale template.
- Capture the runtime per-model config before a load and reapply it when the
load fails, so a failed switch leaves the active model context, KV cache,
template, and speculative settings as they were.
* Make chat template view only for safetensors models
Custom chat template overrides are applied at inference only for GGUF
models, which pass the template to llama-server. The safetensors backend
renders with the model built-in template and ignores the override, so
editing it would save a value that never loads. For safetensors the
config page now opens the template as a read-only preview with a note
that editing is not available yet. This can become editable once
inference support for custom safetensors templates lands in main.
* Fix model picker config edge cases
- Restore prior runtime config when a load no-ops for the active model
- Cap the picker validator request body via the protected prefixes
- Keep the GGUF context slider max above the loaded context
- Fetch subfolder chat templates for uncached Hub repos
- Show the compare side config when reopening the picker
* Keep saved GGUF context above the fallback ceiling
* Show the model config in the run settings sidebar
* Fix model config sidebar reset and context slider
- Stack the remember toggle and action buttons in the sidebar
- Reset the config to defaults instead of the loaded values
- Fetch the native context so the slider max is not the loaded value
* Fix model picker config and download regressions
- Run picker chat template routes off the event loop
- Depth and root guard local template directory scans
- Restore download manager flow for uncached hub picks
- Apply per model context length on reload
- Import model picker symbols from the feature barrel
* Fix model picker config and cached download sorting
- Restore load settings when a Hub run is rejected mid load
- Reuse one NumericValueInput instead of a duplicate copy
- Fix double decode of the model name in the template route
- Remove the unused reset-to-loaded settings action
- Fix cached model download sorting
* Fix model picker per-model config edge cases
Honor a saved or typed max seq length above the model's native context so
RoPE extended values are no longer clamped and silently overwritten. Allow
typing past native while the slider keeps native as a soft ceiling.
Guard the fetch success paths in use-model-defaults against an aborted
signal, and refetch when the HF token changes.
Hash the chat template content in the sidebar remount key instead of its
length. Enable reset for a GGUF whose native context is unknown, and floor
the context slider max so it can never fall below the min.
* Fix GGUF context auto-fit and gated model config token
Stop forcing a 32768 context when a GGUF native context is unknown so the backend auto-fits to VRAM again, while still honoring an explicit context edit.
Send the HF token as a query param so gated safetensors models resolve their max position embeddings.
Derive model default state during render to drop the set-state-in-effect calls.
* Fix native GGUF context ceiling and guard picker template reads
Restore the native context store field so the sidebar slider keeps the
full ceiling for drag and drop GGUFs. Limit local chat template reads to
the browse allowlist, skip malformed repo ids, and drop unused model
picker exports.
* Fix model picker lint boundaries
* Fix model picker review findings
Chat template editor never seeded its draft. Radix only calls onOpenChange
from internal events, so the seed in the nextOpen branch was dead and a model
with a saved override opened empty. Saving then cleared the override. Drop the
dead branch, treat draft as an untouched sentinel, and reset it on every close.
Uncached Hub picks could auto load a model after the user left the chat. Main
detached the staged pick on route exit and on chat context change. Carry the
context key on the pending pick and skip the load when it no longer matches.
Also clear configTarget when the picker closes, restore the onUpdated ref so
variant rows stop resubscribing on every parent render, skip the LRU write when
the entry is already most recent, import NumericValueInput relatively, and drop
the unused ModelUpdateAction barrel export.
* Preserve GGUF context on active reload
* Fix model picker per-model config regressions
- Stop reloading the already loaded model on re-pick
- Hide infra models from the chat picker
- Detect vision support on cached GGUF repos
- Honor saved maxSeqLength on auto load
- Restore default chat template for local GGUFs
- Warn on save failure and revert config on cancel
- Refetch picker inventory on open
- Persist read only per model config safely
* Fix stale model auto load
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix model picker numeric input sizing and constraints
Size value inputs to their content so long context lengths are not clipped,
restrict them to numeric characters, and stop the speculative decoding label
from truncating in the sidebar.
* Fix picker CI tests and harden chat template resolution for PR #6647
- tests: point the descender guard at the moved model-selector.tsx path
- tests: exclude the disabled Reload model button from the regenerate locator so .first targets the real Regenerate
- picker/service.py: reject symlinked template/gguf leaves that resolve outside the browse allowlist (HF cache reads unchanged)
- compare mode: resolve each pane's own remembered chat template instead of inheriting the other pane's from the store
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Protect future-schema per-model configs from deletion for PR #6647
savePerModelConfig already refuses to overwrite a stored config whose schema version is newer than this client understands, but deletePerModelConfig did not. Unchecking Remember on an older client therefore silently destroyed a newer client's saved config. Apply the same guard on delete and surface the blocked case through the existing saveFailed toast.
* Protect future-schema per-model configs from quota eviction for PR #6647
The save and delete guards already refuse to touch a stored config whose schema version is newer than this client understands, but the quota-eviction path did not, so a full store on an older client could still evict a newer client's config. Skip future-schema entries when evicting and fail the save if the budget cannot be met without them.
* Fix GGUF context persistence, compare context, and rollback settings for PR #6647
Persist a GGUF context override from the user's intent instead of collapsing it against the loaded context, which reintroduced the context-reset (f4838782cb reverted the native-baseline fix). model-config-page now collapses the saved value against native, and use-chat-model-runtime and chat-adapter retain the requested context on load so re-saving another setting keeps the override; a null request stays null so a VRAM auto-fit never becomes a stored override.
shared-composer: a compare pane with no explicit GGUF context now loads at native (0) like single-view, not the session maxSeqLength that silently shrank the shown context.
use-chat-model-runtime: restore the previous model's KV cache dtype and chat template on a failed-load rollback so it runs as it was, not with backend defaults.
* Preserve native path token when reloading the active model for PR #6647
handleReloadActiveModel rebuilt the selection without the store's activeNativePathToken, so reloading a file-picked GGUF after a settings change validated the display label as a repo/path and failed. Thread the active native token through the reload selection so native-loaded models reopen correctly.
* Make picker template validation resilient and accept HF generation tags for PR #6647
Import Jinja lazily inside validate_chat_template so a backend without the optional jinja2 package (GGUF-only installs) still starts instead of raising ModuleNotFoundError at import time. Register a no-op extension for the Transformers {% generation %} assistant-mask tag so pasting a valid HF chat template validates, matching the renderer, rather than being rejected as an unknown tag.
* Honor remembered compare config and parse processor chat_template.json for PR #6647
* Fix failed-load rollback context and processor template map fallback for PR #6647
* Restore speculative decoding config on failed-switch rollback
When a model switch fails after the previous model was unloaded, the
rollback reload restored tensor_parallel, KV cache dtype and the chat
template override, but omitted speculative_type and spec_draft_n_max and
cleared their loaded shadows to null. The previous model therefore came
back running at backend defaults (speculation off) while the UI still
showed it enabled, and the status resync confirmed the off state. Resend
the previous model's speculative settings in the rollback load and keep
the store's active and loaded speculative fields in sync with them.
* Reset max sequence length when a model has no saved config
applyPerModelConfigToRuntime reset every per-model field except
maxSeqLength, which it only wrote when the incoming config had one.
maxSeqLength is the sole field carried on store.params, so selecting a
model with no remembered config left the previous model's value in place
and later loaded the new model at that leaked length. Fall back to the
standing default so an unremembered model loads at its own default.
* Surface a message when a variant update cannot start
startManagedUpdate handled the conflict and error start outcomes but let
busy fall through as if the update began, so the confirm dialog closed
with no job created and the cached variant stayed stale. Show an info
message when the repo is busy with a sibling transfer so the click is
not silently dropped.
* Keep per-model speculative choices out of the global default
A staged load with a per-model or one-off config sets keepSpeculative,
which already skips reading the global speculative preference. The
matching save still ran unconditionally, so the model-specific choice was
written to the global unsloth_chat_speculative_type and a later model with
no saved config started from it instead of Auto. Skip saveSpeculativeType
when keepSpeculative so the per-model choice stays isolated.
* Seed non-active model settings from the app default max length
The Run settings page captured initialMaxSeqLength from the loaded
model's runtime params and fell back to it for a model with no saved
config. Opening settings for a different, unloaded model and clicking
Load then sent the active model's context (for example 64k) instead of
the 4096 default, risking validation failures or OOMs. Seed the default
for non-active models and keep the runtime value only for the active one.
* Prefer sidecar tokenizer chat template over the GGUF copy for variants
_chat_template_from_dir returned the embedded GGUF template first when a
variant was selected, reversing the tokenizer-first precedence of the
no-variant path. A model whose chat_template.jinja or tokenizer_config.json
supersedes a stale embedded template then got the wrong template on
variant selection. Keep tokenizer files first regardless of variant; the
variant only picks which GGUF is the fallback. Adds regression tests for
both the tokenizer-wins and gguf-fallback cases.
* Keep per-model speculative choices load-local in autoload and compare
The interactive load path treats a per-model speculative choice as
load-local and skips writing it to the global default. Autoload and
generalized compare still called saveSpeculativeType unconditionally, so a
remembered off or ngram setting leaked into unsloth_chat_speculative_type
and later models with no saved config inherited it. Persist the global
preference only when the value came from the global settings.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: record the compare pane's loaded context in runtime state so the active model's settings and any reload or save use it, not the previous context
* Studio: notify the user when a Hub autoload can't start because another download for the model is already running, instead of silently dropping it
* Studio: drop the merge's orphaned staged-model store helpers and unused alert imports
The main merge left isPendingGguf and pendingSelectionMatches referencing the
removed PendingModelSelection type, and the alert-dialog/alert imports unused
after the permission-mode dropdown replaced the bypass dialog, so tsc -b failed.
* Studio: cache a null default chat template so the viewer stops re-fetching it
A model with no sidecar or embedded template resolves to a terminal null, but
that result was never cached, so reopening the template viewer re-ran the
backend and Hugging Face lookup every time.
* Studio: detect direct-file GGUFs in run settings so Max Tokens uses their context
A GGUF loaded from a local file or custom folder has no variant label, so the
run-settings panel treated it as non-GGUF and clamped Max Tokens to the session
max_seq_length instead of the loaded GGUF context. Detect it via the reported
GGUF context and the .gguf checkpoint suffix, matching the chat page.
* Studio: prompt to re-select a local model file when its lease expired before reload
A file-picked GGUF is reachable only through a native path token that the
desktop host prunes after a TTL. Reloading reused that token blindly, so a
reload long after the initial load failed with an opaque error. Track the
token's expiry and, when it has passed, ask the user to re-select the file
instead of attempting a doomed reload.
* Fix descender-clipping test to tolerate sidebar layout utilities
The sidebar account-block div carries layout utilities (min-w-0, flex-1)
between 'flex' and 'flex-col', so the descender-clipping guard's regex,
which required 'flex' immediately followed by 'flex-col', no longer matched
and the test failed to locate the account-block div. Generalize the prefix
to allow intervening flex utilities while still capturing the leading-*
class before the collapsible visibility utility and asserting leading-tight,
so the guard against clipped glyph descenders is fully preserved.
* Harden picker chat-template resolution
Enforce the 64 KiB chat-template contract at the validate endpoint's request
model so a direct caller cannot submit a template far larger than the frontend
allows (MaxBodyMiddleware only bounds the whole request body, not this field);
oversized templates now return a clean 422.
Apply sidecar-over-GGUF template precedence globally across cached snapshots
instead of per snapshot. A repo with multiple cached revisions previously
returned the first snapshot's template, so a newer GGUF-only revision could
win over an older revision's maintained chat_template.jinja sidecar, which
contradicted the documented intent that sidecars supersede the embedded copy.
* Guard per-model config against future-schema and lossy migration
Two forward-compatibility gaps in the versioned per-model config store:
- The load/apply path returned and normalized a stored record without checking
its schema version, so a record written by a newer client was reinterpreted
under the current schema and applied to a live model load, even though save,
delete and eviction all refuse to touch future-schema records. Reject
future-schema records on load too.
- The one-time legacy migration enforced the storage budget without protecting
the entries it had just migrated and set the completion flag unconditionally.
When storage was already full of future-schema records (which are unevictable
by an older client), the migrated entries were the only evictable ones and
could be dropped while migration was still marked complete. Protect the
migrated keys during eviction and only mark migration complete when they
survive, so it retries once space frees up.
* Discard chat-template validation results after the dialog closes
Server-side template validation is async, but closing or cancelling the editor
did not abort it, so a late-arriving valid response still called onSave and
applied a template the user had already dismissed. Track a validation token
that is bumped on close and ignore any validation result whose token is stale.
* Record native lease expiry when loading a picked GGUF from the chip
The pending-native-model chip loaded via stageOrLoad directly, bypassing
loadNativeModelIntent, so activeNativePathExpiresAtMs was never recorded for a
chip-loaded file. A later reload then either skipped the lease-expiry guard
entirely (expiry left null) or compared against a previously loaded file's
stale expiry, so reload could reuse an already-pruned token or wrongly block a
still-valid one. Route the chip through loadNativeModelIntent, which builds the
same selection and records the expiry.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Prefer sidecar template for a directly selected local GGUF file
A direct .gguf file path read its embedded chat template without checking the
parent directory for a maintained sidecar (chat_template.jinja /
tokenizer_config.json), while directory and variant selections already prefer
the sidecar. That let the config editor preview or save a stale embedded
template for the same model depending on how it was selected. Check the parent
directory sidecars first, then fall back to the embedded copy, and cover both
paths with tests.
* Resolve cached chat template per revision, newest first
The earlier change searched every cached snapshot for a sidecar before
considering any snapshot's embedded GGUF template, which let an obsolete sidecar
from an older revision override the newest revision's template. Restore
per-snapshot resolution (newest first): a revision's sidecar still supersedes
its own embedded GGUF copy, but a newer revision is no longer overridden by an
older revision's sidecar.
* Preserve autoload transport conflicts and surface background busy downloads
- When a Hub autoload hits a transport conflict, keep pendingHubAutoLoad bound
instead of clearing it. Clearing it re-keyed the download surface and its
cleanup cancelled the conflict the toast tells the user to resolve, so the
Hub resume affordance was gone the moment it appeared. Return early on
conflict, mirroring the started branch, so resolving it from the Hub still
auto-loads on completion.
- The background-download branch handled started and conflict but silently
dropped a busy outcome, leaving the user with no feedback when a peer variant
of the same repo was already downloading. Surface the same busy toast the
autoload path uses.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
* Studio: Inkling support fixes (context sizing, tool-call healing, reasoning effort, audio icon)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: reject binary web_search fetches instead of decoding them into replacement chars
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Match web-fetch MIME subtypes exactly and detect control-char binary
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Sniff binary magic bytes and retry undeclared non-UTF-8 pages as text
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Harden web fetch binary sniffing
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Simplify web fetch binary guard
* Sniff unknown MIME types and handle Latin-1
* Sniff ambiguous Office MIME and prefixed magic
* Decode BOM-marked Unicode web content
* Studio: require ASCII evidence for declared Latin-1/cp1252 web fetches
Latin-1 and cp1252 decode every byte to a printable character, so a high-byte
binary body declared as iso-8859-1/windows-1252 decoded cleanly and slipped
past the control-character binary check. Apply the existing ASCII-structure gate
to those declared decodes as well. Scoped to the Latin family so legitimate
non-Latin single-byte pages (Cyrillic, Greek) are not rejected.
* Revert "Studio: require ASCII evidence for declared Latin-1/cp1252 web fetches"
This reverts commit c7fbec216c.
* Studio: tighten web-fetch binary guard comments
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
* Studio: stream live tool output with SSE heartbeats and fix web page extraction
Server-side python/terminal tools now stream incremental stdout to the chat
UI while running (new tool_output SSE event), and every blocking tool
execution emits heartbeat keepalives so reverse proxies (Cloudflare tunnels
cap idle streams at ~100s) cannot drop the connection mid-turn. The tool
loop routes also emit a stall keepalive during silent prompt prefill between
tool iterations. The final role=tool message the model sees is byte-identical
to before, so tool-call parsing, nudging, and healing are untouched.
web_search page fetches now extract main content: GitHub repo root pages are
rewritten to the README API (with HTML fallback), hidden/aria-hidden client
error placeholders are dropped, conversion scopes to article/main, and known
boilerplate fragments are stripped. Non-HTML responses are returned raw
instead of being run through the HTML converter.
The frontend renders live-scrolling tool output inside running python and
terminal cards, and a chat stream that ends without a terminal signal now
surfaces an explicit interrupted state with a Retry action instead of
silently ending the turn.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: fix content-type sniffing, unlimited-timeout drain, and env parity in tool streaming
Content-Type sniffing: get_content_type() defaults to text/plain when the
header is absent, so the sniffing fallback never fired and header-less HTML
came back as raw markup. Report an empty type for a missing header and sniff
the body whenever the declared type is not HTML, so mislabeled text/plain
HTML pages are converted like before the extraction change.
Unlimited timeout drain: with tool_call_timeout disabled the old path used
communicate(timeout=None) and waited for EOF, but the streaming drain capped
the post-exit drain at a 5 second join, truncating output from a grandchild
that holds stdout open. When timeout is None, drain until EOF or the cancel
event fires; finite timeouts keep the bounded remaining-budget join.
Env parity: drop the PYTHONUNBUFFERED=1 injection on the streaming path so
the child invocation is byte-identical with and without streaming (the env
var was model-visible via os.getenv). Live streaming granularity now depends
on the child flushing; unflushed output arrives in ~8 KB chunks or at exit
and the final result is unchanged, with SSE heartbeats covering the gaps.
* Studio: stream tool-call arguments while the model writes them
A model writing a large tool call (a full python game is minutes of
generation) produced nothing on the stream: the structured path
accumulated delta.tool_calls fragments silently after the provisional
card, and the text path's DRAINING state consumed everything until
stream end. The user saw a dead Running spinner while the model was in
fact writing code, and the byte-silent SSE segment was also the window
where proxies drop the connection.
New tool_args SSE events stream the arguments as they generate. The
structured path forwards each fragment once a provisional card exists
(backlog first, so the card starts from the top of the call). The text
path sniffs the drained call for an enabled tool name and streams the
raw call text under the id the stream-end parser assigns its first call
(call_0), so the final tool_start reconciles the same card; the sniff is
gated on enabled names plus the provisional size floor, and prose or
ordinary JSON answers never spawn a card. The safetensors loop streams
the drained render_html call to its existing provisional card the same
way.
The chat adapter accumulates the raw stream per card and feeds a partial
JSON parse (call envelopes and stringified arguments unwrapped) into the
part's args, so the python and terminal cards render the code live and
the render_html canvas builds while streaming; both cards now say
Writing code / Writing command during this phase via useToolArgsStatus.
Display only: the parser input, the executed call, and the conversation
the model sees are byte-identical, covered by new loop-level tests for
the structured path, the text path, and the no-tool JSON answer.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: keep full tool output visible past the model cap; heal /mnt/data habits
Live testing surfaced two issues in the tool streaming UX.
First, a long python stdout ended in '... (truncated' in the finished
card: the model-visible result is capped by tools._truncate
(_MAX_OUTPUT_CHARS, previously 8000 chars) to protect the context
window, and the card rendered that capped text even though the live
stream had already shown everything. The cap stays (raised to 16000,
overridable via UNSLOTH_TOOL_RESULT_MAX_CHARS) but display and model
concerns are now split: the adapter preserves the accumulated live
stream on tool_end whenever it captured more than the final result, and
the finished python/terminal cards prefer it. The live-stream ceiling
rises from 16 KB to 400 KB (chunks batch per poll, so SSE stays cheap),
and both the live pane and the finished card render only the last 2000
lines with a Show all control so a huge output cannot jank the DOM. The
truncation notice now tells the model the user saw the full output and
that written files persist in the working directory. The final result
string remains byte-identical with and without streaming.
Second, models trained on ChatGPT code-interpreter transcripts write to
/mnt/data, which does not exist here (the sandbox CWD is a per-thread
persistent dir). Three layers, all identical across streaming and
non-streaming paths: the python/terminal tool descriptions gain one
sentence saying to use relative paths in the persistent CWD; a failed
execution whose output shows a missing-file error on a known
code-interpreter prefix (/mnt/data, /mnt/outputs, /home/sandbox,
/workspace) gets a model-visible retry hint appended after truncation
so it always survives; and a sitecustomize shim on the sandbox
PYTHONPATH remaps those prefixes onto the CWD in open()/os.makedirs()
with a one-line stderr notice, covering the python tool and any Python
launched from the terminal tool without touching the exec wrapper (so
tracebacks keep their line numbers). Bash-level file operations cannot
be redirected without root or mount namespaces, so they rely on the
description and the hint.
* Studio: fix hidden-element parsing, heartbeat gaps, and tool output id collisions
Review follow-ups on the tool streaming work:
- _html_to_md: treat any present hidden attribute value as hidden (it is an
enumerated attribute whose invalid value default is the Hidden state, so
hidden="false" is still hidden), and implement HTML5 optional end tags so
an unclosed <p hidden> or <li hidden> ends at the next sibling start tag
instead of swallowing every following sibling until the parent closes
- tool_stream_exec: keep heartbeats flowing after the live-output cap; a
tool that keeps printing past the cap kept the queue non-empty, so neither
tool_output nor heartbeat events were emitted and the SSE stream went
silent past proxy idle timeouts
- routes/inference: forward tool heartbeats before the
disable_parallel_tool_use drop window swallows events, so a dropped call
that executes server-side cannot leave the Anthropic stream silent
- llama_cpp: close the provisional text tool card with a tool_end when the
drained call fails to parse (DRAINING false-positive path), so the card
cannot spin forever while the text is delivered as content
- tools: decode terminal output as utf-8 with errors=replace like the python
tool; invalid bytes used to raise UnicodeDecodeError from communicate() on
the non-streaming path and silently truncate the streaming reader, so the
two paths diverged
- sitecustomize: patch io.open alongside builtins.open; pathlib Path.open,
read_text and write_text call io.open directly and bypassed the remap
- frontend: scope the toolLiveOutput/toolFullOutput store keys by pane
(modelType and pairId) and clear stale entries on tool_start; backend ids
like call_0 repeat across turns and across concurrently streaming panes
(compare mode), so a later turn or another pane could display the wrong
preserved output, and run-end cleanup now clears only its own keys
Each backend fix carries a regression test that fails on the previous code;
the byte-identity tests between streaming and non-streaming stay green.
* Studio: keep tool failure status visible and truncation/remap notices truthful
Finished python/terminal cards preferred the fuller live stream by length
alone, so a tool that printed a lot then timed out or exited non-zero showed
the captured stdout but dropped the final result's status (timeout notice,
Exit code N). preferFullToolOutput now shows the stream when the result is
just its truncated prefix, and appends the result otherwise so the failure
tail always survives and the copy button copies both.
The result truncation notice claimed the user was shown the full output, but
the same wrapper serves non-streaming chat/API and direct execute_tool()
callers where nothing is streamed to anyone. The notice is now mode-neutral
and stays byte-identical with and without an output_callback, keeping the
streaming vs non-streaming invariant intact.
The sandbox sitecustomize shim now remaps /tmp/outputs into the working
directory only while it does not already exist, so a real /tmp/outputs the
user's own code created is never shadowed; /tmp/outputs also joins the
missing-path retry-hint list.
* Studio: suppress hidden void elements and keep live output scroll pinned only when at bottom
* Studio: drop capped tool output without concatenating; remap pathlib mkdir
Past the live-output cap stream_tool_execution built item + _drain_pending()
(the current chunk joined with every queued sibling) only to discard it in the
capped branch, so a chatty tool (yes, a tight print loop) could enqueue far
more than one poll interval of text and blow past the memory/CPU ceiling the
cap exists to enforce. Drain and drop queued items without building a combined
string, still counting each drain toward the heartbeat cadence so the SSE
keepalive survives.
Generated code often prepares code-interpreter paths with
Path('/mnt/data').mkdir(parents=True, exist_ok=True); pathlib drives that
through os.mkdir (not the patched os.makedirs) per component and, on
FileExistsError, probes the unpatched os.stat via Path.is_dir(), so the setup
raised before open() ever ran. Patch os.mkdir with the same remap and patch
Path.mkdir so the whole parents/exist_ok dance lands on the mapped working
directory and stays idempotent; real paths still pass through.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: generalize sandbox write remap and hint to any hallucinated absolute path
Models invent absolute paths from seeing their CWD (a DeepSeek run did
open('/home/ubuntu/Sandbox/flappy_bird.html', 'w') and died with
FileNotFoundError). A prefix list cannot enumerate these, so the sitecustomize
shim gains a write-mode fallback in open()/io.open(): when a write/create-mode
open targets an absolute path outside the CWD whose parent directory does not
exist, redirect it to the basename in the CWD and emit the same one-line stderr
notice, echoing the original path. The prefix remaps still run first (they cover
reads and preserve subpaths); read modes never hit the fallback so real system
files fail or succeed truthfully; bytes paths pass through. The fallback is not
applied to mkdir/makedirs/Path.mkdir, since creating an arbitrary absolute
directory can legitimately succeed on the host, and that decision is documented
in a comment.
The model-visible retry hint now echoes the real failing path (parsed from the
traceback tail) instead of the canned /mnt/data example, and fires for any
absolute path outside the working directory, not just the enumerated prefixes,
while a relative miss still gets no hint.
The shim wrapper still adds one frame to tracebacks that surface open() errors;
suppressing only our frame has no clean standard mechanism (a wrapper always
adds a frame), so the frame is left as an accepted compromise.
Tests: hallucinated absolute write remaps to the CWD basename across w/a/x/w+;
reads of a missing absolute path pass through untouched; writes to an existing
external dir pass through; prefix subpaths still preserved; end-to-end write
fallback lands the file in the sandbox workdir identically with and without
streaming; the hint echoes the actual path for convention and non-convention
absolute paths alike.
* Studio: kill exited process groups on drain; bound the over-cap output batch
_drain_process_output killed the process only via _kill_process_tree, which
short-circuits once the parent has exited, so a grandchild that inherited
stdout and outlived the parent was never signaled: a finite-timeout run could
return while it kept holding the pipe, and a timeout=None cancel left it
behind. Capture the setsid process group before waiting and SIGKILL that group
at both give-up points so the whole tree is torn down.
The streaming wrapper's first over-cap batch joined the current chunk with the
entire pending backlog before enforcing the live-output cap, so a chatty tool
could allocate far past the cap on the crossing batch. Bound the drain to the
remaining budget and drop the surplus in place, keeping the truncated output
byte-identical to joining everything.
* Studio: harden sandbox path healing and process/generator cleanup
Sandbox sitecustomize shim:
- Make the generalized write fallback collision-safe: never redirect an
invented absolute path onto an already-present CWD file (refuse and let the
original open raise FileNotFoundError, preserving the workspace file).
- Only w/a/x create a file; r+/rb+ are read-update modes that require the
target to exist, so a bare + no longer trips the write fallback.
- Gate every convention-prefix remap (/mnt/data, /mnt/outputs, /home/sandbox,
/workspace) on the prefix root being absent, so a real host mount is never
shadowed; a miss under an existing real prefix passes through.
- Patch os.open so Path.touch and other low-level creators heal convention
paths too, matching the Path.mkdir patch.
Local code execution (tools.py):
- Capture the setsid process group right after Popen (before any watcher can
poll/reap the leader) and thread it through the cancel watcher and drain.
- Kill the captured group in the non-streaming python/terminal timeout branch
so an exited leader no longer leaks a stdout-holding grandchild (matches the
streaming drain path).
- Guard os.getpgid/os.killpg by platform so streamed execution no longer
raises on Windows; fall back to single-pid kill.
- Judge missing-path hints against the executor's real workdir so a legitimate
miss inside a project workspace outside the sandbox root is not mislabeled.
Tool streaming routes (routes/inference.py):
- Drain a pending next(gen) worker before closing the generator in the
safetensors and Anthropic tool streams, so a disconnect no longer races
gen.close() (generator already executing) or leaks the thread/generator.
HTML to markdown:
- Only drop boilerplate lines composed entirely of known furniture phrases so
real prose that merely quotes one (for example "we use cookies to
authenticate requests") is preserved.
Adds hermetic tests for each change.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: keep aside callouts, contain sandbox path remaps, and keepalive dropped Anthropic tool events
_html_to_md: stop dropping <aside> unconditionally. Documentation pages
render notes/warnings/examples as aside admonition callouts; those inside
the selected article/main scope are real content. A furniture aside outside
the scope is already excluded by the main-content pass.
sitecustomize: contain the code-interpreter path remap under the sandbox
CWD. A hallucinated habit path such as /mnt/data/../other_session/file no
longer escapes the per-conversation workdir; parent-traversal components in
the suffix are dropped and a '.'/'..' write-fallback basename is refused.
routes/inference: emit a rate-limited comment keepalive when the Anthropic
Messages stream drops tool_output/tool_args events. A chatty tool keeps the
generator busy so the stall keepalive never fires and the tool wrapper emits
heartbeats only while idle, which left the SSE stream silent past proxy idle
caps; the OpenAI passthrough paths forward these events, this path now keeps
the connection alive.
* Studio: bound the tool-output chunk that first crosses the live cap
_drain_queue joined the entire chunk that first crossed the live-output
cap before dropping the rest, so a single multi-megabyte line (or any
chunk dequeued once the budget was already met at max_chars <= 0) was
materialized in full only to be truncated away, defeating the memory
ceiling the cap enforces. Slice the crossing chunk to one character past
the budget: that preserves the caller's overflow signal and its
byte-identical truncation while dropping the arbitrarily large remainder
in place.
* Studio: scope missing-path hint to the failing line, keepalive dropped-call output, and preserve truncated tool streams over byte length
- tools._missing_path_hint: the code-interpreter convention-prefix trigger
scanned the whole output, so a convention prefix mentioned only in a
traceback frame (a /workspace project root) or printed by the user's code
would add a misleading 'use a relative path' hint even when the actual
FileNotFoundError was a relative or in-workdir path. Scope the convention
test to the failing-path error line(s), matching _extract_missing_abs_path.
- _anthropic_tool_stream: the tool_output/tool_args rate-limited keepalive sat
after the drop_until_tool_end skip, so under disable_parallel_tool_use a
chatty second-or-later tool call was dropped whole with no keepalive, letting
an idle proxy kill the SSE stream. Check the keepalive branch before the drop
skip (like the heartbeat branch) so dropped-call output keeps the stream alive.
- preferFullToolOutput / chat-adapter: a truncated result can be longer than
the live stream by byte count once its footer, an 'Exit code N:' notice, or an
__IMAGES__ base64 tail is appended, so the length-only gate discarded the full
stream and the finished card fell back to the truncated text. Add a shared
truncation-aware shouldPreserveFullOutput used by both the write and read
sites: preserve the stream whenever the result carries the truncation footer.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: skip the habit-path hint for real project paths under a convention prefix
* Kill captured process group on streamed wait-timeout
The streamed drain path's proc.wait() timeout branch only called
_kill_process_tree(proc). If the leader exits in the narrow window between
the wait timing out and _kill_process_tree sampling its pgid, that helper
short-circuits on the reaped leader and a stdout-holding grandchild in the
same group survives. Also kill the captured pgid there, matching the
non-streaming communicate() timeout path. Adds a hermetic regression test
that models the reaped-leader race by stubbing _kill_process_tree.
* Fix 3.10 pathlib write_text remap and honor cancel in finite drain
On Python < 3.11 pathlib routes Path.open / read_text / write_text through
a module-level accessor singleton whose open attribute captured the original
io.open at import time (_NormalAccessor.open = io.open). Patching io.open in
the sandbox shim therefore never reached that captured reference, so a
Path('/mnt/data/x').write_text(...) raised FileNotFoundError on 3.10 while
passing on 3.11+ (which dropped the accessor and calls io.open at call time).
Repoint _NormalAccessor.open at the same io.open wrapper via a staticmethod,
guarded so it is an idempotent no-op on 3.11+. Keep the test save/restore
helpers symmetric so the accessor is restored too, and add a hermetic
write_text/read_text remap test that covers every version.
Also honor cancellation while draining inherited stdout after the leader
exits. Once the leader is reaped the cancel watcher returns (its loop is
while proc.poll() is None), so the finite-timeout drain did one blocking
reader.join(timeout=remaining) that ignored cancel_event and kept draining a
chatty grandchild for the whole budget after a disconnect/Stop. Poll
cancel_event in 0.5s slices against a deadline like the timeout=None branch
and kill the captured process group promptly on cancel. The normal path still
reaches EOF on its own, so the streamed vs non-streamed result is unchanged.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: port no-tool stream keepalive/drain and fix subprocess/queue/extraction asymmetries
Streaming no-tool paths now match their tool twins:
- _anthropic_plain_stream, safetensors/MLX no-tool stream, and standard GGUF
no-tool stream run next(gen) in a worker with a timed SSE keepalive loop so a
long prompt prefill cannot leave the stream idle past a proxy cap.
- The Anthropic plain and safetensors/MLX no-tool teardowns now drain the
pending next(gen) worker and close the generator on disconnect instead of
leaking the suspended generator.
Other asymmetries:
- Non-streaming _python_exec/_bash_exec always drain via _drain_process_output
(output_callback may be None) so a cancelled run reaps a stdout-holding
grandchild that outlived the leader instead of blocking in communicate(). The
joined bytes are identical to communicate(), so streamed vs non-streamed
results stay byte-identical.
- _build_bypass_env installs the sitecustomize path shim on PYTHONPATH (prepend,
keeping the operator's entries) so /mnt/data remap works in bypass mode too.
- GGUF forwards output_callback to execute_tool only when the callable accepts
it (shared accepts_output_callback), matching safetensors and preserving
legacy monkey-patched signatures.
- tool_stream_exec bounds accepted live output at the producer boundary so a
chatty tool cannot grow the queue without limit under consumer backpressure
and cannot keep the drain spinning and starve heartbeats.
- html_to_md implicit-close now searches past unclosed inline descendants so a
hidden <p>/<li> is closed by a following block; main-content scoping gates on
the largest single <article>/<main> so a swarm of tiny cards cannot pass the
threshold in aggregate and displace the real main.
- preferFullToolOutput re-attaches the "Exit code N:" prefix to the fuller
stream instead of appending the still-prefixed result, so a failed truncated
tool no longer duplicates its stdout in the finished card.
Adds hermetic tests for each.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Preserve short live output on timed-out tools; strip inline-CSS-hidden subtrees and score truncated main-content scopes
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten chat tool streaming comments and docstrings
* Keep HTML READMEs from the GitHub API and preserve interrupted tool output
Convert a 200 HTML README body from the GitHub README API to Markdown
instead of discarding it and falling back to the repo page chrome, and
promote captured live stdout to full output when a tool never reaches
tool_end (stream interrupted or cancelled) so the partial diagnostics
stay on the finished card.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: anchor HTML sniff, keep repeated sandbox writes, reuse textual tool ids
Anchor _looks_like_html to the leading doctype/tag so a Markdown README that
opens with a fenced HTML example stays Markdown (no html_to_markdown
corruption), while bare HTML fragments (<body>/<article>/<section>) are still
detected and converted on a missing/wrong Content-Type.
Let the sandbox write fallback re-serve a target it already healed for the same
invented absolute path, so iterative overwrites of a generated artifact stop
failing with FileNotFoundError while the anti-clobber guard still refuses
unrelated same-basename files.
Reconcile the first textual tool call carrying an explicit id onto the open
provisional TEXT card instead of spawning a duplicate card under that id.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: run implicit-close before skipping tags and keep leading README tables as Markdown
A skipped block (<nav>/<footer>) is an HTML5 optional-end-tag closer of an
open <p>, but handle_starttag returned before the implicit-close bookkeeping,
so a never-closed <p hidden> kept its hidden mark and swallowed every following
sibling. Run _close_implicit before the skip decision so the hidden mark is
released and trailing content renders.
Drop <table> (and its <thead>/<tbody>/<tr>/<td>/<th> children) from the
_looks_like_html leading set: Markdown READMEs routinely open with a raw HTML
<table> badge/layout row, and sniffing that as HTML collapsed the whole
Markdown body through html_to_markdown, exactly like the already-excluded
<div align>/<p align> layout headers.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: make bypass-permissions Popen double faithful to the unified drain path
The non-streaming _python_exec/_bash_exec now share _drain_process_output,
which reads proc.stdout in a reader thread and calls proc.wait(); the test
double only implemented communicate(), so bypass-mode bash returned an
AttributeError instead of the faked output. Give _FakeProc a readable stdout
pipe (yields the fake line then EOF), wait()/poll()/pid, so the test exercises
the real drain path on both the python and bash bypass branches.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: persist sandbox path heals across runs and suppress nested hidden lists
* Studio: run tool Python child unbuffered (-u) so unflushed prints stream live
A long-running snippet doing bare print() without flush=True never reached
the live-output pane: CPython block-buffers stdout when writing to a pipe, so
_drain_process_output's readline() saw nothing until the buffer filled or the
process exited. Launch the child with the interpreter -u flag so stdout is
unbuffered and each print streams as it is produced.
-u is applied unconditionally on both the streaming and non-streaming path, so
the child invocation stays byte-identical with and without streaming and the
final joined result is unchanged (buffering/timing only). Unlike the earlier
PYTHONUNBUFFERED=1 env injection that was removed, -u does not pollute the
child's os.environ and is not visible via os.getenv.
* Render only the selected main-content subtree in html_to_markdown
The main-content heuristic sized each <article>/<main> candidate
individually to pick the largest subtree, but then rendered every
matching tag in the document. A page with one real article plus
sibling related-post cards or comment threads passed the size gate on
the real article yet still emitted the unrelated siblings.
Size and render the same chosen subtree so only the selected
main-content subtree reaches the output.
* Studio: tighten chat-tool-streaming fix comments
* Studio: store tool-output-scope separators as unicode escapes
The pane-scope and tool-output-key separators were literal NUL (0x00) bytes, which made git treat the file as binary and hide its diff and blame. Write them as \u0000 escapes instead; the runtime key value is unchanged.
* Studio: bound tool-stream teardown when the client disconnects
stream_tool_execution ran its yield loop with no try/finally, so a gen.close() on client disconnect (GeneratorExit at a yield) skipped the worker join and never signalled cancellation. A tool that does not poll cancel_event mid-flight (web_search, MCP, search_knowledge_base) then kept request teardown blocked until the tool's own timeout. Thread the request cancel_event into the wrapper, set it only on the abnormal-exit path so a clean multi-tool turn is unaffected, and bound the worker join to a few seconds; the daemon worker cannot outlive the process.
* Studio: sandbox path remap no longer masks missing reads
The sandbox sitecustomize shim remapped code-interpreter prefixes (/mnt/data, /workspace, ...) onto the working directory for every open mode, including reads. A read of a path that truly did not exist was silently redirected onto a same-basename workdir file instead of raising on the path the model used, hiding real missing-input errors. Remap writes and creates as before, but remap a read only when the mapped workdir target already exists (re-reading a just-written artifact); otherwise keep the original absolute path so the failure stays truthful.
* Studio: bound web fetch with one overall deadline and cancellation
The web fetch applied timeouts per network operation, so a GitHub README API attempt plus its HTML fallback plus up to five redirect hops could run well past the tool timeout, and nothing aborted once the client had disconnected. Add a single wall-clock deadline shared across the API attempt, the fallback, every redirect hop and the body read, cap each hop's socket timeout at the time left on the budget, and poll cancel_event. SSRF host pinning, per-hop redirect revalidation, the five-hop cap and the size cap are unchanged.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: keep tool-stream teardown off the event loop on disconnect
The bounded worker join added for disconnect safety still ran on an abnormal close, so a client disconnect could wait the full join timeout; and the safetensors and Anthropic tool streams closed their generator synchronously on the event loop, unlike the GGUF path. On abnormal exit the daemon worker is abandoned, so join with a zero timeout instead of waiting; offload the safetensors and Anthropic gen.close to a thread to match GGUF; and surface a heartbeat as soon as cancel_event is set while the worker is silent so the route regains control at once instead of after a heartbeat interval.
* Studio: extend the web-fetch deadline to DNS, the body read, and search
The overall fetch deadline did not cover host resolution or the response body read, and query-mode web_search ignored cancellation. Resolve hosts (initial and every redirect) on a budget-polled helper so a slow or pre-cancelled getaddrinfo aborts on time; read the capped body in chunks with the budget re-checked between them so a slow-drip server cannot stretch a single read past the deadline; and gate the blocking DDGS query on cancel_event on both sides. SSRF host pinning, per-hop redirect revalidation, the five-hop cap and the size cap are unchanged.
* Studio: defer the sandbox remap notice and tighten os.open create flags
The one-shot remap notice fired while computing the mapping, so a read that kept its original path emitted a false notice and spent the notice a later genuine remap needed. Only emit it once _remap_open commits to the redirect. Separately, os.open classified O_TRUNC / O_APPEND without O_CREAT as creating, but those cannot create a missing file, so a missing target now stays truthful (only O_CREAT maps to the creating mode).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: convert only genuine HTML README bodies, not Markdown with a leading block tag
The GitHub README API returns the raw file, almost always Markdown. _looks_like_html classified a Markdown README opening with a block tag (<ul>, <ol>, <dl>, <pre>, <blockquote>) as HTML, so _fetch_page_text ran it through html_to_markdown and collapsed its headings, lists and fenced code into a single line. Sniff the README body with a stricter document-level check (doctype or a leading <html>/<head>/<body>) so only a real .html README is converted; the general page path is unchanged.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Surface unclassified mid-stream Anthropic errors as SSE error events
The local Anthropic tool-stream and plain-stream paths called
_anthropic_stream_error_event(e) with force defaulting to False, so an
unclassified mid-stream failure (llama-server crash, decode OOM, a
dropped upstream socket) returned no event. The except block then fell
through to emitter.finish(), emitting a normal message_delta and
message_stop that masked a truncated turn as a clean finish.
Pass force = True at both fall-through sites so an unclassified failure
emits a 500 SSE error event and returns, matching the Anthropic
passthrough path that already forces it. Add regression tests covering
both stream paths.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: give each tool run a unique part id so finished cards keep their own output
Backend tool ids restart at call_0 every assistant response, and the
transient toolLiveOutput/toolFullOutput store maps were keyed by pane
scope plus that bare backend id. Two turns in the same pane therefore
shared one key: the stale-clear at tool_start only guards the forward
direction, so when a later call_0 finished and wrote its preserved full
output, every earlier still-mounted finished card reading the same key
re-rendered and displayed the newer tool's output instead of its own.
Mint one per-run-unique part id per backend id (call_0:<uuid>) and route
tool_start/output/args/end through a single resolver so all events for a
call resolve the same id. The durable part carries the unique id, so the
finished-card readers derive a collision-free key with no change, and the
awaiting-confirmation path keeps its own synthesized id. Outbound replay
stays paired (the assistant tool_call id and the role=tool result
tool_call_id both come from the part id) and gains unique ids across
turns, which strict providers require.
* Studio: tighten PR comments
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
* Studio: exclude /api/export/status from request access logs
The frontend polls /api/export/status every 5s to detect export start, so it
fires continuously even when idle. Each poll emitted an info request_completed
access line, making up most of the server access logs. Add it to _EXCLUDED_PATHS
alongside /api/train/status. The endpoint is unchanged; export state is still
logged by the export modules and streamed over SSE, so no signal is lost.
* Studio: collapse hub download-progress polls in the access log
download-status and gguf-download-progress (plus the dataset equivalents)
are polled about twice a second for the whole download, so each emitted an
info request_completed line. Add them to _QUIET_POLL_PATHS so they collapse
to one heartbeat line per 10s instead of one per poll.
* Studio: log hub download progress at 10% steps
The access log carried no real progress, only poll pings. Emit one
hub_download_progress line per 10% step from the shared snapshot progress
reader, so an active download shows actual percentage without a line per
poll. Throttled per job and resynced if the same download restarts.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: drop successful chat thread/project CRUD from the access log
A single chat turn fans out about twenty requests under /api/chat/threads
and /api/chat/projects (list, fetch, per-message forks, and the message
writes) that only reflect the UI re-rendering. Suppress their 2xx access
line so the log keeps the signal (generation, tool calls, code execution,
engine stats) and errors. Non-2xx on these paths still log.
* Studio: silence transformers torch_dtype deprecation warning
transformers logs "`torch_dtype` is deprecated! Use `dtype` instead!" once at
model-config load via logger.warning_once (logging, not warnings), so a warnings
filter cannot catch it. Attach a small logging.Filter in setup_logging, which
runs before any model config is parsed, to drop that record on the transformers
loggers that emit it.
* Studio: quiet inference load-progress polls and log throttled load progress
The frontend polls /api/inference/load-progress about twice a second for the
whole model load, so each emitted a request_completed line. Add it to
_QUIET_POLL_PATHS (heartbeat) and emit one inference_load_progress line per 10%
step from the load-progress route, so a load shows real percentage instead of a
line per poll.
* Studio: fully suppress download/load progress poll access lines
The download-status, download-progress, gguf-download-progress, active-downloads
and transport-status polls (model and dataset), plus inference load-progress,
fire ~2x/s for the whole download or load. Their progress is now reported by the
hub_download_progress / inference_load_progress events (and the viewer's progress
line), so the per-poll access line adds nothing. Drop it on 2xx and keep it on
errors, instead of the prior 10s heartbeat. Chat CRUD suppression is folded into
the same _is_quiet_success helper.
* Studio: suppress training-tab model/dataset download-progress polls
The training tab polls /api/models/download-progress and
/api/datasets/download-progress about twice a second for the whole prep phase.
These are separate routes from the /api/hub equivalents and only scan the cache,
so their 2xx access line adds nothing (on Windows they always read 0 since the
bytes live in snapshots/, not blobs/). Suppress the 2xx line and keep errors,
alongside /api/models/gguf-download-progress.
* Studio: drop transient pre-auth 401 on chat thread/project polls
On first load the SPA fires chat thread/project GETs before the initial token
refresh, so they 401 until /api/auth/refresh runs and the retries succeed. That
pre-auth 401 is a bootstrap artifact, not an error; suppress it alongside the
already-quiet 2xx line. Genuine 4xx/5xx on these paths, the download/load poll
401s, and all /api/auth/* still log.
* Studio: quiet tab-switch list polls and per-poll scan/reconnect logs
Switching between the Train, Export, and Chat tabs refetches list endpoints on a
timer, and each hit re-logs internal detail. Heartbeat /api/train/runs,
/api/models/checkpoints, /api/models/local and /api/rag/knowledge-bases (10s
window, first hit and errors still log), and downgrade two per-poll INFO lines to
debug: the checkpoints scan summary ("Found N training runs") and the
per-reconnect SSE resume line. The meaningful "replayed N missed steps" line,
logged only when steps were actually replayed, stays at info.
* Studio: enable tokenizer parallelism for dataset prep on Windows/macOS
TOKENIZERS_PARALLELISM was forced off everywhere to stop datasets' forked map()
workers from deadlocking, but that fork only happens on Linux. On spawn platforms
(Windows/macOS) dataset.map() runs in-process (dataset_map_num_proc returns None),
so disabling tokenizer parallelism leaves the fast tokenizer single-threaded and
dataset prep runs serially on one core. Keep it off on Linux (fork safety) and on
for spawn platforms, where there is no fork to deadlock. Measured ~7x faster
tokenization (12.5s -> 1.7s for 20k rows on a 32-core Windows box).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: log throttled training status to the server log
Training step/loss/epoch only went to the UI via SSE, so the server log showed
inference engine_stats and train/runs heartbeats but nothing about the actual
run. Emit one throttled training_progress line (step/total, percent, loss, epoch,
eta) from the CUDA event pump: the first step, then at most every 30s, plus the
final step, resyncing when a new run restarts the counter. Per-step UI streaming
is unchanged.
* Studio: quiet llama.cpp update-status polls and log throttled update progress
The prebuilt llama.cpp update polls /api/llama/update-status about twice a second
for the whole download and install. Suppress its 2xx access line (errors still
log) and emit one throttled llama_update_progress line per 10% step from the
status route, so the update shows progress without a line per poll. The existing
"llama update: installing" and "llama update: success" events still bracket it.
* Studio: quiet the export log-tail poll
The Export tab polls /api/export/logs about once a second to stream the export
subprocess output into the UI panel. Suppress its 2xx access line; the real
progress is already logged as event-driven "Export subprocess status: <phase>"
lines plus the subprocess start and checkpoint-loaded events, and errors still log.
* studio: keep errors and mutations visible in access-log suppression
Make the quiet-success access-log suppression GET-only so chat thread/project
mutations (POST/PUT/DELETE) still log; only their list-poll 2xx and the
transient pre-auth 401 are dropped.
Suppress /api/export/status 2xx only (move it out of the all-status exclude
set) so a 401/403/500 on it stays visible.
Legacy /api/models and /api/datasets download-progress polls emit no
hub_download_progress events, so heartbeat them via the 10s quiet-poll window
instead of suppressing outright, keeping download visibility (notably on
Linux). The event-emitting /api/hub download polls stay fully suppressed.
Update and extend the middleware tests to cover GET-only suppression, the
export-status error path, and the legacy download heartbeat.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: tighten access-log and training-progress comments
Comment-only pass: collapse the multi-line explanations in the logging
middleware and the throttled training-progress logger to fewer lines while
keeping the rationale. No behavior change.
* studio: log structured export_progress phases
Emit a structured export_progress event per phase (consolidated in the server
log like training and download progress) instead of a plain status string, and
add a phase milestone at the start of the heavy export step so the
merge/save/convert is visible in the server log, not only in the forwarded
stdout panel.
* Studio: reset training-progress log throttle on each new run
start_training rebuilds the per-run progress state but left _last_progress_log_ts/_last_progress_log_step at their prior values. A run started within 30s of a previous one whose last logged step matched the new run's first step would hit the step == prev short-circuit and drop the promised first training_progress line, then stay suppressed until the old 30s window expired. Reset both fields when a new job is accepted.
* Studio: keep post-bootstrap chat 401s visible in the access log
The chat thread/project 401 suppression dropped every GET 401 on those prefixes, so a genuine expired-session 401 vanished alongside the transient pre-auth race. Gate the 401 drop on a per-middleware bootstrap latch that flips once /api/auth/refresh first succeeds: before that the 401s are the pre-refresh race and are suppressed; after it any chat 401 is a real failure and logs. Add a test for the post-refresh case.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: limit chat access-log suppression to the exact list polls
The chat thread/project suppression matched by startswith, so it also dropped the 2xx access line for detail and message reads (/threads/{id}, /threads/{id}/messages, /threads/{id}/messages/{id}, /projects/{id}) that are not the high-frequency list polls, losing their access and latency logging. Match the two list paths exactly instead, so only the intended list polls (and their pre-auth 401 race) are suppressed while detail and message reads keep their access line. Add a regression test.
* Studio: reset inference load-progress throttle for each load
The load-progress throttle (_last_load_progress_step) is a module global that persisted across loads, so a cached or small load whose first sampled /api/inference/load-progress response already reported fraction=1.0 hit step == prev (10) from a prior completed load and emitted no inference_load_progress line, while that endpoint's access log is suppressed, leaving the new load with no progress signal. Arm the throttle at load initiation in _load_model_impl so each load's first step always logs. Add a regression test.
* Studio: tighten logging comments
Collapse a few verbose comments (tokenizer-parallelism note, torch_dtype filter, legacy download-poll heartbeat, chat list-path suppression) to fewer lines without changing intent or code.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
The edge fades toggle in Settings > Appearance let users swap the panel
edge gradients for thin divider lines. It added little on top of the
default look, so this removes the setting and all of its wiring while
leaving the default edge fades in place.
- drop the edgeFades field, default, and no-edge-fades class from the
appearance customization store
- remove the settings row, switch, and search entry
- drop the html.no-edge-fades rules from index.css and hub.css
- remove the edgeFades label and description from all locales
- drop the edgeFades field from the personalization backend model and
its test references
* Studio: make the Cloudflare tunnel opt-in (off by default)
A wildcard bind (`-H 0.0.0.0`) auto-started a public trycloudflare.com
tunnel, so exposing Studio on the LAN also published it to the public
internet. Flip the default so the tunnel is opt-in.
- `--cloudflare` is now tri-state (Optional[bool], default None = off),
mirroring the existing --enable-tools/--disable-tools handling. Pass
--cloudflare to expose a public HTTPS link for a wildcard bind; --secure
still implies the tunnel.
- --secure + --no-cloudflare is still rejected as a contradiction.
- Update the parent-command guard, re-exec forwarding, startup-banner
wording, the colab comment, README, and tests.
* Studio: update installer/setup launch hints for opt-in Cloudflare
The post-install launch hints only mentioned --secure for a public link.
Now that the tunnel is opt-in, clarify that -H 0.0.0.0 exposes the raw
port on the LAN (not a public URL), and surface --cloudflare as the
explicit opt-in for a public HTTPS link (--secure keeps the raw port
private). Applied to install.ps1, install.sh, and studio/setup.sh.
* Studio: address review - keep cloudflare tri-state + harden run re-exec
Two review points from the bots:
- Gemini: keep `cloudflare` as Optional[bool] in run_server instead of
casting None -> False, so the startup banner can distinguish "OFF (default)"
(unset) from "OFF (--no-cloudflare)" (explicit). `_cloudflare_flag` and the
banner branch now carry the tri-state.
- Codex (P1): `unsloth studio run` re-execs the studio venv's console script,
which can be an older build whose --cloudflare defaulted on; omitting the
flag let it re-enable the tunnel. That path now forwards the default polarity
explicitly (--no-cloudflare, or nothing under --secure since --secure implies
the tunnel). The plain `unsloth studio` path runs the same-version in-tree
run.py (resolved via _find_run_py), so it keeps forwarding only an explicit
polarity and still shows the accurate "(default)" banner.
Tests updated for the tri-state banner labels, the None gate cases, and the
new re-exec forwarding.
* Studio: forward --no-cloudflare on plain re-exec too (mixed install)
Codex follow-up: _find_run_py falls back to STUDIO_HOME/.../studio/backend/
run.py when the package copy is absent, so the plain `unsloth studio` re-exec
can land on an older run.py whose --cloudflare defaults on. Forward the default
polarity explicitly there too (--no-cloudflare, or nothing under --secure),
matching the run subcommand. The common in-venv launch skips the re-exec and
still shows the tri-state "(default)" banner.
* Studio: fix launch hint - --cloudflare needs the wildcard bind
Codex P3: the launch hint listed --cloudflare next to the loopback
`unsloth studio -p 8888` command, but the tunnel only starts for wildcard
binds, so `--cloudflare` alone on 127.0.0.1 does nothing. Show
`-H 0.0.0.0 --cloudflare` in the hints (install.ps1, install.sh,
studio/setup.sh) and clarify the same in the README.
* Studio: cross-platform masked terminal password prompt helper
Per-keystroke '*' echo (POSIX termios cbreak / Windows msvcrt.getwch),
backspace editing, Ctrl-C abort, EOF handling, confirmation loop with
re-prompt on mismatch or policy failure. Pure should_prompt gate for the
--secure/--cloudflare exposure paths.
* Studio CLI: force a terminal password change before public tunnel exposure
When a launch will start the Cloudflare tunnel (--secure, or --cloudflare on
a non-api-only wildcard bind) and the admin account still has its seeded
bootstrap password, prompt for a new password in the terminal (masked with
'*', confirmed, re-prompting until valid) before any re-exec or server
exists. The change is committed in the parent so it never crosses argv or
the environment and older studio-venv children see it immediately. Without
a terminal, warn and fall back to the backend bootstrap shutdown timer.
Mirrors backend update_password semantics in one transaction: rehash,
rotate the JWT secret, clear must_change_password, revoke refresh tokens,
drop the desktop secret, then remove the stale credential files.
* Studio: terminal password gate before the public tunnel (backend backstop)
Never publish a trycloudflare URL while the seeded admin password is
active: run_server now runs a terminal password-change gate after the
tunnel decision and strictly before start_studio_tunnel. Interactive
refusal fails closed (shutdown + exit 1, mirroring the secure gate);
without a tty it warns and keeps the bootstrap deadline. Success applies
the same effects as the change-password route (update_password +
revoke_user_refresh_tokens) and drops the stale
app.state.bootstrap_password. MIN_PASSWORD_LENGTH centralised in
auth/storage.py and referenced by the HTTP schema. terminal_prompt.py
carries the pure gate helper (interactive loop stubbed; supplied by the
masked-input module). Also migrates the studio/setup.ps1 launch footer
that still showed the bare wildcard hint.
* README: reconcile remote-access section with opt-in Cloudflare tunnel
* Studio: harden the terminal password gate after review
- run.py: run the gate BEFORE the uvicorn socket binds. On a wildcard
--cloudflare launch the served HTML injects the bootstrap credential
for first login, so a pre-gate listener would hand the default
password to anyone who reaches the raw port while the operator is
still typing. The gate now also seeds the admin row itself (it can
run before lifespan startup).
- Headless launches that nothing would protect now fail closed: the
bootstrap deadline never arms for api-only serving and
UNSLOTH_STUDIO_BOOTSTRAP_TIMEOUT=0 disables it, so warn-and-proceed
would have promised a shutdown that never comes. Both the CLI and the
backend refuse to publish in that case; the ordinary headless path
still warns and relies on the 1h deadline, and no longer auto-fills
the default credential into HTML served on a public URL.
- storage.update_password gains revoke_refresh_tokens to delete the
user's refresh tokens in the SAME transaction as the password commit;
the change-password route and the backend gate use it (a separable
follow-up delete could fail after the commit and leave a stale
refresh token able to mint access tokens under the rotated secret).
- clear_bootstrap_password is best-effort: a locked/undeletable file
must not surface as a failed password change.
- CLI masked reader: disable ISIG like the backend so Ctrl-Z cannot
suspend the process with the shared terminal stuck in no-echo mode;
handle Ctrl-C/Ctrl-Z as characters; treat stream EOF mid-line as an
abort instead of submitting a partial password. Both readers restore
terminal attrs from a SIGTERM/SIGHUP handler since a finally block
cannot run when a default-disposition signal terminates the process.
- Backend reader: decode byte-at-a-time through an incremental UTF-8
decoder so multi-byte characters split across read boundaries are no
longer dropped; isatty checks tolerate closed/None streams.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: persist bootstrap suppression through lifespan startup
The pre-bind password gate nulled app.state.bootstrap_password, but the
FastAPI lifespan runs after it and re-reads the bootstrap password into
app.state on both admin paths, so a headless public launch could still
serve the injected credential in HTML. Carry a persistent
suppress_bootstrap_injection flag that the lifespan honors instead.
Also drop the quoted Tuple annotation on _terminal_password_gate that
tripped the import-hoist lint (the typing import looked unused).
* Studio CLI: keep the pre-exec auth DB private (0700 dir, 0600 db)
On a fresh install the pre-exposure password gate creates auth/ and
auth.db through the CLI before the backend ever runs, and
sqlite3.connect leaves the DB 0644 under a 022 umask. Mirror backend
storage.get_connection's chmod so the committed password hash and JWT
secret are never world-readable, even if the launch aborts before the
backend applies its own modes.
* Tighten pre-exposure password gate comments
* Studio: delete seeded bootstrap password before headless public re-exec
The headless warn-and-proceed path returns with the default admin
password still active, then re-execs a child Studio process. An old
studio-venv child (mixed-version install) predates the pre-bind gate and
its injection-suppress flag, so its lifespan reads .bootstrap_password
and injects the seeded credential into the public HTML for up to the
bootstrap deadline. A CLI-flag handshake cannot fix this uniformly: the
studio run path uses ignore_unknown_options and an old in-venv child
runs in-process, so it would never reject the flag.
Delete the seeded .bootstrap_password file in the parent before re-exec
so a fresh child of any version reads None and never serves it. This
covers both re-exec paths and both child versions. must_change_password
stays set, so the login page still forces a change and the bootstrap
shutdown timer still arms; only the plaintext-on-disk copy is removed.
Recovery is via a terminal-attached run or reset-password. Backend gate
and CLI warnings updated to match.
* Studio: commit the seeded admin before headless public re-exec
The headless-warn path deletes the seeded .bootstrap_password so a
re-exec'd child cannot inject it, but _ensure_cli_default_admin's INSERT
was never committed and rolled back on conn.close(). On a fresh
STUDIO_HOME an old studio-venv child then found no admin, regenerated a
fresh bootstrap password + file, and injected THAT into the public page,
defeating the deletion.
Commit the seeded admin right after _ensure_cli_default_admin so any
re-exec'd child sees the existing account and does not regenerate.
Regression tests cover both re-exec paths on a fresh (unseeded) DB.
* Studio: fail closed when the bootstrap password file cannot be removed
On the headless public path, deleting .bootstrap_password is the
protection against an old re-exec'd child injecting the seeded
credential. If unlink fails (locked file, read-only auth dir) the file
is still on disk, so warning and proceeding would still leak it for the
bootstrap-timeout window. Abort with a clear error instead. Regression
test covers the unlink-failure fail-closed path.
* Studio: hold no-echo for the whole password line, not per keystroke
The POSIX masked reader set cbreak/no-echo inside _getch_posix and restored
the terminal to echo-on in a finally after every single keystroke, because
_read_password calls _getch once per character. Between one char returning and
the next call re-entering cbreak, ECHO was on, so a keystroke arriving in that
window echoed the password in cleartext.
Move the terminal mode into a _prompt_raw_mode context that _read_password
holds around the entire line (mirroring unsloth_cli/commands/_password_prompt.py,
which already did this), restoring once when the line completes or aborts.
_getch_posix now only reads, since the mode is held by the caller. The context
is a no-op when stdin is not a real terminal, keeping the _getch test seam.
Add a regression test asserting the raw-mode context wraps the read exactly
once and every keystroke is read while it is active.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: strip the seeded bootstrap password when the auth DB check fails
The pre-exposure gate returned early on two auth-DB inspection failures and
proceeded to re-exec without removing the seeded .bootstrap_password:
- _connect_auth_db() failure: a seeded credential from a prior run may still
be on disk.
- the must_change_password read-back failure: worse, _ensure_cli_default_admin
had already seeded the admin and the code committed it (writing
.bootstrap_password) right before the failing SELECT.
In the mixed-version case (a new outer CLI re-execing an old studio-venv child
that predates the pre-bind gate), that child would read the file back and
inject the default admin credential into the public Cloudflare page. The
sibling headless branch already deletes the file for exactly this reason, so
these returns were an inconsistent gap.
Factor the delete-or-fail-closed logic into
_strip_seeded_bootstrap_password_or_exit and call it on both inspection
failures (and reuse it in the headless branch): strip the seeded file first
(version-independent protection), failing closed if the removal itself fails.
must_change_password stays set, so the login page still forces a change and the
bootstrap shutdown timer still arms.
Add tests for both new paths (connect failure and post-commit read-back
failure strip the file and proceed; a failed strip fails closed).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: fail closed when the seeded admin cannot be committed before exposure
The pre-exposure gate wrapped _ensure_cli_default_admin (the INSERT), its
conn.commit(), and the must_change_password read-back in one try, and the
except recovered by stripping .bootstrap_password and proceeding to re-exec on
the assumption the admin was already committed. That assumption only holds when
the failing statement is the SELECT. When the INSERT or the commit itself fails
(e.g. a write lock held past the busy timeout on a fresh install), no admin row
is committed: it rolls back on conn.close(), and a re-exec'd old studio-venv
child (no pre-bind gate) then finds no admin, regenerates a fresh bootstrap
password + file, and serves that default credential on the public Cloudflare
page. Stripping the file cannot stop a regeneration.
Split the seed+commit into its own try that fails closed (refuse the public
launch, best-effort removing any half-written seed file) since we cannot prove a
committed admin; keep the separate read-back failure on the strip-and-proceed
path, where the admin is committed so an old child finds it and will not
regenerate. Add a test for the seed-commit-failure path.
* Studio: decode the CLI masked password reader with errors="replace"
The CLI reader read keystrokes with text-mode sys.stdin.read(1), which raises
UnicodeDecodeError on a pasted non-UTF-8 password (e.g. Latin-1 bytes), or under
PYTHONUTF8 yields a lone surrogate that later crashes the pbkdf2 encode -- either
aborts the launch with a traceback. The backend mirror (terminal_prompt.py)
already reads raw bytes through an incremental decoder with errors="replace".
Mirror that here: read with os.read and an incremental decoder so invalid bytes
map to U+FFFD, iterating over each emitted char (one byte can complete a
replacement plus the next char).
* Studio: resolve the child launcher before the pre-exposure gate
The gate strips the seeded .bootstrap_password on a headless public launch, and
it ran before the re-exec launchability check (studio venv / run.py / console
script present). So a headless launch with an incomplete studio setup would seed
the admin, delete the bootstrap password, then abort because the child could not
be found, leaving the admin at must_change_password=1 with no password ever
shown or injectable: locked out until `unsloth studio reset-password`.
Resolve and validate the child launcher first, in both `studio` (studio_default)
and `studio run`, and only then run the gate, so an unlaunchable setup exits
before anything is stripped. Add a regression test that a missing venv exits
without removing the seeded file.
* Studio: fail closed when the auth DB cannot be opened before exposure
The connect-failure branch of the pre-exposure gate stripped .bootstrap_password
and proceeded, on the assumption a committed admin from a prior run made an old
child find it and not regenerate. But on a fresh public launch whose
_connect_auth_db() itself fails (transient lock during the schema/seed step, or
an unwritable home), no admin is committed, so a mixed-version re-exec child that
predates the backend gate can find no user, generate a fresh bootstrap password,
and serve it on the public Cloudflare page. Stripping a file we cannot vouch for
cannot stop a regeneration.
Make this branch fail closed like the seed/commit failure path: we only continue
past the DB inspection once a committed admin is confirmed. The existing file is
left untouched so a retry (after a transient lock clears) can still prompt.
Update the connect-failure test to assert fail-closed, and give the in-venv
--secure flag test a real STUDIO_HOME with an already-changed admin so the gate
is a no-op rather than relying on a DB-open failure.
* Studio: invalidate seeded bootstrap files before deleting auth.db on reset
reset-password deleted auth.db first, then best-effort unlinked the seeded
.bootstrap_password and desktop secret. unlink() only ignores
FileNotFoundError, so a locked or read-only file (Windows AV, read-only auth
dir) survived while auth.db was gone. The next server start then re-seeded
from that stale plaintext and re-validated the exact credential the reset was
meant to revoke.
Invalidate the credential files first, truncating any that cannot be
unlinked, then delete the DB, so a surviving file can never carry a reusable
secret. clear_bootstrap_password now truncates on unlink failure for the same
reason, and its warning says the contents were cleared rather than claiming
the stale password is already invalid.
* Studio: require a servable frontend before the pre-exposure gate can strip the seeded password
A headless public launch strips the seeded .bootstrap_password before the
re-exec'd child starts. If the child then cannot serve the login page (the only
in-band way to change the seeded password) the admin is locked out
(must_change_password=1, no file, no UI) until reset-password.
Add _require_servable_frontend_or_exit and call it before the gate on both
`unsloth studio` and `unsloth studio run` public launches: fail closed if a
non-api-only public launch has no built frontend dist, before anything is
stripped. A user-supplied --frontend is validated to contain index.html so a
bad path cannot silently bypass the check; an auto-resolved dist is trusted
(_find_frontend_dist already requires index.html) and forwarded to the child.
Model-load aborts on `studio run` remain a residual: the parent must strip for
mixed-version safety (an old studio-venv child has no pre-bind gate) and model
loadability cannot be proven before exec, so that path stays recoverable via
reset-password.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: harden reset-password ordering and validate the in-venv backend before the strip
Three follow-ups to the pre-exposure hardening:
reset-password now deletes auth.db FIRST and proves it is gone before touching
the seeded credential files. If the DB cannot be removed (a running Studio or
Windows holds it open, or a read-only auth dir) it aborts with the credential
files untouched, so a forgotten-password reset is not left half-done with the
recovery credentials deleted while an un-resettable must_change_password=1 DB
survives. After the DB is gone it invalidates the stale credential files
(unlink, else truncate) and fails closed if a file can be neither removed nor
truncated, since a surviving plaintext would be re-seeded by
generate_bootstrap_password() and re-validate the revoked password.
The in-venv (in-process) launch path had no analogue of the re-exec launcher
check: a headless public launch would seed the admin and strip the seeded
.bootstrap_password in the gate before _load_run_module() later failed on a
broken/partial venv, leaving must_change_password=1 with no password to log in.
Add _validate_inproc_backend_before_strip, called on the in-venv path (both
`unsloth studio` and `unsloth studio run`) before the gate on the headless
public path, so a broken backend fails cleanly before anything is stripped. It
is scoped to the headless path so an interactive prompt is not delayed behind a
full backend import.
* Studio: validate the frontend and tunnel before the strip on every public path
Five follow-ups closing the remaining pre-exposure-strip lockouts:
The in-venv (in-process) paths of both `unsloth studio` and `unsloth studio
run` validated the backend but not the frontend before the gate, so a headless
public launch with a missing/bad dist would strip the seeded .bootstrap_password
and then abort in run_server() during frontend setup, leaving
must_change_password=1 with no login page. Both now validate a servable frontend
before the strip (cheap check first, backend import after) and serve the
resolved dist in-process.
The `studio run` re-exec discarded the dist that satisfied the pre-strip check
and only forwarded a user-supplied --frontend. In a shadowed install where the
parent finds a built dist the child cannot, it stripped and exec'd without the
path, and the child aborted during frontend setup. It now forwards the resolved
dist, matching `unsloth studio`.
On a headless --secure launch the bind is loopback, so the Cloudflare tunnel is
the only public exposure. If cloudflared is provably unavailable (found nowhere
and undownloadable) the tunnel cannot start, so stripping the recovery
credential would just lock the user out with no public URL ever served. Add
_tunnel_binary_confirmed_unavailable and, on --secure only, refuse the launch
with the credential preserved rather than strip. Wildcard --cloudflare binds
0.0.0.0 publicly regardless of the tunnel, so it still strips; any uncertainty
(helper not loadable) also still strips, since a possible credential leak
outweighs a recoverable lockout.
clear_bootstrap_password no longer claims it cleared the file's contents when
both unlink and truncate failed; it now reports the stale password is still on
disk and asks the user to remove it manually.
* Studio: fix cloudflared probe path and skip the bootstrap strip for a self-suppressing child
Two follow-ups to the --secure pre-exposure hardening:
The cloudflared availability probe loaded cloudflare_tunnel by file path but not
its backend deps: ensure_cloudflared() -> _cache_path() lazily imports
utils.paths.storage_roots, which only resolves when studio/backend is on
sys.path. From the outer CLI it is not, so the probe saw ensure_cloudflared()
return None (cache unresolvable) and wrongly treated the tunnel as unavailable,
refusing --secure even when cloudflared was cached or downloadable. Add the
backend dir to sys.path for the probe (and remove it after) so the cache path
resolves as it will in the child.
A headless --secure launch stripped the seeded .bootstrap_password before the
child proved the tunnel could actually connect, so a cloudflared that is present
but cannot establish the tunnel (blocked connectivity, Cloudflare outage) left
must_change_password=1 with no recovery credential. But the strip is only needed
when the re-exec'd child is an OLD studio-venv backend with no pre-bind
suppression: this install's own run.py sets app.state.suppress_bootstrap_injection
before binding and never serves the seeded credential publicly. Add
_child_self_suppresses (true in-process, or when the re-exec target is this
install's own run.py by path identity) and skip the strip in that case, keeping
.bootstrap_password as a local recovery credential; the strip stays fully in
force for the studio-venv console-script path and any venv-fallback run.py, where
an old child is actually possible.
* Studio: reword the pre-exposure terminal password prompt
* Studio: warn when -H is overridden by --secure; align pre-exposure prompt wording
- --secure/--secure run: emit a Note (not an error) when -H is a non-loopback
host, since --secure forces the loopback bind and would otherwise discard -H
silently.
- Reword the pre-exposure terminal prompt to 'exposed on the public internet'
in both the backend gate and the CLI mirror.
- Align the CLI success line with the backend ("Password updated for '<user>'.").
- Tests for the new -H warning (present when overridden, absent on loopback).
* Studio: add non-interactive --password to set the initial admin password
Headless hosts (CI, containers, systemd units) have no TTY, so the forced
first-exposure password change could not be completed unattended. Add a
non-interactive way to set the INITIAL admin password before the server binds:
- --password <value>, the UNSLOTH_STUDIO_PASSWORD env var, or --password -
(read one line from stdin). Off by default; unset falls back to the normal
interactive terminal prompt / browser setup.
- Applies on any launch (public --secure/--cloudflare or a headless -H 0.0.0.0
bind), only when the account still has its seeded bootstrap password. An
already-set password is a hard error, never an override; an invalid value
(too short, or equal to the bootstrap) fails closed before bind.
- The CLI applies the change in the parent, never forwards --password to the
re-exec child, and strips UNSLOTH_STUDIO_PASSWORD from the child env so the
secret never crosses to the child. run.py does the same on the direct path and
strips the env var so spawned subprocesses (cloudflared, llama-server, tools)
cannot inherit it.
Mirrors resolve_supplied_password across the CLI and backend, documents the
option in the README (including the argv-visibility caveat), and covers all
flows (env/stdin/literal, fail-closed cases, no-forward, env-strip,
reset-password roundtrip) in the CLI, backend, and unit suites.
* Studio: truncate the stale bootstrap file when unlink fails on a CLI password change
The post-change cleanup in _cli_update_password only warned when
.bootstrap_password could not be unlinked but was still writable (locked file,
read-only auth dir), leaving the old plaintext on disk. If auth.db is later
reset or removed, generate_bootstrap_password() reads that file back and
re-validates the revoked bootstrap password. Truncate the file on unlink
failure so its stale plaintext cannot be re-seeded, mirroring the backend
clear_bootstrap_password(); the password change is already committed, so this
never rolls it back. The warning now states truthfully whether the contents
were cleared or the file must be removed manually.
* Studio: tighten comments
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: permission levels for chat tool calls (Ask, Approve for me, Off, Full access)
Replace the Bypass permissions on/off toggle with a four level permission
selector, available in Settings > General (new Permissions section above
Notifications), the chat settings panel, the composer plus menu, and a new
always visible composer pill.
Levels:
- Ask for approval: every local tool call pauses for allow/deny.
- Approve for me: only calls detected as potentially unsafe pause; the
python/terminal sandbox stays on.
- Off: never pauses; sandbox stays on (previous default behavior).
- Full access: never pauses and the sandbox is disabled. Still requires
the danger confirmation and is never restored across reloads.
Backend adds permission_mode to the OpenAI compatible and Anthropic
passthrough payloads and threads it through both tool loops. Auto mode
uses a fail closed classifier in tools.py: terminal commands must be on
a read only allowlist with no redirection or substitution, python code
is AST scanned for writes, exec, process and network use, MCP tools
auto run only with read only style names. Unknown tools always ask.
Legacy bypass_permissions and confirm_tool_calls keep their exact
behavior for existing API callers.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio permissions: Off is a plain toggle below Full access
Off moves to the bottom of the level menu with a short description and
acts as the feature-off state: the composer pill is hidden entirely
while Off, and reselecting the active level toggles back to Off.
* Studio permissions: higher contrast composer pill text
The permission pill uses a foreground based grey instead of the shared
muted pill color, so it reads darker in light mode and lighter in dark
mode. Full access keeps the danger yellow.
* Studio permissions: panel dropdown layout and shorter tooltip
Chat settings panel: the Bypass permissions label sits on one line with
a full width dropdown underneath, styled like the other panel selects.
Tooltip shortened and wording uses Unsloth instead of Studio.
* Studio permissions: harden auto-mode unsafe detection
Extend the Approve for me classifier to catch write and exec paths that
slipped through:
- terminal: sort -o, tree -o, xxd -r, find -exec/-execdir/-ok/-delete
and find -fprint/-fprintf/-fls now ask; plain read-only forms still
auto-run. awk is no longer allowlisted since its program can write and
call system().
- python: from-imports of mutating names (from os import remove [as rm])
and star imports now ask.
Found by a fuzz and edge-case simulation matrix; pinned in
test_permission_mode.py.
* Studio permissions: split multi-line terminal commands in auto detection
A shell runs each line as its own command, but shlex reads newlines as
whitespace, so "ls\nrm -rf x" demoted rm to argument position and
auto-ran. Normalize newlines and CR to separators, and treat any all
separator token as a command boundary so runs of blank lines still
split. Found by the simulation matrix; pinned in tests.
* Studio permissions: address review feedback on auto-mode detection
Auto-mode (Approve for me) safety classifier hardening:
- Python: flag any reference to a mutating attribute, not only direct
calls, so indirect refs (f = os.remove; f(x)) and aliases ask. Detect
Path.open(mode) write modes and wrap the AST walk to fail closed.
- Terminal: match attached short output flags (sort -o/tmp/out) and keep
find context across grouping parens so find ( -delete ) asks.
- Both: ask before reads that escape the sandbox workdir via parent
traversal or hit credential paths (.ssh, .aws, id_rsa, .pem, etc.).
permission_mode plumbing:
- Fold permission_mode=full into bypass_permissions at the request model
so route-level confirm-gate guards see it as bypass.
- Reject ask/auto on the Anthropic Messages server-tools path, which has
no confirmation channel (mirrors the confirm_tool_calls rejection).
- Keep forced RAG autoinject in auto mode: the safe search_knowledge_base
retrieval never gates, so derive the skip from the real confirm need.
- Reset all local preferences now also clears the legacy confirm key so a
reset restores the fresh default instead of the old level.
Regression tests added for each case.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio permissions: close auto-mode classifier gaps from review round 2
Auto mode ("Approve for me") let a few mutating calls through as safe:
- os.open(...) always creates/writes a descriptor, so treat it as unsafe
even though builtin open in read mode stays safe.
- fd -x/--exec/-X/--exec-batch runs a command per match; scan for these
alongside find's -exec/-delete.
- tempfile writes artefacts and hands back writable handles, so importing
it now asks.
- Calling the result of a call (getattr(os, "remove")("x"), partials) is a
dynamic target the AST can't vet, so fail closed.
- An MCP tool whose name pairs a read verb with a mutating one
(get_or_create_issue, read_and_delete_file) no longer auto-runs on the
read prefix alone.
Also fold permission_mode="off" into confirm_tool_calls=False on both
request models so the non-stream route guard sees the disabled gate, and
drive the Confirm tool calls toggle off permission_mode="ask" so auto no
longer shows it on.
* Harden auto-mode classifier and normalize bypass to full for PR #7079
Approve for me now asks for a few cases it previously auto-ran:
- os.open via an os alias (import os as o; o.open(path, O_CREAT))
- pathlib symlink_to / hardlink_to / link_to
- importlib.import_module dynamic imports
- os.mkfifo / os.mknod / os.utime
Also fold bypass_permissions into full when a stale ask/auto permission_mode
is sent alongside it, so the Anthropic route guard no longer 400s those legacy
callers. Adds classifier and request-model regression tests.
* Close more auto-mode classifier gaps for PR #7079
Approve for me now asks for cases the review surfaced:
- builtin open aliased to a name (f = open; from builtins import open as w)
or looked up dynamically (globals()['open'])
- pickle / marshal / shelve / dill deserialization
- io.FileIO write handles
- sort --compress-program (runs an external program)
- MCP names carrying save/archive/submit/commit/push/sync/register verbs
Also refine the attribute open() write check so an explicit read mode
(ZipFile.open(name, "r")) stays auto while os.open flags still ask. Adds
test coverage for each case.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Close three more auto-mode gaps for PR #7079
- rg runs an arbitrary program per file via --pre / --hostname-bin, so
"Approve for me" now asks for those flags (rg is on the read-only
allowlist).
- A path-qualified command token (./ls, /tmp/cat) is an arbitrary
executable, not the trusted utility its basename matches, so it asks
before running.
- A direct /chat/completions caller that sets permission_mode ask/auto
but omits the legacy confirm_tool_calls flag now self-enables the
confirmation gate, so tools can no longer run ungated on that path.
Adds classifier and request-model tests for each case.
* Close auto-mode classifier gaps from review round 3 for PR #7079
Approve for me now asks for cases the latest pass surfaced:
- short-option clusters bundling a write flag (sort -uo out => -u -o)
- procfs reads that leak a process env/args/memory
(cat /proc/self/environ, /proc/PID/cmdline, maps)
- env-assignment prefixes that change command lookup/loading
(LD_PRELOAD=x ls, PATH=. ls, IFS=x ls); benign FOO=1 cmd stays auto
- os.open imported as a bare callable (from os import open as o)
Also drops ps from the safe terminal allowlist: its BSD environment
flags (ps auxe, ps eww) dump a parent process's unscrubbed env and
cannot be flag-parsed reliably, so ps always asks now. Adds classifier
tests for each case.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Close auto-mode classifier gaps from review round 4 for PR #7079
Terminal (Approve for me now asks for these):
- cd dropped from the safe allowlist: cd /; cat etc/passwd moves the
shell out of the session workdir so a later relative read escapes it
- env -C/--chdir (workdir escape) and -S/--split-string (builds a fresh
command line); wrapper flags are now checked
- /etc//passwd and /etc/./passwd normalize to /etc/passwd before the
sensitive-path scan
- a sensitive path split across an assignment and an argument
(p=/etc; cat $p/passwd) via best-effort NAME=value expansion
Python:
- builtins.exec / builtins.eval attribute calls (dynamic code execution)
- destructured open aliases (f, _ = (open, print); f('out', 'w'))
- a sensitive path composed from literals (os.path.join('/etc','passwd'),
'/etc' + '/passwd')
- ZipFile/TarFile write modes (ZipFile(name, 'w')); the reader stays auto
Adds classifier tests for each case.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Close auto-mode classifier gaps from review round 5 for PR #7079
Terminal (Approve for me now asks for these):
- procfs reads hidden by shell quotes (cat /proc/$PPID/enviro''n) or
quoted/nested-variable assignments (p="/proc/$PPID"; cat $p/environ):
quotes are stripped and NAME=value prefixes expanded before the scan
- LESSOPEN/LESSCLOSE, which make less run an input preprocessor command
Python:
- os.chdir / os.fchdir, which move the cwd so a later relative read
escapes the sandbox workdir
- sensitive paths composed via a pathlib / chain (Path('/etc') / 'passwd')
or an f-string of literals (f'/proc/{pid}/environ')
- runpy (import) and runpy.run_path / run_module, which run arbitrary code
Adds classifier tests for each case.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Close auto-mode classifier gaps from review round 6 for PR #7079
Approve for me now asks for these:
- a mutating callable reached through a getattr alias
(rm = getattr(os, "remove"); rm("f")): calls through a getattr-bound
name fail closed
- compound MCP tool names carrying clone/checkout/comment/fork/tag/
invite/share, which start with a read verb but still mutate
- a sensitive path hidden behind a glob (cat /e??/passwd,
cat /e[t]c/passwd): a ? / * / [..] token is matched against the
sensitive-file set and bracket classes are de-obfuscated; benign
globs (ls *.py) stay auto
Also run first-pass RAG retrieval in off mode: like auto, off never
prompts, so a direct caller passing a stale confirm flag should not lose
document retrieval (both tool loops).
Adds classifier tests for each case.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Close auto-mode classifier gaps from review round 7 for PR #7079
Approve for me now asks for these:
- __builtins__.exec / __builtins__.eval (dynamic code via the dunder)
- terminal reads that hide a credential path behind a backslash escape
(cat /et\c/passwd)
- read-named MCP filesystem calls pointed at a credential path
(mcp__fs__read_file {"path": "/etc/passwd"})
- compound MCP names carrying append / prepend
- open aliased through a subscript or builtins attribute
(f = globals()["open"]; f = builtins.open) then called to write
- open(..., **{"mode": "w"}) where a kwargs splat hides the write mode
- a sensitive path with a dynamic segment (open(f"/etc/{name}"),
os.path.join("/etc", name)); /tmp/{name} stays auto
- urllib3 networking
Also stop folding permission_mode ask/auto into confirm_tool_calls for
external-provider requests: that branch rejects confirm_tool_calls with
tools, and the mode only governs local tool calls. Local requests still
self-gate. Adds tests for each case.
* Close auto-mode classifier gaps from review round 8 for PR #7079
Approve for me now asks for these:
- dbm on the unsafe-module list: dbm.open(file, "c"/"n") creates files,
and importing the family signals a persistence writer
- reads of ~/.azure and ~/.config/gh credential stores (Azure/GitHub
tokens), in terminal, MCP arguments, and Python literals
- compound MCP names carrying upsert / assign
Adds classifier tests for each case.
* Gate secret mounts and fix the composer pill count for PR #7079
- Add Docker/Kubernetes secret mount dirs (/run/secrets,
/var/run/secrets) to the sensitive-path checks, so Approve for me asks
before reading injected credentials (terminal, MCP args, Python).
- Count the always-visible permission pill in the composer's compact
threshold so labels collapse at the intended width instead of
overflowing by one pill.
Adds classifier tests for the secret mount paths.
* Close auto-mode classifier gaps from review round 10 for PR #7079
Approve for me now asks for these:
- qualified pathlib constructors (pathlib.Path('/etc') / name), folded
the same as bare Path(...), so a dynamic sensitive path is detected
- open aliased through an annotated assignment (f: object = open;
f('out', 'w')), tracked like a plain assignment
- recursive searches rooted at an absolute path (grep -R TOKEN /home,
rg TOKEN /, fd pattern /etc), which read host files outside the
sandbox tree; sandbox-relative searches stay auto
Adds classifier tests for each case.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Close auto-mode classifier gaps from review round 11 for PR #7079
Approve for me now asks for these terminal reads, which bash would
expand into a sensitive path only after the classifier had approved:
- a glob that resolves into a secret mount or credential dir
(cat /r?n/secrets/hf_token, cat /root/.s??/id_rsa)
- a recursive search rooted at a tilde home (grep -R TOKEN ~root,
grep -R TOKEN ~/logs)
- a brace expansion that builds a credential path (cat /etc/pass{w,}d)
- a default/alternate parameter expansion that builds one
(cat /etc/pass${x:-wd})
- an input redirection that hides a glob (cat </e??/passwd)
And these python calls:
- a str.format-built sensitive path (open('/etc/{}'.format('passwd')))
- writer methods that persist to disk without open() (numpy.save,
Image.save, plt.savefig, DataFrame.to_csv, json.dump)
Segment-wise directory matching keeps benign globs (ls /home/*/projects)
auto. Adds regression tests for each case and its safe counterpart.
* Close auto-mode classifier gaps from review round 12 for PR #7079
Approve for me now asks for these too:
- a terminal read whose parent traversal hides behind a redirection with
no following space (cat <../../notes)
- a python read whose path is built with str.join
(open(''.join(['/etc', '/passwd']))), told apart from os.path.join
- a dynamic-code builtin reached through an alias
(from builtins import eval as e; e(...); x = builtins.exec; x(...))
Adds regression tests for each case and its safe counterpart.
* Close auto-mode classifier gaps from review round 13 for PR #7079
Approve for me now asks for these too:
- a recursive search whose root is hidden behind an assignment
(p=/; grep -R TOKEN $p): the recursive-root test now runs on the
assignment-expanded tokens as well
- a python read whose sensitive path is split through a literal variable
(base = '/etc'; open(base + '/passwd')), including via an f-string
- numpy ndarray.tofile, which persists without open()
- a sequence brace read (cat /etc/pass{w..w}d), expanded alongside the
comma brace form before the sensitive-path scan
Adds regression tests for each case and its safe counterpart.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Close auto-mode classifier gaps from review round 14 for PR #7079
Approve for me now asks for these python reads that assemble a sensitive
path in a form the fold did not yet recognize:
- a pathlib object reused through a name (p = Path('/etc'); p / 'passwd')
- old-style percent formatting ('%s/%s' % ('/etc', 'passwd'))
- Path.joinpath ('/etc'.joinpath('passwd'))
- a bytes path literal (open(b'/etc/passwd'))
And these terminal reads, which bash expands into a sensitive path only
after the classifier had approved:
- a substring parameter expansion off an assignment
(p=passwd; cat /etc/${p:0:6})
- an ANSI-C quoted path (cat $'/etc/pass\x77d')
- a glob into an Azure or GitHub CLI config dir
(cat /home/*/.az?re/..., cat /home/*/.config/g?/...)
Adds regression tests for each case and its safe counterpart.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Close auto-mode classifier gaps from review round 15 for PR #7079
Approve for me now asks for these terminal reads, which bash expands into
a sensitive path only after the classifier had approved:
- a per-thread procfs env alias (cat /proc/$PPID/task/$PPID/environ)
- a recursive root behind a default parameter (grep -R TOKEN ${root:-/home})
- a path built by pattern replacement (p=passXd; cat /etc/${p/X/w})
And these python reads:
- a pathlib .parent/.parents chain that escapes the session workdir
((Path.cwd().parent / 'other' / 'notes').read_text())
- a sensitive path resolved through glob (glob.glob('/e??/passwd')[0])
Adds regression tests for each case and its safe counterpart.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Close auto-mode classifier gaps from review round 16 for PR #7079
Approve for me now asks for these terminal reads, which bash expands into
a sensitive path only after the classifier had approved:
- a case-modifying parameter expansion (p=PASSWD; cat /etc/${p,,})
- a mutating find action hidden behind an assignment (f=-delete; find . $f)
- a glob assembled through an assignment (g=e??; cat /$g/passwd)
- a POSIX bracket class glob (cat /etc/pass[[:lower:]]d)
And these python reads/writes:
- a glob pattern folded from a literal variable
(base='/e??'; glob.glob(base + '/passwd'))
- a directly imported os.path.join (from os.path import join; join('/etc', 'passwd'))
- a directly imported writer (from numpy import save; save(...))
- an aliased pathlib constructor (from pathlib import Path as P; P('/etc') / 'passwd')
The find/fd and glob scans now run on the assignment/parameter-expanded
command, and pathlib/join/writer import aliases are tracked. Adds
regression tests for each case and its safe counterpart.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Close auto-mode gaps from review round 17 for PR #7079
Two fixes:
- Gate sqlite3 in auto mode. sqlite3.connect(path) creates or mutates a
database file (and runs DDL/DML) with no open()/writer attribute for
the AST checks to catch, so treat the module like dbm and ask.
- Only self-enable confirm_tool_calls for Studio's own tool loop. The
ask/auto fold previously set confirm on every non-provider request,
including a plain client-tool passthrough (client-supplied tools that
Studio does not execute), which then tripped the local-tool
streaming-confirm route guard and rejected the passthrough. Restrict
the fold to requests that actually ask Studio to run tools
(enable_tools / enabled_tools / mcp_enabled).
Adds regression tests for the sqlite3 write and for the passthrough vs
tool-loop confirm behavior.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Close auto-mode gaps from review round 18 for PR #7079
Classifier (auto mode asks for these):
- os.open through a module alias (import os as o; o.open(...)); os/posix
aliases are tracked like the literal module name.
- less/more pagers, whose escapes (+cmd, !shell, -o/--log-file, LESSOPEN)
can run a command or write a file the command-name allowlist cannot
see, so they are no longer auto-approved.
- a read-named MCP tool carrying a mutating query
(query_database {"query": "DELETE FROM runs"}); DML/DDL statements are
matched as whole statements so a natural-language query that merely
contains "delete" stays safe.
- ML persistence helpers (save_pretrained / save_file / save_model /
save_weights / save_lora / save_checkpoint) that export weights to disk.
Route:
- Honor CLI-forced tools when deriving the confirm gate. When a process
policy (unsloth run --enable-tools) opens the local tool loop without a
request-level tool signal, a permission_mode ask/auto request now
derives confirm at the route (GGUF and safetensors paths) so the mode
still gates the call, and a non-streaming ask/auto request is rejected
rather than running unprompted. A plain client-tool passthrough (no
local loop) is unaffected.
Adds regression tests for each case and its safe counterpart.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Close auto-mode classifier gaps from review round 19 for PR #7079
Approve for me now asks for these too:
- a terminal read whose path is built by indirect parameter expansion
(x=passwd; p=x; cat /etc/${!p})
- a bash /dev/tcp or /dev/udp redirection, which opens a network socket
(cat </dev/tcp/host/port)
- a python read via pathlib's receiver-plus-pattern glob
(Path('/etc').glob('passw?'))
- a python read whose sensitive root passes through a normalizer
(os.path.abspath('/etc'), Path('/etc').resolve())
- a pickle-backed loader that can execute code on load
(torch.load, joblib.load, pandas.read_pickle), tracked through module
import aliases
- compiled code wrapped into a callable (compile(...) + types.FunctionType)
Adds regression tests for each case and its safe counterpart.
* Honor unset permission_mode as ask across the local tool loop for PR #7079
Three gaps where an omitted permission_mode did not behave as the
documented default ("ask"):
- The frontend only sent permission_mode / confirm_tool_calls /
bypass_permissions when a tool pill was on. A process policy
(unsloth run --enable-tools) can open the tool loop with no pill, so
the backend never saw the selected gate. Send the three permission
fields at the top level of every local chat payload instead.
- The backend read payload.confirm_tool_calls directly at the
pre-switch guard and both late per-backend derivations, so an unset
mode fell through as no-gate even for an explicit ask/auto. Add
_permission_mode_confirm(payload): explicit confirm_tool_calls wins,
explicit ask/auto engage the gate, off/full never prompt, and an
unset mode defaults to ask only where realizable (streaming), keeping
the legacy no-gate run for non-streaming unset requests.
- A forced ask/auto tool loop (CLI --enable-tools) with no stream now
400s at the pre-switch guard before evicting the resident model,
matching the existing confirm-without-stream rejection.
Adds test_permission_mode_confirm_derivation covering the derivation
truth table.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Declare permission_mode and bypass_permissions on the local chat request type
The previous change moved permission_mode, confirm_tool_calls and
bypass_permissions to the top level of the local chat payload. They had
lived inside a conditional spread, which is not subject to excess
property checking, so the fields were never declared on
OpenAIChatCompletionsRequest. At the top level tsc flagged
permission_mode as unknown (TS2322), failing the frontend build and
every job whose Studio install builds the frontend.
Add permission_mode and bypass_permissions to the request interface
(confirm_tool_calls was already present).
* Close auto-mode classifier gaps from review round 21 for PR #7079
Auto mode ("Approve for me") now asks for these too:
- a pathlib read built from a concrete constructor (PosixPath, WindowsPath
and their Pure* forms), which the folder previously ignored so
PosixPath('/etc') / 'passwd' lost its /etc root and ran unprompted
- a terminal or python read of the ssh host keys under /etc/ssh, which
the sensitive-path regex only covered for passwd/shadow/sudoers
- a read whose path variable is reassigned: the whole-tree pre-scan kept
the last binding, so base = '/etc'; open(base + '/passwd'); base = 'data'
folded to data/passwd and ran even though execution reads /etc/passwd;
any multiply-bound name now folds to the escape sentinel and asks
Also stop the pre-switch guard from rejecting a plain client-tool
passthrough. permission_mode only implies the confirm gate for Studio's
own local tool loop (enable_tools / enabled_tools / mcp_enabled); a
non-streaming client-tool passthrough that carries permission_mode
ask/auto (confirm_tool_calls left unset by the validator) must forward to
the provider branch. Only an explicit confirm_tool_calls=True still forces
the local-confirm rejection there.
Adds regression tests for each case and its safe counterpart.
* Fix permission-pill compaction count and Full-access confirm sync for PR #7079
Two frontend consistency issues in the permission-level UI:
- The composer collapses tool pills to icons above four, but the count
left out the permission pill, which renders in every mode except off.
With one optional pill also shown the row reached five pills without
collapsing and could overflow. Count the pill when it is visible
(permission_mode != off).
- Entering Full access via setPermissionMode('full') or
setBypassPermissions(true) left confirmToolCalls at its previous value,
so a Full-access run (which sends confirm_tool_calls=false) could still
report confirmations as enabled in response metadata. Set
confirmToolCalls false at both entry points.
* Close auto-mode classifier gaps from review round 23 for PR #7079
Auto mode ("Approve for me") now asks for these too:
- a command using an abbreviated GNU long option that reaches a
write/exec action (sort --out= for --output, env --ch= for --chdir,
fd --base-dir= for --base-directory); a prefix of an unsafe long flag
now fails closed
- printf -v NAME, which assigns to a shell variable, so
printf -v PATH %s .; ls can rewrite PATH and run ./ls unprompted
- fd --base-directory / --search-path, which move the search root
outside the session workdir without any positional slash token
- an MCP tool whose compound read name carries a copy-style mutator
(read_and_copy_file, get_and_snapshot_volume): copy, duplicate,
import, export, download, backup, restore, snapshot, mirror
Also treat an omitted permission_mode as its documented default ("ask")
on the Anthropic Messages server-tool path. That branch has no
confirmation channel and already rejects explicit ask/auto, so an
omitted mode now falls into the same rejection instead of silently
running server tools unprompted, unless the caller opted out with
confirm_tool_calls=false (the legacy equivalent of "off"). off/full and
that opt-out still run; the two routing tests that relied on the old
implicit run now set permission_mode="off".
Adds regression tests for each case and its safe counterpart.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Refine permission gating from review round 24 for PR #7079
Four fixes from the latest review:
- Anthropic Messages server tools: an omitted permission_mode no longer
rejects a request that only runs safe server tools (web_search), so
existing Anthropic callers keep working. It still rejects an omitted
mode when a local tool (terminal/python) is selected, and an explicit
ask/auto is still rejected outright. off/full and a
confirm_tool_calls=false opt-out always run.
- Pre-switch confirm-without-stream guard: use
_explicit_studio_tool_loop_requested (the same predicate the
passthrough router uses) instead of the policy-inclusive
_effective_enable_tools, so a process --enable-tools policy no longer
turns a client-tool passthrough into a local-loop rejection.
- Auto mode now asks for `uniq INPUT OUTPUT`: uniq writes its second
file positional, so a second positional (numeric flag values skipped)
is treated like `sort -o`. A lone `uniq file` or piped `... | uniq`
stays safe.
- MCP mutation check now strips SQL comments before matching, so
DELETE/**/FROM and UPDATE/**/users (comment-as-whitespace) no longer
slip past the DML/DDL denylist.
Adds regression tests for each case and its safe counterpart.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Close auto-mode gaps from review round 25 for PR #7079
Auto mode ("Approve for me") now asks for these Python cases too:
- a bare archive constructor with a write mode (from zipfile import
ZipFile; ZipFile('out.zip', 'w')), tracked through import aliases like
the zipfile.ZipFile attribute call already was
- a dynamic lookup aliased through getattr (g = getattr;
rm = g(os, 'remove'); rm('file')), not just direct getattr(...) calls
- a callable that wraps open or a writer via functools.partial
(w = partial(open, mode='w'); w('out.txt')), which hides the write mode
Also:
- Always-safe tools (render_html) stream their early provisional canvas
card in auto mode again. The provisional-card guard mirrored the raw
confirm flag, which suppressed the early card under Approve-for-me; it
now reuses the auto-mode safety decision (is_always_safe_tool).
- The assistant-ui composer no longer counts the permission pill toward
its collapse threshold when the level is Off (the pill renders null
there), matching the other composer.
Adds regression tests for each case and its safe counterpart.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Align permission-mode confirm guards with the router (review round 26)
Three pre-switch confirm-gate checks disagreed with how the tool
loop actually enters, so a valid request could 400 (or an invalid
one could evict the resident model) at the wrong point:
- The /chat/completions pre-switch guard only looked at explicit
request fields, so a process --enable-tools policy that forces the
loop on (request omits enable_tools, no client tools) slipped past
it and only 400ed after _maybe_auto_switch_model had swapped the
model. It now mirrors the router's own loop-entry gate
(_effective_enable_tools or mcp, tool_choice="none" disabling it
unless explicitly asked) while still deferring to client-tool
passthrough, so the policy-forced case is caught before the switch.
- The ChatCompletionRequest full/off fold treated enabled_tools by
itself as a local-loop request and set confirm_tool_calls=True.
The router never starts the loop on enabled_tools alone (it only
filters which tools run), so a non-streaming passthrough carrying
client tools plus enabled_tools 400ed instead of routing verbatim.
The fold now keys off the same enable_tools / mcp_enabled signals.
- The Anthropic /v1/messages unsupported-mode rejection (ask/auto,
or an omitted mode selecting terminal/python) ran inside the
post-switch server-tools block, so an invalid request evicted the
resident model before the 400. It now runs before the auto-switch,
determined from the requested server tools, like the neighboring
malformed- and mixed-tool guards.
Adds regressions for each: a policy-forced non-streaming ask/auto
guard rejection that never reaches the switch, an enabled_tools-only
passthrough that keeps confirm unset, and an Anthropic rejection that
precedes _maybe_auto_switch_model.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Close auto-mode classifier gaps from review round 27 for PR #7079
Auto mode ("Approve for me") now asks for these host-mutating or
host-reading cases it previously ran unprompted (the sandbox does not
jail filesystem reads, and terminal commands can change host state):
- Destructured string literals fold into the scanned path now, so
base, leaf = ('/etc', 'passwd'); open(base + '/' + leaf).read()
resolves to /etc/passwd and asks, like the single-assignment form
already did. The tuple/list unpacking branch tracked only aliases to
open; it now also binds literal and folded-path elements.
- pathlib name rewrites fold to the rewritten path:
Path('/etc/x').with_name('passwd').read_text() (and with_stem /
with_suffix) spell no literal /etc/passwd but resolve to it, so they
are folded and caught. Benign in-sandbox rewrites stay safe.
- hostname NAME (or -F/--file, -b/--boot) sets the hostname, so a
positional or a set flag asks; bare hostname and the display flags
(-f/-i/-I/...) stay read-only.
- date -s/--set STRING and the bare MMDDhhmm... positional set the
system clock and now ask; the display forms stay read-only (+FORMAT,
-u/-R, and -d/-r/-f whose following value is skipped so date -d
tomorrow is not mistaken for a clock-setting positional).
Adds regression rows for each gap and its safe counterpart.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Close more auto-mode classifier gaps from review round 28 for PR #7079
Auto mode ("Approve for me") now asks for these cases too:
- Mapping-style %-formatted paths. '/etc/%(f)s' % {'f': 'passwd'} folds
to /etc/passwd and asks; a dynamic value or a non-literal mapping
leaves the NUL marker so /etc/<dynamic> still fails closed. The path
folder previously handled only tuple/scalar % right-hand sides and
returned None for a dict, hiding the sensitive segment.
- A read-named MCP database tool carrying PostgreSQL COPY. COPY ... FROM
bulk-loads a table and COPY ... TO writes a server-side file, so both
are matched as mutating queries like DELETE/UPDATE already were. A
'copy' substring in a column name stays safe (word boundary).
- logging file handlers. logging.FileHandler('out.log', mode='w') (and
the default append mode, RotatingFileHandler/TimedRotatingFileHandler/
WatchedFileHandler, and the bare from-import form) create or truncate
a file like open(..., 'w'), so they are classified as writer calls.
StreamHandler / NullHandler and logging reads stay safe.
Adds regression rows for each gap and its safe counterpart.
* Fix writer aliases, GraphQL mutations, and auto server tools (review round 29)
- Auto-mode Python: an aliased writer or archive constructor is tracked
like the existing open alias, so from numpy import save; s = save;
s('out.npy', arr) (and z = ZipFile; z('a.zip', 'w'), incl. the
destructured forms) ask instead of running the write unprompted. A
benign builtin alias (x = len) stays safe.
- Auto-mode MCP: a read-named tool carrying a GraphQL mutation now asks.
query_graphql {"query": "mutation { deleteIssue(id: 1) }"} matches a
leading mutation keyword (GraphQL uses # comments, so it scans the raw
payload); GraphQL read queries stay safe.
- Anthropic /v1/messages: permission_mode "auto" no longer 400s a
safe-only server-tool selection. auto only needs a confirmation
channel for an unsafe call, so like the omitted default it runs for
web_search / RAG / render and rejects only when a gate-needing local
terminal/python tool is selected. ask still always rejects (it asks
per call, which this passthrough cannot honor). The rejection stays
ahead of the model auto-switch.
Adds regression rows/cases for each.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Gate asyncio spawn, net clients, default-captured open; allow safe-only auto (round 30)
Auto-mode Python now asks for more process/network/write vectors:
- asyncio process spawners (asyncio.create_subprocess_exec/shell and a
loop's subprocess_exec/shell) run an arbitrary program without the
terminal blocklist, so they gate like os.system/subprocess.
- stdlib network clients imaplib / poplib / nntplib / xmlrpc(.client) /
webbrowser open outbound connections the sandbox does not namespace
off, so their import asks like the other network modules.
- a callable captured as a function or lambda parameter default
(def f(o=open): o('out', 'w')) now binds that parameter into the same
alias set, so the later write through it is gated. A benign default
(o=len) stays safe.
Also, permission_mode "auto" no longer 400s a non-streaming local tool
request whose selection is always-safe-only (web_search / RAG / render).
auto only prompts for a classifier-flagged call, so a safe-only auto
request needs no stream, while ask, an explicit confirm_tool_calls=true,
MCP, and an unrestricted or unsafe selection still require it. Applied
via a shared _confirm_gate_needs_stream helper at the pre-switch, GGUF,
and safetensors confirm-stream guards; the loop's per-call confirm flag
is unchanged.
Adds regression rows/cases for each.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Catch brace-glob paths and attribute writer aliases; unfold auto (round 31)
- Terminal auto mode now runs the glob-sensitive scan over every
expansion candidate, so a brace-expanded glob (cat /e{t,}c/pass?d,
which bash expands to /etc/pass?d and then globs to /etc/passwd) asks.
Brace expansion alone spells no literal /etc/passwd and the glob only
resolves once the brace group is expanded, so scanning both together
is required. A benign brace + glob stays safe.
- Python auto mode now tracks a mutating attribute captured as a plain
name: s = np.save; s('out.npy', arr) binds a writer alias, a captured
.open bound method (p = Path('out').open; p('w')) fails closed on any
call since its mode position varies, and z = zipfile.ZipFile is gated
like the bare import. A benign attribute alias (x = np.mean) stays safe.
- permission_mode "auto" is no longer folded to confirm_tool_calls=true
on the request model. Folding it defeated the safe-only-selection
exception in _confirm_gate_needs_stream (an explicit confirm forces
stream=true), so a non-streaming safe-only auto request was rejected.
Leaving it unset lets the route apply the exception; the mode still
drives the loop's per-call gate. "ask" still folds (it gates every
call).
Adds regression rows/cases for each.
* Harden SQL/GraphQL/writer classification and passthrough guards (round 32)
MCP argument mutation detection (read-named query tools):
- CREATE DDL now matches modifiers and the broader object set, so
CREATE OR REPLACE VIEW, CREATE UNIQUE INDEX, CREATE TEMP TABLE,
CREATE MATERIALIZED VIEW and CREATE FUNCTION ask.
- Stored-procedure invocation (CALL proc(...), EXEC/EXECUTE) and VACUUM
ask; a natural-language "call me back" stays safe via the trailing
"(" / ";" / end lookahead.
- GraphQL # comments are stripped before the mutation match, so
mutation # note\n { deleteIssue(id: 1) } no longer hides the mutation.
Python auto-mode classification:
- numpy.memmap / open_memmap and pandas ExcelWriter / HDFStore create or
truncate a file on construction, so they gate like open(..., "w").
- asyncio networking (asyncio.open_connection, loop.create_connection /
create_server and unix variants) opens outbound connections/listeners
the sandbox does not isolate, so it gates like socket.connect.
Terminal auto-mode: file -C / --compile writes a compiled magic database.
Routing:
- A JSON-schema response_format is guided-decoding passthrough, not a
local tool loop, so a --enable-tools policy no longer 400s a
non-streaming ask/auto structured-output request at the confirm guard.
- An explicit confirm_tool_calls=False opts out of the Anthropic Messages
server-tool gate entirely (it wins over the mode, mirroring
_permission_mode_confirm and the GGUF path), so it runs even under ask.
Adds regression rows/cases for each.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Track path-ctor aliases, exempt empty selection and safe safetensors card (round 33)
- Python auto mode now propagates path constructor / join aliases, so
assigning Path or os.path.join to another local name is still folded:
P = Path; (P('/etc') / 'passwd').read_text() and j = os.path.join;
open(j('/etc', 'passwd')) ask, while a benign /tmp alias stays safe.
- _confirm_gate_needs_stream now distinguishes an omitted enabled_tools
(None, all tools) from an explicit empty list ([], no tools). An empty
selection runs no built-in tool and cannot prompt, so a non-streaming
auto request with enable_tools=true, enabled_tools=[] is no longer
400ed under a --enable-tools policy.
- The safetensors provisional render_html card now uses permission_mode:
render_html is always safe and never prompts, so its early canvas card
streams under auto (which ships confirm_tool_calls=true) instead of
being suppressed, matching the GGUF path's is_always_safe_tool exemption.
Adds regression rows/cases for each.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Extend auto-mode classifier: SQLite mutations, more net/xattr/compressed writers
Additional fail-closed gaps found by a fresh adversarial pass, each with a
reproduction and a benign control:
- MCP read-named tools now ask on SQLite-flavored writes the base DML/DDL regex
missed: ATTACH / DETACH DATABASE, a write-form PRAGMA (PRAGMA journal_mode=WAL
/ user_version=42 / foreign_keys(0), while the read-form PRAGMA journal_mode
stays safe), and load_extension() which loads and runs an arbitrary shared
library.
- Python auto mode now gates the remaining asyncio network entry points
(start_server, open_unix_connection, loop.create_datagram_endpoint,
sock_connect), os.setxattr / os.removexattr metadata writes, the gzip / bz2 /
lzma single-stream writers (GzipFile / BZ2File / LZMAFile, mode-gated like
ZipFile so a read stays safe), pandas to_xml, and the websockets client.
Benign controls (SELECT 1, read-form PRAGMA, asyncio.sleep, gzip read, numpy
read, natural-language "attach"/"analyze") stay safe. Regression rows added to
test_permission_mode.py.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Close follow-up auto-mode gaps: SQLite/GraphQL variants, more writers and net
A fresh adversarial pass on the previous round found consistent extensions of
the same fail-closed rules, each reproduced with a benign control:
- MCP read-named tools: DROP / ALTER now cover the same broad object set as
CREATE (DROP FUNCTION, ALTER INDEX, DROP MATERIALIZED VIEW); ATTACH is caught
without the optional DATABASE keyword via its quoted-path form; a
schema-qualified write PRAGMA (PRAGMA main.user_version=1) is matched; and a
GraphQL mutation carrying directives (mutation M @audit { ... }) is treated as
a mutation.
- Python auto mode: os.startfile (Windows program launch), asyncio
start_unix_server, and the socketserver framework now ask; a gzip/bz2/lzma
open imported under an alias (from gzip import open as gopen) is gated like
builtin open; and a dynamic path prefix that can form a sensitive absolute
root (open(chr(47) + "etc/passwd"), open(os.sep + "etc/passwd")) is treated as
sensitive, while a dynamic prefix with a benign suffix stays safe.
Benign controls (read-form PRAGMA, natural-language "attach ... as", "drop the
idea", SELECT dropped_at, query @cached, gzip read alias, dynamic prefix +
data/file suffix) stay safe. Regression rows added to test_permission_mode.py.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Gate GNU time -o, basicConfig/methodcaller/fileinput, and more SQL mutations
Another adversarial pass surfaced further consistent fail-closed gaps, each
reproduced with a benign control:
- Terminal: GNU time -o/--output/-a/--append truncate or append to a file with
timing output; time is a wrapper, so the flag is checked before the wrapped
command like env -C.
- Python auto mode: logging.basicConfig(filename=...) opens a log file for
write; operator.methodcaller("write_text"/...) hides a writer method behind a
string and is now treated as dynamic dispatch (like getattr/partial);
fileinput.input(..., inplace=True) rewrites a file in place (the default read
form stays safe).
- MCP read-named tools: UPDATE now matches quoted, bracketed, and
schema-qualified targets (UPDATE "users" / public.users / ONLY public.users /
[users] / `users` SET); SELECT ... INTO OUTFILE/DUMPFILE writes a server file;
and state-changing SQL functions inside a SELECT (pg_terminate_backend,
setval, pg_write_file, lo_export, ...) ask.
Benign controls (time ls / time -p, basicConfig(level=), methodcaller("upper"),
fileinput read, NL "update ... set", setval_col column, PL/pgSQL SELECT INTO
var) stay safe. Regression rows added to test_permission_mode.py.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten auto-mode classifier comments
Collapse the multi-line rationale blocks in the permission classifier to one or
two lines each without dropping the exploit each branch closes. Comments and
whitespace only (no code change); the classifier tests are unchanged and pass.
* Retry transient SSE stalls in the tool-calling smoke probes
The tool-calling job flaked with a bare "TimeoutError: timed out": the
server-side python/bash probes stream over post_sse(), which (unlike
post()) had no transport-level retry, so a single stalled stream on a
shared CI runner hard-failed the whole step even though function calling
had already passed.
post_sse() now mirrors post(): a transport-level stall (stream open or a
mid-stream read timing out) is retried once with a fresh request capped
at 300s, while HTTP status errors still surface immediately. The
Linux _run_tool_probe caps each attempt at 360s and treats a stall that
outlives the retry as a failed attempt (rotate to the next seed) instead
of raising, and the web_search probe uses the same 360s cap. A genuine
server wedge still fails (the retry also times out), so real regressions
are not masked. Applied to the Linux, macOS, and Windows inference-smoke
workflows, which share the probe.
* Close five more auto-mode classifier gaps from review
Each reproduces with a benign control:
- Path constructor aliased through an attribute (P = pathlib.Path) now folds
like the bare-name alias, so (P('/etc') / 'passwd').read_text() asks while a
/tmp alias stays safe.
- Callable defaults that are not plain names now bind the parameter: an
attribute writer (def f(s=np.save)), an archive constructor, a captured .open,
and partial(open, mode='w') fold like the equivalent assignment; a benign
default (np.mean) does not.
- A dynamic piece inside a sensitive name (open('/et' + chr(99) + '/passwd'),
which folds to '/et\x00/passwd') now asks: the literals around each dynamic
segment are matched against a credential target with the segment as any run of
non-separator chars, so an all-dynamic ('1 + 1') or segment-spanning
(a + '/' + b) path stays safe.
- MCP read-named tools now ask on REFRESH MATERIALIZED VIEW and REINDEX; a
'refresh' column or natural-language 'refresh' stays safe.
- A writer/open alias handed to a higher-order invoker (map(open, names, modes),
starmap(np.save, ...)) is gated even without a direct call site; a benign
map(len, ...) is unaffected.
Regression rows added to test_permission_mode.py.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Default tool pills off on model load so tool execution is opt-in
resolveToolsEnabledOnLoad turned the web-search and code pills on for
any tool-capable model when the user had expressed no preference. Default
them off instead, so tool execution is enabled only when the person
clicks the pill to turn it on; a saved preference (on or off) is still
honoured, so a user who already enabled tools keeps them on.
* Gate mark/subscribe MCP verbs and qualified higher-order writer invokers
- A read-prefixed MCP tool name carrying mark / subscribe / unsubscribe
(get_and_mark_read, get_and_subscribe) now asks; a 'mark' substring inside
one token (list_bookmarks) stays safe.
- The higher-order writer check now also fires for a qualified invoker
(itertools.starmap(open, ...), functools.reduce(open, ...)), matching the
bare-name map/filter form; the writer-check on the first arg keeps a benign
itertools.starmap(len, ...) or itertools.chain(...) safe.
Regression rows added to test_permission_mode.py.
* Close more auto-mode gaps and align the ask confirm fold across paths
Each classifier change reproduces with a benign control:
- MCP read-named tools now ask on reply / notify verbs (get_and_reply_email,
list_and_notify_users), on catalog writes COMMENT ON / SECURITY LABEL / LOCK
TABLE and CREATE|DROP|ALTER POLICY, and on state-changing PostgreSQL functions
inside a read-shaped SELECT (nextval, set_config, pg_notify, the advisory-lock
family). A 'comment' column, a 'locks' table, and a 'nextval' column prefix
stay safe; the natural-language NOTIFY/SET ROLE statement forms are left out
because SET/NOTIFY overlap ordinary prose.
- Python auto mode now gates loader.exec_module (runs a module's code), archive
extractall (zip-slip file writes), the ensurepip / venv modules (install pip /
build an environment), and pydoc.writedoc. The Hugging Face login token
(~/.cache/huggingface/token and stored_tokens) is now a sensitive path, while
the rest of that cache (model data) stays readable.
- ChatCompletionRequest no longer overwrites an explicit confirm_tool_calls=false
when permission_mode='ask': the fold only self-enables the gate when the flag
is unset, so an explicit opt-out wins on the chat path exactly as it already
does via _permission_mode_confirm and the Anthropic pre-switch guard.
Regression rows added to test_permission_mode.py.
* Gate sort -T, xxd outfile positional, and the legacy HF token path
- sort -T / --temporary-directory writes spill files to a caller-chosen dir,
so it joins -o / --output in sort's unsafe-flag set.
- xxd [infile [outfile]] writes its second positional, like uniq; xxd now uses
the same second-positional-write handling (xxd in.bin out.hex asks, xxd
in.bin and xxd -c 16 in.bin stay read-only).
- The sensitive-path regex now also covers the legacy ~/.huggingface/token
location (optional leading dot), not just ~/.cache/huggingface/token; an
unrelated dir like myhuggingface/token stays safe.
Regression rows added to test_permission_mode.py.
* Catch multi-char SQL mutation targets, globbed credential names, digit outfiles
Three fail-open gaps in the auto-mode classifier, each with a benign control:
- SQL: the trailing word boundary on the MCP mutation regex meant a bare \w
stopped at the first character, so TRUNCATE users, GRANT SELECT ON t, and
REVOKE ALL ON t (multi-character names) slipped through while single-letter
targets matched. Match the whole identifier instead, and accept an explicit
AS alias on UPDATE (UPDATE users AS u SET). The implicit-alias form is left
out because it is indistinguishable from the prose "update <noun> <noun> set".
A truncate_log column and a grants table stay safe.
- A glob that resolves to a credential basename anywhere (cat ~/.huggingface/tok?n
-> token, cat proj/.netr? -> .netrc, cat repo/.aws/cred*) now asks; the fixed
target list only covered a handful of home paths. notes/dra?t.txt and
token_counts.tx? stay safe.
- uniq / xxd counted file positionals but skipped every numeric token to ignore
a flag value, so a file literally named with digits (uniq 123 out) hid the
output positional. Track each command's value-taking flags and consume only
the value, so uniq -f 2 in stays safe while uniq 123 out asks.
Regression rows added to test_permission_mode.py.
* Isolate the permission-mode loop tests from process-global state
The loop-driving tests (auto/off/full/bypass) drove run_safetensors_tool_loop
against a process-global approval registry (state.tool_approvals._pending)
keyed by a single shared session id, and read os.environ. Other backend test
modules mutate both, some at import time, so in the full-suite ordering a stale
pending approval or a leaked env var could make the loop deny or skip a call
these tests expect to run. It passed when the file ran alone but failed only in
the complete tests/ run on CI.
Add an autouse fixture that snapshots and restores os.environ and the approval
registry around each test, and give every _drive call a unique session id so a
leaked approval can never collide. Attach a compact event-stream dump to the
loop assertions so any residual full-suite-only failure reports what the loop
actually did instead of a bare diff.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: harden auto-mode classifier for recursive listers, sort file lists, aliased invokers, single-member extract
Close four fail-open gaps in is_potentially_unsafe_tool_call:
- terminal: tree/du (always recursive) and ls -R rooted at an absolute or
tilde path now ask, matching the existing grep/rg/find recursive-read gate;
relative walks stay safe.
- terminal: sort --files0-from=F reads the file list named in F, so it can
read arbitrary host files indirectly; added to sort's unsafe flags.
- python: track aliases of the higher-order invokers (m = map;
from itertools import starmap as sm) so an aliased invoker handed open/a
writer is still gated; a benign callable (map(len, ...)) stays safe.
- python: single-member archive extract (ZipFile/TarFile.extract) writes to
disk like extractall and is vulnerable to a crafted member path, so gate it.
Also update the stale _FakeExecuteTool in test_permission_mode.py to accept
the thread_id keyword that run_safetensors_tool_loop now forwards to
execute_tool after the main merge, which had broken the five tool-loop tests.
Adds regression rows covering each gap plus benign controls.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: normalize unknown permission_mode to 'ask' instead of a 422
The request models validated permission_mode with Literal[ask, auto, off,
full], so an unrecognized value from a newer UI/client was rejected with a 422
before the tool loops could apply their unknown -> ask fallback
(safetensors_agentic.py:464, llama_cpp.py:9001). That made the intended
forward-compat degradation unreachable at the API boundary for both Chat
Completions and the analogous Anthropic field.
Accept a plain string on both ChatCompletionRequest and AnthropicMessagesRequest
and normalize in a before-validator: None stays unset, the four known modes pass
through, and any other value degrades to the safest gate ('ask'), matching the
loops. Adds a regression test covering unknown/None/known across both models.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: close five more auto-mode classifier gaps
- terminal: xargs is no longer a safe wrapper. It appends arguments read from
stdin that the scan never sees, so `echo -o out /etc/passwd | xargs sort`
forwards to `sort -o out /etc/passwd` (a write + sensitive read) while only
the allow-listed literals are visible. Any xargs command now asks.
- terminal: ionice -p/-P/-u change the I/O priority of an already running
process / group / user instead of forwarding to a wrapped read-only command,
so `ionice -c 3 -p <pid>` now asks. ionice -c 3 <cmd> stays safe.
- MCP: gate ALTER SYSTEM, which persists PostgreSQL server configuration and was
not one of the DDL objects the mutation detector matched.
- MCP: a credential noun in a read-named tool (read_secret, list_tokens,
get_credentials, fetch_api_key) is a sensitive disclosure, so it asks even
without a mutating verb or a path/SQL argument. Scoped *_key nouns keep a
primary_key / keyboard lookup safe.
- render_html: no longer unconditionally safe. A static canvas still auto-runs,
but one whose HTML/JS reaches the network (fetch/WebSocket/remote script) asks,
since it can egress under the canvas CSP when artifact network access is on.
Its early provisional card is suppressed under the auto confirm gate, and the
confirm-without-stream guard now requires a stream when render_html is
selectable.
Adds regression rows and benign controls for each, and updates the render_html
provisional-card and confirm-gate tests to the new behavior.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: extend auto-mode gates for indirect file lists, dynamic lookups, HTML network loads, and Anthropic render_html
Follow-ups on the previous classifier round:
- terminal: wc/du/find --files0-from (and find's -files0-from primary) read a
NUL-separated list of input paths from a file, the same indirect mechanism as
sort --files0-from, so a crafted list reads arbitrary host files past the
literal path/root checks. Gate them like sort.
- python: a namespace lookup through a dict-style call (f =
__builtins__.__dict__.get('open'), globals().get('open'), vars(x).get(...))
can return open/eval/a mutator, so poison the bound name like getattr/subscript
lookups already are. An ordinary dict .get or os.environ.get stays safe.
- render_html: broaden the network detector so a canvas that loads a resource
via CSS url()/@import, srcset, or a root-relative (/path) or protocol-relative
(//host) src/href is treated as networked, not just fetch/WebSocket/remote
script. Relative ./x and url(#id)/data: refs stay static/safe.
- Anthropic /v1/messages: drop render_html from the unprompted-safe server-tool
set. Since it can prompt (networked canvas) and this channel invokes the loop
without confirm, selecting it under ask/auto/omitted now rejects like
terminal/python; off/full (or an explicit confirm opt-out) run it.
Adds regression rows and benign controls for each, plus an Anthropic route test.
* Studio: close six more auto-mode classifier gaps
- terminal: a glob that expands to a project .env (cat .e?v) now asks; .env
joins the sensitive glob-basename set, matching the literal-path gate.
- python: an open bound onto an attribute (box.f = open; box.f('out','w'))
is tracked by attribute name, and open invoked via .__call__
(open.__call__('out','w'), unwrapped to the underlying callable) is gated,
so neither slips past the name-based open-alias checks. Benign attribute
callables and .__call__ on non-writers stay safe.
- python: a namespace lookup via .get/.pop/.setdefault already covered the
builtins case; unchanged here.
- MCP: a mutating HTTP verb in a method/verb argument (get_url
{"method": "DELETE"|"POST"|"PUT"|"PATCH"}) now asks, so a generic HTTP
tool cannot mutate an external service unprompted; GET/HEAD stay safe.
- MCP: a credential/secret environment-variable value (get_env
{"name": "OPENAI_API_KEY"}) is treated as a sensitive read via the same
credential-noun match used for tool names; PATH/HOME stay safe.
- render_html: self-navigation sinks (location.assign/replace, window.open,
assigning a URL to (window.)location(.href)) join the network detector, so a
canvas that navigates itself to an external URL asks; location.reload() /
history.back() stay static.
Adds regression rows and benign controls for each.
* Studio: gate obfuscated canvas egress, sensitive-dir iteration, and MCP metadata-host reads
- render_html: strip block comments before the network scan so fetch/*x*/(...)
cannot hide egress, and match bracket-access forms (window['fetch'](...),
self['open'](...)). Line // comments are left alone so the // in an https URL
is not eaten. A comment-only canvas stays static.
- python: enumerating a directory outside the sandbox (Path('/etc').iterdir(),
os.scandir('/etc'), os.listdir('/home'), os.walk('/')) reads host filenames
the direct /etc/passwd checks would prompt for, so gate it when the target dir
folds to an absolute/tilde/sensitive path; a relative dir stays safe and an
unresolved dynamic dir is left to other checks.
- MCP: a read-named HTTP tool pointed at a cloud-metadata / link-local host
(fetch_url {"url": "http://169.254.169.254/..."}, metadata.google.internal)
reads instance credentials, so classify those URL arguments as sensitive,
mirroring the sandbox SSRF blocklist; ordinary and localhost URLs stay safe.
Adds regression rows and benign controls for each.
* Studio: gate meta-refresh navigation, pandas HTML/markdown exporters, absolute glob roots, and checksum verify mode
* Studio: gate starred open writes, builtins.__import__, computed render_html sinks, and procfs fd reads in auto mode
* Studio: gate remote worker canvases, huggingface_hub downloads, and write callables passed to user helpers in auto mode
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Studio: offer the latest transformers release for brand-new architectures
When a model's config.json model_type is absent from every installed
transformers overlay (base 4.57.x and the .venv_t5_530/550/510 sidecars),
Studio now checks, unauthenticated and cached, whether the newest
transformers ships it:
- utils/transformers_latest.py fetches the latest release version from
https://pypi.org/pypi/transformers/json and the CONFIG_MAPPING_NAMES
sources for that tag and for main from raw.githubusercontent.com
(never api.github.com), parsing them with the same AST extractor the
static router uses (no code execution, no trust_remote_code). Results
are cached in memory and in a JSON snapshot under studio_root()/cache
with a one day ttl; fetches are bounded to 5s with one retry and a
failure backoff, and offline mode or the new kill switch
UNSLOTH_STUDIO_NO_LATEST_TRANSFORMERS=1 short-circuits to None.
- POST /api/inference/validate gains requires_transformers_upgrade plus
a transformers_upgrade payload (model_type, pypi_version,
supported_in_pypi, supported_in_main) so the frontend can raise the
install consent dialog before /load, mirroring the existing
remote-code consent flow. The check fires only when the model_type is
unknown to all installed overlays and the hardcoded tier tables.
- POST /api/inference/install-latest-transformers provisions a new
persistent .venv_t5_latest sidecar after user consent, pinned to the
exact PyPI version (re-verified server-side) with the same
--target/--no-deps recipe as the fixed sidecars. A JSON pin marker
inside the dir records the installed package set, so restarts
revalidate it and routing resolves the new highest-ranked tier
automatically. A dependency preflight (compat_plan) compares the
release's requires_dist against the running env: unsatisfied
tokenizers/safetensors floors are shadow-installed as exact pins into
the sidecar, anything else unsatisfied blocks the install with a
clear message.
Routing for every already-supported model_type is unchanged: the
hardcoded lists and the 530/550/510 static resolver run first, the new
tier only participates once its venv exists, and the probe order gains
the latest sidecar only when provisioned. Verified against live PyPI
and GitHub (transformers 5.13.0: 674 model_types, 26 absent from all
installed overlays, e.g. cosmos3_omni; 4 dev-only on main) and with a
real sidecar install plus restart persistence. 64 new tests; the
existing 200-test transformers_version suite passes unchanged.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Latest-transformers check: fetch outside the lock, serialize installs
Release the module lock during the network refresh so a slow fetch cannot
stall other threads in the ASGI pool; concurrent callers during a fetch get
None (the graceful fallthrough) via an in-flight flag instead of stacking
fetches. Serialize install_latest_transformers with an in-progress flag so
concurrent consents cannot race the sidecar delete and recreate; the loser
gets a structured already-in-progress refusal.
* Latest-transformers check: LoRA bases, pin-gated mapping, live reverify
Run the upgrade check over the [adapter, base] target set so a LoRA whose
base model is a brand-new architecture surfaces the prompt (the worker
activates transformers for the base, not the adapter).
Gate the latest overlay's mapping lookup on a valid pin marker, matching
activation and the probe order, so a partial or manual .venv_t5_latest dir
cannot be routed to and then refused at activation.
Re-verify the requested version against a live PyPI snapshot at install
time, falling back to the cached one on fetch failure, so a release
published inside the cache TTL is not silently missed.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Latest-transformers check: nested config types and latest-tier vision probe
Collect every model_type in the config (top level plus each nested
sub-config) and signal on the first one missing from all installed
overlays, so a supported wrapper carrying a brand-new backbone still
surfaces the upgrade prompt; wrappers instantiate sub-configs through
CONFIG_MAPPING and would fail on the nested type.
Route the vision capability subprocess through the pinned latest sidecar
when the model resolves to the latest tier, so latest-only VLMs are not
misclassified as text-only; every other tier keeps the 5.5 sidecar used
today.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Latest tier: nested routing, vision probe after raw miss, safe upgrades
Route by every model_type in the config: a nested sub-config type can raise
the tier (wrappers instantiate sub-configs through CONFIG_MAPPING), so a
supported wrapper with a latest-only backbone routes to latest once
installed instead of staying on default. An unknown nested type never
vetoes; the primary type keeps its previous semantics. The collector is
shared with the upgrade checker.
Vision detection: when the raw heuristics say False for a model that routes
to the latest tier, run the AutoConfig subprocess under the pinned latest
sidecar instead of trusting heuristics built from older transformers.
Provisioning: stage-and-swap. Build the new sidecar in .venv_t5_latest.staging
and swap it in only when the install and pin marker are complete, so a failed
upgrade never destroys a previously working sidecar; restore the old dir if
the final swap fails.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Latest-transformers checker, vision subprocess, and cache fixes
Require the latest release to support every missing model_type (the
primary included) before prompting; a nested-only match cannot make the
model loadable, so no install is offered for it.
The vision-check subprocess now unions the active sidecar's own
registry mappings into the inlined parent-process detection sets, so
architectures only the sidecar knows classify correctly.
A successful sidecar install clears the tier probe cache, the latest
tier's model_type mapping, and the vision-detection cache so the new
venv takes effect without a restart. Tests for all three.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Aggregate upgrade support flags and keep install off /v1
The upgrade signal now reports supported_in_pypi only when the latest
release covers every missing model_type; a mix with a main-only nested
type surfaces as dev-only so no PyPI install is offered that would
still fail at load. The consented install endpoint moves to
studio_router so it is not reachable through the OpenAI-compatible /v1
mount. Tests for both.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Honor the latest-transformers kill switch in routing
With UNSLOTH_STUDIO_NO_LATEST_TRANSFORMERS set after the sidecar was
provisioned, the latest tier still joined mapping and probe routing
because only the pin was checked. Both admission points now also check
the kill switch, so operators can roll back a problematic sidecar
without deleting files.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Repair the latest sidecar through stage-and-swap
The lazy repair path installed into the live .venv_t5_latest, which
_ensure_venv_dir wipes first, so a failed repair deleted the pinned
sidecar and its marker. Both the consented install and the repair now
share one stage-and-swap helper: the incomplete-but-pinned dir survives
any failure and a later attempt can still repair it.
* Tighten comments
* Remove the staging dir when a latest-sidecar install fails
A pip failure inside _ensure_venv_dir returns False without raising, so
the except cleanup never ran and the partial .venv_t5_latest.staging
leaked until a later attempt. Also note on the validate response fields
that frontend consumption ships in the follow-up PR.
* Add the transformers-upgrade consent dialog to the frontend
When /validate reports requires_transformers_upgrade, every explicit load
path (chat runtime and the compare composer) now pauses on a consent
dialog modeled on the remote-code one: it names the model_type and the
latest PyPI transformers version, and on Accept calls
/api/inference/install-latest-transformers itself, shows an installing
state, and resumes the original load automatically on success. Errors
surface in the dialog with a retry; Cancel aborts the load like the
trust dialog's deny path. Architectures shipped only on transformers
main get a dev-only notice with no install button. Background auto-load
skips upgrade-requiring candidates instead of prompting, mirroring the
trust_remote_code rule. The dialog mounts once in the root layout and
runs before the security dialogs, since no load can proceed without the
runtime.
* Route a non-installable new architecture to the custom-code consent as a last resort
When the upgrade dialog has no installable PyPI release (the architecture
is only on transformers main, which Studio never installs), the dialog now
says so explicitly, and when the model also declares custom (auto_map)
code it offers Continue with custom code: resolving the paused load into
the existing trust_remote_code consent gate instead of hard-aborting.
Models with no custom code keep the Cancel-only notice. The backend
returns no upgrade signal at all for architectures unknown to both PyPI
and main, so those still route straight to the unchanged security gate.
* Force a 16-bit load for models on the latest-transformers sidecar
Live validation with Zyphra/ZAYA1-8B (model_type zaya, shipped by
transformers 5.13.1 but unknown to every installed tier) surfaced a
generation crash when the consented sidecar load kept the default bnb
4-bit quantization: transformers' grouped-MoE kernels feed the packed
uint8 expert weights straight into torch._grouped_mm, and generation
dies (plain 16-bit works). New latest_tier_active_for() mirrors the
sidecar activation's tier resolution and never raises; the inference
worker flips load_in_4bit off when it reports true, and the load route
applies the same flip so the pre-load VRAM guard and the worker command
agree. Fixed tiers are untouched. With the guard, ZAYA1-8B loads and
generates correctly in Studio chat.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Offer the custom-code fallback when a latest-sidecar install fails
* Fail remote mapping fetches wholesale and mirror the 16-bit flip in validate
A transient fetch or parse failure of one auto-mapping file no longer caches
a partial latest-release map for the TTL (a real 404 on pre-5.10 tags is
still tolerated), and validate_model now applies the same latest-sidecar
16-bit sizing flip as /load before the training guard so the two agree.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten comments in the latest-transformers changes
* Resolve remote LoRA bases, fold nested tiers, and guard the sidecar swap
latest_tier_active_for now resolves a remote adapter's base model the same
way worker pre-activation does (and returns early without a sidecar pin), a
hardcoded fast-path tier is raised when a nested sub-config's model_type
needs a higher sidecar, and the install route refuses to swap .venv_t5_latest
while training runs on it and unloads a latest-tier chat model first.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Gate the sidecar install on worker liveness and size installable upgrades 16-bit
The install route now refuses while any training or export runs (tier
re-resolution without the load token is unreliable for gated repos), holds
the inference lifecycle gate across the unload and the swap so no load can
interleave, and passes the model name to unload_model. validate_model runs
the upgrade check before the training guard and sizes an installable
upgrade as 16-bit, matching what /load and the worker will force after the
consented install.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Close the sidecar install races and honor the kill switch over cached mappings
Training starts and mutating export routes now refuse while a transformers
install is in progress (shared is_install_in_progress flag), the chat unload
and idle export-worker teardown moved into a before_swap hook that runs only
once the staged install succeeded, and _config_model_types checks the kill
switch before returning a cached latest mapping.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Reserve the sidecar swap before the gate wait and abort it on failed teardown
The install-in-progress flag moved into a shared sidecar swap reservation in
transformers_version, taken by the install route before awaiting the
inference lifecycle gate (so training and export starts see it for the whole
window) and by the lazy .venv_t5_latest repair path. The before_swap hook
now raises when the chat unload or export teardown reports failure, leaving
the previous sidecar untouched.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Back the sidecar swap reservation with a cross-process lock file
The lazy repair runs inside worker subprocesses, where a module-level flag
is invisible to the parent's route checks. The reservation now also creates
a lock file next to .venv_t5_latest (O_EXCL, owner-only removal, stale after
two hours for crashed owners), so is_install_in_progress sees a repair from
any Studio process.
* Hand the swap reservation to the installer thread and harden pre-swap teardown
A cancelled install request no longer releases the reservation while the
installer thread is still staging (the thread owns and releases it, shielded
from cancellation). The route refuses while another inference request is
generating, export teardown runs before the chat unload and is judged by
worker liveness rather than the cleanup return value, and a live inference
worker with no active model (failed load residue) is shut down before the
swap.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Keep the lifecycle gate with the installer and recheck the swap at spawn time
The gate moved into the shielded install task so a cancelled POST cannot
release the guard /load honors while the installer still runs, cached latest
probe results are ignored while the kill switch is set, and the training and
export subprocess spawns recheck the sidecar swap reservation right before
spawning (the route-level guards are one-shot and validation can outlast an
install's start).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Close the spawn-registration windows against the sidecar install
Training marks the spawn in progress before its reservation recheck and
is_training_active honors the flag, so the install route sees a start that
has passed proc.start() but not yet recorded _proc. Export load-checkpoint
rechecks the reservation after setting _export_active and before tearing
down the old worker, so losing the race keeps the loaded checkpoint instead
of surfacing a 500.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Refine the install-window interleavings around worker teardown
The inference busy count is rechecked under the lifecycle gate (streams
start by taking that gate, so nothing slips past a held gate), the training
handshake moved ahead of the VRAM-freeing before_spawn hook so a lost race
leaves chat/export intact, the export spawn-time check is op-aware (inside
an active op the install is the side that aborts), and the Xet-stall respawn
waits out a transient reservation instead of stranding the run.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Track the install's server-side unload and guard export ops against the swap
The upgrade dialog store records when its install actually ran (the server
unloads the active chat model before swapping), and the load flow then marks
the previous model as unloaded so a later cancelled gate still triggers
rollback; the custom-code fallback leaves the flag unset. _run_export gained
the same reservation handshake as load_checkpoint so an install cannot block
behind an hours-long export op instead of returning 409.
* Tighten comments in the install-guard and upgrade-consent changes
* Surface install-race refusals cleanly and roll back after a failed swap unload
/load refuses while the sidecar swap is reserved so a load cannot succeed
and immediately be unloaded by the pre-swap teardown, worker starts that
lose the install race raise a typed SidecarSwapInProgress mapped to 409
instead of a 500, the install response reports model_unloaded even on a
structured failure so the client can restore its state, and the compare
flow tracks the server-side unload like the primary load path and clears a
stale checkpoint on abort.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Type the export install races, scope the lock release, and keep the unload signal
Export load-checkpoint and export ops raise SidecarSwapInProgress (mapped to
409 in every export route) instead of a 400-shaped failure, the export spawn
check distinguishes repair reservations (always refused) from install ones
(op-aware), the swap lock release only unlinks a lock this process wrote so
a stale-superseded owner cannot drop the new owner's live lock, and the
frontend unload signal survives a superseding consent via read-and-clear
consumption instead of a reset.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Finalize a stalled run when the respawn loses the install race and latch the unload signal
The Xet-stall respawn timeout now finalizes the run as a failure instead of
raising into the pump's broad finalization catch (which stranded it in a
training state with no worker), and a successful install retry ORs the
model_unloaded signal with the latched value so a failed-after-unload first
attempt still triggers rollback.
* Recheck the swap under the load gate and latch the unload before resolver checks
/load rechecks the sidecar reservation after acquiring the lifecycle gate
(an install can reserve while the load queues on it), and the dialog store
latches model_unloaded as soon as the install response arrives, before any
resolver-identity guard, so a superseded consent's unload still reaches
whichever load consumes the signal next.
* Report cleared-state unload failures, guard queued installs, and fold name tiers
A failed chat unload that still cleared the orchestrator's model state now
reports model_unloaded so the client rolls back, the installer aborts with
a 409 when a model load completed while it waited on the lifecycle gate,
and the fixed-tier name fast path consults the config mapping when a latest
sidecar is pinned so an accepted upgrade routes to the sidecar it installed
(no I/O added to the unpinned path).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Report cleared-state unload failures and harden the spawn handshake flag
The failed-unload branch in before_swap now detects that the orchestrator
cleared its model state and reports model_unloaded before aborting (the
earlier commit claimed this fix but a scripting error dropped the edit),
the installer's queued-load check compares a load generation counter so a
same-model reload is caught, and both training spawn sites wrap everything
after the handshake in a guard that resets _spawn_in_progress on any
exception so a failed start cannot wedge is_training_active.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Bump the load generation when the load is published, not at load start
A start-time bump is already visible when the installer snapshots mid-load,
so a same-model reload completing after the snapshot looked unchanged and
could be unloaded by the swap. The counter now increments alongside the
active_model_name publish.
* Self-heal a broken pinned sidecar, guard lazy repairs, and refresh stale retries
A valid pin whose transformers source dir vanished now triggers the repair
from the routing path (with a five minute backoff after failures) instead of
silently routing latest-only models to older tiers, the lazy repair refuses
while parent-visible chat/training/export workers are active since it has no
teardown of its own, and a version-mismatch install failure carries the
superseding release so the dialog's Retry re-requests a version that can
succeed.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Flip latest-tier loads to 16-bit outside chat and protect export state
Training and export workers now apply the same latest-sidecar 16-bit flip
as the chat worker so a brand-new grouped-MoE architecture cannot reach bnb
4-bit through those paths, the latest-tier vision override returns None on
an inconclusive probe so a transient failure is not cached as not-vision,
and the install route refuses while an idle export checkpoint is loaded
rather than discard it with no rollback signal on a failed swap.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address parallel-review findings on the sidecar guards and install checks
The training route sizes latest-tier jobs 16-bit before GPU selection, the
inference subprocess spawn rechecks the swap reservation like training and
export (covering the OpenAI auto-switch path) with the typed error mapped
to a retryable 409, compat_plan blocks the install when dependency metadata
cannot be fetched instead of proceeding unverified, snapshot model-type
lists must contain only strings, and pin-marker package specs are validated
against the sidecar's own package set before ever reaching pip.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Parent-only repairs, live-owner locks, remote-base activation, pre-teardown recheck
Lazy sidecar repairs now refuse inside worker children (whose empty backend
singletons cannot see live siblings) and run only in the parent where the
active-worker guard is real, swap-lock staleness requires the owner pid to
be dead so a slow live install is never superseded, both activation entry
points resolve a remote adapter's base model like the inference worker and
latest_tier_active_for already do, and load_model rechecks the reservation
before tearing down the old worker so losing the race keeps the current
model loaded.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Check workers under the repair reservation and keep state on refused swaps
The lazy repair now reserves first and checks workers under the reservation
(worker starts set their active markers before rechecking, so every
interleaving aborts one side), with export ops and in-flight inference loads
counted as active. The inference pre-teardown and spawn guards refuse only
repair reservations since an install shares the load's lifecycle gate and
aborts via its queued-load snapshot, a SidecarSwapInProgress raised before
teardown no longer clears the live model mirrors, and an export spawn abort
after teardown clears current_checkpoint so the page cannot claim a loaded
checkpoint with no worker.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Repair a present-but-incomplete latest sidecar from routing
The routing self-heal only fired when the pinned sidecar's transformers/
dir was missing. A sidecar that kept transformers/ but lost another pinned
package still routed models to the latest tier, and workers refuse
parent-only repairs, so every load failed until a manual reinstall. Routing
now validates the full pin (via _venv_dir_is_valid) and repairs any
incomplete sidecar under the same swap reservation and 5-minute backoff.
* Treat an unrepaired latest sidecar as unavailable in routing
When the pinned sidecar is incomplete and the lazy repair fails (offline,
pip failure, workers active) or is inside the backoff window, routing
returned the source dir anyway, sending models to a tier whose worker
activation is known to fail. Return None instead so models an older tier
supports keep loading there until a repair succeeds, matching the behavior
when the sidecar dir is missing entirely.
* Harden sidecar swap and repair against crash, survivor, and 16-bit paths
Reclaim a swap lock as soon as its recorded owner PID is dead instead of
waiting out the two-hour cutoff, so a crash mid-install no longer wedges
/load, training, export, and repair for hours. A lock whose PID cannot be
read yet still uses the long cutoff so the create-before-write window is
never mistaken for dead.
Probe process liveness with OpenProcess on Windows: os.kill(pid, 0) there
is CTRL_C_EVENT (a real Ctrl+C via GenerateConsoleCtrlEvent), not a
harmless check, and psutil is not always present.
Return whether _shutdown_subprocess actually killed the worker and keep the
live handle when it survives terminate/kill (an uninterruptible CUDA
syscall can outlive SIGKILL). The pre-swap liveness guard now trusts that
result, so the destructive .venv_t5_latest rename cannot proceed while a
live worker still holds sidecar modules.
Recover a sidecar stranded at .old when a swap's activation rename and its
rollback both fail: reading the pin restores it when no swap holds the
reservation, so latest-tier models are not permanently broken.
Resolve the latest tier in the parent for export loads and for explicitly
16-bit training runs, not only 4-bit ones: tier resolution self-heals an
incomplete sidecar, and repairs are parent-only, so those paths could not
recover before. Sidecar integrity and quantization are independent.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Revert the parent-side latest-tier repair probe on training and export loads
The probe ran before the route freed VRAM, so a resident chat or export worker
made _workers_active_for_repair() refuse the parent-only repair; the route then
tore that worker down and spawned a child that also cannot repair, so an
incomplete sidecar still failed to load. Repairing correctly requires running the
repair between the worker teardown and the child spawn, decoupled from VRAM
sizing, which is a larger change tracked separately. Restore the prior behavior
so these paths match the reviewed form and do not partially attempt a repair that
cannot complete while workers are resident.
* Honor failed worker shutdowns on load and revalidate the cached latest mapping
The fresh-load paths spawned a new worker straight after _shutdown_subprocess
without checking its result, so a worker that outlived terminate/kill (a wedged
CUDA syscall) had its handle overwritten by the replacement while it still held
GPU memory, and is_worker_alive/the pre-swap guard could no longer see it. Both
the inference load and the export checkpoint load now abort when the old worker
did not exit, so the load can be retried once it does.
_config_model_types returned a cached latest mapping without re-checking the
sidecar, so a sidecar deleted or broken in-process after its first parse was
never re-validated: routing kept sending latest-only models to the stale latest
tier while activation failed. The cached latest mapping is now dropped and
re-resolved (self-healing) when the sidecar is no longer intact.
* Drop cached latest mapping when the pin is gone; keep 4-bit for custom-code fallback
_latest_sidecar_intact now returns False when the pin marker itself is gone, not
just when a pinned package is missing. Otherwise a cached latest mapping outlived
a deleted pin: _config_model_types kept returning it, so routing sent latest-only
models to a tier whose worker activation then failed (no pinned version) until
restart. It now drops the cache and re-resolves to no latest tier. The
_overlay_transformers_dir caller already gates on a present pin, so it is
unaffected.
validate_model forced 16-bit sizing whenever a PyPI upgrade was merely offered,
even for a model that can fall back to its own auto_map code. /load loads such a
model 4-bit without the install, and the install route refuses while training is
active, so 16-bit sizing here returned a VRAM 409 for the only viable 4-bit path.
The offered-upgrade flip is now gated on the absence of a custom-code fallback;
an already-active latest sidecar still always sizes 16-bit.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: scope the seeded bootstrap password auto-fill to loopback clients
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: block bootstrap injection through Cloudflare tunnels
* Studio: require loopback host for bootstrap injection
* Studio: add regression test for unparseable Host in bootstrap loopback gate
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Scope bootstrap auto-fill to a direct-loopback client (block proxy/tunnel headers and malformed Host)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Reject scope-id addresses in loopback check (fail closed on ::1%zone Host)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Reject malformed bracketed Host in loopback check (e.g. [::1]evil)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: danielhanchen <23090290+danielhanchen@users.noreply.github.com>
* fix(studio): recover mlx vlm image prompts
* fix(studio): detect serialized vlm media items
* studio: recover MLX VLM prompts when model_type only lives on _config
_mlx_vlm_model_config only fell back to _config when config was entirely
missing, so a model that exposes a config without a model_type (while _config
carries it) skipped model-aware recovery. Prefer whichever of config / _config
actually has a model_type. Adds a focused test.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
oobabooga