* add models for /update endpoint
* add logic for identifying out of date hf models
* add endpoint for updating hf models
* add relevant field to GgufVariantDetail
* make exception handling better
* add update_available flag for cached_models, and moved /update endpoint from inference -> models
* hook up /update endpoint on the frontend
* implement update scenarios for the model picker
* fix bug where downloaded flag for an older revision was being wrongly set to false
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix import and make hf calls async
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* remove has_vision from UpdateRequest
* fix ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* clear cancel event before updating gguf variant
* set _cancel_event back if it was set initially
* add hf_token to get_paths_info
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: harden model update endpoint and update checks
- update_hf_model: pass snapshot_download local_dir (local_path is not a
valid kwarg and 500s when updating bicodec audio models)
- get_gguf_variants: wrap the remote update check so a network, rate-limit,
gated, or offline failure degrades to "no update info" instead of failing
the whole variant listing, matching list_cached_models
- add regression tests for both paths
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: HF model update detection and Update action for cached models
Surface an "Update available" cue and a managed Update action for cached
on-device models. /api/hub/update-status compares each cached main GGUF
file's local blobs against the remote main revision using set membership
across all cached revisions, so a repo that was already updated (and still
holds the old snapshot alongside the new one) is not falsely flagged.
The Update action re-downloads through the download manager so it shows in
the Downloads panel with progress and cancel. The frontend wires the Update
button into the GGUF, on-device, and model-selector cards and keeps the
quant label fully visible when the action buttons crowd the row.
Adds regression tests for the multi-revision update check.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: accept force_download kwarg in hf_xet_fallback test double
The download seam now passes force_download to the attempt callable; the _FakeAttempt mock did not accept it, failing 6 tests with TypeError. Add the keyword (default False) so the scripted-results double matches the seam.
* Fix Studio model update regressions
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address Studio update review feedback
* Address Studio update edge cases
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Share GGUF update status helper
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix GGUF update detection and cache cleanup
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix cached GGUF update badges
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: shimmyshimmer <107991372+shimmyshimmer@users.noreply.github.com>
Co-authored-by: Etherll <61019402+Etherll@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
* (feat) Add project names to studio training runs to avoid models being overwritten when doing similar training runs
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update studio/frontend/src/features/export/export-page.tsx
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* Update studio/frontend/src/features/export/export-page.tsx
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* Update studio/frontend/src/features/export/export-page.tsx
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* better project name sanitization, removed duplicated project name normalization
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* implement checkpoint scanning utilities and tests for base model inference
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Guard project_name against null and use leading important modifiers
* Fix/adjust training project names for PR #6512
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix/adjust training project names for PR #6512
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address project-name review feedback
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Show project names in training recents
* Keep GGUF export directories source-specific
---------
Co-authored-by: NZ-Linix <nz-linix@outlook.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: NZ-Linix <linus.ordowski@outlook.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
* Fix offline checkpoint load/export failing with "tokenizer is weirdly not loaded"
Loading a fine-tuned checkpoint with no internet (e.g. a Studio export) crashed
with "Unsloth: The tokenizer is weirdly not loaded? Please check if there is one."
For a LoRA adapter the loader reassigns model_name to the base model repo id and
only keeps the local checkpoint dir as tokenizer_name when it contains
tokenizer_config.json, tokenizer.json AND special_tokens_map.json. Modern
tokenizers (e.g. Gemma) store special tokens inside tokenizer_config.json and
omit special_tokens_map.json, so tokenizer_name fell back to the base repo id.
The tokenizer/processor loads in vision.py then hit the Hub with no
local_files_only, so with no network they failed (AutoProcessor) or hung for
minutes (AutoTokenizer) even though every file was already cached.
loader.py: keep the local checkpoint dir as tokenizer_name when it has a
tokenizer config plus the actual tokenizer files (tokenizer.json / tokenizer.model
/ vocab files); special_tokens_map.json is no longer required.
vision.py: compute an effective local_files_only (explicit kwarg plus the
HF_HUB_OFFLINE / TRANSFORMERS_OFFLINE env vars, mirroring loader.py and
diffusion.py) and thread it through every AutoConfig, AutoProcessor,
AutoTokenizer and the manual VLM processor fallback, including the
hf_hub_download in that fallback (which now prefers a local file). When a load
fails and no offline env var is set, retry against the local cache. The retry
forces HF offline mode because local_files_only alone does not stop
AutoProcessor / AutoTokenizer from issuing a /api/models request during class
resolution. The final error now explains the offline/cache cause instead of the
misleading "weirdly not loaded" message.
studio export: probe Hub reachability once per checkpoint load and pass
local_files_only when offline so exports use the local checkpoint dir / cache
instead of hanging or crashing with no internet.
Online behavior is unchanged: the new flags default to off and the retry only
runs after a network related failure.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: safer offline forcing, cached fallback config, proxy-aware probe
Follow-up to the offline checkpoint load fix, addressing review feedback:
- vision.py: only flip the process-wide HF offline flag when offline is actually
requested (local_files_only / env) or after a real network failure, never
pre-emptively while we might be online. The flip is now guarded by a lock +
depth counter so nested or concurrent windows restore the flag correctly
(no stale value).
- vision.py: guard the get_auto_processor fallback so a network error there
returns None and the local-cache retry still runs instead of escaping.
- vision.py: in the manual VLM processor fallback, read tokenizer_config.json
via hf_hub_download(..., local_files_only=...) so a cached repo-id config is
still resolved offline and the model-specific image/video tokens are restored.
- studio export: make the reachability probe proxy aware (probe the configured
HTTP(S) proxy egress, honour NO_PROXY, use the endpoint port) so a proxy-only
setup is not wrongly marked offline; allow UNSLOTH_OFFLINE_PROBE=0 to disable.
- studio export: run the audio/vision type-detection probes inside the
forced-offline window when offline, so their config/tokenizer reads hit the
local cache instead of waiting out connection timeouts.
Online behavior remains unchanged.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: gate offline retry, safer tokenizer_name pop, skip audio net probe offline
- vision.py: only force the process-wide HF offline flag on the tokenizer
retry when offline was requested or the captured primary error is actually
network related, so a permanent tokenizer error no longer toggles global
offline mode for other concurrent loads.
- loader.py: always pop tokenizer_name out of kwargs and let a caller-supplied
value win, avoiding a "multiple values for keyword argument 'tokenizer_name'"
TypeError when it is also passed explicitly downstream.
- model_config.py / export.py: add local_files_only to detect_audio_type so the
raw requests.get tokenizer_config fetch is skipped offline (it ignores the HF
offline flag), and pass it from the export probe.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: classify LocalEntryNotFoundError as offline-related
huggingface_hub's LocalEntryNotFoundError subclasses FileNotFoundError, so the
"not isinstance(cur, FileNotFoundError)" guard in _is_offline_related_error was
swallowing it and it could never be recognised as offline, despite being listed
in the network error types. It means "not in cache and the Hub is unreachable",
which is genuinely offline. Capture the class into an isinstance-checkable tuple
(empty, hence a no-op, if the import is unavailable) and exclude it from the
FileNotFoundError guard, so a real offline failure now triggers the local-cache
retry while a plain missing-file error still propagates.
* Address review: require merges.txt for BPE, status-gate HTTP errors, isolate local-only audio cache
- loader.py: a local dir with vocab.json but no merges.txt (and no tokenizer.json)
is not a loadable BPE tokenizer, so do not treat it as self-sufficient; require
merges.txt alongside vocab.json in both gate blocks, otherwise fall back to the
base model tokenizer as before.
- vision.py: _is_offline_related_error no longer buckets every HfHubHTTPError /
requests HTTPError as offline. HTTP errors are judged by status code: only a
transient 5xx triggers the forced local-cache retry, while 401/403 (auth/gated)
and 404 (missing) propagate as the real error instead of being masked. Hard
signals (connection/timeout/OfflineModeIsEnabled/LocalEntryNotFoundError) still
classify as offline.
- model_config.py: include local_files_only in the audio-detection cache key so a
local-only (offline) negative result cannot be reused by a later online probe,
which would otherwise route an audio model through the text loader until restart.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address re-review: fix studio test stubs, force offline env in probe window, drop redundant retry
- studio/backend/tests/test_vision_cache.py: the three _detect_audio_from_tokenizer
stubs were called with the new local_files_only kwarg and raised TypeError, failing
Backend CI. Add local_files_only to the stub signatures and add a test that a
local-only negative does not poison a later online audio probe.
- export.py: the type-detection probe window now also sets HF_HUB_OFFLINE /
TRANSFORMERS_OFFLINE env vars (saved/restored), not just the in-process flag.
transformers_version._load_config_json / _check_tokenizer_config_needs_v5 gate
their urllib fetches on the env vars, and is_vision_model may spawn a subprocess
that inherits os.environ but not the in-process flag; without the env vars a
probe-detected offline export could still block on a network timeout.
- vision.py: only retry the processor load when the first attempt was online and
failed with a network error. When local_files_only was already requested the first
attempt was forced offline, so the previous retry just repeated identical failing
work before the last-resort path.
- model_config.py: correct the _audio_detection_cache type annotation to the 3-tuple
key (name, token_fingerprint, local_files_only).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: thread-safe probe-offline env window, clear error for local dir without config
- export.py: guard the HF_HUB_OFFLINE / TRANSFORMERS_OFFLINE mutation in
_force_offline_probe_window with a lock + depth counter (mirrors _force_hf_offline),
so concurrent / nested export probes only flip on first entry and restore on last
exit. This prevents overlapping export requests from permanently poisoning those
env vars or restoring a stale value.
- vision.py: in the VLM processor fallback, when tokenizer_name is a local directory,
read its tokenizer_config.json directly and raise a clear FileNotFoundError if it is
absent, instead of handing the local path to hf_hub_download (which would treat it as
a repo id and raise a confusing HFValidationError / RepositoryNotFoundError).
hf_hub_download is now only used for actual repo ids.
* Address review: classify raw socket.gaierror DNS failures as offline
Add the platform-specific getaddrinfo / DNS-resolution wording to the offline
detection list in _is_offline_related_error so a bare socket.gaierror (an OSError
subclass) is recovered from the local cache: "Name or service not known" and
"Temporary failure in name resolution" (Linux) and "nodename nor servname
provided" (macOS). Genuine non-network OSErrors (disk full, permission denied)
and plain FileNotFoundError still propagate.
* Address review: retry degraded VLM offline, force offline for text export + patch-tokenizer fallback
- vision.py: a degraded VLM processor (text-only, no image_processor) whose manual
fallback fails offline used to be kept, so image inputs broke even with cached
files. _construct_vlm_processor_fallback now returns its failure error;
_acquire_processor surfaces it, and the caller retries forced-offline when the
result is None OR a degraded VLM and the failure was network related, keeping the
original result if the retry is not strictly better (never regress). The retry is
still gated on an online first attempt + offline-related error so a permanent
error never flips the global offline flag.
- vision.py: wrap the patch_tokenizer except-branch AutoTokenizer.from_pretrained in
the same forced-offline-on-network-error pattern as the primary / last-resort
loads, so an offline export where patch_tokenizer raises does not hang or fail.
- export.py: force HF offline around the two FastLanguageModel loads (text and SNAC)
when the probe detected offline. Their text tokenizer path (load_correct_tokenizer
-> AutoTokenizer) does not forward local_files_only, so without this a text export
could still contact the Hub. Added a small _offline_window_if helper reused by the
probe and load windows.
* Consolidate offline loading into one entry-point decision
Decide offline once per entry point instead of at every HF call site. The
prior approach threaded local_files_only into ~15 scattered config / tokenizer
/ processor / weight loads, each wrapped in its own try-online, classify-error,
retry-forced-offline dance, which is what kept surfacing "another call site you
missed", "another error shape misclassified", and global-flag thread-safety in
review.
FastLanguageModel / FastModel / FastBaseModel.from_pretrained now share an
@_offline_aware_load decorator: when offline (explicit local_files_only kwarg or
HF_HUB_OFFLINE / TRANSFORMERS_OFFLINE env) it sets local_files_only and runs the
whole load inside one _force_hf_offline() window so every nested HF call inherits
it; when online it runs normally and, only if the load fails with a genuinely
network-related error, retries once forced-offline. The online path is unchanged
(no probe added) and 401 / 403 / 404 / permanent errors still propagate.
Centralise the offline helpers in loader_utils.py as the single source of truth
(shared by loader.py, re-exported from vision.py, and reused by the Studio
exporter):
- _force_hf_offline now sets the HF_HUB_OFFLINE / TRANSFORMERS_OFFLINE env vars
AND the in-process huggingface_hub / transformers flags, refcounted under one
lock so nested / concurrent windows restore correctly. Setting the env vars
covers env-gated urllib probes and spawned subprocesses too.
- _get_effective_local_files_only, _is_offline_related_error (unchanged
classifier, retains the 5xx-vs-4xx, LocalEntryNotFound and gaierror handling),
_offline_aware_load, and _resolve_checkpoint_tokenizer_name.
loader.py: wrap both entry points; drop the two duplicated env-var fallback
blocks and the two byte-identical local-tokenizer-gate blocks (now
_resolve_checkpoint_tokenizer_name).
vision.py: drop the per-site force_offline params and the three retry gates
(processor, patch_tokenizer fallback, last-resort). They now just surface the
underlying error so the single entry-point safety net retries forced-offline. A
network fallback error now takes precedence over a permanent primary error so the
offline retry still fires when the manual VLM fallback needs cached repo files.
studio/backend export.py: reuse the unified core _force_hf_offline (env + flags)
and drop the duplicate probe-window primitive; the snac / text branches no longer
need their own window. model_config.py: also gate the raw requests.get audio
fallback on the HF offline env vars so it is covered even without the kwarg.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address 10-reviewer P1 findings: vision cache split, PEFT offline, retry OOM
Split the Studio vision-detection cache by local_files_only, mirroring the audio
cache fix. is_vision_model / _is_vision_model_uncached / _raw_config_has_vision_config
/ load_model_config now thread local_files_only, the cache key includes it, and the
exporter passes it. Offline detection also skips the transformers-5 network
subprocess and stays on the local cache, so an offline negative can no longer be
keyed under the online entry and poison a later online probe. Adds a regression
test mirroring the audio poison test.
Forward local_files_only to both PeftModel.from_pretrained adapter-attach sites in
loader.py so a cached remote LoRA adapter resolves from the local cache under
explicit local-only / offline loads (defence-in-depth alongside the forced-offline
window).
_offline_aware_load: run the forced-offline retry OUTSIDE the except block and
collect + empty the device cache first. An except-scoped exception keeps its
__traceback__, which pins the failed attempt's frame locals (a partially loaded
model) until the block exits; loading the model again while that copy is still
alive could OOM a large VLM. Letting the except block close drops the traceback so
the partial load is freed before the retry reallocates.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address Codex review: env-offline cache key + rebuild HF sessions in offline window
Key the Studio audio and vision detection caches on the EFFECTIVE offline state
(local_files_only OR the HF offline env vars), not just the kwarg. detect_audio_type
and is_vision_model both skip the remote fetch / network subprocess when
HF_HUB_OFFLINE / TRANSFORMERS_OFFLINE is set even with the default
local_files_only=False, so the result reflects offline; storing it under the online
(False) key let an env-offline negative poison a later online lookup once the env var
was cleared. Both now compute effective_offline once and use it for the cache key and
the downstream call. Adds a regression test for the env-offline dimension.
_force_hf_offline now rebuilds huggingface_hub's cached sessions on enter and exit
(best-effort _reset_hf_sessions). On hub 0.x the offline adapter is baked into the
per-thread requests.Session at creation, so flipping the constant alone leaves an
already-cached online session able to hit the network inside the window (and an
offline one stuck offline after restore); resetting forces the next get_session() to
match the current flag. On hub 1.x offline is checked dynamically per request, so
reset_sessions does not exist and the helper is a safe no-op.
The third review point (release the failed load before retrying) was already fixed in
af0f58a: the forced-offline retry now runs outside the except block and frees the
device cache first, so the failed attempt's traceback-pinned partial model is
released before the retry reallocates.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Align Studio _env_offline parsing with the canonical offline helper
model_config._env_offline gates the raw requests.get tokenizer-config fallback in
detect_audio_type and the audio/vision detection cache keys, but it only accepted
unstripped "1"/"true"/"yes". unsloth's offline helpers (loader_utils._env_says_offline
and the from_pretrained env fallback) accept the canonical set {1,true,yes,on} after
strip + lowercase, so HF_HUB_OFFLINE=on or HF_HUB_OFFLINE=" 1 " was treated as offline
by the loaders but online here, leaving the raw network fetch reachable while
"offline". Use the same strip + lowercase {1,true,yes,on} set. Adds parsing tests.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix lint: drop dead offline-helper re-exports from vision.py
The import-hoist verifier (scripts/verify_import_hoist.py) flagged vision.py's
re-export block as HOISTED-IMPORT-UNUSED blockers: it imported eight offline
helpers from loader_utils but only used three internally
(_get_effective_local_files_only, _is_offline_related_error, _offline_aware_load).
The other five were imported purely to preserve `from unsloth.models.vision import
X`, but nothing imports four of them from vision, and loader.py already imports
_resolve_checkpoint_tokenizer_name straight from loader_utils.
Import only the three names vision.py actually uses, and point the Studio exporter
at the canonical source (from unsloth.models.loader_utils import _force_hf_offline)
instead of re-exporting it through vision. loader_utils stays the single source of
truth; no behaviour change.
* Address Opus review: chain probe errors, unify env-offline, status-less HTTP
Chain the original AutoConfig/PeftConfig probe exception into the combined
RuntimeError in both FastLanguageModel.from_pretrained and FastModel.from_pretrained
(`raise RuntimeError(combined_error) from (autoconfig_exc or peft_exc)`). The probes
caught every Exception and stringified it, so the re-raised RuntimeError had no
__cause__/__context__ and _is_offline_related_error could not classify it -- the
network-down-but-cached auto-retry never fired for these entry points. With the
cause chained, the decorator sees a ConnectionError/LocalEntryNotFoundError/5xx and
retries forced-offline from cache; a permanent cause (404 / bad config) is still not
offline-classified and propagates without a wasted retry.
Unify the third offline-env parser: studio/backend/utils/transformers_version._env_offline
now uses the canonical {1,true,yes,on} + strip + lowercase set (matching
loader_utils._env_says_offline and model_config._env_offline), so HF_HUB_OFFLINE=on
or " 1 " no longer leaks the direct urllib metadata fetches to the network.
_is_offline_related_error: a status-less HTTP error (no response / unparseable code)
now falls back to the network-wording check instead of being dropped, so a transient
HTTP failure with clear "couldn't connect" wording is treated as offline. HTTP errors
with a real status code still decide by code (4xx propagates, 5xx is offline).
* Condense offline-loading code comments, drop dead helper, dedupe import for PR #6554
* Add unit tests for offline-loading helpers for PR #6554
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Guard load cleanup with try/finally and add retry-contract tests for PR #6554
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add gc.collect retry-step test for PR #6554
* Tighten offline-loading comments and docstrings for PR #6554
* Raise the both-config-failed error before model-type lookup so offline retry fires for PR #6554
* Prefer offline cause for retry and bound export reachability probe for PR #6554
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Skip remote mapper while offline, harden text-load cleanup, and stop stacked offline retries for PR #6554
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Surface VLM fallback offline errors, probe offline before export version activation, and restore progress bars across retries for PR #6554
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Restore offline env after export version activation so the persistent worker re-decides per load for PR #6554
* Classify socket.gaierror and urllib URLError as offline by type for PR #6554
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Probe offline around export load preflights and never offline-retry TLS failures for PR #6554
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Force in-process offline for export preflights, verify proxy egress in probe, and skip caching offline version negatives for PR #6554
* Snapshot offline constants before forcing env and require local processor files for VLM checkpoints for PR #6554
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: require signed capability tokens for /p preview links
The public /p preview routes added in #6486 run model load and chat
generation as the admin user with no authentication. The only gate is the
preview ref, a deterministic outputs-root path (run or run/checkpoint) that
is guessable rather than secret. On a network-reachable Studio (--secure
tunnel or -H 0.0.0.0), an unauthenticated caller who guesses a ref can
consume GPU and probe a private fine-tuned checkpoint.
Make the share link an unguessable, revocable capability:
- Sign the canonical ref with a dedicated server-side secret (HMAC-SHA256,
stored in app_secrets, independent of the JWT/login secret).
- Require a valid token on every /p chat, models, and page request before
resolving a checkpoint or loading a model; missing or invalid tokens get a
generic 404 so the surface never confirms a ref exists.
- Accept the token via ?k= (browser link and preview page) or
Authorization: Bearer (OpenAI-compatible clients).
- Rotate the secret to revoke every outstanding link
(POST /api/settings/preview-links/rotate).
- Clamp preview generation (max_tokens/max_completion_tokens <= 1024, n = 1)
and set Referrer-Policy: no-referrer on the page so the token is not
leaked via Referer.
Training history hands the authenticated owner the signed token, and the
copy-link button builds /p/{ref}?k={sig}.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: honor a lower caller token limit in the preview clamp
Codex review: when only the legacy max_tokens was sent, the clamp left
max_completion_tokens at the 1024 default, and _effective_max_tokens prefers
max_completion_tokens, so a request like max_tokens=16 could still generate up
to 1024 tokens. Derive one effective limit (max_completion_tokens wins, else the
legacy max_tokens) and pin both fields to it so a caller's lower limit is kept.
* Studio: add preview kill switch, rate limit, and revoke-links UI
Follow-ups to the /p preview capability work:
- Public-sharing kill switch: a persisted setting (default on) gates the public
/p surface. When off, every preview request 404s even with a valid token, and
the owner UI stops offering share links. GET/PUT /api/settings/preview-sharing;
enforced in _verify_or_404.
- Per-IP rate limit on the preview chat route: a coarse in-process sliding-window
limiter (20 req/min/IP) returns 429 + Retry-After before the GPU lock is taken.
Client IP honors X-Forwarded-For only when UNSLOTH_STUDIO_TRUST_FORWARDED is
set, matching the login limiter's trust model.
- Settings UI: a "Preview sharing" section with the public-sharing toggle and a
"Revoke all preview links" button (confirm dialog) that rotates the secret.
Tests cover the kill switch (404 when off), the 429 path, the sliding window,
client-IP trust behavior, and the setting default.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: fix preview-fields sharing arg and refresh sigs after revoke
Codex review:
- P1: get_training_run_detail and update_training_run called _preview_fields
with only output_dir after it gained a required sharing_on parameter, raising
a 500 TypeError once get_run succeeded. Pass get_preview_sharing_enabled() at
both sites; add a detail-endpoint regression test.
- P2: after rotating the preview secret from settings, the history grid still
held stale preview_sig values, so a freshly copied link would 404. Emit
emitTrainingRunsChanged() after a successful revoke so the grid refetches
freshly signed refs.
* Studio: harden preview sharing controls (Codex review)
- Fail closed: a read failure on the preview-sharing kill switch now returns
False instead of defaulting to enabled, so an unavailable settings DB can't
reopen the public surface. A missing key still defaults to enabled.
- Per-IP rate limit behind the managed Cloudflare tunnel: client_ip now honors
CF-Connecting-IP when the socket peer is loopback, so tunneled visitors are
keyed by their real IP instead of collapsing onto the local cloudflared peer.
- GET /p no longer mints key/share_url when sharing is disabled; it returns
sharing_enabled=false so clients don't distribute links that 404.
- Settings UI: toggling public sharing emits the training-runs-changed event so
the history grid shows/hides Copy preview link without a manual refresh.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: harden preview rate limiter and IP keying (Opus review)
From a two-agent review of the PR:
- Rate limiter no longer evicts an active bucket when the table is full: a flood
of distinct keys could otherwise cycle out a throttled bucket and reset its
counter. Evict only aged-out buckets; if the table is full of live clients,
fail closed (deny the new key) instead.
- client_ip keys on the rightmost (proxy-appended) X-Forwarded-For hop when the
trust env is set; the leftmost is client-spoofable. Documented the
append/overwrite-proxy assumption.
- _verify_or_404 checks the capability token before the kill-switch DB read, so
unauthenticated /p spam can't be used as an unbounded settings-DB sink and the
response is identical regardless of the sharing on/off state.
Tests: nested run/checkpoint happy path + wrong-ref rejection, the eviction
fail-closed behavior, and route-level coverage for the rotate / preview-sharing
settings endpoints.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* checkpoint preview endpoint
* harden new preview endpoints
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* address review
* Studio preview: pin adapter, guard streaming submit, robust copy-link
Harden the public per-checkpoint preview surface:
- Pin use_adapter=True in the preview payload sanitizer. Otherwise an
unauthenticated /p caller can POST use_adapter=false, which calls
disable_adapter_layers() on the shared in-memory model without restoring
it; since load_model skips reloads for the same checkpoint, every later
visitor (the page never sends the field) keeps getting base-model output
instead of the fine-tuned checkpoint. Forcing it on also re-enables a
previously disabled adapter and no-ops on merged checkpoints.
- Ignore preview-page submits while a response is streaming. The send
button was disabled but the Enter handler still called requestSubmit(),
so a second request could start before the first reply landed in msgs and
reorder the chat history. Both the keydown and submit handlers now honor
the disabled button.
- Keep the cloudflare-URL polling loop alive across transient startup fetch
errors instead of letting one rejection halt it.
- Build the copy-link from a backend preview_ref (output dir relative to
outputs_root, gated on previewability and the two-segment /p route limit)
so a nested output dir no longer copies a basename-only link that 404s.
Expose preview_ref on training run summaries.
Add route-level security tests (path traversal, payload sanitization,
asset containment, CSP header, HTML title escaping, streaming lock held
until drained) and preview_ref unit tests.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio preview: Safari-safe submit and adapter pin only for LoRA
Follow-ups from cross-browser and route simulations:
- Preview page: send the message from a shared send() helper called by both
the form submit and the Enter key, instead of form.requestSubmit(). The
latter throws on Safari < 16 and older iOS, which broke Enter-to-send there.
Verified across Chromium, Firefox and WebKit with Playwright.
- Only pin use_adapter=True when the resolved checkpoint is a LoRA adapter
(adapter_config.json present); for a merged checkpoint strip it to None.
A merged model has no adapter to toggle, so forcing it on only produced a
per-request "not a PeftModel" warning. The cross-request base-model
contamination fix still holds for LoRA previews.
Add a merged-checkpoint test asserting use_adapter is stripped to None.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio preview: trim verbose comments
Tighten comments across the preview routes, page, checkpoint helpers, and tests
to short single-line notes; drop ones that just restate the code. No behavior
change (verified comment/docstring-only with comment_tools.py check).
* Harden preview routes for PR #6486
- Return a generic 400 detail on a rejected preview path so the public /p
route never echoes the absolute install path (the real reason is logged
server-side instead).
- Strip confirm_tool_calls, session_id and rag_scope in the preview payload
sanitizer so the public surface stays inert regardless of the tool gate.
- Use Path.is_relative_to for the asset containment check, matching the rest
of the codebase.
- Add img-src 'self' and font-src 'self' to the preview page CSP.
- Preview page: on a mid-stream error keep the streamed text, flag the break,
and restore the prompt so the user can retry; drop the unused --font-sans var.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Copy uv `-c`/`-r` paths to a space-free temp dir on macOS/Linux
uv 0.11.x truncates a constraints/requirements path passed via `-c`/`-r`
at the first space, so `unsloth studio` setup from a repo cloned under a
path containing a space (e.g. `/Users/me/Open Source/unsloth`) fails with:
error: File not found: `/Users/me/Open`
_uv_safe_path() already worked around this on Windows via the 8.3 short
path but returned the space-containing path unchanged on macOS/Linux,
which have no 8.3 equivalent. Extend it to copy the (small, flat)
constraints/requirements file into a space-free temp dir and hand uv the
copy; the temp dirs are removed at process exit. Falls back to the
original path on any error, so it is never worse than before.
Refs unslothai/unsloth#6503
* Route UV_OVERRIDE through _uv_safe_path and fix temp-dir leak (#6503)
The -c/-r fix did not cover UV_OVERRIDE, which uv also truncates at the first
space. On Apple Silicon the overrides file is handed to uv via UV_OVERRIDE at
install time (install_python_stack.py) and during the MLX self-heal
(utils.mlx_repair), so a repo under a path containing a space still broke every
uv call there. Move _uv_safe_path into backend.utils.uv_path_safety so both
sites share it, and route UV_OVERRIDE through it.
Also stop leaking the temp dir when shutil.copyfile fails after mkdtemp, and add
tests for the UV_OVERRIDE channel, the TMPDIR-with-space fallback, the atexit
cleanup, and the no-leak path.
---------
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
* Harden MLX self-heal install against supply-chain execution
The Apple Silicon MLX self-heal runs uv pip install on a daemon thread
during Studio startup, default-on with only an env opt-out, before the
post-install stack check. Two things widened the supply-chain surface:
- it accepted source distributions, whose PEP 517 build backends run
arbitrary code at install time; and
- it forwarded the full process environment, exposing Studio secrets to
that code and letting a poisoned env (UV_FIND_LINKS / UV_DEFAULT_INDEX)
repoint the install at a hostile source.
Require pre-built wheels (--only-binary=:all:) and forward only an
allowlist of variables uv needs (PATH/HOME, proxy + CA settings, cache
dir), setting UV_OVERRIDE ourselves. mlx/mlx-metal ship wheels only and
mlx-lm/mlx-vlm publish py3-none-any wheels, so a healthy self-heal is
unaffected; an unavailable wheel just leaves Studio chat-only as before.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Drop cache-dir env vars from the self-heal allowlist
Address review: a poisoned process env could set UV_CACHE_DIR / XDG_CACHE_HOME
to redirect uv at an attacker-staged cache (cache poisoning, symlink writes),
which partly undercut the index-redirect protection. Drop them from the
allowlist; uv falls back to its safe user-owned default cache, still reused
across runs, so there is no normal-path cost. Test now asserts both are
excluded from the install env.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: start the Cloudflare tunnel for --secure even in --api-only, and add --api-only to `unsloth studio run`
--secure exposes ONLY the Cloudflare link (it forces a loopback bind), but
_cloudflare_tunnel_should_start gated the tunnel on `not api_only`, so
`run.py --secure --api-only` started no tunnel and then fail-closed with
"A secure Cloudflare link is not allowed". That blocked the natural headless
use: serve just the API (no web UI) over the authenticated tunnel.
Make --secure start the tunnel regardless of api_only (the non-secure path is
unchanged: tunnel only a 0.0.0.0 bind, never api-only Tauri or Colab). Then
expose --api-only on `unsloth studio run` and forward it through both the
re-exec args and the in-venv run_server call, so
`unsloth studio run --secure --api-only --model ...` is a one-liner secure API
server.
Verified end to end: `run.py --secure --api-only` now brings up the tunnel and
serves /api/health over it (200), with / returning 404 (no UI).
Tests: update the tunnel-gate truth table (secure+api-only now tunnels;
secure+colab still does not) and add --api-only registration + re-exec/in-venv
forwarding coverage to the run CLI tests.
* Trim comments to be succinct (no behavior change)
* studio: address review on parent --api-only and secure api-only CORS
- Reject --api-only on the parent `unsloth studio` group when a subcommand
is invoked, with the same redirect guidance used for --parallel/--secure;
otherwise the flag was silently dropped and the UI served anyway.
- Keep CORS any-origin for secure api-only serving: that mode publishes the
API over Cloudflare for remote browser clients, so the Tauri-only lockdown
(still applied to plain local api-only) would break preflight. Factored the
decision into cors_origins_for_mode() and gate it on api_only and not secure;
run_server exports UNSLOTH_SECURE before importing main.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: suppress TAURI_PORT and de-dup test for headless run --api-only
- run_server gains emit_tauri_port (default True, unchanged for the Tauri/
desktop path). The new headless `run --api-only` path passes False so the
Tauri-only TAURI_PORT= line no longer prepends the documented URL/API key
banner (it ran even under --silent and could break one-liner parsers).
- Remove a duplicate test_reexec_forwards_api_only that shadowed the
parametrized one; fold the --secure --api-only case into it so the secure
headless path is actually collected.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix(studio): sort export checkpoints by step
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
* Studio: honor custom HF_HOME for model download and load
_setup_cache_env always derived HF_HUB_CACHE and HF_XET_CACHE from
XDG_CACHE_HOME / ~/.cache, ignoring a user-set HF_HOME. Because it sets
HF_HUB_CACHE explicitly and that variable takes precedence over HF_HOME
in huggingface_hub, the hub cache was pinned to the standard location: a
model already present under a custom HF_HOME was detected but then
re-downloaded from scratch on load.
Seed HF_HUB_CACHE and HF_XET_CACHE from HF_HOME when the user set it
(HF's own default is $HF_HOME/hub and $HF_HOME/xet), and honor the legacy
HUGGINGFACE_HUB_CACHE alias. The hub download workers call
snapshot_download without a cache_dir for both the Xet and HTTP-fallback
paths, so they follow HF_HUB_CACHE; fixing it here unifies detection and
both transports on one root. Explicit HF_HUB_CACHE / HF_XET_CACHE stay
untouched. Adds tests for the custom-HF_HOME, default, explicit-override,
and legacy-alias cases. Fixes#5182.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: do not crash startup when a custom HF_HOME is not writable
Seeding HF_HUB_CACHE/HF_XET_CACHE from HF_HOME means _setup_cache_env now
mkdir's under a user-controlled path. A non-writable or not-yet-mounted
HF_HOME (typo, offline drive) would raise and crash startup, where the old
code silently fell back. Make the mkdir best-effort; the env var is still
set, so HF reports a clear error at download time. Adds a regression test.
* Studio: strip blank HF_HOME and isolate cache-env tests
Address review: a whitespace-only HF_HOME no longer derives " /hub";
strip it and fall back to the default (matches studio_root). Tests set
UNSLOTH_STUDIO_HOME to a tmp dir so _setup_cache_env's UV/VLLM mkdirs do
not touch the real ~/.unsloth/studio. Adds a whitespace regression test.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Resolve the transformers tier by probing AutoConfig instead of guessing
When the only signal is a 5.x tokenizer class, get_transformers_tier guessed the
lowest 5.x sidecar (530). That misroutes models whose built-in config parser needs
a higher tier: dense NemotronH ships a 5.x tokenizer but its '-' (MLP) layer only
transformers 5.10 can parse, so 5.3/5.5 raise KeyError '-'. The config.json
transformers_version field records the saving version, not the minimum to load, so
it cannot drive routing either.
Replace the weak tokenizer->530 guesses (local and remote) with a probe: parse
config.json with the built-in parser (trust_remote_code=False) in each sidecar,
escalating 530->550->510, and pick the first that succeeds. This generalizes to any
architecture without hardcoded lists. Strong signals stay fast paths (no subprocess);
the probe runs only when the tier is otherwise ambiguous and is cached by (model,
commit sha). It never executes repo code, never downloads weights, never raises, and
falls back to the legacy 530 guess on a transient/auth/offline failure or when no
sidecar is available. UNSLOTH_DISABLE_TIER_PROBE restores the old behavior.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: tier probe fallbacks and cross-platform robustness
Codex:
- Never escalate to 510 on uncertainty. When every sidecar was probed and none
parsed with the built-in parser, the model is a remote-code / custom model_type
that loads via its own code; keep the legacy 530 route instead of jumping to
510 (which would change the behavior of models that worked on the 5.3 stack).
- Only cache the 530 fallback when the result is conclusive (every tier actually
probed). If a sidecar was missing/uninstallable the environment is incomplete,
so return 530 uncached and retry on the next call.
- Do not pin the tier cache under an unknown revision: _resolve_commit_sha no
longer memoizes a None sha (a transient Hub failure is retried), and _probe_tier
only caches a tier when the commit sha is known.
Gemini:
- Wrap Path.exists() in the sha resolver in try/except OSError (a remote repo id
can raise WinError 123 on Windows).
- Probe script writes the error to sys.stderr.buffer as UTF-8 bytes so a non-ASCII
message cannot itself raise UnicodeEncodeError under cp1252.
- subprocess.run decodes stderr with errors="replace" to avoid UnicodeDecodeError
on non-UTF-8 consoles.
Tests: 72 passed (added partial-sidecar uncached, sha-unresolved not cached,
all-failed stays 530 + cached, sha resolver retries None / handles OSError).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review round 2: authenticate tier checks, stop memoizing local sigs
Codex:
- Thread hf_token through _check_config_needs_510/550 and
_check_tokenizer_config_needs_v5 (and the underlying raw fetches). Previously a
gated/private model whose only 5.x signal is tokenizer_config.json never reached
the authenticated probe: the unauthenticated raw fetch failed and cached False,
so the model fell through to the default 4.x tier. The per-check caches are now
keyed by (model, token) so an unauthenticated miss cannot poison a later authed
read, mirroring _load_config_json.
- _resolve_commit_sha no longer memoizes a local directory signature. A local
signature is mutable (size/mtime of config/tokenizer), so a reused/overwritten
checkpoint path would otherwise keep selecting the previous tier; it is now
recomputed every call. Only the immutable remote commit sha is memoized.
Tests: 75 passed (added token-cache isolation + auth header, local signature not
memoized, token threaded into all checks/probe).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review round 3: reach activation with the token, drop SHA tier cache
Codex round 3:
- Thread hf_token into the activation path that actually selects a sidecar. The
token-aware tier checks added last round were unreachable:
activate_transformers_for_subprocess called get_transformers_tier without a
token, and the inference/training/export workers passed only the model name even
though they hold a request-scoped hf_token. activate_transformers_for_subprocess
now takes hf_token and the three workers forward config["hf_token"], so a
gated/private model whose only 5.x signal is an authenticated config/tokenizer is
routed to the right sidecar instead of falling to default 4.x.
- Stop importing huggingface_hub during tier detection. _probe_tier no longer
resolves a commit sha, so it never pulls huggingface_hub into the worker before
the sidecar venv is prepended to sys.path (activation only prepends, never
purges), which would otherwise pin the default-env hub over the sidecar's
pinned huggingface_hub==1.8.0.
- The tier cache is now keyed by model_name for the process lifetime (a model's
required tier is a property of its architecture; cleared on restart). This drops
the mutable-SHA memo that masked remote revision changes and the mutable
local-signature memo, removing _resolve_commit_sha / _local_dir_signature /
_probe_sha_cache entirely.
- Do not cache a probe success that depended on a skipped lower tier: if a lower
sidecar was unavailable, the lowest valid tier may change once it installs, so
the result is returned uncached and re-probed next call.
Tests: 73 passed (probe imports no hub; success uncached when a lower tier is
skipped; activation forwards the token).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Trim comments to be more succinct
* Re-probe overwritten local checkpoints and authenticate the probe child
The AutoConfig tier probe cached its result under the bare model_name, so a
local checkpoint overwritten in place (same path, new config.json) kept serving
the stale sidecar. Fold a cheap config.json signature (size + mtime) into the
cache key for local paths; remote ids stay name-keyed so no huggingface_hub
import lands before the sidecar is activated.
The probe relies on the implicit HF_TOKEN env, so an inherited
HF_HUB_DISABLE_IMPLICIT_TOKEN=1 left it unauthenticated and a gated repo 401ed
into the 530 fail-safe. Clear that flag in the child env when a token is set.
* Keep tier probes off the log-only path and probe new 5.x archs default-first
- get_transformers_tier gains probe=True/False. needs_transformers_5 (a coarse
4-vs-5 boolean used only for a spawn log and a vision-check branch) now passes
probe=False, so a parent/log-only caller never spawns sidecar probes. The real
activation path keeps probe=True and resolves the exact tier in the worker.
- A config.json saved by transformers 5.x but matched by no fast path is now probed
default-first: _probe_tier gains include_default + floor, prepending the ambient
4.57.x tier to the escalation. A model that still parses on the default is left on
it (no mis-route onto a sidecar); only a config the default parser cannot read
escalates to the lowest 5.x tier that parses. The transformers_version field is a
cheap 'worth probing' hint only, read from the already-fetched config (no extra
network); ordinary 4.x configs never probe.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Separate probe cache by mode and keep version-field 5.x visible to needs_transformers_5
- _probe_tier cache was keyed only by config.json signature, so a default-first probe
that returned 'default' could be handed back to a later tokenizer/known-5.x caller
(floor=530), leaving a model with a 5.x-only tokenizer on transformers 4.x. Key the
cache by probe mode (floor + include_default); the legacy 530 mode keeps the bare key.
- The version-field 5.x detection is a cheap config read, not a probe, so run it even
when probe=False: a standard-tokenizer model whose only signal is transformers_version
>= 5 now classifies as 5.x via needs_transformers_5 (returns '530' without spawning a
probe), so the vision-routing fallback uses the 5.x subprocess instead of failing the
default parser and marking it non-vision. The real activation path still probes
default-first and may resolve 'default'.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Don't treat local checkpoints as Hub ids, and fix stale activation test double
- _load_config_json / _check_tokenizer_config_needs_v5: a local checkpoint dir whose
config.json / tokenizer_config.json is not yet present was being fetched from the Hub
as if the path were a repo id, and the 404 miss was cached. A later call after the
file is written (in-progress checkpoint) then served the stale miss, so a
TokenizersBackend checkpoint fell through to the default tier. Skip the Hub fetch for
local dirs and do not cache the miss, so the file is read once it appears.
- test_activate_transformers_version_or_warn_*: the worker now threads hf_token into
_activate_transformers_version (model_name, hf_token); update the one-arg test doubles
to the real two-arg signature so the silent-success path stays silent.
* Tighten comments in the AutoConfig probe and tier-selection paths
* Address review: canonical probe cache key and reuse _token_cache_key
- _probe_cache_key resolves config.json to its absolute realpath before
keying, so a relative path or a changed cwd can't collide with or miss a
prior probe result. Remote ids still fall back to the name (stat raises,
caught).
- _cached_config_json reuses _token_cache_key instead of re-hashing the
token inline, keeping the (model, token) key derivation in one place.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Add HF dataset streaming mode to Studio
* Added default value for datasetStreaming in training-config-store.ts
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Handle None max_steps for streaming validation
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: fast-fail streaming validation and guard incompatible modes
Reject dataset_streaming at the API boundary when hf_dataset is empty,
the dataset is vision/audio, or max_steps is not set. Probe eval split
with get_dataset_split_names before the streaming load so typos fail
immediately instead of mid-training. Guard column_names=None after map
on iterables. Hide the UI toggle for non-text configurations and clear
the stale flag when config becomes incompatible.
* studio: add streaming dataset tests, iterable helper, and streaming template/format support (WIP)
Work-in-progress on top of feat/studio-dataset-streaming-mode (PR #4946):
- new test_training_streaming.py and iterable.py dataset helper
- streaming support in chat_templates.py and format_conversion.py
- additional streaming guards in trainer.py / models / routes
- frontend streaming wiring in params-section and training-config-store
Committed to preserve uncommitted work before merging latest main.
* studio: fix review-team findings for streaming + main merge
BLOCKER: streaming + raw-text/CPT crashed on len(IterableDataset). Guard it in the
start route (reject format_type=="raw" or training_type=="Continued Pretraining")
and in isStreamingSupported (datasetFormat !== "raw").
Also:
- models/training.py: validate hf_dataset/subset/split (charset+length, block ..//);
cap dataset slice indices (le=1e9); note validator ordering
- chat_templates.py: guard _apply_custom_mapping .map() for streaming
- trainer.py: warn when packing+streaming
- training-config-store.ts: persist-migration bump to v11 (standalone datasetStreaming
backfill); add isVisionModel to NON_PERSISTED; toast on silent streamingCompatiblePatch
mutations in the 4 indirect setters
- tests: route rejections (max_steps, raw/cpt), slice cap, unsafe hf_dataset
* studio: enable raw-text/CPT dataset streaming + streaming UX polish
- raw_text: keep the lazy filter but skip len()-based row counting for
IterableDatasets so raw-text / CPT can stream; guard the eval-size log
- routes/trainer: drop the raw/CPT streaming block; add a defensive
not-streaming guard on the eval auto-split (train_test_split)
- dataset-section: streaming toggle is visible-but-disabled and lists the
exact unmet requirement(s) in its tooltip; block embedding models
- training-start-overlay: show "streaming (no full download)" instead of a
stuck download bar for streaming runs
- trim the streaming test suite to the high-value cases
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: address streaming review (MLX/embedding guards, sliced eval split, rehydrate timing)
- routes: reject dataset_streaming for embedding training and on Apple Silicon
(MLX); both loaders materialize the full dataset instead of streaming
- trainer: validate the base eval split name so streaming eval accepts HF slice
syntax such as "validation[:1000]"
- training-config-store: defer the onRehydrateStorage setState to a microtask so
it doesn't hit the store's TDZ during synchronous hydration
- test: streaming start rejects embedding models
* studio: harden HF dataset streaming (column_names, split slicing, empty/eval bounds, gating)
Address a deeper streaming review:
- raw_text: resolve_column_names() guards IterableDataset.column_names=None
(from_generator / unresolved features) so raw-text and CPT streaming no longer
raise TypeError before training
- models/routes: reject HF slice syntax in train_split/eval_split when streaming
(load_dataset(streaming=True) raises "Bad split"); reject mixed sources
(local/S3) and embedding/MLX streaming at the API, not just in the UI
- trainer: an empty post-slice/filter stream fails preflight with a clear message;
streaming eval is capped (STREAMING_EVAL_MAX_SAMPLES) so each eval terminates;
the manual-slice shortcut falls back to a regular load when train_split is sliced
- format_conversion: streaming conversions preflight the first mapped row so
format errors surface before training, not mid-iteration
- frontend: block streaming on Apple Silicon; clear datasetStreaming when a
dataset is detected as image/audio at start
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: fix CI for streaming PR (lint blocker + no-torch sandbox + preflight test)
- trainer.py: drop unused `IterableDataset` import (hoist safety-net blocker).
- test_training_streaming.py: only select real classes (isinstance type) when
locating the trainer class, so a MagicMock-stubbed global is never passed to
object.__new__ (fixes TypeError on the Python 3.10-3.13 jobs).
- no-torch import sandboxes (test_e2e_no_torch_sandbox.py,
test_studio_import_no_torch.py): teach the chat_templates/format_conversion
exec stubs and the full-import-chain copy list about the new `.iterable`
module so the AFTER/runtime cases import without torch again.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Roland Tannous <115670425+rolandtannous@users.noreply.github.com>
Co-authored-by: Roland Tannous <rolandtannous@gravityq.ai>
Co-authored-by: Etherll <61019402+Etherll@users.noreply.github.com>
* Studio: detect transformers 5.3.0 tier from config.json for local checkpoints
A local safetensors folder whose config.json did not match the Gemma4 (510/550)
architecture signals short-circuited get_transformers_tier() to "default"
(transformers 4.57.x), never reaching the name-substring check that routes
Qwen3.5 to the 5.3.0 sidecar. So a local Qwen3.5 checkpoint (model_type
"qwen3_5", needs transformers >= 5.2.0) loaded with 4.57.x and failed with
"does not support Qwen3.5". The same model as a remote HF id worked, because it
has no local config.json to trigger the short-circuit.
Detect the 5.3.0 tier from config.json (model_type "qwen3_5" / architecture
Qwen3_5ForCausalLM) in the local-config branch, mirroring the existing Gemma4
510/550 handling. This is a positive config signal, so it fixes local Qwen3.5
without weakening the directory-name false-positive guard (a llama checkpoint
under a "gemma-4-12b-*" parent still resolves to default).
Adds tests for the config-based 530 detection and local-folder tier resolution.
* Studio: suppress false warning when config.json parse fails for sidecar-tier models
* Studio: generalize local-checkpoint tier detection for all 5.3.0 families
Expands the config.json-based tier detection to cover all known 5.3.0-tier
model families (Qwen3 MoE, GLM-4.7-Flash, LFM2.5-VL) and adds a _name_or_path
fallback so renamed local checkpoints with unrecognised model_type values still
route correctly via the HF ID embedded in their config.json.
- Expand _TRANSFORMERS_530_ARCHITECTURES / _MODEL_TYPES with verified entries
from Qwen3MoeForCausalLM, Glm4MoeLiteForCausalLM, Lfm2VlForConditionalGeneration,
and Qwen3_5ForConditionalGeneration (confirmed from local Qwen3.5-2B config.json)
- Extract _tier_from_name() helper, deduplicating the fast-substring logic used
by both the remote-path branch and the new config _name_or_path fallback
- In the local-config branch: after architecture checks, resolve the tier from
cfg._name_or_path / cfg.model_name before returning "default", preserving the
existing directory-name false-positive guard
- 79 tests passing
* Studio: match 510/550 style for 530 config sets (no inline comments)
* Studio: use _resolve_base_model instead of reinlining _name_or_path lookup
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: recurse into get_transformers_tier for resolved base model (Gemini suggestion)
* Studio: use _tier_from_name in local-config fallback to avoid network probes
Using get_transformers_tier(resolved) on the _name_or_path fallback would
trigger up to 3 network fetches (config.json + tokenizer_config.json, 10s
each) for every ordinary checkpoint whose _name_or_path is a plain HF ID
like meta-llama/Llama-3-8B. The fallback's purpose is name-based detection
on the resolved HF ID, _tier_from_name covers all known cases without I/O.
* Studio: add _check_config_needs_530 to slow HF-ID fallback path
Private or renamed HF repos whose model IDs lack a 5.3 substring were
silently routed to the default tier. _check_config_needs_530 mirrors the
existing 510/550 pattern: fetches config.json once, caches the result, and
is called after the 550 check in the slow path. Includes 5 unit tests.
* Studio: guard _tier_from_name fallback against local-path false positives
When _name_or_path in config.json is an absolute path to the same checkpoint
passed as a relative path, the textual resolved != model_name check passes
and _tier_from_name would scan the directory path for substrings. Split the
fallback: local directories recurse into get_transformers_tier (config check,
no network I/O); HF Hub IDs use _tier_from_name (name-based, no network).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: separator-norm aliases, model_name/_name_or_path fallback, tests
- _norm_separators(): collapse _ . whitespace to - so underscore/dot model
ID variants (Qwen3_5, Qwen3_Next) match the canonical substring list
- _tier_from_name(): apply norm to both name and each substring so aliases
resolve without duplicating the substring lists
- _resolve_base_model(): try model_name then _name_or_path separately so a
self-referential Unsloth model_name doesn't hide the useful HF ID in
_name_or_path
- Gate get_base_model_from_lora on adapter_cfg_path.is_file() to avoid
eagerly importing transformers before the sidecar venv is on sys.path
- 17 new tests covering _norm_separators, separator-insensitive
_tier_from_name, and the model_name/_name_or_path fallback
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: only pre-resolve LoRA adapters in activation callers
activate_transformers_for_subprocess and ensure_transformers_version were
pre-resolving all local checkpoints via _resolve_base_model before calling
get_transformers_tier. After the model_name/_name_or_path fix, a full
checkpoint with a private/offline _name_or_path and no tier substring would
resolve to that HF ID, which can't be probed, bypassing the local config.json
model_type check entirely. Gate pre-resolution on adapter_config.json so full
checkpoints go straight to get_transformers_tier, which reads config.json
directly. LoRA adapters still pre-resolve as before.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: fix Qwen3.5 MoE/Qwen3.6 tier detection and dot-version false positives
- Add Qwen3.5 MoE (qwen3_5_moe / Qwen3_5MoeForConditionalGeneration) and
Qwen3-Next to the 5.3.0 config sets, so renamed local checkpoints route to
the sidecar instead of default transformers
- Let a 510/550 name match override a 530 config match, so Qwen3.6 (which
reuses qwen3_5 / qwen3_5_moe config ids) still routes to the 5.5.0 sidecar
- Stop normalizing version dots to hyphens so size names like Qwen3-5B and
Qwen3-6B are not promoted to a 5.x sidecar; underscore aliases still match
- Skip name matching for resolved values that look like stale local paths
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: close remaining codex P2s: adapter-only LoRA + 530-override path-hint guard
- adapter_model-only LoRA: add import-light _is_lora_adapter_dir/_has_adapter_weights
and gate activation/export pre-resolve on them, so LoRA dirs with
adapter_model*.safetensors but no adapter_config.json still resolve to their base
model (via _resolve_base_model's new unsloth_<model>_<ts> directory-name parse)
instead of tiering off the adapter folder.
- 530 override: only treat a resolved value as a name hint when it is a real Hub id;
a stale/renamed local path in model_name/_name_or_path can no longer flip a correct
530 config to 550. Current folder basename still allowed.
Added 7 regression tests; suite at 116 passing.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: address review feedback on tier detection
- Add Qwen3.5 text-tower model types (qwen3_5_text / qwen3_5_moe_text) to the
5.3.0 config set so text-only configs with stripped architectures still route
to the sidecar
- Apply the Qwen3.6 name override on the remote slow path too, so a renamed or
private repo whose config reuses qwen3_5 ids but names Qwen3.6 in
_name_or_path selects 5.5.0 instead of 5.3.0
- Treat an existing local path (or empty value) as a path, not a Hub id, in
_looks_like_hf_id so a real local checkpoint folder is not name matched
- Guard _resolve_base_model against non-string config values and compare paths
by realpath so relative or absolute self references resolve correctly
- Keep the LoRA adapter is_file check inside the OSError guard
* Studio: harden tier detection against malformed configs and bad paths
- _config_matches_tier no longer raises TypeError when a malformed config.json
carries a non-string model_type (e.g. a list) or non-list architectures; it
fails open to no-match
- guard the model_name-derived is_file/is_dir probes with _safe_is_file /
_safe_is_dir so a pathological or over-long path (e.g. a Windows long path)
fails open to the default tier instead of raising OSError
No routing changes for any valid model; purely defensive. Verified by a
cross-platform simulation (POSIX + NT path semantics) and a before/after tier
matrix that is unchanged for all previously supported models.
* Studio: trim verbose comments in tier detection
Shorten/remove over-long comments and docstrings, mainly on internal helpers,
without changing behavior. Verified code-only via comment_tools.py check; suite
unchanged at 128 passing.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Studio: persistent per-user trust_remote_code approval cache
The consent gate pins each approval to a content fingerprint (sha256 over every
repo .py), but nothing was persisted, so the dialog reappeared on every fresh
load of the same unchanged repo. This adds an on-disk, per-user approval cache
that lets the gate skip the dialog when the same user reloads the same code,
while keeping the safety guarantees intact.
Two-tier validation, both must hold or the user is re-prompted:
- Commit SHA (cheap, one HfApi.model_info().sha, no download): a match means a
byte-identical tree to the approved revision, so the scan/download is skipped.
- Content fingerprint (authoritative): used whenever the SHA is unavailable
(local path / offline) and always recomputed on a SHA miss. A new or edited
.py changes both the SHA and the fingerprint, so it is caught in every mode.
Safety:
- Keyed per subject; one user's approval never auto-runs code for another.
- CRITICAL is never stored or honored (guarded on both write and read), so a
hand-edited store cannot smuggle in an auto-approval.
- The malware (HF unsafe-file) gate stays unconditional.
- Fail-safe: a corrupt store, an unresolvable SHA, or any error degrades to
"ask again", never to "auto-approve". UNSLOTH_TRC_APPROVAL_CACHE_DISABLE=1
turns the cache off entirely.
New module utils/security/remote_code_approvals.py holds the store
(studio_root()/security/remote_code_approvals.json, atomic write, 0600, RLock)
plus the SHA resolvers. Recording happens at the single gate chokepoint when the
caller supplies the matching fingerprint, so subject is just threaded through
inference/training/export (orchestrators, routes, workers). The scan endpoint
returns already_approved so the frontend can skip the dialog on a cache hit.
Tests: new tests/test_trc_approval_cache.py covers cache miss, SHA-match skip,
SHA-moved re-scan, new-file re-consent, CRITICAL never cached (write + forged
read), disable flag, subject isolation, combined adapter+base key, corrupt
store, and no-subject bypass. Full security suite: 101 passed.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: make the approval cache skip only the prompt, never the scan
Codex found that the SHA "no-scan" fast path could run untrusted code without
re-consent. Removed it; the gate now always re-scans and the cache only seeds the
authoritative fingerprint check, so it can skip the dialog but never the scan.
- CRITICAL is hard-blocked on every load (the scan always runs), so a hand-edited
store that downgrades a CRITICAL repo's severity can no longer auto-run it
(P2: do not trust editable severity for SHA approvals).
- The fingerprint covers external auto_map repos, so changed third-party code
always re-prompts even when the primary commit SHA is unchanged; there is no
longer a SHA path that bypasses the fingerprint (P1: external auto_map repos).
- resolve_commit_sha is resolved fresh on every call (no memoization), so a repo
whose default branch moves after approval re-prompts instead of reusing a stale
cached SHA (P1: revalidate mutable Hub SHAs). The SHA is now only a conservative
secondary gate: a fresh resolvable SHA must match the approved revision, else the
seed is withheld; a None (local/offline) falls back to the fingerprint.
- Approvals record the scanner ruleset version (SCAN_RULES_VERSION); the gate
ignores approvals from an older ruleset so reclassified bytes are re-scanned and
re-shown instead of silently auto-approved (P2: invalidate on scan-policy change).
Tests: test_trc_approval_cache.py rewritten around the prompt-skip semantics
(unchanged repo still scans; SHA move / changed code / scanner-version bump /
disable flag all re-prompt; forged downgraded severity still blocks CRITICAL).
105 passed with test_consent_gate.py.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Trim comments to be more succinct
* Keep run-owner subject out of persisted config; serialize approval writes
Threading subject (the run owner's username / API-key id) into the training
config meant _sanitize_db_config persisted it into config_json, which
training-history GET returns to any authenticated user, leaking who started a run
in multi-user installs. Filter subject alongside the token fields; the worker
still receives it from the live config.
The approval store's RLock only guards one process, but approvals are recorded
from separate inference/export/training subprocesses, so concurrent writers could
clobber each other on os.replace and drop an approval (re-prompt). Hold a
best-effort cross-process file lock around the read-modify-write.
* Fail safe on a malformed approval store
A store with the right version but a non-dict shape (e.g. a hand-edited
"subjects": []) passed _load()'s check, then lookup chained .get() on a list and
raised, breaking every remote-code load until the file was removed. Validate that
subjects is a dict in _load(), and tolerate a non-dict per-subject entry in
lookup/record/forget, so a corrupt store fails safe (re-prompt) instead.
* Keep subject out of the MLX W&B run config
_run_mlx_training uploads the whole training config to W&B minus a sensitive set
that only listed hf_token/wandb_token/s3_config, so the authenticated subject
(username / API-key id) was sent to W&B as run config even though DB history
already strips it. Add subject to the W&B-sensitive filter, mirroring
training._sanitize_db_config.
* Tighten the W&B subject-filter comment
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Auto-install SSM kernels (causal-conv1d, mamba-ssm) for inference loads
Mamba/SSM hybrids (Nemotron-H/Nano, Falcon-H1, Granite-4.0-H, ...) lazily import
mamba_ssm / causal_conv1d during from_pretrained, so loading them for chat failed
with 'mamba-ssm is required by the Mamba model but cannot be imported'. The training
worker already wheel-first installs these before a fine-tune; the inference worker
did not. Add utils/ssm_runtime.ensure_ssm_runtime and call it from the inference load
path so the same models load for inference. Training worker is untouched; a drift
test keeps the shared detection and pinned versions in lockstep.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* ssm_runtime: invalidate import caches, skip MLX, cover LoRA base
- Invalidate importlib finder caches in _is_importable and after a successful
wheel install, so a kernel installed earlier in this same process is actually
importable when the modeling code lazy-imports it during from_pretrained.
- Skip the SSM kernel install entirely on the MLX (Apple Silicon) load path:
these are CUDA/ROCm Torch kernels with no MLX use and no macOS prebuilt wheel,
so the source build would fail before the MLX backend loads the model.
- For LoRA loads, also run detection over the resolved base model, since an
adapter id like 'me/my-lora' won't match the SSM heuristics but its SSM base
(Nemotron-H, ...) is what needs the kernels.
Adds tests for cache invalidation and the MLX-skip / LoRA-base worker wiring.
* Tighten SSM autoinstall comments
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* ssm_runtime: verify wheel imports, HIP-aware source build, build heartbeat
Address review feedback:
- Verify a prebuilt wheel actually imports before trusting it; a CUDA/ABI-mismatched
wheel now falls back to a source build instead of returning success and failing later
with the cryptic lazy-import error.
- HIP-aware source build: require hipcc on ROCm, inject clang --gcc-install-dir, and use
the 1800s timeout, mirroring the training worker (ROCm has no prebuilt wheel).
- Emit a status heartbeat every 60s during the source build so a long (ROCm) build does
not trip the orchestrator's 300s inactivity timeout.
Tests cover the wheel-not-importable fallback and the missing-hipcc ROCm bail.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Make causal-conv1d best-effort and harden the SSM source build
- causal-conv1d is a fast path: models that merely want it (Qwen3-Next, LFM2)
fall back to torch, so a failed install must not reject an otherwise loadable
chat model on Windows/CPU/macOS or an ABI without a wheel. Only a true SSM
model's mamba-ssm requirement stays fatal, matching the training worker which
treats causal-conv1d as best-effort.
- The source build is reached only when not importable, including a wheel that
installed but failed to import; add --reinstall/--force-reinstall so it
replaces the broken install instead of no-opping as already satisfied.
- Add --no-cache to the ROCm uv source build to avoid reusing stale artifacts
from a partial HIP build, mirroring the training worker.
* Address review: install SSM kernels before transformers, harden import + Windows
Codex:
- Install the SSM kernels before importing transformers. run_inference_process
imported core.inference.inference (which imports unsloth/transformers) before the
load, and a sidecar transformers can evaluate its optional-backend gates against
the import state; installing causal_conv1d/mamba_ssm afterwards left those gates
unsatisfied and a Nemotron/Falcon/Granite load still failed with "mamba-ssm is
required". The initial model's kernels are now installed in run_inference_process
before the ML import, via a shared _ensure_ssm_kernels helper; _handle_load keeps
calling it (idempotent) for a LoRA's base and for later in-process loads.
- _is_importable now treats any import failure as "not importable", not only
ImportError. An ABI-incompatible native kernel (undefined symbol after a torch/CUDA
upgrade) raises OSError/RuntimeError; letting those escape reported
ssm_runtime_install_failed instead of falling back to reinstall/source build.
- Skip causal-conv1d on Windows (no prebuilt wheel), mirroring the training worker.
A causal-conv1d-only model (Qwen3-Next/LFM2) no longer drops a chat load into a
multi-minute untimed source build; it uses the torch fallback. mamba-ssm is still
attempted for true SSM hybrids.
Tests: test_ssm_runtime.py +5 (broken-kernel exceptions read as not-importable;
causal-conv1d skipped on win32 while mamba-ssm still installs). 36 passed.
* Trim comments to be more succinct
* Run security gates before installing SSM kernels
The SSM kernel auto-install is name-based (model_is_ssm is a substring match, no
config fetch), so a model id merely containing an SSM substring triggered a
native-package install (possibly a slow source build) before the malware and
remote-code consent gates ran. Extract those gates into _run_security_gates and
call it before the kernel install in both the pre-import path of
run_inference_process and in _handle_load, so a blocked or nonexistent model is
refused before any build. The gates are metadata-only and do not import
transformers, so they are safe to run before the pre-import install.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Resolve remote LoRA bases before importing transformers
_resolve_base_model only reads a local adapter_config.json, so a remote LoRA
adapter whose own id has no SSM substring but whose base is a Nemotron/Falcon/
Granite model had its base discovered only by ModelConfig in _handle_load, after
transformers was imported and its optional-backend availability snapshotted, so
the SSM kernel install there was too late. Add _remote_lora_base, a metadata-only
adapter_config.json fetch (no huggingface_hub / transformers import), and use it
in the pre-import path so the base is gated and its kernels pre-installed.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Gate only loaded roots, tier on the resolved base, read offline LoRA cache
Three follow-ups to the pre-import resolution:
- The security gate reused the SSM target list, which for a local full fine-tune
includes the config.json-recorded base. That base is never loaded, so scanning
it could falsely block a safe local checkpoint. Gate only the model plus a
genuine LoRA base (matching _handle_load's mc.is_lora), separate from the
broader SSM-install list.
- Tier activation ran on the raw adapter id, so a remote LoRA whose base needs a
sidecar transformers version imported the default and failed. Resolve the base
once up front and activate on it.
- _remote_lora_base bailed on offline before checking the hub cache, missing a
cached adapter's base. Read the cached adapter_config.json when offline or when
the fetch fails.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Keep the pre-import gate transformers-free; harden remote LoRA resolution
The pre-import security gate called security_load_subdirs, which imports
model_config and thus transformers, snapshotting optional-backend availability
before the SSM kernels are installed and defeating the ordering. Add
compute_subdirs to _run_security_gates and pass False in the preflight so it scans
from the root only (transformers-free); _handle_load still runs the authoritative
gate with full subdir scoping after the import.
_remote_lora_base now skips existing local relative paths (is_local_path) so a
checkpoint like outputs/run1 is never treated as a Hub repo, and distinguishes a
definitive 404 (not a LoRA -> None) from transient/offline failures (read the
cache), so a repo that is now a full model no longer resolves a stale cached base.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Probe a real model id for SSM kernels; respect HF_ENDPOINT
model_is_ssm is a substring match, so an arbitrary name could false-match and
force a mamba-ssm install that fails the load for a non-SSM model:
- a LoRA adapter id like user/falcon-h1-lora (the SSM-relevant code is the base's);
- a local checkpoint under an SSM-named parent dir, e.g. /runs/falcon-h1/llama-ckpt.
Add ssm_probe_identifier, which resolves the base (or a bare local checkpoint's
basename) and feed that to ensure_ssm_runtime from both the pre-import path and
_handle_load, so detection runs against a real model id, never an adapter id or
parent folders.
_remote_lora_base now honors HF_ENDPOINT so enterprise/mirror deployments resolve
the adapter base instead of always hitting huggingface.co.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten comments in the pre-import SSM gate/install path
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <michaelhan2050@gmail.com>
* Route dense NemotronH models to the transformers 5.10 tier
Dense NemotronH models (e.g. unsloth/NVIDIA-Nemotron-3-Nano-4B) describe their
layer stack with a hybrid_override_pattern that includes '-' (MLP) layers.
transformers only learned to parse that ('-' -> 'mlp' in pattern_mapping, 'mlp'
in valid_types and MIXER_TYPES) in 5.10; on 5.3/5.5 the config raises
KeyError: '-'. The model also ships auto_map remote code, so training and
inference that approve trust_remote_code load fine, but a native (TRC=False)
load such as export hits the built-in parser and fails with
'Failed to load checkpoint: -'.
Detect dense NemotronH from config.json (a '-' in hybrid_override_pattern, or
'mlp' in an expanded layers_block_type) and route it to the 5.10 tier, where the
model loads natively without remote code. Pure-MoE NemotronH configs are
unaffected and keep their existing tier.
Covers both the local config.json and the remote HF-id paths, and adds tests for
the detector and the resulting tier selection.
* Tighten _nemotron_h_needs_mlp_support docstring
* Detect dense NemotronH in nested, cached, and resolved-away configs
Three gaps could still route a dense NemotronH (MLP '-' layers) to a tier
below 5.10 and hit KeyError: '-':
- VL wrappers (e.g. NemotronH_Nano_VL_V2) keep the dense language model under
llm_config/text_config; the detector only checked the top-level model_type.
Recurse into nested language configs.
- Offline or blocked config fetches returned None for an already-downloaded
repo. Read config.json from the HF hub cache before any network.
- A local checkpoint resolves to its base before tiering, so an offline/private
base discarded the local config that revealed the dense pattern. Prefer the
higher tier of the resolved base and the original path.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Harden NemotronH tier detection follow-ups
Address review of the nested/cached/resolved-away detection:
- The local re-check ran the full tier detector on the original path, so a bare
LoRA adapter under e.g. /runs/gemma-4-x/llama-lora could upgrade a default base
via directory-name substrings. Gate the re-check on a real local config.json so
it reads metadata, not path names.
- The HF hub cache was read before any network, so an online tier check could
serve stale config.json after the repo changed upstream. Consult the cache only
offline or after a failed fetch.
- Reading the cache imported huggingface_hub during tier detection, which runs
before a sidecar venv is activated and could pin the default-env hub into
sys.modules. Resolve the cache path with stdlib only.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Trim comments to be more succinct
* Select newest hub-cache snapshot by mtime and retry transient config fetches
The HF cache fallback in tier detection picked the lexicographically-first
snapshot when refs/main was absent (commit-pinned downloads), which can be an
older SHA than the Hub would load. Sort snapshots by mtime instead.
A transient online fetch failure cached the hub-cache fallback under the normal
(model_name, token) key, so a long-lived worker kept serving stale metadata even
after connectivity recovered. Return the fallback without memoizing it so the
next call retries the network.
* Harden config.json tier detection against auth failures and transient blips
- _load_config_json: a 401/403/404 from the raw Hub request is a definitive access
answer, not an outage. Return None instead of falling back to the HF hub cache, so
an unauthenticated or wrong-token request can never read another caller's cached
private metadata.
- _check_config_needs_510/550: only memoize the derived tier when the underlying
config read was definitive (local file, offline cache, or a completed fetch).
A transient fetch fallback is no longer pinned, so the tier is re-evaluated once
connectivity returns instead of staying stuck on the lower tier.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten comments in tier-detection auth/cache paths
---------
Co-authored-by: Daniel Han <michaelhan2050@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* studio: persist personalization (profile + theme) server-side
Profile name/nickname/avatar and appearance (theme) were stored only in the
browser's localStorage, so every browser or device that connected to the same
Studio started from defaults and forgot the user's personalization.
Persist them server-side (single-account, stored as one JSON blob in
app_settings) so they follow the account:
- utils/personalization_settings.py + GET/PUT /api/settings/personalization,
with validation (theme/shape enums, avatar must be an image data URL capped at
512 KB) and a 'saved' flag.
- Frontend usePersonalizationSync (mounted in the root layout when signed in)
hydrates the profile + theme stores from the server when a blob exists, and
otherwise migrates the existing local settings up once so nothing is lost;
later changes are written through, debounced. Writers keep using the local
stores unchanged.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix/adjust personalization sync for PR #6516
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix Studio personalization sync edge cases
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
* Windows installer: fix DiskPart UAC, drive-root cache, spurious rename warning, CPU-base messaging
amd-smi gate (DiskPart UAC mid-install): the AMD torch wheel ships hipInfo.exe
inside the venv, and the bitsandbytes fix prepends that venv Scripts dir to PATH.
shutil.which("hipinfo") then found it and flipped _amd_smi_allowed() to True, so
the post-install AMD probe fell through to `amd-smi list` (the venv hipInfo failed
to report gcnArchName, which is why the arch came from the GPU-name table) and
amd-smi elevated, popping the DiskPart UAC. Fix: a hipinfo resolved inside the
active venv (sys.prefix) is the torch-wheel binary, not a HIP SDK, and must not
open the gate. Mirrored in install_python_stack.py, install_llama_prebuilt.py, and
backend utils/hardware/amd.py (the runtime VRAM poller had the same latent prompt).
TORCHINDUCTOR_CACHE_DIR: move from C:\tc to <StudioHome>\TORCHINDUCTOR_CACHE_DIR so
the inductor/Triton cache lives under the user's Studio home, not the system drive
root. Long paths are already enabled above so deep inductor paths still fit.
unsloth.exe rename: skip the rename (and its "pip may fail with WinError 32"
warning) when SKIP_STUDIO_BASE=1. In the install.ps1 flow base packages are not
reinstalled, so unsloth.exe is never rewritten; the self-rename only failed because
setup runs via unsloth.exe (the running launcher holds its own file). The
'studio update' flow still attempts it.
CPU PyTorch messaging: clarify that the CPU base is temporary and setup replaces it
with GPU ROCm wheels, and print an explicit "GPU ROCm PyTorch installed" line after
the AMD wheels land, so the log makes clear the final install is GPU-accelerated.
Adds two regression tests covering the venv-internal vs external hipInfo gate.
Verified end-to-end on a Strix Halo box (Radeon 8060S / gfx1151): install.ps1
--local from this branch completed exit 0 with no DiskPart prompt, no rename
warning, the cache under the Studio home, and "GPU ROCm PyTorch installed
(gfx1151)"; Studio then booted and detected "ROCm (HIP 7.13.99004) -- AMD Radeon
8060S Graphics".
* Windows installer: drop the unreliable unsloth.exe rename and its WinError 32 warning
setup.ps1 used to rename the running unsloth.exe out of the way before the
base-package upgrade so pip could replace it. That rename never actually
worked: setup runs *via* unsloth.exe, so renaming our own running
uv-trampoline launcher failed with a sharing violation (WinError 32) and only
printed a scary 'could not rename unsloth.exe; pip may fail with WinError 32'
warning on every Windows install and update.
It also was not needed. pip tolerates a running/locked console-script .exe: it
moves the old one aside and writes the new one. The base upgrade routes through
pip on Windows, so the upgrade succeeds (or, in the install.ps1 flow with
SKIP_STUDIO_BASE=1, the base is not touched at all) and unsloth.exe is left
intact either way.
Removing the rename block and its failed-install restore block removes the
false warning for all Windows devices in both the install and update flows.
* Windows installer: gate venv-internal hipInfo.exe in PowerShell amd-smi probe; harden venv path checks
Follow-up to PR #6296.
- install.ps1 and setup.ps1: ignore the AMD torch wheel hipInfo.exe that lives
inside the Studio venv when probing for a HIP SDK, so amd-smi no longer reopens
the DiskPart UAC during install/update. Mirrors _path_inside_venv in the Python
installers, which already do this.
- amd.py, install_llama_prebuilt.py, install_python_stack.py: normcase the venv
containment check (Windows paths are case-insensitive) and run the
HIP_PATH/ROCM_PATH candidate through it too.
- setup.ps1: fall back to a short TORCHINDUCTOR cache dir when long paths are
unavailable, and create the dir wildcard-safely.
- tests: isolate sys.prefix in the gate helper, add HIP_PATH/ROCM_PATH cases, and
assert the PowerShell venv exclusion.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Windows installer: install ROCm PyTorch directly for a known AMD arch
When the GPU arch is known (name-inferred from the GPU-name table) but ROCm
could not be probe-verified (no HIP SDK, no amd-smi), the bootstrap installed
a CPU PyTorch base that setup.ps1 then force-reinstalled as ROCm. The
repo.amd.com wheels bundle their own runtime (no HIP SDK required), which
setup.ps1 already relies on, so the CPU base was a pure wasted download/install.
- Gate the ROCm index on a known arch, not only on probe-verified ROCm, so a
mapped arch installs ROCm torch directly. Unmapped arches and no-GPU hosts
still get CPU (unchanged).
- Fall back to a CPU base if the ROCm-index install fails, so a transient
repo.amd.com outage does not abort the install (setup.ps1 retries ROCm).
- Correct the stale comment that claimed ROCm wheels need a confirmed HIP SDK.
- Add a regression test for the arch-based gate.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Windows installer: correct the unsloth.exe rename-removal comment
The comment claimed the base upgrade 'routes through pip on Windows' and that
pip 'moves the old unsloth.exe aside, then writes the new one'. That is not what
the code does. install_python_stack tries uv first; on a locked launcher uv
aborts and falls back to pip, but the pip fallback strips --upgrade-package and
base.txt lists only bare unsloth/unsloth-zoo, so pip finds them already
satisfied and no-ops. The running unsloth.exe is left intact at its current
version either way. Reword the comment to describe the real uv-first /
pip-fallback-no-op behavior. No functional change.
* Windows installer: close two gaps in the venv-internal hipinfo exclusion
Review follow-up. The amd-smi/DiskPart gate could still reopen in two cases:
- setup.ps1 ran the HIP probe long before $VenvDir is assigned, so without
VIRTUAL_ENV (the `unsloth studio update` path) $venvRoots was empty and the
venv-internal hipInfo.exe was not recognized. Seed the venv root from
UNSLOTH_SETUP_PYTHON and the default Studio home too (both installers).
- The HIP_PATH/ROCM_PATH candidate was accepted without the venv filter, so an
env var pointing into the venv (AMD wheel) still set $HipSdkInstalled. Run
Test-HipinfoIsVenvInternal on the candidate as well (both installers).
Extend the PS gate test to assert both. Both .ps1 parse clean; install tests
pass (the venv-internal / HIP probe coverage at 359 passed).
* Windows installer: correct the CPU-base message for arches with no ROCm wheels
After gating the ROCm index on a known arch, a mapped arch sets $ROCmIndexUrl
and installs ROCm directly, so it no longer reaches the "temporary CPU base"
branch. That branch is now reached only by a name-inferred arch with no ROCm
wheels (e.g. RDNA2 gfx103X), where setup.ps1 does NOT install ROCm. The old
text ("setup replaces it with GPU ROCm wheels ... the final install IS
GPU-accelerated") was therefore always wrong there. Say plainly that PyTorch
stays on CPU for this GPU.
* Windows installer: seed the venv-internal hipInfo check from a custom Studio home
Test-HipinfoIsVenvInternal seeded the venv root from VIRTUAL_ENV, VenvDir, the
setup python, and the default %USERPROFILE% path only. A standalone
`unsloth studio update` with a custom UNSLOTH_STUDIO_HOME (or STUDIO_HOME alias)
and none of those set would not recognize the venv hipInfo on PATH, reopening the
amd-smi/DiskPart gate. Seed the custom home too, in both installers, and assert
it in the gate test.
* Studio installer: resolve venv aliases and expand ~ in the hipInfo venv filter
Two review points on the amd-smi/DiskPart UAC gate:
1. _path_inside_venv compared os.path.abspath of sys.prefix and the hipInfo
path, which does not resolve symlinks, junctions, or 8.3 short names. A venv
reached through an aliased path then fails the check, so its bundled
hipInfo.exe is mistaken for an external HIP SDK and amd-smi runs (the
DiskPart prompt this fix exists to suppress). Switch to os.path.realpath in
all three copies (amd.py, install_llama_prebuilt.py, install_python_stack.py).
2. setup.ps1's early venv-internal hipInfo probe seeded the venv root from a
custom Studio home (UNSLOTH_STUDIO_HOME / STUDIO_HOME) without expanding a
leading ~, while the canonical resolver does. With a tilde form,
[IO.Path]::GetFullPath kept the literal ~ relative to cwd, so the custom-home
hipInfo escaped the filter and reopened the gate. Expand ~ in the probe the
same way as the resolver.
tests/studio/install/test_pr5940_followups.py: 30 passed (adds a symlink
realpath case and a setup.ps1 tilde-expansion guard).
* Studio installer: mirror the hipInfo venv filter and ROCm wheel pins into install.ps1
Follow-up review on the same install.ps1 paths:
1. install.ps1's venv-internal hipInfo probe (Test-HipinfoIsVenvInternal)
seeded the venv root from a custom Studio home without expanding a leading
~, unlike the canonical resolver and setup.ps1. A tilde form left
[IO.Path]::GetFullPath with the literal ~ (relative to cwd), so the
custom-home hipInfo escaped the filter and reopened the amd-smi/DiskPart
gate. Expand ~ in the probe, matching the setup.ps1 fix.
2. The AMD ROCm path installed torchvision/torchaudio bare while pinning torch
to below 2.12. AMD's per-arch index publishes the companions independently
and may ship torchvision 0.27 (for torch 2.12) before removing 0.26, so a
bare resolve can pick an ABI-incompatible set and fall back to CPU. Add
torchvision/torchaudio floor maps and pass the pinned specs, mirroring
setup.ps1 and install_python_stack.py.
3. The ROCm-to-CPU fallback torch install used Invoke-InstallCommand (no
retry), the only torch step in the file without it. Switch to
Invoke-InstallCommandRetry so the recovery path survives a transient index
failure.
tests/studio/install/test_pr5940_followups.py: 33 passed (parametrized tilde
check over both installers, a torch/companion floor-map parity test, and a
CPU-fallback retry guard).
* Studio installer: scan all PATH hipinfo so the venv copy can't shadow a real HIP SDK
The amd-smi HIP-SDK probe used shutil.which("hipinfo") / Get-Command hipinfo,
which return only the first hit on PATH. The AMD torch wheel ships hipInfo.exe
inside the venv and the bnb fix (plus the Studio backend) prepend the venv
Scripts dir to PATH, so that venv-internal copy lands first. When a real HIP SDK
hipinfo sits later on PATH with HIP_PATH/ROCM_PATH unset, the first-hit probe
stopped at the venv copy, treated it as "not a HIP SDK", and closed the amd-smi
gate -- AMD users in that PATH-only SDK setup lost amd-smi telemetry and could
fall back to CPU. Scan every PATH entry and keep the first hipinfo that is not
venv-internal; only the venv copy is ignored, so the UAC/DiskPart suppression is
unchanged.
Applied to all three Python copies (install_llama_prebuilt.py,
install_python_stack.py, backend/utils/hardware/amd.py) via a new
_external_hipinfo_on_path helper, and both PowerShell callers (install.ps1,
setup.ps1) now use Get-Command hipinfo -All filtered by Test-HipinfoIsVenvInternal.
tests/studio/install/test_pr5940_followups.py: 36 passed (real-PATH scan tests, a
shadow-regression test for the exact venv-first ordering, and a parity check that
every Python copy uses the scanning helper).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio uninstallers: fix leftovers (false "removed", shared icon, llama lock)
Auditing a dual native+WSL uninstall on a real device surfaced three leftovers:
1. uninstall.ps1 removed the data dir (which holds unsloth.ico) before the
shortcuts that reference that icon, so Explorer's icon cache briefly held it
open. Remove-Item -Recurse reported success yet left the locked file, and the
dir was never re-attempted, so it orphaned with a false "removed" log.
_RemovePath now verifies the path is actually gone (retrying transient locks)
and reports honestly, and the data dir is re-swept after the shortcuts go.
2. install.sh writes a shared unsloth.ico to %LOCALAPPDATA%\Unsloth Studio for
the WSL shortcut, but uninstall.sh never removed it, orphaning the icon (and
dir) after a WSL uninstall. uninstall.sh now drops that icon and the dir when
empty, in both the powershell.exe and drvfs-fallback paths.
3. ~/.unsloth/.llama.cpp.install.lock was never removed, so the rmdir of
~/.unsloth failed and the dir lingered. Both uninstallers now remove the lock.
Verified by running both uninstallers on a real dual install: device fully clean
(no install dirs, shortcuts, PATH/registry entries, shared icon, or lock left).
* install.sh: auto-route Strix Halo WSL to an existing Ubuntu 24.04
ROCm-on-WSL is the GPU runtime for Strix Halo and only targets Ubuntu
24.04. When the installer runs in a newer default distro (e.g. 26.04) it
cannot enable the GPU and silently falls back to CPU. If a 24.04 distro
already exists, re-run the install there and stop in the current one so the
GPU path is taken without the user having to know about the distro
requirement.
Runs before venv creation so the wrong distro is left untouched, guards
against re-route loops via UNSLOTH_WSL_REROUTED, leaves a working ROCm
distro alone (librocdxg present), and skips the GGUF-only / opt-out /
non-Strix cases. When no 24.04 distro exists we keep today's behaviour:
continue to CPU and print the `wsl --install Ubuntu-24.04` guidance, never
auto-downloading a distro.
Adds tests/sh/test_strixhalo_wsl_reroute.sh (hermetic: extracts the
function, rewrites its paths to fixtures, mocks wsl.exe) covering the full
decision matrix, wired into tests/run_all.sh.
* uninstall.ps1: keep shared unsloth.ico for a surviving WSL shortcut
A dual native+WSL install shares %LOCALAPPDATA%\Unsloth Studio\unsloth.ico:
install.sh points the WSL shortcut's icon there while the native install owns the
dir. The native uninstaller removed the whole dir unconditionally, so uninstalling
native while keeping WSL left the WSL shortcut with a blank icon. The old code only
avoided this when Explorer happened to hold the icon open, which is unreliable; on a
real dual install the dir was deleted and the WSL shortcut went blank.
_RemoveDataDirKeepingWslIcon now scans the Start Menu + Desktop for a surviving
"Unsloth Studio (WSL ...).lnk" and, if found, removes everything in the data dir
except unsloth.ico (keeping the dir) instead of deleting it; with no WSL shortcut it
removes the dir as before. uninstall.sh still drops the icon and the empty dir when
WSL itself is uninstalled, so every uninstall order ends clean.
Adds tests/studio/test_uninstall_dual_install_icon.ps1 (AST-extracts the helper and
runs it against a temp dir with controlled shortcut dirs) covering the dual,
native-only, empty, and missing-dir cases, wired into the windows-inference smoke
workflow. Verified on a real dual install: native uninstall now keeps unsloth.ico
and the WSL shortcut's icon stays intact.
* installer: condense AMD/ROCm code comments (no behavior change)
Tighten the comments added for the Strix Halo native+WSL installer work so
they are shorter and clearer without losing intent: the venv-internal hipInfo
amd-smi gate, the ROCm torch/companion floor maps, the WSL 24.04 reroute, and
the dual-install uninstall icon handling. Comment-only; code paths unchanged.
107 insertions, 166 deletions across 11 files.
* install.sh: run the Strix Halo WSL reroute before any STUDIO_HOME write
The reroute fired after mkdir -p "$STUDIO_HOME" and the legacy-venv migration,
so rerouting 26.04 -> 24.04 left an empty ~/.unsloth/studio stub in the origin
distro (and ran venv migration in the distro about to be abandoned). Move the
reroute ahead of the venv section so the origin distro is left untouched, matching
the function's own comment. Behavior is identical on every non-reroute path.
* installer: fix ROCm CPU-fallback, hipinfo gate edge cases, uninstall icon, WSL 22.04
- install.ps1: clear $ROCmIndexUrl/$ROCmTorchFloor after the CPU fallback so the
flavor-repair block does not retry the failed ROCm index and abort the install;
pin the ROCm companion specs ($visionSpec/$audioSpec) in the repair path too.
- install.ps1 + setup.ps1: skip a bare drive root in Test-HipinfoIsVenvInternal so a
non-venv UNSLOTH_SETUP_PYTHON does not match the whole drive; iterate
HIP_PATH/HIP_PATH_57/ROCM_PATH and take the first non-venv hipinfo.
- amd.py, install_llama_prebuilt.py, install_python_stack.py: strip surrounding
quotes from PATH entries before probing for hipinfo.
- install.sh: pipefail the WSL reroute curl|sh; do not reroute supported Ubuntu 22.04.
- uninstall.sh: keep the shared unsloth.ico while any Unsloth shortcut (native or
another WSL distro) still references it, in both the powershell and drvfs paths.
- tests: regression coverage for all of the above.
* installer: forward reroute options, guard ROCm bootstrap, harden hipinfo gate
- install.sh: forward the caller's --package/--python/--verbose/--tauri and a custom
UNSLOTH_STUDIO_HOME into the WSL reroute (was a bare default install); bail on
--local; run the reroute BEFORE dependency/uv install so the origin distro is left
untouched; set UNSLOTH_SKIP_ROCM_WSL_SETUP after a failed reroute so the later
ROCm-on-WSL bootstrap does not install into the unsupported origin distro.
- install.ps1 + setup.ps1: Get-Command hipinfo -CommandType Application so only real
executables match (not an alias/function named hipinfo).
- uninstall.ps1: guard $env:APPDATA when building the default shortcut search dirs.
- tests: cover option forwarding, --local bail, the bootstrap guard, and the gate change.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* installer: guard origin ROCm bootstrap on every CPU-only fallback; harden ~ expansion
WSL reroute: the no-wsl.exe, no-24.04-target and --local fallbacks all tell the
user the install continues CPU-only, but only the failed-reroute branch set
UNSLOTH_SKIP_ROCM_WSL_SETUP=1. The later _maybe_bootstrap_rocm_wsl gate keys off
that flag, so the other three branches could still install ROCm into the
unsupported origin distro (e.g. 26.04). Set the skip guard on all of them.
Forward UNSLOTH_ROCM_WSL_AUTO into the reroute so a Tauri/consented GPU bootstrap
carries through to the rerouted 24.04 child instead of dropping to the prompt path.
install.ps1/setup.ps1: guard the venv-probe ~ expansion on a non-empty
$env:USERPROFILE so Join-Path does not throw on a profile-less service account.
Tests: add no-wsl.exe and UNSLOTH_ROCM_WSL_AUTO reroute cases, the USERPROFILE
guard assertion, and route shell-test fixtures through a single trap-cleaned root.
* installer: pin + soften Windows ROCm Python repair, reroute to 22.04, harden gates
install_python_stack.py: the Windows AMD ROCm repair in _ensure_rocm_torch()
installed bare torch/torchvision/torchaudio via the fatal pip_install -- the same
asymmetry already fixed on the PowerShell side. A transient repo.amd.com failure
could abort the whole install even after install.ps1/setup.ps1 fell back to CPU.
Pin companions per-arch (gfx120X/Strix -> the rocm7.2 trio, mirroring the PS floor
maps) and make the retry nonfatal: keep the existing build and let the user re-run
update to retry ROCm, so the chain install.ps1 -> setup.ps1 -> stack stays CPU-safe.
install.sh: reroute now targets an installed Ubuntu 24.04 OR 22.04 (24.04 preferred);
both are AMD-supported for ROCm-on-WSL, matching the leave-alone set, so a box with
only 22.04 reaches the GPU instead of staying CPU-only.
install.ps1/setup.ps1: a bare ~ for UNSLOTH_STUDIO_HOME left an empty Join-Path child
(PS 5.1 throws); fall back to USERPROFILE directly and only join a real remainder.
_path_inside_venv (amd.py + both installers): guard a root-dir sys.prefix so commonpath
can't classify every path on the drive as venv-internal (defensive; venv never at root).
uninstall.sh: guard an empty LOCALAPPDATA in the PS-interop icon cleanup (mirror APPDATA).
Tests: add 22.04-target reroute cases, Windows ROCm pin+nonfatal coverage (text +
behavioral), root-dir guard coverage, and bare-~/LOCALAPPDATA guard assertions.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install.sh: match WSL reroute target by exact distro name, not substring
The 24.04/22.04 reroute target was chosen with grep -F (substring), so a custom
distro such as 'Ubuntu-24.04-test' (with no exact Ubuntu-24.04) was picked as the
target; the later 'wsl -d Ubuntu-24.04' then fails and the Strix Halo install stays
CPU-only. Match whole lines (grep -ixF) and reuse the matched name so only a real
Ubuntu-24.04/22.04 is targeted. Adds substring-rejection + exact-vs-custom tests.
* install.sh: keep the WSL reroute target to Ubuntu 24.04 (helper-supported only)
The ROCm-on-WSL bootstrap (scripts/install_rocm_wsl_strixhalo.sh) dies on any
VERSION_ID other than 24.04 and pins the noble repo, so treating 22.04 as
GPU-supported let the parent report a successful reroute while the child fell
back to CPU. Drop 22.04 from the supported set and the reroute target list;
24.04 stays the sole target (keeping the exact whole-line distro match). An
already-working ROCm on any other version is still left alone by the librocdxg
check above.
tests: reroute 22.04 cases updated to the 24.04-only behavior; make the
"no wsl.exe" case hermetic so a real host wsl.exe can't leak in on dev boxes;
stop the tauri exit-order check from mis-flagging the reroute helper's
[ "$TAURI_MODE" = true ] && ... --tauri one-liner.
* installer: tighten comment wording across the Strix Halo install/uninstall paths
Condense the verbose multi-line comment blocks (amd-smi hipinfo gate, ROCm
torch install + CPU fallback, WSL reroute, uninstall icon-keep) into fewer,
clearer lines. Comments and a few docstrings only; no code, logic, or
behavior change. Verified with bash -n, the PowerShell parser, and ast.parse,
and the installer test suite still passes.
* add AGPL-3.0 SPDX headers to the .sh/.ps1 scripts missing them
Every shell and PowerShell script under the Studio/installer surface now
carries the standard SPDX-License-Identifier: AGPL-3.0-only + copyright
header (after the shebang where present): the installer (install.sh,
install.ps1), build.sh, the .github and src-tauri scripts, the installer
test suite, and the moe kernel test. Header-only, line endings preserved;
bash -n, the PowerShell parser, and the installer tests all pass.
* installer: drop the duplicate AGPL header from install.sh and install.ps1
Both already carry an SPDX-License-Identifier: AGPL-3.0-only header below
their usage comment block; the prior header pass added a second one at the
top because it only scanned the first few lines. Remove the duplicate so each
file keeps a single original header.
* installer: force-reinstall CPU fallback torch; propagate Tauri NEED_SUDO from reroute
install.ps1/setup.ps1: when the AMD ROCm wheel install fails and we fall back to a
CPU base, force-reinstall the torch/vision/audio triplet. A failed ROCm install can
leave an unpinned ROCm torch (e.g. 2.10.0+rocm on gfx110X/gfx90a) that still
satisfies the CPU torch>=2.4,<2.11.0 range, so without --force-reinstall uv keeps the
ROCm build and only swaps the companions -- a mismatched venv the flavor-repair block
won't fix. setup.ps1 scopes the forced reinstall to the ROCm-fallback path
() so the genuine CPU-only install stays fast.
install.sh: the Strix Halo WSL reroute treated every nonzero child exit as a reroute
failure and fell back to CPU. In --tauri mode the child uses exit 2 ([TAURI:NEED_SUDO])
to ask the desktop app to elevate for the target distro; capture the child's exit code
and propagate exit 2 in Tauri mode (the child already printed the NEED_SUDO line)
instead of masking it. CLI mode still falls back to CPU on a generic failure.
Tests: reroute Tauri exit-2 propagation (and non-Tauri CPU-fallback) cases;
run_func now preserves the child exit code; force-reinstall assertions for both
PowerShell installers.
Note: codex's _rr_q apostrophe finding is a false positive -- the helper already
emits POSIX-correct 'O'\''Brien' and round-trips under both sh and bash.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* setup.ps1: fix $cpuForce array collapse in the ROCm->CPU torch fallback
An if-expression assignment ($cpuForce = if ($ROCmCpuFallback) { @("--force-reinstall") })
collapses the single-element array to a scalar string, so @cpuForce splatting enumerated
it character-by-character into broken single-letter args (- - f o r c e ...), which made
uv/pip reject the install and aborted the whole Studio setup on the AMD ROCm->CPU fallback
path. Build $cpuForce as a real array assigned outside the if-expression so the splat passes
a single --force-reinstall arg. Genuine CPU-only installs stay fast (empty array, no flag).
Test now asserts the array-build form and rejects the if-expression form.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* uninstall: remove the isolated Node.js runtime (~/.unsloth/node)
The isolated Node.js runtime (install_node_prebuilt.py, added with the managed-Node
change) installs to ~/.unsloth/node in default mode -- a sibling of studio, so deleting
<studio> leaves it behind (~200MB orphaned after uninstall). Both uninstallers already
remove the other default-mode siblings (llama.cpp/.cache/.staging); add node alongside
them. uninstall.ps1 also adds it to the handle-lock sweep so a held node.exe can't block
the delete. Env/custom mode nests node under the custom root, removed with that root.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: defer llama.cpp update probes and self-heal MLX on macOS
Two macOS startup problems shared one root area in the FastAPI lifespan:
- The llama.cpp capability + freshness probes ran inline before the server
yielded, so a cold/slow/flaky network on the GitHub freshness check blocked
'Application startup complete' (~34s on CI, longer in the field). Move both
probes to a daemon thread; app.state stays None until ready (status routes
already re-probe at request time). Opt out with UNSLOTH_DISABLE_UPDATE_CHECK=1.
- Train and Export were greyed out because mlx/mlx-lm/mlx-vlm arrive only
transitively and a resolver backtrack silently drops them, so CHAT_ONLY stayed
true. Add utils/mlx_repair.py: when Apple Silicon is detected without MLX,
reinstall mlx/mlx-lm/mlx-vlm by name on a daemon thread and re-run hardware
detection (opt out UNSLOTH_DISABLE_MLX_AUTOREPAIR=1). Surface a chat_only_reason
in /api/health plus a sidebar tooltip so a greyed Train/Export explains itself
instead of failing silently.
* Studio: guard model defaults against a None model name
load_model_defaults(None) called model_name.lower() with no guard, raising
'Error loading model defaults for None' before any model is selected. Return
an empty dict for a falsy/non-str name.
* Studio: drop obsolete upstream macOS + Windows Blackwell prebuilt pins
Both pins worked around gaps in ggml-org upstream prebuilts, but Studio now
routes every GPU host and all of macOS to the unslothai/llama.cpp fork
(published_repo_for_host), which ships the needed bundles, so both pins are
dead code on the default install path:
- macOS b9415: macOS always routes to the fork (its own macOS bundles), and
host_supports_macos_minos() is the backstop. The pin only fired under an
explicit --published-repo ggml-org override.
- Windows Blackwell b9360: Windows-NVIDIA routes to the fork, whose
windows-x64-cuda13 bundle covers Blackwell (manifest max_sm 120, toolkit
13.3), so the pin's self-disable check makes it dormant on every default
install; it could only activate under the same upstream override on a
13.0-13.2 driver.
Remove the pin constants, functions, and call sites. Keep the Blackwell
capability detection (_drop_blackwell_incapable_windows_cuda, _host_is_blackwell,
_windows_cuda_attempt_covers_blackwell) that still drops a non-sm_120 cuda-12.4
build on a Blackwell host. After this, an explicit --published-repo ggml-org
override on a Blackwell 13.0-13.2 host loses its GPU fallback and lands on CPU;
the default fork path is unaffected. Update the install selection-logic and
macOS-compat unit tests for the new no-pin behavior.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: walk back deeper on the macOS upstream prebuilt path
After removing the b9415 macOS pin, the explicit --published-repo ggml-org
upstream path still used the default 2-release fallback, so a pre-macOS-26 host
behind a run of macOS-26-only builds would exhaust two too-new plans (minos is
only checked post-download) and drop to a source build before reaching a
loadable older release. Walk back as deep as the fork macOS path
(DEFAULT_MAX_MACOS_RELEASE_FALLBACKS), turning the removed static pin into
dynamic discovery. Addresses review feedback on the macOS upstream fallback.
* Studio: pin transformers during MLX self-heal so it cannot break Studio
mlx-lm/mlx-vlm declare transformers>=5, but the single-env install pins
transformers==4.57.6. The self-heal used --upgrade with no constraint, so it
could upgrade transformers in the live venv and break the rest of Studio just to
make import mlx.core pass. Pin transformers to the installed version via a
constraint file: the resolver either finds an mlx build compatible with it or
fails (we stay chat-only), never upgrading transformers underneath Studio.
Addresses review feedback on the MLX repair install.
* Studio: harden MLX self-heal against an unsupported mlx-vlm
Pinning transformers alone made uv backtrack mlx-vlm to 0.3.9 (below unsloth-zoo's
mlx-vlm>=0.4.4), which imports but breaks VLM Train/Export -- so the self-heal
could clear chat-only onto a broken stack. Mirror the main installer: set
UV_OVERRIDE=overrides-darwin-arm64.txt so a current mlx-vlm coexists with the
transformers pin, require the same minimum versions unsloth-zoo declares, and
gate/validate on a full mlx_stack_available() check (not a bare import) so an
old or partial stack stays chat-only. Addresses PR review.
* Studio: filter Blackwell-incapable CUDA in resolve_upstream_asset_choice
resolve_upstream_asset_choice returned the first windows-cuda choice unfiltered,
so a Blackwell host could be handed an sm_120-incapable cuda-12.4 build while the
sibling planners drop it. Apply _drop_blackwell_incapable_windows_cuda here too
and fall through to the CPU bundle on a Blackwell host with no capable GPU asset.
Addresses PR review.
* Studio: re-poll health so MLX self-heal reaches an open UI
The sidebar cached the initial /api/health, so a successful background MLX
self-heal (chat_only flips false) did not re-enable Train/Export until a manual
reload. While chat-only for the recoverable mlx_unavailable reason, re-poll
/api/health and stop once Train/Export become available. Addresses PR review.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: make the disabled Train/Export tooltip reachable
The greyed Train/Export items pass a tooltip explaining why (e.g. MLX missing),
but a disabled <button> fires no pointer events and SidebarMenuButton only showed
tooltips while collapsed, so the explanation never appeared. Wrap a disabled
button in a focusable span and show its tooltip while expanded too; enabled items
keep the collapsed-only behavior. Addresses PR review.
* Studio: gate Train/Export on the full MLX stack, not bare mlx.core
detect_hardware enabled MLX training whenever `import mlx.core` worked, but the
MLX self-heal (utils/mlx_repair) treats a stack without mlx-lm/mlx-vlm at the
versions unsloth-zoo requires as inadequate. That asymmetry let the UI enable
Train/Export on exactly the partial/backtracked stack the self-heal is trying to
repair (greyed-in-but-broken VLM export). Gate on the same mlx_stack_available()
criterion so a partial stack stays chat-only (reason mlx_unavailable) and the
background repair restores it. Addresses PR review.
* Fix MLX repair and health auth for PR #6494
* Fix macOS upstream prebuilt fallback for PR #6494
* Fix MLX stack validation for PR #6494
* Fix MLX self-heal validation for PR #6494
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Review fixes: isolate hardware-state test, robust transformers pin
- test_chat_only_reason.py: detect_hardware() assigns module globals directly,
which monkeypatch does not revert; the autouse fixture now saves and restores
DEVICE/CHAT_ONLY/CHAT_ONLY_REASON/IS_ROCM so a chat-only verdict here cannot
leak into other backend tests (e.g. test_utils.py) on a GPU host.
- mlx_repair.py: read the transformers version from importlib.metadata instead of
importing transformers, so the install pin is not silently dropped when
transformers has valid metadata but fails to import.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix CI: model full MLX stack in dispatch tests, keep selection test offline
dispatch (macOS) job:
- detect_hardware now gates MLX on the full stack (mlx_stack_available imports
mlx_lm/mlx_vlm and checks dist versions), so faking only mlx.core makes the
apple_silicon_mlx profile resolve to CPU. The dispatch tests assert the routing
decision when the stack IS usable, so model a complete stack:
test_hardware_dispatch_matrix patches utils.mlx_repair.mlx_stack_available and
test_is_mlx_dispatch_gate patches hardware._has_usable_mlx_stack. The stack
predicate's own internals stay covered by test_mlx_repair.py.
Repo tests (CPU) job:
- test_no_cuda_attempt_on_published_path_for_13_1 fell through to a live
github_release_assets() upstream fetch after the Blackwell filter dropped every
published attempt, which the offline security scanner blocks. Stub that fetch so
the walk-back deterministically finds no usable CUDA build and raises
PrebuiltFallback without network.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Harden MLX self-heal: prepare transformers constraint inside the try
attempt_mlx_repair runs on a daemon thread, but _transformers_constraint_args was
called before the try. A failure there (e.g. tempfile.mkstemp on a full disk or a
bad TMPDIR) would propagate unhandled and silently kill the self-heal thread.
Move the call inside the try and initialize constraint_path so any such failure
is caught and leaves Studio chat-only instead of crashing the thread.
---------
Co-authored-by: Daniel Han <michaelhan2050@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
* Scan auto_map for GGUF-only repo ids in the consent gate
The trust_remote_code consent gate treated any repo classified GGUF-only
(ships .gguf, no transformers-loadable weight) as having no remote code,
so _config_has_auto_map returned False even when a config declared an
auto_map and the repo shipped the referenced .py. The evaluator then
skipped the scan/fingerprint for that target entirely.
GGUF-inertness is a property of the loader, not the repo. A GGUF
selection loads via llama.cpp, which never reads config.json/auto_map,
and that case is already short-circuited upstream by the caller's
is_gguf check (the inference route skips the remote-code preflight for a
GGUF load). Every path that reaches this helper (export, training,
non-GGUF inference) loads through transformers/Unsloth from_pretrained,
which DOES import auto_map even for a repo that only ships .gguf weights:
the custom module runs before from_pretrained fails on the missing
transformers weights. The export path has no is_gguf guard and passes the
source straight to FastLanguageModel.from_pretrained(trust_remote_code=True),
so the in-helper GGUF skip let a repo with config.json (auto_map) +
modeling_x.py + only a .gguf run unreviewed code during export.
Drop the redundant repo-level GGUF short-circuit (and the now-unused
_is_gguf_repo helper). A direct .gguf file reference stays inert via
_is_direct_gguf_file_ref because that genuinely is a single-file llama.cpp
load; repo ids are always scanned. A GGUF repo whose auto_map ships no .py
still allows via the existing empty-code path, so legitimate GGUF loads
are unaffected (and GGUF inference never reaches this helper at all). Only
a repo that actually contains a .gguf can change behavior here; non-GGUF
repos (safetensors, MLX) are byte-identical before and after.
Update the GGUF auto_map test to expect a scan, and add two regression
tests: a GGUF-only repo shipping auto_map Python is scanned and blocked,
and a transformers-style repo (safetensors / MLX .npz) with auto_map stays
scanned and blocked.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Remove trust_remote_code config defaults; consent dialog is the only enabler
trust_remote_code is a per-load decision that must go through the remote-code
consent dialog, which scans the auto_map code and pins the exact version. Two
pre-set paths could still enable it without the user reviewing any code, and the
GGUF consent bypass rode one of them into the export flow:
- 4 model_defaults YAMLs shipped trust_remote_code: true (GLM-4.7-Flash,
Nemotron-3-Nano-30B-A3B, PaddleOCR-VL, ERNIE-4.5-VL).
- The frontend consent hook silently enabled trust_remote_code on a clean scan
whenever the caller flagged the model as needing it.
Remove every trust_remote_code key from the model_defaults YAMLs (the loaders
already default to False when the key is absent) and delete the frontend silent
auto-enable, so trust_remote_code is only turned on after the user approves the
scanned code in the dialog.
The three models that genuinely run custom code ship auto_map, which the consent
gate detects on its own via _config_has_auto_map, so the dialog still fires for
them in inference, training, and export (Nemotron is also re-granted by the
trusted-org auto-enable in the workers). GLM-4.7-Flash has no auto_map:
glm4_moe_lite is native in transformers 5.0+ and it loads with
trust_remote_code=False, so its YAML flag was a no-op.
Adds test_yaml_trust_remote_code_removed.py.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Drop YAML sections emptied by trust_remote_code removal
Removing trust_remote_code from a model YAML whose section had no other key
left a bare `inference:` header, which PyYAML parses as None;
load_inference_config() then does `model_config.get("inference", {}).get(...)`
and crashes on the None. Drop those now-empty section headers (24 model
defaults, all the `inference:` section) so callers fall back to family/default
inference params, which is the same result those models had before (their only
inference override was trust_remote_code).
Strengthens test_yaml_trust_remote_code_removed.py to forbid any empty/None
top-level section and to load the affected models' inference config end to end.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add sweep asserting every model YAML loads via training + inference paths
Loads all model_defaults YAMLs through load_model_defaults (training) and
load_inference_config (inference) with the exact .get() access patterns the
routes use, so a malformed/None section that crashes either loader is caught.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Assert ex-TRC auto_map models still surface the consent dialog
Removing the trust_remote_code YAML default must not suppress the dialog for the
models that genuinely run custom code. The dialog is driven by the repo's auto_map
(via preflight_remote_code_consent_for_targets -> _config_has_auto_map), not the YAML
flag, so Nemotron/PaddleOCR-VL/ERNIE-4.5-VL still require consent; GLM-4.7-Flash (no
auto_map) takes no dialog and loads natively. Mocks only the Hub config + .py reader.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten comments in consent-gate changes
* Trim comments to be more succinct
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <michaelhan2050@gmail.com>
* Studio: use an isolated Node.js for the frontend build instead of replacing the system Node/npm
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: address Node isolation review (no-Node probe crash, PATH refresh, OXC provisioning, venv python, runtime node resolver)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix/adjust Node isolation for PR #6533
* Studio Node: don't cache a negative node resolution; accept Node metadata in setup.sh ownership guard
- node_runtime: memoize only a version-adequate executable so a Node installed
by a separate-process 'studio update' is picked up without a backend restart.
- setup.sh: _studio_owned_adoptable also accepts UNSLOTH_NODE_PREBUILT_INFO.json,
matching the setup.ps1 Node ownership guard (custom-home parity).
* Studio setup.ps1: skip OXC npm install gracefully when npm is absent
Mirror setup.sh's `command -v npm` guard so a pip-installed Studio with no
system Node skips the OXC runtime install (validator degrades at runtime) instead
of exit 1 aborting the whole setup. Tighten test_node_probe_guard.ps1's probe
regex so it only matches the two system-version probes, not this new npm guard.
* Wire test_node_probe_guard.ps1 into Windows CI for PR #6533
* Harden isolated Node install and probes for PR #6533
- install_node_prebuilt.py: keep an existing, still-usable isolated Node
when nodejs.org's dist index is unreachable instead of aborting the
update on a transient outage (existing_install_usable + tolerant fetch).
- install_node_prebuilt.py: pin NPM_CONFIG_PREFIX/npm_config_prefix and
drop NODE_PATH in _run_node so any npm -g stays inside the isolated
prefix; Windows npm otherwise writes to %APPDATA%\npm.
- install_node_prebuilt.py: resolve tar hard-link targets against the
archive root (symlink targets stay link-parent relative).
- setup.ps1: wrap the system node/npm probes in try/catch so a present
but broken shim degrades to the bundled Node instead of aborting setup.
- setup.ps1: run the isolated Node install with the handed-off/venv Python
(ReusedSetupPython); the main resolver runs later and bare python may be
a Store stub this early.
- setup.sh: log when the OXC validator runtime is skipped for missing npm,
matching setup.ps1.
- node_runtime.py: move the version-floor comment onto _version_meets_floor.
- Tests for the offline-reuse and broken-shim paths.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Trim verbose comments across the Studio Node installer for PR #6533
Comments-only pass: collapse the multi-line section banners to single lines,
drop comments that restate obvious code, and tighten the remaining docstrings
and "why" notes without losing intent. No code changes (verified with an AST
comment-only check on the Python files and a non-comment-diff scan on setup.sh
and setup.ps1). Net 109 fewer lines; the install, decision, and probe-guard
suites stay green.
* Harden Node install from review: validated Python, version floor, legacy home, lock race
For PR #6533, addressing the latest review pass:
- setup.ps1: run the isolated Node install with the validated reused/venv Python.
An incompatible reused interpreter (old venv, conda, stale UNSLOTH_SETUP_PYTHON)
is no longer used; fall back to the resolved python instead.
- setup.ps1: a STUDIO_HOME/UNSLOTH_STUDIO_HOME override equal to the legacy default
now uses the legacy sibling node dir (~/.unsloth/node), matching the runtime
resolver and setup.sh, so OXC can find the Node it installed.
- install_node_prebuilt.py: reject an explicit --node-version below the floor
(^20.19 || >=22.12 || >=23) instead of installing a Node the build cannot use.
- install_node_prebuilt.py: atomically rename a stale install lock before unlinking
so two concurrent runs without filelock cannot both acquire it.
Tests added for the version floor (parametrized + explicit-below-floor rejection).
Full install suite: 937 passed, 1 skipped; setup.ps1 parses; decision tests green.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address latest review: armv7l + later-fetch offline reuse for PR #6533
- install_node_prebuilt.py: reject 32-bit ARM (armv7l) up front. Node 24 LTS
ships no linux-armv7l build, so the old path failed late with a confusing
"no sha256"; it now fails fast with a clear unsupported-architecture error.
- install_node_prebuilt.py: extend the offline-reuse fallback to the SHASUMS and
archive fetches. If index.json resolves a newer Node but a later download fails
and a usable isolated Node is already on disk, keep it instead of aborting a
non-force update.
Tests added: armv7l/armhf are unsupported; a SHASUMS failure keeps an existing
usable Node and re-raises when none is present. Full install suite: 941 passed.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add UNSLOTH_STUDIO_HOME node-dir tests (install side + resolver) for PR #6533
* Add regression tests pinning the reuse path read-only and isolating installer writes
Lock in the two invariants behind the isolated-Node design: reusing a good
system Node never mutates the user's Node/npm, and the installer's own npm
calls only ever write inside its install_dir.
- tests/studio/install/test_install_node_prebuilt_logic.py: assert _run_node
redirects NPM_CONFIG_PREFIX/npm_config_prefix into install_dir and drops an
inherited NODE_PATH; assert _ensure_npm_floor scopes the npm self-upgrade to
install_dir (never -g against the system) and is a no-op once npm meets the floor.
- tests/sh/test_system_node_readonly.sh (new, wired into studio-backend-ci.yml):
the setup.sh NODE_SOURCE=system arm runs no global install and sets no
NPM_CONFIG_PREFIX, with a positive control that the bundled arm does.
- tests/studio/test_node_decision.ps1: symmetric structural guard that the prefix
pin and the only global install (bun) live in the bundled branch, not the system arm.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
* Studio: fix llama.cpp update toast tag and reload hint
The post-update toast used the job's to_tag, which is the bare bNNNN build
number (same as installed_tag), so it showed e.g. "b9726" instead of the full
release tag. Use status.latest_tag (e.g. b9726-mix-<sha>) to match the tag the
banner already shows, falling back to to_tag and then a generic label.
Also drop "Reload your model to use it." when there is nothing to reload: only
append it when a local model is loaded, since external-provider models do not
use llama.cpp.
* Fix/adjust llama update toast for PR #6493
---------
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
* Studio: show an actionable message when the GGUF runtime is missing
Selecting a GGUF model with no llama-server installed surfaced a generic
"Invalid model" in the UI, because validate_model's catch-all discarded the
real cause. Add LlamaServerNotFoundError (a RuntimeError subclass) raised by the
GGUF preflight in ModelConfig.from_identifier, and catch it in the validate
route so users get an actionable message: run `unsloth studio setup` to
download the prebuilt llama.cpp runtime. Other validation failures keep the safe
generic message. Adds a regression test.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: also map missing GGUF runtime to a 400 in load_model
validate_model already surfaces the actionable 'install the runtime'
message for LlamaServerNotFoundError; load_model fell through to the
generic 500 'Failed to load model'. Catch it there too so a GGUF load
without llama-server gives the same install hint instead of a 500.
* Trim comments for PR #6327
* Studio: fix stale validate test after #6398 and surface missing GGUF runtime on /load
- test_other_runtime_errors_do_not_get_gguf_message: after merging #6398,
validate_model surfaces a RuntimeError's own message, so a plain RuntimeError
no longer returns "Invalid model". Assert it does not receive the GGUF
install message instead (the prior assertion was stale after the main merge).
- Raise LlamaServerNotFoundError (not a plain RuntimeError) at the backend
load-time missing-binary branch, after diffusion routing, so /load returns the
actionable 400 like remote validation, instead of a generic 500.
- Share LLAMA_SERVER_NOT_FOUND_DETAIL between the from_identifier preflight and
the load-time raise so the message stays in sync.
- Add a propagation regression test for the non-tensor load path.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Reap Studio child processes when the parent dies abnormally
Standalone `unsloth studio` launches orphaned cloudflared and llama-server when
the parent exited without running the cooperative shutdown path (terminal-window
close, Task Manager End Task, SIGKILL): the children reparented to init and kept
running, leaving an authenticated Cloudflare tunnel up for days.
Add utils/process_lifetime.py: a parent-owned Windows Job Object
(JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE, children auto-inherit) plus Linux
PR_SET_PDEATHSIG, behind a best-effort helper that mirrors the desktop app's
windows_job.rs. initialize_parent_lifetime() runs at the top of run_server;
long-lived spawns (cloudflared, llama-server, RAG embedder, llama.cpp updater)
get the PDEATHSIG preexec, multiprocessing workers are adopted into the job, and
_graceful_shutdown plus atexit gain a terminate_all() backstop sweep. The
cooperative shutdown path is otherwise unchanged.
Verified on Linux: killing the parent now reaps cloudflared and llama-server
within ~2s instead of orphaning them.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* test: add real Windows kill-on-job-close integration test
Spawn a parent that installs the job and a child that inherits it, terminate
the parent, and assert the child is reaped. Skipped off Windows. Also make the
liveness probe Windows-safe (os.kill(pid, 0) terminates on Windows).
* Fix Win64 handle truncation in the Job Object calls
Set explicit argtypes so the 64-bit job/process handles are not marshaled as
c_int (which truncated them on Win64, failing AssignProcessToJobObject). Assert
install success in the Windows integration test.
* Bind multiprocessing workers to parent death; harden the sweep
Review follow-ups:
- Multiprocessing workers (inference/export/training/data-recipe/Xet) cannot be
given a preexec_fn by the parent, so adopt_pid alone left them orphanable on a
Linux SIGKILL. They now bind themselves with PR_SET_PDEATHSIG at startup via
bind_current_process_to_parent_lifetime(), wired into the shared
run_without_native_path_secret entrypoint and the Xet child entry.
- Wire the previously-missed data-recipe worker through adopt_pid.
- terminate_all now honors its timeout: SIGTERM, wait, then SIGKILL the
survivors, so cooperative children can exit cleanly.
- Track adopted pids with a /proc starttime identity and add forget_pid, so the
shutdown sweep never signals a recycled pid.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Michael Han <michaelhan2050@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: scale export GGUF size estimates from the real model size
The Export page showed hardcoded, model-independent GGUF quant size
labels (Q8_0 ~8.2 GB, BF16 ~14.2 GB, ...) calibrated for an ~8B model.
For a 35B MoE model like Qwen3.6-35B-A3B (67 GiB bf16, Q8 ~34 GiB) the
picker wrongly reported Q8 ~8.2 GB. Only the displayed estimate was
wrong; the actual export via save_pretrained_gguf was always correct.
Add GET /api/models/export-size, which returns a model's MoE-aware
fp16/bf16-equivalent size and total params using the existing
estimate_fp16_model_size_bytes (safetensors -> config -> local -> vllm).
The result is memoized and degrades to nulls so a size hint can never
break the Export page.
The Export picker now scales each quant from that size
(bytes ~= fp16_bytes * bits_per_weight / 16, GiB units to match the
model selector), and renders no size when it is unknown rather than a
misleading fixed number. The Est. size summary in the page and dialog
is restored now that the value comes from the backend.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio export-size: address review feedback
- Run the size estimate off the event loop with asyncio.to_thread so a slow
Hugging Face request cannot stall other API or SSE endpoints.
- Cache only successful estimates; a transient failure (offline, gated before
credentials) is no longer pinned as unavailable until restart.
- Forward the HF token so private and gated models can be sized, and refetch
when the token changes.
- Clamp the size formatter index so sub-1-byte values cannot pick an
out-of-range unit.
* Studio export-size: address second review pass
- Send the HF token in an X-HF-Token header instead of the query string, so
it never lands in URLs, logs, or browser history.
- Key the estimate cache by model id only (the fp16 size is token independent),
so HF tokens are never retained in the cache.
- Restrict local-path sizing to known Studio roots (outputs/exports/cache/home)
so an authenticated caller cannot trigger a scan of an arbitrary directory.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio export-size: fix CI (import-hoist + isolated-load test stubs)
- Import ExportSizeResponse from models.models in routes/models.py instead of
re-exporting it through models/__init__.py, so the import-hoist lint does not
flag a newly added but un-loaded re-export (models/__init__.py is unchanged).
- Add Header and ExportSizeResponse to the stubbed fastapi / models.models in
test_export_absolute_paths.py, which loads routes/models.py in isolation.
* Studio: validate export-size local path before filesystem access
CodeQL flagged the export-size local-path guard as path injection: the
user-provided model path was resolved and stat-ed before it was checked
for containment under a Studio data root. Decide containment by lexical
normalization (normpath/abspath/expanduser, no filesystem access) and
only touch the filesystem once the path is proven to sit under a trusted
root, so an unvalidated value never reaches a filesystem call. Add a
direct containment unit test (under-root, root itself, missing, /etc,
and '..' traversal).
* Studio: trim export-size comments to be more concise
Shorten docstrings and comments on the export-size endpoint, helpers, tests,
and frontend size utilities; drop comments that just restate the code. Verified
code-identical (comments only) via AST/TS-compiler check. No behavior change.
* Studio: harden export-size local-path handling
Address review feedback on the export-size endpoint's local sizing:
- Resolve symlinks and re-verify containment in _is_sizable_local_path so a
symlink inside a Studio root can't point the sizer outside it.
- Re-validate the resolved LoRA base before sizing, so a crafted adapter
whose base_model points outside the roots can't redirect the scan.
- Skip nested checkpoint-*/global_step* snapshots when summing local weight
sizes so a run dir's intermediate checkpoints don't inflate the estimate.
- Size the checkpoint directory for full fine-tune checkpoint exports (whose
base may be a local/custom path), keeping base-model sizing for adapters.
Adds tests for the adapter-base escape, symlink escape, and nested-checkpoint
exclusion.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
* Harden model fetching: consent gate for trust_remote_code
Add a load-path consent gate that scans a model's auto_map repository code
before it executes and blocks CRITICAL/HIGH findings unless the user pins
approval of that exact code version. Capability detection stays code-free,
reading raw config.json instead of AutoConfig.
- Scan config.json and tokenizer_config.json auto_map, nested local helpers,
and external owner/name--module repos; fail closed on partial downloads.
- Gate inference, training, and export workers, including the MLX path and a
LoRA's base model, and report requires_trust_remote_code from the raw config
so chat and auto-load surface the dialog.
- Verify trusted-org auto-enable against the Hub with the request token and key
the verdict cache by token; reject local-path and spoofed names.
- Add a consent dialog showing the flagged file, line, and surrounding code.
- Thread hf_token through the scan and load paths for gated repos.
* Address review: token handling, tokenizer/LoRA scan coverage, rollback
- Send the HF token for remote-code scans in the POST body, not the URL, so it
never lands in a log or browser history.
- Collect tokenizer_config.json auto_map files directly instead of relying only
on the repo file listing.
- Resolve a LoRA's base model for the validate flag and the scan endpoint so the
dialog scans the code the workers actually gate.
- Pass the request token to the training YAML trusted-org auto-enable.
- Resend a previously approved fingerprint when rolling back to a custom-code
model after a failed switch.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Consent UX: drop legacy chat toggle, fix decline copy, purge declined downloads
The per-model consent dialog is now the single approval path for custom
(auto_map) code in chat, so three leftovers from before it existed are removed:
- Remove the "Enable custom code" switch from Chat Settings and stop persisting
trust_remote_code, so a previously saved blanket-on cannot linger and load a
model without going through per-version review. The flag stays as an internal
YAML/preset default (e.g. first-party auto-enable); the load path still gates
every custom-code load on a fingerprint only the dialog produces.
- Reword the decline message and the auto-load toast to describe approving the
model's code from the dialog, not a missing settings toggle.
- On decline, purge the repo the scan downloaded so untrusted code is not left
on disk. A new /api/models/discard-remote-code endpoint deletes only a
metadata-only cache entry the scan created; it refuses local paths, loaded
models, and any repo with weight files cached, so a model the user already had
or pre-downloaded is always left untouched. The frontend only calls it when
the scan reported created_by_scan.
Adds discard-endpoint tests (delete metadata-only, refuse on weights/gguf,
refuse local, no-op when not cached) and a created_by_scan payload assertion.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Export: remove the user-facing trust remote code toggle
The Export page kept a "Trust remote code" switch (default on) next to the HF
token field. Like chat, custom (auto_map) code should be approved per model
through the load-time review dialog, not a persistent blanket switch, so the
toggle is removed. The export load path already routes through the same consent
dialog: an HF source now starts with trust_remote_code off and only enables it
when the user approves the scanned code in the dialog (a local checkpoint the
user exported stays trusted by default). With the dialog unreachable and no
approval, an HF source loads with trust_remote_code off, which fails closed
rather than running unreviewed code.
* Block loads of repos with unsafe files using Hugging Face's security scan
The trust_remote_code consent gate covers one load-time RCE vector (a repo's
auto_map Python). It does not cover the other: a malicious pickle inside a weight
file (pytorch_model.bin, *.pkl, *.dat) deserializes during from_pretrained even
with trust_remote_code False, so a repo with a normal config plus a poisoned
pickle slips past the existing gate.
Add a metadata-only malware gate that uses Hugging Face's own scan (picklescan +
ClamAV), read via model_info(securityStatus=True).security_repo_status. It never
downloads, opens, or unpickles the flagged files; it only reads the Hub's verdict
and surfaces the flagged file names. New evaluate_file_security runs
unconditionally (independent of trust_remote_code) in every load path (inference,
training SFT/MLX, export), blocking the load when a file is flagged
unsafe/suspicious/malicious. The /remote-code-scan preflight and the validate
endpoint also report the result so the consent dialog opens as a hard block (no
override) listing the flagged files, even for a repo with no custom code.
Policy: hard block with no user override; fail open when the scan is unavailable
(offline/unscanned) so legitimate loads are not broken; no first-party exemption
(a poisoned pickle in a compromised trusted repo still blocks); local paths and
GGUF are skipped (no Hub scan, non-pickle format). Blocking does not gate on
scansDone, since that is often false for clean repos and a file already flagged
unsafe is unsafe regardless.
Adds test_file_security.py covering the block/allow/fail-open/skip matrix.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: scan list-form tokenizer auto_map, gate unsafe files on all load paths
Fixes from a 10-reviewer pass on the model-fetching hardening:
- The remote-code scanner skipped tokenizer auto_map encoded as a [slow, fast]
list (transformers' standard tokenizer shape, e.g.
{"AutoTokenizer": ["owner/repo--tokenization_x.Slow", null]}). External
tokenizer code in that form was never fetched, scanned, or fingerprinted, so an
AutoTokenizer(trust_remote_code=True) load could run it. _auto_map_refs now
flattens string, list, and nested values. Adds a regression test.
- Compare-mode chat loads and background auto-load only gated on
requires_trust_remote_code, so a repo flagged unsafe by the Hub scan but with no
custom code skipped the hard-block dialog. Both now also gate on
requires_security_review, matching the main chat path.
- The /remote-code-scan and /validate routes collapsed a LoRA adapter to its base
before the malware scan, so unsafe files in the adapter repo itself were missed
in the pre-load review (the workers already scan both). Both routes now run the
file-security scan over the adapter and the base.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Require approval for all HIGH remote code, fail closed when unscannable
Tighten the load-time security gates based on review:
Consent gate
- HIGH-severity auto_map code now requires explicit, per-version approval for
every repo, including first-party unsloth/nvidia. The org is no longer a
blanket bypass: a compromised first-party repo with HIGH code still warrants
review. CRITICAL stays a hard block; clean code still loads after the consent
prompt.
- Fail closed when auto_map code is present but cannot be fully fetched or
listed to scan (gated, offline, transient, or a repo-listing failure that
could hide an imported helper). We cannot fingerprint code we cannot see, so
this is a non-approvable block, retryable once the repo is reachable.
- Scan auto_map from every config that can carry one (model, tokenizer, image
and feature processor, processor, video processor), not just config.json and
tokenizer_config.json, so a custom-processor model is not missed. The file
list is the single source of truth in remote_code_scan and is pinned to the
transformers filename constants by a guard test.
- Distinguish a genuine 404 (config truly absent) from a transient error: only
the latter forces a scan, so a repo with no config is correctly a no-op.
Malware gate
- Scan a remote repo even when its name ends in .gguf; only local paths skip the
Hub scan, so a repo cannot dodge the scan by naming itself "*.gguf".
- Correct the docstring: a file already flagged unsafe blocks regardless of
scansDone; the only fail-open path is an unavailable scan.
Coverage
- Resolve a remote LoRA adapter's base model (not just local directories) so the
base, where the code and weights actually execute, is scanned in validate,
the scan route, and the training and export workers.
- Gate the embedding training path (FastSentenceTransformer) with the malware
and consent checks, matching the other load paths.
Tests updated and added for each change.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Scope malware gate to the load-path vector; stop false-blocking first-party models
Follow-up hardening from a second review pass + a broad live model matrix
(unsloth/* , nvidia/* , third-party, and the eicar malware repo).
Malware / unsafe-file gate
- Scope the block to the actual RCE vector: a root-level file in a code-executing
format. from_pretrained deserializes weight files at the repo ROOT, so a flag is
only a load-path pickle vector there. Two exclusions, because neither is loaded:
inert formats (safetensors is tensor-only, gguf is non-pickle, configs/text/
images) and files in subdirectories. This keeps eicar blocked (its *.pkl/*.dat/
eicar_test_file sit at the repo root) while no longer false-blocking legitimate
first-party repos: nvidia/Nemotron-H-8B-Base-8K ships root safetensors plus NeMo
pickle checkpoints under nemo/ that the loader never touches, and the Hub flags
both; the gate previously hard-blocked it.
- Unknown / future non-"safe" levels now fail closed (block) instead of being
silently allowed, so Hub schema drift cannot introduce a bypass; in-progress
("pending"/"scanning"/"error") levels stay non-blocking to avoid false blocks.
Consent gate
- Ignore a STALE own-repo auto_map target that is absent from the repo listing (an
older config pointing at a file the repo no longer ships) instead of failing the
whole repo closed as unscannable. The present .py are still fully scanned, which
is the stronger coverage, and a file that is not there cannot execute. This
unblocks first-party models like unsloth/PaddleOCR-VL (its tokenizer_config.json
names processing_ppocrvl.py while the repo ships processing_paddleocr_vl.py). A
referenced .py that IS present but cannot be fetched, and a repo-listing failure,
still fail closed.
Remote LoRA base resolution
- Distinguish a genuine 404 (not a LoRA / repo absent -> None) from a transient
error: the transient case is retried once, then logged as a WARNING (a missed
base is scanned by neither gate) rather than silently skipped.
Discard endpoint
- Treat .onnx and .ckpt as weights so a repo whose only heavy artifact is one of
those is never eligible for the declined-download purge.
Tests added for each: load-path scoping (safetensors/subdir/Nemotron-H shapes,
unknown-level fail-closed, pending non-block), stale own-repo auto_map ref, remote
LoRA transient retry, and the empty-config-list (all-404 -> []) semantics.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Make LoRA-base transient-warning test robust to logging backend
Assert on the logger object directly instead of capsys, so the test does not
depend on whether the real structlog logger or the module-stub logger is active
(which varies with test collection order).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Allow a repo with auto_map but no executable code (e.g. GGUF) instead of blocking
A config can declare an auto_map yet the repo ship NO executable .py -- most
commonly a GGUF repo whose config.json carries an auto_map copied from the original
model (e.g. unsloth/Llama-3_1-Nemotron-Ultra-253B-v1-GGUF references
modeling_decilm.py, which the GGUF-only repo does not contain). A GGUF model loads
through llama.cpp, which never executes auto_map, and transformers cannot run a file
that is not present, so there is nothing to scan and trust_remote_code is a no-op.
The fail-closed change treated this empty result the same as "code is present but we
could not fetch it" and hard-blocked the load. Distinguish the two: repo_remote_code_files
now RAISES RemoteCodeUnscannable when code is present but cannot be fully fetched or
listed (offline / gated / transient / a present .py that 404s / a listing failure),
and returns an empty dict only when the listing succeeded and the repo genuinely ships
no executable .py. The consent gate blocks on the exception (fail closed) and allows the
empty case as a no-op. Real unscannable code still hard-blocks; eicar and CRITICAL/HIGH
custom code are unaffected.
Verified against all 37 unsloth/*Nemotron* models (two GGUF repos were false-blocked,
now load) and the existing matrix (eicar still blocks; DeepSeek-OCR / NVLM-D-72B still
prompt approvable consent). Tests updated to expect the raise for unscannable cases and
added for the no-executable-code no-op.
* Ignore vestigial auto_map in GGUF repos (llama.cpp never runs it)
A GGUF repo's config.json is often copied verbatim from the original
transformers model, auto_map and all, but a GGUF load goes through
llama.cpp which never executes auto_map, so the config is inert. Treat
a direct .gguf reference, and a repo that ships .gguf weights with no
.safetensors, as having no remote code so the consent flow is never
triggered. A mixed repo with both .gguf and .safetensors is still gated,
since the safetensors variant would load through transformers where
auto_map does run. The check sits behind the existing auto_map-present
gate so normal models pay no extra repo listing.
* Add scanner-result copy to the remote-code consent dialog
Make the consent dialog state the scan outcome in plain language for
every model. When the static scan finds nothing, reassure the user with
'Our automatic scanner did not flag any worrying files, but please
double check.' (shown only for the clean, approvable case). When the
scan flags custom code or unsafe files, label the list with 'Our
automatic scanner flagged issues including:'. The Hugging Face
attribution for unsafe files stays in the dialog description.
* Close GGUF-suffix consent bypass for repo ids ending in .gguf
The .gguf short-circuit in _config_has_auto_map skipped the scan for any
model name ending in .gguf, including a bare two-segment repo id like
'evil/model.gguf'. Such a repo can still ship safetensors plus auto_map
Python that transformers would execute, so skipping the scan was an
asymmetric bypass (file_security already scans those repos). Restrict the
short-circuit to genuine direct GGUF file references via
_is_direct_gguf_file_ref: a local .gguf path, or a remote repo_id plus
filename (three or more segments). A two-segment repo id named *.gguf now
falls through to the config scan and _is_gguf_repo file inspection, so it
only skips consent when it actually ships .gguf weights and no safetensors.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Align consent dialog body with the title and fix narrow-width overflow
The scan results (the 'Our automatic scanner...' label, finding/unsafe
cards, and the clean-scan reassurance) sat at the dialog's left padding
while the title and description were indented past the status icon, so
the body did not line up under the description. Move the title,
description and results into one column to the right of the icon so they
share a left edge, and let that column fill its width so the description
no longer wraps early.
Also stop a wide code snippet from pushing the dialog off-screen on
narrow viewports: AlertDialogHeader is a grid with place-items-center,
which sized the content row to its content; give the row w-full so it
fills the track, and add min-w-0 down the results chain so the snippet
scrolls inside its card instead of widening the dialog. Verified aligned
and contained from mobile portrait through ultrawide.
* Treat a repo as GGUF-only only when it ships no transformers weights
_is_gguf_repo excluded only .safetensors, so a repo with a .gguf and a
pytorch_model.bin (or .pt/.pth/.h5/.msgpack/.onnx/.ckpt) and no
safetensors was treated as GGUF-only and skipped the consent scan, even
though transformers can load that weight set and execute the repo's
auto_map code. Require the absence of ANY transformers-loadable weight
before treating the repo as a llama.cpp-only GGUF load. A genuine
GGUF-only repo (only .gguf) is still inert; a mixed repo with any pickle
or safetensors weight is gated. Adds a regression test across all the
non-safetensors weight formats.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Block flagged subdir weight shards referenced by a root index
The malware gate treated every subdirectory file as non-loadable, but
from_pretrained deserializes a subdir shard a root index references
(pytorch_model.bin.index.json -> shards/...-00001-of-00002.bin). Read the
root weight indexes and block a flagged subdir pickle the weight_map
points at; a flagged subdir pickle no index lists (NeMo nemo/*.distcp)
stays non-blocking, and an inconclusive index lookup fails closed.
* Pass hf_token to the export checkpoint load
ExportBackend.load_checkpoint scanned with hf_token in the worker but
loaded the weights unauthenticated, so a gated/private checkpoint passed
preflight then 401'd at from_pretrained. Add hf_token to load_checkpoint
and forward token to every from_pretrained branch; the worker passes the
command's hf_token.
* Scope created_by_scan to every HF cache the discard searches
created_by_scan used get_cache_path (active HF_HUB_CACHE only) while
/discard-remote-code deletes across active, legacy, and default caches. A
repo the user already had in a legacy/default cache was marked
scan-created and deleted on decline. Check all three caches for the repo
dir before declaring the scan created it.
* Scan the full .py closure of external auto_map repos
An auto_map cross-repo ref (owner/name--module.Class) only had its entry
file downloaded, but transformers also fetches that file's relative
imports from the same repo, so a dangerous helper.py was left outside the
scanned fingerprint. List each external repo's .py and scan the whole set
(plus the referenced entry files); fail closed if the repo cannot be
listed or fetched.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fail closed when a weight index cannot be fully read
_indexed_shard_paths treated a partial result as definitive: if one weight
index read cleanly but another failed transiently, it returned the shard
paths it did see. A flagged subdirectory pickle listed only by the index we
could not read would then be classed as "not a load input" and skipped,
re-opening the very fail-open this guard was added to close.
Return None whenever any index read is inconclusive, even if another read
cleanly, so the caller blocks the already-flagged subdir pickle. A repo that
ships no index files raises EntryNotFoundError for each (never inconclusive)
and still returns an empty set.
* Match cached repos case-insensitively in the created_by_scan guard
_repo_in_any_hf_cache resolved casing only against the active cache and then
probed every cache with an exact directory name. A case-variant already
present in a legacy or default cache (models--Unsloth--Foo for a scan of
unsloth/foo) was missed, so the repo was marked created_by_scan and deleted
on decline -- but discard_remote_code_download deletes case-insensitively,
so that delete would hit the user's pre-existing cache entry. Detect
case-insensitively too, mirroring the deletion path.
* Skip remote-code and security review for selected GGUF variants
validate_model ran the trust_remote_code and Hugging Face security-scan
preflight against the repo even when the selected artifact is a .gguf. A
GGUF loads through llama.cpp, which never executes the repo's auto_map
Python and never deserializes root pickle weights, so repo-level Transformers
artifacts (a config.json with auto_map, or an unsafe pytorch_model.bin next
to the .gguf in a mixed repo) are inert for that load. Gating the GGUF on
them is a false positive. Run both preflights only for non-GGUF loads.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Scope the malware gate to actual load roots and serialized files
Two fixes to evaluate_file_security so it neither misses a load-path pickle nor
false-blocks an inert file:
- Honor subdirectory load roots. Spark-TTS / BiCodec call from_pretrained on the
snapshot's LLM subdirectory, so a flagged pickle directly under it is a
root-level load artifact there. A new load_subdirs parameter (set from the
model's audio type via security_load_subdirs) reclassifies those files relative
to the load root and looks for weight indexes under it, so a flagged shard in
that subdir is no longer skipped as "not root-level".
- Exempt source files. A root .py is never deserialized by from_pretrained;
executable repo code runs only through auto_map, which the remote-code consent
gate scans. Flagging a Python helper here would false-block a repo that merely
ships a build or train script.
* Scan a LoRA adapter and base as one consent unit, and gate MEDIUM code
A LoRA load runs both the adapter's and the base's repo code. The consent gate
scanned them separately and pinned one fingerprint per repo, so an adapter that
shipped its own auto_map code was either never shown in the dialog (which only
saw the base) or impossible to approve with the base's fingerprint.
evaluate_remote_code_consent_for_targets now scans all of a load's repos as a
single combined unit and pins ONE fingerprint over the union of their code, so
approving the load approves every repo's code together. evaluate_remote_code_consent
becomes a thin single-target wrapper, and an unscannable target fails the whole
load closed.
Also gate MEDIUM findings: like HIGH they now block pending pinned approval, so a
direct API caller cannot run flagged code by setting trust_remote_code=True
without consenting. Only a clean scan loads without a fingerprint.
* Preflight a LoRA load's adapter and base as one combined consent scan
scan_model_remote_code rewrote a LoRA adapter to its base and scanned only the
base for remote code, so the dialog never surfaced an adapter's own auto_map
code. Scan the adapter and base together through
preflight_remote_code_consent_for_targets, which pins one combined fingerprint
the worker gate accepts. The malware preflight is also scoped to each target's
load subdirectories.
* Apply combined consent and subdir-aware malware scan in load workers
Each load worker (inference, export, training) evaluated remote-code consent
once per target with a single shared fingerprint, so a LoRA adapter that ships
its own auto_map code could not be approved by the base's fingerprint. They now
scan the adapter and base together via evaluate_remote_code_consent_for_targets,
which pins one combined fingerprint over the union of their code. The malware
scan in each worker is also scoped to the model's load subdirectories so a
flagged pickle under a from_pretrained load subdir is not missed.
* Report a consistent trust_remote_code requirement after a model loads
validate_model reports requires_trust_remote_code from the YAML default OR the
raw auto_map, but the load, already-loaded, and status responses reported only
the YAML default. A custom-code model approved and loaded via auto_map was then
reported as not requiring trust_remote_code, so the frontend stored false and a
later retry or rollback sent trust_remote_code=false and failed.
A shared resolver reports the same requirement for a loaded model (a value
stored at load time, else the trust_remote_code the load used, else the YAML
default, else the raw auto_map check), and the load response persists it so the
status and already-loaded paths stay consistent. The selected-GGUF security
review is also scoped to the model's load subdirectories.
* Run the consent gate on training resume and for YAML-only trust_remote_code
Three frontend gaps left a model loading without the trust_remote_code it needs:
- The shared consent helper returned early when the scan found no auto_map and no
unsafe files, dropping a requirement that comes from a model's Studio YAML
default (e.g. GLM-4.7-Flash). It now grants the caller's requirement with an
empty pin instead of sending trust_remote_code=false.
- Resume-from-history called startTraining directly with no consent gate, so a
resumed run whose model needs custom code (or an old run with no approved
fingerprint) hit the worker block with no dialog. It now runs the same gate as
a fresh start.
- HF export passed requiresTrustRemoteCode=false for every HF source, so a
YAML-only model could not flip the flag before export. It now signals the
requirement for HF sources.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Cover both LoRA repos in validate, report GGUF as inert, purge all declined repos
Three follow-on gaps from the combined adapter+base consent work:
- validate_model resolved requires_trust_remote_code from the base alone, so a
LoRA adapter that ships its OWN auto_map code (with a plain base) was reported
as not needing trust_remote_code and the consent dialog never opened. It now
checks the [adapter, base] target set, matching the scan route and the workers
(which already gate both) and the security review already running over both.
- The already-loaded, loaded, and status responses for a selected GGUF reported
requires_trust_remote_code from the model's YAML default. A GGUF loads through
llama.cpp, which never executes the repo's auto_map Python, so the requirement
is inert for that load. They now report False, matching validate_model (which
already skips both gates for GGUF) so a status refresh cannot flip the flag
back on.
- The remote-code scan downloads both the adapter's and the base's config, but
created_by_scan tracked only the primary, so a base the scan was first to pull
into the cache was left on disk when the user declined. The scan now reports
scan_created_repos (every repo it newly cached) and the decline cleanup purges
each; created_by_scan stays for older clients. The frontend falls back to the
primary flag when the list is absent.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Scan the repo the load fetches, purge external code on decline, harden consent pins
Six follow-on hardening fixes from a fresh review pass over the gate:
- The malware gate scanned the literal "Spark-TTS-0.5B/LLM" alias, but the trainer
downloads it as unsloth/Spark-TTS-0.5B and loads LLM/, so the alias 404'd and
failed open, missing a flagged LLM/ pickle. evaluate_file_security now resolves
the alias to the repo the loader fetches and scans LLM/ as a load root.
- security_load_subdirs relied only on tokenizer detection, which fails on an
unresolved alias or offline; it now also honors the Studio YAML audio_type
default, so a BiCodec LLM/ load root is not missed.
- The remote-code scan downloads external auto_map repos (owner/name--module.Class),
but the decline cleanup tracked only the model/adapter/base, leaving the external
untrusted code cached. The scan now enumerates external auto_map repos and reports
the ones it created in scan_created_repos, so a decline purges them too.
- External auto_map refs failed the whole load closed on a stale or mis-derived
dotted ref (sub.mod.py vs the real sub/mod.py) even though the actual file was
present and scanned. They now drop such refs when the repo listing is real, exactly
like the own-repo path; an empty/incomplete listing still fetches and fails closed.
- The combined consent fingerprint keyed code by the raw target string, so the scan
endpoint's canonicalized casing and a worker's raw user input produced different
pins for identical code, rejecting a valid approval. Hub repo ids are now folded to
lowercase in the key (local paths stay case-sensitive), so the pin tracks the code.
- Export threaded hf_token into the weight load but not into detect_audio_type /
is_vision_model, so a gated multimodal base 404'd in detection and fell through to
the text loader. Both probes now use the same token.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Thread the token through check-vision and guard the gate's parallel sites
The /check-vision endpoint classified a model without the hf_token, so a gated or
private vision model 404'd in the probe and was reported as a plain text model --
the same dropped-token shape as the export probes, at a sibling site. It now passes
the token like the neighboring /check-embedding endpoint.
Add deterministic consistency guards (tests/test_security_gate_consistency.py) that
enumerate the gate's parallel sites mechanically instead of relying on a review to
spot a missed sibling: every is_vision_model / is_embedding_model / detect_audio_type
caller under routes/ and core/ must thread the token, every GGUF response must report
trust_remote_code via the resolver or False (never the raw YAML default), and every
load worker that runs the malware or consent gate must resolve the LoRA base. A new
site that drops the token or mis-reports the requirement now fails CI directly.
* Narrow the LLM alias rewrite and make audio detection token-aware
Three fixes from the confirmatory review, one a regression from the previous round:
- _load_scan_target rewrote EVERY remote repo ending in "/LLM" to unsloth/<parent>,
so a real third-party repo named "<owner>/LLM" was scanned as unsloth/<owner>
while the loader still fetched the real repo -- a fail-open hole introduced when
the Spark-TTS alias handling was added. It now rewrites only a registry-known
bicodec alias; every other "/LLM" repo is scanned as itself.
- detect_audio_type cached results under the bare model name, so an unauthenticated
probe of a gated/private repo cached None and poisoned a later authenticated call
with the token. The cache is now keyed by (normalized_name, token_fingerprint),
matching the vision cache.
- The training fallback /check-vision call dropped the hf_token, misclassifying a
gated/private VLM when the config endpoint failed. It now passes the token, like
the getModelConfig call it falls back from; checkEmbeddingModel takes the token too.
Extend the consistency guards: every capability cache must be keyed by a tuple
including the token, so a cache re-declared as Dict[str, ...] fails CI.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Document the broad .py scan as deliberate and enforce it with a test
The remote-code scanner scans every .py in a repo once an auto_map exists, not
just the auto_map entry's static import closure. This is intentional: the entry
module can reach a sibling via an absolute import, importlib, or exec, none of
which a static relative-import closure follows, so closure-only scanning would be
a real bypass of a load-time RCE gate. The broad scan never under-scans; the cost
is that an unrelated benign script can over-block, which is the safe failure
direction (HIGH stays approvable; only CRITICAL hard-blocks).
Spell this out at both the local and remote scan sites so the choice reads as
deliberate, and add a test asserting an unrelated, never-imported .py is still
scanned -- so a future narrowing to the static closure fails CI.
* Purge a declined remote LoRA adapter the scan downloaded
scan_model_remote_code probed the created-by-scan state AFTER resolving the base,
but get_base_model_from_lora_identifier downloads a remote adapter's own
adapter_config.json, so the adapter looked already-cached and was dropped from
scan_created_repos. On decline the adapter -- including the auto_map .py the
preflight fetched -- was left on disk, defeating the "untrusted code is not left
on disk" guarantee for the adapter itself.
Snapshot the primary's cache state BEFORE base resolution and use it when marking
the adapter scan-created; on any probe error treat it as pre-existing so a decline
never deletes it. The base and external repos are unaffected (their configs are not
downloaded before their own probe). Add a test that models the mid-scan download
side effect, which the prior static-stub tests did not.
* Clear remote-code approval when the training model changes
Switching the training model from an approved custom-code model to a clean one
kept the previous model's trust_remote_code=true and approved fingerprint in the
store: setSelectedModel reset visionImageSize on a true switch but not the
remote-code approval. The clean model then trained with trust_remote_code=true,
which bypasses the compiler and disables fused cross-entropy.
Reset trustRemoteCode and approvedRemoteCodeFingerprint on a true model switch.
The new model's own YAML default is re-applied by loadAndApplyModelDefaults, and a
custom-code model still re-opens the consent dialog before training starts, so the
only change is that a clean model no longer inherits a stale approval.
* Trim verbose comments across the model-fetching hardening changes
Condense the explanatory comments and docstrings introduced across the
trust_remote_code consent gate, the malware/unsafe-file gate, the remote-code
scanner, the load workers, the model routes, and the security frontend into
fewer, tighter lines while preserving every security rationale (fail-open vs
fail-closed direction, the deliberate broad-scan anti-bypass note, the
empty-vs-unscannable distinction, stale-ref handling, and the alias-rewrite
spoof guard).
Comments and docstrings only. No code, logic, identifiers, or test behaviour
changed; verified comment-only via the AST/TypeScript checker (40/40), with the
backend test suite and frontend tsc green.
* Do not cache transient audio-detection failures
detect_audio_type cached _detect_audio_from_tokenizer's result
unconditionally, so a transient read failure (network error or 5xx,
returned as None) poisoned the cache and the later successful probe never
ran. Mirror the vision cache: _detect_audio_from_tokenizer now returns
(audio_type, definitive) and the caller caches only definitive results.
A read that succeeds with no audio tokens, or clean 404s for every
tokenizer path, stays a cacheable None; only a genuine transient failure
(connection error, timeout, 5xx, malformed body) skips the cache so the
next call retries.
---------
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: pin CUDA_DEVICE_ORDER=PCI_BUS_ID and list GPUs at startup
On a mixed-GPU host, Studio could load a model onto a different physical
GPU than the one it selected. The free-VRAM probe numbers GPUs via
nvidia-smi (PCI-bus order), but CUDA defaults to FASTEST_FIRST ordering,
so a selected index written into CUDA_VISIBLE_DEVICES resolved to the
wrong card. Example: 5090 + RTX PRO 6000, the picker chose the emptier
RTX PRO 6000 (nvidia-smi index 1) but CUDA read index 1 as the 5090.
Pin CUDA_DEVICE_ORDER=PCI_BUS_ID at import (before any CUDA context is
created) in both the Studio entrypoint and the hardware module, so torch,
nvidia-smi, and CUDA_VISIBLE_DEVICES share one index space. setdefault
keeps an explicit user override intact. Child processes inherit it via
os.environ.
Also list every detected CUDA GPU with its index at startup instead of
naming only device 0, matching nvidia-smi -L and making the selected
index unambiguous on multi-GPU hosts.
* Studio: make CUDA_DEVICE_ORDER tests exercise module import and respect user override
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: guard full _print_cuda_device_list body and fix test PYTHONPATH trailing separator
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: stop the llama.cpp update banner flickering and show the download size
The banner animated in and out with a motion opacity + scale + translate
transition. That transform/opacity transition promotes a GPU compositing
layer whose first and last frame can flash for a moment on real displays,
which reads as a flicker on appear and again on dismiss/snooze. Drop the
animation and render the banner as a plain conditional mount: it appears
and leaves cleanly with nothing to flash.
Also surface the download size. update-status now reports the size of the
prebuilt that Update would fetch (the latest-release asset matching this
host's bundle), and the banner shows it as whole MB next to the no-restart
note, so the cost of the update is clear before clicking.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Show llama.cpp update size for upstream and source-build installs
update_download_size_bytes now accepts the upstream ggml-org ubuntu-/win-
asset suffixes and falls back to the marker's binary_repo, so the size
resolves for CPU/ROCm prebuilts (the fork publish repo only carries the
app-* and macOS bundles). The source-build update path now populates
update_size_bytes from the resolved asset, matching the marker path.
Both fail open to null. Adds regression tests for the upstream and
source-build size lookups and the route field round-trip.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: add 'Load on selection' toggle to configure load options before loading
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: seed staged speculative decoding from the standing default
* Studio: address PR review for load-on-selection staging
* Studio: handle direct GGUF staging and stale-stage edge cases from load-on-selection review
* Studio: cancel replaced staged downloads and keep staged pick on load failure
* Studio: centralize staged-download cancel and guard staged-load restore
* fix: address staged GGUF load review
* fix: honor staged GGUF load metadata
* fix: clarify load-on-selection tooltip
Keep the load-on-selection hint visually anchored to the control and make the on/off behavior explicit without changing the broader deferred-load flow.
* Studio: reset orphaned staged knobs on abandon and cap Max Tokens to staged context
* Studio: remove dead code and cancel staged download when loading a different model
* fix: surface staged model in run settings before deferred load
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: imagineer99 <samleejackson0@gmail.com>
* Studio: add shared Xet-primary download helper with HTTP stall fallback
Xet is the fast default transport in huggingface_hub, but a stalled Xet
transfer hangs with no progress and no exception, and a blocked native thread
cannot be killed. The safetensors inference path already recovers (subprocess
watchdog + respawn with HF_HUB_DISABLE_XET=1); the GGUF and training paths do
not. Add a reusable helper that the in-process paths can adopt.
utils/hf_xet_fallback.py:
- DownloadStallError (moved here from core/inference/orchestrator.py, which now
imports it; behavior unchanged, still a RuntimeError subclass).
- get_hf_download_state / start_watchdog: a no-progress watchdog built on the
sparse-aware hub.utils.hf_cache_state helpers; fires only while a .incomplete
is present and the on-disk byte total is unchanged for stall_timeout.
- hf_hub_download_with_xet_fallback: cached files short-circuit; otherwise the
download runs in a spawn child (own process group) supervised by the watchdog.
On a stall it kills the child, makes the partial safe for HTTP via
prepare_cache_for_transport, and respawns once with HF_HUB_DISABLE_XET=1. Cancel
and deterministic errors (auth/missing/disk) propagate without a fallback.
Tests cover the watchdog state machine, the transport decision logic, and a
regression lock that HF_HUB_DISABLE_XET is honored in a fresh interpreter.
* Studio: route GGUF Chat-Mode downloads through the Xet->HTTP fallback
The GGUF load path (_download_gguf main+shards, _download_companion_gguf for
mmproj/MTP) called a bare blocking hf_hub_download with no recovery, so a Xet
stall hung the Chat-Mode load with no fallback. Route those three calls through
hf_hub_download_with_xet_fallback: Xet stays primary, HTTP is used only if Xet
stalls, per-file so finished shards stay cached. The existing _cancel_event is
threaded through, the Cancelled sentinel is preserved, and companions stay
best-effort (a terminal stall is swallowed to None). Cached files short-circuit
in the helper with no subprocess, so the fast path is unchanged.
The two offline mmproj tests are repointed from huggingface_hub.hf_hub_download
to the new call boundary (the helper) since the download now goes through it.
* Studio: recover a stalled training model-load via Xet->HTTP respawn
Training runs in a spawn subprocess and FastModel.from_pretrained downloads
internally, so the download cannot be wrapped per-file like GGUF. Instead the
worker now watches the HF cache during the model-load phase (emitting
model_load_started / model_load_completed and a stall event), and the parent
recovers a stall by terminating the worker and respawning it once with
HF_HUB_DISABLE_XET=1.
worker.py: set HF_HUB_DISABLE_XET=1 before any HF import when the parent passes
disable_xet (respawn), and wrap trainer.load_model with start_watchdog.
training.py: plumb disable_xet through the config; track the model-load window;
on a first-load stall arm a one-shot respawn (handled on the exiting pump thread,
so no pump self-join) that preserves the DB run row (history is not duplicated)
and re-runs the load over HTTP. A second stall, or a stall outside model-load,
surfaces as a normal error. W&B init happens after model-load, so a pre-load
respawn cannot duplicate it; the dataset is re-formatted in the new worker.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: add gated test-only fault-injection hook for the Xet stall path
UNSLOTH_HF_XET_FORCE_STALL=1 makes the Xet download attempt write a partial
blob and hang, so the no-progress watchdog and the HTTP fallback can be
exercised end to end against a real repo (never set in production). Used to
verify recovery on real models: a forced Xet stall on a 5.37GB Qwen3.5-35B-A3B
shard triggered the watchdog and the HTTP retry downloaded the correct file
(sha256 verified).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: tighten Xet-fallback comments and consolidate its tests
Trim docstrings and inline comments across the Xet->HTTP fallback code to
the non-obvious why (spawn-not-thread, killpg-not-getpgid, the sparse-partial
HTTP-resume hazard); drop comments that merely restate the code. Verified
comment-only with an AST signature check.
Merge the three helper-level test files (watchdog, transport policy, and the
HF_HUB_DISABLE_XET regression lock) into tests/test_hf_xet_fallback.py, and
prefer the real structlog over a bare stub so test collection order cannot
leak an incomplete module to others that log at import.
Full backend suite: 3455 passed, 14 pre-existing flash-attn failures only.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: log transformers version-switching decisions and stop swallowing MLX activation failures
Two logging gaps in dynamic transformers version switching (issue #6103):
1. get_transformers_tier returned a tier with no trace of why. Add an
info log at each decision point naming the model and the trigger
(which substring matched, or which config check fired), so a model
landing on the wrong tier is diagnosable.
2. The MLX fast-path in run_training_process activated the transformers
version inside a bare 'except Exception: pass', silently swallowing
failures while the non-MLX path reports them. A missing or broken
version venv (e.g. Gemma-4 needing 5.5.0) left no trace and only a
confusing downstream crash. Extract a small _activate_transformers_version_or_warn
helper that logs a warning on failure while keeping the non-fatal
fall-through, and call it from the MLX path.
Adds tier-selection logging tests and helper warn/silent tests.
* Studio: clarify path-prepend log, warn on venv version mismatch, log per-package install progress
Completes the remaining logging items of #6103 in studio/backend/utils/transformers_version.py:
- activate_transformers_for_subprocess: the early "Activated transformers X.X.X" line was misleading because at that point only the venv directory has been prepended to sys.path, not imported. It now says it prepended the venv to sys.path and notes the loaded version is confirmed later by "Subprocess loaded transformers ...".
- _venv_dir_is_valid: a detected version mismatch is logged at warning instead of info, since it immediately triggers a full venv wipe and reinstall that should be visible in the logs.
- _ensure_venv_dir: log each package as it starts installing with an N/M progress counter, so a slow runtime install is not mistaken for a hang (pip/uv output is piped and only surfaced on error).
Adds tests covering all three behaviours; pre-existing unused imports are left untouched.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: make tier log-capture tests independent of import order
The new issue #6103 caplog assertions in test_transformers_version.py
relied on the module-level sys.modules.setdefault("loggers", stub)
winning the import race. In a full backend pytest run another module
(for example test_log_filter_no_truncation, collected earlier) imports
the real loggers first, so the setdefault is a no-op and
transformers_version.logger becomes a structlog/stdout logger that
caplog cannot capture -- the tier, activation, venv-mismatch and
install-progress log assertions then fail even though the line was
emitted.
Bind a real stdlib logger to transformers_version.logger for the
duration of each test via an autouse fixture, so the module logs through
logging and caplog captures them regardless of collection order.
* Studio: log local checkpoint tier decisions and warn on MLX inference activation
- get_transformers_tier: the local config.json fast path returned a tier
without logging it, so local checkpoints stayed opaque while HF ids were
traceable. Log each decision there too, with a caplog regression test.
- inference worker: the MLX path swallowed _activate_transformers_version
failures with a bare except, the same gap issue #6103 fixed for training.
Warn instead, keeping the non-fatal fall-through.
---------
Co-authored-by: Daniel Han <michaelhan2050@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: make lifespan shutdown resilient to a dead default executor
On an abrupt shutdown (closing the Windows console window, or interpreter
teardown racing uvicorn's graceful stop) the event loop's default thread-pool
executor can already be shut down by the time the FastAPI lifespan shutdown
runs. The first post-yield statement was an unguarded
`await asyncio.to_thread(terminate_hub_downloads)`, so executor.submit raised
`RuntimeError: cannot schedule new futures after shutdown`. That raise
propagated up through every nested merged_lifespan __aexit__, aborted the rest
of the cleanup (DEVICE reset, compiled-cache clear), and surfaced as
"Application shutdown failed. Exiting."
Extract the post-yield cleanup into utils/lifespan_shutdown.run_lifespan_shutdown
and guard each step independently. On the to_thread RuntimeError, fall back to
running the (already best-effort, quick) terminate inline on the loop thread so
shutdown still completes cleanly. The helper is dependency-injected and free of
the heavy backend import graph, so it is unit-tested in isolation.
Add tests/test_lifespan_shutdown.py (4 cases: dead-executor survival, normal
path, terminate error, clear error). Validated on windows-latest and
ubuntu-latest runners on Python 3.13.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: only retry terminate inline when scheduling fails, not when the body raises
Address review feedback on the shutdown helper: the previous
`except RuntimeError` after `asyncio.to_thread(terminate_downloads)` could
not tell a dead-executor scheduling failure from a RuntimeError raised by
terminate_downloads itself, so a body-side RuntimeError on a healthy executor
ran the cleanup a second time inline.
Schedule via loop.run_in_executor and await separately: a dead default executor
raises synchronously at submit time (inline fallback), while a body exception
only surfaces when the future is awaited (logged, never retried). Add a
regression test that a body RuntimeError runs terminate exactly once.
* Studio: run terminate cleanup with a copied context (parity with asyncio.to_thread)
Simulation across the executor-state x exception x DEVICE matrix surfaced the one
behavioural difference from the original implementation: asyncio.to_thread copies
the caller's contextvars into the worker thread, while a bare
run_in_executor(None, fn) does not. Restore exact parity by scheduling
ctx.run(terminate_downloads) from a contextvars.copy_context(), so the refactor
is a behavioural no-op apart from the intended dead-executor recovery. Add a
regression test asserting the copied context is visible to terminate_downloads.
* Studio: tighten comments in lifespan_shutdown helper and tests
Condense the verbose docstrings/comments to the non-obvious rationale and drop
the self-evident ones. Verified comment-only with comment_tools.py check
--strip-docstrings (code signature unchanged); tests and sims still green.
* Studio: address review nits on lifespan shutdown helper
Annotate hw_module as types.ModuleType, reword the schedule/await comment
(inline fallback runs, it does not retry), and add a test for the public
loop.shutdown_default_executor() path (the 'Executor shutdown has been called'
RuntimeError that real uvicorn shutdown takes, distinct from the submit-time
'cannot schedule new futures after shutdown').
---------
Co-authored-by: Daniel Han <michaelhan2050@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: show llama.cpp version and GPU specs in the About panel
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: run the hardware endpoint off the event loop
* Studio: show the full llama.cpp release tag (incl -mix-<sha>) in the About panel
* Studio: order About-panel GPUs by visible ordinal and skip the llama.cpp probe during updates
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix hardware info refresh and endpoint scope
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Etherll <61019402+Etherll@users.noreply.github.com>
* Studio: enable stdio MCP servers on a loopback bind
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: address codex review on stdio MCP loopback gate
* Studio: fix banner URL and preserve stdio MCP env opt-in on network binds
* Studio: scope loopback to exact aliases and honor force-disable on run_server reuse
* Studio: cover force-disable across a public re-bind and fix a stale test comment
* Studio: keep stdio MCP off on Colab loopback launches
* Studio: set tool policy before server startup
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: imagineer99 <samleejackson0@gmail.com>
The update banner is driven by update_available from
/api/llama/update-status. Prebuilt (marker) installs decide this via
freshness.is_behind, which is mix-aware: a release at the same upstream
base build but with a new -mix-<sha> suffix counts as behind.
Source-build installs went through _source_build_status, which compared
only the numeric base build (installed_build < latest_build). When a new
prebuilt shared the installed upstream base (our usual mix re-tag at the
same base), it returned update_available=False and the banner never
showed. This is the common macOS case, where a failed prebuilt fetch
falls back to a source build that lacks the mix patches.
Make the source-build path mix-aware to match the marker path: same base
plus a -mix-<sha> tag now offers the update (and displays the mix tag),
a bare same-base rebuild does not, and the downgrade guard and unknown
-version fallback are unchanged.
* Studio: derive training output dir from model basename for local-drive models
A LoRA/QLoRA run started from a model loaded by absolute path (common when
models live on a non-system drive, e.g. G:\modelsAI\...\gemma-4-12B-it) seeded
the default output dir with that full path. resolve_output_dir then raised
"path escapes root ... is not under the studio outputs folder", so training
could not start from a model stored off the system drive.
Add default_run_dir_name(): Hugging Face repo ids keep their namespace
(org/model becomes org_model), while local paths collapse to their final
component so an absolute source path can no longer leak into the output dir.
Use it at the three worker derivation sites and add a regression test.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: cap run dir name length and drop redundant resolve
Apply PR review feedback: length-cap the auto-generated output dir component so an unusually long model name stays under the filesystem name limit, and drop the redundant double resolve_output_dir at the embedding site so all three derivation sites match.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: keep llama-server discovery from crashing on an access-denied candidate
_find_llama_server_binary probed candidates with Path.is_file(), which raises
PermissionError (WinError 5) when a path exists but is momentarily inaccessible
(antivirus lock, an install replace in flight, an elevated-install ACL),
aborting model validation. Treat a denied-but-present path as the real binary
so discovery returns it; absent paths still skip.
* Retry a transiently locked binary instead of returning a denied path
Returning a still-denied path only moved the PermissionError to the next
is_file() (probe_server_capabilities). Retry briefly so a transient lock
clears and discovery returns an accessible path; on a persistent lock return
nothing rather than a path downstream cannot stat.
* Studio: do not fall back to another llama-server when a pinned one is locked
A denied LLAMA_SERVER_PATH made discovery skip the explicit pin and run a
lower-priority managed or PATH binary, so a load could silently use a stale or
incompatible server. Split the probe into a file/absent/denied status: when the
pinned path exists but stays access-denied, warn and stop rather than falling
back to a different executable.
* Studio: never downgrade past a denied pinned or managed llama-server
Extend the no-fallback rule beyond LLAMA_SERVER_PATH: a present-but-denied
UNSLOTH_LLAMA_CPP_PATH or managed ($STUDIO_HOME/llama.cpp, ~/.unsloth/llama.cpp)
binary now reports temporarily-unavailable instead of silently launching a
lower-priority legacy or PATH server. Shared _scan_pinned/_unavailable helpers;
legacy in-tree and PATH stay genuine fallbacks (a denied candidate there just
continues).
* Studio: let diffusion asset lookup use a locked llama-server path for its dir
DiffusionGemma does not run llama-server; _find_diffusion_assets only needs the
install dir to find the adjacent llama-diffusion-gemma-visual-server. The
no-fallback rule returning None on a transiently locked llama-server therefore
hid an available visual-server and raised 'runner not found'. Add an
include_denied option so diffusion lookup gets the locked path (its dir is all
it needs), while inference keeps the no-denied-path, no-downgrade behavior.
* Studio: report a locked llama-server as temporarily unavailable, not missing
When the pinned/managed binary stays access-denied through the retries, discovery
returns None and load_model raised 'binary not found', a terminal error that
points users at reinstalling rather than retrying a transient AV/install lock.
Reuse include_denied to detect the locked path and raise a distinct
temporarily-unavailable, retry message instead.
* Studio: GGUF preflight treats a locked llama-server as present
The pre-download preflight (and so /api/inference/validate) used the default
discovery, which returns None for a transiently access-denied binary, so it
raised 'binary not found' for a binary that merely needs the lock to clear. Use
include_denied so the existence check counts a locked binary as present; the
load itself still reports a still-locked binary as temporarily unavailable.
list_gguf_variants() keys files by _extract_quant_label(), which only captured the base quant token. Repos that ship the same base quant at multiple bits-per-weight (e.g. byteshape/Qwen3.6-35B-A3B-MTP-GGUF with three IQ4_XS files at 3.53/3.97/4.19 bpw) collapsed into a single row and Studio summed their sizes (~48 GB).
Extend the regex to capture an optional trailing -<N>(.<N>)?bpw modifier so each flavor produces a unique label. Round-trips through _find_local_gguf_by_variant and _download_gguf since both sides use the same extractor.
Fixes#5728.
Co-authored-by: Etherll <61019402+Etherll@users.noreply.github.com>
* fix(studio): keep local GGUF vision on llama-server
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio): lower local GGUF vision log level
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio): find GGUF companions from variant dirs
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: imagineer99 <samleejackson0@gmail.com>
* Studio: fix Downloaded model list disappearing and order it by last download
The chat model picker scan for cached GGUF and safetensors models aborted
whenever an auxiliary Hugging Face cache dir (such as ~/.cache/huggingface/hub)
was unreadable, returning an empty list. That hid the Downloaded section and
let already downloaded models appear under Recommended. Isolate each cache
probe so an inaccessible directory is skipped instead of failing the scan.
Also order Downloaded newest-first using cached blob mtimes (multi-quant repos
group by their most recent quant), keep the section visible while searching,
and make the per-quant downloaded check per-snapshot and mmproj aware so a
Recommended quant is never falsely marked downloaded.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: harden gguf-variants scan and dedupe by newest timestamp
Guard f.stat() per file so a broken symlink or unreadable file in a
snapshot no longer aborts the downloaded check early, and match quant
labels case-insensitively. When the same repo is present in multiple
caches with equal size, keep the newest last_modified so Downloaded
ordering reflects the most recent copy.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: apply cache-scan guards to sibling endpoints found in review
Extend the inaccessible-cache guard and mmproj/stat hardening to the
parallel HF cache code paths flagged in review:
- list_local_models and the Hub inventory scan now skip an unreadable
auxiliary cache instead of returning 500.
- The GGUF download-progress endpoint excludes mmproj adapters and
guards f.stat() so one bad file does not zero a repo's progress.
- The offline snapshot scanner guards its is_dir() probes.
- The chat-only picker no longer renders a blank list when a search
matches only cached non-GGUF models.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix: allow absolute save_directory in export paths to prevent cross-drive copy failures
The GGUF export pipeline (and all other export flows) forced every
save_directory through resolve_export_dir(), which always resolved
the path under exports_root() — typically ~/.unsloth/studio/exports/
on the system drive (C: on Windows).
When a user selected an output directory on a different drive (E:):
1. The absolute path was rejected at the Pydantic validator level.
2. Even if it got through, resolve_export_dir would re-resolve it
under C:\Users\.unsloth\studio\exports\.
3. After GGUF conversion completed on E:, the relocation step would
try to move/copy the finished files to C:, causing:
- WinError 17 (cross-drive move failure when shutil.move falls
through to a cross-filesystem copy)
- WinError 112 (disk full on C:)
Fix both layers:
- _validate_save_directory: accept absolute paths (they represent an
explicit user choice of output location).
- resolve_export_dir, resolve_output_dir, resolve_tensorboard_dir:
return absolute paths as-is instead of forcing them under the
default root. Keep the existing safety checks (null bytes, '..'
segments) and fall through to resolve_under_root for relative paths.
Fixes: https://github.com/unslothai/unsloth/issues/6082
* refactor: centralize user path validation into _resolve_user_path helper
Addresses code review feedback: the null-byte, '..', and absolute-path
checks were duplicated across resolve_output_dir, resolve_export_dir,
and resolve_tensorboard_dir. Extract a single _resolve_user_path helper
that all three delegate to.
No behavioral change — pure consolidation.
* fix: address code review — contain destructive cleanup and scope absolute paths
Address all review feedback from gemini-code-assist:
1. P1: destructive subdirectory cleanup (export_gguf)
The flattening loop in export_gguf previously rmtree'd every
subdirectory under abs_save_dir. When targeting an existing user
directory on a different drive (#6082), this could nuke unrelated
subdirectories. Now snapshot existing subdirectories before the
export and only clean up dirs created during this run.
2. P2: keep scan/read endpoints contained
Only resolve_export_dir accepts absolute paths (export is a write
path where user picks location). Reverted resolve_output_dir and
resolve_tensorboard_dir to use resolve_under_root directly — these
are used by scan/read/training endpoints that must stay contained
under their respective roots.
3. Centralization feedback
Removed the _resolve_user_path helper since it's no longer needed
with the narrowed scope. resolve_export_dir has the absolute path
logic inline with a clear docstring.
* fix: skip pre-existing subdirs in GGUF flatten loop and clean stale export intermediates
Two issues caught in code review (chatgpt-codex-connector):
1. The flattening loop moved ALL .gguf files from ALL subdirectories
into abs_save_dir, including pre-existing unrelated user subdirs.
Now skip pre-existing subdirs entirely unless they are known
export-owned intermediates (model/, model_gguf/).
2. After a failed export, known export-owned subdirectories (model/,
model_gguf/) were snapshotted as pre-existing on retry and never
cleaned up. These are now always cleaned up regardless, since they
are known intermediates created by the export pipeline.
* fix: separate write vs read export paths, guard same-dir rmtree
Three issues caught in code review (chatgpt-codex-connector):
1. P1: scan endpoint containment
resolve_export_dir was changed to accept absolute paths, but it's
also used by scan/read endpoints (routes/models.py) that must stay
contained under exports_root(). Split into:
- resolve_export_dir: contained, used by scans
- resolve_export_write_dir: accepts absolute paths, used by export
backend only
2. P1: same-directory rmtree
When a non-PEFT checkpoint's gguf_dir resolves to the same path as
abs_save_dir (user selected the checkpoint's gguf output as their
export directory), shutil.rmtree(gguf_dir) would delete the user's
chosen output directory. Now skip relocation when both paths resolve
to the same location.
3. P1: pre-existing subdir flatten loop
Reverted _EXPORT_OWNED_SUBDIRS logic — 'model/' and 'model_gguf/'
are common directory names in shared model folders and don't prove
export ownership. Now only clean up subdirs that didn't exist before
the export started.
* fix: remove dead _EXPORT_OWNED_SUBDIRS and fix _export_details for absolute paths
Two fixes from review comments:
1. Remove unused _EXPORT_OWNED_SUBDIRS declaration (leftover from
previous iteration that was intentionally removed).
2. _export_details now returns the full absolute path when the export
target is outside exports_root(), instead of truncating to basename.
Users who export to E:\ can now see the full destination path in
the success dialog.
* fix: use unique tmp dir for GGUF intermediates to avoid overwriting user dirs
When exporting to an absolute destination that already contains a
model/ subdirectory (e.g. a shared models folder), the hard-coded
model_save_path would overwrite files in that unrelated directory.
Use _tmp_model_<uuid> as the intermediate path instead, so user
directories are never touched. The tmp dir is created as a new subdir
of abs_save_dir and cleaned up by the flatten loop after GGUF files
are relocated.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix GGUF local export paths for PR #6088
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address GGUF export follow-ups for PR #6088
* Clean GGUF temp dirs on export failure for PR #6088
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix/adjust export path tests for PR #6088
* Fix/adjust export path review findings for PR #6088
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix/adjust home export path handling for PR #6088
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
The post-install path cleared only the in-memory freshness caches and then
re-primed the 24h disk cache with a forced GitHub refresh. When that refresh
cannot reach GitHub, latest_published_release falls back to the last-good disk
value, so a still-fresh same-base mix tag cached before the swap (b9596-mix-aaa
vs the just-installed b9596-mix-bbb) is replayed and the prebuilt reads as
behind, surfacing a false update banner that points back at the build that was
just replaced.
Give reset_caches a drop_disk option and use it on the update path: with the
disk cache gone, an offline post-install refresh leaves latest as None and the
banner fails open (off) instead of lingering on the stale same-base value. The
no-arg form stays in-memory only. Adds regression coverage for the drop, the
default no-op, and the fail-open vs stale-replay contrast.
IOReport energy counters can reset (sleep/wake, power gating), making a poll
delta negative. Return None for a negative total so the monitor shows -- for
that poll instead of a bogus negative wattage; it self-corrects next poll.
* Studio: show Apple GPU temperature and power in the GPU monitor (macOS)
The GPU monitor on Apple Silicon always showed -- for Temperature and
Power: the MLX branch of get_gpu_utilization() hardcoded None because
ioreg's AGXAccelerator PerformanceStatistics carries neither metric.
Add utils/hardware/apple.py, mirroring macmon's no-sudo approach:
- Temperature: average of the AppleSMC "Tg*" float keys via the
AppleSMCKeysEndpoint user client (ctypes/IOKit, macOS 14+).
- Power: IOReport "Energy Model" group, "GPU Energy" channels; each
poll diffs the energy counter against the previous poll's sample, so
the value is the average wattage over the polling window. The first
poll only sets the baseline and returns None.
Both readers latch to None on first failure and never raise, so
non-Mac platforms and locked-down hosts keep the previous behavior.
* Sample IOReport with the subscribed channels descriptor for PR #6187
IOReportCreateSubscription writes the channel descriptor that later samples
must use; sampling with the original requested group can return no Energy
Model entries on hosts that normalize the channel set, leaving power_draw_w
null after the baseline. Use the subscribed descriptor (matching macmon) and
fall back to the requested channels if the OS leaves it unset.
---------
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: danielhanchen <23090290+danielhanchen@users.noreply.github.com>
* Studio: llama.cpp update banner redesign, About tab license info, inline system prompt editing, naming cleanup
- Redesign the llama.cpp update banner to match the chat composer surface
(borderless rounded card, composer shadow, Hellix Medium title), rename
actions to Update and add a 15 minute Remind me later snooze
- Keep the banner up until the user explicitly acts on it; drop the
outside click dismissal
- Add a Settings > General > Notifications toggle to disable the banner
for training-only setups (on by default)
- Rename the Help settings tab to About and add a License section
(Unsloth Studio AGPL-3.0, Unsloth Core Apache-2.0) linking to the
license files in this repo
- Make the run settings system prompt box an inline editable textarea;
the popup editor opens when the prompt overflows the box
- Pointer cursor on the preset dropdown chevron
- Dark mode toasts use the chat composer surface color
- Replace standalone Studio with Unsloth in user facing strings; keep
Unsloth Studio, LM Studio, Fine-tuning Studio, Recipe Studio and CLI
commands unchanged
* Studio: open the system prompt popup on box click, balance banner padding
- The system prompt box opens the Edit System Prompt dialog on click,
matching the pencil action
- Slightly more bottom padding on the llama.cpp update banner so the
spacing reads even next to the action pills
* Studio: replace unsloth studio update with the installer commands in update guidance
- The unsloth studio update command no longer works, so the About tab
update section now shows the one-line installer (curl or irm) for
PyPI and unknown installs, and git pull plus the local installer for
checkouts
- Add a short note that unsloth studio update is no longer supported
- Link the Installation, Updating and Windows install docs pages
- The package update banner now copies the platform installer command
instead of unsloth studio update
* Studio: rounder account menu, inline system prompt box with popup from the label
- Account menu corners go from 14px to 18px via a specific override,
since list menus pin border-radius globally
- llama.cpp banner bottom padding 22px
- System prompt is an inline editable textarea again; clicking the
System Prompt label opens the popup editor, and an overflowing
prompt opens it on box click
* Studio: show the standard install commands in the About update section
- Both one-line install commands (MacOS/Linux/WSL and Windows
PowerShell) are always shown, labeled like the docs, since running
them again updates an existing install
- Drop the unsloth studio update deprecation note
- Add the Mac install guide to the docs links
* Studio: clearer platform toggle and layout in the About update section
- Section heading is Update
- Platform picker is a pair of pill buttons, MacOS / Linux and Windows,
and only the selected platform's install command is shown
- Intro reads: To install or update Unsloth
- Local update heading separates checkout guidance from the standard
install command
* Studio: report GitHub branch instead of dev for source checkouts
A source checkout not on an exact release tag now shows
GitHub <branch> (e.g. GitHub main) as the Studio version in About.
Detached or unusual HEADs still fall back to dev.
* Studio: tighten the About update section copy and toggle styling
- Platform toggle buttons are borderless pills
- Shorter local update wording and restart note
- Docs links read Mac and Windows
* Studio: tighten line spacing in the sidebar account button
* Studio: fix vanishing compact MCP icon on hover, single line pill tooltips
- Compact caret pills (MCP, RAG) keep their icon on hover for inactive
pills too; the off switch hover rules hid the icon while compact mode
hid the X, leaving an empty slot
- Compact icon tooltips and single line compact tooltips render as full
pills; wrapped tooltips keep the 9px corners. TooltipContent measures
line count in a ref callback since Radix mounts portal content
without re-rendering the wrapper
- 1px gap between the name and Unsloth lines in the sidebar account
button
* Studio: Projects hover plus button, align recents with the label
- Hovering the Projects nav item reveals a plus button that opens the
New project dialog, with the same circular hover treatment as the
chat row actions
- Recent chat titles start at the same x as the Recents label
- The system prompt overflow lock only engages for a non-empty prompt
with a laid-out box, so a mis-measure cannot turn clicks into the
popup
* Clip system prompt overflow inside the rounded box
Wrap the inline system prompt textarea in a rounded overflow-hidden
surface so scrolled text and the scrollbar stay inside the box. The
focus ring moves to the wrapper via focus-within.
* Add updating progress bar to llama banner and shorten settings copy
While an update is applying, the banner action row becomes an
indeterminate progress bar that keeps animating under reduced motion,
matching the other loading indicators. Settings descriptions across
General, Profile, Appearance, Chat, Connections, API, and About are
trimmed without losing meaning.
* Address review: desktop update note, server platform detection, zh-CN keys
The About tab no longer shows terminal install commands in the desktop
app, where the bundled backend updates through the built-in updater;
it shows a short note and the docs links instead.
fetchDeviceType now sends the auth token to /api/health, which only
reports the server platform to authed callers, and caches only a
server-reported value. Copied install commands then match the host
platform rather than the browser when they differ (WSL, SSH).
zh-CN gains translations for the new notification and license keys,
the renamed About tab title, and the desktop update note.
* Real download progress for llama.cpp updates, prompt and sidebar polish
The update worker now streams the installer output and parses its
download percent lines into job progress, exposed via the update-status
API. The installer emits finer non-tty milestones when
UNSLOTH_PROGRESS_PERCENT_STEP is set; the worker requests 5 percent
steps. The banner renders a determinate bar from the reported fraction
and falls back to the sweep until the first percent arrives.
Also removes the focus ring on the inline system prompt box and
slightly shrinks the Projects hover plus icon.
Anish Umale