The pinned-index work landed for install.sh and install_python_stack.py, but the
Windows installers still picked the wheel index from GPU probing. Extend the same
UNSLOTH_TORCH_INDEX_URL / _FAMILY contract so a pinned index wins on every platform:
- install.ps1: Get-TorchIndexUrl returns the pinned URL/family before nvidia-smi
probing; the AMD ROCm reroute is skipped when the index is pinned, so an explicit
cpu/cu* pin on an AMD host is not overwritten.
- studio/setup.ps1: add shared Get-PinnedTorchIndexUrl / Get-TorchIndexLeaf helpers;
the stale-venv check, the install selection and the AMD reroute all honor the pin,
and the CPU/CUDA install pulls from the resolved index URL.
- tests: parity test that all four installers read both override vars and the two
Windows installers gate the AMD reroute on the pinned flag.
Follow-up to the override work in this PR: the get_torch_index_url / install.sh
reroute already respect a pinned UNSLOTH_TORCH_INDEX_URL / _FAMILY, but the
Python repair helpers in install_python_stack.py still re-probed the GPU and
could overwrite the pinned family. Make the pin authoritative there too:
- _ensure_cuda_torch: an explicit cu* pin commits to CUDA wheels, so repair a
ROCm-poisoned venv even when no NVIDIA GPU is visible here (headless /
container / CI cross-install), instead of bailing on the GPU-presence gate.
- _ensure_rocm_torch: skip the AMD per-gfx (Strix) reroute when a ROCm index is
pinned, and in the generic reinstall path install from the pinned URL verbatim
rather than re-detecting the host ROCm version. gfx*/rocm7.2 indexes serve
torch 2.11+, so select the 2.11 package specs for a gfx leaf.
- install.sh: raise the torch constraint to 2.11 for */gfx* indexes too, matching
rocm7.2, so a pinned full-URL/family override that returns early keeps a valid
constraint.
Add _explicit_torch_index_url / _explicit_rocm_torch_index_url helpers and tests
covering the no-GPU CUDA pin repair and the explicit gfx index honored verbatim.
* Studio Colab: add opt-in shareable Cloudflare tunnel link
colab.start(cloudflare=True) opts in to a free Cloudflare quick tunnel and
shows a trycloudflare.com link above the proxy iframe, reachable from any
device. Default OFF: bare start() keeps the in-tab Colab-proxy behavior.
run_server suppresses the tunnel on Colab by design, so colab.py starts it
directly via cloudflare_tunnel.start_studio_tunnel(); failures degrade to
the Colab proxy only.
* Studio Colab notebook: surface opt-in cloudflare=True in start cell
* Studio Colab: reskin shareable Cloudflare link to match the proxy banner
Retrofit _shareable_link_html to reuse the original Colab proxy banner skin
from show_link (white card, black border, Unsloth gem, black Open button)
instead of the plain dark box, so the shareable Cloudflare link gets the same
prominent 'Ready!' treatment.
* Studio Colab: address review feedback on Cloudflare tunnel
- try/finally around tunnel start + embed + keepalive so a KeyboardInterrupt
while the tunnel is starting or the iframe is rendering tears it down instead
of orphaning the cloudflared process (Gemini review).
- Publish the directly-started tunnel URL onto app.state.cloudflare_url via a new
_publish_cloudflare_url helper so /api/health advertises it; otherwise the
frontend's API examples fall back to the unreachable raw server_url (Codex P2).
_stop_cloudflare_tunnel now also clears it so health stops showing a dead tunnel.
- Notebook: make cloudflare=True a replacement for start(), not an addition, since
start() blocks and the second call would never run if both are left in (Codex P2).
* Studio Colab: gate Cloudflare tunnel on auth + honor opt-out in run_server
- Refuse to open the Cloudflare tunnel while the admin still holds its seeded
bootstrap password. While requires_password_change is true the server injects
that password into same-origin index GETs, and a public tunnel request counts
as same-origin, so sharing the link would leak admin access. New
_bootstrap_password_pending() gate (fails safe) blocks the tunnel and tells the
user to change the password first, then re-run start(cloudflare=True) (P1).
- Pass cloudflare=False into run_server so the opt-out holds even when Colab
detection fails; this helper is now the sole owner of the tunnel decision,
preventing run_server from opening a tunnel on the 0.0.0.0 bind by default (P2).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio Colab: drop duplicate tunnel link log and simplify start cell guidance
* Studio Colab: validate /api/health identity before reusing or tunneling a port
* Studio Colab: condense verbose docstrings and comments
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Fix offline checkpoint load/export failing with "tokenizer is weirdly not loaded"
Loading a fine-tuned checkpoint with no internet (e.g. a Studio export) crashed
with "Unsloth: The tokenizer is weirdly not loaded? Please check if there is one."
For a LoRA adapter the loader reassigns model_name to the base model repo id and
only keeps the local checkpoint dir as tokenizer_name when it contains
tokenizer_config.json, tokenizer.json AND special_tokens_map.json. Modern
tokenizers (e.g. Gemma) store special tokens inside tokenizer_config.json and
omit special_tokens_map.json, so tokenizer_name fell back to the base repo id.
The tokenizer/processor loads in vision.py then hit the Hub with no
local_files_only, so with no network they failed (AutoProcessor) or hung for
minutes (AutoTokenizer) even though every file was already cached.
loader.py: keep the local checkpoint dir as tokenizer_name when it has a
tokenizer config plus the actual tokenizer files (tokenizer.json / tokenizer.model
/ vocab files); special_tokens_map.json is no longer required.
vision.py: compute an effective local_files_only (explicit kwarg plus the
HF_HUB_OFFLINE / TRANSFORMERS_OFFLINE env vars, mirroring loader.py and
diffusion.py) and thread it through every AutoConfig, AutoProcessor,
AutoTokenizer and the manual VLM processor fallback, including the
hf_hub_download in that fallback (which now prefers a local file). When a load
fails and no offline env var is set, retry against the local cache. The retry
forces HF offline mode because local_files_only alone does not stop
AutoProcessor / AutoTokenizer from issuing a /api/models request during class
resolution. The final error now explains the offline/cache cause instead of the
misleading "weirdly not loaded" message.
studio export: probe Hub reachability once per checkpoint load and pass
local_files_only when offline so exports use the local checkpoint dir / cache
instead of hanging or crashing with no internet.
Online behavior is unchanged: the new flags default to off and the retry only
runs after a network related failure.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: safer offline forcing, cached fallback config, proxy-aware probe
Follow-up to the offline checkpoint load fix, addressing review feedback:
- vision.py: only flip the process-wide HF offline flag when offline is actually
requested (local_files_only / env) or after a real network failure, never
pre-emptively while we might be online. The flip is now guarded by a lock +
depth counter so nested or concurrent windows restore the flag correctly
(no stale value).
- vision.py: guard the get_auto_processor fallback so a network error there
returns None and the local-cache retry still runs instead of escaping.
- vision.py: in the manual VLM processor fallback, read tokenizer_config.json
via hf_hub_download(..., local_files_only=...) so a cached repo-id config is
still resolved offline and the model-specific image/video tokens are restored.
- studio export: make the reachability probe proxy aware (probe the configured
HTTP(S) proxy egress, honour NO_PROXY, use the endpoint port) so a proxy-only
setup is not wrongly marked offline; allow UNSLOTH_OFFLINE_PROBE=0 to disable.
- studio export: run the audio/vision type-detection probes inside the
forced-offline window when offline, so their config/tokenizer reads hit the
local cache instead of waiting out connection timeouts.
Online behavior remains unchanged.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: gate offline retry, safer tokenizer_name pop, skip audio net probe offline
- vision.py: only force the process-wide HF offline flag on the tokenizer
retry when offline was requested or the captured primary error is actually
network related, so a permanent tokenizer error no longer toggles global
offline mode for other concurrent loads.
- loader.py: always pop tokenizer_name out of kwargs and let a caller-supplied
value win, avoiding a "multiple values for keyword argument 'tokenizer_name'"
TypeError when it is also passed explicitly downstream.
- model_config.py / export.py: add local_files_only to detect_audio_type so the
raw requests.get tokenizer_config fetch is skipped offline (it ignores the HF
offline flag), and pass it from the export probe.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: classify LocalEntryNotFoundError as offline-related
huggingface_hub's LocalEntryNotFoundError subclasses FileNotFoundError, so the
"not isinstance(cur, FileNotFoundError)" guard in _is_offline_related_error was
swallowing it and it could never be recognised as offline, despite being listed
in the network error types. It means "not in cache and the Hub is unreachable",
which is genuinely offline. Capture the class into an isinstance-checkable tuple
(empty, hence a no-op, if the import is unavailable) and exclude it from the
FileNotFoundError guard, so a real offline failure now triggers the local-cache
retry while a plain missing-file error still propagates.
* Address review: require merges.txt for BPE, status-gate HTTP errors, isolate local-only audio cache
- loader.py: a local dir with vocab.json but no merges.txt (and no tokenizer.json)
is not a loadable BPE tokenizer, so do not treat it as self-sufficient; require
merges.txt alongside vocab.json in both gate blocks, otherwise fall back to the
base model tokenizer as before.
- vision.py: _is_offline_related_error no longer buckets every HfHubHTTPError /
requests HTTPError as offline. HTTP errors are judged by status code: only a
transient 5xx triggers the forced local-cache retry, while 401/403 (auth/gated)
and 404 (missing) propagate as the real error instead of being masked. Hard
signals (connection/timeout/OfflineModeIsEnabled/LocalEntryNotFoundError) still
classify as offline.
- model_config.py: include local_files_only in the audio-detection cache key so a
local-only (offline) negative result cannot be reused by a later online probe,
which would otherwise route an audio model through the text loader until restart.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address re-review: fix studio test stubs, force offline env in probe window, drop redundant retry
- studio/backend/tests/test_vision_cache.py: the three _detect_audio_from_tokenizer
stubs were called with the new local_files_only kwarg and raised TypeError, failing
Backend CI. Add local_files_only to the stub signatures and add a test that a
local-only negative does not poison a later online audio probe.
- export.py: the type-detection probe window now also sets HF_HUB_OFFLINE /
TRANSFORMERS_OFFLINE env vars (saved/restored), not just the in-process flag.
transformers_version._load_config_json / _check_tokenizer_config_needs_v5 gate
their urllib fetches on the env vars, and is_vision_model may spawn a subprocess
that inherits os.environ but not the in-process flag; without the env vars a
probe-detected offline export could still block on a network timeout.
- vision.py: only retry the processor load when the first attempt was online and
failed with a network error. When local_files_only was already requested the first
attempt was forced offline, so the previous retry just repeated identical failing
work before the last-resort path.
- model_config.py: correct the _audio_detection_cache type annotation to the 3-tuple
key (name, token_fingerprint, local_files_only).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: thread-safe probe-offline env window, clear error for local dir without config
- export.py: guard the HF_HUB_OFFLINE / TRANSFORMERS_OFFLINE mutation in
_force_offline_probe_window with a lock + depth counter (mirrors _force_hf_offline),
so concurrent / nested export probes only flip on first entry and restore on last
exit. This prevents overlapping export requests from permanently poisoning those
env vars or restoring a stale value.
- vision.py: in the VLM processor fallback, when tokenizer_name is a local directory,
read its tokenizer_config.json directly and raise a clear FileNotFoundError if it is
absent, instead of handing the local path to hf_hub_download (which would treat it as
a repo id and raise a confusing HFValidationError / RepositoryNotFoundError).
hf_hub_download is now only used for actual repo ids.
* Address review: classify raw socket.gaierror DNS failures as offline
Add the platform-specific getaddrinfo / DNS-resolution wording to the offline
detection list in _is_offline_related_error so a bare socket.gaierror (an OSError
subclass) is recovered from the local cache: "Name or service not known" and
"Temporary failure in name resolution" (Linux) and "nodename nor servname
provided" (macOS). Genuine non-network OSErrors (disk full, permission denied)
and plain FileNotFoundError still propagate.
* Address review: retry degraded VLM offline, force offline for text export + patch-tokenizer fallback
- vision.py: a degraded VLM processor (text-only, no image_processor) whose manual
fallback fails offline used to be kept, so image inputs broke even with cached
files. _construct_vlm_processor_fallback now returns its failure error;
_acquire_processor surfaces it, and the caller retries forced-offline when the
result is None OR a degraded VLM and the failure was network related, keeping the
original result if the retry is not strictly better (never regress). The retry is
still gated on an online first attempt + offline-related error so a permanent
error never flips the global offline flag.
- vision.py: wrap the patch_tokenizer except-branch AutoTokenizer.from_pretrained in
the same forced-offline-on-network-error pattern as the primary / last-resort
loads, so an offline export where patch_tokenizer raises does not hang or fail.
- export.py: force HF offline around the two FastLanguageModel loads (text and SNAC)
when the probe detected offline. Their text tokenizer path (load_correct_tokenizer
-> AutoTokenizer) does not forward local_files_only, so without this a text export
could still contact the Hub. Added a small _offline_window_if helper reused by the
probe and load windows.
* Consolidate offline loading into one entry-point decision
Decide offline once per entry point instead of at every HF call site. The
prior approach threaded local_files_only into ~15 scattered config / tokenizer
/ processor / weight loads, each wrapped in its own try-online, classify-error,
retry-forced-offline dance, which is what kept surfacing "another call site you
missed", "another error shape misclassified", and global-flag thread-safety in
review.
FastLanguageModel / FastModel / FastBaseModel.from_pretrained now share an
@_offline_aware_load decorator: when offline (explicit local_files_only kwarg or
HF_HUB_OFFLINE / TRANSFORMERS_OFFLINE env) it sets local_files_only and runs the
whole load inside one _force_hf_offline() window so every nested HF call inherits
it; when online it runs normally and, only if the load fails with a genuinely
network-related error, retries once forced-offline. The online path is unchanged
(no probe added) and 401 / 403 / 404 / permanent errors still propagate.
Centralise the offline helpers in loader_utils.py as the single source of truth
(shared by loader.py, re-exported from vision.py, and reused by the Studio
exporter):
- _force_hf_offline now sets the HF_HUB_OFFLINE / TRANSFORMERS_OFFLINE env vars
AND the in-process huggingface_hub / transformers flags, refcounted under one
lock so nested / concurrent windows restore correctly. Setting the env vars
covers env-gated urllib probes and spawned subprocesses too.
- _get_effective_local_files_only, _is_offline_related_error (unchanged
classifier, retains the 5xx-vs-4xx, LocalEntryNotFound and gaierror handling),
_offline_aware_load, and _resolve_checkpoint_tokenizer_name.
loader.py: wrap both entry points; drop the two duplicated env-var fallback
blocks and the two byte-identical local-tokenizer-gate blocks (now
_resolve_checkpoint_tokenizer_name).
vision.py: drop the per-site force_offline params and the three retry gates
(processor, patch_tokenizer fallback, last-resort). They now just surface the
underlying error so the single entry-point safety net retries forced-offline. A
network fallback error now takes precedence over a permanent primary error so the
offline retry still fires when the manual VLM fallback needs cached repo files.
studio/backend export.py: reuse the unified core _force_hf_offline (env + flags)
and drop the duplicate probe-window primitive; the snac / text branches no longer
need their own window. model_config.py: also gate the raw requests.get audio
fallback on the HF offline env vars so it is covered even without the kwarg.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address 10-reviewer P1 findings: vision cache split, PEFT offline, retry OOM
Split the Studio vision-detection cache by local_files_only, mirroring the audio
cache fix. is_vision_model / _is_vision_model_uncached / _raw_config_has_vision_config
/ load_model_config now thread local_files_only, the cache key includes it, and the
exporter passes it. Offline detection also skips the transformers-5 network
subprocess and stays on the local cache, so an offline negative can no longer be
keyed under the online entry and poison a later online probe. Adds a regression
test mirroring the audio poison test.
Forward local_files_only to both PeftModel.from_pretrained adapter-attach sites in
loader.py so a cached remote LoRA adapter resolves from the local cache under
explicit local-only / offline loads (defence-in-depth alongside the forced-offline
window).
_offline_aware_load: run the forced-offline retry OUTSIDE the except block and
collect + empty the device cache first. An except-scoped exception keeps its
__traceback__, which pins the failed attempt's frame locals (a partially loaded
model) until the block exits; loading the model again while that copy is still
alive could OOM a large VLM. Letting the except block close drops the traceback so
the partial load is freed before the retry reallocates.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address Codex review: env-offline cache key + rebuild HF sessions in offline window
Key the Studio audio and vision detection caches on the EFFECTIVE offline state
(local_files_only OR the HF offline env vars), not just the kwarg. detect_audio_type
and is_vision_model both skip the remote fetch / network subprocess when
HF_HUB_OFFLINE / TRANSFORMERS_OFFLINE is set even with the default
local_files_only=False, so the result reflects offline; storing it under the online
(False) key let an env-offline negative poison a later online lookup once the env var
was cleared. Both now compute effective_offline once and use it for the cache key and
the downstream call. Adds a regression test for the env-offline dimension.
_force_hf_offline now rebuilds huggingface_hub's cached sessions on enter and exit
(best-effort _reset_hf_sessions). On hub 0.x the offline adapter is baked into the
per-thread requests.Session at creation, so flipping the constant alone leaves an
already-cached online session able to hit the network inside the window (and an
offline one stuck offline after restore); resetting forces the next get_session() to
match the current flag. On hub 1.x offline is checked dynamically per request, so
reset_sessions does not exist and the helper is a safe no-op.
The third review point (release the failed load before retrying) was already fixed in
af0f58a: the forced-offline retry now runs outside the except block and frees the
device cache first, so the failed attempt's traceback-pinned partial model is
released before the retry reallocates.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Align Studio _env_offline parsing with the canonical offline helper
model_config._env_offline gates the raw requests.get tokenizer-config fallback in
detect_audio_type and the audio/vision detection cache keys, but it only accepted
unstripped "1"/"true"/"yes". unsloth's offline helpers (loader_utils._env_says_offline
and the from_pretrained env fallback) accept the canonical set {1,true,yes,on} after
strip + lowercase, so HF_HUB_OFFLINE=on or HF_HUB_OFFLINE=" 1 " was treated as offline
by the loaders but online here, leaving the raw network fetch reachable while
"offline". Use the same strip + lowercase {1,true,yes,on} set. Adds parsing tests.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix lint: drop dead offline-helper re-exports from vision.py
The import-hoist verifier (scripts/verify_import_hoist.py) flagged vision.py's
re-export block as HOISTED-IMPORT-UNUSED blockers: it imported eight offline
helpers from loader_utils but only used three internally
(_get_effective_local_files_only, _is_offline_related_error, _offline_aware_load).
The other five were imported purely to preserve `from unsloth.models.vision import
X`, but nothing imports four of them from vision, and loader.py already imports
_resolve_checkpoint_tokenizer_name straight from loader_utils.
Import only the three names vision.py actually uses, and point the Studio exporter
at the canonical source (from unsloth.models.loader_utils import _force_hf_offline)
instead of re-exporting it through vision. loader_utils stays the single source of
truth; no behaviour change.
* Address Opus review: chain probe errors, unify env-offline, status-less HTTP
Chain the original AutoConfig/PeftConfig probe exception into the combined
RuntimeError in both FastLanguageModel.from_pretrained and FastModel.from_pretrained
(`raise RuntimeError(combined_error) from (autoconfig_exc or peft_exc)`). The probes
caught every Exception and stringified it, so the re-raised RuntimeError had no
__cause__/__context__ and _is_offline_related_error could not classify it -- the
network-down-but-cached auto-retry never fired for these entry points. With the
cause chained, the decorator sees a ConnectionError/LocalEntryNotFoundError/5xx and
retries forced-offline from cache; a permanent cause (404 / bad config) is still not
offline-classified and propagates without a wasted retry.
Unify the third offline-env parser: studio/backend/utils/transformers_version._env_offline
now uses the canonical {1,true,yes,on} + strip + lowercase set (matching
loader_utils._env_says_offline and model_config._env_offline), so HF_HUB_OFFLINE=on
or " 1 " no longer leaks the direct urllib metadata fetches to the network.
_is_offline_related_error: a status-less HTTP error (no response / unparseable code)
now falls back to the network-wording check instead of being dropped, so a transient
HTTP failure with clear "couldn't connect" wording is treated as offline. HTTP errors
with a real status code still decide by code (4xx propagates, 5xx is offline).
* Condense offline-loading code comments, drop dead helper, dedupe import for PR #6554
* Add unit tests for offline-loading helpers for PR #6554
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Guard load cleanup with try/finally and add retry-contract tests for PR #6554
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add gc.collect retry-step test for PR #6554
* Tighten offline-loading comments and docstrings for PR #6554
* Raise the both-config-failed error before model-type lookup so offline retry fires for PR #6554
* Prefer offline cause for retry and bound export reachability probe for PR #6554
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Skip remote mapper while offline, harden text-load cleanup, and stop stacked offline retries for PR #6554
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Surface VLM fallback offline errors, probe offline before export version activation, and restore progress bars across retries for PR #6554
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Restore offline env after export version activation so the persistent worker re-decides per load for PR #6554
* Classify socket.gaierror and urllib URLError as offline by type for PR #6554
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Probe offline around export load preflights and never offline-retry TLS failures for PR #6554
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Force in-process offline for export preflights, verify proxy egress in probe, and skip caching offline version negatives for PR #6554
* Snapshot offline constants before forcing env and require local processor files for VLM checkpoints for PR #6554
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Address review feedback on the override added in this PR so a pinned index is
honoured everywhere, not just in get_torch_index_url:
- Skip the WSL ROCm bootstrap (root privilege + large downloads, probes
/dev/dxg) when UNSLOTH_TORCH_INDEX_URL / _FAMILY is set; it previously ran
before the override was consulted.
- Skip the Radeon/Strix rerouting (which re-probes the GPU and overwrites the
resolved URL with repo.radeon.com / repo.amd.com) when the index is pinned, so
an explicit ROCm override (e.g. UNSLOTH_TORCH_INDEX_FAMILY=rocm6.4) is kept.
- install_python_stack.py: derive _TORCH_BACKEND from the override when
UNSLOTH_TORCH_BACKEND is unset (standalone studio update), so _ensure_rocm_torch
/ _ensure_cuda_torch repair to the requested family instead of re-detecting.
- Strip ALL leading/trailing slashes in the shell override to match the Python
side (avoids 404s on strict pip proxies).
Adds test cases for double-slash and leading/trailing-slash overrides.
get_torch_index_url (and the studio-update mirror _detect_cuda_torch_index_url)
chose the torch wheel family solely by probing the host GPU, with no override.
In a headless / container / CI build the host driver is visible via the
/proc/driver/nvidia/gpus fallback but nvidia-smi cannot report a CUDA version,
so the function fell back to its cu126 default and installed the wrong wheels
(e.g. a cu128 image got cu126 torch).
Add an explicit override checked before any probing, in both the shell installer
and the Python studio-update path:
- UNSLOTH_TORCH_INDEX_URL full index URL, used verbatim (wins)
- UNSLOTH_TORCH_INDEX_FAMILY family (cpu, cu128, rocm6.4, ...) appended to the
mirror base (UNSLOTH_PYTORCH_MIRROR still honoured)
This matches how the published GPU images select CUDA -- vLLM and SGLang take the
CUDA version from an explicit build ARG rather than detecting it, and the Unsloth
Docker base image already pins the cu128 index directly. Desktop installs are
unchanged: with no override set, detection runs exactly as before.
Adds test_get_torch_index_url.sh cases for the override (family, full URL,
precedence, mirror base, trailing-slash strip, empty-ignored).
* Studio: require signed capability tokens for /p preview links
The public /p preview routes added in #6486 run model load and chat
generation as the admin user with no authentication. The only gate is the
preview ref, a deterministic outputs-root path (run or run/checkpoint) that
is guessable rather than secret. On a network-reachable Studio (--secure
tunnel or -H 0.0.0.0), an unauthenticated caller who guesses a ref can
consume GPU and probe a private fine-tuned checkpoint.
Make the share link an unguessable, revocable capability:
- Sign the canonical ref with a dedicated server-side secret (HMAC-SHA256,
stored in app_secrets, independent of the JWT/login secret).
- Require a valid token on every /p chat, models, and page request before
resolving a checkpoint or loading a model; missing or invalid tokens get a
generic 404 so the surface never confirms a ref exists.
- Accept the token via ?k= (browser link and preview page) or
Authorization: Bearer (OpenAI-compatible clients).
- Rotate the secret to revoke every outstanding link
(POST /api/settings/preview-links/rotate).
- Clamp preview generation (max_tokens/max_completion_tokens <= 1024, n = 1)
and set Referrer-Policy: no-referrer on the page so the token is not
leaked via Referer.
Training history hands the authenticated owner the signed token, and the
copy-link button builds /p/{ref}?k={sig}.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: honor a lower caller token limit in the preview clamp
Codex review: when only the legacy max_tokens was sent, the clamp left
max_completion_tokens at the 1024 default, and _effective_max_tokens prefers
max_completion_tokens, so a request like max_tokens=16 could still generate up
to 1024 tokens. Derive one effective limit (max_completion_tokens wins, else the
legacy max_tokens) and pin both fields to it so a caller's lower limit is kept.
* Studio: add preview kill switch, rate limit, and revoke-links UI
Follow-ups to the /p preview capability work:
- Public-sharing kill switch: a persisted setting (default on) gates the public
/p surface. When off, every preview request 404s even with a valid token, and
the owner UI stops offering share links. GET/PUT /api/settings/preview-sharing;
enforced in _verify_or_404.
- Per-IP rate limit on the preview chat route: a coarse in-process sliding-window
limiter (20 req/min/IP) returns 429 + Retry-After before the GPU lock is taken.
Client IP honors X-Forwarded-For only when UNSLOTH_STUDIO_TRUST_FORWARDED is
set, matching the login limiter's trust model.
- Settings UI: a "Preview sharing" section with the public-sharing toggle and a
"Revoke all preview links" button (confirm dialog) that rotates the secret.
Tests cover the kill switch (404 when off), the 429 path, the sliding window,
client-IP trust behavior, and the setting default.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: fix preview-fields sharing arg and refresh sigs after revoke
Codex review:
- P1: get_training_run_detail and update_training_run called _preview_fields
with only output_dir after it gained a required sharing_on parameter, raising
a 500 TypeError once get_run succeeded. Pass get_preview_sharing_enabled() at
both sites; add a detail-endpoint regression test.
- P2: after rotating the preview secret from settings, the history grid still
held stale preview_sig values, so a freshly copied link would 404. Emit
emitTrainingRunsChanged() after a successful revoke so the grid refetches
freshly signed refs.
* Studio: harden preview sharing controls (Codex review)
- Fail closed: a read failure on the preview-sharing kill switch now returns
False instead of defaulting to enabled, so an unavailable settings DB can't
reopen the public surface. A missing key still defaults to enabled.
- Per-IP rate limit behind the managed Cloudflare tunnel: client_ip now honors
CF-Connecting-IP when the socket peer is loopback, so tunneled visitors are
keyed by their real IP instead of collapsing onto the local cloudflared peer.
- GET /p no longer mints key/share_url when sharing is disabled; it returns
sharing_enabled=false so clients don't distribute links that 404.
- Settings UI: toggling public sharing emits the training-runs-changed event so
the history grid shows/hides Copy preview link without a manual refresh.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: harden preview rate limiter and IP keying (Opus review)
From a two-agent review of the PR:
- Rate limiter no longer evicts an active bucket when the table is full: a flood
of distinct keys could otherwise cycle out a throttled bucket and reset its
counter. Evict only aged-out buckets; if the table is full of live clients,
fail closed (deny the new key) instead.
- client_ip keys on the rightmost (proxy-appended) X-Forwarded-For hop when the
trust env is set; the leftmost is client-spoofable. Documented the
append/overwrite-proxy assumption.
- _verify_or_404 checks the capability token before the kill-switch DB read, so
unauthenticated /p spam can't be used as an unbounded settings-DB sink and the
response is identical regardless of the sharing on/off state.
Tests: nested run/checkpoint happy path + wrong-ref rejection, the eviction
fail-closed behavior, and route-level coverage for the rotate / preview-sharing
settings endpoints.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Verify linuxdeploy AppImage digest before use in desktop release
The desktop release workflow downloaded linuxdeploy-x86_64.AppImage from a
GitHub release and ran chmod +x with no integrity check. Pinning the
versioned release path is reproducibility, not integrity: a release asset
can be replaced (or its delivery path compromised) after upload. The next
step builds the AppImage with the Tauri signing private key and a
contents:write GITHUB_TOKEN in scope, so a substituted linuxdeploy that
ran during packaging could exfiltrate signing material or tamper with
published release artifacts.
Pin the immutable SHA-256 of the asset and verify it with sha256sum -c
before chmod +x, so a mismatch fails the job closed before the binary is
ever executable. Extend the existing in-workflow guard to require both the
pinned digest and the verification step, so a future edit cannot silently
drop the check.
* Scope linuxdeploy guard to real step content, not its own text
The self-check searched every workflow line, so the digest assertion was
satisfied by the guard's own expectedLinuxdeployDigest line and the
verification assertion by a comment. Deleting the LINUXDEPLOY_SHA256 env
pin or the actual sha256sum -c command would still have passed.
Match the digest against the LINUXDEPLOY_SHA256 env line specifically and
require sha256sum -c on a non-comment line, so dropping either the pin or
the verification now fails the guard.
* Scope linuxdeploy guard to the Pin step block and check ordering
The previous predicate still scanned the whole workflow, so the literal
sha256sum -c in the guard's own code satisfied the verification check; a
deleted or post-chmod verification command would still pass.
Extract the 'Pin linuxdeploy for AppImage' step block and assert within it:
the LINUXDEPLOY_SHA256 env pins the expected digest, a non-comment line
runs sha256sum -c, and that verification precedes chmod +x.
* feat: add GPU-aware model filtering and For You section- Add fit filter toggle (All / Fits GPU / Comfortable) to Hub discover tab- Add For You section showing only hardware-compatible models- Fix MoE active parameter extraction (Qwen3.5-35B-A3B now correctly reads as 3B active, not 35B)- Add gpu-fit-filter.ts with instant VRAM estimation from HF metadata without fetching model configs- Add fit badges to model cards and table rows- No backend changes- Closes#6556
* fix: handle unified memory systems in GPU fit classification
* fix: tighten GPU model fit filtering
---------
Co-authored-by: imagineer99 <samleejackson0@gmail.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
* Studio: stop leaking the auth token through HTML canvas preview frames
The artifact preview frame placed the Studio bearer token in the iframe URL
(?token=) whenever canvas network access was enabled. Untrusted canvas HTML
runs in that frame and can read its own window.location.href, and the
network-mode CSP allows outbound http/https, so the token could be
exfiltrated and replayed against authenticated Studio APIs. The auto-render
HTML cards widened the reach: ordinary or prompt-injected assistant html
fences become a Preview card that opens this same frame, and the render_html
tool path auto-opens it without a click.
Root cause: never put the token in the frame URL. The preview shell is a
static document that only renders HTML posted to it by its embedder, and
frame-ancestors plus the no-same-origin sandbox already constrain it, so the
endpoint no longer accepts or validates the token and selects the network
CSP from allow_network alone. No credential ever reaches the frame.
Defense in depth: only tool-rendered canvases may opt into network mode;
fences auto-extracted from assistant text never do.
* Studio: stop strict canvas frames from self-upgrading to network mode
Network mode is selected from the allow_network query param alone, so untrusted
canvas code in a strict frame could navigate its own iframe to
?allow_network=1; the frame's onLoad handler then reposted the same untrusted
HTML into the now network-enabled frame, giving a no-network or fenced canvas
unauthorized network egress.
Only inject the artifact for loads we initiated (mount or a src change), tracked
by a pending flag set when src changes. A self-navigation also fires onLoad but
is no longer fed, so the upgraded frame stays the inert shell. The strict CSP
default-src 'none' already blocks the child-iframe variant.
* Studio: trim comments in the canvas artifact security fix
Condense the added explanatory comments and the artifact-preview-frame docstring
to one line each while keeping the security rationale. No code change (verified
comment-only).
* Studio: keep the training event pump alive so progress can't silently freeze
The parent-side event pump is the only writer of the in-memory progress state
that SSE /progress, /status, /metrics and the DB history all read. It ran in a
single unsupervised daemon thread with no guard around event handling, so one
malformed event or a transient queue/DB error would terminate it permanently.
The worker subprocess keeps training regardless (mp.Queue puts never block on an
unbounded queue), so a run kept burning GPU for hours while every progress
surface froze on the last step the pump saw.
- Guard each pump iteration: a bad event or queue-read error is logged and
skipped instead of ending the loop. _read_queue now reads any error as
"no event", not just Empty/EOFError/OSError/ValueError.
- Add a _pump_running flag and an _ensure_pump_alive watchdog wired into
is_training_active, so a pump that dies while the worker is alive is restarted
on the next status poll and the UI catches up from the still-open queue.
- Start respawned and restarted pumps under the lock so the watchdog can never
spawn a duplicate during the brief start window.
Adds tests/test_training_pump_resilience.py covering both guarantees.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio training pump: address review (drain guard, start race, read backoff, respawn flag)
Follow-up to the event-pump resilience change, closing four edge cases a
review surfaced in the same pump/queue surface:
- _drain_queue now tolerates any error during the worker-exit drain and
finalizes with whatever it drained, instead of skipping finalization and
leaving the run wedged "active" with a dead worker.
- start_training clears a stale _pump_running flag during reset and assigns
the subprocess handles plus starts the pump under the lock, so a concurrent
status/SSE poll can't spawn a duplicate pump during setup.
- _read_queue goes back to the narrow EOFError/OSError/ValueError catch;
truly unexpected errors are left to _pump_loop's guarded read, which logs
and backs off so a persistently raising queue can't spin a hot loop.
- The xet respawn-failure path clears _pump_running so a later run can't
inherit a stale flag.
Adds regression tests for all four.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: revive a crashed pump after worker exit + stop test module pollution
Two review follow-ups on the training event pump:
- _ensure_pump_alive refused to restart once the worker had exited
(not self._proc.is_alive()), so a pump that crashed just before the worker
finished never drained the terminal complete/error events still sitting in
the queue. progress.is_training stayed True and is_training_active() returned
True forever, leaving the run stuck "running" behind a dead pump. A True
_pump_running flag with a dead thread is an unambiguous crash regardless of
worker state, so restart there too: the fresh pump drains the backlog and
finalizes. Updated the watchdog test to assert the revive-and-finalize.
- The resilience test imports core.training.training while heavy module-level
deps are stubbed, then restores the stubs -- but the cached training module
kept the stubs bound in its globals, so a later test in the same session
could exercise the fakes (e.g. prepare_gpu_selection) instead of the real
code. Evict the training module (and its package) after import when this file
created it, so subsequent tests re-import it cleanly.
* Studio: finalize training run when queue reads keep failing on a dead worker
reviewer.py follow-up. _read_queue only swallows EOFError/OSError/ValueError;
an unexpected error escapes to the pump's outer guard, which logged, slept and
`continue`d. If those reads keep raising after the worker has already exited
(e.g. a broken queue pipe), the loop never reaches the dead-worker finalize
block, so the pump spins on with _pump_running True and progress.is_training
stuck True -- the run looks like it is still training forever. On a read failure
now fall through to finalize when the worker is gone, only backing off and
retrying while it is still alive. Mirrors the data-recipe pump fix; added a
regression test.
* Tighten training pump resilience comments and docstrings
Condense the verbose explanatory comments and docstrings on the training event
pump and its tests to shorter, clearer forms. Comment/whitespace only; verified
no code changed via AST diff. No behaviour change.
* Studio: create the training DB run before starting the event pump
start_training started the event pump before the eager _ensure_db_run_created()
call, so for a worker that completes or fails immediately the pump could race the
main thread into creating and finalizing the same run row (duplicate INSERT, or a
finalize skipped while _db_run_created was still false). Create the run first; the
pump then only ever finalizes. Adds a regression test asserting the pump observes
an already-created run.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Keep pad-named pad_tokens; defer pad repair to shared unsloth_zoo.pad_token
A pad-named token (e.g. <|vision_pad|>) is a valid pad. The narrow fallback that
stripped vision pad tokens on text-only models is now a no-op; the active path
delegates to the shared fix_pad_token in unsloth_zoo, which keeps pad-named tokens
and only heals missing / eos-collision / out-of-range pads.
This fixes the Qwen3-4B-Base load crash (its config ships pad_token=<|vision_pad|>):
the old swap could not find a safe text pad (eos is <|endoftext|>, no unk_token) and
left the tokenizer broken. Removes the unused _VISION_PAD_TOKENS / _SAFE_TEXT_PAD_TOKENS
sets. Tests updated.
Pairs with unslothai/unsloth-zoo#831.
* Remove _fix_vision_pad_token; inline the no-op fallback
A pad-named token (e.g. <|vision_pad|>) is a valid pad, so the old vision-pad swap
helper has no purpose. _fix_pad_token now returns the tokenizer unchanged when the
shared unsloth_zoo.pad_token module is unavailable, instead of routing through a
no-op helper. Test WANTED set updated.
* Studio: don't time out the live progress stream during pre-first-step prep
The live progress SSE counts every 1s poll without a step update toward a
30-minute stall timeout, after which it emits an error event and ends the
stream. But that counter also runs during the pre-first-step phase (model
load + tokenizing the dataset), which is never reset because no step has
happened yet. On a large dataset that prep can take well over 30 minutes, so
the live view is torn down with an error while the run is perfectly healthy
and still preparing -- the run then trains on in the background with the UI
showing nothing, exactly the "no progress for hours" decoupling.
Apply the stall timeout only once the stream has actually seen a live step.
Before the first step the run is preparing and may legitimately emit no step
for a long time; heartbeats still flow so the client stays connected and the
worker's liveness still ends the loop when training finishes. A genuine
post-step stall still times out. Extracted the threshold to a module constant
so it can be tuned/tested.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: seed seen_live_step from the resume point on reconnect
Review follow-up: seen_live_step reset to False on every SSE request, so a
client reconnecting past the first step (Last-Event-ID set, or the run already
has step history) only receives heartbeats and never flips it true. A worker
that hangs after step N would then never trip the stall timeout for that
reconnected client. Initialize it from resume_from_step / existing step
history so reconnects keep the post-step timeout behavior, while a genuine
pre-first-step run still stays exempt. Added a reconnect regression test.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten prep-phase progress timeout comments
Condense the verbose explanatory comments and docstring on the prep-phase stall
timeout exemption to shorter, clearer forms. Comment/whitespace only; verified no
code changed via AST diff. No behaviour change.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: UNSLOTH_NPM_REGISTRY opt-in for corporate npm mirrors (#6491)
studio/frontend/.npmrc pins registry=https://registry.npmjs.org/ as a
supply-chain lock. A project-level pin takes precedence over a user's
~/.npmrc, so behind a corporate firewall that blocks npmjs.org the
frontend bun/npm install hit npmjs.org directly and failed with 403.
Add an opt-in UNSLOTH_NPM_REGISTRY env var (off by default). When set it
is threaded as --registry into every registry-touching install in
setup.sh, setup.ps1 and build.sh (bun bootstrap, bun install + retry, npm
fallback, OXC validator runtime). --registry is the highest-precedence
override for both bun and npm and leaves min-release-age and save-exact
in force, so the default lock is unchanged for everyone else.
On an install failure that looks like a blocked registry, print guidance
pointing at UNSLOTH_NPM_REGISTRY and auto-suggest the mirror already set
in the user's npm config. Registries are never switched automatically.
Also correct the .npmrc comment: the pin does not block an ambient
NPM_CONFIG_REGISTRY env var (npm and bun honor that at higher precedence);
it only guards against a lower-precedence stale ~/.npmrc.
* Studio: make the registry hint reachable under set -e; clean temp log (#6491)
run_quiet_no_exit returns non-zero on failure, which under `set -euo
pipefail` exits the script at the call site before the exit code is
captured, so the new UNSLOTH_NPM_REGISTRY hint never printed on the npm
fallback and OXC validator paths. Guard both with `|| _rc=$?` (the same
idiom every other run_quiet_no_exit caller already uses) so the failure
branch runs, and remove the _FRONTEND_INSTALL_LOG temp file on the
early-exit path.
* Studio: detect the user's mirror outside the pinned frontend dir (#6491)
_suggest_npm_registry / Show-NpmRegistryHint run while the cwd is still
studio/frontend, whose .npmrc pins registry=https://registry.npmjs.org/.
So `npm config get registry` returned that pin instead of the user's
~/.npmrc mirror, and the "Detected a registry" branch was skipped for the
main corporate case (mirror set in ~/.npmrc). Run the lookup from a
directory with no project .npmrc (/ in bash, the temp dir in PowerShell)
so the user/global mirror is surfaced. The NPM_CONFIG_REGISTRY env check
is unchanged and still takes precedence.
The post-filter safety net for 'Train on completions' fires when
train_on_responses_only() masks every token in too many rows. Its trigger is
a row-drop ratio, not a token-length check, but the message hardcoded
"max_seq_length is too short, try increasing (e.g. 8192)" -- advice that
fires identically at any max_seq_length and can recommend a value below the
user's current setting (telling someone already at 16384 to use 8192).
The dominant real cause is that the model's response template is not found in
the formatted samples: the dataset is already formatted, or its structure
doesn't match the model's chat template, so every token gets masked and the
rows are dropped. Reword the error (and the comment above it) to lead with
that cause and the actionable fix (turn off 'Train on completions'), and
mention max_seq_length only as a secondary possibility without a hardcoded
recommendation.
* Studio: clean up empty leftover quant folders so they can be deleted
An interrupted or cancelled split GGUF download leaves snapshots/<rev>/<quant>/
behind with no shards. Such a folder is neither a completed download nor a
tracked partial (no .incomplete blobs, no manifest), so it was invisible in the
variant list and a per-variant delete returned 404, leaving it on disk forever.
- list_empty_gguf_variant_dirs: detect quant folders that are empty in every
snapshot, excluding any quant that has shards in another snapshot.
- get_gguf_variants_response: surface those quants as partial (cleanable) so the
UI shows a delete affordance.
- _delete_gguf_variant_from_repos: remove the empty (or just-emptied) quant
subfolder and count it toward the result so the delete succeeds instead of 404.
Adds hub/tests/test_empty_variant_folder.py.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: simplify empty-dir check to any(iterdir())
* Studio: tighten comments on empty-quant-folder cleanup
* Studio: surface empty-folder removal failures and cleanables on local/offline paths
Address review feedback on the empty leftover quant folder cleanup:
- _remove_empty_variant_dirs now returns removal failures (read-only cache or a
locked dir), and the variant delete raises 409 instead of a misleading 404; a
concurrent download refilling the dir (ENOTEMPTY) is still treated as a skip.
- Empty leftover folders are surfaced as cleanable on every variant-listing path
(prefer_local_cache / offline / HF-fallback), not just a remote listing, via a
single post-process that flips a listed quant to partial or appends an
unlisted one.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: surface empty-folder cleanables even when metadata fetch fails
When the cache holds only an empty leftover snapshots/<rev>/<quant>/ folder
from an interrupted split download and the client is offline or the HF
metadata request fails, _compute() re-raised before cleanables were marked,
leaving the folder undeletable. Now fall back to marking cleanables against an
empty response and return them if any; otherwise re-raise the original error.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Installer: make UV_OVERRIDE space-safe on Apple Silicon (#6503)
On Apple Silicon, install.sh exports UV_OVERRIDE pointing at the bundled
overrides-darwin-arm64.txt. uv splits UV_OVERRIDE on whitespace, so a repo
cloned under a path containing a space (e.g. /Users/me/Open Source/unsloth)
truncates the value and every later uv call aborts with
'error: File not found: <truncated>' (the PyTorch install step in #6503).
Copy the overrides file into a space-free temp dir and point uv at the copy
when the path contains a space, mirroring the macOS/Linux handling already
merged for the Python installer in #6534. The temp dir is removed in the
exit trap, and the code falls back to the original path when no space-free
temp dir is available, so the no-space and non-macOS paths are unchanged.
Adds tests/sh/test_install_uv_override_space.sh, which extracts and runs the
install.sh hardening block and checks the spaced, no-space, and
spaced-TMPDIR fallback cases.
* Installer: match all whitespace (not just spaces) in UV_OVERRIDE handling
uv splits UV_OVERRIDE on any whitespace, so use the POSIX class
*[[:space:]]* rather than a literal space in install.sh (catches tabs and
newlines in the path too) and the matching test assertions. Use the portable
awk bracket expression [$] instead of \$ in the extraction so the test runs
the same under BSD awk (macOS) and GNU awk (Linux). Adds a tab-in-path case.
* Installer: clear _UV_OVERRIDE_TMPDIR before the exit trap
The exit trap rm -rf's _UV_OVERRIDE_TMPDIR. Initialize it to empty before
registering the trap so an inherited environment value can never be removed;
only a temp dir this script creates (Apple Silicon, spaced path) is cleaned.
Adds a structural test asserting the init precedes the trap.
* Run the install.sh UV_OVERRIDE space test in CI via a pytest wrapper
The Shell installer tests job uses a fixed script list (not tests/run_all.sh),
so the new shell test would not run on PRs. Add a pytest wrapper under
tests/python/ that invokes it; the auto-discovered repo CPU test job collects
tests/python/ and so executes the Apple Silicon spaced-path regression.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: honor stream=false on the GGUF agentic tool path (#6570)
* Studio: dedup the #6570 non-streaming tool tests and cover cached_tokens
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: cover the cached_tokens metadata fix and clarify the drain comment (#6570)
* Studio: align the GGUF tool drain naming and tighten its comment (#6570)
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
ensure_diffusion_visual_server() downloaded the visual-server release
asset with the unverified download_file() and marked it executable,
bypassing the approved-checksum manifest that gates every other prebuilt
llama.cpp artifact. The backend later auto-discovers that binary and
launches it through DG_VISUAL_BIN, so a compromised or substituted
release asset could place attacker-controlled native code in the install
tree and have it executed under the Studio user.
Require the matched asset to be present in the approved checksum manifest
and download it through download_file_verified() with the published
sha256. A name-matching asset that is absent from the manifest is refused
rather than executed.
Add regression tests covering the verified-download path and the refusal
of an unapproved asset.
* checkpoint preview endpoint
* harden new preview endpoints
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* address review
* Studio preview: pin adapter, guard streaming submit, robust copy-link
Harden the public per-checkpoint preview surface:
- Pin use_adapter=True in the preview payload sanitizer. Otherwise an
unauthenticated /p caller can POST use_adapter=false, which calls
disable_adapter_layers() on the shared in-memory model without restoring
it; since load_model skips reloads for the same checkpoint, every later
visitor (the page never sends the field) keeps getting base-model output
instead of the fine-tuned checkpoint. Forcing it on also re-enables a
previously disabled adapter and no-ops on merged checkpoints.
- Ignore preview-page submits while a response is streaming. The send
button was disabled but the Enter handler still called requestSubmit(),
so a second request could start before the first reply landed in msgs and
reorder the chat history. Both the keydown and submit handlers now honor
the disabled button.
- Keep the cloudflare-URL polling loop alive across transient startup fetch
errors instead of letting one rejection halt it.
- Build the copy-link from a backend preview_ref (output dir relative to
outputs_root, gated on previewability and the two-segment /p route limit)
so a nested output dir no longer copies a basename-only link that 404s.
Expose preview_ref on training run summaries.
Add route-level security tests (path traversal, payload sanitization,
asset containment, CSP header, HTML title escaping, streaming lock held
until drained) and preview_ref unit tests.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio preview: Safari-safe submit and adapter pin only for LoRA
Follow-ups from cross-browser and route simulations:
- Preview page: send the message from a shared send() helper called by both
the form submit and the Enter key, instead of form.requestSubmit(). The
latter throws on Safari < 16 and older iOS, which broke Enter-to-send there.
Verified across Chromium, Firefox and WebKit with Playwright.
- Only pin use_adapter=True when the resolved checkpoint is a LoRA adapter
(adapter_config.json present); for a merged checkpoint strip it to None.
A merged model has no adapter to toggle, so forcing it on only produced a
per-request "not a PeftModel" warning. The cross-request base-model
contamination fix still holds for LoRA previews.
Add a merged-checkpoint test asserting use_adapter is stripped to None.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio preview: trim verbose comments
Tighten comments across the preview routes, page, checkpoint helpers, and tests
to short single-line notes; drop ones that just restate the code. No behavior
change (verified comment/docstring-only with comment_tools.py check).
* Harden preview routes for PR #6486
- Return a generic 400 detail on a rejected preview path so the public /p
route never echoes the absolute install path (the real reason is logged
server-side instead).
- Strip confirm_tool_calls, session_id and rag_scope in the preview payload
sanitizer so the public surface stays inert regardless of the tool gate.
- Use Path.is_relative_to for the asset containment check, matching the rest
of the codebase.
- Add img-src 'self' and font-src 'self' to the preview page CSP.
- Preview page: on a mid-stream error keep the streamed text, flag the break,
and restore the prompt so the user can retry; drop the unused --font-sans var.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Pin isolated Node.js installer to committed sha256 digests
The isolated Node installer verified each downloaded archive only against
SHASUMS256.txt fetched from the same nodejs.org origin as the archive, so a
compromised CDN or TLS path could serve a malicious archive plus a matching
checksum and gain code execution when the extracted node is run during the
npm floor check and version probe.
Anchor trust in studio/node_prebuilt_pins.json, a committed manifest of
per-arch sha256 digests, and verify archives against it. The default channel
installs the pinned version and never fetches the remote SHASUMS. Unpinned
lts, latest, or explicit versions fail closed via UnpinnedNodeRefused unless
UNSLOTH_NODE_ALLOW_UNVERIFIED=1, and the refusal is not swallowed by the
keep-existing-on-transient-failure path. Ship the manifest in package-data.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review nits on the pinned Node installer
- Drop the unused npm_min_major field from node_prebuilt_pins.json; the floor is
the NPM_MIN_MAJOR module constant and the dead field could silently drift.
- Reword the unpinned-refusal message so it does not tell a user already on the
default to install it, and point the "add a pin" hint at the exact asset.
- Decode the opt-in SHASUMS body with errors="replace" so a non-UTF8 response
yields a clean PrebuiltFallback instead of an uncaught UnicodeDecodeError.
- Tests: assert the refusal message (guards the main() catch order, not just the
exit code), cover malformed-manifest parsing, and drive the opt-in remote-SHASUMS
path end to end through install_prebuilt.
* Tighten comments in the pinned Node installer
Collapse multi-line rationale comments to single lines, drop docstrings on the
obvious internal helpers (load_pins, pinned_sha256), and shorten the manifest
note. Comments/docstrings only; verified code-unchanged via AST comparison.
* Address Codex review: verify pins on existing installs; tomllib fallback
- existing_install_matches now takes an expected_sha and the short-circuit passes
the committed pin, so a version-matching but non-pinned or tampered install (e.g.
from the old remote-SHASUMS path) is re-verified instead of kept. An unpinned
target without opt-in no longer short-circuits on an existing install; it falls
through to the UnpinnedNodeRefused fail-closed path.
- The package-data test uses pytest.importorskip(tomllib/tomli) so it does not
ModuleNotFoundError on the supported 3.9/3.10 interpreters.
* Make the transient-failure keep-existing path pin-aware
The previous commit added the pinned-digest check to the existing-install
short-circuit but not to the post-download-failure fallback, which still kept any
runnable same-version install via existing_install_usable(). A same-version
install whose recorded sha256 is not the pin could therefore be kept on a
transient download failure, the exact artifact the short-circuit rejects. Refuse
to keep a same-version pin-mismatched install there too; a different usable
version is still kept for offline resilience.
* Bump pinned default Node to the current 24 LTS (24.18.0)
Node 24 LTS moved to 24.18.0; since the default channel now resolves straight to
the manifest, a frozen 24.17.0 would downgrade fresh installs and make
UNSLOTH_NODE_VERSION=lts refuse the current LTS as unpinned. Update default_version
and all six per-arch digests (verified against the official SHASUMS256.txt), and
point the test INDEX/short-circuit fixtures at the new LTS.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
The fade is recomputed on a group's open/close state flip, but the groups
animate their height, so it measured scrollHeight mid-animation and the
fade could vanish at random. Re-measure on the collapsible animationend,
and add the missing pinnedOpen dep to the recompute effect.
Footer padding moves from pt-3 pb-4 to pb-3 with a conditional top: pt-1.5
when the update card is shown so the fade hugs it, pt-2.5 for the profile
on its own. When the update card is shown, shorten the fade above it
(h-10 -> h-3) so the list reads closer to the card.
* Studio: group the project export menu by Combined and Per chat
Replace the repeated (combined)/(per chat) suffix on every export row with a
section subheading, so the rows read Raw JSONL / CSV / ShareGPT JSONL under
Combined and Per chat headings.
* Studio: group export sections with DropdownMenuGroup
Wrap each Combined and Per chat section in DropdownMenuGroup for screen-reader
semantics, and drop the redundant px-3 already set by DropdownMenuLabel.
* Studio: cap GGUF context to unified memory on Apple Silicon
* Studio: tighten Apple ctx-cap comments and drop the overstated MLX-sync claim
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: reserve flat MTP fraction and floor sparse-KV ctx in the Apple unified-memory cap
The Apple Silicon GGUF context cap mirrored the discrete-GPU auto-fit branch but
missed two protections the discrete path already applies:
- It passed the full unified-memory budget with budget_frac=1.0 without first
reserving the flat MTP fraction the discrete path takes off via _pin_fraction.
With an MTP draft whose KV cannot be byte-sized (e.g. Qwen3.6-MTP, #6529), the
cap filled the whole budget and left nothing for the draft, so unified memory
could still over-commit. Reserve _flat_mtp_reserve up front; this is a no-op
when MTP is not engaged.
- It required _can_estimate_kv(), so a GGUF with sparse KV metadata skipped the
cap entirely and launched at full native context. Mirror the discrete
file-size-only fallback and floor the auto context to 4096 when the cache
cannot be sized.
Adds regression tests for both paths.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: tighten comments in the Apple unified-memory context cap
Condense the verbose comment blocks in the Apple budget helper, the no-GPU
Metal branch, and the context-fit tests. Comments only, no code change
(verified with ast-based comment_tools check); suite still green.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
The Launch-section blurb described --secure as a secure HTTPS link
instead of a raw port and stressed that the raw port is never exposed,
which reads as the more private option. The Cloudflare tunnel actually
publishes Studio at a public trycloudflare.com URL, and server-side
tools let anyone with the API key run code. Reword to state the public
exposure and the code-execution caveat, matching the installer hints and
the Remote access section.
* Copy uv `-c`/`-r` paths to a space-free temp dir on macOS/Linux
uv 0.11.x truncates a constraints/requirements path passed via `-c`/`-r`
at the first space, so `unsloth studio` setup from a repo cloned under a
path containing a space (e.g. `/Users/me/Open Source/unsloth`) fails with:
error: File not found: `/Users/me/Open`
_uv_safe_path() already worked around this on Windows via the 8.3 short
path but returned the space-containing path unchanged on macOS/Linux,
which have no 8.3 equivalent. Extend it to copy the (small, flat)
constraints/requirements file into a space-free temp dir and hand uv the
copy; the temp dirs are removed at process exit. Falls back to the
original path on any error, so it is never worse than before.
Refs unslothai/unsloth#6503
* Route UV_OVERRIDE through _uv_safe_path and fix temp-dir leak (#6503)
The -c/-r fix did not cover UV_OVERRIDE, which uv also truncates at the first
space. On Apple Silicon the overrides file is handed to uv via UV_OVERRIDE at
install time (install_python_stack.py) and during the MLX self-heal
(utils.mlx_repair), so a repo under a path containing a space still broke every
uv call there. Move _uv_safe_path into backend.utils.uv_path_safety so both
sites share it, and route UV_OVERRIDE through it.
Also stop leaking the temp dir when shutil.copyfile fails after mkdtemp, and add
tests for the UV_OVERRIDE channel, the TMPDIR-with-space fallback, the atexit
cleanup, and the no-leak path.
---------
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
* Tidy verbose Studio launch messages
The reachability failure hint, the loopback deploy block, and the stop
hint were several lines longer than they needed to be, and the banner
printed a Tip line that just repeated the URL already shown above. Shorten
them while keeping the useful detail (cloud provider names, the SSH
local-forward workaround, the relaunch command, the trusted-network and
macOS Ctrl+C notes). No behavior change, output wording only.
* Refine launch-message wording after review
Apply review feedback so the launch messages read well for both experts
and general users:
- reachability hint: restore the 'from your own computer' cue and put the
ssh command on its own line so it stops wrapping, name the cloud rules
precisely (GCP firewall / Azure NSG rule), and restore 'in your browser'
- loopback banner: add the missing colon, drop the Ctrl+C duplication
(the stop hint already covers it), and explain the exposure in plain
language instead of 'exposes the API on every interface'
- stop hint: trim so it fits an 80-column terminal without wrapping
- replace two pre-existing em dashes with --
The installer/setup launch hints described --secure as merely allowing
HTTPS. In practice --secure forces a loopback bind and opens a public
Cloudflare quick tunnel (https://*.trycloudflare.com) to Studio, which
serves Python/terminal tools by default, so the old wording understated
the exposure. Update the hint to say it is a public Cloudflare HTTPS link
and that anyone with the API key can run code, matching the runtime
secure-mode banner.
* Fix _SameTaskStreamingResponse disconnect test bypassing __init__
test_same_task_response_closes_body_iterator_on_send_disconnect builds the
response via __new__ to skip Starlette's __init__, then wires body_iterator,
background, and stream_response by hand. It never set _unstarted_cleanup, so the
disconnect-before-first-chunk branch of __call__ raised AttributeError instead of
ClientDisconnect, failing the Backend CI "Repo tests (CPU)" job on main.
Set response._unstarted_cleanup = None in the manual construction, matching the
default __init__ assigns.
* Shorten the _unstarted_cleanup comment to one line
* Studio: confirm saved Hugging Face token with a tick
Pasting a token and clicking away saved it silently with no feedback.
Show a green tick in the field once a non-empty token is committed and
the input still matches the stored value, so the save is visible. Adds
the tokenSaved label to the en and zh-CN locales.
* Studio: let clicks pass through the saved-token tick
The decorative tick sat over the input and swallowed clicks, so clicking
it would not focus the field. Add pointer-events-none so clicks reach the
input; drop the now-unreachable title and keep an aria-label via role=img.
* Studio: slim the GLM-5.2 thinking menu width
The high|max effort menu has short labels, so drop it from min-w-44 to
min-w-40 for GLM-5.2 only. Other models keep the wider dropdown.
* Studio: keep Preserve thinking menus at full width
Only narrow the effort menu when it has no Preserve thinking row. That
row is longer than the high|max labels and wraps to two lines at the
min-w-40 floor, so gate the skinnier width on no preserve-thinking.
* Fix construct_chat_template leaking {INPUT}/{OUTPUT} sentinel into the template
In construct_chat_template's inner process() helper, the branch handling a
section that starts with the {INPUT}/{OUTPUT} sentinel sliced the part from
part.find(which) (which is 0 in that branch), so the literal sentinel was
re-included in the generated Jinja chat template. The endswith branch already
slices correctly with part[:part.find(which)]; this slices past the sentinel
with part[len(which):], so a template whose input or output section begins
with the sentinel (for example a user turn that starts with {INPUT}) renders
correctly instead of emitting a literal {INPUT}/{OUTPUT}.
Added a regression test covering {INPUT}-leading and {OUTPUT}-leading sections.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix SyntheticDataKit.chunk_data dropping single-chunk documents
chunk_data turns the n boundary points from np.linspace into n-1 ranges
via the boundaries[:-1] / [1:] pairing. When a document fits in a single
chunk (n_chunks == 1) that produces zero ranges, so the loop writes no
files and the whole document is silently dropped. Emit the full
[0, length] range when n_chunks <= 1; the multi-chunk path is unchanged.
Added a regression test covering the single-chunk and multi-chunk cases.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* chunk_data: emit nothing for an empty document (no empty chunk file)
Addresses review feedback: when the input document is empty (length == 0),
return no chunks instead of writing a single empty chunk file. Added a
regression test for the empty-document case.
* chunk_data: reject overlap >= chunk size (non-positive stride)
Per review feedback: when overlap >= max_tokens the chunk stride is
non-positive, which would divide by zero or silently emit one oversized
chunk. Raise a clear RuntimeError for that unusable configuration. Added
a regression test.
* Broaden single-chunk guard to length <= max_tokens (also fixes sub-overlap docs); expand tests
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
* Studio: drop OpenEnv and unused ExecuTorch/open_spiel install deps
* Studio: drop 8 more unused install deps from extras
* Studio: restore tomli<3.11 for kernels; tidy dep-cleanup comments and tests
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: refresh scan-packages baseline for scipy _external + unsloth-zoo tests
scipy moved its vendored array_api_compat from scipy/_lib to
scipy/_external, so the four allowlisted array_api_compat __init__.py
entries stopped matching and resurfaced as unsuppressed CRITICAL
"Downloads and executes remote code" findings on all three pip
scan-packages shards (extras, hf-stack, studio). Add the _external
paths next to the existing _lib ones so both scipy layouts stay covered.
Allowlist two unsloth-zoo test-file false positives now present in the
hf-stack shard: tests/test_mlx_save_export_regressions.py (writes to
/tmp dropper) and tests/test_mlx_trainer_internals.py (obfuscation plus
exec/eval).
Drop nine stale entries for packages removed from the Studio
requirements and no longer in any shard closure (evaluate, pytest,
hypothesis, kgb, langid), confirmed absent via with-deps resolution of
all three shards.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Harden MLX self-heal install against supply-chain execution
The Apple Silicon MLX self-heal runs uv pip install on a daemon thread
during Studio startup, default-on with only an env opt-out, before the
post-install stack check. Two things widened the supply-chain surface:
- it accepted source distributions, whose PEP 517 build backends run
arbitrary code at install time; and
- it forwarded the full process environment, exposing Studio secrets to
that code and letting a poisoned env (UV_FIND_LINKS / UV_DEFAULT_INDEX)
repoint the install at a hostile source.
Require pre-built wheels (--only-binary=:all:) and forward only an
allowlist of variables uv needs (PATH/HOME, proxy + CA settings, cache
dir), setting UV_OVERRIDE ourselves. mlx/mlx-metal ship wheels only and
mlx-lm/mlx-vlm publish py3-none-any wheels, so a healthy self-heal is
unaffected; an unavailable wheel just leaves Studio chat-only as before.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Drop cache-dir env vars from the self-heal allowlist
Address review: a poisoned process env could set UV_CACHE_DIR / XDG_CACHE_HOME
to redirect uv at an attacker-staged cache (cache poisoning, symlink writes),
which partly undercut the index-redirect protection. Drop them from the
allowlist; uv falls back to its safe user-owned default cache, still reused
across runs, so there is no normal-path cost. Test now asserts both are
excluded from the install env.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix Qwen3 NaN: self-heal vision pad_token in load_correct_tokenizer
Text-only Qwen3 (and Qwen2.5) models share Qwen3-VL's vocab, so their
Hub tokenizer configs ship <|vision_pad|> as pad_token. Padding text-only
training with a vision token corrupts attention/loss and produces NaN
losses and gradients on affected stacks.
patch_tokenizer already heals this, but only when a model with config is
passed. The standalone load_correct_tokenizer path (and custom training
loops) still returned <|vision_pad|>. This adds a model-independent guard
in load_correct_tokenizer that replaces a vision pad_token on text-only
tokenizers with the first safe text token (<|endoftext|>, <pad>, [PAD],
<unk>), falling back to eos_token only if it differs from pad_token.
The result now matches upstream Qwen configs (pad_token <|endoftext|>,
id 151643) with no new token added. Vision processors (image_processor
present) and non-vision pad tokens (Llama, Qwen2) are left untouched.
Fixes#3155
* Format pad_token helper for ruff kwarg-spacing hook (pre-commit)
* Harden vision pad_token fix: drop unk candidate, guard get_vocab, skip vision eos
* Tighten code comments (no logic change)
* Delegate pad_token fix to shared unsloth_zoo.pad_token
Generalize the narrow _fix_vision_pad_token by delegating to unsloth_zoo's
shared fix_pad_token (single source of truth, AGPL-3.0), which scans the
reserved-token families instead of only the vision-pad case. A guarded import
keeps this working against an older unsloth_zoo that has not shipped the module
yet: on ImportError it falls back to _fix_vision_pad_token.
allow_add=False is passed so the early load_correct_tokenizer call stays
side-effect free (no model here to resize embeddings); the later model-aware
patch_tokenizer call finishes the job and is idempotent.
Adds tests/python/test_pad_token_fix.py covering both dispatch paths offline.
* Fix CPOTrainer crash with multimodal processors
CPOTrainer shares build_tokenized_answer/tokenize_row and __init__ with
ORPOTrainer, but the ORPO replacement functions that route tokenization
through the underlying text tokenizer and resolve pad_token_id were only
registered for orpo_trainer. With a multimodal processing class (e.g.
Gemma4Processor) the positional self.processing_class(prompt, ...) call binds
prompt to images=, leaving text=None and raising
TypeError: 'NoneType' object is not subscriptable.
Register the existing orpo_trainer_text_tokenizer and
orpo_trainer_processor_pad_token under cpo_trainer as well so CPO/SimPO
fine-tuning of multimodal models works. No change for plain tokenizers.
* Add CPO processor tokenizer regression test
Static, CPU-only checks that cpo_trainer registers the same
orpo_trainer_text_tokenizer and orpo_trainer_processor_pad_token rewriters as
orpo_trainer, and that the rewriter drops the broken positional
self.processing_class(prompt, ...) call. Guards against issue #4952 regressing.
* Format CPO test assert for ruff line length (pre-commit)
* Bind CPO __init__ pad/eos token reads to underlying tokenizer
TRL 0.28+ CPOTrainer.__init__ reads bare processing_class.pad_token and
processing_class.eos_token before pad_token_id, which raises AttributeError
for multimodal processors (e.g. Gemma) where those live on .tokenizer.
Extend orpo_trainer_processor_pad_token to route that block through the
underlying tokenizer, and add a regression test.
* Tighten code comments (no logic change)
* Make CPO/ORPO rewriters reach the trainer on TRL 1.x
TRL 1.x moved CPOTrainer and ORPOTrainer out of trl.trainer into
trl.experimental.<algo> and dropped the trl.trainer.<algo>_trainer shim that
older TRL (0.26 - 0.28) kept. patch_trl_rl_trainers() discovers trainers via
dir(trl.trainer), so on TRL 1.x cpo_trainer and orpo_trainer are never found and
the multimodal-processor tokenization fix (#4952) silently stops applying, even
though the rewriters themselves still match the source.
Re-expose experimental-only trainers that Unsloth has rewriters for (RL_FUNCTIONS
keys) under trl.trainer before discovery, so the existing patch machinery and its
thin-wrapper resolution work unchanged. The alias is a no-op on older TRL where
trl.trainer.<algo>_trainer already exists.
Also rebind the patched Trainer/Config into every already-imported trl.* module
that holds the original class so the fix is visible at the experimental import
site (from trl.experimental.cpo import CPOTrainer), not only via trl.trainer.
Verified on transformers 4.57.6 + trl 0.22.2, transformers 4.57.6 + trl 0.27.1,
and transformers 5.12.1 + trl 1.6.0: CPOTrainer with a multimodal processor
tokenizes through the underlying text tokenizer with no crash on all three, and
the SFT/GRPO/DPO patch paths are unchanged.
* Format for ruff (pre-commit)
* Simplify CPO fix to mirror ORPO registrations (#4952)
Register the existing ORPO row-tokenizer/pad-token rewriters for cpo_trainer.
Under the trl<=0.24.0 pin CPOTrainer lives in trl.trainer.cpo_trainer (found by
dir(trl.trainer)), shares ORPO's build_tokenized_answer and uses
processing_class.pad_token_id, so the two registrations are sufficient.
Drop the trl 1.x experimental aliasing/rebind machinery in rl.py and the
bare-pad_token rewriter: trl 1.x (CPO in trl.experimental) and the bare
pad_token pattern (trl>=0.28) are not installable under the pin.
* CPO: route bare pad_token/eos_token default through inner tokenizer
TRL 1.x CPO/ORPO __init__ (the trl.experimental source unsloth resolves on TRL
0.26+) defaults processing_class.pad_token from processing_class.eos_token
before tokenizing. Multimodal processors (Gemma3/Gemma4 Processor) expose those
attributes on .tokenizer, not on the processor, so that bare access raises
AttributeError during __init__ even with the pad_token_id fallback registered.
Extend orpo_trainer_processor_pad_token to rewrite that defaulting block to run
on the inner tokenizer. The pinned TRL range (<=0.24.0) has no such block, so
the regex is a no-op there and only the existing pad_token_id fallback applies.
Verified the rewrite against the real trl 1.6.0 experimental CPOTrainer.__init__
(bare access removed, result compiles, a processor without pad_token no longer
raises) and added offline regression tests for both the rewrite and its no-op.
---------
Co-authored-by: Daniel Han <michaelhan2050@gmail.com>
pre-commit-ci[bot]