Adds a configurable Hugging Face model download cache location to Unsloth Studio, selectable from Settings, with per-cache download manifests, scoped deletion, and read-only inventory of previously selected caches.
* Studio: offer the latest transformers release for brand-new architectures
When a model's config.json model_type is absent from every installed
transformers overlay (base 4.57.x and the .venv_t5_530/550/510 sidecars),
Studio now checks, unauthenticated and cached, whether the newest
transformers ships it:
- utils/transformers_latest.py fetches the latest release version from
https://pypi.org/pypi/transformers/json and the CONFIG_MAPPING_NAMES
sources for that tag and for main from raw.githubusercontent.com
(never api.github.com), parsing them with the same AST extractor the
static router uses (no code execution, no trust_remote_code). Results
are cached in memory and in a JSON snapshot under studio_root()/cache
with a one day ttl; fetches are bounded to 5s with one retry and a
failure backoff, and offline mode or the new kill switch
UNSLOTH_STUDIO_NO_LATEST_TRANSFORMERS=1 short-circuits to None.
- POST /api/inference/validate gains requires_transformers_upgrade plus
a transformers_upgrade payload (model_type, pypi_version,
supported_in_pypi, supported_in_main) so the frontend can raise the
install consent dialog before /load, mirroring the existing
remote-code consent flow. The check fires only when the model_type is
unknown to all installed overlays and the hardcoded tier tables.
- POST /api/inference/install-latest-transformers provisions a new
persistent .venv_t5_latest sidecar after user consent, pinned to the
exact PyPI version (re-verified server-side) with the same
--target/--no-deps recipe as the fixed sidecars. A JSON pin marker
inside the dir records the installed package set, so restarts
revalidate it and routing resolves the new highest-ranked tier
automatically. A dependency preflight (compat_plan) compares the
release's requires_dist against the running env: unsatisfied
tokenizers/safetensors floors are shadow-installed as exact pins into
the sidecar, anything else unsatisfied blocks the install with a
clear message.
Routing for every already-supported model_type is unchanged: the
hardcoded lists and the 530/550/510 static resolver run first, the new
tier only participates once its venv exists, and the probe order gains
the latest sidecar only when provisioned. Verified against live PyPI
and GitHub (transformers 5.13.0: 674 model_types, 26 absent from all
installed overlays, e.g. cosmos3_omni; 4 dev-only on main) and with a
real sidecar install plus restart persistence. 64 new tests; the
existing 200-test transformers_version suite passes unchanged.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Latest-transformers check: fetch outside the lock, serialize installs
Release the module lock during the network refresh so a slow fetch cannot
stall other threads in the ASGI pool; concurrent callers during a fetch get
None (the graceful fallthrough) via an in-flight flag instead of stacking
fetches. Serialize install_latest_transformers with an in-progress flag so
concurrent consents cannot race the sidecar delete and recreate; the loser
gets a structured already-in-progress refusal.
* Latest-transformers check: LoRA bases, pin-gated mapping, live reverify
Run the upgrade check over the [adapter, base] target set so a LoRA whose
base model is a brand-new architecture surfaces the prompt (the worker
activates transformers for the base, not the adapter).
Gate the latest overlay's mapping lookup on a valid pin marker, matching
activation and the probe order, so a partial or manual .venv_t5_latest dir
cannot be routed to and then refused at activation.
Re-verify the requested version against a live PyPI snapshot at install
time, falling back to the cached one on fetch failure, so a release
published inside the cache TTL is not silently missed.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Latest-transformers check: nested config types and latest-tier vision probe
Collect every model_type in the config (top level plus each nested
sub-config) and signal on the first one missing from all installed
overlays, so a supported wrapper carrying a brand-new backbone still
surfaces the upgrade prompt; wrappers instantiate sub-configs through
CONFIG_MAPPING and would fail on the nested type.
Route the vision capability subprocess through the pinned latest sidecar
when the model resolves to the latest tier, so latest-only VLMs are not
misclassified as text-only; every other tier keeps the 5.5 sidecar used
today.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Latest tier: nested routing, vision probe after raw miss, safe upgrades
Route by every model_type in the config: a nested sub-config type can raise
the tier (wrappers instantiate sub-configs through CONFIG_MAPPING), so a
supported wrapper with a latest-only backbone routes to latest once
installed instead of staying on default. An unknown nested type never
vetoes; the primary type keeps its previous semantics. The collector is
shared with the upgrade checker.
Vision detection: when the raw heuristics say False for a model that routes
to the latest tier, run the AutoConfig subprocess under the pinned latest
sidecar instead of trusting heuristics built from older transformers.
Provisioning: stage-and-swap. Build the new sidecar in .venv_t5_latest.staging
and swap it in only when the install and pin marker are complete, so a failed
upgrade never destroys a previously working sidecar; restore the old dir if
the final swap fails.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Latest-transformers checker, vision subprocess, and cache fixes
Require the latest release to support every missing model_type (the
primary included) before prompting; a nested-only match cannot make the
model loadable, so no install is offered for it.
The vision-check subprocess now unions the active sidecar's own
registry mappings into the inlined parent-process detection sets, so
architectures only the sidecar knows classify correctly.
A successful sidecar install clears the tier probe cache, the latest
tier's model_type mapping, and the vision-detection cache so the new
venv takes effect without a restart. Tests for all three.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Aggregate upgrade support flags and keep install off /v1
The upgrade signal now reports supported_in_pypi only when the latest
release covers every missing model_type; a mix with a main-only nested
type surfaces as dev-only so no PyPI install is offered that would
still fail at load. The consented install endpoint moves to
studio_router so it is not reachable through the OpenAI-compatible /v1
mount. Tests for both.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Honor the latest-transformers kill switch in routing
With UNSLOTH_STUDIO_NO_LATEST_TRANSFORMERS set after the sidecar was
provisioned, the latest tier still joined mapping and probe routing
because only the pin was checked. Both admission points now also check
the kill switch, so operators can roll back a problematic sidecar
without deleting files.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Repair the latest sidecar through stage-and-swap
The lazy repair path installed into the live .venv_t5_latest, which
_ensure_venv_dir wipes first, so a failed repair deleted the pinned
sidecar and its marker. Both the consented install and the repair now
share one stage-and-swap helper: the incomplete-but-pinned dir survives
any failure and a later attempt can still repair it.
* Tighten comments
* Remove the staging dir when a latest-sidecar install fails
A pip failure inside _ensure_venv_dir returns False without raising, so
the except cleanup never ran and the partial .venv_t5_latest.staging
leaked until a later attempt. Also note on the validate response fields
that frontend consumption ships in the follow-up PR.
* Add the transformers-upgrade consent dialog to the frontend
When /validate reports requires_transformers_upgrade, every explicit load
path (chat runtime and the compare composer) now pauses on a consent
dialog modeled on the remote-code one: it names the model_type and the
latest PyPI transformers version, and on Accept calls
/api/inference/install-latest-transformers itself, shows an installing
state, and resumes the original load automatically on success. Errors
surface in the dialog with a retry; Cancel aborts the load like the
trust dialog's deny path. Architectures shipped only on transformers
main get a dev-only notice with no install button. Background auto-load
skips upgrade-requiring candidates instead of prompting, mirroring the
trust_remote_code rule. The dialog mounts once in the root layout and
runs before the security dialogs, since no load can proceed without the
runtime.
* Route a non-installable new architecture to the custom-code consent as a last resort
When the upgrade dialog has no installable PyPI release (the architecture
is only on transformers main, which Studio never installs), the dialog now
says so explicitly, and when the model also declares custom (auto_map)
code it offers Continue with custom code: resolving the paused load into
the existing trust_remote_code consent gate instead of hard-aborting.
Models with no custom code keep the Cancel-only notice. The backend
returns no upgrade signal at all for architectures unknown to both PyPI
and main, so those still route straight to the unchanged security gate.
* Force a 16-bit load for models on the latest-transformers sidecar
Live validation with Zyphra/ZAYA1-8B (model_type zaya, shipped by
transformers 5.13.1 but unknown to every installed tier) surfaced a
generation crash when the consented sidecar load kept the default bnb
4-bit quantization: transformers' grouped-MoE kernels feed the packed
uint8 expert weights straight into torch._grouped_mm, and generation
dies (plain 16-bit works). New latest_tier_active_for() mirrors the
sidecar activation's tier resolution and never raises; the inference
worker flips load_in_4bit off when it reports true, and the load route
applies the same flip so the pre-load VRAM guard and the worker command
agree. Fixed tiers are untouched. With the guard, ZAYA1-8B loads and
generates correctly in Studio chat.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Offer the custom-code fallback when a latest-sidecar install fails
* Fail remote mapping fetches wholesale and mirror the 16-bit flip in validate
A transient fetch or parse failure of one auto-mapping file no longer caches
a partial latest-release map for the TTL (a real 404 on pre-5.10 tags is
still tolerated), and validate_model now applies the same latest-sidecar
16-bit sizing flip as /load before the training guard so the two agree.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten comments in the latest-transformers changes
* Resolve remote LoRA bases, fold nested tiers, and guard the sidecar swap
latest_tier_active_for now resolves a remote adapter's base model the same
way worker pre-activation does (and returns early without a sidecar pin), a
hardcoded fast-path tier is raised when a nested sub-config's model_type
needs a higher sidecar, and the install route refuses to swap .venv_t5_latest
while training runs on it and unloads a latest-tier chat model first.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Gate the sidecar install on worker liveness and size installable upgrades 16-bit
The install route now refuses while any training or export runs (tier
re-resolution without the load token is unreliable for gated repos), holds
the inference lifecycle gate across the unload and the swap so no load can
interleave, and passes the model name to unload_model. validate_model runs
the upgrade check before the training guard and sizes an installable
upgrade as 16-bit, matching what /load and the worker will force after the
consented install.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Close the sidecar install races and honor the kill switch over cached mappings
Training starts and mutating export routes now refuse while a transformers
install is in progress (shared is_install_in_progress flag), the chat unload
and idle export-worker teardown moved into a before_swap hook that runs only
once the staged install succeeded, and _config_model_types checks the kill
switch before returning a cached latest mapping.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Reserve the sidecar swap before the gate wait and abort it on failed teardown
The install-in-progress flag moved into a shared sidecar swap reservation in
transformers_version, taken by the install route before awaiting the
inference lifecycle gate (so training and export starts see it for the whole
window) and by the lazy .venv_t5_latest repair path. The before_swap hook
now raises when the chat unload or export teardown reports failure, leaving
the previous sidecar untouched.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Back the sidecar swap reservation with a cross-process lock file
The lazy repair runs inside worker subprocesses, where a module-level flag
is invisible to the parent's route checks. The reservation now also creates
a lock file next to .venv_t5_latest (O_EXCL, owner-only removal, stale after
two hours for crashed owners), so is_install_in_progress sees a repair from
any Studio process.
* Hand the swap reservation to the installer thread and harden pre-swap teardown
A cancelled install request no longer releases the reservation while the
installer thread is still staging (the thread owns and releases it, shielded
from cancellation). The route refuses while another inference request is
generating, export teardown runs before the chat unload and is judged by
worker liveness rather than the cleanup return value, and a live inference
worker with no active model (failed load residue) is shut down before the
swap.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Keep the lifecycle gate with the installer and recheck the swap at spawn time
The gate moved into the shielded install task so a cancelled POST cannot
release the guard /load honors while the installer still runs, cached latest
probe results are ignored while the kill switch is set, and the training and
export subprocess spawns recheck the sidecar swap reservation right before
spawning (the route-level guards are one-shot and validation can outlast an
install's start).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Close the spawn-registration windows against the sidecar install
Training marks the spawn in progress before its reservation recheck and
is_training_active honors the flag, so the install route sees a start that
has passed proc.start() but not yet recorded _proc. Export load-checkpoint
rechecks the reservation after setting _export_active and before tearing
down the old worker, so losing the race keeps the loaded checkpoint instead
of surfacing a 500.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Refine the install-window interleavings around worker teardown
The inference busy count is rechecked under the lifecycle gate (streams
start by taking that gate, so nothing slips past a held gate), the training
handshake moved ahead of the VRAM-freeing before_spawn hook so a lost race
leaves chat/export intact, the export spawn-time check is op-aware (inside
an active op the install is the side that aborts), and the Xet-stall respawn
waits out a transient reservation instead of stranding the run.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Track the install's server-side unload and guard export ops against the swap
The upgrade dialog store records when its install actually ran (the server
unloads the active chat model before swapping), and the load flow then marks
the previous model as unloaded so a later cancelled gate still triggers
rollback; the custom-code fallback leaves the flag unset. _run_export gained
the same reservation handshake as load_checkpoint so an install cannot block
behind an hours-long export op instead of returning 409.
* Tighten comments in the install-guard and upgrade-consent changes
* Surface install-race refusals cleanly and roll back after a failed swap unload
/load refuses while the sidecar swap is reserved so a load cannot succeed
and immediately be unloaded by the pre-swap teardown, worker starts that
lose the install race raise a typed SidecarSwapInProgress mapped to 409
instead of a 500, the install response reports model_unloaded even on a
structured failure so the client can restore its state, and the compare
flow tracks the server-side unload like the primary load path and clears a
stale checkpoint on abort.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Type the export install races, scope the lock release, and keep the unload signal
Export load-checkpoint and export ops raise SidecarSwapInProgress (mapped to
409 in every export route) instead of a 400-shaped failure, the export spawn
check distinguishes repair reservations (always refused) from install ones
(op-aware), the swap lock release only unlinks a lock this process wrote so
a stale-superseded owner cannot drop the new owner's live lock, and the
frontend unload signal survives a superseding consent via read-and-clear
consumption instead of a reset.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Finalize a stalled run when the respawn loses the install race and latch the unload signal
The Xet-stall respawn timeout now finalizes the run as a failure instead of
raising into the pump's broad finalization catch (which stranded it in a
training state with no worker), and a successful install retry ORs the
model_unloaded signal with the latched value so a failed-after-unload first
attempt still triggers rollback.
* Recheck the swap under the load gate and latch the unload before resolver checks
/load rechecks the sidecar reservation after acquiring the lifecycle gate
(an install can reserve while the load queues on it), and the dialog store
latches model_unloaded as soon as the install response arrives, before any
resolver-identity guard, so a superseded consent's unload still reaches
whichever load consumes the signal next.
* Report cleared-state unload failures, guard queued installs, and fold name tiers
A failed chat unload that still cleared the orchestrator's model state now
reports model_unloaded so the client rolls back, the installer aborts with
a 409 when a model load completed while it waited on the lifecycle gate,
and the fixed-tier name fast path consults the config mapping when a latest
sidecar is pinned so an accepted upgrade routes to the sidecar it installed
(no I/O added to the unpinned path).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Report cleared-state unload failures and harden the spawn handshake flag
The failed-unload branch in before_swap now detects that the orchestrator
cleared its model state and reports model_unloaded before aborting (the
earlier commit claimed this fix but a scripting error dropped the edit),
the installer's queued-load check compares a load generation counter so a
same-model reload is caught, and both training spawn sites wrap everything
after the handshake in a guard that resets _spawn_in_progress on any
exception so a failed start cannot wedge is_training_active.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Bump the load generation when the load is published, not at load start
A start-time bump is already visible when the installer snapshots mid-load,
so a same-model reload completing after the snapshot looked unchanged and
could be unloaded by the swap. The counter now increments alongside the
active_model_name publish.
* Self-heal a broken pinned sidecar, guard lazy repairs, and refresh stale retries
A valid pin whose transformers source dir vanished now triggers the repair
from the routing path (with a five minute backoff after failures) instead of
silently routing latest-only models to older tiers, the lazy repair refuses
while parent-visible chat/training/export workers are active since it has no
teardown of its own, and a version-mismatch install failure carries the
superseding release so the dialog's Retry re-requests a version that can
succeed.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Flip latest-tier loads to 16-bit outside chat and protect export state
Training and export workers now apply the same latest-sidecar 16-bit flip
as the chat worker so a brand-new grouped-MoE architecture cannot reach bnb
4-bit through those paths, the latest-tier vision override returns None on
an inconclusive probe so a transient failure is not cached as not-vision,
and the install route refuses while an idle export checkpoint is loaded
rather than discard it with no rollback signal on a failed swap.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address parallel-review findings on the sidecar guards and install checks
The training route sizes latest-tier jobs 16-bit before GPU selection, the
inference subprocess spawn rechecks the swap reservation like training and
export (covering the OpenAI auto-switch path) with the typed error mapped
to a retryable 409, compat_plan blocks the install when dependency metadata
cannot be fetched instead of proceeding unverified, snapshot model-type
lists must contain only strings, and pin-marker package specs are validated
against the sidecar's own package set before ever reaching pip.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Parent-only repairs, live-owner locks, remote-base activation, pre-teardown recheck
Lazy sidecar repairs now refuse inside worker children (whose empty backend
singletons cannot see live siblings) and run only in the parent where the
active-worker guard is real, swap-lock staleness requires the owner pid to
be dead so a slow live install is never superseded, both activation entry
points resolve a remote adapter's base model like the inference worker and
latest_tier_active_for already do, and load_model rechecks the reservation
before tearing down the old worker so losing the race keeps the current
model loaded.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Check workers under the repair reservation and keep state on refused swaps
The lazy repair now reserves first and checks workers under the reservation
(worker starts set their active markers before rechecking, so every
interleaving aborts one side), with export ops and in-flight inference loads
counted as active. The inference pre-teardown and spawn guards refuse only
repair reservations since an install shares the load's lifecycle gate and
aborts via its queued-load snapshot, a SidecarSwapInProgress raised before
teardown no longer clears the live model mirrors, and an export spawn abort
after teardown clears current_checkpoint so the page cannot claim a loaded
checkpoint with no worker.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Repair a present-but-incomplete latest sidecar from routing
The routing self-heal only fired when the pinned sidecar's transformers/
dir was missing. A sidecar that kept transformers/ but lost another pinned
package still routed models to the latest tier, and workers refuse
parent-only repairs, so every load failed until a manual reinstall. Routing
now validates the full pin (via _venv_dir_is_valid) and repairs any
incomplete sidecar under the same swap reservation and 5-minute backoff.
* Treat an unrepaired latest sidecar as unavailable in routing
When the pinned sidecar is incomplete and the lazy repair fails (offline,
pip failure, workers active) or is inside the backoff window, routing
returned the source dir anyway, sending models to a tier whose worker
activation is known to fail. Return None instead so models an older tier
supports keep loading there until a repair succeeds, matching the behavior
when the sidecar dir is missing entirely.
* Harden sidecar swap and repair against crash, survivor, and 16-bit paths
Reclaim a swap lock as soon as its recorded owner PID is dead instead of
waiting out the two-hour cutoff, so a crash mid-install no longer wedges
/load, training, export, and repair for hours. A lock whose PID cannot be
read yet still uses the long cutoff so the create-before-write window is
never mistaken for dead.
Probe process liveness with OpenProcess on Windows: os.kill(pid, 0) there
is CTRL_C_EVENT (a real Ctrl+C via GenerateConsoleCtrlEvent), not a
harmless check, and psutil is not always present.
Return whether _shutdown_subprocess actually killed the worker and keep the
live handle when it survives terminate/kill (an uninterruptible CUDA
syscall can outlive SIGKILL). The pre-swap liveness guard now trusts that
result, so the destructive .venv_t5_latest rename cannot proceed while a
live worker still holds sidecar modules.
Recover a sidecar stranded at .old when a swap's activation rename and its
rollback both fail: reading the pin restores it when no swap holds the
reservation, so latest-tier models are not permanently broken.
Resolve the latest tier in the parent for export loads and for explicitly
16-bit training runs, not only 4-bit ones: tier resolution self-heals an
incomplete sidecar, and repairs are parent-only, so those paths could not
recover before. Sidecar integrity and quantization are independent.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Revert the parent-side latest-tier repair probe on training and export loads
The probe ran before the route freed VRAM, so a resident chat or export worker
made _workers_active_for_repair() refuse the parent-only repair; the route then
tore that worker down and spawned a child that also cannot repair, so an
incomplete sidecar still failed to load. Repairing correctly requires running the
repair between the worker teardown and the child spawn, decoupled from VRAM
sizing, which is a larger change tracked separately. Restore the prior behavior
so these paths match the reviewed form and do not partially attempt a repair that
cannot complete while workers are resident.
* Honor failed worker shutdowns on load and revalidate the cached latest mapping
The fresh-load paths spawned a new worker straight after _shutdown_subprocess
without checking its result, so a worker that outlived terminate/kill (a wedged
CUDA syscall) had its handle overwritten by the replacement while it still held
GPU memory, and is_worker_alive/the pre-swap guard could no longer see it. Both
the inference load and the export checkpoint load now abort when the old worker
did not exit, so the load can be retried once it does.
_config_model_types returned a cached latest mapping without re-checking the
sidecar, so a sidecar deleted or broken in-process after its first parse was
never re-validated: routing kept sending latest-only models to the stale latest
tier while activation failed. The cached latest mapping is now dropped and
re-resolved (self-healing) when the sidecar is no longer intact.
* Drop cached latest mapping when the pin is gone; keep 4-bit for custom-code fallback
_latest_sidecar_intact now returns False when the pin marker itself is gone, not
just when a pinned package is missing. Otherwise a cached latest mapping outlived
a deleted pin: _config_model_types kept returning it, so routing sent latest-only
models to a tier whose worker activation then failed (no pinned version) until
restart. It now drops the cache and re-resolves to no latest tier. The
_overlay_transformers_dir caller already gates on a present pin, so it is
unaffected.
validate_model forced 16-bit sizing whenever a PyPI upgrade was merely offered,
even for a model that can fall back to its own auto_map code. /load loads such a
model 4-bit without the install, and the install route refuses while training is
active, so 16-bit sizing here returned a VRAM 409 for the only viable 4-bit path.
The offered-upgrade flip is now gated on the absence of a custom-code fallback;
an already-active latest sidecar still always sizes 16-bit.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix offline checkpoint load/export failing with "tokenizer is weirdly not loaded"
Loading a fine-tuned checkpoint with no internet (e.g. a Studio export) crashed
with "Unsloth: The tokenizer is weirdly not loaded? Please check if there is one."
For a LoRA adapter the loader reassigns model_name to the base model repo id and
only keeps the local checkpoint dir as tokenizer_name when it contains
tokenizer_config.json, tokenizer.json AND special_tokens_map.json. Modern
tokenizers (e.g. Gemma) store special tokens inside tokenizer_config.json and
omit special_tokens_map.json, so tokenizer_name fell back to the base repo id.
The tokenizer/processor loads in vision.py then hit the Hub with no
local_files_only, so with no network they failed (AutoProcessor) or hung for
minutes (AutoTokenizer) even though every file was already cached.
loader.py: keep the local checkpoint dir as tokenizer_name when it has a
tokenizer config plus the actual tokenizer files (tokenizer.json / tokenizer.model
/ vocab files); special_tokens_map.json is no longer required.
vision.py: compute an effective local_files_only (explicit kwarg plus the
HF_HUB_OFFLINE / TRANSFORMERS_OFFLINE env vars, mirroring loader.py and
diffusion.py) and thread it through every AutoConfig, AutoProcessor,
AutoTokenizer and the manual VLM processor fallback, including the
hf_hub_download in that fallback (which now prefers a local file). When a load
fails and no offline env var is set, retry against the local cache. The retry
forces HF offline mode because local_files_only alone does not stop
AutoProcessor / AutoTokenizer from issuing a /api/models request during class
resolution. The final error now explains the offline/cache cause instead of the
misleading "weirdly not loaded" message.
studio export: probe Hub reachability once per checkpoint load and pass
local_files_only when offline so exports use the local checkpoint dir / cache
instead of hanging or crashing with no internet.
Online behavior is unchanged: the new flags default to off and the retry only
runs after a network related failure.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: safer offline forcing, cached fallback config, proxy-aware probe
Follow-up to the offline checkpoint load fix, addressing review feedback:
- vision.py: only flip the process-wide HF offline flag when offline is actually
requested (local_files_only / env) or after a real network failure, never
pre-emptively while we might be online. The flip is now guarded by a lock +
depth counter so nested or concurrent windows restore the flag correctly
(no stale value).
- vision.py: guard the get_auto_processor fallback so a network error there
returns None and the local-cache retry still runs instead of escaping.
- vision.py: in the manual VLM processor fallback, read tokenizer_config.json
via hf_hub_download(..., local_files_only=...) so a cached repo-id config is
still resolved offline and the model-specific image/video tokens are restored.
- studio export: make the reachability probe proxy aware (probe the configured
HTTP(S) proxy egress, honour NO_PROXY, use the endpoint port) so a proxy-only
setup is not wrongly marked offline; allow UNSLOTH_OFFLINE_PROBE=0 to disable.
- studio export: run the audio/vision type-detection probes inside the
forced-offline window when offline, so their config/tokenizer reads hit the
local cache instead of waiting out connection timeouts.
Online behavior remains unchanged.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: gate offline retry, safer tokenizer_name pop, skip audio net probe offline
- vision.py: only force the process-wide HF offline flag on the tokenizer
retry when offline was requested or the captured primary error is actually
network related, so a permanent tokenizer error no longer toggles global
offline mode for other concurrent loads.
- loader.py: always pop tokenizer_name out of kwargs and let a caller-supplied
value win, avoiding a "multiple values for keyword argument 'tokenizer_name'"
TypeError when it is also passed explicitly downstream.
- model_config.py / export.py: add local_files_only to detect_audio_type so the
raw requests.get tokenizer_config fetch is skipped offline (it ignores the HF
offline flag), and pass it from the export probe.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: classify LocalEntryNotFoundError as offline-related
huggingface_hub's LocalEntryNotFoundError subclasses FileNotFoundError, so the
"not isinstance(cur, FileNotFoundError)" guard in _is_offline_related_error was
swallowing it and it could never be recognised as offline, despite being listed
in the network error types. It means "not in cache and the Hub is unreachable",
which is genuinely offline. Capture the class into an isinstance-checkable tuple
(empty, hence a no-op, if the import is unavailable) and exclude it from the
FileNotFoundError guard, so a real offline failure now triggers the local-cache
retry while a plain missing-file error still propagates.
* Address review: require merges.txt for BPE, status-gate HTTP errors, isolate local-only audio cache
- loader.py: a local dir with vocab.json but no merges.txt (and no tokenizer.json)
is not a loadable BPE tokenizer, so do not treat it as self-sufficient; require
merges.txt alongside vocab.json in both gate blocks, otherwise fall back to the
base model tokenizer as before.
- vision.py: _is_offline_related_error no longer buckets every HfHubHTTPError /
requests HTTPError as offline. HTTP errors are judged by status code: only a
transient 5xx triggers the forced local-cache retry, while 401/403 (auth/gated)
and 404 (missing) propagate as the real error instead of being masked. Hard
signals (connection/timeout/OfflineModeIsEnabled/LocalEntryNotFoundError) still
classify as offline.
- model_config.py: include local_files_only in the audio-detection cache key so a
local-only (offline) negative result cannot be reused by a later online probe,
which would otherwise route an audio model through the text loader until restart.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address re-review: fix studio test stubs, force offline env in probe window, drop redundant retry
- studio/backend/tests/test_vision_cache.py: the three _detect_audio_from_tokenizer
stubs were called with the new local_files_only kwarg and raised TypeError, failing
Backend CI. Add local_files_only to the stub signatures and add a test that a
local-only negative does not poison a later online audio probe.
- export.py: the type-detection probe window now also sets HF_HUB_OFFLINE /
TRANSFORMERS_OFFLINE env vars (saved/restored), not just the in-process flag.
transformers_version._load_config_json / _check_tokenizer_config_needs_v5 gate
their urllib fetches on the env vars, and is_vision_model may spawn a subprocess
that inherits os.environ but not the in-process flag; without the env vars a
probe-detected offline export could still block on a network timeout.
- vision.py: only retry the processor load when the first attempt was online and
failed with a network error. When local_files_only was already requested the first
attempt was forced offline, so the previous retry just repeated identical failing
work before the last-resort path.
- model_config.py: correct the _audio_detection_cache type annotation to the 3-tuple
key (name, token_fingerprint, local_files_only).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: thread-safe probe-offline env window, clear error for local dir without config
- export.py: guard the HF_HUB_OFFLINE / TRANSFORMERS_OFFLINE mutation in
_force_offline_probe_window with a lock + depth counter (mirrors _force_hf_offline),
so concurrent / nested export probes only flip on first entry and restore on last
exit. This prevents overlapping export requests from permanently poisoning those
env vars or restoring a stale value.
- vision.py: in the VLM processor fallback, when tokenizer_name is a local directory,
read its tokenizer_config.json directly and raise a clear FileNotFoundError if it is
absent, instead of handing the local path to hf_hub_download (which would treat it as
a repo id and raise a confusing HFValidationError / RepositoryNotFoundError).
hf_hub_download is now only used for actual repo ids.
* Address review: classify raw socket.gaierror DNS failures as offline
Add the platform-specific getaddrinfo / DNS-resolution wording to the offline
detection list in _is_offline_related_error so a bare socket.gaierror (an OSError
subclass) is recovered from the local cache: "Name or service not known" and
"Temporary failure in name resolution" (Linux) and "nodename nor servname
provided" (macOS). Genuine non-network OSErrors (disk full, permission denied)
and plain FileNotFoundError still propagate.
* Address review: retry degraded VLM offline, force offline for text export + patch-tokenizer fallback
- vision.py: a degraded VLM processor (text-only, no image_processor) whose manual
fallback fails offline used to be kept, so image inputs broke even with cached
files. _construct_vlm_processor_fallback now returns its failure error;
_acquire_processor surfaces it, and the caller retries forced-offline when the
result is None OR a degraded VLM and the failure was network related, keeping the
original result if the retry is not strictly better (never regress). The retry is
still gated on an online first attempt + offline-related error so a permanent
error never flips the global offline flag.
- vision.py: wrap the patch_tokenizer except-branch AutoTokenizer.from_pretrained in
the same forced-offline-on-network-error pattern as the primary / last-resort
loads, so an offline export where patch_tokenizer raises does not hang or fail.
- export.py: force HF offline around the two FastLanguageModel loads (text and SNAC)
when the probe detected offline. Their text tokenizer path (load_correct_tokenizer
-> AutoTokenizer) does not forward local_files_only, so without this a text export
could still contact the Hub. Added a small _offline_window_if helper reused by the
probe and load windows.
* Consolidate offline loading into one entry-point decision
Decide offline once per entry point instead of at every HF call site. The
prior approach threaded local_files_only into ~15 scattered config / tokenizer
/ processor / weight loads, each wrapped in its own try-online, classify-error,
retry-forced-offline dance, which is what kept surfacing "another call site you
missed", "another error shape misclassified", and global-flag thread-safety in
review.
FastLanguageModel / FastModel / FastBaseModel.from_pretrained now share an
@_offline_aware_load decorator: when offline (explicit local_files_only kwarg or
HF_HUB_OFFLINE / TRANSFORMERS_OFFLINE env) it sets local_files_only and runs the
whole load inside one _force_hf_offline() window so every nested HF call inherits
it; when online it runs normally and, only if the load fails with a genuinely
network-related error, retries once forced-offline. The online path is unchanged
(no probe added) and 401 / 403 / 404 / permanent errors still propagate.
Centralise the offline helpers in loader_utils.py as the single source of truth
(shared by loader.py, re-exported from vision.py, and reused by the Studio
exporter):
- _force_hf_offline now sets the HF_HUB_OFFLINE / TRANSFORMERS_OFFLINE env vars
AND the in-process huggingface_hub / transformers flags, refcounted under one
lock so nested / concurrent windows restore correctly. Setting the env vars
covers env-gated urllib probes and spawned subprocesses too.
- _get_effective_local_files_only, _is_offline_related_error (unchanged
classifier, retains the 5xx-vs-4xx, LocalEntryNotFound and gaierror handling),
_offline_aware_load, and _resolve_checkpoint_tokenizer_name.
loader.py: wrap both entry points; drop the two duplicated env-var fallback
blocks and the two byte-identical local-tokenizer-gate blocks (now
_resolve_checkpoint_tokenizer_name).
vision.py: drop the per-site force_offline params and the three retry gates
(processor, patch_tokenizer fallback, last-resort). They now just surface the
underlying error so the single entry-point safety net retries forced-offline. A
network fallback error now takes precedence over a permanent primary error so the
offline retry still fires when the manual VLM fallback needs cached repo files.
studio/backend export.py: reuse the unified core _force_hf_offline (env + flags)
and drop the duplicate probe-window primitive; the snac / text branches no longer
need their own window. model_config.py: also gate the raw requests.get audio
fallback on the HF offline env vars so it is covered even without the kwarg.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address 10-reviewer P1 findings: vision cache split, PEFT offline, retry OOM
Split the Studio vision-detection cache by local_files_only, mirroring the audio
cache fix. is_vision_model / _is_vision_model_uncached / _raw_config_has_vision_config
/ load_model_config now thread local_files_only, the cache key includes it, and the
exporter passes it. Offline detection also skips the transformers-5 network
subprocess and stays on the local cache, so an offline negative can no longer be
keyed under the online entry and poison a later online probe. Adds a regression
test mirroring the audio poison test.
Forward local_files_only to both PeftModel.from_pretrained adapter-attach sites in
loader.py so a cached remote LoRA adapter resolves from the local cache under
explicit local-only / offline loads (defence-in-depth alongside the forced-offline
window).
_offline_aware_load: run the forced-offline retry OUTSIDE the except block and
collect + empty the device cache first. An except-scoped exception keeps its
__traceback__, which pins the failed attempt's frame locals (a partially loaded
model) until the block exits; loading the model again while that copy is still
alive could OOM a large VLM. Letting the except block close drops the traceback so
the partial load is freed before the retry reallocates.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address Codex review: env-offline cache key + rebuild HF sessions in offline window
Key the Studio audio and vision detection caches on the EFFECTIVE offline state
(local_files_only OR the HF offline env vars), not just the kwarg. detect_audio_type
and is_vision_model both skip the remote fetch / network subprocess when
HF_HUB_OFFLINE / TRANSFORMERS_OFFLINE is set even with the default
local_files_only=False, so the result reflects offline; storing it under the online
(False) key let an env-offline negative poison a later online lookup once the env var
was cleared. Both now compute effective_offline once and use it for the cache key and
the downstream call. Adds a regression test for the env-offline dimension.
_force_hf_offline now rebuilds huggingface_hub's cached sessions on enter and exit
(best-effort _reset_hf_sessions). On hub 0.x the offline adapter is baked into the
per-thread requests.Session at creation, so flipping the constant alone leaves an
already-cached online session able to hit the network inside the window (and an
offline one stuck offline after restore); resetting forces the next get_session() to
match the current flag. On hub 1.x offline is checked dynamically per request, so
reset_sessions does not exist and the helper is a safe no-op.
The third review point (release the failed load before retrying) was already fixed in
af0f58a: the forced-offline retry now runs outside the except block and frees the
device cache first, so the failed attempt's traceback-pinned partial model is
released before the retry reallocates.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Align Studio _env_offline parsing with the canonical offline helper
model_config._env_offline gates the raw requests.get tokenizer-config fallback in
detect_audio_type and the audio/vision detection cache keys, but it only accepted
unstripped "1"/"true"/"yes". unsloth's offline helpers (loader_utils._env_says_offline
and the from_pretrained env fallback) accept the canonical set {1,true,yes,on} after
strip + lowercase, so HF_HUB_OFFLINE=on or HF_HUB_OFFLINE=" 1 " was treated as offline
by the loaders but online here, leaving the raw network fetch reachable while
"offline". Use the same strip + lowercase {1,true,yes,on} set. Adds parsing tests.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix lint: drop dead offline-helper re-exports from vision.py
The import-hoist verifier (scripts/verify_import_hoist.py) flagged vision.py's
re-export block as HOISTED-IMPORT-UNUSED blockers: it imported eight offline
helpers from loader_utils but only used three internally
(_get_effective_local_files_only, _is_offline_related_error, _offline_aware_load).
The other five were imported purely to preserve `from unsloth.models.vision import
X`, but nothing imports four of them from vision, and loader.py already imports
_resolve_checkpoint_tokenizer_name straight from loader_utils.
Import only the three names vision.py actually uses, and point the Studio exporter
at the canonical source (from unsloth.models.loader_utils import _force_hf_offline)
instead of re-exporting it through vision. loader_utils stays the single source of
truth; no behaviour change.
* Address Opus review: chain probe errors, unify env-offline, status-less HTTP
Chain the original AutoConfig/PeftConfig probe exception into the combined
RuntimeError in both FastLanguageModel.from_pretrained and FastModel.from_pretrained
(`raise RuntimeError(combined_error) from (autoconfig_exc or peft_exc)`). The probes
caught every Exception and stringified it, so the re-raised RuntimeError had no
__cause__/__context__ and _is_offline_related_error could not classify it -- the
network-down-but-cached auto-retry never fired for these entry points. With the
cause chained, the decorator sees a ConnectionError/LocalEntryNotFoundError/5xx and
retries forced-offline from cache; a permanent cause (404 / bad config) is still not
offline-classified and propagates without a wasted retry.
Unify the third offline-env parser: studio/backend/utils/transformers_version._env_offline
now uses the canonical {1,true,yes,on} + strip + lowercase set (matching
loader_utils._env_says_offline and model_config._env_offline), so HF_HUB_OFFLINE=on
or " 1 " no longer leaks the direct urllib metadata fetches to the network.
_is_offline_related_error: a status-less HTTP error (no response / unparseable code)
now falls back to the network-wording check instead of being dropped, so a transient
HTTP failure with clear "couldn't connect" wording is treated as offline. HTTP errors
with a real status code still decide by code (4xx propagates, 5xx is offline).
* Condense offline-loading code comments, drop dead helper, dedupe import for PR #6554
* Add unit tests for offline-loading helpers for PR #6554
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Guard load cleanup with try/finally and add retry-contract tests for PR #6554
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add gc.collect retry-step test for PR #6554
* Tighten offline-loading comments and docstrings for PR #6554
* Raise the both-config-failed error before model-type lookup so offline retry fires for PR #6554
* Prefer offline cause for retry and bound export reachability probe for PR #6554
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Skip remote mapper while offline, harden text-load cleanup, and stop stacked offline retries for PR #6554
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Surface VLM fallback offline errors, probe offline before export version activation, and restore progress bars across retries for PR #6554
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Restore offline env after export version activation so the persistent worker re-decides per load for PR #6554
* Classify socket.gaierror and urllib URLError as offline by type for PR #6554
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Probe offline around export load preflights and never offline-retry TLS failures for PR #6554
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Force in-process offline for export preflights, verify proxy egress in probe, and skip caching offline version negatives for PR #6554
* Snapshot offline constants before forcing env and require local processor files for VLM checkpoints for PR #6554
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Resolve the transformers tier by probing AutoConfig instead of guessing
When the only signal is a 5.x tokenizer class, get_transformers_tier guessed the
lowest 5.x sidecar (530). That misroutes models whose built-in config parser needs
a higher tier: dense NemotronH ships a 5.x tokenizer but its '-' (MLP) layer only
transformers 5.10 can parse, so 5.3/5.5 raise KeyError '-'. The config.json
transformers_version field records the saving version, not the minimum to load, so
it cannot drive routing either.
Replace the weak tokenizer->530 guesses (local and remote) with a probe: parse
config.json with the built-in parser (trust_remote_code=False) in each sidecar,
escalating 530->550->510, and pick the first that succeeds. This generalizes to any
architecture without hardcoded lists. Strong signals stay fast paths (no subprocess);
the probe runs only when the tier is otherwise ambiguous and is cached by (model,
commit sha). It never executes repo code, never downloads weights, never raises, and
falls back to the legacy 530 guess on a transient/auth/offline failure or when no
sidecar is available. UNSLOTH_DISABLE_TIER_PROBE restores the old behavior.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: tier probe fallbacks and cross-platform robustness
Codex:
- Never escalate to 510 on uncertainty. When every sidecar was probed and none
parsed with the built-in parser, the model is a remote-code / custom model_type
that loads via its own code; keep the legacy 530 route instead of jumping to
510 (which would change the behavior of models that worked on the 5.3 stack).
- Only cache the 530 fallback when the result is conclusive (every tier actually
probed). If a sidecar was missing/uninstallable the environment is incomplete,
so return 530 uncached and retry on the next call.
- Do not pin the tier cache under an unknown revision: _resolve_commit_sha no
longer memoizes a None sha (a transient Hub failure is retried), and _probe_tier
only caches a tier when the commit sha is known.
Gemini:
- Wrap Path.exists() in the sha resolver in try/except OSError (a remote repo id
can raise WinError 123 on Windows).
- Probe script writes the error to sys.stderr.buffer as UTF-8 bytes so a non-ASCII
message cannot itself raise UnicodeEncodeError under cp1252.
- subprocess.run decodes stderr with errors="replace" to avoid UnicodeDecodeError
on non-UTF-8 consoles.
Tests: 72 passed (added partial-sidecar uncached, sha-unresolved not cached,
all-failed stays 530 + cached, sha resolver retries None / handles OSError).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review round 2: authenticate tier checks, stop memoizing local sigs
Codex:
- Thread hf_token through _check_config_needs_510/550 and
_check_tokenizer_config_needs_v5 (and the underlying raw fetches). Previously a
gated/private model whose only 5.x signal is tokenizer_config.json never reached
the authenticated probe: the unauthenticated raw fetch failed and cached False,
so the model fell through to the default 4.x tier. The per-check caches are now
keyed by (model, token) so an unauthenticated miss cannot poison a later authed
read, mirroring _load_config_json.
- _resolve_commit_sha no longer memoizes a local directory signature. A local
signature is mutable (size/mtime of config/tokenizer), so a reused/overwritten
checkpoint path would otherwise keep selecting the previous tier; it is now
recomputed every call. Only the immutable remote commit sha is memoized.
Tests: 75 passed (added token-cache isolation + auth header, local signature not
memoized, token threaded into all checks/probe).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review round 3: reach activation with the token, drop SHA tier cache
Codex round 3:
- Thread hf_token into the activation path that actually selects a sidecar. The
token-aware tier checks added last round were unreachable:
activate_transformers_for_subprocess called get_transformers_tier without a
token, and the inference/training/export workers passed only the model name even
though they hold a request-scoped hf_token. activate_transformers_for_subprocess
now takes hf_token and the three workers forward config["hf_token"], so a
gated/private model whose only 5.x signal is an authenticated config/tokenizer is
routed to the right sidecar instead of falling to default 4.x.
- Stop importing huggingface_hub during tier detection. _probe_tier no longer
resolves a commit sha, so it never pulls huggingface_hub into the worker before
the sidecar venv is prepended to sys.path (activation only prepends, never
purges), which would otherwise pin the default-env hub over the sidecar's
pinned huggingface_hub==1.8.0.
- The tier cache is now keyed by model_name for the process lifetime (a model's
required tier is a property of its architecture; cleared on restart). This drops
the mutable-SHA memo that masked remote revision changes and the mutable
local-signature memo, removing _resolve_commit_sha / _local_dir_signature /
_probe_sha_cache entirely.
- Do not cache a probe success that depended on a skipped lower tier: if a lower
sidecar was unavailable, the lowest valid tier may change once it installs, so
the result is returned uncached and re-probed next call.
Tests: 73 passed (probe imports no hub; success uncached when a lower tier is
skipped; activation forwards the token).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Trim comments to be more succinct
* Re-probe overwritten local checkpoints and authenticate the probe child
The AutoConfig tier probe cached its result under the bare model_name, so a
local checkpoint overwritten in place (same path, new config.json) kept serving
the stale sidecar. Fold a cheap config.json signature (size + mtime) into the
cache key for local paths; remote ids stay name-keyed so no huggingface_hub
import lands before the sidecar is activated.
The probe relies on the implicit HF_TOKEN env, so an inherited
HF_HUB_DISABLE_IMPLICIT_TOKEN=1 left it unauthenticated and a gated repo 401ed
into the 530 fail-safe. Clear that flag in the child env when a token is set.
* Keep tier probes off the log-only path and probe new 5.x archs default-first
- get_transformers_tier gains probe=True/False. needs_transformers_5 (a coarse
4-vs-5 boolean used only for a spawn log and a vision-check branch) now passes
probe=False, so a parent/log-only caller never spawns sidecar probes. The real
activation path keeps probe=True and resolves the exact tier in the worker.
- A config.json saved by transformers 5.x but matched by no fast path is now probed
default-first: _probe_tier gains include_default + floor, prepending the ambient
4.57.x tier to the escalation. A model that still parses on the default is left on
it (no mis-route onto a sidecar); only a config the default parser cannot read
escalates to the lowest 5.x tier that parses. The transformers_version field is a
cheap 'worth probing' hint only, read from the already-fetched config (no extra
network); ordinary 4.x configs never probe.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Separate probe cache by mode and keep version-field 5.x visible to needs_transformers_5
- _probe_tier cache was keyed only by config.json signature, so a default-first probe
that returned 'default' could be handed back to a later tokenizer/known-5.x caller
(floor=530), leaving a model with a 5.x-only tokenizer on transformers 4.x. Key the
cache by probe mode (floor + include_default); the legacy 530 mode keeps the bare key.
- The version-field 5.x detection is a cheap config read, not a probe, so run it even
when probe=False: a standard-tokenizer model whose only signal is transformers_version
>= 5 now classifies as 5.x via needs_transformers_5 (returns '530' without spawning a
probe), so the vision-routing fallback uses the 5.x subprocess instead of failing the
default parser and marking it non-vision. The real activation path still probes
default-first and may resolve 'default'.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Don't treat local checkpoints as Hub ids, and fix stale activation test double
- _load_config_json / _check_tokenizer_config_needs_v5: a local checkpoint dir whose
config.json / tokenizer_config.json is not yet present was being fetched from the Hub
as if the path were a repo id, and the 404 miss was cached. A later call after the
file is written (in-progress checkpoint) then served the stale miss, so a
TokenizersBackend checkpoint fell through to the default tier. Skip the Hub fetch for
local dirs and do not cache the miss, so the file is read once it appears.
- test_activate_transformers_version_or_warn_*: the worker now threads hf_token into
_activate_transformers_version (model_name, hf_token); update the one-arg test doubles
to the real two-arg signature so the silent-success path stays silent.
* Tighten comments in the AutoConfig probe and tier-selection paths
* Address review: canonical probe cache key and reuse _token_cache_key
- _probe_cache_key resolves config.json to its absolute realpath before
keying, so a relative path or a changed cwd can't collide with or miss a
prior probe result. Remote ids still fall back to the name (stat raises,
caught).
- _cached_config_json reuses _token_cache_key instead of re-hashing the
token inline, keeping the (model, token) key derivation in one place.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: detect transformers 5.3.0 tier from config.json for local checkpoints
A local safetensors folder whose config.json did not match the Gemma4 (510/550)
architecture signals short-circuited get_transformers_tier() to "default"
(transformers 4.57.x), never reaching the name-substring check that routes
Qwen3.5 to the 5.3.0 sidecar. So a local Qwen3.5 checkpoint (model_type
"qwen3_5", needs transformers >= 5.2.0) loaded with 4.57.x and failed with
"does not support Qwen3.5". The same model as a remote HF id worked, because it
has no local config.json to trigger the short-circuit.
Detect the 5.3.0 tier from config.json (model_type "qwen3_5" / architecture
Qwen3_5ForCausalLM) in the local-config branch, mirroring the existing Gemma4
510/550 handling. This is a positive config signal, so it fixes local Qwen3.5
without weakening the directory-name false-positive guard (a llama checkpoint
under a "gemma-4-12b-*" parent still resolves to default).
Adds tests for the config-based 530 detection and local-folder tier resolution.
* Studio: suppress false warning when config.json parse fails for sidecar-tier models
* Studio: generalize local-checkpoint tier detection for all 5.3.0 families
Expands the config.json-based tier detection to cover all known 5.3.0-tier
model families (Qwen3 MoE, GLM-4.7-Flash, LFM2.5-VL) and adds a _name_or_path
fallback so renamed local checkpoints with unrecognised model_type values still
route correctly via the HF ID embedded in their config.json.
- Expand _TRANSFORMERS_530_ARCHITECTURES / _MODEL_TYPES with verified entries
from Qwen3MoeForCausalLM, Glm4MoeLiteForCausalLM, Lfm2VlForConditionalGeneration,
and Qwen3_5ForConditionalGeneration (confirmed from local Qwen3.5-2B config.json)
- Extract _tier_from_name() helper, deduplicating the fast-substring logic used
by both the remote-path branch and the new config _name_or_path fallback
- In the local-config branch: after architecture checks, resolve the tier from
cfg._name_or_path / cfg.model_name before returning "default", preserving the
existing directory-name false-positive guard
- 79 tests passing
* Studio: match 510/550 style for 530 config sets (no inline comments)
* Studio: use _resolve_base_model instead of reinlining _name_or_path lookup
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: recurse into get_transformers_tier for resolved base model (Gemini suggestion)
* Studio: use _tier_from_name in local-config fallback to avoid network probes
Using get_transformers_tier(resolved) on the _name_or_path fallback would
trigger up to 3 network fetches (config.json + tokenizer_config.json, 10s
each) for every ordinary checkpoint whose _name_or_path is a plain HF ID
like meta-llama/Llama-3-8B. The fallback's purpose is name-based detection
on the resolved HF ID, _tier_from_name covers all known cases without I/O.
* Studio: add _check_config_needs_530 to slow HF-ID fallback path
Private or renamed HF repos whose model IDs lack a 5.3 substring were
silently routed to the default tier. _check_config_needs_530 mirrors the
existing 510/550 pattern: fetches config.json once, caches the result, and
is called after the 550 check in the slow path. Includes 5 unit tests.
* Studio: guard _tier_from_name fallback against local-path false positives
When _name_or_path in config.json is an absolute path to the same checkpoint
passed as a relative path, the textual resolved != model_name check passes
and _tier_from_name would scan the directory path for substrings. Split the
fallback: local directories recurse into get_transformers_tier (config check,
no network I/O); HF Hub IDs use _tier_from_name (name-based, no network).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: separator-norm aliases, model_name/_name_or_path fallback, tests
- _norm_separators(): collapse _ . whitespace to - so underscore/dot model
ID variants (Qwen3_5, Qwen3_Next) match the canonical substring list
- _tier_from_name(): apply norm to both name and each substring so aliases
resolve without duplicating the substring lists
- _resolve_base_model(): try model_name then _name_or_path separately so a
self-referential Unsloth model_name doesn't hide the useful HF ID in
_name_or_path
- Gate get_base_model_from_lora on adapter_cfg_path.is_file() to avoid
eagerly importing transformers before the sidecar venv is on sys.path
- 17 new tests covering _norm_separators, separator-insensitive
_tier_from_name, and the model_name/_name_or_path fallback
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: only pre-resolve LoRA adapters in activation callers
activate_transformers_for_subprocess and ensure_transformers_version were
pre-resolving all local checkpoints via _resolve_base_model before calling
get_transformers_tier. After the model_name/_name_or_path fix, a full
checkpoint with a private/offline _name_or_path and no tier substring would
resolve to that HF ID, which can't be probed, bypassing the local config.json
model_type check entirely. Gate pre-resolution on adapter_config.json so full
checkpoints go straight to get_transformers_tier, which reads config.json
directly. LoRA adapters still pre-resolve as before.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: fix Qwen3.5 MoE/Qwen3.6 tier detection and dot-version false positives
- Add Qwen3.5 MoE (qwen3_5_moe / Qwen3_5MoeForConditionalGeneration) and
Qwen3-Next to the 5.3.0 config sets, so renamed local checkpoints route to
the sidecar instead of default transformers
- Let a 510/550 name match override a 530 config match, so Qwen3.6 (which
reuses qwen3_5 / qwen3_5_moe config ids) still routes to the 5.5.0 sidecar
- Stop normalizing version dots to hyphens so size names like Qwen3-5B and
Qwen3-6B are not promoted to a 5.x sidecar; underscore aliases still match
- Skip name matching for resolved values that look like stale local paths
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: close remaining codex P2s: adapter-only LoRA + 530-override path-hint guard
- adapter_model-only LoRA: add import-light _is_lora_adapter_dir/_has_adapter_weights
and gate activation/export pre-resolve on them, so LoRA dirs with
adapter_model*.safetensors but no adapter_config.json still resolve to their base
model (via _resolve_base_model's new unsloth_<model>_<ts> directory-name parse)
instead of tiering off the adapter folder.
- 530 override: only treat a resolved value as a name hint when it is a real Hub id;
a stale/renamed local path in model_name/_name_or_path can no longer flip a correct
530 config to 550. Current folder basename still allowed.
Added 7 regression tests; suite at 116 passing.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: address review feedback on tier detection
- Add Qwen3.5 text-tower model types (qwen3_5_text / qwen3_5_moe_text) to the
5.3.0 config set so text-only configs with stripped architectures still route
to the sidecar
- Apply the Qwen3.6 name override on the remote slow path too, so a renamed or
private repo whose config reuses qwen3_5 ids but names Qwen3.6 in
_name_or_path selects 5.5.0 instead of 5.3.0
- Treat an existing local path (or empty value) as a path, not a Hub id, in
_looks_like_hf_id so a real local checkpoint folder is not name matched
- Guard _resolve_base_model against non-string config values and compare paths
by realpath so relative or absolute self references resolve correctly
- Keep the LoRA adapter is_file check inside the OSError guard
* Studio: harden tier detection against malformed configs and bad paths
- _config_matches_tier no longer raises TypeError when a malformed config.json
carries a non-string model_type (e.g. a list) or non-list architectures; it
fails open to no-match
- guard the model_name-derived is_file/is_dir probes with _safe_is_file /
_safe_is_dir so a pathological or over-long path (e.g. a Windows long path)
fails open to the default tier instead of raising OSError
No routing changes for any valid model; purely defensive. Verified by a
cross-platform simulation (POSIX + NT path semantics) and a before/after tier
matrix that is unchanged for all previously supported models.
* Studio: trim verbose comments in tier detection
Shorten/remove over-long comments and docstrings, mainly on internal helpers,
without changing behavior. Verified code-only via comment_tools.py check; suite
unchanged at 128 passing.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Auto-install SSM kernels (causal-conv1d, mamba-ssm) for inference loads
Mamba/SSM hybrids (Nemotron-H/Nano, Falcon-H1, Granite-4.0-H, ...) lazily import
mamba_ssm / causal_conv1d during from_pretrained, so loading them for chat failed
with 'mamba-ssm is required by the Mamba model but cannot be imported'. The training
worker already wheel-first installs these before a fine-tune; the inference worker
did not. Add utils/ssm_runtime.ensure_ssm_runtime and call it from the inference load
path so the same models load for inference. Training worker is untouched; a drift
test keeps the shared detection and pinned versions in lockstep.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* ssm_runtime: invalidate import caches, skip MLX, cover LoRA base
- Invalidate importlib finder caches in _is_importable and after a successful
wheel install, so a kernel installed earlier in this same process is actually
importable when the modeling code lazy-imports it during from_pretrained.
- Skip the SSM kernel install entirely on the MLX (Apple Silicon) load path:
these are CUDA/ROCm Torch kernels with no MLX use and no macOS prebuilt wheel,
so the source build would fail before the MLX backend loads the model.
- For LoRA loads, also run detection over the resolved base model, since an
adapter id like 'me/my-lora' won't match the SSM heuristics but its SSM base
(Nemotron-H, ...) is what needs the kernels.
Adds tests for cache invalidation and the MLX-skip / LoRA-base worker wiring.
* Tighten SSM autoinstall comments
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* ssm_runtime: verify wheel imports, HIP-aware source build, build heartbeat
Address review feedback:
- Verify a prebuilt wheel actually imports before trusting it; a CUDA/ABI-mismatched
wheel now falls back to a source build instead of returning success and failing later
with the cryptic lazy-import error.
- HIP-aware source build: require hipcc on ROCm, inject clang --gcc-install-dir, and use
the 1800s timeout, mirroring the training worker (ROCm has no prebuilt wheel).
- Emit a status heartbeat every 60s during the source build so a long (ROCm) build does
not trip the orchestrator's 300s inactivity timeout.
Tests cover the wheel-not-importable fallback and the missing-hipcc ROCm bail.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Make causal-conv1d best-effort and harden the SSM source build
- causal-conv1d is a fast path: models that merely want it (Qwen3-Next, LFM2)
fall back to torch, so a failed install must not reject an otherwise loadable
chat model on Windows/CPU/macOS or an ABI without a wheel. Only a true SSM
model's mamba-ssm requirement stays fatal, matching the training worker which
treats causal-conv1d as best-effort.
- The source build is reached only when not importable, including a wheel that
installed but failed to import; add --reinstall/--force-reinstall so it
replaces the broken install instead of no-opping as already satisfied.
- Add --no-cache to the ROCm uv source build to avoid reusing stale artifacts
from a partial HIP build, mirroring the training worker.
* Address review: install SSM kernels before transformers, harden import + Windows
Codex:
- Install the SSM kernels before importing transformers. run_inference_process
imported core.inference.inference (which imports unsloth/transformers) before the
load, and a sidecar transformers can evaluate its optional-backend gates against
the import state; installing causal_conv1d/mamba_ssm afterwards left those gates
unsatisfied and a Nemotron/Falcon/Granite load still failed with "mamba-ssm is
required". The initial model's kernels are now installed in run_inference_process
before the ML import, via a shared _ensure_ssm_kernels helper; _handle_load keeps
calling it (idempotent) for a LoRA's base and for later in-process loads.
- _is_importable now treats any import failure as "not importable", not only
ImportError. An ABI-incompatible native kernel (undefined symbol after a torch/CUDA
upgrade) raises OSError/RuntimeError; letting those escape reported
ssm_runtime_install_failed instead of falling back to reinstall/source build.
- Skip causal-conv1d on Windows (no prebuilt wheel), mirroring the training worker.
A causal-conv1d-only model (Qwen3-Next/LFM2) no longer drops a chat load into a
multi-minute untimed source build; it uses the torch fallback. mamba-ssm is still
attempted for true SSM hybrids.
Tests: test_ssm_runtime.py +5 (broken-kernel exceptions read as not-importable;
causal-conv1d skipped on win32 while mamba-ssm still installs). 36 passed.
* Trim comments to be more succinct
* Run security gates before installing SSM kernels
The SSM kernel auto-install is name-based (model_is_ssm is a substring match, no
config fetch), so a model id merely containing an SSM substring triggered a
native-package install (possibly a slow source build) before the malware and
remote-code consent gates ran. Extract those gates into _run_security_gates and
call it before the kernel install in both the pre-import path of
run_inference_process and in _handle_load, so a blocked or nonexistent model is
refused before any build. The gates are metadata-only and do not import
transformers, so they are safe to run before the pre-import install.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Resolve remote LoRA bases before importing transformers
_resolve_base_model only reads a local adapter_config.json, so a remote LoRA
adapter whose own id has no SSM substring but whose base is a Nemotron/Falcon/
Granite model had its base discovered only by ModelConfig in _handle_load, after
transformers was imported and its optional-backend availability snapshotted, so
the SSM kernel install there was too late. Add _remote_lora_base, a metadata-only
adapter_config.json fetch (no huggingface_hub / transformers import), and use it
in the pre-import path so the base is gated and its kernels pre-installed.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Gate only loaded roots, tier on the resolved base, read offline LoRA cache
Three follow-ups to the pre-import resolution:
- The security gate reused the SSM target list, which for a local full fine-tune
includes the config.json-recorded base. That base is never loaded, so scanning
it could falsely block a safe local checkpoint. Gate only the model plus a
genuine LoRA base (matching _handle_load's mc.is_lora), separate from the
broader SSM-install list.
- Tier activation ran on the raw adapter id, so a remote LoRA whose base needs a
sidecar transformers version imported the default and failed. Resolve the base
once up front and activate on it.
- _remote_lora_base bailed on offline before checking the hub cache, missing a
cached adapter's base. Read the cached adapter_config.json when offline or when
the fetch fails.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Keep the pre-import gate transformers-free; harden remote LoRA resolution
The pre-import security gate called security_load_subdirs, which imports
model_config and thus transformers, snapshotting optional-backend availability
before the SSM kernels are installed and defeating the ordering. Add
compute_subdirs to _run_security_gates and pass False in the preflight so it scans
from the root only (transformers-free); _handle_load still runs the authoritative
gate with full subdir scoping after the import.
_remote_lora_base now skips existing local relative paths (is_local_path) so a
checkpoint like outputs/run1 is never treated as a Hub repo, and distinguishes a
definitive 404 (not a LoRA -> None) from transient/offline failures (read the
cache), so a repo that is now a full model no longer resolves a stale cached base.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Probe a real model id for SSM kernels; respect HF_ENDPOINT
model_is_ssm is a substring match, so an arbitrary name could false-match and
force a mamba-ssm install that fails the load for a non-SSM model:
- a LoRA adapter id like user/falcon-h1-lora (the SSM-relevant code is the base's);
- a local checkpoint under an SSM-named parent dir, e.g. /runs/falcon-h1/llama-ckpt.
Add ssm_probe_identifier, which resolves the base (or a bare local checkpoint's
basename) and feed that to ensure_ssm_runtime from both the pre-import path and
_handle_load, so detection runs against a real model id, never an adapter id or
parent folders.
_remote_lora_base now honors HF_ENDPOINT so enterprise/mirror deployments resolve
the adapter base instead of always hitting huggingface.co.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten comments in the pre-import SSM gate/install path
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <michaelhan2050@gmail.com>
* Route dense NemotronH models to the transformers 5.10 tier
Dense NemotronH models (e.g. unsloth/NVIDIA-Nemotron-3-Nano-4B) describe their
layer stack with a hybrid_override_pattern that includes '-' (MLP) layers.
transformers only learned to parse that ('-' -> 'mlp' in pattern_mapping, 'mlp'
in valid_types and MIXER_TYPES) in 5.10; on 5.3/5.5 the config raises
KeyError: '-'. The model also ships auto_map remote code, so training and
inference that approve trust_remote_code load fine, but a native (TRC=False)
load such as export hits the built-in parser and fails with
'Failed to load checkpoint: -'.
Detect dense NemotronH from config.json (a '-' in hybrid_override_pattern, or
'mlp' in an expanded layers_block_type) and route it to the 5.10 tier, where the
model loads natively without remote code. Pure-MoE NemotronH configs are
unaffected and keep their existing tier.
Covers both the local config.json and the remote HF-id paths, and adds tests for
the detector and the resulting tier selection.
* Tighten _nemotron_h_needs_mlp_support docstring
* Detect dense NemotronH in nested, cached, and resolved-away configs
Three gaps could still route a dense NemotronH (MLP '-' layers) to a tier
below 5.10 and hit KeyError: '-':
- VL wrappers (e.g. NemotronH_Nano_VL_V2) keep the dense language model under
llm_config/text_config; the detector only checked the top-level model_type.
Recurse into nested language configs.
- Offline or blocked config fetches returned None for an already-downloaded
repo. Read config.json from the HF hub cache before any network.
- A local checkpoint resolves to its base before tiering, so an offline/private
base discarded the local config that revealed the dense pattern. Prefer the
higher tier of the resolved base and the original path.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Harden NemotronH tier detection follow-ups
Address review of the nested/cached/resolved-away detection:
- The local re-check ran the full tier detector on the original path, so a bare
LoRA adapter under e.g. /runs/gemma-4-x/llama-lora could upgrade a default base
via directory-name substrings. Gate the re-check on a real local config.json so
it reads metadata, not path names.
- The HF hub cache was read before any network, so an online tier check could
serve stale config.json after the repo changed upstream. Consult the cache only
offline or after a failed fetch.
- Reading the cache imported huggingface_hub during tier detection, which runs
before a sidecar venv is activated and could pin the default-env hub into
sys.modules. Resolve the cache path with stdlib only.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Trim comments to be more succinct
* Select newest hub-cache snapshot by mtime and retry transient config fetches
The HF cache fallback in tier detection picked the lexicographically-first
snapshot when refs/main was absent (commit-pinned downloads), which can be an
older SHA than the Hub would load. Sort snapshots by mtime instead.
A transient online fetch failure cached the hub-cache fallback under the normal
(model_name, token) key, so a long-lived worker kept serving stale metadata even
after connectivity recovered. Return the fallback without memoizing it so the
next call retries the network.
* Harden config.json tier detection against auth failures and transient blips
- _load_config_json: a 401/403/404 from the raw Hub request is a definitive access
answer, not an outage. Return None instead of falling back to the HF hub cache, so
an unauthenticated or wrong-token request can never read another caller's cached
private metadata.
- _check_config_needs_510/550: only memoize the derived tier when the underlying
config read was definitive (local file, offline cache, or a completed fetch).
A transient fetch fallback is no longer pinned, so the tier is re-evaluated once
connectivity returns instead of staying stuck on the lower tier.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten comments in tier-detection auth/cache paths
---------
Co-authored-by: Daniel Han <michaelhan2050@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: log transformers version-switching decisions and stop swallowing MLX activation failures
Two logging gaps in dynamic transformers version switching (issue #6103):
1. get_transformers_tier returned a tier with no trace of why. Add an
info log at each decision point naming the model and the trigger
(which substring matched, or which config check fired), so a model
landing on the wrong tier is diagnosable.
2. The MLX fast-path in run_training_process activated the transformers
version inside a bare 'except Exception: pass', silently swallowing
failures while the non-MLX path reports them. A missing or broken
version venv (e.g. Gemma-4 needing 5.5.0) left no trace and only a
confusing downstream crash. Extract a small _activate_transformers_version_or_warn
helper that logs a warning on failure while keeping the non-fatal
fall-through, and call it from the MLX path.
Adds tier-selection logging tests and helper warn/silent tests.
* Studio: clarify path-prepend log, warn on venv version mismatch, log per-package install progress
Completes the remaining logging items of #6103 in studio/backend/utils/transformers_version.py:
- activate_transformers_for_subprocess: the early "Activated transformers X.X.X" line was misleading because at that point only the venv directory has been prepended to sys.path, not imported. It now says it prepended the venv to sys.path and notes the loaded version is confirmed later by "Subprocess loaded transformers ...".
- _venv_dir_is_valid: a detected version mismatch is logged at warning instead of info, since it immediately triggers a full venv wipe and reinstall that should be visible in the logs.
- _ensure_venv_dir: log each package as it starts installing with an N/M progress counter, so a slow runtime install is not mistaken for a hang (pip/uv output is piped and only surfaced on error).
Adds tests covering all three behaviours; pre-existing unused imports are left untouched.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: make tier log-capture tests independent of import order
The new issue #6103 caplog assertions in test_transformers_version.py
relied on the module-level sys.modules.setdefault("loggers", stub)
winning the import race. In a full backend pytest run another module
(for example test_log_filter_no_truncation, collected earlier) imports
the real loggers first, so the setdefault is a no-op and
transformers_version.logger becomes a structlog/stdout logger that
caplog cannot capture -- the tier, activation, venv-mismatch and
install-progress log assertions then fail even though the line was
emitted.
Bind a real stdlib logger to transformers_version.logger for the
duration of each test via an autouse fixture, so the module logs through
logging and caplog captures them regardless of collection order.
* Studio: log local checkpoint tier decisions and warn on MLX inference activation
- get_transformers_tier: the local config.json fast path returned a tier
without logging it, so local checkpoints stayed opaque while HF ids were
traceable. Log each decision there too, with a caplog regression test.
- inference worker: the MLX path swallowed _activate_transformers_version
failures with a bare except, the same gap issue #6103 fixed for training.
Warn instead, keeping the non-fatal fall-through.
---------
Co-authored-by: Daniel Han <michaelhan2050@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix Studio Python, Gemma 4 Unified sidecar, and worker crash messages
* Clean up Gemma 4 sidecar test patch contexts
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Polish inference worker crash message
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address transformers tier review feedback
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Route Gemma 4 assistant models to transformers 5.10
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Trim and tighten code comments and docstrings across the repository. Comment-only: every changed file verified code-identical to main via AST/token comparison.
Trim and tighten code comments and docstrings across studio/ Python. Comment-only: every changed file verified code-identical to main via AST/token comparison.
Raise ruff line-length to 100 and extend the local pre-commit format pipeline (def-signature magic-comma normalization, short multi-line assert collapse, kwarg '=' spacing, blank-line-after-short-import removal, adjacent string-literal / f-string+plain merge, redundant-pass pruning). Every transform re-checks the file AST and is dropped if it would differ; the whole-repo reformat is verified AST-identical per file and idempotent.
* split venv_t5 into venv_t5_530 and venv_t5_550 for tiered transformers 5.x support
* fix bfloat16 crash on T4 for FORCE_FLOAT32 models and disable trust_remote_code auto-enable for native t5 models
* revert FORCE_FLOAT32 dtype change
* restrict trust_remote_code auto-enable to Nemotron models only
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* use config.json model_type for tier detection, add unsloth/nvidia namespace guard
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Revert "[pre-commit.ci] auto fixes from pre-commit.com hooks"
This reverts commit fb43d468e2.
* Revert "use config.json model_type for tier detection, add unsloth/nvidia namespace guard"
This reverts commit fc49ae2453.
* add unsloth/nvidia namespace guard to Nemotron trust_remote_code auto-enable
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* reorder tier checks: all substring matches before config.json fetches
* extract shared activate_transformers_for_subprocess into transformers_version.py
* narrow Nemotron trust_remote_code to nemotron_h/nemotron-3-nano, add to export worker
* clean venv_t5 dirs before re-install in setup.sh, clarify version alias comment
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* run venv_t5 migration outside deps fast-path gate in both setup scripts
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* feat: support full model GGUF export, disable incompatible methods in UI
* fix: resolve base model from config.json for venv_t5 export switching
* feat: detect BNB-quantized models and disable all export methods for quantized non-PEFT checkpoints
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: relocate Ollama Modelfile alongside GGUFs during non-PEFT export cleanup
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
oobabooga