* Studio: allow torch 2.11.x on the CUDA install path
The CUDA torch repair path (_ensure_cuda_torch) installs torch/torchvision/
torchaudio from an exclusive --index-url, so _CUDA_TORCH_PKG_SPEC decides
exactly which torch the Studio venv gets. It was capped at torch<2.11.0, so on
a cu128/cu130 host the venv resolved torch 2.10.x even though the CUDA indexes
now publish torch 2.11.0. That left the Studio venv a torch minor behind the
torch 2.11.0 Docker base image, so the CUDA dedup step would relink base libs
under a mismatched torch.
Raise the upper bound to <2.12.0 (torchvision <0.27.0, torchaudio <2.12.0) so
the CUDA install path lands on torch 2.11.x, matching the rocm7.2 spec and the
base image. The torchao selector already maps torch 2.11 -> torchao 0.17.0, and
_ensure_flash_attn degrades gracefully when no prebuilt wheel matches (Blackwell
skips it outright; non-Blackwell prints a warning and continues), so no other
pin needs to move.
Add test_cuda_torch_spec.py to lock the bound (torch 2.11.x in, 2.12.x out) and
assert the CUDA and rocm7.2 upper bounds stay in lockstep.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* test: use zip(strict=True) so a spec length mismatch fails loudly
* install.sh: widen the CUDA torch ceiling to <2.12.0 so a fresh install matches the base
Raising _CUDA_TORCH_PKG_SPEC alone was not enough: that spec only feeds
_ensure_cuda_torch(), the ROCm-poisoning repair path that early-returns on a
normal NVIDIA host. A fresh CUDA install (including the studio Docker build,
which runs `bash install.sh --local`) takes its torch from install.sh's
TORCH_CONSTRAINT, which was still capped at torch>=2.4,<2.11.0, so cu12x/cu13x
resolved torch 2.10.x and the venv landed a minor behind the torch 2.11.0 base
image.
Extend the existing `case "$TORCH_INDEX_URL"` block (which already relaxes
rocm7.2) with a `*/cu[0-9]*` branch that widens the ceiling to <2.12.0, keeping
the >=2.4 floor so an older CUDA index (e.g. cu118) that tops out below 2.11
still resolves. The CPU wheel and older ROCm tags stay on <2.11.0 (the glob
does not match /cpu). torchvision/torchaudio are bare on this install line and
resolve their compatible companions via wheel metadata, matching the rocm7.2
pattern.
Add behavioral tests (Python + shell) exercising the case block: cu118/124/126/
128/130 widen to <2.12.0, rocm7.2 stays 2.11.x, and /cpu plus older ROCm keep
the default <2.11.0.
* install.sh: key the CUDA torch widening off the index leaf, not the full URL
The `*/cu[0-9]*` glob matched a `cu<digit>` segment anywhere in TORCH_INDEX_URL,
so a custom UNSLOTH_PYTORCH_MIRROR whose base path contains e.g. cu128 but whose
final leaf is cpu or an older ROCm tag would still widen TORCH_CONSTRAINT to
<2.12.0, contradicting the block's own comment and letting a CPU / older-ROCm
mirror resolve torch 2.11.x. Match on _torch_index_leaf (the final path segment
the backend classification just above already computes) so only a real cu*/
rocm7.2 leaf is affected; cpu and older ROCm keep the default <2.11.0. Update
the Python + shell tests to mirror the leaf-anchored case and add regression
cases for a mirror base that contains cu128 but resolves to a cpu / rocm7.1 leaf.
* install: freeze the torch trio during the with-deps unsloth installs
Released unsloth wheels can pin an older torch than Step 1 installed
(unsloth 2026.7.2 declares torch<2.11.0), so the with-deps resolve from
PyPI silently downgrades the pinned +cuXXX torch trio to PyPI's default
wheel. The flavor guard cannot catch every such swap: PyPI's torch 2.10
default is itself cu128-flavored, so the cuXXX tag comparison still
matches while the version silently drops. Freeze the just-installed trio
with uv --overrides (overrides replace dependency requirements during
resolution), keeping torch 2.11.0+cuXXX in place while unsloth's other
dependencies resolve normally. Verified on the cu128 path: without the
override torch drops 2.11.0+cu128 -> 2.10.0; with it the trio survives
and unsloth 2026.7.2 + unsloth-zoo install cleanly.
* install: fold UV_OVERRIDE env files into the torch-trio overrides file
The CLI --overrides flag is the command-line form of UV_OVERRIDE, so
passing it replaced any overrides file already exported for the process;
macOS arm64 exports UV_OVERRIDE=overrides-darwin-arm64.txt for the same
generic install path and would have lost those pins. Concatenate any
UV_OVERRIDE files into the temp trio file so both keep applying.
* install: extend the torch-trio overrides guard to migrated installs
Four follow-ups to the Step-2 --overrides guard, all empirically verified:
1. The migrated-environment with-deps unsloth install resolved
unsloth>=2026.7.2 (which pins torch<2.11.0) without the overrides file,
so a migrated CUDA venv on torch 2.11 was silently downgraded -- the
exact bug this branch fixes on the fresh path. The overrides build is
now a function (_build_unsloth_torch_overrides, reading the trio
installed at call time) invoked by both with-deps paths; the migrated
no-torch path installs --no-deps and stays unguarded.
2. The overrides temp file is now cleaned by the EXIT trap (same pattern
as _UV_OVERRIDE_TMPDIR, pre-initialized empty so an inherited value can
never reach the trap's rm); previously any Step-2 failure leaked it.
3. Folding UV_OVERRIDE files used cat, which joins the last requirement of
a file lacking a trailing newline onto the next file's first requirement
(reproduced: idna==3.10certifi==2025.1.31 makes uv fail parsing).
4. Inherited torch/torchvision/torchaudio override lines are now filtered
out when folding: uv intersects duplicate overrides rather than
last-wins (verified on uv 0.10.12: direct conflict is unsatisfiable,
transitive conflict silently backtracks), so a conflicting inherited
trio pin would break the resolve the generated exact pins protect.
Both 3 and 4 are handled by a single newline-terminating awk filter
that preserves non-trio overrides (torchmetrics, torchao, ...).
test_unsloth_torch_override.sh extended: migrated-path coverage, trap
assertion, and a functional fold test (14 checks).
* installer: tighten comments
* install: keep the existing torch release when re-running the installer
Re-running `curl -fsSL https://unsloth.ai/install.sh | sh` over an existing
install rebuilds the venv for clean state, which silently moved users to the
newest torch in range (2.10 -> 2.11 once the constraint widened). A torch the
user already validated must survive an unsloth update.
Before the old venv is moved aside for rollback, its torch version is probed
(last stdout line only, so sitecustomize noise cannot corrupt it). After the
index leaf is chosen, _previous_torch_pin turns that version into a
torch==X.Y.Z pin, but only when it cannot do harm:
- cu*/cpu leaves only; rocm leaves keep their floors (rocm7.2 must land 2.11
for the Strix _grouped_mm fix) and the Radeon wheel-matching path is
untouched.
- The wheel's flavor tag must match the freshly chosen leaf, so a flavor
change (cpu -> cuda, cu126 -> cu130) still installs the correct new build.
- The base must look like a release, so probe noise never becomes a pin.
- UNSLOTH_TORCH_UPGRADE=1 opts out and restores the old always-newest
behavior; the substep line advertises it.
The supported range is kept in _PREV_FALLBACK_CONSTRAINT: if the exact
release is not resolvable from the chosen index (custom mirrors prune old
wheels), the install warns and falls back to the newest supported release
instead of failing the whole run. The later flavor-mismatch repair reuses
TORCH_CONSTRAINT, so a mid-install clobber is repaired back to the kept
release rather than the newest one.
Verified end to end: a venv seeded with torch 2.10.0+cu130 re-run through the
full installer finishes with torch 2.10.0+cu130 (previously 2.11.0+cu130).
Tests: tests/sh/test_previous_torch_pin.sh covers keep/flavor-change/rocm/
noise/opt-out plus wiring (probe ordering before venv replacement, fallback
present, SKIP_TORCH gate).
* install: constrain kept torch pins to the supported window
Review caught that _previous_torch_pin pinned the previous venv's torch on
flavor match alone, so a release outside the installer's active range (a
2.3.x manual install below the >=2.4 floor, or a 2.12.x manual upgrade above
the ceiling) replaced the bounds computed just above it and a rerun kept a
torch the installer otherwise deliberately excludes.
New _torch_release_in_window checks the probed base against the active
TORCH_CONSTRAINT ("torch>=A.B[,<C.D.F]") at major.minor granularity, which
is exact for the windows this script uses (ceilings are always X.Y.0; a
non-.0 ceiling would only make it conservative). Anything unparseable
answers no, so probe noise or a malformed window fails toward the supported
range instead of becoming a pin. _previous_torch_pin takes the active
constraint as a third argument and refuses out-of-window releases; the
in-window keep behavior is unchanged.
Tests: out-of-window rows (2.3.x floor, 2.12.x ceiling, boundary keeps, cpu
and macOS windows, malformed/empty windows) plus direct
_torch_release_in_window coverage.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* test(version-compat): keep GRPO fake-run logits finite on CPU
The GRPO fake-run test samples completions from a tiny untrained model on
CPU. Such a model can emit non-finite logits, so torch.multinomial inside
generate() intermittently raises "probability tensor contains either inf,
nan or element < 0" -- a nondeterministic sampling failure, not a regression
(the Trainer already fixes the seed, but CPU reduction order is not
bit-reproducible). Add a forward hook that sanitizes the LM head logits to a
finite bounded range before sampling, so the fake run reliably exercises the
whole train loop; the test checks the loop runs, not the numerics.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* test(version-compat): drop redundant nan_to_num bounds (clamp handles them)
* test(version-compat): scope GRPO finite-logits guard to the GRPO test
Only test_grpo_trains_on_cpu autoregressively samples completions, so it is
the only canary that can hit the non-finite-logits torch.multinomial crash.
Move the _guard_finite_logits hook out of the shared _load_plain() and into
test_grpo_trains_on_cpu so the SFT and DPO canaries keep asserting against the
model's true, unclamped logits.
---------
Co-authored-by: Daniel Han <unslothai@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix(registry): don't register deepseek models at import time
`_deepseek.py` called `register_deepseek_models(include_original_model=True)`
at module scope, so merely importing `unsloth.registry` registered models
(and reached the hub via `list_models`) as a side effect. None of the other
five families (`_gemma`/`_llama`/`_mistral`/`_phi`/`_qwen`) do this; they only
register when `register_models()` asks them to.
Two consequences:
- Importing the registry populated MODEL_REGISTRY on its own (32 entries,
including 10 `deepseek-ai` original models that no other family leaks) and
did network I/O at import time.
- Because the import-time call set the `_IS_DEEPSEEK_*_REGISTERED` guards with
`include_original_model=True`, the later `register_models()` call (which uses
the default `include_original_model=False`) early-returned, so the
original-model set won permanently.
Remove the stray module-level call. The `if __name__ == "__main__"` block below
still registers with `include_original_model=True` for standalone use, so the
generator script is unaffected.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* test(registry): make import-side-effect test pass on CPU-only runners
The new test spawned a fresh `python -c "import unsloth.registry"` that did
not inherit tests/conftest.py's GPU-free harness, so on no-accelerator CI
runners the child raised NotImplementedError from unsloth_zoo.device_type
before printing REGISTRY_SIZE. With check=True this surfaced only as an
opaque CalledProcessError, turning the "Repo tests (CPU)" job red even
though the registry fix is correct.
Import this directory's conftest inside the child first so it applies the
same device_type stubs and torch.cuda probe patches. Also use check=False
and include the child stdout/stderr in the assertion message so a future
import regression is legible instead of an opaque non-zero exit.
* test(registry): assert register_models() leaks no upstream originals
Adds a fresh-interpreter test that register_models() registers only
unsloth-org models (deepseek still present via the normal path) and never
leaks the upstream deepseek-ai originals that the import-time guard poisoning
used to leak (129 -> 139). Factors the conftest-harness subprocess runner
into a shared helper reused by both registry import tests.
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <unslothai@gmail.com>
* Replace standalone Studio wording with Unsloth
Replace the single word Studio with Unsloth wherever it is used as
shorthand for Unsloth Studio in docs, CLI output, UI strings, i18n
locales, workflow display names, comments and docstrings.
Kept unchanged: the full name Unsloth Studio, third party product
names (LM Studio, Visual Studio, Mac Studio), feature names
(Recipe Studio, Fine-tuning Studio and its translations), and all
identifiers such as env vars, commands, paths and filenames.
* Address review feedback on the Studio wording rename
Use "an" before Unsloth where the rename left the article as "a".
Restore the split brand where Unsloth and Studio render as two halves
of the full product name: the onboarding sidebar subtitle and the
IPv6 localhost warning. Scope two messages to the full name Unsloth
Studio where plain Unsloth was misleading: the AMD README bullet and
the CLI studio setup error.
Cross-platform staging CI surfaced five Repo tests (CPU) failures where
main's tests assert on script internals this branch legitimately
changed; each reconciled on its merits.
The WoA native-wheel probe still used uv's deprecated --index-url alias
that main's test suite now forbids in favor of --default-index (same
semantics, and --default-index is what overrides inherited uv index
defaults); the probe now matches the convention.
The CUDA provision gate spelled its linked-dir guard with the
:-false default form that main's prune-refactor test blacklists
file-wide. The variable is unconditionally initialized far above, so the
guard now uses the plain spelling with identical semantics. The variable
guard itself stays: unlike a symlink test, it also covers the
canonical-location reuse case where the linked dir is not a symlink.
The gpu-detection tests extract named shell functions into a sandbox,
so _setup_has_usable_nvidia_gpu's new _resolve_nvsmi dependency made the
sandboxed helper die on command-not-found and report not_usable for
usable cases; the extraction list now includes the resolver, and the
driver-version hardening assertion tracks the resolved-path spelling
while still requiring the timeout wrapper. Also hardened the resolver
assignment with an explicit empty fallback so a future non-condition
call site cannot trip set -e.
The staging run also showed the Mac Studio Update uninstall step dying
mid-run, consistent with the round-seven group kill signalling its own
process group; the round-eight self-pgid guard already fixes that and
this push carries it to CI.
Verified: the five failing tests pass locally at this head (the one
remaining local red, test_negative_control_no_tokenizers, fails
identically with these changes stashed and did not fail in CI), bash -n,
PowerShell AST parse, and the sh battery matches the branch baseline.
* fix(dataprep): skip .jsonl lines that are valid JSON but not objects
`_read_file_by_format` json.loads each line and hands the result to
`_extract_text_from_json`, which assumes a dict:
for field in self._TEXT_FIELDS:
if field in data and isinstance(data[field], str):
A JSON line does not have to be an object -- `"context"`, `["text"]` and
`42` are all valid JSON. For those, `field in data` stops being a key
lookup and becomes a substring/membership test, so `data[field]` raises:
"context" -> "text" in "context" is True (substring!)
-> TypeError: string indices must be integers
["text", "foo"] -> TypeError: list indices must be integers
42 -> TypeError: argument of type 'int' is not iterable
The TypeError escapes past `except json.JSONDecodeError: continue`, so the
whole load dies on one odd line.
That except clause is also the tell: a *malformed* line is already skipped
gracefully. A *well-formed* line that happens not to be an object should be
too -- it carries no text either way. This makes the two agree.
Reachable from `unsloth-cli.py:253` (`--dataset foo.jsonl` auto-detect) and
`RawTextDataLoader` is exported from `unsloth/__init__.py`.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Slim the non-object jsonl regression test and shorten the guard comment
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Stabilize Studio regression tests
Rebuild on current main. Restore the set-membership sidebar account-block matcher
(#6647, which fixed the same order-sensitive regex, was reverted on main, so the
guard is failing on main again) and keep the watchdog replacement-race fix, whose
blocked-watchdog stub now waits without a timeout so a superseded watchdog stays
alive until cleanup regardless of scheduler load.
* Tighten the blocked-watchdog stub comment
---------
Co-authored-by: Daniel Han <unslothshared@gmail.com>
* fix(tokenizer): check for tokenizer.model after saving it, not before
`fix_sentencepiece_tokenizer` creates its temporary directory, then returns
early unless that directory already contains a tokenizer.model:
if not os.path.exists(temporary_location):
os.makedirs(temporary_location) # fresh, empty
if not os.path.isfile(f"{temporary_location}/tokenizer.model"):
return new_tokenizer # always true
old_tokenizer.save_pretrained(temporary_location) # writes that file
The file only appears on the line after the check, so the guard is always
true and the body never runs. Nothing else writes that path either --
`convert_to_fast_tokenizer` saves into a per-name subdirectory, not
`{temporary_location}/tokenizer.model`.
Both call sites are in `get_chat_template` and are commented "Must fix the
sentence piece tokenizer since there's no tokenizer.model file!" -- the
guard defeats the exact intent the caller states. The effect is silent: the
caller still gets a working `new_tokenizer`, but the sentencepiece piece
rename is skipped, so the mapped token (e.g. the eos token remapped to
`<|im_end|>`) is missing from tokenizer.model and GGUF/llama.cpp exports
carry the old piece.
`check_if_sentencepiece_model` in save.py does the same probe in the right
order -- makedirs, save_pretrained, then isfile. Match it.
Tests are added under tests/saving/ next to the existing sentencepiece
coverage, and to the two Bucket-A lists in consolidated-tests-ci.yml, since
Repo tests (CPU) --ignores tests/saving and these need protobuf.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Clear stale tokenizer.model before the sentencepiece guard
The guard now runs after old_tokenizer.save_pretrained, but the default
temporary_location is a fixed reusable directory. A fast-only tokenizer writes
no tokenizer.model, so a stale file from an earlier sentencepiece call could
pass the guard and patch the wrong model (e.g. mixing models in one process,
like a long-running server). Remove any existing tokenizer.model first, and add
a regression test.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Empty the reusable sentencepiece scratch directory each call
The final AutoTokenizer.from_pretrained reloads the whole temporary_location, so
removing only a stale tokenizer.model still let other artifacts from a previous
tokenizer (added_tokens.json, chat template, etc.) leak into the reload when the
default reusable directory is used across models in one process. Recreate the
directory instead, and add a regression test for the leaked-artifact case.
* Clear only top-level scratch files, keep subdirectories
Recreating the whole reusable directory deleted the {name} subtree that
convert_to_fast_tokenizer stores a converted tokenizer's source vocab in, so
old_tokenizer.save_pretrained could not copy tokenizer.model and the guard
returned the tokenizer unpatched for those legacy converted tokenizers. Remove
only stale top-level files (all the final reload reads) and leave subdirectories
intact. Add a regression test for the converted-source subdirectory.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Keep the current tokenizer's own source vocab when clearing
On a repeated get_chat_template(map_eos_token=True) call, the returned tokenizer's
vocab_file points back at the top-level tokenizer.model, and the cleanup deleted
that source before old_tokenizer.save_pretrained could re-emit it, so the guard
returned the tokenizer unpatched. Skip removing the old tokenizer's own source
vocab while still clearing stale files from a different tokenizer, and add a
regression test.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Use a per-call temporary directory for the sentencepiece fix
The scratch directory defaulted to a single shared path, so concurrent or repeated
get_chat_template(map_eos_token=True) calls could delete or overwrite each other's
tokenizer.model between save and reload (tripping the piece assertion or reloading
the wrong model), and stale files from an earlier tokenizer could leak into the
reload. Work in a unique per-call subdirectory instead: this isolates every call
without deleting anything the caller owns, and replaces the earlier per-file cleanup.
Tests updated to read the patched model from the reloaded directory and to cover
isolation and source-vocab preservation.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Pass only the applied token mappings into the sentencepiece fix
get_chat_template mirrors token remaps into tokenizer.model via
fix_sentencepiece_tokenizer, but two caller paths passed a mapping that did not
match what they wrote to the fast tokenizer JSON, so once the sentencepiece patch
runs the model and JSON disagree:
- the mapped-token path skipped entries whose target already existed but still
passed the full mapping, renaming a piece the JSON never changed;
- the EOS-swap path swapped both tokens in the JSON but passed only one direction,
leaving two stop_word pieces and no old EOS piece.
Pass the applied mapping (and both swap directions) instead. Add regression tests.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten sentencepiece guard comments
* Add SPDX license identifier to sentencepiece guard test
* Reclaim the per-call sentencepiece scratch directory
The per-call tempfile.mkdtemp fixed the shared-directory race but never cleaned
up, so a long-running process leaked one scratch dir per call. The dir cannot be
deleted eagerly for sentencepiece tokenizers because the returned tokenizer's
vocab_file points into it (a later save_pretrained copies the patched
tokenizer.model from there). Reclaim it correctly instead: remove the dir right
away on the fast-only path (the returned tokenizer never references it), and
attach a weakref.finalize so the sentencepiece dir is removed once its tokenizer
is garbage collected. Add regression tests for both.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten the scratch-dir reclaim comment
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: danielhanchen <unslothshared@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
* Propagate fp8 block_size before the early return in get_lora_parameters_bias
get_lora_parameters_bias set the fp8 block_size on W/W_quant only after the
disable_adapters/merged early return, so on the merged or disabled path (merged
inference, DPO reference model) a block-fp8 weight lost its real block_size and
downstream fp8 kernels fell back to [128, 128]. The non-bias sibling
get_lora_parameters already sets block_size before its early return; move the
block so both behave the same.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Guard the fp8 block_size against a missing quant state
A decompressed compressed-tensors layer keeps quant_method == "fp8" while its
weight is back to bf16, so it has no quant state and get_lora_parameters_bias
must still return W_quant None for fast_linear_forward to fall back to a plain
matmul. Only attach block_size when a quant state was actually found.
* Guard the sibling get_lora_parameters fp8 block_size against a missing quant state
Mirror the get_lora_parameters_bias guard so a decompressed compressed-tensors
layer (quant_method fp8, bf16 weight, no quant state) does not raise
AttributeError on the fused-LoRA path. Add a CPU-only regression test.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* refactor(studio): move chat model picker into features/model-picker
Relocate model-selector + its support files from components/assistant-ui
into a self-contained features/model-picker feature (own barrel), mirroring
the modular Hub layout. Pure move + import repoint; no behaviour change.
* feat(model-picker): add per-model config persistence layer
Superset PerModelConfig (customContextLength, kvCacheDtype, speculativeType,
specDraftNMax, tensorParallel, chatTemplateOverride, trustRemoteCode) persisted
to localStorage (unsloth_model_configs) with schema versioning + LRU budget.
KV-dtype and speculative value sets match main's sidebar (no q4_0/ngram-simple).
Reuses features/hub/lib/model-identity for normalization; adds storage-key layer
and applyPerModelConfigToRuntime (sets tensorParallel, which the old PR omitted).
* feat(picker): modular backend for chat-template validate + default fetch
New studio/backend/picker package (schemas/service/routes) mounted at /api/picker:
- POST /api/picker/validate-chat-template (Jinja syntax validation, no false positives)
- GET /api/picker/chat-template/{model_name} (default template from tokenizer_config.json,
reusing get_cache_path/resolve_cached_repo_id_case; graceful null, no model-code exec)
Frontend api/templates.ts client + hooks/use-model-defaults lazy cache. No backend
changes to the existing inference load route (per-model load fields already supported).
* feat(model-picker): bind picker on-device list to shared hub inventory
Picker now sources cached + local models from useHubInventory (the Hub's shared
store) via a thin adapter, replacing its own /api/models/* fetchers + module
caches. Hub, download manager, and picker now share one source of truth, so
completed downloads reflect in the picker automatically. Partial/live-download
rows are filtered from the cached lists (unchanged rendering). Local naming/search
preserved via additive LocalInventoryRow modelId/displayName. Variant expander,
scan-folder management, recommended-fit, search, external providers untouched.
Known minor: cached 'Downloaded date' sort tiebreak degrades to alphabetical
(hub cached rows carry no mtime); default 'recent' (load-time) sort preserved.
* feat(model-picker): per-model config step inside the picker
Picking a (non-external) model now opens an in-picker config view built from
main's current load controls (context length, KV cache dtype, speculative
decoding, draft tokens, tensor parallel) plus a chat-template editor backed by
the picker validate/default endpoints. 'Remember for this model' persists the
config per model+variant; Run forwards the config to the existing load flow via
meta.config. External models bypass the step. Two-view orchestration lives in
model-selector (single interception point); pickers.tsx call sites untouched.
trustRemoteCode dropped from PerModelConfig to preserve main's per-load consent.
* feat(chat): apply/persist per-model config through the load flow
handleCheckpointChange threads meta.config into the selection; stageOrLoad and
the autoload/Hub-run paths now apply the picker config (explicit pick or saved
remembered config) via applyPerModelConfigToRuntime before staging/loading, with
keepSpeculative set so a remembered speculative mode survives the model switch.
Replaces the old remembered-load-settings seeding (resolveInitialConfig now the
single source). SelectedModelInput carries config.
* refactor(chat): remove per-model load config from the right sidebar
The load knobs (context, KV cache, speculative, draft tokens, tensor parallel)
and the chat-template editor now live only in the picker config step. The sheet's
Model section keeps the staged Load/Cancel flow (config is applied at pick time);
sampling params, system prompt, and RAG are unchanged. Deletes the superseded
remembered-load-settings module + the store's applyRememberedLoadSettings action,
removes the now-dead sheet state/imports, and points the settings reset at
unsloth_model_configs. Delete-cleanup deferred (stale config is LRU-capped).
* fix(model-picker): remove leftover sidebar-staging cogwheel + empty Model section
The downloaded-variant gear (ModelLoadSettingsAction) staged a model straight
into the right-sidebar Run-settings flow -- the old 'configure before load' path
now fully replaced by the in-picker config step. Removed the gear + its component.
Also gate the sheet's 'Model' section to staged picks only (pendingSelection):
after the load-knob strip its content is staged-only, so it was rendering an
empty section header whenever a model was merely loaded.
* chore(chat): remove dead per-model-config setters + modelControlsDisabled
After the load-config UI moved into the picker, the store's per-model setters
(setKvCacheDtype/setSpeculativeType/setSpecDraftNMax/setTensorParallel/
setCustomContextLength/setChatTemplateOverride) had zero callers
(applyPerModelConfigToRuntime writes via setState), and the sheet's
modelControlsDisabled was unreferenced. Verified dead across the whole tree.
* fix(chat): config-step Load actually loads (ignore Load-on-selection)
Root cause: with Settings > Chat > 'Load on selection' turned OFF, the config
step's load went down the deferred-staging path -- opening the right sidebar with
'<model> is staged, not loaded yet / Choose Load model'. The in-picker config step
IS the deliberate load action, so its Load now loads immediately (or downloads +
auto-loads when not cached) regardless of the toggle. Renamed the button
'Run model' -> 'Load model' to match. Native/dropped picks still honor the toggle.
* refactor(chat,hub): retire 'Load on selection' — config step is the only load flow
The in-picker config step (and the Hub Run button) now fully supersede the old
stage-to-sidebar flow, so the Load-on-selection toggle is removed everywhere:
- chat stageOrLoad: every pick loads immediately, or downloads + auto-loads when
not cached (the previous default behaviour, now universal).
- hub Run: drops the stage branch; downloaded GGUFs load directly with their saved
per-model config (no collision with the chat config step — both end at selectModel).
- store: removed loadOnSelection field/setter/key/default; Settings>Chat toggle and
its settings-reset entry removed.
- staged sidebar section is now a download-progress view (auto-loads on completion).
No manual staging remains; stageModel is used only for background auto-load downloads.
* feat(model-picker): default chat template from GGUF + thread variant through config flow
Read the embedded tokenizer.chat_template from GGUF files (read_gguf_chat_template
in gguf_metadata) and use it as the per-model default. Plumb gguf_variant through
the picker service, /api/picker/chat-template route, frontend templates API, and
use-model-defaults so the right variant's template is fetched.
Also refine the picker config-page/model-selector wiring, drop the dead
ggufNativeContextLength runtime path, and add the per-model-config storage keys to
the settings prefs export.
* feat(model-picker): read safetensors chat template + hide editor where it has no effect
Resolve the default chat template for safetensors models: prefer the modern
chat_template.jinja, fall back to the tokenizer_config.json chat_template field,
then chat_template.json (multimodal processor), then the GGUF embedded template.
Applied to local dirs, the HF cache snapshot scan, and the HF remote fetch.
Hide the chat-template editor in the picker for safetensors models — the override
is only applied at load by the GGUF/llama.cpp backend, so editing it on safetensors
currently has no effect. GGUF keeps the editor. Nothing removed; the dialog stays
for when the safetensors apply path is wired up in a later branch.
* fix(model-picker): set legacy-migration flag only after the write succeeds
Set unsloth_model_configs_migrated only once writeMap confirms the migrated
map persisted, so a quota/storage failure no longer marks migration done and
silently drops the user's pre-existing remembered settings — the next load retries.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* MVP model picker fixes
* MVP picker config fix
* MVP safetensors config
* MVP max seq config
* MVP max seq fix
* Fix static max tokens cap ignoring model context
* Fix picker GGUF scan parity
* fix(studio): harden model picker config loading
Apply remembered per-model configs consistently from picker and Hub loads, keep default configs from overriding standing speculative settings, add config access for direct local GGUF files, and support saving or forgetting active model settings without a reload.
* Fix model picker config flow
* Fix model picker config loads
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Avoid recursive per-model config migration reads
* Apply the displayed context length when loading a GGUF
* Fix template validation, cached template lookup, and failed load rollback
- Validate chat templates with the loopcontrols extension so templates
that use break or continue tags pass the picker validator, matching the
inference renderer that already accepts them.
- Read the default chat template from the newest cache snapshot rather than
an arbitrary iterdir order, so an older cached revision no longer prefills
a stale template.
- Capture the runtime per-model config before a load and reapply it when the
load fails, so a failed switch leaves the active model context, KV cache,
template, and speculative settings as they were.
* Make chat template view only for safetensors models
Custom chat template overrides are applied at inference only for GGUF
models, which pass the template to llama-server. The safetensors backend
renders with the model built-in template and ignores the override, so
editing it would save a value that never loads. For safetensors the
config page now opens the template as a read-only preview with a note
that editing is not available yet. This can become editable once
inference support for custom safetensors templates lands in main.
* Fix model picker config edge cases
- Restore prior runtime config when a load no-ops for the active model
- Cap the picker validator request body via the protected prefixes
- Keep the GGUF context slider max above the loaded context
- Fetch subfolder chat templates for uncached Hub repos
- Show the compare side config when reopening the picker
* Keep saved GGUF context above the fallback ceiling
* Show the model config in the run settings sidebar
* Fix model config sidebar reset and context slider
- Stack the remember toggle and action buttons in the sidebar
- Reset the config to defaults instead of the loaded values
- Fetch the native context so the slider max is not the loaded value
* Fix model picker config and download regressions
- Run picker chat template routes off the event loop
- Depth and root guard local template directory scans
- Restore download manager flow for uncached hub picks
- Apply per model context length on reload
- Import model picker symbols from the feature barrel
* Fix model picker config and cached download sorting
- Restore load settings when a Hub run is rejected mid load
- Reuse one NumericValueInput instead of a duplicate copy
- Fix double decode of the model name in the template route
- Remove the unused reset-to-loaded settings action
- Fix cached model download sorting
* Fix model picker per-model config edge cases
Honor a saved or typed max seq length above the model's native context so
RoPE extended values are no longer clamped and silently overwritten. Allow
typing past native while the slider keeps native as a soft ceiling.
Guard the fetch success paths in use-model-defaults against an aborted
signal, and refetch when the HF token changes.
Hash the chat template content in the sidebar remount key instead of its
length. Enable reset for a GGUF whose native context is unknown, and floor
the context slider max so it can never fall below the min.
* Fix GGUF context auto-fit and gated model config token
Stop forcing a 32768 context when a GGUF native context is unknown so the backend auto-fits to VRAM again, while still honoring an explicit context edit.
Send the HF token as a query param so gated safetensors models resolve their max position embeddings.
Derive model default state during render to drop the set-state-in-effect calls.
* Fix native GGUF context ceiling and guard picker template reads
Restore the native context store field so the sidebar slider keeps the
full ceiling for drag and drop GGUFs. Limit local chat template reads to
the browse allowlist, skip malformed repo ids, and drop unused model
picker exports.
* Fix model picker lint boundaries
* Fix model picker review findings
Chat template editor never seeded its draft. Radix only calls onOpenChange
from internal events, so the seed in the nextOpen branch was dead and a model
with a saved override opened empty. Saving then cleared the override. Drop the
dead branch, treat draft as an untouched sentinel, and reset it on every close.
Uncached Hub picks could auto load a model after the user left the chat. Main
detached the staged pick on route exit and on chat context change. Carry the
context key on the pending pick and skip the load when it no longer matches.
Also clear configTarget when the picker closes, restore the onUpdated ref so
variant rows stop resubscribing on every parent render, skip the LRU write when
the entry is already most recent, import NumericValueInput relatively, and drop
the unused ModelUpdateAction barrel export.
* Preserve GGUF context on active reload
* Fix model picker per-model config regressions
- Stop reloading the already loaded model on re-pick
- Hide infra models from the chat picker
- Detect vision support on cached GGUF repos
- Honor saved maxSeqLength on auto load
- Restore default chat template for local GGUFs
- Warn on save failure and revert config on cancel
- Refetch picker inventory on open
- Persist read only per model config safely
* Fix stale model auto load
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix model picker numeric input sizing and constraints
Size value inputs to their content so long context lengths are not clipped,
restrict them to numeric characters, and stop the speculative decoding label
from truncating in the sidebar.
* Fix picker CI tests and harden chat template resolution for PR #6647
- tests: point the descender guard at the moved model-selector.tsx path
- tests: exclude the disabled Reload model button from the regenerate locator so .first targets the real Regenerate
- picker/service.py: reject symlinked template/gguf leaves that resolve outside the browse allowlist (HF cache reads unchanged)
- compare mode: resolve each pane's own remembered chat template instead of inheriting the other pane's from the store
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Protect future-schema per-model configs from deletion for PR #6647
savePerModelConfig already refuses to overwrite a stored config whose schema version is newer than this client understands, but deletePerModelConfig did not. Unchecking Remember on an older client therefore silently destroyed a newer client's saved config. Apply the same guard on delete and surface the blocked case through the existing saveFailed toast.
* Protect future-schema per-model configs from quota eviction for PR #6647
The save and delete guards already refuse to touch a stored config whose schema version is newer than this client understands, but the quota-eviction path did not, so a full store on an older client could still evict a newer client's config. Skip future-schema entries when evicting and fail the save if the budget cannot be met without them.
* Fix GGUF context persistence, compare context, and rollback settings for PR #6647
Persist a GGUF context override from the user's intent instead of collapsing it against the loaded context, which reintroduced the context-reset (f4838782cb reverted the native-baseline fix). model-config-page now collapses the saved value against native, and use-chat-model-runtime and chat-adapter retain the requested context on load so re-saving another setting keeps the override; a null request stays null so a VRAM auto-fit never becomes a stored override.
shared-composer: a compare pane with no explicit GGUF context now loads at native (0) like single-view, not the session maxSeqLength that silently shrank the shown context.
use-chat-model-runtime: restore the previous model's KV cache dtype and chat template on a failed-load rollback so it runs as it was, not with backend defaults.
* Preserve native path token when reloading the active model for PR #6647
handleReloadActiveModel rebuilt the selection without the store's activeNativePathToken, so reloading a file-picked GGUF after a settings change validated the display label as a repo/path and failed. Thread the active native token through the reload selection so native-loaded models reopen correctly.
* Make picker template validation resilient and accept HF generation tags for PR #6647
Import Jinja lazily inside validate_chat_template so a backend without the optional jinja2 package (GGUF-only installs) still starts instead of raising ModuleNotFoundError at import time. Register a no-op extension for the Transformers {% generation %} assistant-mask tag so pasting a valid HF chat template validates, matching the renderer, rather than being rejected as an unknown tag.
* Honor remembered compare config and parse processor chat_template.json for PR #6647
* Fix failed-load rollback context and processor template map fallback for PR #6647
* Restore speculative decoding config on failed-switch rollback
When a model switch fails after the previous model was unloaded, the
rollback reload restored tensor_parallel, KV cache dtype and the chat
template override, but omitted speculative_type and spec_draft_n_max and
cleared their loaded shadows to null. The previous model therefore came
back running at backend defaults (speculation off) while the UI still
showed it enabled, and the status resync confirmed the off state. Resend
the previous model's speculative settings in the rollback load and keep
the store's active and loaded speculative fields in sync with them.
* Reset max sequence length when a model has no saved config
applyPerModelConfigToRuntime reset every per-model field except
maxSeqLength, which it only wrote when the incoming config had one.
maxSeqLength is the sole field carried on store.params, so selecting a
model with no remembered config left the previous model's value in place
and later loaded the new model at that leaked length. Fall back to the
standing default so an unremembered model loads at its own default.
* Surface a message when a variant update cannot start
startManagedUpdate handled the conflict and error start outcomes but let
busy fall through as if the update began, so the confirm dialog closed
with no job created and the cached variant stayed stale. Show an info
message when the repo is busy with a sibling transfer so the click is
not silently dropped.
* Keep per-model speculative choices out of the global default
A staged load with a per-model or one-off config sets keepSpeculative,
which already skips reading the global speculative preference. The
matching save still ran unconditionally, so the model-specific choice was
written to the global unsloth_chat_speculative_type and a later model with
no saved config started from it instead of Auto. Skip saveSpeculativeType
when keepSpeculative so the per-model choice stays isolated.
* Seed non-active model settings from the app default max length
The Run settings page captured initialMaxSeqLength from the loaded
model's runtime params and fell back to it for a model with no saved
config. Opening settings for a different, unloaded model and clicking
Load then sent the active model's context (for example 64k) instead of
the 4096 default, risking validation failures or OOMs. Seed the default
for non-active models and keep the runtime value only for the active one.
* Prefer sidecar tokenizer chat template over the GGUF copy for variants
_chat_template_from_dir returned the embedded GGUF template first when a
variant was selected, reversing the tokenizer-first precedence of the
no-variant path. A model whose chat_template.jinja or tokenizer_config.json
supersedes a stale embedded template then got the wrong template on
variant selection. Keep tokenizer files first regardless of variant; the
variant only picks which GGUF is the fallback. Adds regression tests for
both the tokenizer-wins and gguf-fallback cases.
* Keep per-model speculative choices load-local in autoload and compare
The interactive load path treats a per-model speculative choice as
load-local and skips writing it to the global default. Autoload and
generalized compare still called saveSpeculativeType unconditionally, so a
remembered off or ngram setting leaked into unsloth_chat_speculative_type
and later models with no saved config inherited it. Persist the global
preference only when the value came from the global settings.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: record the compare pane's loaded context in runtime state so the active model's settings and any reload or save use it, not the previous context
* Studio: notify the user when a Hub autoload can't start because another download for the model is already running, instead of silently dropping it
* Studio: drop the merge's orphaned staged-model store helpers and unused alert imports
The main merge left isPendingGguf and pendingSelectionMatches referencing the
removed PendingModelSelection type, and the alert-dialog/alert imports unused
after the permission-mode dropdown replaced the bypass dialog, so tsc -b failed.
* Studio: cache a null default chat template so the viewer stops re-fetching it
A model with no sidecar or embedded template resolves to a terminal null, but
that result was never cached, so reopening the template viewer re-ran the
backend and Hugging Face lookup every time.
* Studio: detect direct-file GGUFs in run settings so Max Tokens uses their context
A GGUF loaded from a local file or custom folder has no variant label, so the
run-settings panel treated it as non-GGUF and clamped Max Tokens to the session
max_seq_length instead of the loaded GGUF context. Detect it via the reported
GGUF context and the .gguf checkpoint suffix, matching the chat page.
* Studio: prompt to re-select a local model file when its lease expired before reload
A file-picked GGUF is reachable only through a native path token that the
desktop host prunes after a TTL. Reloading reused that token blindly, so a
reload long after the initial load failed with an opaque error. Track the
token's expiry and, when it has passed, ask the user to re-select the file
instead of attempting a doomed reload.
* Fix descender-clipping test to tolerate sidebar layout utilities
The sidebar account-block div carries layout utilities (min-w-0, flex-1)
between 'flex' and 'flex-col', so the descender-clipping guard's regex,
which required 'flex' immediately followed by 'flex-col', no longer matched
and the test failed to locate the account-block div. Generalize the prefix
to allow intervening flex utilities while still capturing the leading-*
class before the collapsible visibility utility and asserting leading-tight,
so the guard against clipped glyph descenders is fully preserved.
* Harden picker chat-template resolution
Enforce the 64 KiB chat-template contract at the validate endpoint's request
model so a direct caller cannot submit a template far larger than the frontend
allows (MaxBodyMiddleware only bounds the whole request body, not this field);
oversized templates now return a clean 422.
Apply sidecar-over-GGUF template precedence globally across cached snapshots
instead of per snapshot. A repo with multiple cached revisions previously
returned the first snapshot's template, so a newer GGUF-only revision could
win over an older revision's maintained chat_template.jinja sidecar, which
contradicted the documented intent that sidecars supersede the embedded copy.
* Guard per-model config against future-schema and lossy migration
Two forward-compatibility gaps in the versioned per-model config store:
- The load/apply path returned and normalized a stored record without checking
its schema version, so a record written by a newer client was reinterpreted
under the current schema and applied to a live model load, even though save,
delete and eviction all refuse to touch future-schema records. Reject
future-schema records on load too.
- The one-time legacy migration enforced the storage budget without protecting
the entries it had just migrated and set the completion flag unconditionally.
When storage was already full of future-schema records (which are unevictable
by an older client), the migrated entries were the only evictable ones and
could be dropped while migration was still marked complete. Protect the
migrated keys during eviction and only mark migration complete when they
survive, so it retries once space frees up.
* Discard chat-template validation results after the dialog closes
Server-side template validation is async, but closing or cancelling the editor
did not abort it, so a late-arriving valid response still called onSave and
applied a template the user had already dismissed. Track a validation token
that is bumped on close and ignore any validation result whose token is stale.
* Record native lease expiry when loading a picked GGUF from the chip
The pending-native-model chip loaded via stageOrLoad directly, bypassing
loadNativeModelIntent, so activeNativePathExpiresAtMs was never recorded for a
chip-loaded file. A later reload then either skipped the lease-expiry guard
entirely (expiry left null) or compared against a previously loaded file's
stale expiry, so reload could reuse an already-pruned token or wrongly block a
still-valid one. Route the chip through loadNativeModelIntent, which builds the
same selection and records the expiry.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Prefer sidecar template for a directly selected local GGUF file
A direct .gguf file path read its embedded chat template without checking the
parent directory for a maintained sidecar (chat_template.jinja /
tokenizer_config.json), while directory and variant selections already prefer
the sidecar. That let the config editor preview or save a stale embedded
template for the same model depending on how it was selected. Check the parent
directory sidecars first, then fall back to the embedded copy, and cover both
paths with tests.
* Resolve cached chat template per revision, newest first
The earlier change searched every cached snapshot for a sidecar before
considering any snapshot's embedded GGUF template, which let an obsolete sidecar
from an older revision override the newest revision's template. Restore
per-snapshot resolution (newest first): a revision's sidecar still supersedes
its own embedded GGUF copy, but a newer revision is no longer overridden by an
older revision's sidecar.
* Preserve autoload transport conflicts and surface background busy downloads
- When a Hub autoload hits a transport conflict, keep pendingHubAutoLoad bound
instead of clearing it. Clearing it re-keyed the download surface and its
cleanup cancelled the conflict the toast tells the user to resolve, so the
Hub resume affordance was gone the moment it appeared. Return early on
conflict, mirroring the started branch, so resolving it from the Hub still
auto-loads on completion.
- The background-download branch handled started and conflict but silently
dropped a busy outcome, leaving the user with no feedback when a peer variant
of the same repo was already downloading. Surface the same busy toast the
autoload path uses.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
* fix(save): unsloth_push_to_hub_gguf(save_method="lora") raises NameError
unsloth_push_to_hub_gguf reads is_main_process at save.py:3181 but never
declares it. Its twin unsloth_save_pretrained_gguf declares it (2783) and
uses it the same way (2839) -- the LoRA branch was copied between the twins,
the parameter it depends on was not. There is no module-level global, so the
name resolves as a global load and the branch raises NameError 100% of the
time.
save_pretrained_gguf(save_method="lora", push_to_hub=True) raises a
ValueError that tells users to "use .push_to_hub_gguf(save_method='lora')
instead" -- the documented escape hatch is the broken call.
Add is_main_process to the signature, positioned as in the twin, and forward
it to unsloth_save_pretrained_gguf on the merged path so the parameter is not
silently ignored there. Default stays True, so nothing changes for existing
callers.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix(save): preserve GGUF push compatibility
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Etherll <61019402+Etherll@users.noreply.github.com>
* fix(dataprep): smart_chunk_text single-chunk path leaks internal tensor type when eos_token_id is None
RawTextDataLoader.smart_chunk_text()'s single-chunk branch only
converts `tokens` to a plain Python list inside the
`if eos_token_id is not None:` guard. When a tokenizer has no
eos_token_id configured, that conversion is skipped entirely and the
function returns whatever internal tensor-like object came out of
the tokenizer normalization step (e.g. a torch.Tensor) as
"input_ids", instead of a list of ints.
The sibling multi-chunk branch a few lines below does the conversion
unconditionally, before checking eos_token_id -- the two branches of
the same method disagree on output type depending purely on whether
the tokenizer has an EOS token. Downstream, create_causal_dataset()
does `labels = [list(ids) for ids in input_ids]`; list()'ing a
tensor produces a list of 0-d tensor elements rather than plain
ints, inconsistent with every multi-chunk sample and liable to break
type inference in Dataset.from_dict()/downstream collation.
Fix: move the list conversion out of the eos_token_id guard,
matching the multi-chunk branch's existing pattern.
Added test_smart_chunk_text_single_chunk_no_eos_returns_plain_list
to tests/test_raw_text.py, confirmed red against unfixed code
(assertion failure: input_ids was a MockTensor, not a list) and
green after the fix. Full tests/test_raw_text.py (both test
functions) passes. ruff check + the repo's ruff-format-with-kwargs
script: clean.
Note: tests/test_raw_text.py does not appear to be wired into any
.github/workflows/*.yml CI job (a pre-existing repo characteristic,
not something introduced by this change) -- verified locally via
`python3 tests/test_raw_text.py`.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Etherl <61019402+Etherll@users.noreply.github.com>
* Fix Studio toast close-button positioning
* Use UTF-8 for locale regression test
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Harden toast close-button positioning
* Limit language menu height
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Single-pass GGUF export for direct outtypes + parallel multi-quant
save_to_gguf defaulted first_conversion to model_dtype before the block
that picks the optimal base conversion, leaving that block dead since it
landed (#3356). Every default export (fast_quantized -> q8_0) therefore
ran two passes: convert HF -> 16-bit GGUF, then llama-quantize -> q8_0,
writing a 2x-size intermediate that the cleanup step deletes again.
- Route single-output exports whose type convert_hf_to_gguf.py emits
directly (f32/f16/bf16/q8_0) through one conversion pass with no
16-bit intermediate. Measured on Qwen2.5-0.5B-Instruct (8-core CPU):
bytes written 1525 MB -> 531 MB (2.9x less), peak extra disk 994 MB
-> 0, wall time neutral on local NVMe (14.8s vs 15.4s). The dequantized
q8_0 tensors are bit-identical to the two-pass output (max diff 0 over
all 290 tensors, same quant-type table). On disk-capped runtimes
(Kaggle 20 GB, Colab) the removed intermediate is the difference
between an export that fits and one that dies - see the Kaggle error
text this file already carries. imatrix runs keep the two-pass route
since only llama-quantize can apply one; explicit first_conversion is
still honored.
- Run independent llama-quantize passes two at a time when several
quant methods are requested (thread budget split between workers,
outputs byte-identical, order preserved). Measured 1.38x wall-clock
on q4_k_m+q5_k_m+q6_k. Sequential under UNSLOTH_ENABLE_LOGGING=1 to
keep subprocess logs readable; kill switch
UNSLOTH_PARALLEL_GGUF_QUANTS=0. Duplicate methods now quantize once.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Guard parallel GGUF quant on Kaggle and make multi-quant failures atomic
Each llama-quantize pass loads the whole model into RAM, so running two at once on Kaggle can OOM a host that succeeded sequentially; skip the parallel path there. On a failed multi-quant export, stop launching queued passes and remove orphaned quant outputs so a failure leaves no partial GGUFs behind, keeping the 16-bit base for retry. Also accept 0/false/no/off/empty for UNSLOTH_PARALLEL_GGUF_QUANTS so a well-meant 'false' actually disables parallelism, and add tests/saving/test_gguf_single_pass_export.py to the CI saving bucket so the new tests run.
* Preserve pre-existing outputs for canceled quant passes on failure
The parallel cleanup unlinked every requested output name, so a failed rerun could delete a valid model.<METHOD>.gguf left by an earlier successful export for a method whose pass was canceled and never ran this session. Skip canceled futures and only remove outputs from passes that actually executed.
* Gate parallel GGUF quant on available memory and preserve prior outputs
Skip the two-worker path when RAM cannot hold two full-model quantizations at once (and on Colab as well as Kaggle), so a multi-quant export that fit sequentially no longer OOMs. On failure, remove only outputs this run newly created, tracked against a pre-launch snapshot, so a rerun into an existing _gguf directory never deletes a valid artifact from an earlier export.
---------
Co-authored-by: djs <dschroers2@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
RawTextDataLoader.smart_chunk_text takes chunk_size and stride as its own
arguments, so a direct call with stride >= chunk_size bypasses the
constructor validation. In that case `start_idx += chunk_size - stride` is
non-positive, so start_idx never advances past the first window and the
chunking loop never terminates (hangs).
Re-add the chunk_size/stride guard at the top of smart_chunk_text so direct
callers fail fast with a clear ValueError. The constructor keeps its own
guard for the internal callers (defense in depth). Add a regression test
that calls smart_chunk_text directly with stride == chunk_size and
stride > chunk_size and asserts it raises instead of hanging.
Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
* Unsloth: appearance palettes, customization options, and control restyle
Adds Standard, Classic, and Minimal color palettes to Appearance settings,
each adapting to light and dark mode. Classic is a neutral enterprise look
that reserves its blue accent for toggles, badges, and focus rings; Minimal
is strictly black, grey, and white.
Adds customization options scoped to the active mode: accent, background,
and foreground colors with an in-app color picker, UI and code fonts with a
searchable dropdown covering bundled, device, and imported fonts, font file
import, UI and code font sizes, contrast, pointer cursors, reduce motion,
font smoothing, and translucent sidebar. Settings persist through the
personalization API with backend validation and sync across devices.
Restyles core controls for a cleaner, flatter look in both modes: bordered
white input fields, fully rounded pills for single-row controls, no drop
shadows, simple straight-line chevrons replacing all rounded arrow icons,
and consistent hover tones in dropdown menus. Popovers now portal into the
open dialog so their lists scroll correctly inside modal dialogs.
Moves Language into General settings and Chat defaults into the Chat tab
above the Canvas section.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Unsloth: appearance follow-ups, font options, and settings search
Neutralizes focus and selection rings across all palettes so highlighted
elements, including typing boxes and the selected palette card, never take
the accent color. The custom accent no longer recolors rings.
Restyles the color controls as filled pills showing the hex value inside,
with text and border contrast picked from the color's luminance. Menus in
popovers now match the app's dropdown menus: rounded-lg corners, tighter
padding, accent hover rows, and a bordered search field. Popovers inside
modal dialogs are modal so their lists scroll with the wheel. Outline
buttons share the same dark fills as dropdown triggers.
Adds heading and chat font options next to the UI and code fonts, each
using the searchable font dropdown and persisting through the
personalization API. Removes the translucent sidebar option end to end.
Adds settings search: a search field at the top of the settings sidebar
that filters setting names across every tab, grouped by tab with icons,
and jumps to the tab on click.
* Unsloth: use the shared accent token for dark hover fills
The settings dialog nav, its close button, the model selector, and the
project switcher hovered with hardcoded blue tinted greys (#3a3d43,
#2d2e32) in dark mode while every menu and sidebar uses --accent. All
hover and active pill fills now use the accent token so dark hovers are
the same everywhere and adapt to the active palette.
* Unsloth: settings search polish and jump to matched setting
Widens the settings dialog to 880px and the sidebar column to 248px so
the search field has more room. The search pill aligns with the left
start of the Settings title, gets more spacing above and below, and its
icon and placeholder sit slightly further left.
Search results now jump to the exact setting: rows and sections expose
their label as a data attribute, and picking a result opens the tab,
scrolls the matched row into view, and flashes it briefly.
* Unsloth: settings search bar spans the full nav pill width
The search field now starts and ends at the same edges as the nav hover
pills instead of being inset to the title text.
* Unsloth: address review findings on motion, sync, and font limits
Reduce motion Off now opts back out of the OS reduced-motion preference
for CSS animations via a force-motion class that the media rules skip,
and forcing reduce motion On keeps the loader exceptions (spinners,
loading dots, progress bars) animating.
When the color scheme follows the system, the resolved mode is now part
of the theme store snapshot, so an OS scheme flip re-renders consumers
and reapplies per-mode custom colors instead of leaving stale inline
variables from the previous mode.
Imported fonts get an aggregate size cap (4.4M characters) on both the
frontend sanitizer and the backend model so the persisted store always
fits browser localStorage quotas, with a clear error toast when an
import would exceed it. Backend validation also tightens imported font
names (rejects CSS delimiter characters) and requires strict base64
font data URLs, matching the frontend patterns.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Unsloth: profile toggle to hide the sloth in the chat greeting
Adds a Show greeting sloth switch to Settings > Profile. The chat welcome
hides the mascot when it is off. The preference persists locally and
through the personalization API, with backend validation and tests, and
the row is reachable from settings search in all four locales.
* Unsloth: control restyle, dropdown scrolling, and palette consistency
Settings sidebar puts search on top with the tab list under a small
Settings label. Combobox popups scroll with the wheel inside dialogs by
falling back to manual list scrolling while a dialog scroll lock is
active, and the local model selector popover became modal for the same
reason. Number inputs swap native spinners for a shared grey stepper
that clamps to min, max, and step. Run settings fields in light mode use
the same white fill and border as the settings dialog. Selection and
focus rings derive from each palette's border color instead of near
black, hover borders soften the same way, the Classic sidebar stays
white like Standard, decorative greens follow the palette accent, and
meaning-carrying marks like the hub verified badge keep the brand green
in every palette.
* Unsloth: palette card selection keyed off the palette attribute
Switching palettes restyles the whole page the moment data-palette lands
on the html element, but the React re-render that moves the selection
classes arrives later, so the ring and check briefly stayed on the
previous card with the new palette's colors. The active ring and check
now key off html[data-palette] in CSS, so they swap in the same style
pass that swaps the tokens. Also adds breathing room around the settings
search bar and under the Settings label, shortens the greeting sloth
description, and renames the avatar section to Or pick a sloth profile
picture in all locales.
* Unsloth: restore neutral rings, drop the palette check, sidebar spacing
Puts the ring tokens back to their fixed per palette values and removes
the hover border darkening, undoing the derived border experiment. The
selected palette card no longer shows a check since the ring already
marks it. The settings sidebar search bar, nav pills, and search results
get a little side padding, and the Settings label lines up with the pill
text.
* Unsloth: indicator restyle, sidebar menu customization, edge fade toggle
- Derive focus and selection rings from the border color so indicators
stay 1px and adapt to every theme and palette
- Suppress mouse focus rings except on pressed controls to remove the
selection flash on the avatar and palette pickers
- Defer settings panel rendering so the active nav pill updates instantly
- Customizable sidebar user menu with drag to reorder and shortcuts to
the settings tabs
- Grey hover for the standard light palette instead of green
- Borderless controls in dark mode with fill based focus states
- Profile picture: no picture option, pencil edit icon, atomic selection
- Font dropdowns: narrower triggers and the resolved default shown as
Inter Variable (Default)
- System prompt border darkens on focus
- New appearance setting to swap edge fades for thin divider lines
- Move the theme bootstrap to an external script to satisfy CSP
* Unsloth: harden theme boot and Firefox scroll container focus
- Guard the theme and palette storage reads separately so a blocked
localStorage (private browsing) still resolves a mode from the OS
preference instead of skipping the boot entirely
- Firefox makes scrollable containers keyboard focusable and drew its
3px UA outline on them; swap it for the app's soft 1px indicator
* Unsloth: make the UI and code font settings reach the font utilities
The theme block declared the sans and mono stacks as literals, so
Tailwind inlined them into every font-sans and font-mono utility at
build time and the runtime overrides from Settings > Appearance never
applied. Reference the :root tokens instead, matching how the color
tokens already work.
* Unsloth: in-dropdown font upload, accent meters and avatar, naming cleanup
- Move font importing into each font dropdown: Upload and Select folder
sit side by side under the list, imported fonts get an inline remove,
and the standalone Import font row is gone
- Uploads reuse fonts the user already has (bundled, imported, or
installed, matched by file name with style suffixes stripped) instead
of embedding a duplicate copy; only new fonts are embedded
- Folder scan lists font files from a picked folder in every dropdown
for the session; picking one imports it through the same path
- Fallback avatar uses the control accent with a readable foreground
instead of the neutral primary that rendered black outside standard
- Monitor bars, progress defaults, sliders, and usage meters use the
control accent; warning and danger tiers stay amber and red
- User facing strings that called the app just Studio now say Unsloth
in all four locales, keeping Unsloth Studio and LM Studio intact
* Unsloth: left align the font upload actions and divide them
Upload and Select folder now read from the left like the list items,
with a short vertical rule between the two.
* Unsloth: keep sliders neutral and the chat greeting on Hellix
- Sliders are controls, not meters, so their fill goes back to the
neutral primary instead of the palette accent
- The base h1 rule reads --font-heading with !important and the chat
thread root resets that variable to the sans stack, which pulled the
greeting off Hellix; restore the stack on the greeting element
* Unsloth: move the None avatar cell last and keep footer actions on one line
- None sits after the sloth pictures instead of leading the grid
- Upload shrinks to its label so Select folder no longer wraps
* Unsloth: size the folder action to its label
Both footer actions now hug their content so the hover pill does not
stretch across the leftover row width.
* Unsloth: separators only between unrelated settings clusters
Rows inside a titled section are related, so the per row divide-y is
gone from SettingsSection. A SettingsGroupDivider marks the two real
boundaries in the theme section (colors to fonts, fonts to contrast)
and the Clear all chats row gets its destructive border back now that
divide-y no longer draws one for it.
* Unsloth: balance the two font upload actions
Both actions share the footer row evenly again; nowrap keeps Select
folder on one line at the narrower width.
* Unsloth: drop the theme section dividers and split the chat menu groups
The colors, fonts, and contrast rows read fine without rules, and the
chat menu gains its one real boundary between the pin toggles and the
disclaimer rows.
* Unsloth: normalize oversized sidebar menus and reject newline font data URLs
Two backend validation fixes in PersonalizationCustomization:
- sidebarMenu refused any list longer than the number of distinct ids
because Field(max_length) is enforced before the dedupe validator runs.
A stale or duplicated payload that would normalize to one entry per id
was rejected outright, defeating the normalizer that exists for exactly
that case. Cap the incoming list at a generous multiple so it reaches
the validator; a pathologically long list is still refused.
- The imported font dataUrl validator used re.match on a pattern ending
in $, which also matches just before a trailing newline, so
"data:font/woff2;base64,AAAA\n" passed even though the frontend JS
pattern rejects it. Use re.fullmatch for parity.
Adds covering tests for both.
* Unsloth: preview fonts in their own typeface and slim the color pills
- Every font dropdown entry, the default item, and the closed trigger
render in the font they name, falling back to the UI stack for
families the browser cannot resolve
- Color swatch pills drop from 36px to 28px so they sit closer to the
row label height
* Unsloth: drop the font row and theme section descriptions
The labels carry the meaning on their own; the mode switching note in
particular read long and confusing.
* Unsloth: let the chat greeting follow the heading font setting
The greeting stays on Hellix by default but adopts a chosen heading
font through a --custom-heading-font variable the applier sets only
while an override exists, so the thread root's sans reset for chat
prose no longer hides the user's pick from the greeting.
* Unsloth: divide the theme section clusters and align the color pill height
Separators return between colors and fonts and between fonts and
contrast, and the color pills share the 32px height of the font
dropdown triggers.
* Unsloth: color pills at half the dropdown width
Fixed w-24 against the w-48 font triggers, with tighter padding so the
hex value still fits.
* Studio: update dep-removal test after next-themes was replaced
The frontend no longer declares next-themes or imports it in src (it was
replaced by the custom theme store and boot script), so the checker now
reports its removal as a safe no-op. The C1 and C8 fixtures in
test_frontend_dep_removal.py still asserted next-themes was a used
dependency, which fails the studio frontend CI dependency-removal safety
check. Update C1 to expect a no-op PASS and drop next-themes from the C8
expected failures so the suite matches the checker's correct output.
* Studio: remove unused ageLabel and exportCollectionJsonl helpers
* Studio: fix blocked-storage theme desync, search jump race, font validation
- theme-store.ts: keep an in-memory currentTheme/currentPalette so a selected
value survives when localStorage is blocked (private browsing). The snapshots
previously re-read empty storage and reverted React state to the default while
the DOM already changed. The matchMedia handler no longer re-reads storage, so
it cannot clobber the in-memory choice; cross-tab storage events still adopt.
- settings-dialog.tsx: the search jump waited a single fixed 60ms for the
deferred tab panel to render, then silently missed under render lag. Retry
across animation frames until the target row exists, then scroll and flash.
- settings.py: apply the font-name character check to the four selected-font
fields (uiFont/headingFont/chatFont/codeFont), and forbid backslash, comma,
slash and control characters so a name cannot escape the quoted CSS
font-family or smuggle extra fallbacks. Adds covering tests.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix appearance customization edge cases for PR #7077
- Reset all local preferences now also clears palette and appearance customization
- Number input wrapper keeps full width so fields fill their flex/grid cell, and the stepper stays pinned to the field edge
- Number stepper snaps to the min anchored step grid like the native spinner instead of leaving a step-invalid value
- Code font now applies to chat code fences and inline code via a dedicated token
- Reduce motion (on/off) is honored by onboarding/tour confetti and the theme toggle view transition
- Re-importing a font under the same name with new bytes now swaps the FontFace
- Keep local customization when a synced record predates the customization field, and re-push it
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Align client font name sanitization with server validation for PR #7077
sanitizeFont now strips the same characters the backend _FONT_NAME_FORBIDDEN
rejects (backslash, slash, comma, backtick) plus control chars, so a locally
chosen font name can no longer pass the client but fail the personalization PUT
and silently stall appearance sync.
* Address follow-up review items for PR #7077
- Number input wrapper carries React Flow interaction classes (nodrag/nopan/nowheel) so clicking the stepper arrows increments instead of dragging the node
- Preserve local palette and greeting-sloth toggle when the synced record predates those fields, and re-push them, mirroring the customization handling (new paletteSaved and greetingSlothSaved response flags)
- Add settings-search scroll targets (data-settings-label) for the Profile title, description, display name, nickname, and avatar shape rows
* Preserve absent personalization fields on PUT for PR #7077
A stale client that omits palette or customization previously had those
defaults materialized by model_dump() and persisted, which flipped
paletteSaved/customizationSaved to true and defeated the legacy detection.
The PUT now dumps only the request's set fields and merges them onto the
stored record, so omitted fields keep whatever was already stored.
* Persist theme and palette via a fixed allow-list for PR #7077
The theme/palette values reach setTheme/setPalette from the authenticated
personalization sync, which made the CodeQL clear-text-storage query treat
writing them to localStorage as storing sensitive data. Store a re-derived
literal from a constant map instead, so a plain UI preference is not tracked
as sensitive; behavior is unchanged.
* Harden imported-font handling for PR #7077
- syncImportedFonts: a rejected FontFace.load() only clears the registry entry
if it still points at that face, so a same-name re-import while the old load
was pending is no longer untracked/leaked.
- Cap imported-font names to the backend length (100) so an over-long name can
no longer pass the client but fail the personalization PUT and stall sync.
- Add a backend test that a stale PUT preserves an existing stored palette and
customization (not just that absent fields stay absent).
* Return the merged personalization record from PUT
The PUT /personalization handler returned the request payload, which
Pydantic had already filled with defaults for any field the client
omitted. A partial or stale write (for example a client sending only
theme) therefore got back a response that contradicted both storage and
the next GET: preserved fields like palette and the custom font showed
their defaults instead of the stored values.
Return model_validate(merged) so the response mirrors what was stored.
The stored record is still the full merged dict, so legacy fields the
model does not know about are preserved as before.
* Fix small UI and keyboard-focus defects in appearance settings
- Settings search now scrolls to the result within its destination tab
instead of a same-named row in the previously rendered deferred tab
(for example "Storage" and "Models folder" appear in both General and
Resources).
- The reduce-motion segmented control honors its own Off/On/System choice
by reading useReducedMotionConfig instead of the OS-only useReducedMotion.
- The color picker saturation/value area is operable by keyboard, so the
role="slider" surface responds to the arrow keys it advertises.
- Profile avatars and palette cards show a visible keyboard focus ring
again.
- Guard the persisted appearance-customization write so a blocked or full
localStorage does not throw out of a store action, matching the theme
store.
- Import the appearance store symbols from the settings feature barrel.
* Tighten appearance fix comments
---------
Co-authored-by: shimmyshimmer <shimmyshimmer@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Studio: resolve llama.cpp prebuilts via the release-assets CDN to avoid GitHub API rate limits
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: resolve manifest-named prebuilt assets on the download-host fast path
Add tag-pinned CDN URLs for any manifest artifact whose hash is keyed under an
upstream-tag alias in the checksum asset, so the fast path resolves the same
assets the API path does. Cover the resolve body directly (only download_bytes
stubbed) and soften the doc's validation-equivalence wording.
* Studio: pin llama.cpp fast path to the releases/latest redirect tag
Derive the authoritative latest tag from GitHub's /releases/latest redirect
target instead of trusting the checksum asset's self-reported release_tag, so the
existing release_tag cross-check in parse_approved_release_checksums is a real
check again: a stale or mis-tagged checksum asset now falls back to the API. Pin
every fast-path URL to that tag. Fall back to the API on a manifest 404 as well,
since an in-progress release can publish the checksum asset before the manifest,
matching the sha256 404 handling. Document the releases/latest (created_at /
make_latest) versus published_at ordering divergence and why it is an accepted,
mitigated tradeoff.
* Studio: drop the llama.cpp prebuilt-resolution doc
Remove studio/docs/llama-cpp-prebuilt-resolution.md and the docstring pointer to
it; the resolution rationale (the created_at/make_latest vs published_at ordering
nuance) stays inline in _download_host_latest_release_tag.
* Studio: tighten llama.cpp download-host fast-path comments
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Handle linked instruction files in Bash cleanup
* Limit instruction cleanup to managed dependencies
* Make Bash cleanup test portable
* Run junction cleanup regression on Windows
* Keep instruction cleanup CI focused
* Studio: remove AGENTS.md from install artifacts
* Studio: prune CLAUDE.md from install artifacts
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix Studio instruction cleanup edge cases
* Trim Studio cleanup comments
* Make Studio cleanup safe on PowerShell 5.1
* Fix Studio cleanup ownership boundaries
* Simplify Windows link detection
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix SyntheticDataKit.chunk_data emitting chunks over max_tokens
The multi-chunk path built boundaries from np.linspace(..., n_chunks), but
pairing boundaries[:-1] with boundaries[1:] turns N points into N-1 ranges,
so it produced one fewer, oversized chunk: every chunk exceeded max_tokens
and a document just over the threshold came back as a single unsplit chunk.
Use n_chunks + 1 points so exactly n_chunks ranges are emitted, each within
max_tokens.
Also base n_chunks on the non-overlapped span: consecutive chunks overlap by
overlap, so covering length needs ceil((length - overlap) / stride) chunks, not
ceil(length / stride). The looser count over-counted by one just past a stride
multiple (a 673-token doc became 3 chunks of ~267 instead of 2 of ~369),
emitting an extra redundant chunk. Coverage and overlap are unchanged and every
chunk still stays within max_tokens.
* Condense chunk_data comments and clarify over-split test for PR #7073
---------
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
* Restore dropped FP8 weight_scale_inv tensors on load
Some block-scale FP8 checkpoints (for example Qwen3.6-27B-FP8, issue #6200) load
with transformers leaving an mlp.gate_proj as a plain bf16 Linear instead of an
fp8 module. Its raw quantized values are read into the bf16 weight and the
weight_scale_inv is dropped as an unexpected key, so the weight is used un-scaled
and the base model is garbage (perplexity around 2 million).
After load, for every checkpoint weight_scale_inv whose live weight is not fp8,
dequantize the orphaned weight in place using the block scale from the checkpoint
index. Modules that were converted correctly keep an fp8 weight and are skipped,
so healthy checkpoints and single-file checkpoints are a no-op.
Verified on Qwen3.6-27B-FP8: 64 gate_proj scales restored, perplexity 2028902 to
8.9. No-op on Qwen3-8B-FP8 (all scales already live).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Harden FP8 weight_scale_inv restore from review
- Skip restore when the model has no fp8 weights, so an intentionally
dequantized load (load_in_16bit) is never re-scaled and corrupted.
- Thread revision, subfolder and cache_dir through the index and shard
downloads so scales come from the same snapshot as the weights.
- Cover unsharded single-file model.safetensors checkpoints (no index).
- Handle transposed block-scale layouts and skip on a true grid mismatch
instead of applying a wrong scale.
- Match text-only VLM loads where the language_model prefix was stripped.
- Restore on the FastLanguageModel text path too, not only vision.
- Handle a scalar weight_block_size; per-tensor error handling so one bad
tensor cannot abort the rest or hide a partial mutation.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address second review round on FP8 scale restore
- Bound peak memory: dequantize block views in place with the fp32 scale
broadcast instead of materializing a full expanded scale and fp32 copy,
so a near-VRAM-limit load is not pushed into OOM by the repair.
- Restore on the sequence-classification load path too.
- Cover more VLM key remappings (language_model.model.* to
model.language_model.*) when matching modules.
- Skip the restore for variant loads (variant=...) rather than risk
applying default-checkpoint scales to variant weights.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Align FP8 scale restore revision with the loaded weights and warn on disk-offloaded layers
In llama.py the CausalLM/SequenceClassification weight loads resolve model_name on its
default branch (revision is not forwarded there), so read the dropped weight_scale_inv
tensors from the same default branch instead of the requested revision, avoiding rescaling
default-branch weights with scales from another revision.
In loader_utils.py a disk-offloaded layer keeps its weight on the meta device until the
offload hook materializes it, so the scale cannot be applied in place. Skip such layers
explicitly and print a warning rather than silently leaving them unscaled.
* Tighten comments in the FP8 scale restore path
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
parse_direct_linux_release_bundle and direct_linux_release_plan are no
longer reached by any live code path. Fork Linux installs resolve through
_fork_manifest_release_plans -> _linux_published_attempts, and the upstream
(ggml-org) path uses direct_upstream_release_plan. The dead parser also
called _resolve_linux_bundle_profile, which no longer exists, so its CUDA
branch would raise NameError if ever executed.
Drop both functions and the obsolete TestDirectLinuxNvidiaCpuGate; its live
equivalent TestLinuxPublishedAttemptsNvidiaCpuGate already covers the
NVIDIA no-silent-CPU behaviour.
* Keep native RoPE scaling when extending context; carry rope_theta for linear
When max_seq_length exceeds a model's native window, the loader overwrote the
model's rope_scaling with linear scaling. For models that already ship a scaled
RoPE (llama3/yarn/longrope) that is far worse for long context, and on
transformers v5 the linear dict omitted rope_theta (v5 keeps it under
rope_parameters), so the rotary base fell back to 10000 and broke past ~8K tokens.
Keep the native scaling and just widen the window; only synthesize linear for
plain-RoPE models, and carry rope_theta so v5 keeps the real base.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Only preserve native llama3 when extending context; keep linear fallback otherwise
The patched attention constructor (patch_llama_rope_scaling) rebuilds only linear,
llama3 and longrope and its longrope branch reads a top-level
original_max_position_embeddings, so preserving yarn or a nested-only longrope config
would raise during construction on transformers <= 4.47.1. Keep only llama3 native;
yarn/longrope/other types fall back to the linear override, still carrying rope_theta.
* Correct long-context extension comment to match llama3-only preservation
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix fast_gemv crash on compressed-tensors FP8 models
Loading a compressed-tensors FP8 checkpoint (for example
unsloth/Llama-3.2-1B-Instruct-FP8-Block) with fast_inference=False and
running a forward crashed with 'Parameter object has no attribute absmax'
inside fast_gemv.
A compressed-tensors CompressedLinear exposes an already dequantized bf16
weight at forward time while keeping a weight_scale Parameter. The quant
state resolution in get_lora_parameters/get_lora_parameters_bias fell back
to that weight_scale, so a bf16 weight was routed into the bitsandbytes
fast_gemv/fast_dequantize path, which expects a bitsandbytes QuantState
with an absmax attribute.
Only fall back to weight_scale_inv/weight_scale when the weight is still
fp8. A decompressed bf16 weight then resolves to no quant state and flows
through the normal bf16 path, which already handles bias and the LoRA
backward. Real fp8 and bitsandbytes 4bit weights are unchanged.
* Skip the fast_gemv dispatch test before importing unsloth when bitsandbytes is absent
* Fix Windows installer torch index override
* Clear inherited uv index env vars for pinned installs in studio/setup.ps1 (#6898)
* Harden setup.ps1 index-var clearing to truly remove vars (#6898)
* Apply UV_DEFAULT_INDEX torch index fix to Linux/Mac install.sh (#6898)
* Neutralize all uv index env vars for pinned torch installs (#6898)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix FastSentenceTransformer Qwen embedding preprocessing
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Document Transformer.load embedding modality fix for #6881
* Harden #6881 fix and add forwards/backwards-compatible regression tests
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fall back to Transformer constructor on legacy sentence-transformers without Hub-capable load
* Mirror legacy sentence-transformers fallback in embedding-parity tripwire test
* Tighten #6881 comments and docstrings
* Skip embedding-parity test on CPU-only runners since FastSentenceTransformer requires CUDA
* Honor the transformer module's saved subfolder when loading
modules.json records a path for the Transformer module (root for
decoder embedders like Qwen3-Embedding, 0_Transformer for the classic
layout). Pooling/Normalize already load from their saved path; thread the
same path into Transformer.load as subfolder so config and tokenizer
resolve like stock ST. stays a no-op, so single-module models are
unchanged.
* Make embedding-parity test bf16-aware
fp16 overflows to NaN on bf16-native embedders such as EmbeddingGemma
(Gemma3), producing a false parity failure. Prefer bf16 when the GPU
supports it so the tripwire can guard the full documented embedding
matrix (Qwen3-Embedding, EmbeddingGemma, BGE-M3, all-MiniLM, GTE-ModernBERT),
not just fp16-safe models.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
* Retry the Studio UI shutdown re-login on transient goto timeout
The Chat UI Playwright smoke intermittently failed at the pre-shutdown
re-login: page.goto('/login') can hit a 60s TimeoutError on a slow runner
even while the server is healthy, and the surrounding except only tolerated
ERR_ABORTED / interrupted-navigation, so a plain timeout hard-failed the job.
Wrap the re-login goto/wait/fill/submit in the same 3-attempt retry the
change-password step already uses (recover_or_replace_page between tries,
per-attempt fail screenshots, wait_for_health pre-gate). The composer wait
stays outside the loop so a retry never re-navigates after login has set
tokens (which would redirect to /chat via the guest guard); it remains the
authoritative confirmation, so a genuinely broken login still fails.
* Catch transient login-request failures and preserve error listeners on recovery
Wait on the /api/auth/login POST inside the retry (via click_and_wait_for_response)
so a transient 4xx/5xx is retried in-loop instead of surfacing only at the
out-of-loop composer wait, matching the change-password step. When
recover_or_replace_page swaps in a fresh page, re-attach the pageerror/console
listeners so error tracking survives the replacement.
* Stabilize floating monitor drag
* Restore floating monitor exit animation
* Harden Windows Studio smoke checks
* Keep API menu badge removed
* Apply no-build-tools env overrides in-script
The runner does not apply step-level env keys containing parentheses,
so ProgramFiles(x86) kept its real value and Find-VsBuildTools still
detected VS through vswhere. Set the overrides inside each pwsh step
instead; child processes inherit them. The resolver step moves to pwsh
because bash cannot export a variable named ProgramFiles(x86).
* Reset chat UI session without a second browser context
macOS runs Chromium with --single-process, where closing the last
context tears down the whole browser, so the shutdown re-login died
with TargetClosedError on new_page. Clear cookies and swap pages
inside the same context instead, opening the replacement page before
closing the old one.
* Keep the no-build-tools Path filtered across session refreshes
install.ps1's Refresh-SessionPath and setup.ps1's Refresh-Environment
rebuild the session Path from the Machine and User registry scopes, so
the process-level filter could be undone mid-install and re-expose
CMake. Filter those scopes in the Prepare step with normalized dir
matching and restore them in cleanup.
* Drop stale localStorage auth tokens before re-login
Auth tokens live in localStorage, not cookies, and the login guest
guard redirects on their mere presence. Remove them during the session
reset so the /login navigation is deterministic instead of relying on
the tolerated redirect bounce.
* fix: Remove moot has_blackwell_gpu() function
Fixes unslothai/unsloth#6961. This function skipped flash-attn on Blackwell GPUs because no prebuilt wheel existed;
Dao-AILab now ships one and url_exists() already gates resolution.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix: use torchao 0.17.0 for Blackwell
Fixes#6961. Torchao 0.16.0's cpp extensions are built against CUDA 12, so on a CUDA-13
torch (cu130 / Blackwell) they fail to load with "libcudart.so.12: cannot
open shared object file". Select 0.17.0 there instead: its cpp targets torch
2.11, so it is skipped cleanly rather than crashing. CUDA-12 / ROCm / CPU
torch 2.10 keeps 0.16.0 and its working kernels.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Condense torchao version-selection comments (no behavior change)
* Support torch 2.11 in the Studio installer via the torch2.10 prebuilt wheels
Map torch 2.11 to the torch2.10 prebuilt wheels for flash-attn, causal-conv1d,
and mamba through wheel_utils.prebuilt_wheel_torch_mm, applied in direct_wheel_url
(filename) and flash_attn_wheel_url (version). Those torch2.10 CUDA wheels load and
pass each project's own test suite on torch 2.11 (verified on B200), so a torch 2.11
environment gets the prebuilt accelerators instead of skipping or building from source.
Raise _CUDA_TORCH_PKG_SPEC to <2.12.0 (torchvision <0.27.0, torchaudio <2.12.0) so
the CUDA torch repair path can install torch 2.11, where torchao 0.17's cpp kernels
load cleanly. Add tests for the mapping.
* Keep has_blackwell_gpu as a False stub for future arch gating
* Restore has_blackwell_gpu as a return-False probe kept for future arch gating
Keep the nvidia-smi compute_cap detection and its two call sites, but short-circuit
with return False at the top so flash-attn is no longer skipped on Blackwell (sm_100+
now has prebuilt wheels and url_exists gates resolution). Drop the early return to
re-enable arch-based detection later.
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Studio: source CPU llama.cpp prebuilts from the unslothai fork
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: reject unknown Linux CPU arches and keep ROCm-tooling hosts off the CPU prebuilt
* Studio: extend the resolve-prebuilt ROCm-tooling guard to Windows
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: let ROCm-SDK-only CPU hosts take the fork CPU prebuilt
* Studio: accept windows-arm64 prebuilt kind and refresh stale fork-routing comments
* Studio: correct stale fork-routing comments and --resolve-prebuilt help
* Refresh stale ggml-org routing comments
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* feat: detect installed coding agent CLIs in Studio settings
The API-keys panel only ever showed the "claude" flavor of the
`unsloth start` command, so anyone using Codex, OpenCode, OpenClaw,
Hermes, or Pi had to manually rewrite the copied command by hand.
Add a backend check that looks for each agent's CLI binary on PATH
(shutil.which, mirroring the pattern already used elsewhere in
studio/backend/utils) and expose it as GET /api/settings/coding-agents.
The API-keys panel now renders a picker for all six supported agents,
marks the ones it finds installed, and defaults to one of those instead
of always falling back to claude.
Includes unit tests for the detection helper.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* address review feedback on coding-agent detection
Three fixes from PR review:
- detect_installed_coding_agents now treats a PATH lookup failure as
"not installed" instead of letting it bubble up and break the
settings endpoint; added a regression test for it.
- CodingAgentsResponse.agents is now typed as an immutable tuple
instead of a list built from one, matching CODING_AGENTS itself.
- Fixed a race in the API-keys panel: picking an agent while the
installed-CLI check is still in flight could get silently overwritten
once that check resolved. A ref now tracks whether the user has made
a manual choice, so the auto-detected default only applies before
that happens.
* Address Codex feedback: GGUF gating and remote-detection scope
- codex refuses to launch against a non-GGUF (transformers-backed) model
(unsloth_cli's _require_gguf_for_codex), so auto-defaulting to it produced
a copy-pasteable command that fails immediately whenever the loaded model
isn't GGUF. Add useActiveModelIsGguf() (looks up the active checkpoint in
the chat runtime store) and a correction effect that steers the auto-pick
away from codex unless the loaded model qualifies, without ever touching a
choice the user made by hand.
- Detection runs via shutil.which on the Studio backend host, which isn't
the same machine as the browser in a tunnel/remote session. Reword the
'installed'/'detected' copy to say so explicitly when the tunnel URL is
in use, instead of implying the check ran on the viewer's own device.
* Rework auto-default per review: loopback gating + inline GGUF check
Replaces the previous approach with the exact shape discussed on the PR:
- Export isLoopbackHost/normalizeHost from agent-command.ts. The detection
endpoint runs shutil.which on the Studio backend, which only describes the
browser's own machine when the base this panel targets resolves to
loopback. For a LAN or tunnel/remote base, gate the whole thing off --
don't mark anything as "detected" and don't let it drive the default --
instead of just relabeling the copy.
- Drop the separate GGUF-correction effect and useActiveModelIsGguf hook.
Read useChatRuntimeStore.getState().activeGgufVariant inline inside the
existing detection effect's .then() (so it doesn't need to sit in the
effect's deps), and pick the first detected agent that isn't codex unless
the loaded model is GGUF, leaving the existing default untouched when no
compatible agent is detected.
Verified both branches (loopback vs LAN/tunnel base, gguf vs non-gguf,
manual pick preserved, no-compatible-agent fallback) with a standalone
port of the .then() logic.
* Address latest Codex findings: stale detection, model swap, cache
- Clear detectedAgents (and skip the network call entirely) when the panel
leaves a loopback base, instead of leaving a previous loopback detection
result marked 'installed' for a command that now targets a LAN/tunnel/
remote host.
- Add a separate, network-free correction effect keyed on the live
activeGgufVariant: if codex was auto-picked while a GGUF model was loaded
and the user then switches to a transformers-backed model while this panel
stays mounted, steer away from codex instead of leaving a command that
unsloth_cli's _require_gguf_for_codex will now reject. Never touches a
manual pick.
- Drop coding-agents.ts's module-lifetime cache. Installed-CLI detection is
environment state, not a persisted setting, so a stale positive/negative
from before the user installed something (or reopened the tab) is worse
than one extra cheap local API call per mount; keep only the in-flight
de-dupe for concurrent callers.
Verified the correction-effect logic (gguf->non-gguf swap with/without a
fallback, still-gguf no-op, manual pick never overridden) with a standalone
port of the effect.
* Make the codex/GGUF auto-pick symmetric in both directions
The correction effect only steered away from codex when the model stopped
being GGUF; it never steered back toward codex if the model became GGUF
*after* a non-GGUF-gated fallback had already picked something else (e.g.
codex is the only detected CLI, a transformers model is loaded so the
selection correctly falls back to the claude default, then the user loads a
GGUF model while the panel stays mounted -- codex never gets reconsidered).
Consolidate into one effect that re-derives the preferred detected agent
from scratch whenever detectedAgents or activeGgufVariant changes, in either
direction, instead of only reacting to the codex-specific downgrade case.
The fetch effect now only populates detectedAgents/availableAgents; this
effect is the single source of truth for what gets auto-picked from that
list. Never overrides a manual choice.
Verified both transition directions plus the manual-pick-survives and
initial-detection cases with a standalone port of the derivation logic.
* Reset the auto-pick to the default when it stops being trustworthy
Two more real gaps from the latest Codex pass on d988f52:
- The unified derivation effect only handled the case where a *different*
detected agent could take over. If codex was the only detected agent and
auto-picked while a GGUF model was loaded, then the model stopped being
GGUF, 'preferred' came back undefined and the effect silently left the
selection on codex -- exactly the command unsloth_cli's
_require_gguf_for_codex now rejects. Fall back to DEFAULT_AGENT in that
case instead of leaving it untouched.
- Leaving a loopback base cleared detectedAgents (so the 'installed' badges
correctly disappear) but left whatever agent had been auto-picked from
that now-stale, server-side-only detection still selected. Reset to
DEFAULT_AGENT there too, unless the user picked by hand.
Introduces a shared DEFAULT_AGENT constant instead of repeating the "claude"
literal at each reset site. Verified all five cases (both new resets, both
manual-pick-survives variants, and the existing multi-detected-agent
fallback still preferring another compatible agent over resetting) with a
standalone port of the effects.
* Derive GGUF-ness from the actual loaded state, not just the variant string
activeGgufVariant only covers an HF-repo GGUF pick (a specific quant
variant string). A direct local .gguf file -- custom folder, LM
Studio, or drag-drop -- is just as much a GGUF the codex preflight
(unsloth_cli's _require_gguf_for_codex) would accept, but it never has
a "variant" to report, so it read as non-GGUF here even though
/api/inference/status correctly reports is_gguf: true for it. That
mismatch could leave a Codex-only install not auto-selected, or reset
an auto-picked Codex, for a model that actually supports it.
Combined activeGgufVariant with activeNativePathToken (covers the
drag-drop/picked-file case) and ggufContextLength (only ever populated
when the backend last reported is_gguf: true for the active model, see
applyActiveModelStatusToStore) so all three paths a model can be GGUF
through are covered, matching the same is_gguf-or-equivalent check
hasGgufSource already applies to a staged pick elsewhere in this
codebase.
* Clear stale native-path token on a non-GGUF status refresh
When a native (drag-dropped or picked) GGUF was loaded and the backend later
switches to a transformers model outside the UI load path, refresh() adopts the
new /api/inference/status via setCheckpoint and applyActiveModelStatusToStore.
Those reset activeGgufVariant and ggufContextLength but never clear
activeNativePathToken, so the isGguf OR stays true after the switch and a
Codex-only detection auto-selects unsloth start codex for a non-GGUF model its
preflight rejects.
Drop activeNativePathToken in applyActiveModelStatusToStore whenever the status
is non-GGUF. A real GGUF load reports is_gguf: true, so its token is preserved
(the load path owns it); only a non-GGUF status clears it.
* Add the AGPL-3.0 header to the new studio contract test
* Fix/adjust agent detection for PR #6909
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
Co-authored-by: wasimysaid <112766706+wasimysaid@users.noreply.github.com>
Daniel Han