* Studio: enable MTP for sub-3B Gemma separate-drafter GGUFs
The sub-3B auto-drop to ngram-mod was tuned for an embedded draft head
(Qwen), whose per-token cost regresses below 3B. Gemma ships the head as a
separate root mtp-*.gguf drafter, a tiny standalone model that is cheap
enough to win below 3B: B200 Q4_K_XL bench, draft-mtp n=2 vs spec-off,
gemma-4-E2B (2B) = 1.21x (accept ~0.65) while ngram-mod is 1.00x.
Exempt a separate drafter from the sub-3B gate everywhere the threshold is
applied: the resolver (_mtp_too_small), the auto-fit VRAM reserve, the
drafter auto-download decision, and the reload-skip mirror via a
has_separate_drafter flag on _auto_mode_drops_mtp. Embedded sub-3B heads
(Qwen) still drop to ngram-mod. A drafter the binary cannot build (older
prebuilt, or a CUDA kernel limit) still aborts the spawn and the load
retries once without speculative decoding.
Adds the full Qwen3.5 + Gemma-4 (regular and QAT) auto/off/forced resolver
matrix, plus explicit sub-3B exemption tests.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Always compare the separate drafter in the reload-skip mirror
The sub-3B wrapper around the drafter compare could skip it when the drafter
was deleted out from under a running sub-3B server (detected None, stored set),
leaving a stale launch. The resolved-path compare is cheap and already handles
every case, so drop the _auto_mode_drops_mtp guard (and its now-unused imports)
and always compare when the mode can use a drafter and the user does not own
--spec-type. Addresses review feedback on #6191.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* -Added catch for unsloth running inside of system32 (for windows)
- changed import statement to import os as _os instead of only os.path as _osp
- replaced __osp mention with __os.path
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Adressed geminis codereview. Did not change Error message as "System32" is precise enough and the _system32 variable is lowercase only
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Etherll <61019402+Etherll@users.noreply.github.com>
* Studio: fetch the release source asset for exact (mix) source builds
The source-build fallback rebuilt the codeload/archive URL from the
source repo and commit. A mix build's merged tree is never pushed to any
repo (it ships only as the release's llama.cpp-source-commit-<sha>.tar.gz
asset), so codeload 404s on the merge commit and an uncovered host could
not build from source. When an exact-source asset exists, fetch it
directly from the release and keep codeload as the fallback for vanilla
builds whose commit is real.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Keep audio feature extractors right padded when loading processors
FastBaseModel.from_pretrained passes padding_side=left to
AutoProcessor.from_pretrained for generation, and ProcessorMixin forwards
the kwarg to every sub-component, including audio feature extractors.
Stock transformers right-pads audio: frame-validity masks assume trailing
padding. The leaked left padding shifts mel content to the end of the 30s
window for Whisper and gives Gemma 4 one extra valid mel frame on clip
lengths off the hop boundary, desyncing audio features from placeholder
tokens and crashing training on transformers 5.5.0 to 5.9.x with 'Audio
features and audio tokens do not match'.
Reset the feature extractor to right padding at the single processor
finalization point. Text tokenizer padding stays left. Verified on
transformers 5.5.0: the loaded processor now matches a fresh stock
AutoProcessor exactly, and a 9-combination Gemma 4 audio forward probe
goes from 4 failures to none.
* Add explicit None check before reading feature extractor attributes
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten comments
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix: ignore unsupported env proxy during Studio startup
* fix: handle missing socksio env proxy at startup
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Match printf logging style and inline the proxy predicate for PR #6102
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Installer: harden GPU detection follow-ups after #6174
Ports the NVIDIA-priority and /proc/driver/nvidia/gpus hardening from #6174
to the remaining pathways and adds recovery for already-poisoned venvs:
- install_python_stack.py: add _ensure_cuda_torch so 'unsloth studio update'
force-reinstalls CUDA torch when the venv carries a ROCm build on an NVIDIA
Linux host (the pre-#6174 poisoning signature). Honors UNSLOTH_TORCH_BACKEND,
UNSLOTH_ROCM_TORCH_INSTALLED, and CUDA_VISIBLE_DEVICES=-1/'' opt-outs; never
touches healthy CUDA, deliberate CPU wheels, macOS, or Windows.
- install_llama_prebuilt.py: detect_host gains the /proc NVIDIA fallback and
skips ROCm probes when NVIDIA is usable; forwarded --rocm-gfx/--has-rocm
overrides still win.
- setup.sh: GPU summary classifies NVIDIA first through a timeout-bounded
probe with the /proc fallback; AMD probes are bounded and gain a KFD
vendor_id 4098 fallback; the llama.cpp source build only selects
GGML_CUDA/GGML_HIP when the matching GPU is actually detected.
- install.sh: bound both nvidia-smi calls with a 10s timeout (no behavior
change when healthy or when the timeout binary is absent); classify the
exported UNSLOTH_TORCH_BACKEND on the final index path segment so custom
mirrors containing 'rocm'/'gfx' in their base path are not mislabeled.
- install.ps1 + setup.ps1: NVIDIA probes now require a real 'GPU N:' row from
nvidia-smi -L under a 10s bound instead of bare exit code 0; later CUDA
version and compute_cap queries are bounded too.
Tests: 3 new test files (50+ tests), suite at 788 passed.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix Resolve-CudaToolkit driver probe for extracted-function unit test
tests/studio/test_resolve_cuda_toolkit.ps1 extracts Resolve-CudaToolkit alone
into a child pwsh and stubs nvidia-smi with a .ps1 script. The bounded runner
is not in scope there (and ProcessStartInfo cannot dispatch .ps1 stubs), so
the DriverMaxCuda parse silently returned nothing and the major-mismatch
scenarios failed. Fall back to direct invocation when Invoke-NvidiaSmiBounded
is unavailable; production setup.ps1 always has it defined and keeps the
10s bound.
* Treat CUDA_VISIBLE_DEVICES empty or -1 as hidden in NVIDIA-first guards
The NVIDIA-first guards added in this branch only special-cased
CUDA_VISIBLE_DEVICES=-1 at two setup.sh gates and ignored the empty-string
form entirely, while the Python detector (install_llama_prebuilt.py)
already treats both as hidden. On a mixed AMD+NVIDIA host steered to the
AMD card via CUDA_VISIBLE_DEVICES, the guards suppressed the AMD probes,
so setup.sh fell to a CPU llama.cpp build and install.sh picked CUDA
wheels instead of ROCm.
Move the policy into the helpers so every consumer agrees:
- install.sh: new _cvd_hides_nvidia checked first in _has_usable_nvidia_gpu
- studio/setup.sh: same via _setup_cvd_hides_nvidia; the two ad-hoc
CUDA_VISIBLE_DEVICES=-1 gate conditions are now redundant and removed
- studio/install_python_stack.py: _has_usable_nvidia_gpu returns False
when CUDA_VISIBLE_DEVICES is set to or -1 (whitespace tolerated)
Tests: 5 new sh scenarios (hidden via , -1, padded -1, visible device,
and mixed host with hidden NVIDIA restoring the ROCm route) plus a pytest
class covering all three implementations behaviourally.
Addresses the review comment on the NVIDIA-first setup.sh block.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Retrigger CI after PyPI 503 outage during the previous run
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
The import-hoist safety step built its changed-file list with a two-dot
diff against the base branch tip on a shallow clone. Once the base
branch moves past a PR's branch point, that diff includes every file
the base changed since, and the verifier compares newer base code
(BEFORE) against the PR's older snapshot (AFTER). This time-reversed
comparison reports the base branch's own refactors as blockers on PRs
that never touched those files, forcing branch updates or admin merges.
Resolve the true merge-base through the compare API, fetch that single
commit by SHA (clone stays shallow), and use it for both the file list
and --before. Verified against the PR 6137 failure: with the base tip
the step flags studio files from a later main commit; with the
merge-base the file list contains only the PR's own files and the
verifier passes.
* Add FastDiffusionModel slow path for text-diffusion models (DiffusionGemma)
Text-diffusion models (DiffusionGemma) use a block-diffusion generate loop and a novel
backbone, so Unsloth's autoregressive kernel/compile patching does not apply. FastDiffusionModel
loads the unmodified HuggingFace model (outputs stay bit-identical to transformers) and adds only
the safe conveniences: 4bit/8bit loading, PEFT LoRA (attention + dense MLP; the fused 3D MoE
experts are noted as a follow-up), the (model, tokenizer) API, and for_inference/for_training.
FastModel.from_pretrained auto-routes diffusion model_types to this path, including a fallback that
aliases the legacy "diffusion_gemma" config to the "diffusion_gemma4" classes current transformers
ships. New file unsloth/models/diffusion.py; loader.py adds the dispatch and the slow-path-aware
get_peft_model / for_inference / for_training.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* FastDiffusionModel: use unsloth's standard dtype resolution
Match the FastModel loader instead of the one-line bf16/fp16 pick:
SUPPORTS_BFLOAT16 = is_bfloat16_supported(), default to bf16 when supported
else fp16, downgrade an explicit bf16 on unsupported hardware with a warning,
and assert a valid dtype.
---------
Co-authored-by: danielhanchen <danielhanchen@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: offer the in-app llama.cpp update for source-build (markerless) installs
Source-build installs have no UNSLOTH_PREBUILT_INFO.json marker, so freshness
reported supported=False and the Update button never showed (notably on macOS,
where the fork shipped no prebuilt before b9585 and setup fell back to a source
build). When an install has no marker but an official prebuilt now exists for
the host, surface the update and let one click swap it in place.
- install_llama_prebuilt.py: published_repo_for_host() (the setup.sh host->repo
rule in Python) and a --resolve-prebuilt mode that reports whether a prebuilt
exists for this host without downloading.
- llama_cpp_update.py: markerless branch in get_update_status/start_update,
version-suppressed so source builds already newer than latest are not nagged;
fail-open throughout.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: run llama update detection off the event loop, expose source_build
The markerless source-build check probes the host and reads GitHub, so run
get_update_status and start_update in a worker thread to keep the API
responsive. Expose source_build in the status response so the banner can label
the source-build switch.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Keep the llama-route auth stub out of sys.modules for the rest of the suite
test_llama_route.py replaced sys.modules['auth.authentication'] with a
bare stub at collection time and never restored it, so every later test
importing create_access_token got the stub: 17 failures across
test_desktop_auth, test_middleware, test_openai_tool_passthrough and
test_rag_preview on all four Backend CI Python versions. Import the
real module when its deps are available and only stub in minimal envs,
popping the stubs after the standalone route load either way.
* Studio: address review on the source-build update path
- published_repo_for_host: route CPU-only Windows to ggml-org too (mirrors
setup.ps1; the fork ships no win-cpu bundle), macOS always the fork.
- markerless detection compares/display the upstream llama_tag, not a possible
fork wrapper release_tag, so a source build is not wrongly judged newer.
- do not offer when there is no resolvable install root (a pinned
LLAMA_SERVER_PATH outside a managed dir): an apply would not take effect.
* Ignore version probes in the update tests' subprocess capture
The status polls in these tests trigger the new source-build detection,
which shells out to llama-server --version through the same patched
subprocess.run. On slow runners that probe lands after the installer
call and clobbers the single captured argv, failing the flag
assertions (seen on the 3.10/3.11 Backend CI jobs). Skip probe calls
in all three fakes so only the installer invocation is captured.
* Skip markerless re-detection while the update job is swapping the tree
On a source-build install the frontend polls update-status every 3s
during an apply, and each poll ran _source_build_status, which execs
the very llama-server binary the job is concurrently replacing. On
Windows that exec can hold the exe long enough to fail the installer's
os.replace; everywhere it is a per-poll subprocess spawn for a status
the poller does not read (it only consumes job progress). Gate the
markerless branch on the job not running; the marked path is probe-free
and still returns the live job state.
* Studio: tighten source-build update root, repo routing, and downgrade guard
Only manage a markerless install when the active binary lives under a
resolvable llama.cpp root (marker dir, UNSLOTH_LLAMA_CPP_PATH it sits in,
or a llama.cpp ancestor); a pinned LLAMA_SERVER_PATH or a PATH/system
binary is left alone so an apply cannot install where it would not take
effect. Gate start_update on the same suppression as detection so a
direct POST cannot downgrade a source build newer than the latest
prebuilt. Route Linux hosts with AMD tooling (rocminfo/amd-smi/hipconfig/
hipinfo) to the fork in --resolve-prebuilt, matching setup.sh, so a HIP
source build is not offered an upstream CPU prebuilt.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: cover inactive env root and pinned llama.cpp checkout in update root tests
---------
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
#5963 folded the manifest resolver into the simple-path resolver,
removed the installer's --simple-policy flag, and moved macOS prebuilts
to the unslothai/llama.cpp fork. The MLX CI prebuilt step still passed
the deleted flag, so argparse exits 2 and the workflow has been red on
every main push since. Point the step at the fork with no extra flags,
exactly like studio/setup.sh on macOS.
* Studio: forward preserve_thinking + reasoning_effort on the OpenAI passthrough
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Provide _request_reasoning_kwargs on the responses passthrough test backend mock
The OpenAI passthrough body builder now asks the active backend for
capability-gated reasoning kwargs. The responses stream adapter test fakes
the backend with a bare SimpleNamespace, so give it the same method a
non-reasoning template would expose (returns None, keeping
chat_template_kwargs out of the captured body).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Shorten the backend mock comment
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
PR #6153 taught the chat UI driver to find Compare chat inside the
decluttered plus menu (#6140) but the extra UI driver kept looking for
the old top-level item, so all three Studio UI CI workflows have been
red on main since (Compare nav not found). Mirror the same
hover-then-click More fallback here.
* fix: prevent ROCm torch from installing on NVIDIA Linux hosts
NVIDIA's open kernel module (driver 560+) registers GPU topology nodes in
the KFD sysfs hierarchy with non-zero gpu_id values. The _has_amd_rocm_gpu
(install.sh) and _has_rocm_gpu (install_python_stack.py) sysfs fallbacks
previously treated any non-zero gpu_id as proof of an AMD GPU, so an
NVIDIA-only host with the open kernel driver was misrouted to the ROCm
install path, replacing the correctly-installed CUDA torch with ROCm wheels.
Fixes:
1. install.sh _has_amd_rocm_gpu sysfs fallback: require vendor_id 4098
(AMD 0x1002) in the KFD node properties file before declaring an AMD
GPU present. NVIDIA KFD nodes carry vendor_id 4318 (0x10DE) and are
now skipped.
2. install_python_stack.py _has_rocm_gpu sysfs fallback: same vendor_id
guard. Also preserves the existing fallback for older kernels that
don't ship a properties file (trusts gpu_id alone there).
3. install.sh now exports UNSLOTH_TORCH_BACKEND ("cuda"/"rocm"/"cpu")
immediately after get_torch_index_url() resolves the wheel family.
install_python_stack.py reads this as _TORCH_BACKEND and short-circuits
_ensure_rocm_torch() entirely on cuda/cpu hosts, providing a second
layer of defense that is independent of subprocess GPU detection.
Tests: 9 new cases in TestHasRocmGpuKfdVendorGuard,
TestEnsureRocmTorch, and TestInstallShStructure cover all three changes.
Full test_rocm_support.py suite: 289 passed, 2 skipped, 0 failed.
Closes#6172
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: show actual torch backend in progress step labels
The 'ROCm torch check' and 'ROCm torch (final)' step labels were
hardcoded regardless of whether the installer was targeting CUDA, ROCm,
or CPU. On NVIDIA hosts they showed 'ROCm' even though no ROCm wheels
were being installed, which was misleading.
Add _torch_step_label(suffix) which reads UNSLOTH_TORCH_BACKEND (set by
install.sh) and formats the label as e.g. 'torch check (cuda)' or
'torch final (rocm)'. Falls back to live GPU detection for standalone
studio update runs that bypass install.sh.
* fix: make KFD sysfs vendor check conservative -- skip if no properties file
The previous implementation fell through to `return True` when the KFD
node's properties file was missing (OSError), intending to support older
kernels. But NVIDIA open driver KFD nodes can also lack a properties file
on some kernel versions, so the fallback still produced a false positive.
Change the `except OSError: pass` to `continue` so any node without a
readable properties file is skipped rather than trusted. KFD properties
files exist on every kernel version that actually exposes /sys/class/kfd,
so this does not regress real AMD GPU detection -- if the directory exists
at all, properties files will be present for genuine GPU nodes.
* fix: bulletproof NVIDIA vs AMD GPU detection
Four changes that together ensure ROCm torch can never be installed on an
NVIDIA host regardless of which detection path fires:
1. _has_rocm_gpu() (Python): NVIDIA guard at the top -- returns False
immediately when _has_usable_nvidia_gpu() is True, blocking rocminfo,
amd-smi, and KFD sysfs from producing a false positive even when ROCm
tools are co-installed alongside the NVIDIA driver.
2. _has_amd_rocm_gpu() (install.sh): same NVIDIA guard -- calls
_has_usable_nvidia_gpu first and returns 1 if it succeeds.
3. _has_usable_nvidia_gpu() (Python): adds /proc/driver/nvidia/gpus/
sysfs fallback. The NVIDIA driver populates this directory on Linux
regardless of nvidia-smi state, so a subprocess PATH gap, timeout, or
driver initialisation race can no longer silence NVIDIA detection.
4. _has_usable_nvidia_gpu() (install.sh): same /proc/driver/nvidia/gpus
fallback, tried after nvidia-smi -L rather than instead of it.
Together: NVIDIA wins at every decision point. If nvidia-smi works, it
confirms NVIDIA. If it fails, /proc/driver/nvidia confirms NVIDIA. If
somehow both fail, _has_rocm_gpu still checks NVIDIA first before any AMD
path runs.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: two KFD/proc-only corner cases from Codex review
1. KFD awk state not reset per node file (Ryzen+NVIDIA false positive):
The awk glob processes all topology node properties files in one pass.
Without FNR==1 reset, a Ryzen+NVIDIA host where an AMD CPU-agent node
sets amd=1 (vendor_id 4098, gpu_id 0) can combine with a later NVIDIA
node setting gpu=1 (gpu_id > 0), triggering found=1 before vendor_id
4318 is seen. Added FNR==1{ gpu=0; amd=0 } to reset per file.
2. proc-only NVIDIA not reaching CUDA wheel selection:
_has_usable_nvidia_gpu returning true via /proc/driver/nvidia fallback
left _smi empty, so get_torch_index_url entered the AMD/CPU branch and
selected CPU wheels despite NVIDIA being confirmed. Introduced
_nvidia_detected flag (separate from _smi) so the AMD branch is skipped
whenever NVIDIA is confirmed by any path, while _cuda_ver reads from
_smi when available (with the existing cu126 fallback when _smi is absent).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Follow-up to #6097. The update banner appeared 8s after load and auto-hid after
about 10s. Show it about 1s after a newer prebuilt is detected and keep it up
until the user dismisses it (click outside or the X) or runs the update; it
stays during an in-progress update so the progress is visible.
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
* Studio: gracefully disable MTP when the model has no head or drafter
Selecting MTP or MTP+Ngram in Speculative Decoding on a GGUF with no nextn
head and no separate drafter aborted the whole load. llama-server does not
no-op an empty draft-mtp request: it exits with 'failed to measure MTP
context memory: failed to create llama_context', surfaced to the user as a
generic 'llama-server failed to start. Check that the GGUF file is valid
and you have enough memory.'
Build-time fix in _build_speculative_flags: when a forced mtp / mtp+ngram
mode targets a model with no MTP head and no drafter (is_mtp_model is
False), default back instead of emitting draft-mtp. mtp falls back to
--spec-default; mtp+ngram keeps the ngram-mod half, which needs no head.
Real MTP models (embedded head or separate drafter), sub-3B MTP overrides,
and the auto path are unchanged.
Runtime hardening: the existing post-launch MTP retry only fired for
separate-file drafters (--model-draft in spec_flags), so an embedded-head
model that the binary cannot build still hard-failed. Gate the retry on the
spec block requesting MTP, recognise the embedded-head abort strings
('failed to measure MTP context memory', 'failed to create llama_context'),
and make the drafter name None-safe in the warning.
Tests: extend the resolver matrix (forced mtp / mtp+ngram on a non-MTP
model) and add two cases asserting the default-back emission.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Adds an in-app "Update llama.cpp" banner and button to Unsloth Studio. When the installed prebuilt is behind the latest published release, a non-invasive banner appears; clicking Update downloads the latest prebuilt for this host and swaps it in place in the background, with no restart.
Detection reuses the freshness check from #5529. The update re-runs install_llama_prebuilt.py the same way setup.sh and setup.ps1 do after #5963: it forwards the published repo and the AMD gfx target derived from the install marker, and does not pass the removed --simple-policy or the arm64-only --cpu-fallback.
While the installer swaps binaries the backend enters a maintenance state (flag set under the serial load lock, active server unloaded) so a concurrent load cannot start a server from a half-swapped binary; the next load uses the new build. The banner also handles refused responses and jobs started in another tab so it never sticks on "Updating...".
Verified end to end on an NVIDIA B200: installed b9493, detected the update, applied it, and confirmed the binary at the same path advanced to b9585 in the same process. Hermetic backend tests and the frontend type-check pass.
* feat(hub): enable Run/New Chat for downloaded GGUF models, fix README bottom spacing
- enable the Run / New Chat action for GGUF models that are already downloaded, across the download card, on-device card, and hub page
- remove the extra bottom spacing under the model inspector README
* fix(hub): sync active GGUF variant on mount
* feat(hub): open a new chat immediately when Run is clicked
---------
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
* Chat template: restore the preview box with a smaller font
Bring back the template preview under the Chat Template label at 10px
instead of 13px, clamped to four lines. The label is a plain span
again; the preview box and the edit icon open the editor.
* Circular hover for chat action buttons; smaller template edit icon
- Message action bar buttons (copy, retry, delete, more) and the
branch picker chevrons hover as circles instead of 10px rectangles
- The three dot button next to sidebar chats gets the same circular
hover and open state
- Chat template pencil icon drops from 14px to 12px
* Chat template: drop the preview box, add Reset to the editor dialog
- Remove the template preview under the Chat Template label; the row
is just the label with the revert and edit icons
- The Apply and Reset buttons in the panel no longer trigger on
template-only changes, they remain for model settings that need a
reload
- The editor dialog gets a Reset button that restores the default
template in the draft, disabled when the draft already matches
* Sidebar: Gemini-style row vs action hover; clickable template label
- Hovering the 3 dot action next to a chat shows only the action's
hover circle; the row pill highlight is suppressed while the action
is hovered and applies only when hovering the row itself
- Chat Template label opens the editor and the gap below the row is
slightly tighter
* Plus menu More submenu styling and order; sidebar action contrast
- The More submenu was missing the unsloth-plus-menu class, so its
items hovered with the green accent instead of the shared grey
- Saved prompts moves above Compare chat
- Opening the 3 dot menu no longer highlights the whole chat row, only
the action circle; the circle is a step darker than the row hover so
the two read separately
- Panel icon buttons (template pencil, revert) hover as circles
* Chat template: confirm saves with a toast
Template-only edits no longer surface the panel Apply button, so a
saved override sat pending with no indicator. Saving now shows a toast
stating the change applies on the next model reload, with a separate
message when the save clears the override.
* Chat welcome: raise the greeting block to 27.5vh
* Composer: tighten trailing padding on caret pills
The RAG and MCP pills end in a chevron that carries its own
whitespace, so their hover pill looked over padded on the right.
Pills with a caret drop from 10px to 6px right padding; label only
pills keep the wider padding.
* Compact RAG and MCP pills open their menu instead of toggling off
When the composer collapses pills to icons, the RAG and MCP glyph is
the whole button, so its turn-off click handler made the menu
unreachable. In compact mode the glyph click now falls through to the
dropdown trigger, and the hover X swap is skipped since the click no
longer turns the pill off. Full size pills keep the icon-as-off-switch
behavior.
* Compact pills: name tooltip on hover; pill shaped tooltips
- Collapsed composer icons (Search, Code, Images, Fetch, Canvas, RAG,
MCP) show their name in a hover tooltip via data-pill-label, since
the label itself is hidden in compact mode
- Tooltips (.tooltip-compact and the new pill tooltip) are rounded
pills instead of 10px rectangles
* System prompt edit icon and reset; soft borderless dialogs
- System Prompt section header gets the same pencil icon as Chat
Template, opening the prompt editor; CollapsibleSection grows an
optional headerAction slot so the icon is not a button inside the
toggle button
- The prompt editor dialog gets a Reset button that clears the draft,
disabled when already empty
- All dialogs swap the border for a shared dialog-soft-surface class:
borderless with the chatbox shadow in light mode, flat card surface
with no shadow in dark mode
* Editor dialogs: borderless text areas
The system prompt and chat template textareas drop their border to
match the borderless dialog surface; the soft fill alone defines the
input area.
* System prompt: tighten the gap between the header and the text box
* Studio: fall back to text-only when llama.cpp is too old for a model's vision projector
Loading a GGUF vision model starts llama-server with --mmproj <projector>. When the installed llama.cpp prebuilt predates the model's projector format, llama-server aborts at startup with 'clip.cpp: Unknown projector type' (exit -6), and the whole load failed even though the base GGUF is a fine text/tools chat model. Seen with gemma-4 on a 3-day-old prebuilt (build b9496).
load_model now retries the launch once without --mmproj when the captured startup output indicates a projector-format incompatibility. The retry runs the model text-only, marks the session non-vision (is_vision False, mmproj audio dropped) so the status/capabilities the frontend reads stay consistent, and warns the user to update llama.cpp. Detection is generic, not model-specific, and conservative: OOM, bad GGUF, port-bind, missing-file and other failures keep their existing handling and never retry. If the text-only retry also fails, it errors out with the real reason.
Adds _is_projector_incompatibility and _strip_mmproj_args (unit-tested with the real gemma-4 abort plus negatives) and extracts _start_llama_process so both the initial start and the retry share one spawn path and each logs its argv.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Reformat mmproj fallback files to match main (ruff line-length 100 + kwarg spacing)
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
* Studio: route arm64 Linux CUDA hosts to linux-arm64-cuda prebuilts
* Studio: SM-aware selection for windows-cuda app bundles
* Studio: select published ROCm bundles by gfx target (linux + windows)
* Studio: route macOS installs to the fork's prebuilt bundles
* Studio: fix windows cuda13 driver-13.0 gate and ROCm gfx prefix overreach
* Fix Blackwell Windows pin shadowing native app-bundle (b9360 over b9457)
* Match Windows cuda12 driver floor to Linux (12.x minor-version compat)
* Fix Windows app-bundle dropped when runtime DLLs come from torch/lib
* Fold the manifest resolver into the simple-path resolver (one entry, no dormant full path)
* Remove unused UNSLOTH_LLAMA_PUBLISHED_REPO override
* Route Windows GPU hosts to the fork prebuilts in setup.ps1
* Document sm_103 path divergence and mark --simple-policy as a no-op
* Note sm_103 coverage now comes from the producer manifest
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Remove the now-vestigial --simple-policy flag (one resolver handles all hosts)
* Unify the fork onto the manifest path; drop the linux-x64 filename path and hardcoded coverage tables
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Strip whitespace from manifest gfx_target/mapped_targets when parsing
* Windows CUDA: sort coverage-unknown bundles last so they can't outrank targeted ones
* Share the SM-coverage sort key between the linux and windows selectors via _sm_range
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: reject approved releases with an exact source archive but no source repo to clone from
* Studio: accept the fork's windows-rocm kind in the Windows reinstall check
* Studio: accept a manifest-bundle source repo in the exact-source release check
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: route Linux hosts to the fork only when a usable GPU is present
* Fix Windows AMD lemonade tag resolution for PR #5963
The fork release scan passes each scanned release's upstream tag
(b9518, ...) to the lemonade lookup, but lemonade publishes its own tag
series (b1292, ...) that never contains upstream tag numbers. On a
Windows AMD host every scanned release therefore 404s the lemonade
fetch twice, the upstream HIP zip is dropped by the approved-hash gate,
and the scan walks the whole release history until it dies on the
unauthenticated GitHub rate limit or falls to a HIP source build. The
Linux path already passes the requested tag ("latest") and works.
Thread the requested tag through resolve_release_asset_choice ->
resolve_asset_choice -> resolve_upstream_asset_choice as lemonade_tag,
used only by the lemonade lookups. Upstream asset names keep the
concrete per-release tag and all new parameters default to the old
behavior.
Verified on a gfx1151 box: before, the native Windows install scanned
b9518..b8811 and aborted on rate limit; after, it selects
llama-b1292-windows-rocm-gfx1151-x64.zip (lemonade) from fork release
b9518, passes staged validation, and the installed llama-server
enumerates ROCm0. WSL keeps selecting the matching ubuntu bundle.
Adds a regression test pinning that the Windows fork path resolves
lemonade via /releases/latest, never /releases/tags/<fork-tag>.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Plan lemonade for Linux ROCm hosts on the ggml-org direct path for PR #5963
Audit follow-up to 72f32364 across the other selection pathways. The
ggml-org direct planner kept its lemonade attempt for Windows ROCm
hosts but planned only the CPU tarball for Linux ROCm hosts, so an AMD
Linux box routed to ggml-org (for example a --published-repo override)
silently installed the CPU build. That lemonade planning used to live
in the --simple-policy dispatcher this PR removed.
Add the lemonade attempt ahead of the CPU tarball in the Linux x86_64
branch, mirroring the Windows branch, with the lookup keyed to the
requested tag. Adds a regression test asserting lemonade is the first
attempt for a Linux ROCm host on the direct path.
Also re-verified the other pathways on a gfx1151 box: the fork-routed
flows pass the requested tag everywhere, repeat runs over an existing
lemonade install correctly skip with "already matches selected release
b9518" on both native Windows and WSL, and macOS, CUDA and CPU
selection are untouched. Suites: 328 passed on Linux, Windows matches
the pre-existing baseline.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
* Studio: sync detected model capabilities into models[] after load
The chat composer gates audio upload on activeModel.hasAudioInput, but
/api/models/list omits audio fields for default and active-GGUF entries
and the single chat load path never wrote the load response's
capability flags back into the store. Audio-capable models such as the
Gemma 4 GGUFs therefore never unlocked audio input in the main chat,
while the compare composer (which does sync) worked.
Add syncModelCapabilities and call it after a successful load and after
the status fetch in refresh, so the flags also survive F5 and are not
clobbered by stale catalog data.
* Studio: merge audio upload into the Add photos & files picker
Remove the separate Upload audio row from the composer plus menu and
register an AudioAttachmentAdapter in the shared attachment pipeline,
so the standard picker and drag-drop accept wav, mp3, m4a, ogg, flac
and webm directly. Gating matches images: the picker always lists
audio and models without audio input get a toast at add() time. The
50MB limit is kept and the file shows as a normal attachment chip.
On send the adapter emits an audio content part on the attachment and
findLatestUserAudioBase64 now also scans attachment content, so the
request still carries audio_base64 exactly as before.
* Studio: extract AudioAttachmentAdapter into its own module
Move the adapter out of runtime-provider.tsx so it is importable in
isolation, export the audio send-path and capability-sync helpers for
tests, and guard attachment id generation for non-secure contexts
(crypto.randomUUID is undefined over plain HTTP on a LAN, matching the
existing guard in startCompare).
* Studio: do not claim .webm by extension in the audio adapter
A video/webm file would match the .webm extension entry and route to
the audio adapter. Real audio webm (MediaRecorder output) always
reports the audio/webm MIME, so matching webm by MIME only keeps video
files out while keeping recorded audio working.
* Studio: only send audio from the newest user message
audio_base64 switches the backend onto the audio generation path
(generate_whisper_response ignores chat messages entirely and
generate_audio_input_response bypasses the normal streaming path), so
replaying audio from an older turn hijacked text-only follow-ups:
Whisper would retranscribe the stale clip instead of erroring cleanly,
and audio VLMs lost tools and streaming. Stop the scan at the newest
user message, matching the consumed-on-send semantics of the legacy
pendingAudio path. Regenerating the audio turn itself still resends
its audio since it is the newest user message in that run.
Also guard extractAudioPartBase64 against null parts in deserialized
history content.
* Studio: forward audio input to llama-server for GGUF models (#6096)
* Studio: forward audio input to llama-server for GGUF models
* Studio: harden GGUF audio input handling (multi-format decode, size cap)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: carry GGUF audio in the message list so it works with tools
* Studio: bound decoded audio length and make the soundfile decoder optional
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Handle audio attachment edge cases
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: gate audio file picker by loaded model capability (#6142)
* Gate audio attachments by loaded model
* Use conditional spread for audio attachment adapter
* Preserve audio fallback while filtering picker
---------
Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local>
Co-authored-by: oobabooga <oobabooga4@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: imagineer99 <samleejackson0@gmail.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
* Studio: support separate-file MTP GGUF drafters (Gemma 4)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: fix review findings for separate-file MTP drafters
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: pair local MTP drafters by name and include them in reload dedup
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: manage --model-draft in extras and reject MTP/ copies as models
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix Studio Python, Gemma 4 Unified sidecar, and worker crash messages
* Clean up Gemma 4 sidecar test patch contexts
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Polish inference worker crash message
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address transformers tier review feedback
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Route Gemma 4 assistant models to transformers 5.10
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Two hardening fixes from the fleet-validation audit.
Blackwell Windows hosts drop windows-cuda attempts that cannot offload
sm_120 instead of leaving them ranked behind the b9360 pin. A cuda-12.4
upstream build loads and passes the functional validator but runs the
model on a slow non-native path (an RTX 5090 measured 7.1 tok/s vs
551.2 on cuda-13.3), so one failed pin download away from that is too
close. The coverage check now also reads manifest SM metadata first, so
published cuda12 app bundles (toolkit 12.8, sm_120 included) stay
selectable and make the pin go dormant correctly.
The fork-release Linux planner no longer appends the linux-cpu bundle
for NVIDIA hosts whose CUDA selection produced nothing; it raises so
the caller walks back to an older release with a usable CUDA line,
mirroring the deliberate ROCm policy. Today's walk-back only works
because partial releases ship no CPU bundle; this keeps it working if
a future partial release does.
* Studio: training survives a non-writable HF datasets cache
A shared HF datasets cache can contain subtrees owned by another user
(for example populated by an earlier root-run job). datasets then dies
with "[Errno 13] Permission denied: ..._builder.lock" while locking
the cached builder and the training run fails. load_dataset in the
training worker and trainer now goes through a wrapper that catches the
EACCES and rebuilds the dataset in a Studio-owned cache under
cache_root()/hf-datasets, logging the fallback.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Scope the HF_DATASETS_CACHE override to the fallback load
* Route non-streaming dataset preview loads through the cache-safe wrapper
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: mascot images degrade gracefully instead of showing alt text
Mascots loaded from raw public-folder URLs with no error handling, so a
missing or unreachable file (version skew during updates, subpath
mounts, transient blips) painted the alt text, like the empty mascot on
the training start overlay. A shared MascotImg now resolves against
BASE_URL, retries once with a cache-buster, then swaps to a 4 KB
fallback sloth bundled as a data URI that cannot 404. Applied to the
chat greeting, 404 page, training start overlay, auth form, onboarding
splash and wizard.
* Reset mascot retry state via key remount instead of render-time tracking
* Use MascotImg for the artifact generating panel sloth
* Call patch_compiling_bitsandbytes on the FastLanguageModel dispatch path
* Guard the FastLanguageModel patch call so an old unsloth_zoo cannot crash loads
* Auto-set BNB_ROCM_VERSION from the installed wheel on Windows + ROCm
bitsandbytes derives its ROCm backend DLL name from `torch.version.hip`.
AMD's Windows bitsandbytes prerelease wheel ships a single
`libbitsandbytes_rocm<NN>.dll` whose suffix does not always match the torch
HIP version: e.g. `torch==2.11.0+rocm7.13.0` reports HIP 7.13, so bitsandbytes
looks for `libbitsandbytes_rocm713.dll`, but the wheel only ships
`libbitsandbytes_rocm72.dll`. The names disagree, the native library fails to
load, and every 4-bit / 8-bit path breaks for users running `import unsloth`
directly (Unsloth Studio already works around this in its worker).
Detect the suffix from the actually-installed wheel and pin BNB_ROCM_VERSION
before bitsandbytes is first imported (unsloth_zoo.device_type imports it during
`from .models import *`), so the correct backend loads. This is precisely the
override bitsandbytes itself recommends when the build/runtime ROCm versions
differ.
Strict no-op unless ALL of: running on Windows, a ROCm torch build, the var is
unset, and a `libbitsandbytes_rocm*.dll` is actually installed. Linux ROCm is
untouched (its multi-backend bitsandbytes resolves the backend correctly from
torch.version.hip). Honors a user-provided BNB_ROCM_VERSION and an explicit
opt-out (UNSLOTH_SKIP_BNB_ROCM_VERSION=1).
Verified on an AMD Radeon 8060S (gfx1151, Strix Halo) Windows 11 + ROCm box:
`import unsloth` now auto-sets BNB_ROCM_VERSION=72 and a native 4-bit
quantize/dequantize roundtrip succeeds with the var unset; previously it failed
to load `libbitsandbytes_rocm713.dll`.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Gate BNB_ROCM_VERSION on the actual torch build, not runtime hints
_is_rocm_torch_build() falls back to environment and filesystem hints
(HIP_PATH, ROCM_PATH, ...) that are routinely present on Windows boxes
with the AMD HIP SDK installed but a CUDA or CPU torch. If such a box
also has a bitsandbytes wheel that ships a rocm DLL (AMD's Windows
prerelease wheel ships rocm72 alongside all the cuda DLLs), setting
BNB_ROCM_VERSION makes bitsandbytes raise at import on its CUDA build
and `import unsloth` breaks.
Add _is_hip_torch_build(): wheel version tag first (no torch import),
then torch.version.hip for untagged custom/source HIP builds, and use
it as the gate. The broader hint-based helper keeps its other callers.
Verified on a gfx1151 Windows box: True on the ROCm venv
(2.11.0+rocm7.13.0), False on a torch-less interpreter; 4 new unit
tests including the HIP-SDK-on-CUDA-box false-positive regression.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Relocate wiring so this PR composes with the bnb arch-detection PR
Both this PR and the fix_bitsandbytes_rocm_arch_detection PR anchored
their _gpu_init.py wiring and import_fixes.py additions on the same
configure_amdgpu_asic_id_table_path lines, so whichever merged second
hit a textual conflict in both files (verified by merging both onto
main in each order).
Move maybe_set_windows_rocm_bnb_version's wiring to a self-contained
block after the import-order warning (still before `import
unsloth_zoo`, which is what pulls in bitsandbytes on ROCm) and append
the helpers at the end of import_fixes.py. The hunks no longer
overlap, so the two PRs merge cleanly in either order. No behavior
change: the env var only needs to be set before bitsandbytes is first
imported, and it still is.
* Redetect sitecustomize-seeded BNB_ROCM_VERSION for PR #5986
After #6048, every Studio venv process starts with BNB_ROCM_VERSION seeded
by the managed sitecustomize.py block, which made this gate a no-op inside
Studio venvs and blind to wheel updates. Treat values marked
UNSLOTH_BNB_ROCM_VERSION_SOURCE=sitecustomize as redetectable defaults,
stamp redetected values as detected, and keep the seeded value when no DLL
is found. Explicit caller values still win and the opt-out is unchanged.
Also merges latest main.
* Make BNB_ROCM_VERSION opt-out drop the sitecustomize-seeded default for PR #5986
UNSLOTH_SKIP_BNB_ROCM_VERSION=1 previously no-opped the helper but left a
sitecustomize-seeded BNB_ROCM_VERSION in the environment, so bitsandbytes
still consumed the override the user disabled. The opt-out now removes
values carrying the sitecustomize source marker; explicit user values have
no marker and are untouched. Adds tests for the opt-out paths and the
empty-string edge.
* Tighten comments and docstrings for PR #5986
Comment-only pass: shorten verbose docstrings on the internal helpers,
collapse multi-line inline comments, and drop wording that restates the
code. Verified code-identical via the comment_tools.py AST signature
check (3/3 files unchanged).
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix UnboundLocalError in _detect_rocm_version dpkg/rpm fallback
A leftover local import re inside the amd-smi branch made re function
local for the whole scope. When amd-smi and hipconfig are absent and
dpkg-query or rpm reports rocm-core, the epoch strip at the dpkg/rpm
fallback hit re.sub before any local binding existed and crashed the
installer with UnboundLocalError. Drop the local import (the module
already imports re at top level) and add a regression test covering the
dpkg path without hipconfig.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* tests: follow Compare chat into the More submenu in the chat UI driver
The plus-menu declutter moved the Compare chat item into a More
submenu (DropdownMenuSub in thread.tsx / shared-composer.tsx), so the
Playwright driver stopped finding it at the top level and Chat UI Tests
went red on every PR regardless of its diff. Open the More sub-trigger
(hover, then click as a fallback) before asserting the item; the direct
lookup stays first so the driver still passes if the item moves back.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Add regression guard for batched left-padded generation (#1066, #3699)
Three layers of tests plus a path-filtered CI workflow so the left-padding
position_ids / attention-mask bug class cannot silently return:
- tests/utils/test_prepare_inputs_ast_guard.py: import-free AST checks on
_fast_prepare_inputs_for_generation (cumsum-from-mask branch present,
cache_position only as fallback, no mask truncation, model families wired)
- tests/utils/test_prepare_inputs_leftpad.py: CPU behavioral unit test with
synthetic left-padded masks and fake caches; exact expected position_ids
for prefill and cached decode
- tests/utils/test_batched_leftpad_generation_gpu.py: optional GPU e2e,
solo vs batched prefix match, skipped without CUDA
- .github/workflows/batch-inference-guard.yml: ubuntu-latest CPU job running
the two deterministic layers on PRs touching unsloth/models/**
Validated: all pass on main; both CPU layers fail at 6d0f8643~1 (pre #4100)
and at 332eabf3~1 (pre #2216), reproducing the historical bug signatures.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Cite staging proof in batch-inference-guard header (staging-2 PRs 170/171)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fold left-padding guard into consolidated Core CI; merge AST + behavioral tests
No new workflow and no new CI job: the guard now runs as one HARD GATE step
inside consolidated-tests-ci.yml, right after the callback signature drift
detector, where the CPU torch stack is already installed. The AST structural
checks and the behavioral unit tests live in a single file
(tests/utils/test_prepare_inputs_leftpad.py); the AST layer stays stdlib-only
with unsloth imported lazily inside the behavioral tests, so import breakage
cannot mask the structural checks.
Revalidated after the merge: 11 assertions pass on main, 8 fail at
6d0f8643~1 (pre #4100).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update staging proof reference for consolidated gate (PRs 170/172)
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix(studio): infer mlx vlm resized image layout
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Run the HF cache redirect before import fixes that can freeze Hub constants
unsloth-zoo's redirect_hf_cache_if_readonly() repoints HF_HOME,
HF_HUB_CACHE, and HF_XET_CACHE at import time when the default cache is
read-only. In _gpu_init.py the early fix block can import vllm
(disable_broken_vllm), transformers (check_fbgemm_gpu_version), or
huggingface_hub (fix_huggingface_hub) before import unsloth_zoo runs
the redirect. huggingface_hub computes its cache constants when its
constants module is first executed, so those probes can freeze the old
read-only paths and the redirect then never reaches Hub.
Recent huggingface_hub and vllm releases import lazily, which masks
this, but older supported versions import eagerly: older vllm pulls in
transformers at top level, and on hub versions where is_offline_mode
still exists the hasattr probe materializes the constants module.
Load unsloth_zoo/hf_cache.py straight from its file (it is stdlib-only)
and run the redirect before the fix calls. The zoo's own call later is
an idempotent no-op, and an older unsloth_zoo without hf_cache.py is
skipped silently.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: danielhanchen <danielhanchen@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio UI polish: search dialog shadow, picker pills, sidebar spacing, white Hub
- Chat search dialog: keep a page-bg shadow in dark mode so the dialog
does not merge into same-color content behind it
- Model selector: hover and selected rows are fully rounded pills
- Sidebar: nudge nav items, recents and section labels 2px right and
widen the hover pill 2px left so its side gaps match
- Hub: pure white page background in light mode and drop the ambient
card glow that tinted the page gray
* Lighten sidebar border, widen model picker rows
- Sidebar border is #f2f2f2 in light mode
- Model picker rows extend 4px further left and right, lining up with
the search bar
* Plus menu: restore the intended 18px corner radius
The global 14px !important dropdown radius overrode the menu's own
18px, so the plus menu and its submenus rendered squarer than designed.
Mark the menu radius !important so it wins.
* Projects: borderless import/export button; picker search border #f2f2f2
- The import/export button on the Projects page drops its outline
border and matches the Sort by pill next to it
- The model picker search inputs use a #f2f2f2 border in light mode,
dark mode keeps the default
* Model selector: align the popup with the trigger label
The popover lined up with the trigger button edge, 14px left of the
label text. A 10px alignOffset starts it just before the label.
* Model picker: shorten search placeholder to Search models
* Pill-shaped model trigger and panel controls; chat template edit button
- Select model trigger: rounded pill with slightly tighter horizontal
padding
- KV cache, speculative decoding, draft N and preset controls in the
chat settings panel are rounded pills
- Chat Template: replace the three-line preview box with an edit icon
button next to the label; it opens the existing editor dialog
* Training overlay: borderless console, sloth attached to the box
- Drop the terminal border for the training start console
- Remove the gap between the sloth image and the console so they touch
* Settings panel: more left padding in pill controls, smaller template edit icon
- f16, Auto and draft N pills get 12px left padding, widths bumped 4px
to keep the values from truncating
- Chat template edit icon shrinks to 14px and the Chat Template label
also opens the editor
* Model picker: more padding inside row pills, hugeicons delete icon
- Row pills get 12px horizontal padding so text is not pressed against
the highlight boundary
- The cached model delete button uses the hugeicons Delete02 icon
instead of the lucide trash icon
* Address review: hub bg layering, dead shadow utilities, merged paddings
- Remove bg-background from the Hub root: the white rule lives in the
base layer and the utility was winning the cascade, so the white
background never applied
- Drop the inline shadow utilities on the chat search dialog: the
chat-search-surface rules sit later in the utilities layer and
already win, making the inline ones dead
- Merge symmetric pl/pr pairs into px on the sidebar group wrappers
Completes the allowScripts rollout: upgrade CI npm to the 11.x line
(node 22 bundles 10.x, which predates the gate) with a loud version
guard so the flag can never silently degrade into a warning, then run
npm ci --strict-allow-scripts. A dependency that introduces install
scripts not covered by the committed policy now fails the job with
npm's approve-scripts/deny-scripts instructions; the pre-commit sync
hook keeps existing pins fresh after bumps.
At UD-IQ2_XXS the temp-0 greedy answer to the capital-of-France probe is
hardware dependent: GitHub ubuntu runners deterministically answer
city=France while other CPUs answer Paris, because the 2-bit argmax
flips with the SIMD kernel path. Seeds do not rescue it: a staging-fork
sweep on the affected runners measured 1/5 Paris at temp 0.7 and 1/5 at
temp 1.0 (different winning seeds on different hardware), so a
retry-across-seeds assert would still flake about a third of runs.
The same sweep on UD-Q4_K_XL answered Paris 13/13 across temp 0, 0.7
and 1.0 with 5 seeds each, including 3x deterministic greedy. Bump the
job's quant (roughly 570 MiB to 1.1 GiB, cache key already includes the
variant) and leave the assertion exactly as it was.
* Restore config use_cache in for_inference after gradient checkpointing prep
unsloth_zoo's prepare_model_for_training now sets use_cache=False on the
model config and nested sub-configs when gradient checkpointing is on
(unsloth-zoo PR 715) and records the original values. Wire the
counterpart into both for_inference implementations so the original
values come back for inference, and re-disable in for_training when a
record exists so resumed training keeps the config consistent. Both
calls import lazily and tolerate older unsloth_zoo without the helpers,
so version skew in either direction is a no-op. The MLX shims in
__init__.py are deliberately untouched.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix(vision): validate dataset video paths before training
* fix(vision): remove redundant warnings import, add pytest tests for #5085
* fix(trainer): auto-validate video paths in UnslothVisionDataCollator on first batch (#5085)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* refactor(vision): use str.removeprefix instead of slicing (Datta0 review)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* style: replace em dashes with hyphens in error message and docstring
* Update unsloth/models/vision.py
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* Split: keep only 1 file(s)
* fix(vision): broaden video-path validation to all collator inputs
- check_dataset_for_missing_videos now accepts every example shape that
UnslothVisionDataCollator forwards to process_vision_info:
dict rows with "messages"/"conversations"/"prompt"/"completion", and raw
message-list rows. Earlier logic only handled {"messages": [...]}, so
conversations/prompt/completion datasets silently skipped validation and
raw message-list rows crashed on list.get.
- Guard against non-dict message entries; a bare string inside a message
list no longer raises AttributeError.
- Decode file:// URIs via urllib so percent-encoded paths, absolute
Windows URIs (file:///C:/...) and host-qualified URIs
(file://localhost/abs/path) map back to their real filesystem path.
- Expose an optional "checked" set so callers can reuse dedup state across
invocations.
- Docstring warns that passing a streaming IterableDataset consumes the
iterator.
UnslothVisionDataCollator now validates every batch (not just batch 0)
and applies formatting_func before validation, matching the base
collator's own ordering so formatter-generated video paths are also
checked. The already-checked set is shared across batches, so per-batch
cost stays proportional to newly seen paths.
* fix(vision): robust URI + scheme handling in video-path validator
- _local_path_from_video_value now treats anything with a "://" prefix
as a URI and validates only file:// URIs. This prevents false
FileNotFoundError on remote schemes that were silently passed through
before (s3://, gs://, hf://, ftp://, az://, ...).
- Non-localhost file authorities (e.g. file://nas-server/share/clip.mp4)
are now skipped instead of being stripped and validated against the
local filesystem; RFC 8089 only permits empty host or "localhost" for
local files.
- Drop the explicit unquote call: urllib.request.url2pathname already
unquotes, so the previous url2pathname(unquote(path)) double-decoded
any filename with a literal percent (e.g. a file named "clip%20.mp4").
- Remove the Windows drive-letter strip block; nturl2path.url2pathname
handles "/C:/foo" -> "C:\\foo" itself, leaving nothing for the guard to
match on either OS.
- Return None when the resolved path is empty (bare "file://" or
"file://hostname") so the caller skips it instead of reporting a
blank " - " entry in the error message.
- Add a runtime guard in check_dataset_for_missing_videos that warns and
returns early when handed a datasets.IterableDataset, matching the
docstring contract and preventing silent iterator exhaustion.
Windows native paths like "C:/path/x.mp4" stay valid because the
scheme check uses the "://" substring (not urlparse's single-letter
scheme surface).
* tests: consolidate video-path validation coverage into one file
New coverage for tests/test_video_path_validation.py:
- every-batch validation with cross-batch dedup (replaces the old
first-batch-only assertion which no longer matches the implementation).
- all collator-supported input shapes: messages, conversations,
prompt/completion, raw-message-list rows; non-dict message entries.
- file:// URI robustness: percent-encoded paths, localhost netloc,
non-localhost netloc skipped, bare / hostname-only URIs skipped,
double-encoded filenames single-unquote correctly.
- non-file remote schemes (s3, gs, hf, ftp, az) skipped without raising.
- Windows-style absolute path not mistaken for a URI scheme.
- formatting_func applied before validation inside the collator wrapper.
- IterableDataset runtime guard warns and returns without consuming.
The pre-existing test_collator_validates_only_once assertion has been
replaced by test_collator_validates_every_batch / dedupes_across_batches
because the wrapper now validates every batch.
The session-scoped AST fallback fixture was extended to extract the
helper functions that the rewritten check_dataset_for_missing_videos
depends on, so the Windows/no-triton code path still loads the module
surface.
* Fix CI iter 1
* Omit release-desktop.yml from the PR diff
The workflow file landed on origin/main after the PR branched off; our
fork-scoped push token cannot touch .github/workflows/**. Drop it from
this branch so the PR diff stays within the author's authorisable
surface. The file remains on origin/main and will return after this PR
merges upstream.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(vision): cache only validated paths in checked set
Missing paths were added to the dedup cache before the existence check,
so a caller that caught FileNotFoundError and retried with the same
collator/checked set would silently skip the bad path on the second call.
Only add a path to checked after os.path.isfile confirms it exists, so
missing paths are re-validated on every call until they are fixed.
* fix(ci): resolve two CI failures introduced by this PR
- Add __all__ to models/__init__.py so the HOISTED-IMPORT-UNUSED linter
check passes for check_dataset_for_missing_videos
- Replace Dataset.from_list() in test helpers with plain list literals;
the CI environment mocks datasets with a MockDataset that only has
from_dict, but check_dataset_for_missing_videos accepts any iterable
so no Dataset wrapper is needed
- Guard test_iterable_dataset_warns_and_skips with pytest.importorskip
so it skips cleanly when the real datasets package is unavailable
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(ci): correct HOISTED-IMPORT-UNUSED without breaking wildcard exports
Adding __all__ to models/__init__.py was too aggressive - it restricted
from .models import * in _gpu_init.py to only check_dataset_for_missing_videos,
hiding FastLanguageModel, FastVisionModel etc and breaking
test_fast_model_class_surface_under_spoof.
Instead: remove __all__, and add an explicit named import in _gpu_init.py
so the linter sees the symbol consumed in the re-export chain.
* Deduplicate missing video paths and skip data URIs in validator
check_dataset_for_missing_videos appended a path to the missing list on every
occurrence, so a path referenced by multiple rows was reported N times and the
error header read the wrong count. Track missing paths in a per-call set so each
is listed once, kept separate from the checked cache so retries still re-check
missing files. This restores the dedup behaviour the docstring promises and the
existing test_duplicate_paths_deduplicated test asserts.
Also skip data: URIs in _local_path_from_video_value so inline base64 payloads
are not flagged as missing files.
Add tests for the data URI case, warn-only dedup, and real integration against
the unsloth_zoo UnslothVisionDataCollator base (verifying validation gates the
base call, formatting_func is applied once and restored even when the base
raises).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(trainer): declare __slots__ on UnslothVisionDataCollator subclass
* fix(ci): hoist check_dataset_for_missing_videos to trainer module level; guard IterableDataset skip
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(vision): accept tuple message content in video path validator
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten comments and docstrings for PR #5136
Comment-only pass over the new video path validation code: shorten the
collator and validator docstrings, collapse multi-line inline comments,
and reduce test docstrings to one-liners. No code changes; verified with
comment_tools.py check --strip-docstrings (3/3 code unchanged) and the
full test suite (35 passed).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
