The main merge left listStoredChatThreads imported twice -- once as a standalone import from
the deep utils path and once via the @/features/chat barrel (which re-exports it) -- tripping
TS2300 'Duplicate identifier' and failing the Tauri frontend build. Keep the barrel import,
grouped with the other chat imports.
* Fix Windows installer torch index override
* Clear inherited uv index env vars for pinned installs in studio/setup.ps1 (#6898)
* Harden setup.ps1 index-var clearing to truly remove vars (#6898)
* Apply UV_DEFAULT_INDEX torch index fix to Linux/Mac install.sh (#6898)
* Neutralize all uv index env vars for pinned torch installs (#6898)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: add Vulkan llama.cpp support
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address gemini's feedback
* Studio: move the Vulkan VRAM probe into a standalone script
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Improve Vulkan probe error reporting
* Resolve llama-server symlink so Vulkan build is detected
* Drop unreachable Vulkan fallback in GPU free-memory dispatcher
* Skip the Intel GPU probe when NVIDIA or ROCm is present
* Reserve host RAM headroom for Vulkan integrated GPUs
* Add a `UNSLOTH_FORCE_VULKAN` environment variable
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Honor GGML_VK_VISIBLE_DEVICES, reserve discrete Vulkan VRAM headroom, and clear Intel GPU on --cpu-fallback
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Route Intel and forced-Vulkan hosts to the upstream Vulkan prebuilt, add arm64 Vulkan, keep Vulkan out of RAG auto-detect
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Clear the fork release pin when routing a Vulkan host to the upstream repo
* Gate auto-Vulkan routing on no physical NVIDIA so hidden CUDA devices aren't used
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Pin Vulkan launches with --device Vulkan<i> instead of the raw GGML_VK_VISIBLE_DEVICES index space
* Let user --device override the Vulkan pin, and gate direct Vulkan asset picks on no physical NVIDIA
* Update RAG auto-backend test mocks for the _resolve_auto binary and Vulkan probes
* Keep the add_dll_directory handle alive through the Vulkan probe DLL loads
* Revert RAG auto Vulkan guard, guard multi-backend Vulkan detection, and preserve forced Vulkan across updates
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Use getattr for RTLD_GLOBAL in the Vulkan probe CDLL mode
* Skip CUDA/ROCm APU and datacenter GPU tuning on Vulkan builds
On a Vulkan llama.cpp build gpu_indices are ggml compact ordinals, not
CUDA/ROCm physical ids, so _amd_apu_wants_unified_memory and
_apply_datacenter_env were reading the wrong device. On a mixed AMD APU
plus discrete GPU host that could raise a spurious system-RAM shortfall
and block a valid discrete-GPU load. Gate all three call sites on
not is_vulkan_backend; the Vulkan path already reserves iGPU host
headroom and the backend ignores GGML_CUDA_* anyway.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten Vulkan-guard comment in load_model
* Reduce comments in Vulkan support to be more succinct
* Resolve shell-wrapper llama-server entrypoint to the real lib dir
create_exec_entrypoint falls back to a #!/bin/sh wrapper at the install
root when it cannot symlink into build/bin. _find_llama_server_binary
returns that root entrypoint, but Path.resolve() does not follow a shell
wrapper, so _llama_lib_dir returned the install root and _is_vulkan_backend
missed libggml-vulkan.so -- silently skipping the Vulkan probe and --device
pin on an otherwise valid Vulkan install. Follow the wrapper's exec target
to build/bin. Regression test: test_shell_wrapper_entrypoint_resolves_to_real_lib_dir.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
* Studio /v1/messages: accept thinking and unknown content blocks
The Anthropic-compatible /v1/messages endpoint modeled a message's content as
Union[str, list[{text|image|tool_use|tool_result}]], so any other block type
made Pydantic reject the whole request with
`messages.N.content.str: Input should be a valid string`. Resuming a Claude
session commonly replays assistant turns that carry `thinking` (extended
thinking) blocks, and sometimes a null content for a tool-only turn, both of
which tripped this and returned a 400.
Accept them:
- Add a permissive AnthropicUnknownBlock fallback (any block whose type is not
one of the four known ones), so thinking/redacted_thinking/provider-specific/
future blocks validate. A validator keeps known types on their typed models,
so a malformed known block (e.g. a tool_use without id) still fails cleanly.
- Coerce a null message (and tool_result) content to "" so the converter's
`for block in content` stays safe.
The converter already drops block types it does not translate, so a thinking
block is not forwarded to the model.
* Studio /v1/messages: keep user content validation strict
Make the thinking/null leniency role-aware so it never silently drops real
user input. Assistant turns (replayed history) still accept unknown/thinking
blocks and coerce a null tool-only turn to empty. User turns keep the strict
boundary: a null user content is rejected, and a content block the converter
cannot translate is rejected instead of being dropped into an empty prompt.
Also remove an empty file committed by accident.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio /v1/messages: coalesce resumed user turns and tighten content checks
- The /v1/messages count and generation paths now coalesce the adjacent user
turns that dropping an empty or null assistant turn can leave behind, so a
strict GGUF chat template no longer 400s on non-alternating roles.
- A user content block with a non-string type (list / dict) is rejected as a
clean 400 instead of raising TypeError and escaping as a 500.
- The assistant null-to-empty coercion only applies to an explicit null; an
assistant turn that omits content entirely still fails required-field
validation instead of being silently coerced to an empty string.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio /v1/messages: tighten comments
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* unsloth start: add --resume to persist and reopen agent sessions
`unsloth start <agent>` launches a coding agent whose home is a throwaway
temp dir wiped on exit, so codex/openclaw/hermes/pi (which relocate their
whole home there) cannot resume a conversation after you quit. opencode and
claude keep their session data in a fixed user dir, so they already resume.
Add an opt-in --resume/--no-resume flag: it routes the launch to the stable
Unsloth agents dir (the same one --no-launch already uses) so the session
survives the exit, never touching the user's own ~/.<agent>. A bare --resume
also reopens the last conversation via the agent's native flag (codex
`resume --last`, opencode/claude/pi `--continue`). The default is unchanged:
a plain launch still uses a temp dir and persists nothing.
Add a dispatch-only `resume` job to the Local Agent Guides CI that drives the
real launch path and asserts the split: codex/pi are wiped without --resume
and persist with it, while opencode/claude persist either way.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* unsloth start: rename --resume to --persist
The session flag collided with agents' own resume flags. `unsloth start
claude --resume <id>` used to forward `--resume <id>` straight to Claude
(which keeps its history in ~/.claude regardless), so a boolean --resume on
unsloth start would have swallowed the session id and turned it into a stray
prompt. Name the persistence flag --persist instead, so every agent's native
resume flag (claude --resume <id>, codex resume, opencode --continue, ...)
still passes through untouched. Behavior is otherwise identical: --persist
keeps a launched agent's session under the Unsloth agents dir, and a bare
--persist reopens the last conversation.
Add a regression test that `--resume <id>` passes through verbatim, and in the
CI resume experiment skip the redundant second pass for opencode/claude (they
persist either way, and a second CPU turn only risks a timeout).
* unsloth start: correct --persist help and drop the buggy auto-resume
Reword the --persist help to be accurate: claude and opencode keep sessions in
the user's own stores and resume regardless, so --persist only stabilizes the
otherwise-ephemeral relocated home of codex/openclaw/hermes/pi. Drop the
bare-launch auto-append of native resume tokens: it errored on a first launch
with no prior session, and was inconsistent between launch and no-launch.
--persist now only keeps the session dir; resume via the agent's own command
(e.g. `unsloth start codex --persist resume`), which now finds it.
In the CI resume experiment, fail the pass when the launched turn exits
non-zero, so a write-then-error is not misread as PERSISTED.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
The main merge left listStoredChatThreads imported twice -- once as a standalone import from
the deep utils path and once via the @/features/chat barrel (which re-exports it) -- tripping
TS2300 'Duplicate identifier' and failing the Tauri frontend build. Keep the barrel import,
grouped with the other chat imports.
* Silence torch._check_is_size FutureWarning and shim it if torch removes it
bitsandbytes 4-bit dequant calls torch._check_is_size, which torch
deprecated with a FutureWarning ("Use _check(i >= 0) instead") that prints
on every bnb-4bit load. Silence that warning in suppress_cuda_printf, and
add fix_torch_check_is_size so a future torch that removes _check_is_size
gets it shimmed to _check(i >= 0) (honoring the max bound) and bitsandbytes
keeps working.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten fix_torch_check_is_size docstring
Lead with what the shim does and drop the redundant line; two lines
instead of three, same intent.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Mirror dynamo/inductor config sets into defaults so torch 2.12 worker threads honor them
torch 2.12 stores config user overrides in ContextVars, so direct
assignments like torch._dynamo.config.recompile_limit = 1024 no longer
reach the autograd engine worker threads. Gradient checkpointing
recomputes fullgraph-compiled gpt-oss kernels inside backward on those
threads, which then read the default recompile limit of 8 and raise
FailOnRecompileLimitHit at step 0 of GRPO/SFT. Mirror direct config
assignments into the process-global entry defaults on torch >= 2.12,
restoring the torch <= 2.11 cross-thread semantics while leaving the
context-scoped config.patch API untouched.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Keep config.patch thread-local when mirroring dynamo/inductor sets
config.patch(...) also assigns through ConfigModule.__setattr__, so the
default-mirror was leaking its scoped, thread-local writes into the
process-global entry default. Track patch enter/exit with a per-thread
depth counter (wrapping ConfigModule.patch) and skip mirroring while
inside a patch, so only genuine direct assignments restore the torch
2.11 cross-thread semantics and config.patch stays context-local.
* Also keep config.load_config thread-local when mirroring config sets
load_config restores a saved dynamo/inductor config by calling setattr
per key, which the default-mirror would otherwise leak process-wide just
like config.patch did. Wrap load_config with the same per-thread depth
counter (renamed to _scoped_depth) so both scoped writers skip the mirror
and stay context-local, while genuine direct assignments still restore the
torch 2.11 cross-thread default.
* Drop the pre-existing override replay from the config thread fix
The replay was redundant: this runs from _gpu_init before unsloth sets any
dynamo/inductor config, so the __setattr__ wrapper already mirrors every
later assignment (recompile_limit included). It could also read a value
that belonged to a config.patch context still active at import time and
write that thread-local override into the global default. Removing it keeps
the cross-thread fix and drops the now-unused _inductor.config import.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Address the Codex review round on the video/quant work:
- Companion auto-quant now honors an explicit Speed=off. Both loaders already pin the DiT dense
under an explicit off (bit-exact reference), but the unset text-encoder / VAE quant still promoted
to auto and silently fp8/int8'd the companions, breaking the bit-exact request. An UNSET speed
still auto-quantises; an explicit companion scheme still forces it.
- The HunyuanVideo joint-attention trim is a speed lever (it swaps to the fused SDPA kernel), so gate
it on a non-off speed tier exactly like the adjacent attention-backend selection -- the off path
keeps the stock dense-mask attention.
- Explicit torchao text-encoder modes (int8 / fp8_dynamic / nvfp4) now run the same kernel smoke
test the auto ladder uses. They could clear the capability gate yet fail the real GEMM on a build
where quantize_ wraps the encoder but the kernel is broken; the caster's try/except only covers the
cast, not the first forward, so the load would report engaged then crash at generation. Now it
falls back to dense. Layerwise fp8 has no torchao GEMM, so the probe is a no-op for it.
- The trim pre-hook's fallback restores the caller's original kwargs (it may have emptied the image
stream / trimmed a text stream before failing), so the stock dense-mask path runs on exactly what
it expects, matching the empty-prompt guard.
- video_speedmem_bench mirrors the loader: installs the Hunyuan trim before the backend set (gated on
an active tier) and skips the auto int8 quant when it is the fp8-denied memory fallback and dense
fits resident, so the shipped/auto rows measure what the loader actually runs.
Tests: TE explicit-mode kernel probe (+ layerwise-fp8 bypass), trim mid-trim restore, and loader-level
speed=off companion suppression + trim skip for both backends. 262 backend tests pass; ruff clean.
When a coding agent is missing, `unsloth start <agent>` offers to run the
vendor's own installer (curl | bash, irm | iex, or npm) after an interactive
confirm. Those installers execute with the user's privileges and there is no
signature or hash check on the fetched content, so a blind "yes" is a
supply-chain risk if the delivery path is compromised.
Keep the auto-install convenience but make consent informed: before the prompt,
name the exact remote source the installer fetches (or the command it runs for a
package installer) and state that nothing verifies a signature or hash. Behavior
is otherwise unchanged: non-interactive stdin still never executes anything, and
the confirm still defaults to no.
The Recommended list's fit-on-device toggle filtered the live Hub rows and the
flat curated rows, but the canonical catalog GROUP rows (Images / Video pages)
were gated only by the format filter. A bare click on a filtered list could then
still start an OOM load the toggle was meant to hide (LTX-2 base at 90 GB, the
Wan2.2-A14B MoE at 114 GB, both bf16-only with no GGUF fallback).
Add catalogGroupFitsDevice: a group stays visible when at least one artifact can
actually run here (already downloaded, a GGUF whose quant ladder self-fits, or a
sized artifact within 0.7*GPU + 0.7*RAM), mirroring the Recommended fit predicate
across a group's formats. Gate the search (matchedCatalogGroups) and the
Recommended-section catalog rows (render + roving keys) on it. Node-native
catalog:check assertions cover the over-budget, GGUF-fallback, downloaded, unknown
-budget, and datacenter-budget cases.
- diffusion_cache: do not engage FBCache when the selected pipeline opens no cache_context.
A CacheMixin transformer is necessary but not sufficient -- Flux Kontext / img2img /
inpaint / controlnet reuse the CacheMixin FluxTransformer2DModel yet their __call__ never
opens a cache_context, so the First-Block-Cache hook raised 'No context is set' on the
first forward, crashing every default FLUX.1-Kontext edit (28 steps, above the FBCache
threshold). Detect it from the pipeline __call__ source, resolved off the instance so the
per-expert proxy view delegates to the real pipe.
- diffusion_attention: honor an explicit aiter backend on ROCm/AMD targets instead of
dropping it via the NVIDIA-only guard (aiter is the AMD ROCm kernel; it only works there).
- video: clear the CUDA cache on a failed load so a partially built pipeline's reserved VRAM
does not OOM the next load (mirrors the image backend), and re-check cancellation after the
export/mux so a clip cancelled during the blocking encode is discarded, not persisted.
- diffusion_auto_policy / diffusion_prequant: validate a request-supplied prequant path
override (present AND allowlisted) before budgeting the small prequant plan, so the loader
does not skip the dense shards and then rebuild dense after evicting the resident pipeline.
- diffusion_controlnet: family-gate a curated ControlNet addressed by its full repo id, not
only its short catalog id, so a cross-family repo id 400s up front instead of downloading
and loading through the wrong ControlNet class.
* Fix FastSentenceTransformer Qwen embedding preprocessing
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Document Transformer.load embedding modality fix for #6881
* Harden #6881 fix and add forwards/backwards-compatible regression tests
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fall back to Transformer constructor on legacy sentence-transformers without Hub-capable load
* Mirror legacy sentence-transformers fallback in embedding-parity tripwire test
* Tighten #6881 comments and docstrings
* Skip embedding-parity test on CPU-only runners since FastSentenceTransformer requires CUDA
* Honor the transformer module's saved subfolder when loading
modules.json records a path for the Transformer module (root for
decoder embedders like Qwen3-Embedding, 0_Transformer for the classic
layout). Pooling/Normalize already load from their saved path; thread the
same path into Transformer.load as subfolder so config and tokenizer
resolve like stock ST. stays a no-op, so single-module models are
unchanged.
* Make embedding-parity test bf16-aware
fp16 overflows to NaN on bf16-native embedders such as EmbeddingGemma
(Gemma3), producing a false parity failure. Prefer bf16 when the GPU
supports it so the tripwire can guard the full documented embedding
matrix (Qwen3-Embedding, EmbeddingGemma, BGE-M3, all-MiniLM, GTE-ModernBERT),
not just fp16-safe models.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
* Retry the Studio UI shutdown re-login on transient goto timeout
The Chat UI Playwright smoke intermittently failed at the pre-shutdown
re-login: page.goto('/login') can hit a 60s TimeoutError on a slow runner
even while the server is healthy, and the surrounding except only tolerated
ERR_ABORTED / interrupted-navigation, so a plain timeout hard-failed the job.
Wrap the re-login goto/wait/fill/submit in the same 3-attempt retry the
change-password step already uses (recover_or_replace_page between tries,
per-attempt fail screenshots, wait_for_health pre-gate). The composer wait
stays outside the loop so a retry never re-navigates after login has set
tokens (which would redirect to /chat via the guest guard); it remains the
authoritative confirmation, so a genuinely broken login still fails.
* Catch transient login-request failures and preserve error listeners on recovery
Wait on the /api/auth/login POST inside the retry (via click_and_wait_for_response)
so a transient 4xx/5xx is retried in-loop instead of surfacing only at the
out-of-loop composer wait, matching the change-password step. When
recover_or_replace_page swaps in a fresh page, re-attach the pageerror/console
listeners so error tracking survives the replacement.
Two fidelity fixes to the video speed/mem bench so its numbers match production:
- _build_pipe loaded the pipeline with a scalar bf16 torch_dtype and then upcast the
VAE, which truncates the fp32-stored Wan VAE at load (a later .to(float32) only widens
the lossy values). Pin the VAE fp32 per-component like the production loader
({"vae": fp32, "default": bf16}) so the bench decodes the same weights production does.
- The persisted reference frames were written/read as a single unkeyed ref_frames.npz in
the fixed default --out dir, so a reference-less run of a different family/seed/steps/
frames/resolution scored LPIPS against a stale baseline. Key the cache by those
parameters so a run only reuses a reference computed for the same parameters.
The image load route calls engine.begin_load(..., vae_quant=request.vae_quant, ...)
uniformly for both engines, but the native SdCppDiffusionBackend.begin_load accepted
every other diffusers-only knob except vae_quant and had no **kwargs, so a native
(CPU-only / MPS / forced-native) image GGUF load raised TypeError on every request
(vae_quant is always passed, defaulting to None). Accept and ignore it like the other
diffusers-only knobs; sd.cpp has no torchao VAE quant.
* Stabilize floating monitor drag
* Restore floating monitor exit animation
* Harden Windows Studio smoke checks
* Keep API menu badge removed
* Apply no-build-tools env overrides in-script
The runner does not apply step-level env keys containing parentheses,
so ProgramFiles(x86) kept its real value and Find-VsBuildTools still
detected VS through vswhere. Set the overrides inside each pwsh step
instead; child processes inherit them. The resolver step moves to pwsh
because bash cannot export a variable named ProgramFiles(x86).
* Reset chat UI session without a second browser context
macOS runs Chromium with --single-process, where closing the last
context tears down the whole browser, so the shutdown re-login died
with TargetClosedError on new_page. Clear cookies and swap pages
inside the same context instead, opening the replacement page before
closing the old one.
* Keep the no-build-tools Path filtered across session refreshes
install.ps1's Refresh-SessionPath and setup.ps1's Refresh-Environment
rebuild the session Path from the Machine and User registry scopes, so
the process-level filter could be undone mid-install and re-expose
CMake. Filter those scopes in the Prepare step with normalized dir
matching and restore them in cleanup.
* Drop stale localStorage auth tokens before re-login
Auth tokens live in localStorage, not cookies, and the login guest
guard redirects on their mere presence. Remove them during the session
reset so the /login navigation is deterministic instead of relying on
the tolerated redirect bounce.
test_diffusion_attention.py documents itself as hermetic with no torch/diffusers
needed, but the HunyuanVideo trim tests added a module-level 'import torch' that
aborted collection of the whole file when torch is absent. Move those 15 tests to
test_diffusion_attention_trim.py (which declares the torch dependency) so the
attention-backend policy tests stay collectable and runnable without torch.
Five studio diffusion source files added by earlier sub-PRs (Krea 2 Turbo,
Images LoRA, ControlNet) were missing the AGPL-3.0 SPDX header the rest of
studio/backend carries. Add it so the whole backend is consistently licensed.
torch.cuda.is_bf16_supported() reports True on pre-Ampere GPUs that only
emulate bf16, so the SDXL LoRA trainer would keep bf16 there and fail at
load/forward. Use native_bf16_supported() (the same compute-capability
probe the DiT trainer already uses) so T4 / V100 / RTX 20xx fall back to
fp16 instead.
HunyuanVideo-1.5's DiT runs a joint [video; text] self-attention and, on every
block and step, builds a dense [B,1,N,N] boolean mask so the video never attends
to the padded text. A dense bool attn_mask disables every fused SDPA kernel
(flash rejects it; cuDNN and memory-efficient fall back), so the attention runs
the slow math-style path: at the production shape (121 frames, 480p, N about 50k)
one attention call is ~421ms with the mask vs ~19ms with attn_mask=None. The text
is ~99.5% padding (a t2v prompt fills ~9 of ~1985 slots), so nearly all of that
cost is spent masking padding.
install_hunyuan_attention_trim installs an eager forward pre-hook that drops the
all-zero image stream (t2v) and trims the mllm/byt5 text streams to their
globally-valid columns, plus a null-mask attention processor that runs
attn_mask=None once no partially-padded column remains (the batch-1 /
per-guidance-branch case) and otherwise delegates to the stock dense-mask
processor. The model already zeroes and masks the padded text and discards its
attention output (only the video split feeds proj_out), so removing it is exact
for the video; the only numeric change is the SDPA kernel (masked fallback to
fused). Measured on a B200: 23.3s to 1.3s per DiT forward at 121 frames (~18x with
regional compile, 0 graph breaks); per-forward cosine 0.99998 vs stock; equal
distance to an fp32 reference (LPIPS fp32-vs-stock 0.292, fp32-vs-trim 0.307), so
it is not less accurate than the current bf16 default.
Wired auto-on for HunyuanVideo-1.5 in the video loader, before the attention
backend set so the requested kernel pins onto the new processors; a no-op for
every other family and reversible (stock dense-mask path on any anomaly). Adds
hermetic tests and the diagnostic/validation scripts.
* unstructured block removal
* Enhance unstructured block handling
* Restrict block cleanup to upload UIDs
* cleanup for seed block uploads
* upload cleanup queue for unstructured blocks in recipe studio
* Fix unstructured upload cleanup edge cases
* Fix unstructured upload import ownership
* Fix-unstructured-import-path-ownership
* Guard failed-delete restore against stale block in unstructured drop zone
* Drain queued upload cleanups when autosave is skipped
---------
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: imagineer99 <samleejackson0@gmail.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Root-caused "HunyuanVideo-1.5 int8 is slower than dense" with a per-forward profiler
(scripts/hunyuan_int8_profile.py, dynamo-reset, back-to-back on a clean B200): int8 compiles
cleanly (0 recompiles, 0 graph breaks, steady 268.3 ms/forward) and is only ~7% slower than dense
+ regional compile (250.5 ms/forward), not the 38% a contended-GPU bench run suggested. int8 is
also less accurate (LPIPS 0.085 vs dense+compile 0.037). So for a family where fp8 is denied
(Hunyuan black-frames on per-row fp8), int8 is a MEMORY lever, not a speed win, yet the auto-quant
default quantised it even when the dense DiT already fit resident.
Fix: is_int8_memory_fallback(target, family) is True only when AUTO quant lands on int8 as a
denied/black-frame fallback on a data-center, fp8-capable GPU (fp8 would be the arch pick but is
denied for the family). The video loader now skips the auto-quant and runs dense+compile when that
holds AND the bf16 memory plan already fits resident (offload_policy == none), so there is no new
OOM risk. Scoped tightly: only an AUTO request (explicit int8/fp8 honored), only int8-fallback
families (Wan / LTX resolve to fp8 -> keep quantising), only data-center fp8-capable parts (consumer
GPUs and pre-Ada, where int8 is a genuine accelerator, keep int8), and only when dense provably
fits; a memory-constrained plan still quantises. Result: Hunyuan on a resident-fit B200 now runs
faster AND more accurate, quantising only when memory is the constraint.
Also resets dynamo per config in the video bench (so compiled graphs cannot leak across configs in
one process) and adds the per-forward profiler used for the diagnosis.
Adds wan2.2-t2v-a14b and hunyuanvideo-1.5-720p to the bench family table and makes _apply_levers
quantize / compile BOTH experts of a dual-expert MoE (Wan2.2-A14B) via a _SecondExpertView proxy
that mirrors the loader's _SecondDiTView, so A14B latency and accuracy are measured on the real
two-DiT path instead of only the first expert.
Used to validate that every video family is on the fastest DiT quant scheme that is not less
accurate than its alternative (B200, 512x320, 25 frames, 30 steps, no cache, LPIPS vs dense bf16):
- Wan2.2-TI2V-5B fp8 49.9 ms/step vs int8 64.6 vs dense 59.8; LPIPS 0.129 vs 0.180
- Wan2.2-T2V-A14B fp8 195.8 ms/step vs int8 193.5 vs dense 369.1; LPIPS 0.288 vs 0.349 (both experts)
- LTX-2 / LTX-2.3 fp8 133.5 ms/step vs int8 138.4 vs dense 204.7; LPIPS 0.026 vs 0.027
- HunyuanVideo-1.5 fp8 is black (denied, not localizable); int8 21.2 s vs dense+compile 15.4 s -- a
memory saving at a speed cost, so int8 stays only because fp8 is impossible there.
fp8 is faster than dense AND more accurate than int8 on all three Wan/LTX families (the two Wan ones
via the condition_embedder exclude; LTX-2 needs none -- its conditioning has no zero-amax padding
rows). The auto-ladder + deny + exclude already select exactly these schemes, so no scheme-selection
change was needed; this commit is the bench faithfulness improvement that let the campaign confirm it.
The Wan fp8 black frame was root-caused (scripts/fp8_layer_ablation.py,
measured on B200 with the production torch._scaled_mm path): per-row fp8
scales each activation row by row_amax/448, and the text prompt is padded to
512 tokens (~all padding for a short prompt), so condition_embedder's text
embedder divides a zero padding row by a zero scale, which infs and renders
every frame black. That embedder's bias makes every downstream row non-zero,
so the whole 30-block attn1/attn2/ffn stack is fp8-clean (fp8-except-
condition_embedder measured cosine 0.9998 vs bf16, 0 non-finite; fp8-
everywhere is 100% non-finite).
So the blanket fp8 deny was heavier than needed for Wan. Remove fp8 from the
Wan deny and keep only condition_embedder in bf16 via a new
_FP8_FAMILY_EXCLUDE_NAME_TOKENS; auto now restores fp8 (the Blackwell ladder
head) for Wan2.2-TI2V-5B and -T2V-A14B (shared DiT class and padded-text
conditioning). Full-generation check (512x320, 25 frames, 30 steps, cache on
and off): mixed-fp8 is non-black (mean luma 182.6 vs dense 181.2), more
accurate than int8 (LPIPS 0.129 vs 0.180 no-cache, 0.224 vs 0.251 with
FBCache), faster (49.9 vs 64.6 ms/step; int8 was a per-step regression vs the
59.8 ms/step dense), at the same memory (19.34 GB, both -20% vs dense).
HunyuanVideo-1.5 keeps the fp8 deny: its MMDiT masks the padding text tokens
to zero inside every block, so the per-block context stream (add_*_proj /
to_add_out / ff_context) regenerates zero rows layer after layer (fp8 on only
the main blocks is 100% non-finite) so no small exclude set exists and int8
stays. mxfp8 / nvfp4 remain denied for Wan (same per-row scaled_mm family, not
separately validated).
exclude_tokens_for_scheme now takes an optional family, threaded through the
runtime quantiser and the offline prequant builder + validator so offline ==
runtime (a stale Wan fp8 checkpoint baked without the exclude is rejected and
re-quantised rather than loaded). Adds scripts/fp8_layer_ablation.py (the
per-layer ablation probe) and a mean-luma black-frame metric plus mixed-fp8
vs int8 configs to the video bench.
* Studio: render thinking blocks for safetensors inference with prefilled <think> templates
Reasoning templates like Qwen3.6 end the generation prompt with an open
<think> tag. skip_prompt streaming drops it, so the frontend never sees
the opening tag and shows reasoning as plain text. Detect the prefill
and re-emit it at the start of the stream on the transformers and MLX
paths. Also stop stripping think tags in _clean_generated_text when a
tokenizer marks them special.
* Studio: guard think re-emit for special close tags, yield prefill early
Address review feedback:
- Guard: skip re-emitting the open <think> when the tokenizer marks </think>
as a special token, since skip_special_tokens would strip the model's close
tag and leave an unclosed block that swallows the answer. Falls back to
plain text (pre-fix behaviour) for those tokenizers.
- Yield the prefilled <think> before the first token so the thinking block
renders during prompt prefill instead of after the first generated token.
- Drop the now-unnecessary _clean_generated_text think-tag exemption; the
guard handles the special-token case at the source.
No mainstream reasoning model (Qwen3.6, Qwen3, DeepSeek-R1, QwQ, GLM-4.6)
marks think tags special, so behaviour is unchanged for them.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lyxot <longyixing331@gmail.com>
Measured the fp8 DiT auto-quant path across the remaining dense-pipeline video families on
B200 (production torch._scaled_mm per-row fp8, no MSLK):
- HunyuanVideo-1.5 (480p + 720p repacks): every frame black (mean luma 0.0, LPIPS 0.82);
int8 is clean (mean 102.7 vs dense 99.9). Same failure as Wan / qwen-image.
- LTX-2: fp8 renders clean (mean 153.7, matches int8's 157.7) -- NOT a black-frame family.
So deny fp8/mxfp8/nvfp4 for hunyuanvideo-1.5 and hunyuanvideo-1.5-720p (fall to int8), and
deliberately leave LTX-2 on fp8. The deny stays measured per family, not a blanket video rule:
a blanket deny would have wrongly forced LTX-2 off fp8. Adds a Hunyuan deny test that also
asserts LTX-2 keeps fp8; 49/49 transformer-quant tests pass.
video_speedmem_bench.py gains guidance_via_guider support (HunyuanVideo-1.5 sets CFG on a
guider component and its __call__ takes no guidance_scale / callback_on_step_end), so the
harness can drive Hunyuan the same way the loader does.
PyPI release unsloth 2026.7.2 is now live. Bumps the pinned floor in
install.sh and install.ps1 from 2026.7.1 to 2026.7.2 for both unsloth and
unsloth-zoo across all 5 install commands (no-torch / reinstall / upgrade /
local / auto torch backend paths) so fresh installs resolve to the new wheel.
Follows the same pattern as #5716.
video_speedmem_bench.py drives the real video-loader lever functions
(quantize_transformer / quantize_text_encoders / quantize_vae / apply_speed_optims
/ apply_attention_backend / apply_step_cache) with the loader's own defaults, so each
measured config reflects a real load. It decomposes the video speed/memory stack
(compile, cuDNN attention, First-Block-Cache, DiT/TE/VAE quant) with per-step latency,
peak resident GB, and per-frame LPIPS vs a bit-exact reference. This is the harness that
surfaced and validated the Wan fp8 black-frame fix.
quant_speedmem_bench.py gains the DiT-quant mode (dense vs fp8/int8/mxfp8 speed, peak
memory, and LPIPS vs the dense render) plus the shared LPIPS(AlexNet) helper.
The dense video default engages transformer auto-quant, and on Blackwell the
auto ladder leads with fp8. On the Wan DiT the production per-row fp8 path
(torch._scaled_mm) renders every frame black (mean luma 0.0 at 512x320 and
704x480, LPIPS ~0.80 vs bf16): Wan's activation outliers exceed per-row fp8's
range, the same failure already denied for qwen-image. First-Block-Cache then
over-caches the degenerate activations (per-step collapses to ~10ms),
compounding it.
Add the Wan families (wan2.2-ti2v-5b, wan2.2-t2v-a14b, same WanTransformer3DModel)
to _FAMILY_SCHEME_DENY for fp8/mxfp8/nvfp4 so auto falls through to int8, which is
clean on Wan (per-token, outlier-robust), saves the same weight memory on the DiT,
and lets First-Block-Cache engage normally instead of over-caching. mxfp8/nvfp4 are
denied alongside fp8 conservatively so auto lands on the battle-tested int8; they
can be re-enabled per family once validated in-bar, like the nvfp4 auto-ladder TODO.
Validated on B200: the shipped video default now selects int8 for the Wan DiT and
renders clean frames (mean 172.6) at 15.6 GB resident (down from 24.2 GB dense),
with First-Block-Cache engaged. Adds two deny tests; 48/48 transformer-quant tests pass.
* Studio: allow CPU-only DiffusionGemma by granting the diffusion runner the CPU device
* Studio: mark CPU-only DiffusionGemma as non-GPU-resident for training VRAM preflight
* Studio: keep the CPU DiffusionGemma change minimal (revert VRAM-flag tweak; Metal hosts still hold unified memory)
* Studio: keep CPU DiffusionGemma fallback fully CPU-masked so a masked GPU host does not re-expose GPU 0
worker.py imports has_blackwell_gpu from utils.wheel_utils, but _load_worker_module
stubs utils.wheel_utils with a fixed name tuple that omitted it, so loading the worker
raised ImportError (cannot import name 'has_blackwell_gpu') and Backend CI could not
collect test_mlx_training_worker_config.py. Add the name to the stub so it matches
worker.py's imports.
* fix: Remove moot has_blackwell_gpu() function
Fixes unslothai/unsloth#6961. This function skipped flash-attn on Blackwell GPUs because no prebuilt wheel existed;
Dao-AILab now ships one and url_exists() already gates resolution.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* fix: use torchao 0.17.0 for Blackwell
Fixes#6961. Torchao 0.16.0's cpp extensions are built against CUDA 12, so on a CUDA-13
torch (cu130 / Blackwell) they fail to load with "libcudart.so.12: cannot
open shared object file". Select 0.17.0 there instead: its cpp targets torch
2.11, so it is skipped cleanly rather than crashing. CUDA-12 / ROCm / CPU
torch 2.10 keeps 0.16.0 and its working kernels.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Condense torchao version-selection comments (no behavior change)
* Support torch 2.11 in the Studio installer via the torch2.10 prebuilt wheels
Map torch 2.11 to the torch2.10 prebuilt wheels for flash-attn, causal-conv1d,
and mamba through wheel_utils.prebuilt_wheel_torch_mm, applied in direct_wheel_url
(filename) and flash_attn_wheel_url (version). Those torch2.10 CUDA wheels load and
pass each project's own test suite on torch 2.11 (verified on B200), so a torch 2.11
environment gets the prebuilt accelerators instead of skipping or building from source.
Raise _CUDA_TORCH_PKG_SPEC to <2.12.0 (torchvision <0.27.0, torchaudio <2.12.0) so
the CUDA torch repair path can install torch 2.11, where torchao 0.17's cpp kernels
load cleanly. Add tests for the mapping.
* Keep has_blackwell_gpu as a False stub for future arch gating
* Restore has_blackwell_gpu as a return-False probe kept for future arch gating
Keep the nvidia-smi compute_cap detection and its two call sites, but short-circuit
with return False at the top so flash-attn is no longer skipped on Blackwell (sm_100+
now has prebuilt wheels and url_exists gates resolution). Drop the early return to
re-enable arch-based detection later.
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* models: auto-target per-expert Linear MoE experts for LoRA (gpt-oss 4bit)
MoE checkpoints whose experts are stored as per-expert nn.Linear ModuleLists
could not receive expert LoRA. gpt-oss bnb-4bit is the canonical case: its
experts live at mlp.experts.gate_up_projs.<i> and mlp.experts.down_projs.<i> as
per-expert Linear4bit modules, not a fused nn.Parameter. The target_parameters
path only handles the fused nn.Parameter layout, and the plain
gate_proj/up_proj/down_proj leaf names do not match the per-expert indices, so
get_peft_model attached LoRA to attention only and left every expert frozen
(0 of 1536 on gpt-oss-20b) even though the grouped bnb-4bit training forward
exists.
Add get_moe_target_modules, the module-LoRA counterpart of
get_moe_target_parameters: it detects per-expert Linear ModuleLists under an
experts container and returns their suffix target_modules names
(gate_up_projs.<i> / down_projs.<i>). get_peft_model in both llama.py and
vision.py extends target_modules with these, handling the explicit leaf-list
form and the regex form (auto / all-linear / scoped). It is gated on the same
MLP-in-scope condition as the parameter path, so an attention-only request still
skips the experts.
Also gate get_moe_target_parameters on the fused parameter actually existing, so
a per-expert-Linear layout no longer produces a dead target_parameters path or a
misleading "Enabling LoRA on MoE parameters" line; those experts are handled
through target_modules instead.
Validated on gpt-oss-20b-unsloth-bnb-4bit (transformers 5.5.0): experts attach
(1536 modules, trainable 0.036 percent to 1.65 percent) across the default, None
and all-linear paths; training memorizes and the LoRA adapter reproduces exactly
after a cold reload in a fresh process. No regression: fused-parameter MoEs
(Qwen3-30B-A3B-4bit), non-MoE models, and attention-only requests are unaffected
(get_moe_target_modules returns an empty list).
Merging these per-expert adapters into a merged_16bit checkpoint is handled by a
companion unsloth-zoo change (saving_utils folds each per-expert delta into the
fused gate_up_proj / down_proj tensor). With both, the LoRA adapter and the
merged_16bit checkpoint reload the trained behavior identically.
* models: scope per-expert MoE targets, keep repeat get_peft_model idempotent, warn on old zoo
Address review of the per-expert Linear MoE targeting:
- Scope get_moe_target_modules to the requested projection leaves (gate/up map to
the gate_up ModuleList, down maps to the down ModuleList), so a narrowed request
such as target_modules=["down_proj"] no longer also trains gate_up_projs, matching
get_moe_target_parameters.
- Detect experts through a PEFT-wrapped base_layer as well, and recompute the
auto-added expert targets in the llama.py existing-adapter check, so a repeat
get_peft_model call with the same arguments stays idempotent instead of raising on
the saved expert targets.
- Warn when the installed unsloth_zoo cannot fold these per-expert experts into a
merged_16bit checkpoint (older releases keep the fused gate_up_proj / down_proj
tensors and drop the per-expert deltas), so the expert LoRA is not silently lost on
save_pretrained_merged; the fold lands in unsloth-zoo #885. The LoRA adapter itself
is unaffected.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix(studio/hub): apply repo_id length limit per segment, not whole string
is_valid_repo_id() applied the 96-char limit to the full "namespace/repo_name"
string, so a repo with a valid (<=96 char) name but a long combined id was
falsely rejected. Match huggingface_hub.validate_repo_id by checking the length
per segment instead. Fixes#6946.
* Fix long repo id state filenames
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Etherll <61019402+Etherll@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: source CPU llama.cpp prebuilts from the unslothai fork
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: reject unknown Linux CPU arches and keep ROCm-tooling hosts off the CPU prebuilt
* Studio: extend the resolve-prebuilt ROCm-tooling guard to Windows
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: let ROCm-SDK-only CPU hosts take the fork CPU prebuilt
* Studio: accept windows-arm64 prebuilt kind and refresh stale fork-routing comments
* Studio: correct stale fork-routing comments and --resolve-prebuilt help
* Refresh stale ggml-org routing comments
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Studio: keep transformers off sys.modules until the training worker activates the sidecar
The training worker (core/training/worker.py:run_training_process) decides the per-worker
Xet env flip during preflight by importing utils/hf_xet_fallback.py, which eagerly imported
unsloth_zoo at module load. unsloth_zoo's __init__ imports transformers, so the default
transformers 4.57.x was cached in sys.modules before activate_transformers_for_subprocess
prepended the 5.x sidecar to sys.path. Since activation only edits sys.path, the already
cached module won, and 5.x models failed to load their tokenizer or config:
- Qwen3.5 / GLM-4.7 (tokenizer_class TokenizersBackend): "Tokenizer class TokenizersBackend
does not exist or is not currently imported."
- gemma-4: "... is not supported yet in transformers==4.57.6."
Fix: load the shared unsloth_zoo backend lazily (only when a heavy download helper is first
used, which is after activation). child_should_disable_xet and the DEFAULT_* constants are
defined locally so importing the shim stays light. The download wrappers, the DownloadStallError
class, start_watchdog and get_hf_download_state resolve the shared backend on first use, and the
degraded no-unsloth_zoo fallback is preserved.
Tests:
- test_hf_xet_fallback.py: existing suite kept green via the restored _shared_* seam; the
GPU-init retry test now triggers the lazy load explicitly; new guard asserts importing
child_should_disable_xet does not import transformers/unsloth_zoo.
- test_training_worker_import_discipline.py: new invariant test that the worker preflight
imports leave transformers unimported, so this class of regression cannot return silently.
Runs in studio-backend-ci (CPU only, no network/GPU/weights).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: CPU-only guard that activation switches transformers to the model's sidecar version
Adds test_worker_activates_correct_transformers.py: runs the real worker preflight
(from utils.hf_xet_fallback import child_should_disable_xet) plus the real tier
detection and activate_transformers_for_subprocess for a transformers-5.x model
(Qwen3.5, tier 530), then asserts the in-process transformers actually switched to
the 5.x sidecar. A stale pre-activation import leaves 4.57.x pinned and fails the
assertion, which is exactly the TokenizersBackend regression (#6951).
Self-contained CUDA spoof (mirrors tests/_zoo_aggressive_cuda_spoof.py) forces
unsloth_zoo down its full, transformers-importing init path on a GPU-less runner;
without it unsloth_zoo degrades and never preloads transformers, masking the bug.
A one-line stub sidecar stands in for the 5.x venv, so no GPU, network, weights, or
real sidecar are needed. Passes on this fix, fails on buggy main.
* Studio: load the repo's canonical CUDA spoof in the correct-version guard
Load tests/_zoo_aggressive_cuda_spoof.py (the committed spoof the consolidated CI
already relies on) as the single source of truth so the guard matches CI and stays
robust on a CPU-only torch wheel, where a partial hand-rolled spoof could miss a
torch.cuda call and let the unsloth_zoo import raise (masking the bug). Falls back to
a minimal inline spoof for a standalone studio checkout. Verified: passes on this fix,
fails on buggy main, and the fallback path passes when the spoof file is absent.
* Studio: declare the lazily-resolved xet names so ruff F822 stays green
DownloadStallError, start_watchdog and get_hf_download_state are provided via the
module __getattr__ (PEP 562), so ruff F822 flagged them as undefined names in __all__
and the Source-lint / pre-commit checks went red. Add annotation-only declarations
(no value bound, so __getattr__ still resolves them lazily to the shared unsloth_zoo
backend) to mark them defined for the linter while keeping F822 active for the rest
of __all__.
* Studio: tighten comments on the sidecar-activation fix and its tests
* Studio: mirror the new MLX-dispatch preflight import in the import-discipline guard
The worker preflight now also runs 'from core.training.training import
is_apple_silicon_training_platform, should_use_mlx_training_backend' before it
activates the transformers sidecar. Add that import (guarded) to the guard's
preflight snippet so the invariant test stays a faithful mirror: a future change
that makes core.training.training pull transformers/unsloth_zoo eagerly would then
be caught too. Verified clean on the current tree (no leak).
---------
Co-authored-by: danielhanchen <unslothai@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
* feat: detect installed coding agent CLIs in Studio settings
The API-keys panel only ever showed the "claude" flavor of the
`unsloth start` command, so anyone using Codex, OpenCode, OpenClaw,
Hermes, or Pi had to manually rewrite the copied command by hand.
Add a backend check that looks for each agent's CLI binary on PATH
(shutil.which, mirroring the pattern already used elsewhere in
studio/backend/utils) and expose it as GET /api/settings/coding-agents.
The API-keys panel now renders a picker for all six supported agents,
marks the ones it finds installed, and defaults to one of those instead
of always falling back to claude.
Includes unit tests for the detection helper.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* address review feedback on coding-agent detection
Three fixes from PR review:
- detect_installed_coding_agents now treats a PATH lookup failure as
"not installed" instead of letting it bubble up and break the
settings endpoint; added a regression test for it.
- CodingAgentsResponse.agents is now typed as an immutable tuple
instead of a list built from one, matching CODING_AGENTS itself.
- Fixed a race in the API-keys panel: picking an agent while the
installed-CLI check is still in flight could get silently overwritten
once that check resolved. A ref now tracks whether the user has made
a manual choice, so the auto-detected default only applies before
that happens.
* Address Codex feedback: GGUF gating and remote-detection scope
- codex refuses to launch against a non-GGUF (transformers-backed) model
(unsloth_cli's _require_gguf_for_codex), so auto-defaulting to it produced
a copy-pasteable command that fails immediately whenever the loaded model
isn't GGUF. Add useActiveModelIsGguf() (looks up the active checkpoint in
the chat runtime store) and a correction effect that steers the auto-pick
away from codex unless the loaded model qualifies, without ever touching a
choice the user made by hand.
- Detection runs via shutil.which on the Studio backend host, which isn't
the same machine as the browser in a tunnel/remote session. Reword the
'installed'/'detected' copy to say so explicitly when the tunnel URL is
in use, instead of implying the check ran on the viewer's own device.
* Rework auto-default per review: loopback gating + inline GGUF check
Replaces the previous approach with the exact shape discussed on the PR:
- Export isLoopbackHost/normalizeHost from agent-command.ts. The detection
endpoint runs shutil.which on the Studio backend, which only describes the
browser's own machine when the base this panel targets resolves to
loopback. For a LAN or tunnel/remote base, gate the whole thing off --
don't mark anything as "detected" and don't let it drive the default --
instead of just relabeling the copy.
- Drop the separate GGUF-correction effect and useActiveModelIsGguf hook.
Read useChatRuntimeStore.getState().activeGgufVariant inline inside the
existing detection effect's .then() (so it doesn't need to sit in the
effect's deps), and pick the first detected agent that isn't codex unless
the loaded model is GGUF, leaving the existing default untouched when no
compatible agent is detected.
Verified both branches (loopback vs LAN/tunnel base, gguf vs non-gguf,
manual pick preserved, no-compatible-agent fallback) with a standalone
port of the .then() logic.
* Address latest Codex findings: stale detection, model swap, cache
- Clear detectedAgents (and skip the network call entirely) when the panel
leaves a loopback base, instead of leaving a previous loopback detection
result marked 'installed' for a command that now targets a LAN/tunnel/
remote host.
- Add a separate, network-free correction effect keyed on the live
activeGgufVariant: if codex was auto-picked while a GGUF model was loaded
and the user then switches to a transformers-backed model while this panel
stays mounted, steer away from codex instead of leaving a command that
unsloth_cli's _require_gguf_for_codex will now reject. Never touches a
manual pick.
- Drop coding-agents.ts's module-lifetime cache. Installed-CLI detection is
environment state, not a persisted setting, so a stale positive/negative
from before the user installed something (or reopened the tab) is worse
than one extra cheap local API call per mount; keep only the in-flight
de-dupe for concurrent callers.
Verified the correction-effect logic (gguf->non-gguf swap with/without a
fallback, still-gguf no-op, manual pick never overridden) with a standalone
port of the effect.
* Make the codex/GGUF auto-pick symmetric in both directions
The correction effect only steered away from codex when the model stopped
being GGUF; it never steered back toward codex if the model became GGUF
*after* a non-GGUF-gated fallback had already picked something else (e.g.
codex is the only detected CLI, a transformers model is loaded so the
selection correctly falls back to the claude default, then the user loads a
GGUF model while the panel stays mounted -- codex never gets reconsidered).
Consolidate into one effect that re-derives the preferred detected agent
from scratch whenever detectedAgents or activeGgufVariant changes, in either
direction, instead of only reacting to the codex-specific downgrade case.
The fetch effect now only populates detectedAgents/availableAgents; this
effect is the single source of truth for what gets auto-picked from that
list. Never overrides a manual choice.
Verified both transition directions plus the manual-pick-survives and
initial-detection cases with a standalone port of the derivation logic.
* Reset the auto-pick to the default when it stops being trustworthy
Two more real gaps from the latest Codex pass on d988f52:
- The unified derivation effect only handled the case where a *different*
detected agent could take over. If codex was the only detected agent and
auto-picked while a GGUF model was loaded, then the model stopped being
GGUF, 'preferred' came back undefined and the effect silently left the
selection on codex -- exactly the command unsloth_cli's
_require_gguf_for_codex now rejects. Fall back to DEFAULT_AGENT in that
case instead of leaving it untouched.
- Leaving a loopback base cleared detectedAgents (so the 'installed' badges
correctly disappear) but left whatever agent had been auto-picked from
that now-stale, server-side-only detection still selected. Reset to
DEFAULT_AGENT there too, unless the user picked by hand.
Introduces a shared DEFAULT_AGENT constant instead of repeating the "claude"
literal at each reset site. Verified all five cases (both new resets, both
manual-pick-survives variants, and the existing multi-detected-agent
fallback still preferring another compatible agent over resetting) with a
standalone port of the effects.
* Derive GGUF-ness from the actual loaded state, not just the variant string
activeGgufVariant only covers an HF-repo GGUF pick (a specific quant
variant string). A direct local .gguf file -- custom folder, LM
Studio, or drag-drop -- is just as much a GGUF the codex preflight
(unsloth_cli's _require_gguf_for_codex) would accept, but it never has
a "variant" to report, so it read as non-GGUF here even though
/api/inference/status correctly reports is_gguf: true for it. That
mismatch could leave a Codex-only install not auto-selected, or reset
an auto-picked Codex, for a model that actually supports it.
Combined activeGgufVariant with activeNativePathToken (covers the
drag-drop/picked-file case) and ggufContextLength (only ever populated
when the backend last reported is_gguf: true for the active model, see
applyActiveModelStatusToStore) so all three paths a model can be GGUF
through are covered, matching the same is_gguf-or-equivalent check
hasGgufSource already applies to a staged pick elsewhere in this
codebase.
* Clear stale native-path token on a non-GGUF status refresh
When a native (drag-dropped or picked) GGUF was loaded and the backend later
switches to a transformers model outside the UI load path, refresh() adopts the
new /api/inference/status via setCheckpoint and applyActiveModelStatusToStore.
Those reset activeGgufVariant and ggufContextLength but never clear
activeNativePathToken, so the isGguf OR stays true after the switch and a
Codex-only detection auto-selects unsloth start codex for a non-GGUF model its
preflight rejects.
Drop activeNativePathToken in applyActiveModelStatusToStore whenever the status
is non-GGUF. A real GGUF load reports is_gguf: true, so its token is preserved
(the load path owns it); only a non-GGUF status clears it.
* Add the AGPL-3.0 header to the new studio contract test
* Fix/adjust agent detection for PR #6909
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
Co-authored-by: wasimysaid <112766706+wasimysaid@users.noreply.github.com>
