* Fix Studio silently exiting on Windows without error output
On Windows, `unsloth studio` launches a child process via
subprocess.Popen to run the server in the studio venv. If the child
crashes (e.g. due to a missing package), the parent just calls
typer.Exit(rc) with no message -- the user sees "Launching Unsloth
Studio... Please wait..." and then the prompt returns with zero
feedback.
Root cause: `data_designer_unstructured_seed` is imported at the top
level in seed.py. If this package is not installed in the studio venv,
the entire import chain (seed.py -> routes/__init__.py -> main.py ->
run_server()) crashes with ModuleNotFoundError. Since run.py has no
try/except around run_server() and studio.py does not report nonzero
exit codes, the failure is completely silent.
Changes:
- run.py: wrap run_server() in try/except, print clear error with
traceback to stderr. Also reconfigure stderr encoding on Windows so
tracebacks with non-ASCII paths do not cause secondary failures.
- studio.py: print an error message when the child process exits with
a nonzero code on Windows, so the user knows something went wrong.
- seed.py: make data_designer_unstructured_seed import optional with
a try/except fallback. The server starts normally and only returns
HTTP 500 if the unstructured seed endpoints are actually called.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Skip Anaconda/Miniconda Python when creating Studio venv on Windows
Conda-bundled CPython ships modified DLL search paths that prevent
torch from loading c10.dll on Windows. The Studio server fails
silently at startup because the venv was created with conda's Python.
Standalone CPython (python.org, winget, uv) does not have this issue.
Both install.ps1 and setup.ps1 now skip any Python binary whose path
contains conda, miniconda, anaconda, miniforge, or mambaforge when
selecting the interpreter for the studio venv. If only conda Python
is available, the scripts print an error with instructions to install
standalone CPython.
* Fix multi-file preview crash and improve setup.ps1 Python discovery
Addresses review findings [10/10] and [8/10]:
1. seed.py: _read_preview_rows_from_multi_files() had a hard import
of build_multi_file_preview_rows inside the function body, bypassing
the optional-plugin guard. Moved it into the top-level try/except
block and added a None guard matching the other functions.
2. setup.ps1: Python discovery now probes py.exe (Python Launcher)
first, uses Get-Command -All to look past conda entries that shadow
standalone CPython further down PATH, skips WindowsApps stubs, and
resolves the actual executable path so venv creation does not
re-resolve back to a conda interpreter.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Check sys.base_prefix to catch venvs created from conda Python
A venv created from conda Python (e.g. C:\Users\danie\.venv) has a
path that does not contain "conda", but sys.base_prefix still points
to the conda install (e.g. C:\Users\danie\miniconda3). The previous
path-only check missed this case entirely.
Both install.ps1 and setup.ps1 now use a Test-IsConda helper that
checks both the executable path AND sys.base_prefix against the
conda/miniconda/anaconda/miniforge/mambaforge pattern. This catches:
- Direct conda Python executables
- Venvs created from conda Python (base_prefix reveals the origin)
* Fix install.ps1 passing version string to uv venv instead of resolved path
Find-CompatiblePython returned a bare version string (e.g. "3.13")
which was passed to `uv venv --python 3.13`. uv performs its own
interpreter discovery and can resolve that version string back to a
conda Python, defeating the entire conda-skip logic.
Now Find-CompatiblePython returns a hashtable with both .Version (for
display) and .Path (the resolved absolute executable path). The venv
is created with `uv venv --python <absolute-path>`, ensuring uv uses
the exact interpreter we validated.
* Quote resolved Python path in uv venv call for paths with spaces
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix(studio): prevent ModuleNotFoundError in dataset.map() on Windows
On Windows, dataset.map() uses "spawn", which requires workers to
import compiled modules from disk. Previously, clear_unsloth_compiled_cache()
deleted the entire directory, causing workers to crash when looking for
UnslothSFTTrainer.py.
Changes:
1. Added `preserve_patterns` to cache cleanup to keep `Unsloth*Trainer.py`
on Windows while clearing model-specific files.
2. Added the cache directory to PYTHONPATH for spawn workers.
Linux/macOS behavior is unchanged.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix spawn-platform coverage, CWD path mismatch, and race condition for PR #4473
- Extend platform guard from win32-only to include macOS (also uses spawn
since Python 3.8, same ModuleNotFoundError would occur)
- Replace fragile CWD-based PYTHONPATH registration with centralized
register_compiled_cache_on_path() that uses the same __file__-relative
_CACHE_DIRS already used by cache_cleanup -- fixes path mismatch when
studio is launched from a directory other than the repo root
- Move PYTHONPATH registration to the top of _train_worker(), before any
dataset.map() call (previously it ran late in config assembly, after
dataset formatting which also calls dataset.map())
- Update inference.py model-unload to preserve trainer files on spawn
platforms, preventing a race where unloading a model via inference tab
would delete UnslothSFTTrainer.py while training workers are importing it
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix cache-dir precedence reversal in register_compiled_cache_on_path()
Iterating _CACHE_DIRS in forward order while calling insert(0) each time
reverses the declared priority: later entries shadow earlier ones. When
multiple compiled-cache directories exist, spawned workers could import a
stale trainer from the wrong cache.
Fix: iterate in reverse so that the highest-priority entry (first in
_CACHE_DIRS) is inserted last and ends up at position 0 in sys.path and
PYTHONPATH.
* fix: harden worker-count helpers against cpu_count=None and desired<=0
- safe_num_proc: guard os.cpu_count() with `or 1`, clamp multi-GPU
path with max(1, min(4, desired)), clamp return with max(1, desired)
- safe_thread_num_proc: same os.cpu_count() guard and return clamp
- Add regression tests (31 L1 unit + 10 sandbox edge-case tests)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* remove regression tests from PR
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
The previous prompt "Show me a live weather dashboard, no API key needed"
was too vague. The new wording explicitly asks for HTML code, which
produces more useful and consistent responses.
* fix(install.ps1): split torch+unsloth install to fix non-NVIDIA package resolution
--torch-backend=auto on a non-NVIDIA Windows machine causes uv to resolve
unsloth==2024.8 (pre-CLI, no unsloth.exe). Fix: detect GPU robustly (PATH +
hardcoded fallback paths, mirrors setup.ps1), install torch first with an
explicit --index-url (CUDA variant for NVIDIA, CPU for everyone else), then
install unsloth separately without --torch-backend so the solver always picks
a modern release that ships the Studio CLI.
Closes the remaining gap flagged in #4478.
* fix(install.ps1): align warning with setup.ps1, add --upgrade, handle CUDA 11.x
- Match the no-GPU warning message to studio/setup.ps1 wording
(chat-only GGUF mode, driver download link)
- Add CUDA 11.x floor check in Get-TorchIndexUrl so old drivers
fall back to CPU wheels instead of silently getting cu124
- Log a warning when nvidia-smi output cannot be parsed
- Add --upgrade to both uv pip install calls so re-runs pick up
newer package versions
* revert --upgrade from uv pip install calls
uv pip install already resolves to the latest satisfying version;
--upgrade is unnecessary and could force unwanted re-installs.
* fix: replace frozen cu124 fallbacks with cu126, guard CUDA 11.x
cu124 wheels are frozen at torch 2.6.0 -- falling back to them pins
users to an outdated PyTorch. Three issues fixed in both install.ps1
and setup.ps1:
1. CUDA 12.0-12.5 now maps to cu126 (was cu124).
2. CUDA 11.x and older now falls back to cpu (was cu124, which would
silently install incompatible GPU wheels).
3. Parse-failure and no-nvidia-smi fallbacks updated to cu126/cpu.
Adds tests/test_cuda_wheel_mapping.py covering the mapping logic,
nvidia-smi parsing, PS1 file sync, PyTorch index URL validation,
and sandbox torch installs.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* remove test file from PR branch
Test file kept locally, not needed in the PR.
* fix: map CUDA 11.x to cu118 instead of cpu
PyTorch still publishes cu118 wheels (up to torch 2.7.1), so CUDA 11.x
users get GPU-accelerated torch rather than being forced to CPU-only.
Only CUDA 10.x and older fall back to cpu.
* fix: revert CUDA 12.0-12.5 to cu124, handle cpu tag in setup.ps1
CUDA 12.0-12.5 drivers only support up to their reported CUDA version,
so cu126 wheels (built with CUDA 12.6) fail to load. Revert the catch-
all for 12.0-12.5 back to cu124.
Also fix setup.ps1 caller: when Get-PytorchCudaTag returns "cpu" (e.g.
CUDA 10.x driver), the installer now correctly skips Triton and prints
"CPU-only" instead of "CUDA support (cpu)".
* fix: add --upgrade to unsloth install for stale venv repair
On reruns against an existing venv, uv pip install unsloth makes no
changes if unsloth==2024.8 is already installed (it satisfies the
constraint). Adding --upgrade only to the unsloth install ensures
stale installs get repaired without forcing a multi-GB torch
re-download.
* fix: use --upgrade-package to avoid clobbering torch CUDA wheels
`--upgrade unsloth` re-resolves torch from default PyPI, stripping the
+cuXXX suffix installed in step 1. `--upgrade-package unsloth unsloth`
upgrades only unsloth (and pulls missing deps like transformers, trl)
while preserving the pinned torch from the CUDA-specific index.
* docs: explain why split-install and --upgrade-package are needed
Expand the inline comment block to document both design decisions:
1. Why torch is installed separately (solver fallback to 2024.8)
2. Why --upgrade-package is used instead of --upgrade (preserves CUDA wheels)
---------
Co-authored-by: LeoBorcherding <LeoBorcherding@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix Studio crash on Anaconda Python due to platform._sys_version() parse failure
Anaconda and conda-forge modify sys.version to include distributor
metadata between pipe characters, e.g.:
3.12.4 | packaged by Anaconda, Inc. | (main, ...) [MSC v.1929 ...]
Python's platform._sys_version() has a hardcoded regex that cannot
parse this format, raising ValueError. CPython closed this as "not
planned" (cpython#102396) since Anaconda modified the binary.
This breaks the import chain: run.py -> structlog -> rich -> attrs,
which calls platform.python_implementation() at module scope.
Fix: before any library imports, strip the pipe segments, parse the
cleaned version string via the standard parser, and cache the result
under the original sys.version key so all subsequent platform calls
hit the cache.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add defensive fallback for unpaired pipe edge cases in version patch
Address Gemini review suggestion: if the paired-pipe regex leaves
residual pipes (hypothetical single-pipe distributor metadata), fall
back to extracting the version number and the parenthesized build
info directly. Wrap the entire patch in try/except so unexpected
version string formats degrade gracefully instead of crashing the
patch itself.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Refactor into shared _platform_compat module, cover colab.py entrypoint
Address reviewer feedback:
1. Extract the Anaconda/conda-forge sys.version fix into a shared
_platform_compat.py module that wraps platform._sys_version() with
a retry-on-ValueError fallback. This is more robust than cache-seeding
because it handles all future platform._sys_version() calls, not just
the first one.
2. Import the fix from both run.py and colab.py entrypoints, so Studio
no longer crashes on Anaconda Python regardless of the launch path.
3. The wrapper is idempotent (guarded by a flag) and handles edge cases:
paired pipes (Anaconda, conda-forge), unpaired pipes (hypothetical),
and standard CPython strings (no-op since ValueError is never raised).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Replace monkey-patch with cache-prime, fix colab.py duplicate sys.path, cover main.py
- Rewrite _platform_compat.py: replace function-wrapping monkey-patch with
one-shot cache seed (_seed_sys_version_cache). Parses cleaned sys.version
once and seeds platform._sys_version_cache so the stdlib parser never sees
the problematic Anaconda/conda-forge pipe-delimited string. No function
replacement, no idempotency flag, no reload edge cases.
- colab.py: remove duplicate backend_path sys.path insertion after
_bootstrap_studio_venv(). The early insertion (before _platform_compat
import) already covers it. This also fixes backend/ ending up behind
venv site-packages in sys.path ordering.
- run.py: move PYTHONWARNINGS=ignore before _platform_compat import to
preserve original intent of suppressing warnings early.
- main.py: add sys.path + _platform_compat import before route imports,
covering the direct `uvicorn main:app` launch path.
- Add test_platform_compat.py with 7 tests covering Anaconda, conda-forge,
and standard CPython version strings, plus the loggers import chain.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Remove test_platform_compat.py from PR
* Handle Format B conda-forge version strings with duplicate paren groups
Some conda-forge builds produce sys.version with the build info both
before and after the pipe label (e.g. "3.9.7 (default, ...) | packaged
by conda-forge | (default, ...) \n[GCC 7.5.0]"). After stripping the
pipe segment, two consecutive (...) groups remain, which still fails
platform._sys_version(). Add a second regex pass to drop the duplicate
paren group.
* Guard _sys_version call with try/except to avoid making things worse
If the cleaned version string is still unparseable by the stdlib regex
(e.g. nested parens, exotic multi-pipe formats), silently give up
instead of letting ValueError propagate at import time -- which would
be a worse crash than the original deferred one.
---------
Co-authored-by: Daniel Han <danielhanchen@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix: handle Windows subprocess crash during dataset.map()
Windows uses spawn (not fork) for multiprocessing. Spawned workers
cannot resolve Unsloth's dynamically compiled cache modules from
unsloth_compiled_cache/, causing ModuleNotFoundError and RuntimeError
during dataset.map() tokenization.
Add two platform-guarded patches for sys.platform == "win32":
1. Force HF_DATASETS_MULTITHREADING_MAX_WORKERS=1 and set spawn method
2. Monkey-patch Dataset.map() to force num_proc=None
Fixes#4490
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* address review: extend spawn fix to macOS, add multiprocess fallback
- Change platform checks from sys.platform == "win32" to
sys.platform != "linux" so macOS (also spawn-based) is covered
- Wrap multiprocess import in try/except falling back to stdlib
multiprocessing when the multiprocess package isn't installed
- Rename _win32_safe_map to _spawn_safe_map to reflect broader scope
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: replace global Dataset.map monkey-patch with targeted num_proc routing
The previous approach had issues: Patch 1 set HF_DATASETS_MULTITHREADING_MAX_WORKERS
and forced set_start_method (dead code on platforms already using spawn), and Patch 2
globally monkey-patched Dataset.map() (too broad, missed Dataset.filter()).
Replace with a two-layer fix:
1. Studio layer: Add dataset_map_num_proc() that returns None on spawn platforms
(Windows, macOS). Unlike num_proc=1 which still creates Pool(1) and spawns a
worker, num_proc=None runs Dataset.map()/filter() truly in-process.
Update all dataset.map() callsites to use it. ThreadPoolExecutor callers
(format_conversion.py) keep using safe_num_proc() since threads are unaffected.
2. Root-cause layer: Propagate UNSLOTH_COMPILE_LOCATION via PYTHONPATH on spawn
platforms so spawned workers can import compiled modules. Mirrors the .venv_t5
pattern in worker.py. Does not import unsloth_zoo.compiler (heavy torch/triton
imports). Completely skipped on Linux.
Also extend safe_num_proc() to return 1 on macOS (was only guarding Windows),
and narrow the transformers 5.x dataloader guard from != "linux" to explicit
("win32", "darwin").
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: add safe_thread_num_proc() for ThreadPoolExecutor callsites
safe_num_proc() correctly caps to 1 on macOS/Windows for process-based
multiprocessing, but format_conversion.py reuses it for ThreadPoolExecutor
workers. Threads share address space and are unaffected by spawn, so
capping to 1 makes image URL downloads sequential -- a real regression.
Add safe_thread_num_proc() that skips the platform guard but keeps the
cpu_count heuristic, and switch both ThreadPoolExecutor callsites in
format_conversion.py to use it.
* fix: remove double-wrap in dataset_num_proc + fix num_proc=1 in datasets route
- trainer.py:3009: Replace safe_num_proc(max(1, os.cpu_count() // 4))
with max(1, (os.cpu_count() or 1) // 4) to avoid double-wrapping
inside dataset_map_num_proc which already calls safe_num_proc
- trainer.py:15-20: Clarify comment on PYTHONPATH propagation
- datasets.py:445: Change num_proc=1 to num_proc=None for 10-row
preview slice (avoids unnecessary multiprocessing overhead)
* fix: guard os.cpu_count() against None in worker-count helpers
os.cpu_count() can return None on some platforms. Use (os.cpu_count() or 1)
to prevent TypeError in safe_num_proc() and safe_thread_num_proc().
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* gate on min uv version and shortcut python candidate search if known
* fix sort -V cross compat issue, run_quiet early exit on llamacpp, autolaunch
* update launch message
* Fix PR comments
* auto launch and find open port
* remove dev install
* Fix review findings: major-version guard, non-fatal port fallback, tty comment, restore local
* Remove autolaunch, clean up dead state and debug noise
- Remove find_open_port, TTY-gated autolaunch, and </dev/tty
redirection from install.sh; just print launch instructions
- Remove unused BEST_MAJOR variable from studio/setup.sh
- Remove stray "finished finding best python" debug echo
- Fix stale comment "below 3.12" to "below 3.11"
* Reject prerelease uv at exact minimum version boundary
* Remove 2>/dev/null from version_ge numeric comparisons
Let non-numeric version parts surface errors on stderr
instead of being silently swallowed.
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* fix: reconfigure stdout UTF-8 on Windows to prevent UnicodeEncodeError from emoji
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: default frontend_path when None to fix blank page when venv is pre-activated
* Restore Windows UTF-8 stdout fix dropped in earlier commit
The cp1252 console encoding on Windows cannot render emoji characters
used in startup messages (e.g. print("✅ Frontend loaded ...")).
This causes UnicodeEncodeError and crashes the server before it starts.
Place sys.stdout.reconfigure(encoding="utf-8", errors="replace") at the
top of run_server(), unconditionally before any print() or structlog
call, so all emoji output is covered -- including the frontend status
messages and silent=True paths that the original placement missed.
Guarded by sys.platform == "win32" and hasattr check, so it is a no-op
on Linux/macOS and safe in non-standard stdout environments (Jupyter,
piped IO).
* fix: preserve run_server(None) as headless, fix CLI frontend kwarg
Remove the frontend_path=None fallback in run_server() that changed
None from "headless/API-only" to "mount bundled frontend", breaking
backwards compatibility for embedders.
The blank-page bug was actually caused by the CLI wrappers always
passing frontend_path=frontend (even when frontend=None), which
overrode run_server()'s default. Fix studio.py and ui.py to only
pass frontend_path when the user explicitly sets --frontend.
* fix: use timeout loop for shutdown event in ui command
Match studio_default()'s shutdown loop that uses a 1-second timeout
on Event.wait(). Without a timeout, the bare wait() blocks at the C
level on Linux, preventing Python from delivering SIGINT (Ctrl+C).
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* fix: add CUDA minimum version check and abort for llama.cpp (>= 12.4)
- setup.ps1/setup.sh: abort with clear error if CUDA toolkit < 12.4
(llama.cpp requirement); link to cuda-toolkit-archive for upgrade
- setup.ps1: promote CUDA VS integration copy failure from WARN to
ERROR + exit 1; remove manual-copy hack instructions per Roland —
correct fix is re-installing CUDA/MSBuild, not a manual workaround
Fixes: https://github.com/unslothai/unsloth/issues/4437
Reported by: Sebastien
* fix: wipe stale studio venv when torch CUDA tag changes
When the NVIDIA driver is updated, the required PyTorch CUDA tag changes
(e.g. cu124 -> cu130) but setup.ps1 was silently reusing the existing
.venv, leaving the old torch wheel in place and breaking the UI for
everyone on the next setup run.
Before creating/reusing the venv, inspect the installed torch version
string. If its CUDA tag does not match what the current driver requires,
wipe the venv so we always get a clean, correct install.
* Fix CUDA version check: portability, non-fatal fallback, stale venv detection
- setup.sh: Replace grep -oP with POSIX sed for macOS compatibility
- setup.sh: Replace exit 1 with NVCC_PATH="" to fall back to CPU-only build
- setup.sh: Move version check before -DGGML_CUDA=ON append
- setup.sh: Add else branch warning when nvcc version is unparseable
- setup.ps1: Replace exit 1 with $NvccPath=$null for non-fatal CUDA fallback
- setup.ps1: Add driver vs toolkit guidance in version warning
- setup.ps1: Guard CUDA env/VS integration setup with if ($NvccPath)
- setup.ps1: VS integration catch: downgrade to WARN, restore source/dest paths
- setup.ps1: Stale venv: detect CPU torch and untagged wheels, not just +cuNNN
- setup.ps1: Stale venv: rebuild on failed torch import
- setup.ps1: Stale venv: wrap Remove-Item in try/catch for locked files
* Remove incorrect CUDA >= 12.4 check, keep only stale venv detection
llama.cpp has no hard minimum CUDA version -- it builds with CUDA as old
as 11.2 and degrades features gracefully via #if CUDART_VERSION guards.
The 12.4 figure was the default Docker/CI baseline, not a build requirement.
Reverted:
- CUDA version check in setup.sh (entirely removed)
- CUDA version check in setup.ps1 (entirely removed)
- VS integration catch block cosmetic changes (restored to main)
- if ($NvccPath) guard around CUDA env setup (not needed without version check)
Kept:
- Stale venv detection in setup.ps1: detects torch CUDA tag mismatch
(cu124 vs cu130, cpu vs cuXXX, broken torch import) and rebuilds venv
* Fix stale venv detection: incomplete venvs, timeout, fatal delete failure
- Add 30s timeout for torch import probe via ProcessStartInfo/WaitForExit
- Use Test-Path -PathType Container to reject files masquerading as venv dir
- Trigger rebuild when python.exe is missing (incomplete venv)
- Make Remove-Item failure fatal ([ERROR] + exit 1) instead of warn-and-continue
- Move $expectedTorchTag computation inside -not $shouldRebuild guard
---------
Co-authored-by: LeoBorcherding <LeoBorcherding@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* feat(chat): add server-side timings and context display for GGUF
Extract timings/usage metadata from llama-server SSE stream and forward
through the full stack. Replace client-side estimates with accurate
server-reported metrics (prompt eval, tok/s, token counts, cache hits).
Add context window usage bar to chat top nav.
* feat(chat): source badges with hover cards and 2-row collapse
- Add hover cards to source badges showing favicon, title, URL and
snippet description on hover
- Limit source badges to 2 rows with +X more expand/collapse
- Parse snippet from web search results for hover card descriptions
- Replace individual Source rendering with grouped SourcesGroup component
* fix(chat): add null guards for server timings edge cases
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(chat): reset contextUsage on thread switch, remove unused context-display
* fix(chat): stop double-counting completion tokens in tool-calling path
* fix(chat): skip metadata events in llm_assist consumers
* fix(chat): hide context usage bar in compare mode
* fix(chat): harden timings pipeline and context usage persistence
Accumulate prompt_ms, predicted_ms, and predicted_n from intermediate
tool-detection passes so the final metadata reflects total server work.
Persist contextUsage in message metadata (Dexie) and restore on thread
load. Add type guard in gguf_stream_chunks for unexpected dict events.
Clear contextUsage when entering compare mode.
* feat(chat): make GGUF stream metadata OpenAI-compatible
* fix(chat): address PR review feedback
* feat(chat): address PR review feedback
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix(recipe-studio): prevent fitView from zooming to wrong location on recipe load
* feat: add pymupdf/python-docx deps and unstructured uploads storage root
* feat: add POST /seed/upload-unstructured-file endpoint
* feat: add multi-file chunking with source_file column
* feat: update frontend types and API layer for multi-file upload
* feat: round-robin preview rows across source files
Ensures every uploaded file is represented in the preview table
by cycling through sources instead of just taking the first N rows.
* fix: disable OCR, fix auto-load timing, fix persistence on reload
- Disable pymupdf4llm OCR with write_images=False, show_progress=False
- Replace onAllUploaded callback with useEffect that detects uploading→done
transition (avoids stale closure reading empty file IDs)
- Fix importer to preserve file IDs from saved recipes instead of clearing
(clearing only happens at share time via sanitizeSeedForShare)
* fix: harden unstructured upload with input validation and state fixes
Validate block_id/file_id with alphanumeric regex to prevent path
traversal, use exact stem match for file deletion, add error handling
for metadata writes and empty files, fix React stale closures and
object mutations in upload loop, and correct validation logic for
unstructured seed resolved_paths.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: address PR review - legacy path import, share sanitizer, sync effect
Promote legacy source.path into resolved_paths for old unstructured
recipes, clear source.paths in share sanitizer to prevent leaking local
filesystem paths, and gate file sync effect to dialog open transition
so users can actually delete all uploaded files.
* fix: CSV column fix (BOM + whitespace + unnamed index re-save) for #4470
* fix: harden unstructured upload flow and polish dialog UX
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* feat: support full model GGUF export, disable incompatible methods in UI
* fix: resolve base model from config.json for venv_t5 export switching
* feat: detect BNB-quantized models and disable all export methods for quantized non-PEFT checkpoints
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: relocate Ollama Modelfile alongside GGUFs during non-PEFT export cleanup
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* full finetuning studio
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update studio/backend/core/training/trainer.py
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* One liner setup for unsloth studio
* Fix install scripts: system deps, activation bugs, curl/wget support
- install.sh: detect platform (macOS/Linux/WSL) and check for missing
system dependencies (cmake, git, build-essential, libcurl4-openssl-dev).
Prompt user once for permission to install all missing packages via
brew (macOS) or sudo apt-get (Linux/WSL). Add wget fallback via
download() helper since curl is not always present on minimal Linux
installs. Fix nested curl|sh stdin stealing by downloading uv installer
to a tempfile first. Replace venv activation (no-op in a pipe subshell)
with explicit --python flag for uv pip install and direct venv binary
invocation. Add idempotency guard for venv creation. Redirect stdin
on unsloth studio setup to prevent pipe consumption. On macOS, check
for Xcode Command Line Tools and trigger install if missing.
- install.ps1: wrap script body in Install-UnslothStudio function so
that errors use return instead of exit (exit kills the terminal when
run via irm|iex). Remove activate.ps1 invocation entirely -- use
explicit --python path for uv pip install and & $UnslothExe for
studio setup. This avoids both the child-scope activation bug (& vs
dot-source) and the execution policy error on default Windows systems.
Add winget availability check with clear error message. Fix PATH
refresh to append registry paths instead of replacing the session PATH.
Add uv installer fallback via astral.sh PowerShell script if winget
install does not put uv on PATH. Broaden Python version check to
accept 3.11-3.13. Add idempotency guard for venv creation.
- README.md: add wget one-liner alternative for systems without curl.
* Fix Tailwind CSS v4 .gitignore bug on Windows (#4444)
- Add .gitignore hiding workaround to setup.ps1 (matching existing
setup.sh logic) so venv .gitignore files containing "*" don't prevent
Tailwind's oxide scanner from finding .tsx source files
- Add CSS size validation to setup.sh, setup.ps1, and build.sh to catch
truncated Tailwind builds early
- Remove stray force-rebuild overrides that made the "skip build if
current" cache check dead code in both setup scripts
- Add rm -rf dist to build.sh to force clean rebuilds for wheel packaging
* Change default port 8000 to 8888, fix installer bugs, improve UX
- Change default Studio port from 8000 to 8888 across all entry points
(run.py, studio.py, ui.py, colab.py, vite.config.ts, setup scripts)
- Update launch banner: "Launching with studio venv..." to
"Launching Unsloth Studio... Please wait..."
- Add "Open your web browser" banner and rename labels
(Local -> Local Access, External -> Worldwide Web Address)
- Fix venv idempotency: check for bin/python instead of just directory
existence, clean up partial venvs on retry
- Fix build.sh CSS validation: handle empty CSS case that silently
bypassed the check with "integer expression expected"
- Fix install.sh sudo handling: try apt-get without sudo first (works
when root), then escalate with per-package tracking and user prompt
- Fix install.ps1: check exit code from studio setup, fail on error
- Add pciutils to WSL GGUF build dependencies
- Apply same smart apt-get escalation pattern to studio/setup.sh
* Use detected Python version for venv, abort on non-apt Linux
- install.ps1: detect existing Python 3.11/3.12/3.13 and use that
version for venv creation instead of always forcing 3.13
- install.sh: exit with error on non-apt Linux distros when required
packages cannot be auto-installed, instead of silently continuing
* Make sudo permission prompt more prominent with warning banner
* Add Accept [Y/n] sudo prompt to studio/setup.sh for consistency
* Fix native command exit code handling and sudo decline flow
install.ps1: Add $LASTEXITCODE checks after winget (Python), uv venv,
and uv pip install calls. $ErrorActionPreference only catches PowerShell
cmdlet errors, not native executable failures. The Python check also
handles winget returning non-zero for "already installed".
setup.sh: Skip llama-server build when user declines sudo or sudo is
unavailable. Previously the script continued to section 8 which would
fail with confusing errors (e.g. "gcc: command not found") since
build-essential was never installed.
* Move rm -rf llama.cpp inside build branch to preserve existing install
When _SKIP_GGUF_BUILD is set (user declined sudo or sudo unavailable),
the previous rm -rf would destroy an already-working llama-server before
the skip check ran. Move it inside the else branch so existing builds
are preserved when the rebuild is skipped.
---------
Co-authored-by: Daniel Han <danielhanchen@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Fixing Qwen3.5 bug and adding Outetts dependencies
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Apply suggestion from @danielhanchen
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Fix studio crash on Mac: vendor check_signal_escape_patterns from unsloth_zoo
Vendor the `check_signal_escape_patterns` function from
`unsloth_zoo.rl_environments` directly into `tools.py`. The function is
pure Python (only uses stdlib `ast`) and has zero GPU dependencies, but
importing it from unsloth_zoo triggers `unsloth_zoo.__init__` which calls
`get_device_type()` at module scope -- raising NotImplementedError on
Apple Silicon Macs.
By vendoring the code, the safety checks still run on all platforms
(Mac, Linux, Windows) without needing unsloth_zoo at all.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
- tool-ui-python.tsx: use explicit tuple type instead of `as const` to
match the mutable `[BundledTheme, BundledTheme]` expected by Streamdown
- chat-adapter.ts: add missing `argsText` field required by
ToolCallMessagePart and fix `args` type to use ReadonlyJSONObject
* Add elapsed timer to tool status pill in Studio
Show a count-up seconds timer (0s, 1s, 2s, ...) next to the tool status
text in the composer area. Helps users gauge how long a tool call (web
search, code execution) has been running. Timer resets when a new tool
starts and disappears when all tools finish.
* Fix tool call parsing, add tool outputs panel and reasoning copy button
Backend:
- Rewrite tool call XML parser to use balanced-brace JSON extraction
instead of greedy regex, fixing truncation on nested braces in
code/JSON arguments
- Handle optional closing tags (</tool_call>, </function>, </parameter>)
that models frequently omit
- Support bare <function=...> tags without <tool_call> wrapper
- Strip tool call markup from streamed content so raw XML never leaks
into the chat UI
- Use a persistent ~/studio_sandbox/ working directory for tool
execution so files persist across calls within a session
- Emit tool_start/tool_end SSE events so the frontend can display
tool inputs and outputs
Frontend:
- Add collapsible "Tool Outputs" panel below assistant messages showing
each tool call's input and output with copy buttons
- Add copy button to reasoning blocks
- Add elapsed timer to tool status pill
- Update project URLs in pyproject.toml (http -> https, add docs link)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add interactive HTML preview with fullscreen toggle for code blocks
HTML code fences now render an interactive sandboxed iframe preview
below the syntax-highlighted code, similar to how SVG fences show
an image preview. The iframe uses sandbox="allow-scripts" to allow
JavaScript execution while blocking access to the parent page.
Includes a fullscreen toggle (enlarge/minimize button) that expands
the preview into a viewport overlay, dismissible via button, Escape
key, or backdrop click. A streaming placeholder prevents partial
HTML from rendering mid-stream.
* Add tool call settings: auto-heal toggle, max iterations, timeout
Add three user-configurable tool call settings to the Studio Settings panel:
- Auto Heal Tool Calls: toggle to control fallback XML parsing of malformed
tool calls from model output (default: on)
- Max Tool Calls Per Message: slider 0-40 + Max to cap tool call iterations
per message (default: 10)
- Max Tool Call Duration: slider 1-30 minutes + Max to set per-tool-call
execution timeout (default: 5 minutes)
All settings persist to localStorage and flow through the full stack:
frontend store -> API request -> Pydantic model -> route -> llama_cpp -> tools.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix tool call timeout: respect no-limit and apply to web search
- Use a sentinel to distinguish timeout=None (no limit) from the default
(300s). Previously None was silently replaced with _EXEC_TIMEOUT.
- Pass the configured timeout to DDGS() for web searches so the setting
applies uniformly to all tool types.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add input validation bounds and per-thread sandbox isolation
- Add ge=0 constraint to max_tool_calls_per_message (rejects negative values)
- Add ge=1 constraint to tool_call_timeout (minimum 1 second)
- Thread session_id from frontend through backend to tool execution
- Scope sandbox directories per conversation: ~/studio_sandbox/{thread_id}/
- Backwards compatible: API callers without session_id use ~/studio_sandbox/
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix non-monotonic streaming and Python temp script path
- Split tool markup stripping into closed-only (mid-stream) and full
(final flush) to prevent cumulative text from shrinking mid-stream
- Enforce monotonicity: only emit when cleaned text grows, so the
proxy's delta logic (cumulative[len(prev_text):]) never breaks
- Place Python temp scripts in the sandbox workdir instead of /tmp so
sys.path[0] points to the sandbox and cross-call imports work
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Sanitize session_id to prevent path traversal in sandbox
Strip path separators and parent-dir references from session_id before
using it as a directory name. Verify the resolved path stays under
~/studio_sandbox/ as a second guard.
* feat(chat): proper assistant-ui tool call UIs with sources
Replace custom metadata-based ToolOutputsGroup with native assistant-ui
tool-call content parts. Backend SSE tool_start/tool_end events now emit
proper { type: "tool-call" } parts from the adapter, enabling per-tool
UIs registered via tools.by_name in MessagePrimitive.Parts.
- Web search: Globe icon, Source badges with favicons, auto-collapse
when LLM starts responding
- Python: Code icon, syntax-highlighted code via Streamdown/shiki,
output block with copy
- Terminal: Terminal icon, command in trigger, output with copy
- ToolGroup wraps consecutive tool calls (skips for single calls)
- Sources component renders URL badges at end of message
- Flattened code block CSS (single border, no nested boxes)
* fix(inference): respect empty enabled_tools allowlist
`if payload.enabled_tools:` is falsy for [], falling through to
ALL_TOOLS. Use `is not None` so an explicit empty list disables
all tools as intended.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Shine1i <wasimysdev@gmail.com>
* revert: remove frontend build caching from setup scripts
The mtime-based caching introduced in #4404/#4413 can incorrectly skip
frontend builds -- e.g. after git pull when filesystem timestamps are
not preserved, or after our Tailwind v4 discovery that the site-packages
.gitignore must be hidden before vite build (which the cached path
doesn't handle).
Always rebuild the frontend on setup. The build takes ~15s and is
safer than risking a stale dist/.
* revert: disable frontend build caching, keep code commented out
Caching disabled by always setting _NEED_FRONTEND_BUILD=true.
The mtime-based logic is preserved in comments for future re-enabling.
Reasons for disabling:
- Git does not preserve file timestamps, so cached dist/ can appear
newer than freshly checked-out source after a pull
- Tailwind v4 requires hiding site-packages/.gitignore before vite
build; the cache path bypasses this, producing broken CSS
* revert: always rebuild frontend, remove mtime caching
* revert: always rebuild frontend, override caching with _NEED_FRONTEND_BUILD=true
* Fix inference stall during prefill by removing retry storm
The _stream_with_retry method used a 0.5s read timeout and retried by
sending a brand new POST request each time. During prompt prefill (which
can take 5-30+ seconds for long contexts or reasoning models), this
caused 10-60 duplicate requests that forced llama-server to restart
processing from scratch each time, resulting in 10-20s stalls visible
as "Generating" with no progress in the UI.
Fix: send the request ONCE with a 120s read timeout for the initial
response headers. Cancel support during the prefill wait is handled by
a background thread that monitors cancel_event (checked every 0.3s)
and closes the response to unblock the httpx read immediately. This
preserves the ability to stop/cancel/refresh during generation.
The existing 0.5s timeout on the httpx.Client is still used by
_iter_text_cancellable for per-token cancel checking during streaming
(after prefill), which is unaffected by this change.
* Fix race in cancel watcher when response is not yet created
When cancel_event fires before client.stream() returns (response is
still None), the watcher would hit return and exit without closing
anything. The main thread stays blocked for up to 120s.
Fix: after cancel is requested, keep polling _response_ref every 0.1s
until the response object appears (then close it) or _cancel_closed
is set (main thread finished on its own).
* Minor cleanup: remove redundant None check, add debug logging in cancel watcher
Address Gemini review: cancel_event is guaranteed non-None when the
watcher thread runs, and logging the close exception aids debugging.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Retry r.close() on failure instead of giving up
If r.close() raises, stay in the polling loop and retry rather than
returning and leaving the main thread blocked for up to 120s.
* fix: keep short read timeout during token streaming
The prefill_timeout (read=120s) was passed to client.stream(), which
applied to ALL reads -- not just the initial response headers. This
meant _iter_text_cancellable's ReadTimeout-based cancel checking was
broken during token streaming: the Stop button could take up to 120s
to respond instead of 0.5s.
Fix: keep the client's short read timeout (0.5s) for the stream call.
During prefill, catch ReadTimeout in a loop and re-check cancel_event
instead of re-sending the POST (which was the original retry storm).
Once the first bytes arrive, yield the response with a PrependStream
wrapper so iter_text() sees the buffered first chunk.
This preserves both:
- Fast cancel during prefill (via cancel watcher + ReadTimeout loop)
- Fast cancel during streaming (via _iter_text_cancellable's 0.5s
ReadTimeout, which now fires correctly again)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: swap to short-timeout stream after prefill completes
Address two review issues:
1. _PrependStream did not inherit from httpx.SyncByteStream, so
Response.iter_raw() would raise RuntimeError. Replaced with a
_ShortTimeoutStream that inherits SyncByteStream properly.
2. client.stream() entry itself raises ReadTimeout during slow prefill
(before headers arrive). The previous fix tried to catch this at
the body-read level but missed the connection-level timeout.
New approach: keep the 120s read timeout for client.stream() so the
connection survives long prefills. Once headers arrive, replace the
response stream with _ShortTimeoutStream -- a wrapper that uses a
background reader thread and a Queue with a short get() timeout to
re-raise ReadTimeout at the original 0.5s interval. This way
_iter_text_cancellable's cancel-checking remains responsive during
token streaming while prefill gets the long timeout it needs.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: move _ShortTimeoutStream before LlamaCppBackend class
The class was placed inside LlamaCppBackend's body, splitting the
class in two and making _codec_mgr and other attributes unreachable.
Move it to module level before LlamaCppBackend.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: remove _ShortTimeoutStream, use watcher for all cancel
_ShortTimeoutStream had two critical issues:
1. Raising ReadTimeout from a generator kills it -- Python finalizes
generators after an uncaught exception, so the next next() call
hits StopIteration and streaming ends mid-response.
2. The unbounded Queue in the background reader loses backpressure,
causing memory spikes with slow clients.
Simpler approach: use the 120s read timeout for the entire stream and
rely on the cancel watcher thread for all cancellation (both prefill
and streaming). The watcher closes the response on cancel_event,
which unblocks any blocking httpx read within ~0.3s. This eliminates
the need for short timeout tricks entirely.
Cancel latency:
- Prefill: ~0.3s (watcher polls cancel_event every 0.3s)
- Streaming: ~0.3s (same watcher mechanism)
- Both faster than the old 0.5s ReadTimeout approach
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* docs: clarify cancel limitations in _stream_with_retry
The docstrings claimed ~0.3s cancel in all cases, but httpx cannot
interrupt a blocked read before the response object exists. Update
the docstrings to accurately describe the behavior:
- Cancel during prefill (header wait) is deferred until headers arrive
- Cancel during streaming works via response.close() from the watcher
- _iter_text_cancellable docstring updated to reflect the watcher-based
cancel mechanism instead of the old ReadTimeout polling
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Allow Windows setup to complete without NVIDIA GPU
setup.ps1 previously hard-exited if nvidia-smi was not found, blocking
setup entirely on CPU-only or non-NVIDIA machines. The backend already
supports CPU and MLX (Apple Silicon) in chat-only GGUF mode, and the
Linux/Mac setup.sh handles missing GPUs gracefully.
Changes:
- Convert the GPU check from a hard exit to a warning
- Guard CUDA toolkit installation behind $HasNvidiaSmi
- Install CPU-only PyTorch when no GPU is detected
- Build llama.cpp without CUDA flags when no GPU is present
- Update doc comment to reflect CPU support
* Cache frontend build across setup runs
Skip the frontend npm install + build if frontend/dist already exists.
Previously setup.ps1 nuked node_modules and package-lock.json on every
run, and both scripts always rebuilt even when dist/ was already present.
On a git clone editable install, the first setup run still builds the
frontend as before. Subsequent runs skip it, saving several minutes.
To force a rebuild, delete frontend/dist and re-run setup.
* Show pip progress for PyTorch download on Windows
The torch CUDA wheel is ~2.8 GB and the CPU wheel is ~300 MB. With
| Out-Null suppressing all output, the install appeared completely
frozen with no feedback. Remove | Out-Null for the torch install
lines so pip's download progress bar is visible. Add a size hint
so users know the download is expected to take a while.
Also moves the Triton success message inside the GPU branch so it
only prints when Triton was actually installed.
* Guard CUDA env re-sanitization behind GPU check in llama.cpp build
The CUDA_PATH re-sanitization block (lines 1020-1033) references
$CudaToolkitRoot which is only set when $HasNvidiaSmi is true and
the CUDA Toolkit section runs. On CPU-only machines, $CudaToolkitRoot
is null, causing Split-Path to throw:
Split-Path : Cannot bind argument to parameter 'Path' because it is null.
Wrap the entire block in `if ($HasNvidiaSmi -and $CudaToolkitRoot)`.
* Rebuild frontend when source files are newer than dist/
Instead of only checking if dist/ exists, compare source file timestamps
against the dist/ directory. If any file in frontend/src/ is newer than
dist/, trigger a rebuild. This handles the case where a developer pulls
new frontend changes and re-runs setup -- stale assets get rebuilt
automatically.
* Fix cmake not found on Windows after winget install
Two issues fixed:
1. After winget installs cmake, Refresh-Environment may not pick up the
new PATH entry (MSI PATH changes sometimes need a new shell). Added a
fallback that probes cmake's default install locations (Program Files,
LocalAppData) and adds the directory to PATH explicitly if found.
2. If cmake is still unavailable when the llama.cpp build starts (e.g.
winget failed silently or PATH was not updated), the build now skips
gracefully with a [SKIP] warning instead of crashing with
"cmake : The term 'cmake' is not recognized".
* Fix frontend rebuild detection and decouple oxc-validator install
Address review feedback:
- Check entire frontend/ directory for changes, not just src/.
The build also depends on package.json, vite.config.ts,
tailwind.config.ts, public/, and other config files. A change
to any of these now triggers a rebuild.
- Move oxc-validator npm install outside the frontend build gate
in setup.sh so it always runs on setup, matching setup.ps1
which already had it outside the gate.
* Show cmake errors on failure and retry CUDA VS integration with elevation
Two fixes for issue #4405 (Windows setup fails at cmake configure):
1. cmake configure: capture output and display it on failure instead of
piping to Out-Null. When the error mentions "No CUDA toolset found",
print a hint about the CUDA VS integration files.
2. CUDA VS integration copy: when the direct Copy-Item fails (needs
admin access to write to Program Files), retry with Start-Process
-Verb RunAs to prompt for elevation. This is the root cause of the
"No CUDA toolset found" cmake failure -- the .targets files that let
MSBuild compile .cu files are missing from the VS BuildCustomizations
directory.
* Address reviewer feedback: cmake PATH persistence, stale cache, torch error check
1. Persist cmake PATH to user registry so Refresh-Environment cannot
drop it later in the same setup run. Previously the process-only
PATH addition at phase 1 could vanish when Refresh-Environment
rebuilt PATH from registry during phase 2/3 installs.
2. Clean stale CMake cache before configure. If a previous run built
with CUDA and the user reruns without a GPU (or vice versa), the
cached GGML_CUDA value would persist. Now the build dir is removed
before configure.
3. Explicitly set -DGGML_CUDA=OFF for CPU-only builds instead of just
omitting CUDA flags. This prevents cmake from auto-detecting a
partial CUDA installation.
4. Fix CUDA cmake flag indentation -- was misaligned from the original
PR, now consistently indented inside the if/else block.
5. Fail hard if pip install torch returns a non-zero exit code instead
of silently continuing with a broken environment.
* Remove extra CUDA cmake flags to align Windows with Linux build
Drop GGML_CUDA_FA_ALL_QUANTS, GGML_CUDA_F16, GGML_CUDA_GRAPHS,
GGML_CUDA_FORCE_CUBLAS, and GGML_CUDA_PEER_MAX_BATCH_SIZE flags.
The Linux build in setup.sh only sets GGML_CUDA=ON and lets llama.cpp
use its defaults for everything else. Keep Windows consistent.
* Address reviewer round 2: GPU probe fallback, Triton check, stale binary rebuild
1. GPU detection: fallback to default nvidia-smi install locations
(Program Files\NVIDIA Corporation\NVSMI, System32) when nvidia-smi
is not on PATH. Prevents silent CPU-only provisioning on machines
that have a GPU but a broken PATH.
2. Triton: check $LASTEXITCODE after pip install and print [WARN]
on failure instead of unconditional [OK].
3. Stale llama-server: check CMakeCache.txt for GGML_CUDA setting
and rebuild if the existing binary does not match the current GPU
mode (e.g. CUDA binary on a now-CPU-only rerun, or vice versa).
* Fix frontend rebuild detection and npm dependency issues
Addresses reviewer feedback on the frontend caching logic:
1. setup.sh: Fix broken find command that caused exit under pipefail.
The piped `find | xargs find -newer` had paths after the expression
which GNU find rejects. Replaced with a simpler `find -maxdepth 1
-type f -newer dist/` that checks ALL top-level files (catches
index.html, bun.lock, etc. that the extension allowlist missed).
2. setup.sh: Guard oxc-validator npm install behind `command -v npm`
check. When the frontend build is skipped (dist/ is cached), Node
bootstrap is also skipped, so npm may not be available.
3. setup.ps1: Replace Get-ChildItem -Include with explicit path
probing for src/ and public/. PowerShell's -Include without a
trailing wildcard silently returns nothing, so src/public changes
were never detected. Also check ALL top-level files instead of
just .json/.ts/.js/.mjs extensions.
* Fix studio setup: venv isolation, centralized .venv_t5, uv targeting
- All platforms (including Colab) now create ~/.unsloth/studio/.venv
with --without-pip fallback for broken ensurepip environments
- Add --python sys.executable to uv pip install in install_python_stack.py
so uv targets the correct venv instead of system Python
- Centralize .venv_t5 bootstrap in transformers_version.py with proper
validation (checks required packages exist, not just non-empty dir)
- Replace ~150 lines of duplicated install code across 3 worker files
with calls to the shared _ensure_venv_t5_exists() helper
- Use uv-if-present with pip fallback; do not install uv at runtime
- Add site.addsitedir() shim in colab.py so notebook cells can import
studio packages from the venv without system-Python double-install
- Update .venv_t5 packages: huggingface_hub 1.3.0->1.7.1, add hf_xet
- Bump transformers pin 4.57.1->4.57.6 in requirements + constraints
- Add Fast-Install helper to setup.ps1 with uv+pip fallback
- Keep Colab-specific completion banner in setup.sh
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix nvidia-smi PATH persistence and cmake requirement for CPU-only
1. Store nvidia-smi as an absolute path ($NvidiaSmiExe) on first
detection. All later calls (Get-CudaComputeCapability,
Get-PytorchCudaTag, CUDA toolkit detection) use this absolute
path instead of relying on PATH. This survives Refresh-Environment
which rebuilds PATH from the registry and drops process-only
additions.
2. Make cmake fatal for CPU-only installs. CPU-only machines depend
entirely on llama-server for GGUF chat mode, so reporting "Setup
Complete!" without it is misleading. GPU machines can still skip
the llama-server build since they have other inference paths.
* Fix broken frontend freshness detection in setup scripts
- setup.sh: Replace broken `find | xargs find -newer` pipeline with
single `find ... -newer` call. The old pipeline produced "paths must
precede expression" errors (silently suppressed by 2>/dev/null),
causing top-level config changes to never trigger a rebuild.
- setup.sh: Add `command -v npm` guard to oxc-validator block so it
does not fail when Node was not installed (build-skip path).
- setup.ps1: Replace `Get-ChildItem -Include` (unreliable without
-Recurse on PS 5.1) with explicit directory paths for src/ and
public/ scanning.
- Both: Add *.html to tracked file patterns so index.html (Vite
entry point) changes trigger a rebuild.
- Both: Use -print -quit instead of piping to head -1 for efficiency.
* Fix bugs found during review of PRs #4404, #4400, #4399
- setup.sh: Add || true guard to find command that checks frontend/src
and frontend/public dirs, preventing script abort under set -euo
pipefail when either directory is missing
- colab.py: Use sys.path.insert(0, ...) instead of site.addsitedir()
so Studio venv packages take priority over system copies. Add warning
when venv is missing instead of silently failing.
- transformers_version.py: _venv_t5_is_valid() now checks installed
package versions via .dist-info metadata, not just directory presence.
Prevents false positives from stale or wrong-version packages.
- transformers_version.py: _install_to_venv_t5() now passes --upgrade
so pip replaces existing stale packages in the target directory.
- setup.ps1: CPU-only PyTorch install uses --index-url for cpu wheel
and all install commands use Fast-Install (uv with pip fallback).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix _venv_t5_is_valid dist-info loop exiting after first directory
Remove premature break that caused the loop over .dist-info directories
to exit after the first match even if it had no METADATA file. Now
continues iterating until a valid METADATA is found or all dirs are
exhausted.
* Capture error output on failure instead of discarding with Out-Null
setup.ps1: 6 locations changed from `| Out-Null` to `| Out-String` with
output shown on failure -- PyTorch GPU/CPU install, Triton install,
venv_t5 package loop, cmake llama-server and llama-quantize builds.
transformers_version.py: clean stale .venv_t5 directory before reinstall
when validation detects missing or version-mismatched packages.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix ModuleNotFoundError when CLI imports studio.backend.core
The backend uses bare "from utils.*" imports everywhere, relying on
backend/ being on sys.path. Workers and routes add it at startup, but
the CLI imports studio.backend.core as a package -- backend/ was never
added. Add sys.path setup at the top of core/__init__.py so lazy
imports resolve correctly regardless of entry point.
Fixes: unsloth inference unsloth/Qwen3-8B "who are you" crashing with
"No module named 'utils'"
* Fix frontend freshness check to detect all top-level file changes
The extension allowlist (*.json, *.ts, *.js, *.mjs, *.html) missed
files like bun.lock, so lockfile-only dependency changes could skip
the frontend rebuild. Check all top-level files instead.
* Add tiktoken to .venv_t5 for Qwen-family tokenizers
Qwen models use tiktoken-based tokenizers which fail when routed through
the transformers 5.x overlay without tiktoken installed. Add it to the
setup scripts (with deps for Windows) and runtime fallback list.
Integrates PR #4418.
* Fix tiktoken crash in _venv_t5_is_valid and stray brace in setup.ps1
_venv_t5_is_valid() crashed with ValueError on unpinned packages like
"tiktoken" (no ==version). Handle by splitting safely and skipping
version check for unpinned packages (existence check only).
Also remove stray closing brace in setup.ps1 tiktoken install block.
---------
Co-authored-by: Daniel Han <danielhanchen@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* feat(studio): infinite scroll for recommended models list
The model selector showed a hard cap of 4 GGUFs + 4 safetensors in the
Recommended section. Users who wanted to browse more had to search
manually on Hugging Face.
Backend: increase the default model pool from 8+8 to 40+40 (the HF
fetch already pulls 80, so no extra network cost).
Frontend: replace the static 4+4 cap with on-demand lazy loading.
A page counter tracks how many groups of 4 to show per category.
An IntersectionObserver on a sentinel div at the bottom of the list
increments the page when the user scrolls down. Models are interleaved
in groups of 4 GGUFs then 4 hub models per page for a balanced view.
Key implementation details:
- Callback ref for the sentinel so the observer attaches reliably on
first popover open (useRef would miss the initial mount)
- Observer disconnects after each fire and re-attaches via useEffect
with a 100ms layout delay to prevent runaway page loading
- VRAM info fetched incrementally via useRecommendedModelVram on the
visible slice only
- recommendedSet uses visible IDs so HF search dedup stays correct
* refactor: address review feedback on recommended infinite scroll
- Simplify visibleRecommendedIds: use findIndex to locate the GGUF/hub
split point instead of re-filtering the entire array each time.
recommendedIds is already sorted GGUF-first, so a single slice is
enough.
- Fix VRAM refetch churn: pass the full recommendedIds (stable across
page increments) to useRecommendedModelVram instead of the growing
visibleRecommendedIds slice. The hook derives its stableKey from the
sorted+joined input, so passing the same pool on every page avoids
redundant HF modelInfo requests.
* Comment out large unused packages from Studio setup requirements
Audited all packages installed by `unsloth studio setup` against actual
imports in unsloth, unsloth_zoo, and studio/backend. The following have
zero imports anywhere and are the largest offenders by disk size:
- gradio (148 MB) in studio.txt -- Studio uses React + FastAPI, not Gradio
- executorch (41.5 MB) in extras-no-deps.txt -- no imports found
- scikit-learn (31.8 MB) in extras.txt -- no imports found
- MeCab (19.9 MB) in extras.txt -- Japanese tokenizer, no imports found
- coremltools (10.2 MB) in extras.txt -- Apple CoreML, no imports found
- uroman (4.0 MB) in extras.txt -- romanization tool, no imports found
Total savings: ~255 MB (~32% of the 805 MB installed by setup).
Each line is commented out with the package size annotated so they can be
re-enabled easily if needed in the future.
* Restore scikit-learn -- needed by sentence_transformers
sentence_transformers is installed with --no-deps in extras-no-deps.txt,
so its sklearn dependency is not auto-resolved. Multiple modules in
sentence_transformers import sklearn at the top level (evaluation,
util/similarity), so removing scikit-learn would break embedding jobs.
* compare for 2 diff models
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* resolving gemini comments
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio): refine model-load toast stop action and compare selector sizing (#4369)
Co-authored-by: imagineer99 <samleejackson0@gmail.com>
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: imagineer99 <samleejackson0@gmail.com>
* studio: improve onboarding UX, tooltips, and training defaults
- Change splash text to "Train and run LLMs locally"
- Add "Chat Only" card with BubbleChatIcon to skip directly to chat
- Add Skip/Skip to Chat buttons in sidebar and footer
- Back button on step 1 returns to splash screen instead of being disabled
- Change "Watch video guide" to "Get started with our guide" with new URL
- Update intro text to mention all model types + chat
- Make all tooltips clickable (in addition to hover) via React context
- Strip surrounding quotes from pasted HF tokens
- Rename "Eval Split" to "Evaluation Split"
- Add SparklesIcon to "Auto Detect" format option
- Change step 4 heading to "Choose your training parameters"
- Default max_steps to 60
- Learning rate displayed in scientific notation with +/- stepper
- Context length options capped by model's max_position_embeddings (via AutoConfig)
- Fix "QLORA"/"LORA" to "QLoRA"/"LoRA" in summary step
- Backend: add max_position_embeddings to model config endpoint
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* compare for 2 diff models
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* resolving gemini comments
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: disable thinking for Qwen3.5 <9B and always for AI Assist
- Change Qwen3.5 thinking threshold from <=2B to <9B (0.8B, 2B, 4B
all disable thinking by default; 9B+ enables it)
- Always pass enable_thinking=False in AI Assist helper calls
(_run_with_helper and _generate_with_backend) regardless of chat
thinking settings
* studio: address PR review comments
- Extract _get_max_position_embeddings helper to DRY config extraction
- Fix "Skip to Chat" to navigate to /chat on step 1 (was /studio)
* fix: comment out debug print statements
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: skip Shiki highlighting for incomplete SVG code fences
While streaming SVG content, the syntax highlighter (Shiki) re-parses
the entire growing SVG on every token, blocking the main thread and
freezing the code area until the fence closes. Show a plain-text
preview for incomplete SVG fences instead, similar to how Mermaid
diagrams show a placeholder while streaming.
* studio: fix default top_k from 50/40 to 20 for chat inference
Per Qwen3.5 docs (unsloth.ai/docs/models/qwen3.5), top_k should be 20
for both thinking and non-thinking modes. The model-specific config in
inference_defaults.json already had top_k=20 for Qwen3.5, but the
generic fallback defaults were wrong:
- Frontend DEFAULT_INFERENCE_PARAMS.topK: 50 -> 20
- Backend generate_chat_completion top_k: 40 -> 20
- Backend generate_chat_completion_with_tools top_k: 40 -> 20
- Frontend title generation top_k: 40 -> 20
* studio: set universal inference defaults for unknown models
Default params for any model without specific config:
temperature=0.6, top_p=0.95, top_k=20, min_p=0.01,
presence_penalty=0.0, repetition_penalty=1.0
Models with entries in inference_defaults.json (Qwen3.5, Gemma-3,
Llama, etc.) override these with their recommended values.
Updated in: frontend DEFAULT_INFERENCE_PARAMS, backend Pydantic
request models, and backend generate_chat_completion defaults.
* studio: only trust_remote_code for unsloth/ models in AutoConfig
Only set trust_remote_code=True when the model name starts with
"unsloth/". All other models default to False for safety.
* studio: move Generating spinner above the composer
The "Generating" spinner was below the send message bar, causing
the bar to jump up and down. Move it above the composer in both
the regular thread view and the welcome/empty view.
* studio: adjust toast close button position away from edge
Move the X close button on toasts (like "Starting model...") from
top-1.5 to top-3 and add right-3, giving more breathing room from
the top-right corner.
* studio: make Think button smaller with tighter icon-text gap
Reduce gap from 1.5 to 0.5, padding from px-2.5/py-1 to px-2/py-0.5,
and icon from size-3.5 to size-3.
* studio: multiple onboarding and chat UX improvements
- Move Generating spinner above composer (fixes jumping send bar)
- Make Think button smaller with tighter icon-text gap
- Chat card now inside grid (same size as Audio/Embeddings cards)
- Rename "Chat Only" to "Chat"
- Chat card requires Continue to proceed (no auto-advance)
- Continue on Chat selection skips onboarding and goes to /chat
- Tooltip (i) click on Chat card doesn't trigger navigation
- Step 1 footer Back button goes back to splash (label is "Back")
- Splash "Skip Onboarding" renamed to "Skip to Chat", navigates to /chat
- Toast close button moved away from edge
* studio: align Skip to Chat button, add Skip to footer
- Sidebar "Skip to Chat" now uses primary (green) Button style with
arrow icon, full width, aligned like step items. Shows on all steps.
- Footer: added "Skip" outline button next to Continue that goes
directly to /studio with progress saved (markOnboardingDone)
* studio: change default max steps from 30 to 60 in toggle hook
The DEFAULT_MAX_STEPS in use-max-steps-epochs-toggle.ts was still 30,
used as fallback when toggling from epochs back to max steps.
* studio: extend context length options to 262K
CONTEXT_LENGTHS now includes 65536, 131072, 262144 in addition to
the existing 512-32768 range. The onboarding step filters these by
the model's max_position_embeddings (e.g. Nemotron-3-Nano-4B has
262144), showing powers of 2 up to the model's maximum.
* studio: auto-select LoRA vs QLoRA based on model size and GPU memory
After selecting a model in onboarding, detect the total model weight
file size from HF Hub (safetensors/bin files). Then estimate memory
needed: model_size_gb * 1.5 * context_scale, where context_scale is:
- <=8192 tokens: 1.0x
- >8192 tokens: 1.7x
- >=16384 tokens: 2.0x
- >=32768 tokens: 4.0x
If the estimate fits in free GPU VRAM, default to LoRA (16-bit).
Otherwise default to QLoRA (4-bit).
Backend changes:
- Add model_size_bytes to ModelDetails (models.py)
- Add _get_model_size_bytes() using HfApi.repo_info (routes/models.py)
- Add vram_free_gb to get_gpu_summary (hardware.py)
Frontend changes:
- Add autoSelectTrainingMethod() in training-config-store.ts
- Called after model defaults are loaded
- Add model_size_bytes to ModelConfigResponse type
- Add vramFreeGb to HardwareInfo hook
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: rename "Importing ML libraries..." to "Importing Unsloth..."
* studio: show model/dataset in training status, fix LoRA/QLoRA casing
- Training status now shows 'Training "model_name"' and 'Dataset = ...'
instead of generic "Starting training..."
- Fix Studio progress section to show QLoRA/LoRA instead of QLORA/LORA
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: rename 'Skip to Chat' to 'Skip Onboarding' on splash screen
* studio: add presence_penalty support for chat inference
Add presence_penalty as a parameter across the full stack:
- Backend: llama_cpp.py generate_chat_completion/with_tools, Pydantic
models (inference.py), routes/inference.py pass-through
- Frontend: InferenceParams type, DEFAULT_INFERENCE_PARAMS (0.0),
chat-adapter.ts payload, chat-settings-sheet.tsx slider (0-2),
model defaults loading from inference_defaults.json
- Set Qwen3.5 default presence_penalty to 1.5 per official docs
- Default for unknown models is 0.0 (off)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: fix Chat card deselecting Text and aligning with other cards
* studio: fix presence_penalty not loading from inference defaults
The inference_config.py load_inference_config() was not including
presence_penalty in the returned config dict, so the Qwen3.5
default of 1.5 from inference_defaults.json never reached the
frontend. Added it to the config builder.
* studio: add delete button for cached models in model selector
Add trash icon on each downloaded model row (GGUF and safetensors) with
confirmation dialog. Backend DELETE /api/models/delete-cached endpoint
uses huggingface_hub scan_cache_dir + delete_revisions to cleanly remove
cached repos, refusing if the model is currently loaded.
* studio: restore inference defaults, reasoning, and tools on page refresh
On page refresh with a model already loaded, the frontend was not
re-applying model-specific inference defaults (presence_penalty,
temperature, etc.) or restoring reasoning/tools support flags.
Backend: Add inference config, supports_reasoning, supports_tools,
and context_length to InferenceStatusResponse.
Frontend: In the refresh callback, when an active model is detected,
apply mergeRecommendedInference and restore reasoning/tools flags
with proper Qwen3.5 size-based defaults.
* studio: fix delete dialog closing before async completes
Prevent AlertDialogAction's default close behavior with
e.preventDefault() so the dialog stays open during deletion.
Also block onOpenChange dismiss while deleting is in progress.
* fix: add Dict and Any imports to inference models
* studio: fix Qwen3.5 reasoning threshold in frontend load path
The frontend loadModel handler had the old threshold (<=2) for
disabling reasoning on small Qwen3.5 models. Changed to <9 to
match the backend. This was causing 4B to not properly disable
thinking by default when auto-loaded.
* studio: move GGUF delete to per-variant level
For GGUF repos, the trash icon now appears on each downloaded variant
row inside the quantization expander instead of on the repo-level row.
Backend accepts optional variant param to delete specific GGUF files
(blob + symlink) rather than the entire repo cache.
* studio: restore ggufContextLength on page refresh
The Max Tokens slider was capped at 32768 on page refresh because
ggufContextLength was not restored from the status response.
Now set it from statusRes.context_length on reconnect.
* fix: remove <think> from Qwen3.5 response template marker
The train-on-responses-only feature uses template markers to find
where the assistant response starts. The Qwen3.5 response marker
included '<think>\n' which is only present when thinking mode is
enabled. With thinking disabled (default for <9B), the marker
never matched, causing 100% of samples to be dropped.
Changed response marker from '<|im_start|>assistant\n<think>\n'
to '<|im_start|>assistant\n' which works regardless of thinking mode.
* studio: fix sloth ASCII art alignment in training overlay
* fix: correct sloth ASCII art alignment to match Unsloth banner
* studio: add Python and terminal tool calling to chat
Register python and terminal tools alongside web search. Python
executor validates imports (stdlib only) via unsloth_zoo
rl_environments, runs code in a subprocess sandbox with 5-min
timeout and cancel support. Terminal executor blocks dangerous
commands (rm, sudo, etc.) and runs in a temp directory.
Update llama_cpp tool loop to show tool-specific status messages
and pass cancel_event through to executors. Rename composer
toggle from "Search" to "Tools" and show TerminalIcon for
execution status pills.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: fix Nemotron/transformers 5.x support, onboarding navigation, port binding
Backend:
- Dynamic transformers 5.x detection via tokenizer_config.json fetch
(checks for TokenizersBackend class, cached per-model)
- Bump transformers 5.x version from 5.2.0 to 5.3.0 across all workers,
setup scripts (setup.sh, setup.ps1)
- Auto-enable trust_remote_code for unsloth/* models needing transformers 5.x
(workaround for NemotronH config parsing bug in transformers)
- Auto-install mamba-ssm/causal-conv1d for SSM models (NemotronH, Falcon-H1)
with --no-build-isolation --no-deps to avoid torch version conflicts
- Add SO_REUSEADDR to port check in run.py (fixes Colab proxy stale connection
falsely reporting port as in-use)
Frontend:
- Fix "Skip to Chat" navigation: use window.location.href instead of React
Router navigate() to bypass useEffect redirect race
- Fix "Skip Onboarding" on splash: navigates to /studio (not /chat)
- Fix onboarding guard: only check isOnboardingDone() on initial mount
- Fix Chat card on step 1: add sr-only spacer for consistent alignment
- Fix Chat+Text both selected: clear RadioGroup value when Chat is selected
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: split tools toggle into Search and Code buttons
Replace the single "Tools" toggle with two independent toggles:
- "Search" (globe icon) enables web search only
- "Code" (terminal icon) enables Python and terminal execution
Add enabled_tools list field to the inference payload so the
backend only registers the tools the user has toggled on. Both
toggles appear in the main composer and the compare composer.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: fix tool calling import validation and error logging
Replace unsloth_zoo-dependent import checker with a standalone
ast-based validator using sys.stdlib_module_names. This properly
blocks non-stdlib imports (numpy, requests, etc.) and returns a
clear error message to the model so it can rewrite using only
stdlib.
Add full traceback to tool streaming error logs for debugging.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: parse gpt-oss harmony channels for clean safetensors chat output
gpt-oss models emit multi-channel output via harmony protocol tokens
(<|channel|>analysis<|message|>... and <|channel|>final<|message|>...).
TextIteratorStreamer with skip_special_tokens=True strips the special
tokens but leaves channel names concatenated with content, producing
garbled output like "analysisWe need to...assistantfinalHello!".
Add HarmonyTextStreamer that decodes with skip_special_tokens=False,
parses harmony markup via regex, and emits <think>analysis</think>
for the analysis channel and plain text for the final channel --
reusing the existing frontend reasoning UI.
Also expose supports_reasoning=True for non-GGUF gpt-oss models in
the /status endpoint so the frontend enables the Think toggle.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: use unsloth_zoo for Python sandbox validation
Set UNSLOTH_IS_PRESENT=1 and import check_python_modules and
check_signal_escape_patterns directly from unsloth_zoo instead
of a standalone fallback. This gives us the full Unsloth
validation including stdlib-only import checks and signal/timeout
escape pattern detection.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: allow all imports in Python tool sandbox
Remove stdlib-only import restriction. Keep signal escape
pattern detection via unsloth_zoo for safety.
* studio: fix ReadTimeout on tool streaming final pass
The 0.5s read timeout used for cancel-checking during streaming
also fires when waiting for the first response from llama-server
(e.g. reasoning model thinking for 15+ seconds). Add
_stream_with_retry() context manager that retries on ReadTimeout
while checking cancel_event, so the model has unlimited time to
think before producing the first token. Applied to both the
regular streaming path and the tool-calling final pass.
* fix: rewrite HarmonyTextStreamer with stateful incremental parsing
The delta-on-transformed approach had two critical bugs:
1. Before the full <|channel|>X<|message|> pattern was complete, the
strip-tokens fallback emitted "analysis" as plain text. Then when
the regex matched, _transform returned a completely different format
(<think>...</think>) and the delta was computed against the wrong
base string, producing fragments like "think>", "nk>", ">".
2. Even with full matches, the closing </think> tag shifted position
as content grew, so text[prev_len:] produced garbled deltas.
Replace with stateful incremental parsing that:
- Buffers until a complete channel+message pair is seen
- Emits <think> once when analysis channel first appears
- Streams analysis content deltas (computed on channel content directly)
- Emits </think> once when final channel first appears
- Streams final content deltas
- Closes open think tags in end()
Also skip the generic all_special_tokens stripping in
_clean_generated_text for gpt-oss since HarmonyTextStreamer already
produces clean output and the generic stripping was mangling <think>
tags.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: strip all <|...|> tokens in gpt-oss cleanup, not just harmony subset
The gpt-oss tokenizer has added tokens like <|return|> (id=200002) that
are not part of the harmony channel protocol but can leak into output.
The previous regex only stripped channel|message|start|end tokens.
Broaden the _clean_generated_text regex for gpt-oss to <\|[a-z_]+\|>
which catches all pipe-delimited tokens (return, constrain, reserved,
etc.) without matching <think>/<\/think> tags.
Verified: gpt-oss all_special_tokens are only <|return|>,
<|reserved_200017|>, <|startoftext|> -- none overlap with <think>.
The harmony tokens (channel, message, start, end) are added_tokens
but not in all_special_tokens.
* fix: hide config-only model repos from cached models list
Repos that only have metadata/config files cached (no .safetensors or
.bin weight files) were showing up in the Downloaded list with tiny
sizes like "1.8 KB" or "24 KB". These are just leftover config
snapshots from architecture checks, not usable models.
Filter the cached-models endpoint to only include repos that contain
actual model weight files (.safetensors or .bin).
* studio: fix toast description text contrast in dark mode
Add explicit !text-muted-foreground to toast description classNames
so secondary text (e.g. "Releases VRAM and resets inference state.")
is readable in dark mode.
* studio: fix Chat card icon alignment with size-4 spacer
Replace sr-only span (takes no space) with a size-4 shrink-0 div
matching the RadioGroupItem dimensions in other cards, so the Chat
icon aligns vertically with Text/Audio/Vision/Embeddings icons.
---------
Co-authored-by: workspace <user@workspace.local>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Manan17 <shahmanan170602@gmail.com>
Co-authored-by: Roland Tannous <rolandtannous@gravityq.ai>
* fix(llm_assist): disable thinking mode for helper model JSON output
Pass enable_thinking=False to generate_chat_completion() in both
_run_with_helper() and _generate_with_backend() so the Qwen3.5-4B
helper model produces clean JSON instead of wrapping responses in
<think> tags.
* fix(llm_assist): log per-request enable_thinking=False override
Add info-level log lines so the user can see that each helper/advisor
request overrides the server-level thinking default to False.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
- Add SVG preview rendering below code blocks using safe data URI
in <img> tag. Includes sanitization to block script/event handlers.
- Fix GGUF streaming crash: cache response.iter_text() iterator
instead of creating a new one on every loop iteration.
- Fix model selector showing "Select model..." after auto-load by
re-reading store state after setCheckpoint before setParams.
- Remove unused warmupToastShown variable (TS6133 build error).
- Change default suggestion to "Draw an SVG of a cute sloth".
The streaming loop used response.iter_text() with timeout=None, which
blocks until the next chunk arrives from llama-server. On large models
like Qwen3.5-27B where each token takes seconds, pressing Stop in the
UI would not take effect until the next token was produced.
Fix by using a 0.5s read timeout and a new _iter_text_cancellable()
helper that checks cancel_event between timeout windows and explicitly
closes the response when cancelled. Applied to both the regular chat
completion and tool-calling streaming paths.
Creative: temperature=1.5, min_p=0.1, top_p=Off (1.0), top_k=Off (0)
Precise: temperature=0.1, top_p=0.95, top_k=80, min_p=0.01
Also show "Off" in the slider label for top_p=1.0, top_k=0, and
repetition_penalty=1.0 since those values disable their respective
samplers. Changed top_k slider min from -1 to 0.
