* fix: reconfigure stdout UTF-8 on Windows to prevent UnicodeEncodeError from emoji
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: default frontend_path when None to fix blank page when venv is pre-activated
* Restore Windows UTF-8 stdout fix dropped in earlier commit
The cp1252 console encoding on Windows cannot render emoji characters
used in startup messages (e.g. print("✅ Frontend loaded ...")).
This causes UnicodeEncodeError and crashes the server before it starts.
Place sys.stdout.reconfigure(encoding="utf-8", errors="replace") at the
top of run_server(), unconditionally before any print() or structlog
call, so all emoji output is covered -- including the frontend status
messages and silent=True paths that the original placement missed.
Guarded by sys.platform == "win32" and hasattr check, so it is a no-op
on Linux/macOS and safe in non-standard stdout environments (Jupyter,
piped IO).
* fix: preserve run_server(None) as headless, fix CLI frontend kwarg
Remove the frontend_path=None fallback in run_server() that changed
None from "headless/API-only" to "mount bundled frontend", breaking
backwards compatibility for embedders.
The blank-page bug was actually caused by the CLI wrappers always
passing frontend_path=frontend (even when frontend=None), which
overrode run_server()'s default. Fix studio.py and ui.py to only
pass frontend_path when the user explicitly sets --frontend.
* fix: use timeout loop for shutdown event in ui command
Match studio_default()'s shutdown loop that uses a 1-second timeout
on Event.wait(). Without a timeout, the bare wait() blocks at the C
level on Linux, preventing Python from delivering SIGINT (Ctrl+C).
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* fix: add CUDA minimum version check and abort for llama.cpp (>= 12.4)
- setup.ps1/setup.sh: abort with clear error if CUDA toolkit < 12.4
(llama.cpp requirement); link to cuda-toolkit-archive for upgrade
- setup.ps1: promote CUDA VS integration copy failure from WARN to
ERROR + exit 1; remove manual-copy hack instructions per Roland —
correct fix is re-installing CUDA/MSBuild, not a manual workaround
Fixes: https://github.com/unslothai/unsloth/issues/4437
Reported by: Sebastien
* fix: wipe stale studio venv when torch CUDA tag changes
When the NVIDIA driver is updated, the required PyTorch CUDA tag changes
(e.g. cu124 -> cu130) but setup.ps1 was silently reusing the existing
.venv, leaving the old torch wheel in place and breaking the UI for
everyone on the next setup run.
Before creating/reusing the venv, inspect the installed torch version
string. If its CUDA tag does not match what the current driver requires,
wipe the venv so we always get a clean, correct install.
* Fix CUDA version check: portability, non-fatal fallback, stale venv detection
- setup.sh: Replace grep -oP with POSIX sed for macOS compatibility
- setup.sh: Replace exit 1 with NVCC_PATH="" to fall back to CPU-only build
- setup.sh: Move version check before -DGGML_CUDA=ON append
- setup.sh: Add else branch warning when nvcc version is unparseable
- setup.ps1: Replace exit 1 with $NvccPath=$null for non-fatal CUDA fallback
- setup.ps1: Add driver vs toolkit guidance in version warning
- setup.ps1: Guard CUDA env/VS integration setup with if ($NvccPath)
- setup.ps1: VS integration catch: downgrade to WARN, restore source/dest paths
- setup.ps1: Stale venv: detect CPU torch and untagged wheels, not just +cuNNN
- setup.ps1: Stale venv: rebuild on failed torch import
- setup.ps1: Stale venv: wrap Remove-Item in try/catch for locked files
* Remove incorrect CUDA >= 12.4 check, keep only stale venv detection
llama.cpp has no hard minimum CUDA version -- it builds with CUDA as old
as 11.2 and degrades features gracefully via #if CUDART_VERSION guards.
The 12.4 figure was the default Docker/CI baseline, not a build requirement.
Reverted:
- CUDA version check in setup.sh (entirely removed)
- CUDA version check in setup.ps1 (entirely removed)
- VS integration catch block cosmetic changes (restored to main)
- if ($NvccPath) guard around CUDA env setup (not needed without version check)
Kept:
- Stale venv detection in setup.ps1: detects torch CUDA tag mismatch
(cu124 vs cu130, cpu vs cuXXX, broken torch import) and rebuilds venv
* Fix stale venv detection: incomplete venvs, timeout, fatal delete failure
- Add 30s timeout for torch import probe via ProcessStartInfo/WaitForExit
- Use Test-Path -PathType Container to reject files masquerading as venv dir
- Trigger rebuild when python.exe is missing (incomplete venv)
- Make Remove-Item failure fatal ([ERROR] + exit 1) instead of warn-and-continue
- Move $expectedTorchTag computation inside -not $shouldRebuild guard
---------
Co-authored-by: LeoBorcherding <LeoBorcherding@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
The ruff pre-commit hook runs on all file types by default, including
.ipynb notebooks. Colab notebooks are authored in Colab's editor and
can contain IPython magics (%cd, !git) that ruff cannot parse. This
causes pre-commit.ci to fail on unrelated PRs when a notebook on main
has syntax ruff does not understand.
Add `exclude: '\.ipynb$'` to the ruff hook so notebooks are skipped.
* feat(chat): add server-side timings and context display for GGUF
Extract timings/usage metadata from llama-server SSE stream and forward
through the full stack. Replace client-side estimates with accurate
server-reported metrics (prompt eval, tok/s, token counts, cache hits).
Add context window usage bar to chat top nav.
* feat(chat): source badges with hover cards and 2-row collapse
- Add hover cards to source badges showing favicon, title, URL and
snippet description on hover
- Limit source badges to 2 rows with +X more expand/collapse
- Parse snippet from web search results for hover card descriptions
- Replace individual Source rendering with grouped SourcesGroup component
* fix(chat): add null guards for server timings edge cases
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(chat): reset contextUsage on thread switch, remove unused context-display
* fix(chat): stop double-counting completion tokens in tool-calling path
* fix(chat): skip metadata events in llm_assist consumers
* fix(chat): hide context usage bar in compare mode
* fix(chat): harden timings pipeline and context usage persistence
Accumulate prompt_ms, predicted_ms, and predicted_n from intermediate
tool-detection passes so the final metadata reflects total server work.
Persist contextUsage in message metadata (Dexie) and restore on thread
load. Add type guard in gguf_stream_chunks for unexpected dict events.
Clear contextUsage when entering compare mode.
* feat(chat): make GGUF stream metadata OpenAI-compatible
* fix(chat): address PR review feedback
* feat(chat): address PR review feedback
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix(recipe-studio): prevent fitView from zooming to wrong location on recipe load
* feat: add pymupdf/python-docx deps and unstructured uploads storage root
* feat: add POST /seed/upload-unstructured-file endpoint
* feat: add multi-file chunking with source_file column
* feat: update frontend types and API layer for multi-file upload
* feat: round-robin preview rows across source files
Ensures every uploaded file is represented in the preview table
by cycling through sources instead of just taking the first N rows.
* fix: disable OCR, fix auto-load timing, fix persistence on reload
- Disable pymupdf4llm OCR with write_images=False, show_progress=False
- Replace onAllUploaded callback with useEffect that detects uploading→done
transition (avoids stale closure reading empty file IDs)
- Fix importer to preserve file IDs from saved recipes instead of clearing
(clearing only happens at share time via sanitizeSeedForShare)
* fix: harden unstructured upload with input validation and state fixes
Validate block_id/file_id with alphanumeric regex to prevent path
traversal, use exact stem match for file deletion, add error handling
for metadata writes and empty files, fix React stale closures and
object mutations in upload loop, and correct validation logic for
unstructured seed resolved_paths.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: address PR review - legacy path import, share sanitizer, sync effect
Promote legacy source.path into resolved_paths for old unstructured
recipes, clear source.paths in share sanitizer to prevent leaking local
filesystem paths, and gate file sync effect to dialog open transition
so users can actually delete all uploaded files.
* fix: CSV column fix (BOM + whitespace + unnamed index re-save) for #4470
* fix: harden unstructured upload flow and polish dialog UX
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix Windows installer Python detection and winget error handling
The PowerShell installer crashes on some Windows machines due to two
issues:
1. Windows Store App Execution Aliases: Get-Command finds the stub at
WindowsApps\python.exe, then python --version writes to stderr.
With $ErrorActionPreference = "Stop" on PowerShell 5.1, stderr
from native commands becomes a terminating error, killing the
script before it tries to install Python.
2. winget "already installed" exit code: winget returns -1978335189
(APPINSTALLER_CLI_ERROR_UPDATE_NOT_APPLICABLE) when the package is
already at the latest version. The script treated any non-zero exit
as failure. The fallback Get-Command check could also find the
Store stub or fail if Python was partially uninstalled.
Changes:
- Add Find-CompatiblePython helper that tries the py launcher first,
then python3/python via Get-Command -All, explicitly skipping any
WindowsApps stubs. All invocations wrapped in try-catch so stderr
never triggers ErrorActionPreference.
- Replace exit-code-based winget error handling with outcome-based:
re-detect Python after install, retry with --force if not found,
show actionable manual install instructions on final failure.
- Deduplicate PATH entries in Refresh-SessionPath to prevent unbounded
growth from repeated machine+user path prepending.
* Address reviewer feedback: wrap winget calls, remove blanket WindowsApps filter
Three fixes based on code review:
1. Wrap all winget install calls in $ErrorActionPreference = "Continue"
blocks so that winget stderr (progress bars, warnings) does not
become a terminating error on PowerShell 5.1. This matches the
pattern already used in studio/setup.ps1 line 983.
2. Remove the blanket *\WindowsApps\* path filter that rejected all
WindowsApps executables including valid Microsoft Store Python
installs. Instead, rely on the existing try-catch + version regex
probing to determine if a candidate is functional. Non-functional
entries (App Execution Alias stubs) fail the try-catch and are
skipped naturally.
3. Use $pyLauncher.Source (resolved path) instead of bare py name,
add -CommandType Application to avoid matching aliases/functions,
and derive winget package ID from $PythonVersion variable instead
of hardcoding Python.Python.3.13.
* Add back WindowsApps filter for python3/python fallback path
The App Execution Alias stubs in WindowsApps can open the Microsoft
Store as a side effect when invoked, even though the try-catch handles
the error. Since the py launcher (tried first) already detects
legitimate Store Python -- Store packages include py since Python
3.11 -- filtering WindowsApps in the python3/python fallback is safe
and avoids the Store popup.
---------
Co-authored-by: Daniel Han <danielhanchen@users.noreply.github.com>
* fix(install.ps1): detect AMD/no-NVIDIA GPU early and guard unsloth.exe existence
When a user has an AMD GPU (no nvidia-smi), uv's --torch-backend=auto
resolves to CPU torch, which constrains the solver to unsloth==2024.8.
That ancient release has no unsloth.exe CLI entry point, so the subsequent
& \ studio setup call throws a confusing PowerShell
'module could not be loaded' CommandNotFoundException instead of a
clear error.
Two fixes:
- Detect nvidia-smi early; if no NVIDIA GPU is found, print a clear
error explaining AMD/Intel GPUs are unsupported and exit before
wasting time installing the wrong package version.
- Guard Test-Path \ before invoking it, so any future case
where the CLI entry point is missing produces a readable error
instead of a cryptic PowerShell exception.
Fixes: unsloth_studio\Scripts\unsloth.exe CommandNotFoundException
on AMD GPU systems (Windows).
* fix(install.ps1): correct GPU support message - AMD is Linux-only via ROCm
* Slim down to just the unsloth.exe existence guard
Remove the early NVIDIA GPU detection gate -- Studio supports Windows
and Mac without a GPU (finetuning is simply disabled). The GPU gate
was blocking legitimate non-NVIDIA users from installing.
Keep only the Test-Path guard on unsloth.exe before invoking it. This
turns the confusing PowerShell CommandNotFoundException into a clear
error message pointing at the likely cause (older unsloth version
resolved by the package solver that does not include the Studio CLI).
* Fix quickstart link in unsloth.exe guard message
---------
Co-authored-by: LeoBorcherding <LeoBorcherding@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* feat: support full model GGUF export, disable incompatible methods in UI
* fix: resolve base model from config.json for venv_t5 export switching
* feat: detect BNB-quantized models and disable all export methods for quantized non-PEFT checkpoints
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: relocate Ollama Modelfile alongside GGUFs during non-PEFT export cleanup
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix macOS install.sh: stdin consumption and Python discovery
Two issues when running `curl | sh` on macOS:
1. Commands like `brew install` consume bytes from the piped stdin,
causing the shell to lose its place in the script. The remaining
source code gets printed as text instead of being executed, so
users have to run the installer twice. Fixed by redirecting stdin
from /dev/null for brew, apt-get, xcode-select, and the uv
installer subprocess.
2. setup.sh searches for Python 3.11-3.13 on the system PATH via
`compgen -c`. On macOS systems that only have Python 3.9 and/or
3.14, this fails with "No Python version between 3.11 and 3.13
found" even though uv already installed Python 3.13 into the
venv. Fixed by adding the venv's bin/ to PATH before invoking
`unsloth studio setup`.
* Guard PATH export against empty VENV_ABS_BIN
If cd into the venv bin/ fails, VENV_ABS_BIN would be empty and
PATH would start with ":", causing the current directory to be
searched for executables. Wrap the export in a non-empty check.
* full finetuning studio
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update studio/backend/core/training/trainer.py
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* One liner setup for unsloth studio
* Fix install scripts: system deps, activation bugs, curl/wget support
- install.sh: detect platform (macOS/Linux/WSL) and check for missing
system dependencies (cmake, git, build-essential, libcurl4-openssl-dev).
Prompt user once for permission to install all missing packages via
brew (macOS) or sudo apt-get (Linux/WSL). Add wget fallback via
download() helper since curl is not always present on minimal Linux
installs. Fix nested curl|sh stdin stealing by downloading uv installer
to a tempfile first. Replace venv activation (no-op in a pipe subshell)
with explicit --python flag for uv pip install and direct venv binary
invocation. Add idempotency guard for venv creation. Redirect stdin
on unsloth studio setup to prevent pipe consumption. On macOS, check
for Xcode Command Line Tools and trigger install if missing.
- install.ps1: wrap script body in Install-UnslothStudio function so
that errors use return instead of exit (exit kills the terminal when
run via irm|iex). Remove activate.ps1 invocation entirely -- use
explicit --python path for uv pip install and & $UnslothExe for
studio setup. This avoids both the child-scope activation bug (& vs
dot-source) and the execution policy error on default Windows systems.
Add winget availability check with clear error message. Fix PATH
refresh to append registry paths instead of replacing the session PATH.
Add uv installer fallback via astral.sh PowerShell script if winget
install does not put uv on PATH. Broaden Python version check to
accept 3.11-3.13. Add idempotency guard for venv creation.
- README.md: add wget one-liner alternative for systems without curl.
* Fix Tailwind CSS v4 .gitignore bug on Windows (#4444)
- Add .gitignore hiding workaround to setup.ps1 (matching existing
setup.sh logic) so venv .gitignore files containing "*" don't prevent
Tailwind's oxide scanner from finding .tsx source files
- Add CSS size validation to setup.sh, setup.ps1, and build.sh to catch
truncated Tailwind builds early
- Remove stray force-rebuild overrides that made the "skip build if
current" cache check dead code in both setup scripts
- Add rm -rf dist to build.sh to force clean rebuilds for wheel packaging
* Change default port 8000 to 8888, fix installer bugs, improve UX
- Change default Studio port from 8000 to 8888 across all entry points
(run.py, studio.py, ui.py, colab.py, vite.config.ts, setup scripts)
- Update launch banner: "Launching with studio venv..." to
"Launching Unsloth Studio... Please wait..."
- Add "Open your web browser" banner and rename labels
(Local -> Local Access, External -> Worldwide Web Address)
- Fix venv idempotency: check for bin/python instead of just directory
existence, clean up partial venvs on retry
- Fix build.sh CSS validation: handle empty CSS case that silently
bypassed the check with "integer expression expected"
- Fix install.sh sudo handling: try apt-get without sudo first (works
when root), then escalate with per-package tracking and user prompt
- Fix install.ps1: check exit code from studio setup, fail on error
- Add pciutils to WSL GGUF build dependencies
- Apply same smart apt-get escalation pattern to studio/setup.sh
* Use detected Python version for venv, abort on non-apt Linux
- install.ps1: detect existing Python 3.11/3.12/3.13 and use that
version for venv creation instead of always forcing 3.13
- install.sh: exit with error on non-apt Linux distros when required
packages cannot be auto-installed, instead of silently continuing
* Make sudo permission prompt more prominent with warning banner
* Add Accept [Y/n] sudo prompt to studio/setup.sh for consistency
* Fix native command exit code handling and sudo decline flow
install.ps1: Add $LASTEXITCODE checks after winget (Python), uv venv,
and uv pip install calls. $ErrorActionPreference only catches PowerShell
cmdlet errors, not native executable failures. The Python check also
handles winget returning non-zero for "already installed".
setup.sh: Skip llama-server build when user declines sudo or sudo is
unavailable. Previously the script continued to section 8 which would
fail with confusing errors (e.g. "gcc: command not found") since
build-essential was never installed.
* Move rm -rf llama.cpp inside build branch to preserve existing install
When _SKIP_GGUF_BUILD is set (user declined sudo or sudo unavailable),
the previous rm -rf would destroy an already-working llama-server before
the skip check ran. Move it inside the else branch so existing builds
are preserved when the rebuild is skipped.
---------
Co-authored-by: Daniel Han <danielhanchen@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Fixing Qwen3.5 bug and adding Outetts dependencies
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Apply suggestion from @danielhanchen
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Fix studio crash on Mac: vendor check_signal_escape_patterns from unsloth_zoo
Vendor the `check_signal_escape_patterns` function from
`unsloth_zoo.rl_environments` directly into `tools.py`. The function is
pure Python (only uses stdlib `ast`) and has zero GPU dependencies, but
importing it from unsloth_zoo triggers `unsloth_zoo.__init__` which calls
`get_device_type()` at module scope -- raising NotImplementedError on
Apple Silicon Macs.
By vendoring the code, the safety checks still run on all platforms
(Mac, Linux, Windows) without needing unsloth_zoo at all.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
- tool-ui-python.tsx: use explicit tuple type instead of `as const` to
match the mutable `[BundledTheme, BundledTheme]` expected by Streamdown
- chat-adapter.ts: add missing `argsText` field required by
ToolCallMessagePart and fix `args` type to use ReadonlyJSONObject
* Add elapsed timer to tool status pill in Studio
Show a count-up seconds timer (0s, 1s, 2s, ...) next to the tool status
text in the composer area. Helps users gauge how long a tool call (web
search, code execution) has been running. Timer resets when a new tool
starts and disappears when all tools finish.
* Fix tool call parsing, add tool outputs panel and reasoning copy button
Backend:
- Rewrite tool call XML parser to use balanced-brace JSON extraction
instead of greedy regex, fixing truncation on nested braces in
code/JSON arguments
- Handle optional closing tags (</tool_call>, </function>, </parameter>)
that models frequently omit
- Support bare <function=...> tags without <tool_call> wrapper
- Strip tool call markup from streamed content so raw XML never leaks
into the chat UI
- Use a persistent ~/studio_sandbox/ working directory for tool
execution so files persist across calls within a session
- Emit tool_start/tool_end SSE events so the frontend can display
tool inputs and outputs
Frontend:
- Add collapsible "Tool Outputs" panel below assistant messages showing
each tool call's input and output with copy buttons
- Add copy button to reasoning blocks
- Add elapsed timer to tool status pill
- Update project URLs in pyproject.toml (http -> https, add docs link)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add interactive HTML preview with fullscreen toggle for code blocks
HTML code fences now render an interactive sandboxed iframe preview
below the syntax-highlighted code, similar to how SVG fences show
an image preview. The iframe uses sandbox="allow-scripts" to allow
JavaScript execution while blocking access to the parent page.
Includes a fullscreen toggle (enlarge/minimize button) that expands
the preview into a viewport overlay, dismissible via button, Escape
key, or backdrop click. A streaming placeholder prevents partial
HTML from rendering mid-stream.
* Add tool call settings: auto-heal toggle, max iterations, timeout
Add three user-configurable tool call settings to the Studio Settings panel:
- Auto Heal Tool Calls: toggle to control fallback XML parsing of malformed
tool calls from model output (default: on)
- Max Tool Calls Per Message: slider 0-40 + Max to cap tool call iterations
per message (default: 10)
- Max Tool Call Duration: slider 1-30 minutes + Max to set per-tool-call
execution timeout (default: 5 minutes)
All settings persist to localStorage and flow through the full stack:
frontend store -> API request -> Pydantic model -> route -> llama_cpp -> tools.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix tool call timeout: respect no-limit and apply to web search
- Use a sentinel to distinguish timeout=None (no limit) from the default
(300s). Previously None was silently replaced with _EXEC_TIMEOUT.
- Pass the configured timeout to DDGS() for web searches so the setting
applies uniformly to all tool types.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add input validation bounds and per-thread sandbox isolation
- Add ge=0 constraint to max_tool_calls_per_message (rejects negative values)
- Add ge=1 constraint to tool_call_timeout (minimum 1 second)
- Thread session_id from frontend through backend to tool execution
- Scope sandbox directories per conversation: ~/studio_sandbox/{thread_id}/
- Backwards compatible: API callers without session_id use ~/studio_sandbox/
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix non-monotonic streaming and Python temp script path
- Split tool markup stripping into closed-only (mid-stream) and full
(final flush) to prevent cumulative text from shrinking mid-stream
- Enforce monotonicity: only emit when cleaned text grows, so the
proxy's delta logic (cumulative[len(prev_text):]) never breaks
- Place Python temp scripts in the sandbox workdir instead of /tmp so
sys.path[0] points to the sandbox and cross-call imports work
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Sanitize session_id to prevent path traversal in sandbox
Strip path separators and parent-dir references from session_id before
using it as a directory name. Verify the resolved path stays under
~/studio_sandbox/ as a second guard.
* feat(chat): proper assistant-ui tool call UIs with sources
Replace custom metadata-based ToolOutputsGroup with native assistant-ui
tool-call content parts. Backend SSE tool_start/tool_end events now emit
proper { type: "tool-call" } parts from the adapter, enabling per-tool
UIs registered via tools.by_name in MessagePrimitive.Parts.
- Web search: Globe icon, Source badges with favicons, auto-collapse
when LLM starts responding
- Python: Code icon, syntax-highlighted code via Streamdown/shiki,
output block with copy
- Terminal: Terminal icon, command in trigger, output with copy
- ToolGroup wraps consecutive tool calls (skips for single calls)
- Sources component renders URL badges at end of message
- Flattened code block CSS (single border, no nested boxes)
* fix(inference): respect empty enabled_tools allowlist
`if payload.enabled_tools:` is falsy for [], falling through to
ALL_TOOLS. Use `is not None` so an explicit empty list disables
all tools as intended.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Shine1i <wasimysdev@gmail.com>
* revert: remove frontend build caching from setup scripts
The mtime-based caching introduced in #4404/#4413 can incorrectly skip
frontend builds -- e.g. after git pull when filesystem timestamps are
not preserved, or after our Tailwind v4 discovery that the site-packages
.gitignore must be hidden before vite build (which the cached path
doesn't handle).
Always rebuild the frontend on setup. The build takes ~15s and is
safer than risking a stale dist/.
* revert: disable frontend build caching, keep code commented out
Caching disabled by always setting _NEED_FRONTEND_BUILD=true.
The mtime-based logic is preserved in comments for future re-enabling.
Reasons for disabling:
- Git does not preserve file timestamps, so cached dist/ can appear
newer than freshly checked-out source after a pull
- Tailwind v4 requires hiding site-packages/.gitignore before vite
build; the cache path bypasses this, producing broken CSS
* revert: always rebuild frontend, remove mtime caching
* revert: always rebuild frontend, override caching with _NEED_FRONTEND_BUILD=true
* fix: exclude nemotron_h from flex_attention
NemotronHForCausalLM does not support flex_attention and raises:
NotImplementedError: NemotronHForCausalLM does not support an
attention implementation through torch's flex_attention.
Add nemotron_h to the exclusion list alongside gpt_oss and mllama
so Unsloth falls back to the default attention implementation.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix inference stall during prefill by removing retry storm
The _stream_with_retry method used a 0.5s read timeout and retried by
sending a brand new POST request each time. During prompt prefill (which
can take 5-30+ seconds for long contexts or reasoning models), this
caused 10-60 duplicate requests that forced llama-server to restart
processing from scratch each time, resulting in 10-20s stalls visible
as "Generating" with no progress in the UI.
Fix: send the request ONCE with a 120s read timeout for the initial
response headers. Cancel support during the prefill wait is handled by
a background thread that monitors cancel_event (checked every 0.3s)
and closes the response to unblock the httpx read immediately. This
preserves the ability to stop/cancel/refresh during generation.
The existing 0.5s timeout on the httpx.Client is still used by
_iter_text_cancellable for per-token cancel checking during streaming
(after prefill), which is unaffected by this change.
* Fix race in cancel watcher when response is not yet created
When cancel_event fires before client.stream() returns (response is
still None), the watcher would hit return and exit without closing
anything. The main thread stays blocked for up to 120s.
Fix: after cancel is requested, keep polling _response_ref every 0.1s
until the response object appears (then close it) or _cancel_closed
is set (main thread finished on its own).
* Minor cleanup: remove redundant None check, add debug logging in cancel watcher
Address Gemini review: cancel_event is guaranteed non-None when the
watcher thread runs, and logging the close exception aids debugging.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Retry r.close() on failure instead of giving up
If r.close() raises, stay in the polling loop and retry rather than
returning and leaving the main thread blocked for up to 120s.
* fix: keep short read timeout during token streaming
The prefill_timeout (read=120s) was passed to client.stream(), which
applied to ALL reads -- not just the initial response headers. This
meant _iter_text_cancellable's ReadTimeout-based cancel checking was
broken during token streaming: the Stop button could take up to 120s
to respond instead of 0.5s.
Fix: keep the client's short read timeout (0.5s) for the stream call.
During prefill, catch ReadTimeout in a loop and re-check cancel_event
instead of re-sending the POST (which was the original retry storm).
Once the first bytes arrive, yield the response with a PrependStream
wrapper so iter_text() sees the buffered first chunk.
This preserves both:
- Fast cancel during prefill (via cancel watcher + ReadTimeout loop)
- Fast cancel during streaming (via _iter_text_cancellable's 0.5s
ReadTimeout, which now fires correctly again)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: swap to short-timeout stream after prefill completes
Address two review issues:
1. _PrependStream did not inherit from httpx.SyncByteStream, so
Response.iter_raw() would raise RuntimeError. Replaced with a
_ShortTimeoutStream that inherits SyncByteStream properly.
2. client.stream() entry itself raises ReadTimeout during slow prefill
(before headers arrive). The previous fix tried to catch this at
the body-read level but missed the connection-level timeout.
New approach: keep the 120s read timeout for client.stream() so the
connection survives long prefills. Once headers arrive, replace the
response stream with _ShortTimeoutStream -- a wrapper that uses a
background reader thread and a Queue with a short get() timeout to
re-raise ReadTimeout at the original 0.5s interval. This way
_iter_text_cancellable's cancel-checking remains responsive during
token streaming while prefill gets the long timeout it needs.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: move _ShortTimeoutStream before LlamaCppBackend class
The class was placed inside LlamaCppBackend's body, splitting the
class in two and making _codec_mgr and other attributes unreachable.
Move it to module level before LlamaCppBackend.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: remove _ShortTimeoutStream, use watcher for all cancel
_ShortTimeoutStream had two critical issues:
1. Raising ReadTimeout from a generator kills it -- Python finalizes
generators after an uncaught exception, so the next next() call
hits StopIteration and streaming ends mid-response.
2. The unbounded Queue in the background reader loses backpressure,
causing memory spikes with slow clients.
Simpler approach: use the 120s read timeout for the entire stream and
rely on the cancel watcher thread for all cancellation (both prefill
and streaming). The watcher closes the response on cancel_event,
which unblocks any blocking httpx read within ~0.3s. This eliminates
the need for short timeout tricks entirely.
Cancel latency:
- Prefill: ~0.3s (watcher polls cancel_event every 0.3s)
- Streaming: ~0.3s (same watcher mechanism)
- Both faster than the old 0.5s ReadTimeout approach
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* docs: clarify cancel limitations in _stream_with_retry
The docstrings claimed ~0.3s cancel in all cases, but httpx cannot
interrupt a blocked read before the response object exists. Update
the docstrings to accurately describe the behavior:
- Cancel during prefill (header wait) is deferred until headers arrive
- Cancel during streaming works via response.close() from the watcher
- _iter_text_cancellable docstring updated to reflect the watcher-based
cancel mechanism instead of the old ReadTimeout polling
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
