* Fix Studio crash on Anaconda Python due to platform._sys_version() parse failure
Anaconda and conda-forge modify sys.version to include distributor
metadata between pipe characters, e.g.:
3.12.4 | packaged by Anaconda, Inc. | (main, ...) [MSC v.1929 ...]
Python's platform._sys_version() has a hardcoded regex that cannot
parse this format, raising ValueError. CPython closed this as "not
planned" (cpython#102396) since Anaconda modified the binary.
This breaks the import chain: run.py -> structlog -> rich -> attrs,
which calls platform.python_implementation() at module scope.
Fix: before any library imports, strip the pipe segments, parse the
cleaned version string via the standard parser, and cache the result
under the original sys.version key so all subsequent platform calls
hit the cache.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add defensive fallback for unpaired pipe edge cases in version patch
Address Gemini review suggestion: if the paired-pipe regex leaves
residual pipes (hypothetical single-pipe distributor metadata), fall
back to extracting the version number and the parenthesized build
info directly. Wrap the entire patch in try/except so unexpected
version string formats degrade gracefully instead of crashing the
patch itself.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Refactor into shared _platform_compat module, cover colab.py entrypoint
Address reviewer feedback:
1. Extract the Anaconda/conda-forge sys.version fix into a shared
_platform_compat.py module that wraps platform._sys_version() with
a retry-on-ValueError fallback. This is more robust than cache-seeding
because it handles all future platform._sys_version() calls, not just
the first one.
2. Import the fix from both run.py and colab.py entrypoints, so Studio
no longer crashes on Anaconda Python regardless of the launch path.
3. The wrapper is idempotent (guarded by a flag) and handles edge cases:
paired pipes (Anaconda, conda-forge), unpaired pipes (hypothetical),
and standard CPython strings (no-op since ValueError is never raised).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Replace monkey-patch with cache-prime, fix colab.py duplicate sys.path, cover main.py
- Rewrite _platform_compat.py: replace function-wrapping monkey-patch with
one-shot cache seed (_seed_sys_version_cache). Parses cleaned sys.version
once and seeds platform._sys_version_cache so the stdlib parser never sees
the problematic Anaconda/conda-forge pipe-delimited string. No function
replacement, no idempotency flag, no reload edge cases.
- colab.py: remove duplicate backend_path sys.path insertion after
_bootstrap_studio_venv(). The early insertion (before _platform_compat
import) already covers it. This also fixes backend/ ending up behind
venv site-packages in sys.path ordering.
- run.py: move PYTHONWARNINGS=ignore before _platform_compat import to
preserve original intent of suppressing warnings early.
- main.py: add sys.path + _platform_compat import before route imports,
covering the direct `uvicorn main:app` launch path.
- Add test_platform_compat.py with 7 tests covering Anaconda, conda-forge,
and standard CPython version strings, plus the loggers import chain.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Remove test_platform_compat.py from PR
* Handle Format B conda-forge version strings with duplicate paren groups
Some conda-forge builds produce sys.version with the build info both
before and after the pipe label (e.g. "3.9.7 (default, ...) | packaged
by conda-forge | (default, ...) \n[GCC 7.5.0]"). After stripping the
pipe segment, two consecutive (...) groups remain, which still fails
platform._sys_version(). Add a second regex pass to drop the duplicate
paren group.
* Guard _sys_version call with try/except to avoid making things worse
If the cleaned version string is still unparseable by the stdlib regex
(e.g. nested parens, exotic multi-pipe formats), silently give up
instead of letting ValueError propagate at import time -- which would
be a worse crash than the original deferred one.
---------
Co-authored-by: Daniel Han <danielhanchen@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix: reconfigure stdout UTF-8 on Windows to prevent UnicodeEncodeError from emoji
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: default frontend_path when None to fix blank page when venv is pre-activated
* Restore Windows UTF-8 stdout fix dropped in earlier commit
The cp1252 console encoding on Windows cannot render emoji characters
used in startup messages (e.g. print("✅ Frontend loaded ...")).
This causes UnicodeEncodeError and crashes the server before it starts.
Place sys.stdout.reconfigure(encoding="utf-8", errors="replace") at the
top of run_server(), unconditionally before any print() or structlog
call, so all emoji output is covered -- including the frontend status
messages and silent=True paths that the original placement missed.
Guarded by sys.platform == "win32" and hasattr check, so it is a no-op
on Linux/macOS and safe in non-standard stdout environments (Jupyter,
piped IO).
* fix: preserve run_server(None) as headless, fix CLI frontend kwarg
Remove the frontend_path=None fallback in run_server() that changed
None from "headless/API-only" to "mount bundled frontend", breaking
backwards compatibility for embedders.
The blank-page bug was actually caused by the CLI wrappers always
passing frontend_path=frontend (even when frontend=None), which
overrode run_server()'s default. Fix studio.py and ui.py to only
pass frontend_path when the user explicitly sets --frontend.
* fix: use timeout loop for shutdown event in ui command
Match studio_default()'s shutdown loop that uses a 1-second timeout
on Event.wait(). Without a timeout, the bare wait() blocks at the C
level on Linux, preventing Python from delivering SIGINT (Ctrl+C).
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* One liner setup for unsloth studio
* Fix install scripts: system deps, activation bugs, curl/wget support
- install.sh: detect platform (macOS/Linux/WSL) and check for missing
system dependencies (cmake, git, build-essential, libcurl4-openssl-dev).
Prompt user once for permission to install all missing packages via
brew (macOS) or sudo apt-get (Linux/WSL). Add wget fallback via
download() helper since curl is not always present on minimal Linux
installs. Fix nested curl|sh stdin stealing by downloading uv installer
to a tempfile first. Replace venv activation (no-op in a pipe subshell)
with explicit --python flag for uv pip install and direct venv binary
invocation. Add idempotency guard for venv creation. Redirect stdin
on unsloth studio setup to prevent pipe consumption. On macOS, check
for Xcode Command Line Tools and trigger install if missing.
- install.ps1: wrap script body in Install-UnslothStudio function so
that errors use return instead of exit (exit kills the terminal when
run via irm|iex). Remove activate.ps1 invocation entirely -- use
explicit --python path for uv pip install and & $UnslothExe for
studio setup. This avoids both the child-scope activation bug (& vs
dot-source) and the execution policy error on default Windows systems.
Add winget availability check with clear error message. Fix PATH
refresh to append registry paths instead of replacing the session PATH.
Add uv installer fallback via astral.sh PowerShell script if winget
install does not put uv on PATH. Broaden Python version check to
accept 3.11-3.13. Add idempotency guard for venv creation.
- README.md: add wget one-liner alternative for systems without curl.
* Fix Tailwind CSS v4 .gitignore bug on Windows (#4444)
- Add .gitignore hiding workaround to setup.ps1 (matching existing
setup.sh logic) so venv .gitignore files containing "*" don't prevent
Tailwind's oxide scanner from finding .tsx source files
- Add CSS size validation to setup.sh, setup.ps1, and build.sh to catch
truncated Tailwind builds early
- Remove stray force-rebuild overrides that made the "skip build if
current" cache check dead code in both setup scripts
- Add rm -rf dist to build.sh to force clean rebuilds for wheel packaging
* Change default port 8000 to 8888, fix installer bugs, improve UX
- Change default Studio port from 8000 to 8888 across all entry points
(run.py, studio.py, ui.py, colab.py, vite.config.ts, setup scripts)
- Update launch banner: "Launching with studio venv..." to
"Launching Unsloth Studio... Please wait..."
- Add "Open your web browser" banner and rename labels
(Local -> Local Access, External -> Worldwide Web Address)
- Fix venv idempotency: check for bin/python instead of just directory
existence, clean up partial venvs on retry
- Fix build.sh CSS validation: handle empty CSS case that silently
bypassed the check with "integer expression expected"
- Fix install.sh sudo handling: try apt-get without sudo first (works
when root), then escalate with per-package tracking and user prompt
- Fix install.ps1: check exit code from studio setup, fail on error
- Add pciutils to WSL GGUF build dependencies
- Apply same smart apt-get escalation pattern to studio/setup.sh
* Use detected Python version for venv, abort on non-apt Linux
- install.ps1: detect existing Python 3.11/3.12/3.13 and use that
version for venv creation instead of always forcing 3.13
- install.sh: exit with error on non-apt Linux distros when required
packages cannot be auto-installed, instead of silently continuing
* Make sudo permission prompt more prominent with warning banner
* Add Accept [Y/n] sudo prompt to studio/setup.sh for consistency
* Fix native command exit code handling and sudo decline flow
install.ps1: Add $LASTEXITCODE checks after winget (Python), uv venv,
and uv pip install calls. $ErrorActionPreference only catches PowerShell
cmdlet errors, not native executable failures. The Python check also
handles winget returning non-zero for "already installed".
setup.sh: Skip llama-server build when user declines sudo or sudo is
unavailable. Previously the script continued to section 8 which would
fail with confusing errors (e.g. "gcc: command not found") since
build-essential was never installed.
* Move rm -rf llama.cpp inside build branch to preserve existing install
When _SKIP_GGUF_BUILD is set (user declined sudo or sudo unavailable),
the previous rm -rf would destroy an already-working llama-server before
the skip check ran. Move it inside the else branch so existing builds
are preserved when the rebuild is skipped.
---------
Co-authored-by: Daniel Han <danielhanchen@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* studio: improve onboarding UX, tooltips, and training defaults
- Change splash text to "Train and run LLMs locally"
- Add "Chat Only" card with BubbleChatIcon to skip directly to chat
- Add Skip/Skip to Chat buttons in sidebar and footer
- Back button on step 1 returns to splash screen instead of being disabled
- Change "Watch video guide" to "Get started with our guide" with new URL
- Update intro text to mention all model types + chat
- Make all tooltips clickable (in addition to hover) via React context
- Strip surrounding quotes from pasted HF tokens
- Rename "Eval Split" to "Evaluation Split"
- Add SparklesIcon to "Auto Detect" format option
- Change step 4 heading to "Choose your training parameters"
- Default max_steps to 60
- Learning rate displayed in scientific notation with +/- stepper
- Context length options capped by model's max_position_embeddings (via AutoConfig)
- Fix "QLORA"/"LORA" to "QLoRA"/"LoRA" in summary step
- Backend: add max_position_embeddings to model config endpoint
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* compare for 2 diff models
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* resolving gemini comments
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: disable thinking for Qwen3.5 <9B and always for AI Assist
- Change Qwen3.5 thinking threshold from <=2B to <9B (0.8B, 2B, 4B
all disable thinking by default; 9B+ enables it)
- Always pass enable_thinking=False in AI Assist helper calls
(_run_with_helper and _generate_with_backend) regardless of chat
thinking settings
* studio: address PR review comments
- Extract _get_max_position_embeddings helper to DRY config extraction
- Fix "Skip to Chat" to navigate to /chat on step 1 (was /studio)
* fix: comment out debug print statements
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: skip Shiki highlighting for incomplete SVG code fences
While streaming SVG content, the syntax highlighter (Shiki) re-parses
the entire growing SVG on every token, blocking the main thread and
freezing the code area until the fence closes. Show a plain-text
preview for incomplete SVG fences instead, similar to how Mermaid
diagrams show a placeholder while streaming.
* studio: fix default top_k from 50/40 to 20 for chat inference
Per Qwen3.5 docs (unsloth.ai/docs/models/qwen3.5), top_k should be 20
for both thinking and non-thinking modes. The model-specific config in
inference_defaults.json already had top_k=20 for Qwen3.5, but the
generic fallback defaults were wrong:
- Frontend DEFAULT_INFERENCE_PARAMS.topK: 50 -> 20
- Backend generate_chat_completion top_k: 40 -> 20
- Backend generate_chat_completion_with_tools top_k: 40 -> 20
- Frontend title generation top_k: 40 -> 20
* studio: set universal inference defaults for unknown models
Default params for any model without specific config:
temperature=0.6, top_p=0.95, top_k=20, min_p=0.01,
presence_penalty=0.0, repetition_penalty=1.0
Models with entries in inference_defaults.json (Qwen3.5, Gemma-3,
Llama, etc.) override these with their recommended values.
Updated in: frontend DEFAULT_INFERENCE_PARAMS, backend Pydantic
request models, and backend generate_chat_completion defaults.
* studio: only trust_remote_code for unsloth/ models in AutoConfig
Only set trust_remote_code=True when the model name starts with
"unsloth/". All other models default to False for safety.
* studio: move Generating spinner above the composer
The "Generating" spinner was below the send message bar, causing
the bar to jump up and down. Move it above the composer in both
the regular thread view and the welcome/empty view.
* studio: adjust toast close button position away from edge
Move the X close button on toasts (like "Starting model...") from
top-1.5 to top-3 and add right-3, giving more breathing room from
the top-right corner.
* studio: make Think button smaller with tighter icon-text gap
Reduce gap from 1.5 to 0.5, padding from px-2.5/py-1 to px-2/py-0.5,
and icon from size-3.5 to size-3.
* studio: multiple onboarding and chat UX improvements
- Move Generating spinner above composer (fixes jumping send bar)
- Make Think button smaller with tighter icon-text gap
- Chat card now inside grid (same size as Audio/Embeddings cards)
- Rename "Chat Only" to "Chat"
- Chat card requires Continue to proceed (no auto-advance)
- Continue on Chat selection skips onboarding and goes to /chat
- Tooltip (i) click on Chat card doesn't trigger navigation
- Step 1 footer Back button goes back to splash (label is "Back")
- Splash "Skip Onboarding" renamed to "Skip to Chat", navigates to /chat
- Toast close button moved away from edge
* studio: align Skip to Chat button, add Skip to footer
- Sidebar "Skip to Chat" now uses primary (green) Button style with
arrow icon, full width, aligned like step items. Shows on all steps.
- Footer: added "Skip" outline button next to Continue that goes
directly to /studio with progress saved (markOnboardingDone)
* studio: change default max steps from 30 to 60 in toggle hook
The DEFAULT_MAX_STEPS in use-max-steps-epochs-toggle.ts was still 30,
used as fallback when toggling from epochs back to max steps.
* studio: extend context length options to 262K
CONTEXT_LENGTHS now includes 65536, 131072, 262144 in addition to
the existing 512-32768 range. The onboarding step filters these by
the model's max_position_embeddings (e.g. Nemotron-3-Nano-4B has
262144), showing powers of 2 up to the model's maximum.
* studio: auto-select LoRA vs QLoRA based on model size and GPU memory
After selecting a model in onboarding, detect the total model weight
file size from HF Hub (safetensors/bin files). Then estimate memory
needed: model_size_gb * 1.5 * context_scale, where context_scale is:
- <=8192 tokens: 1.0x
- >8192 tokens: 1.7x
- >=16384 tokens: 2.0x
- >=32768 tokens: 4.0x
If the estimate fits in free GPU VRAM, default to LoRA (16-bit).
Otherwise default to QLoRA (4-bit).
Backend changes:
- Add model_size_bytes to ModelDetails (models.py)
- Add _get_model_size_bytes() using HfApi.repo_info (routes/models.py)
- Add vram_free_gb to get_gpu_summary (hardware.py)
Frontend changes:
- Add autoSelectTrainingMethod() in training-config-store.ts
- Called after model defaults are loaded
- Add model_size_bytes to ModelConfigResponse type
- Add vramFreeGb to HardwareInfo hook
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: rename "Importing ML libraries..." to "Importing Unsloth..."
* studio: show model/dataset in training status, fix LoRA/QLoRA casing
- Training status now shows 'Training "model_name"' and 'Dataset = ...'
instead of generic "Starting training..."
- Fix Studio progress section to show QLoRA/LoRA instead of QLORA/LORA
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: rename 'Skip to Chat' to 'Skip Onboarding' on splash screen
* studio: add presence_penalty support for chat inference
Add presence_penalty as a parameter across the full stack:
- Backend: llama_cpp.py generate_chat_completion/with_tools, Pydantic
models (inference.py), routes/inference.py pass-through
- Frontend: InferenceParams type, DEFAULT_INFERENCE_PARAMS (0.0),
chat-adapter.ts payload, chat-settings-sheet.tsx slider (0-2),
model defaults loading from inference_defaults.json
- Set Qwen3.5 default presence_penalty to 1.5 per official docs
- Default for unknown models is 0.0 (off)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: fix Chat card deselecting Text and aligning with other cards
* studio: fix presence_penalty not loading from inference defaults
The inference_config.py load_inference_config() was not including
presence_penalty in the returned config dict, so the Qwen3.5
default of 1.5 from inference_defaults.json never reached the
frontend. Added it to the config builder.
* studio: add delete button for cached models in model selector
Add trash icon on each downloaded model row (GGUF and safetensors) with
confirmation dialog. Backend DELETE /api/models/delete-cached endpoint
uses huggingface_hub scan_cache_dir + delete_revisions to cleanly remove
cached repos, refusing if the model is currently loaded.
* studio: restore inference defaults, reasoning, and tools on page refresh
On page refresh with a model already loaded, the frontend was not
re-applying model-specific inference defaults (presence_penalty,
temperature, etc.) or restoring reasoning/tools support flags.
Backend: Add inference config, supports_reasoning, supports_tools,
and context_length to InferenceStatusResponse.
Frontend: In the refresh callback, when an active model is detected,
apply mergeRecommendedInference and restore reasoning/tools flags
with proper Qwen3.5 size-based defaults.
* studio: fix delete dialog closing before async completes
Prevent AlertDialogAction's default close behavior with
e.preventDefault() so the dialog stays open during deletion.
Also block onOpenChange dismiss while deleting is in progress.
* fix: add Dict and Any imports to inference models
* studio: fix Qwen3.5 reasoning threshold in frontend load path
The frontend loadModel handler had the old threshold (<=2) for
disabling reasoning on small Qwen3.5 models. Changed to <9 to
match the backend. This was causing 4B to not properly disable
thinking by default when auto-loaded.
* studio: move GGUF delete to per-variant level
For GGUF repos, the trash icon now appears on each downloaded variant
row inside the quantization expander instead of on the repo-level row.
Backend accepts optional variant param to delete specific GGUF files
(blob + symlink) rather than the entire repo cache.
* studio: restore ggufContextLength on page refresh
The Max Tokens slider was capped at 32768 on page refresh because
ggufContextLength was not restored from the status response.
Now set it from statusRes.context_length on reconnect.
* fix: remove <think> from Qwen3.5 response template marker
The train-on-responses-only feature uses template markers to find
where the assistant response starts. The Qwen3.5 response marker
included '<think>\n' which is only present when thinking mode is
enabled. With thinking disabled (default for <9B), the marker
never matched, causing 100% of samples to be dropped.
Changed response marker from '<|im_start|>assistant\n<think>\n'
to '<|im_start|>assistant\n' which works regardless of thinking mode.
* studio: fix sloth ASCII art alignment in training overlay
* fix: correct sloth ASCII art alignment to match Unsloth banner
* studio: add Python and terminal tool calling to chat
Register python and terminal tools alongside web search. Python
executor validates imports (stdlib only) via unsloth_zoo
rl_environments, runs code in a subprocess sandbox with 5-min
timeout and cancel support. Terminal executor blocks dangerous
commands (rm, sudo, etc.) and runs in a temp directory.
Update llama_cpp tool loop to show tool-specific status messages
and pass cancel_event through to executors. Rename composer
toggle from "Search" to "Tools" and show TerminalIcon for
execution status pills.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: fix Nemotron/transformers 5.x support, onboarding navigation, port binding
Backend:
- Dynamic transformers 5.x detection via tokenizer_config.json fetch
(checks for TokenizersBackend class, cached per-model)
- Bump transformers 5.x version from 5.2.0 to 5.3.0 across all workers,
setup scripts (setup.sh, setup.ps1)
- Auto-enable trust_remote_code for unsloth/* models needing transformers 5.x
(workaround for NemotronH config parsing bug in transformers)
- Auto-install mamba-ssm/causal-conv1d for SSM models (NemotronH, Falcon-H1)
with --no-build-isolation --no-deps to avoid torch version conflicts
- Add SO_REUSEADDR to port check in run.py (fixes Colab proxy stale connection
falsely reporting port as in-use)
Frontend:
- Fix "Skip to Chat" navigation: use window.location.href instead of React
Router navigate() to bypass useEffect redirect race
- Fix "Skip Onboarding" on splash: navigates to /studio (not /chat)
- Fix onboarding guard: only check isOnboardingDone() on initial mount
- Fix Chat card on step 1: add sr-only spacer for consistent alignment
- Fix Chat+Text both selected: clear RadioGroup value when Chat is selected
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: split tools toggle into Search and Code buttons
Replace the single "Tools" toggle with two independent toggles:
- "Search" (globe icon) enables web search only
- "Code" (terminal icon) enables Python and terminal execution
Add enabled_tools list field to the inference payload so the
backend only registers the tools the user has toggled on. Both
toggles appear in the main composer and the compare composer.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: fix tool calling import validation and error logging
Replace unsloth_zoo-dependent import checker with a standalone
ast-based validator using sys.stdlib_module_names. This properly
blocks non-stdlib imports (numpy, requests, etc.) and returns a
clear error message to the model so it can rewrite using only
stdlib.
Add full traceback to tool streaming error logs for debugging.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: parse gpt-oss harmony channels for clean safetensors chat output
gpt-oss models emit multi-channel output via harmony protocol tokens
(<|channel|>analysis<|message|>... and <|channel|>final<|message|>...).
TextIteratorStreamer with skip_special_tokens=True strips the special
tokens but leaves channel names concatenated with content, producing
garbled output like "analysisWe need to...assistantfinalHello!".
Add HarmonyTextStreamer that decodes with skip_special_tokens=False,
parses harmony markup via regex, and emits <think>analysis</think>
for the analysis channel and plain text for the final channel --
reusing the existing frontend reasoning UI.
Also expose supports_reasoning=True for non-GGUF gpt-oss models in
the /status endpoint so the frontend enables the Think toggle.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: use unsloth_zoo for Python sandbox validation
Set UNSLOTH_IS_PRESENT=1 and import check_python_modules and
check_signal_escape_patterns directly from unsloth_zoo instead
of a standalone fallback. This gives us the full Unsloth
validation including stdlib-only import checks and signal/timeout
escape pattern detection.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: allow all imports in Python tool sandbox
Remove stdlib-only import restriction. Keep signal escape
pattern detection via unsloth_zoo for safety.
* studio: fix ReadTimeout on tool streaming final pass
The 0.5s read timeout used for cancel-checking during streaming
also fires when waiting for the first response from llama-server
(e.g. reasoning model thinking for 15+ seconds). Add
_stream_with_retry() context manager that retries on ReadTimeout
while checking cancel_event, so the model has unlimited time to
think before producing the first token. Applied to both the
regular streaming path and the tool-calling final pass.
* fix: rewrite HarmonyTextStreamer with stateful incremental parsing
The delta-on-transformed approach had two critical bugs:
1. Before the full <|channel|>X<|message|> pattern was complete, the
strip-tokens fallback emitted "analysis" as plain text. Then when
the regex matched, _transform returned a completely different format
(<think>...</think>) and the delta was computed against the wrong
base string, producing fragments like "think>", "nk>", ">".
2. Even with full matches, the closing </think> tag shifted position
as content grew, so text[prev_len:] produced garbled deltas.
Replace with stateful incremental parsing that:
- Buffers until a complete channel+message pair is seen
- Emits <think> once when analysis channel first appears
- Streams analysis content deltas (computed on channel content directly)
- Emits </think> once when final channel first appears
- Streams final content deltas
- Closes open think tags in end()
Also skip the generic all_special_tokens stripping in
_clean_generated_text for gpt-oss since HarmonyTextStreamer already
produces clean output and the generic stripping was mangling <think>
tags.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: strip all <|...|> tokens in gpt-oss cleanup, not just harmony subset
The gpt-oss tokenizer has added tokens like <|return|> (id=200002) that
are not part of the harmony channel protocol but can leak into output.
The previous regex only stripped channel|message|start|end tokens.
Broaden the _clean_generated_text regex for gpt-oss to <\|[a-z_]+\|>
which catches all pipe-delimited tokens (return, constrain, reserved,
etc.) without matching <think>/<\/think> tags.
Verified: gpt-oss all_special_tokens are only <|return|>,
<|reserved_200017|>, <|startoftext|> -- none overlap with <think>.
The harmony tokens (channel, message, start, end) are added_tokens
but not in all_special_tokens.
* fix: hide config-only model repos from cached models list
Repos that only have metadata/config files cached (no .safetensors or
.bin weight files) were showing up in the Downloaded list with tiny
sizes like "1.8 KB" or "24 KB". These are just leftover config
snapshots from architecture checks, not usable models.
Filter the cached-models endpoint to only include repos that contain
actual model weight files (.safetensors or .bin).
* studio: fix toast description text contrast in dark mode
Add explicit !text-muted-foreground to toast description classNames
so secondary text (e.g. "Releases VRAM and resets inference state.")
is readable in dark mode.
* studio: fix Chat card icon alignment with size-4 spacer
Replace sr-only span (takes no space) with a size-4 shrink-0 div
matching the RadioGroupItem dimensions in other cards, so the Chat
icon aligns vertically with Text/Audio/Vision/Embeddings icons.
---------
Co-authored-by: workspace <user@workspace.local>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Manan17 <shahmanan170602@gmail.com>
Co-authored-by: Roland Tannous <rolandtannous@gravityq.ai>
* fix: Ctrl+C not breaking out of backend on Linux
threading.Event.wait() without a timeout blocks at the C level on
Linux, preventing Python from delivering SIGINT. Use a 1-second
timeout loop so the interpreter can process pending signals.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix: graceful shutdown on Windows (signal handlers for Ctrl+C)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
If the requested port (default 8000) is already in use, auto-
increment and try the next port, up to 20 attempts. Prints a
message like "Port 8000 is in use, using port 8001 instead".
Previously, if port 8000 was busy, uvicorn would fail with
"[Errno 98] address already in use" and the studio would not
start. Now it gracefully finds the next free port.
Uses socket.bind() to check availability before starting uvicorn.
Cross-platform (Linux, macOS, Windows).
- Workers now compute backend_path and venv_t5 locally via Path(__file__)
- Moved .venv_t5 to ~/.unsloth/studio/.venv_t5
- Added ensure_studio_directories() call on server startup
- Expanded CLI studio command into sub-app with setup subcommand
