* feat(studio): add Continued Pretraining (CPT) support
Implements CPT as a first-class training method in Unsloth Studio,
resolving feature request #4565.
Changes:
- frontend/src/types/training.ts: add 'cpt' to TrainingMethod union
- frontend/src/lib/vram.ts: add 'cpt' to VramTrainingMethod (fp16 footprint)
- frontend/src/features/export/constants.ts: add CPT to METHOD_LABELS
- frontend/src/features/training/api/mappers.ts: map 'cpt' -> 'Continued Pretraining',
force packing=true and train_on_completions=false for CPT payloads
- frontend/src/features/studio/sections/model-section.tsx: add 'Continued Pretraining'
option (purple dot) to Method selector; update tooltip
- frontend/src/features/onboarding/.../model-selection-step.tsx: add CPT to
onboarding wizard method dropdown
- backend/models/training.py: update training_type field description
- backend/core/training/worker.py: detect is_cpt flag, force packing=True,
train_on_completions=False, pass is_cpt to _train_worker
- backend/core/training/trainer.py: _train_worker reads is_cpt kwarg, forces
packing on, skips train_on_responses_only for raw-text pretraining
CPT behaviour:
- Full model weights (no LoRA adapters), same as Full Finetuning
- Sequence packing always enabled for GPU efficiency
- Trains on every token (no chat-format masking)
- VRAM estimated at fp16 (2.0 bytes/param)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update mappers.ts
* Add CPT raw dataset support and UI fixes
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add missing training methods module
* Handle invalid raw-text rows and expose raw in onboarding
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Roland Tannous <115670425+rolandtannous@users.noreply.github.com>
Co-authored-by: Etherll <61019402+Etherll@users.noreply.github.com>
Co-authored-by: Etherll <mrmrmidessam@gmail.com>
* Add Apple Silicon MLX routing
Rewrite __init__.py: detect MLX on macOS arm64 before any torch imports
Extract original GPU init to _gpu_init.py (unchanged)
MLX path imports FastMLXModel from unsloth_zoo, skips all GPU code
GPU path unchanged: from ._gpu_init import *
* Add Apple Silicon MLX routing
- Rewrite __init__.py: detect MLX on macOS arm64 before any torch imports
- Extract original GPU init to _gpu_init.py (unchanged)
- MLX path imports FastMLXModel from unsloth_zoo, skips all GPU code
- GPU path unchanged: from ._gpu_init import *
* mlx with studio
* mlx with studio
* updating temporary install.sh
* updating temporary install.sh
* adding t_v5 path
* adding t_v5 path
* fixing vision training
* fixing vision training
* adding chat
* adding chat
* minor
* minor
* Adding export and fixing training issues, inference with lora adaptors
* Adding export and fixing training issues, inference with lora adaptors
* fix: MLX worker pass load_in_4bit, override is_vlm based on dataset, streaming for VLM
* fix: MLX worker pass load_in_4bit, override is_vlm based on dataset, streaming for VLM
* Merge mlx-apple-silicon into main
* update install.sh to point to main branch
* update install.sh to point to main branch
* fix: export returns 3 values (success, message, output_path) matching upstream worker
* fix: export returns 3 values (success, message, output_path) matching upstream worker
* fix(mlx): show training-process peak memory in Studio UI, not system-wide
Studio UI was showing ~95 GB during MLX training because get_gpu_utilization
read "In use system memory" from IORegistry's AGXAccelerator — system-wide
GPU memory across all processes (training + backend + browser + Display).
Now the trainer's mx.get_peak_memory value is forwarded through the
progress event and surfaced via /api/train/hardware while training is
active. Falls back to the system-wide reading when training is not running.
* fix(mlx): show training-process peak memory in Studio UI, not system-wide
Studio UI was showing ~95 GB during MLX training because get_gpu_utilization
read "In use system memory" from IORegistry's AGXAccelerator — system-wide
GPU memory across all processes (training + backend + browser + Display).
Now the trainer's mx.get_peak_memory() value is forwarded through the
progress event and surfaced via /api/train/hardware while training is
active. Falls back to the system-wide reading when training is not running.
* fix(mlx): make is_bfloat16_supported detect M1/M2 (no native bf16)
M1 and M2 chips emulate bf16 in software on the GPU, causing 40-70%
slower prefill compared to native fp16. M3+ have native bf16 (macOS
Sonoma+ MPSGraph). Replaces the always-True stub with chip-aware
detection via mx.device_info.
* fix(mlx): make is_bfloat16_supported() detect M1/M2 (no native bf16)
M1 and M2 chips emulate bf16 in software on the GPU, causing 40-70%
slower prefill compared to native fp16. M3+ have native bf16 (macOS
Sonoma+ MPSGraph). Replaces the always-True stub with chip-aware
detection via mx.device_info().
* feat(mlx): wire training_type="Full Finetuning" through MLX worker
Compute use_lora from the UI's training_type before loading the model,
pass full_finetuning=not use_lora to FastMLXModel.from_pretrained, and
let the existing 'if use_lora' branch skip get_peft_model. Matches the
GPU worker's flow.
* feat(mlx): wire training_type="Full Finetuning" through MLX worker
Compute use_lora from the UI's training_type before loading the model,
pass full_finetuning=not use_lora to FastMLXModel.from_pretrained, and
let the existing 'if use_lora' branch skip get_peft_model. Matches the
GPU worker's flow.
* fix(mlx): pass save_method='merged_16bit' from Studio's export page
Previously the MLX path called save_pretrained_merged with no
save_method, which fell through to a no-op that didn't actually fuse
LoRA into the base. Now Studio's "Merged Model" export properly
fuses LoRA + dequantizes any 4-bit base to bf16, matching the GPU
behavior for the same UI option.
* fix(mlx): pass save_method='merged_16bit' from Studio's export page
Previously the MLX path called save_pretrained_merged() with no
save_method, which fell through to a no-op that didn't actually fuse
LoRA into the base. Now Studio's "Merged Model" export properly
fuses LoRA + dequantizes any 4-bit base to bf16, matching the GPU
behavior for the same UI option.
* fix(studio): pass private to MLX push, return 3-tuples consistently
MLX push_to_hub branch now forwards private=private (matches GPU)
Existing 2-tuple early-returns ('repo_id+token required', 'PEFT model
needed') were tripping the route's 3-tuple unpack. Added a None
output_path so the unpack always succeeds.
* fix(studio): pass private to MLX push, return 3-tuples consistently
- MLX push_to_hub branch now forwards private=private (matches GPU)
- Existing 2-tuple early-returns ('repo_id+token required', 'PEFT model
needed') were tripping the route's 3-tuple unpack. Added a None
output_path so the unpack always succeeds.
* studio wirings
* studio wirings
* Merge pull request #5 from Manan17/feat/quant_config
studio wirings
* fix(mlx): wire train_on_completions for VLM via per-template lookup
Mirror the GPU worker: stop excluding VLMs and stop hardcoding
template detection. Look up the model in MODEL_TO_TEMPLATE_MAPPER and
fetch the per-template instruction/response markers from
TEMPLATE_TO_RESPONSES_MAPPER. The frontend already force-disables
train_on_completions for vision+image and audio cases, so backend
just trusts the flag.
* fix(mlx): wire train_on_completions for VLM via per-template lookup
Mirror the GPU worker: stop excluding VLMs and stop hardcoding
template detection. Look up the model in MODEL_TO_TEMPLATE_MAPPER and
fetch the per-template instruction/response markers from
TEMPLATE_TO_RESPONSES_MAPPER. The frontend already force-disables
train_on_completions for vision+image and audio cases, so backend
just trusts the flag.
* wire in lora rslora, init lora weights, random_state
* wire in lora rslora, init lora weights, random_state
* loftq studio error message fix
* loftq studio error message fix
* handle unknown optim and lr scheduler
* handle unknown optim and lr scheduler
* Merge pull request #6 from Manan17/update/peftkwargs
Update/peftkwargs
* feat(mlx): pass finetune_language/attention/mlp/vision flags to FastMLXModel
Studio's four UI checkboxes now actually flow through to MLX get_peft_model
(which was just updated in unsloth-zoo to honor them). Also drops the
incorrect train_projector wiring that tied projector LoRA to the
attn/mlp flags — those are language-side toggles, not projector toggles.
Co-Authored-By: Manan17 <shahmanan170602@gmail.com>
* feat(mlx): pass finetune_language/attention/mlp/vision flags to FastMLXModel
Studio's four UI checkboxes now actually flow through to MLX get_peft_model
(which was just updated in unsloth-zoo to honor them). Also drops the
incorrect train_projector wiring that tied projector LoRA to the
attn/mlp flags — those are language-side toggles, not projector toggles.
Co-Authored-By: Manan17 <shahmanan170602@gmail.com>
* feat(mlx,ux): auto-imply finetune_language_layers when user picks attn/mlp
UI guardrail. The four checkboxes (vision/language/attention/MLP) carry
"scope × module-type" semantics that aren't obvious — picking just
"Attention modules" + "MLP modules" without "Language layers" naturally
reads as "fine-tune attn/mlp" but our backend reads it as "fine-tune
attn/mlp modules in *no* tower" → empty target_modules → zero
trainable params → crash inside value_and_grad.
If user selected attn or mlp module types but no layer scope, default
to language scope. Power users can still explicitly choose
language=False, vision=True if they want vision-only fine-tuning of
attn/mlp.
Co-Authored-By: Manan17 <shahmanan170602@gmail.com>
* feat(mlx,ux): auto-imply finetune_language_layers when user picks attn/mlp
UI guardrail. The four checkboxes (vision/language/attention/MLP) carry
"scope × module-type" semantics that aren't obvious — picking just
"Attention modules" + "MLP modules" without "Language layers" naturally
reads as "fine-tune attn/mlp" but our backend reads it as "fine-tune
attn/mlp modules in *no* tower" → empty target_modules → zero
trainable params → crash inside value_and_grad.
If user selected attn or mlp module types but no layer scope, default
to language scope. Power users can still explicitly choose
language=False, vision=True if they want vision-only fine-tuning of
attn/mlp.
Co-Authored-By: Manan17 <shahmanan170602@gmail.com>
* fix(mlx): wire top_k, repetition_penalty, and VLM top_p through to mlx-lm/mlx-vlm
Inference UI sliders for top_k and repetition_penalty had no effect on
MLX, and VLM top_p was also silently dropped. Plus a latent pre-existing
bug: mlx_vlm.generate_step expects temperature= (long form), but we
were passing temp= which silently fell into **kwargs — every VLM chat
was effectively greedy regardless of the temperature slider.
Text path (_generate_text):
make_sampler now receives top_k in addition to temp/top_p
make_logits_processors built and forwarded when repetition_penalty is
non-trivial (skip when 0.0/1.0 to avoid pointless overhead)
VLM path (_generate_vlm):
Pass top_p, top_k, repetition_penalty as kwargs (mlx_vlm.stream_generate
forwards them to generate_step's sampler/logits_processor builders)
Rename temp= → temperature= so it's actually consumed
Verified end-to-end with a smoke test on Qwen2.5-0.5B-Instruct (text) and
Qwen2.5-VL-3B-Instruct (VLM): each of {greedy, top_p=0.5, top_k=10,
rep_pen=1.5} now produces a distinct output, proving the parameters
reach the sampler.
Co-Authored-By: Manan17 <shahmanan170602@gmail.com>
* fix(mlx): wire top_k, repetition_penalty, and VLM top_p through to mlx-lm/mlx-vlm
Inference UI sliders for top_k and repetition_penalty had no effect on
MLX, and VLM top_p was also silently dropped. Plus a latent pre-existing
bug: mlx_vlm.generate_step expects temperature= (long form), but we
were passing temp= which silently fell into **kwargs — every VLM chat
was effectively greedy regardless of the temperature slider.
Text path (_generate_text):
- make_sampler now receives top_k in addition to temp/top_p
- make_logits_processors built and forwarded when repetition_penalty is
non-trivial (skip when 0.0/1.0 to avoid pointless overhead)
VLM path (_generate_vlm):
- Pass top_p, top_k, repetition_penalty as kwargs (mlx_vlm.stream_generate
forwards them to generate_step's sampler/logits_processor builders)
- Rename temp= → temperature= so it's actually consumed
Verified end-to-end with a smoke test on Qwen2.5-0.5B-Instruct (text) and
Qwen2.5-VL-3B-Instruct (VLM): each of {greedy, top_p=0.5, top_k=10,
rep_pen=1.5} now produces a distinct output, proving the parameters
reach the sampler.
Co-Authored-By: Manan17 <shahmanan170602@gmail.com>
* feat(mlx): map format_type to MLX save_method, reuse local save dir for hub push
export_merged_model: format_type="4-bit (FP4)" → save_method="merged_4bit"
(was hardcoded merged_16bit, ignoring the UI choice).
Both export_merged_model and export_base_model now pass save_directory=
to push_to_hub_merged so it reuses the just-written local folder
instead of re-saving under a relative "username/model" directory.
Co-Authored-By: Manan17 <shahmanan170602@gmail.com>
* feat(mlx): map format_type to MLX save_method, reuse local save dir for hub push
- export_merged_model: format_type="4-bit (FP4)" → save_method="merged_4bit"
(was hardcoded merged_16bit, ignoring the UI choice).
- Both export_merged_model and export_base_model now pass save_directory=
to push_to_hub_merged so it reuses the just-written local folder
instead of re-saving under a relative "username/model" directory.
Co-Authored-By: Manan17 <shahmanan170602@gmail.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* restore install
* restore install
* fix(mlx): restore FastVisionModel as a distinct class
unsloth/__init__.py was assigning `FastVisionModel = FastLanguageModel`
right after defining `class FastVisionModel(FastLanguageModel)` with a
`for_training` static method. The alias erased the class binding, so
the documented `FastVisionModel.for_training(model)` call from upstream
Unsloth's VLM notebooks raised `AttributeError` on MLX.
Remove the offending alias. `FastVisionModel` is now a real subclass of
`FastLanguageModel` again — inherits `from_pretrained` /
`get_peft_model` / `for_inference`, exposes `for_training` as a no-op
pass-through (no-op because MLX doesn't have a train/eval mode flag;
the call exists purely for GPU/MLX notebook parity).
Verified end-to-end: Qwen3-VL-2B + LaTeX_OCR LoRA + vision LoRA via
FastVisionModel.from_pretrained → get_peft_model → for_training →
MLXTrainer.train runs 10 steps cleanly (loss 1.10 → 0.12, no NaNs,
peak 5.89 GB).
Studio's path (FastLanguageModel.from_pretrained for any repo,
auto-detect VLM in the loader) is unaffected. Tier-1 review finding #8.
* fix(mlx): restore FastVisionModel as a distinct class
unsloth/__init__.py was assigning `FastVisionModel = FastLanguageModel`
right after defining `class FastVisionModel(FastLanguageModel)` with a
`for_training` static method. The alias erased the class binding, so
the documented `FastVisionModel.for_training(model)` call from upstream
Unsloth's VLM notebooks raised `AttributeError` on MLX.
Remove the offending alias. `FastVisionModel` is now a real subclass of
`FastLanguageModel` again — inherits `from_pretrained` /
`get_peft_model` / `for_inference`, exposes `for_training` as a no-op
pass-through (no-op because MLX doesn't have a train/eval mode flag;
the call exists purely for GPU/MLX notebook parity).
Verified end-to-end: Qwen3-VL-2B + LaTeX_OCR LoRA + vision LoRA via
FastVisionModel.from_pretrained → get_peft_model → for_training →
MLXTrainer.train() runs 10 steps cleanly (loss 1.10 → 0.12, no NaNs,
peak 5.89 GB).
Studio's path (FastLanguageModel.from_pretrained for any repo,
auto-detect VLM in the loader) is unaffected. Tier-1 review finding #8.
* Studio: harden MLX training and export, restore GPU init guards
Studio export
Restore Tuple[bool, str, Optional[str]] contract on export_merged_model,
export_base_model, export_gguf, and export_lora_adapter, populating
output_path on successful local saves so routes/worker/CLI/frontend
details.output_path is non-empty again.
Lift the GPU save_method assignment out of the local-save branch so
Hub-only merged exports (save_directory='', push_to_hub=True) no longer
hit UnboundLocalError on the push branch.
For MLX merged and base hub-only export, stage to a tempfile.TemporaryDirectory
before push_to_hub_merged instead of passing save_directory=''.
Source _IS_MLX from unsloth instead of recomputing the platform check
(single source of truth, also enforces mlx-package availability).
Studio MLX training/inference
Pass token=hf_token into FastMLXModel.from_pretrained for gated/private
models, matching the inference path.
Strip hf_token and wandb_token from wandb.init(config=...) so secrets
do not leak into the W&B run config.
Replace load_from_disk(local_datasets[0]) with the existing
UnslothTrainer._resolve_local_files / _loader_for_files helpers so
uploaded JSON/JSONL/CSV/Parquet files train through the normal datasets
loader (load_from_disk still used for HF save_to_disk directories).
Make the dataset slice helper inclusive at the end and treat 0 as a real
index instead of "unset", matching the GPU and embedding paths.
Add a status_message -> message alias inside _send so the existing parent
pump (training.py) renders MLX status updates instead of blanks.
Forward min_p through generate_chat_response into _generate_text /
_generate_vlm and into make_sampler / vlm_kwargs so the sampling control
is no longer a no-op on MLX.
Wrap unsloth_zoo.mlx_loader / mlx_trainer imports with a clearer
ImportError pointing users at install.sh for Apple Silicon.
Exit the MLX stop-polling thread on EOFError/OSError instead of
busy-looping when the queue/pipe is permanently closed (one-line
why-safe rationale inline).
Studio frontend
ParamsSection subscribes to platform deviceType via the Zustand hook so
the gradient checkpointing dropdown re-renders after the async device
fetch completes.
Studio hardware
get_gpu_utilization MLX branch now reads _read_apple_gpu_stats once and
derives VRAM totals from psutil, removing the second ioreg subprocess
per utilization poll.
Unsloth core
Restore the os.geteuid == 0 guard around the CUDA ldconfig recovery
that was lost when GPU initialization moved into _gpu_init.py, plus the
non-root manual-fix warning branch. Non-root CUDA users no longer shell
out to ldconfig at import time.
Load dataprep/raw_text via importlib so the MLX import path no longer
pulls torch in through dataprep/__init__.py -> synthetic.py.
FastVisionModel.from_pretrained overrides the inherited delegator only
to inject text_only=False; this is an extension, not a duplication, and
is needed so VLM checkpoint loads keep the vision tower.
Wrap the MLX-branch unsloth_zoo import with a clearer ImportError.
* Studio: regression tests for MLX training/export and GPU init ldconfig guard
tests/python/test_gpu_init_ldconfig_guard.py asserts the geteuid root
check still wraps the ldconfig recovery and the non-root branch warns
bnb users; AST + source-text inspection so the test runs without torch.
tests/studio/test_export_output_path_contract.py covers the
Tuple[bool, str, Optional[str]] return contract on every export method,
the output_path assignment after successful local save, the Hub-only
GPU save_method binding fix, the MLX hub-only TemporaryDirectory
staging, and the single-source `_IS_MLX` import from unsloth.
tests/studio/test_mlx_training_worker_behaviors.py covers token
forwarding to FastMLXModel.from_pretrained, wandb config secret
stripping, file-aware local dataset loading, status_message ->
message aliasing, inclusive slice semantics, EOFError/OSError stop
thread exit, and the friendly mlx_loader / mlx_trainer ImportError.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(mlx): cap inference memory + release wired on unload + tame worker pre-pin
Three memory-hardening fixes for Studio's MLX path:
1. Inference applies the same Metal caps as the trainer.
load_model previously only called set_wired_limit(100% of recommended)
with no upper memory_limit, leaving large VLM checkpoints unbounded
during the loader allocation. Add _configure_memory_limits() that sets
memory_limit to 85% of recommended and wired_limit to min(recommended,
memory_limit) — matching MLXTrainer's defaults so behavior is the same
whether the user trains or just runs inference.
2. unload_model releases pinned memory back to the OS — but only when
the cache is empty. Without this, pinned wired bytes stayed allocated
to MLX after the model was gone, starving other apps. The release is
guarded on `not self.models` so unloading one of several cached
models doesn't un-pin weights still in use.
3. Worker pre-cap is conservative instead of aggressive.
The previous pre-pin set_wired_limit(100% of recommended) competed
with MLXTrainer's later more conservative cap. Replace with the same
85%-memory / min(rec, memory) pair that the trainer applies later
(idempotent re-apply). Bounds the model load + LoRA setup window
without over-pinning.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* tests/studio: regression tests for the _IS_MLX dispatch gate
Two gates drive every MLX-vs-CUDA dispatch decision in Studio:
1. unsloth._IS_MLX in unsloth/__init__.py — evaluated once at import
time, read by Studio worker code to choose the GPU vs MLX trainer
and inference paths. Defined as
Darwin AND arm64 AND find_spec("mlx") is not None.
2. utils.hardware.detect_hardware() — runtime probe with priority
CUDA > XPU > MLX > CPU. The MLX branch is reached only when both
CUDA and XPU are unavailable and the host is Apple Silicon and
mlx is importable.
Neither gate had a direct test. Adds tests/studio/test_is_mlx_dispatch_gate.py
with six tests:
test_is_mlx_gate_uses_three_required_predicates
AST-walks unsloth/__init__.py and asserts the _IS_MLX assignment
is a BoolOp(And) of platform.system()=="Darwin",
platform.machine()=="arm64", and find_spec("mlx") is not None.
Catches accidental rewrites that drop a predicate.
test_is_mlx_gate_true_on_apple_silicon_with_mlx_present
Spoofs platform to Darwin/arm64, injects a fake mlx module so
find_spec returns a real ModuleSpec, re-evaluates the gate
expression. Verifies it flips True under the exact conditions
Studio expects.
test_is_mlx_gate_false_when_mlx_missing
Spoofs Apple Silicon but with mlx absent. Verifies the gate stays
False (so a Mac without mlx installed does not pretend to have
MLX support).
test_is_mlx_gate_false_on_non_apple_silicon
Canary on the actual Linux+CUDA / AMD / Intel test host: the gate
must remain False regardless of whether mlx happens to be
importable. Protects existing GPU users from accidental MLX
hijack when MLX support evolves.
test_detect_hardware_picks_mlx_when_only_apple_silicon_available
Forces torch.cuda and torch.xpu off, spoofs Apple Silicon, injects
fake mlx and mlx.core. detect_hardware() must return DeviceType.MLX.
test_detect_hardware_picks_cuda_on_real_host
Canary: on a real CUDA host detect_hardware() must return
DeviceType.CUDA. Protects against the MLX branch shadowing CUDA
dispatch on NVIDIA / AMD ROCm hosts.
Uses the same monkeypatch.setitem(sys.modules, ...) fake-mlx pattern as
the existing test_mlx_inference_backend.py — no new test infrastructure,
no real mlx install required.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add AGPL-3.0 SPDX header to Studio MLX regression tests
Four Studio MLX test files shipped without an SPDX-License-Identifier:
studio/backend/tests/test_mlx_training_worker_config.py
tests/studio/test_mlx_training_worker_behaviors.py
tests/studio/test_export_output_path_contract.py
tests/studio/test_is_mlx_dispatch_gate.py
They sit in or alongside studio/backend/, which is governed by
studio/LICENSE.AGPL-3.0, and exercise AGPL Studio code. Add the same
"# SPDX-License-Identifier: AGPL-3.0-only" header that's already on
test_mlx_inference_backend.py so the license declaration matches
the code under test rather than defaulting to the repo-root
Apache-2.0.
* Wrap MLX submodule imports with friendly install hint
The _IS_MLX block at the top of unsloth/__init__.py already catches the
missing-package case with a friendly install hint, but the follow-up
"from unsloth_zoo.mlx_trainer import ..." and "from unsloth_zoo.mlx_loader import ..."
lines run unguarded. An Apple Silicon user who has unsloth-zoo installed
but on an older version (e.g. the current PyPI release, before the MLX
modules ship) sees a raw ImportError on the submodule rather than the
hint that points at install.sh.
Wrap the two submodule imports in the same try/except shape so the
friendly install message fires whether the package is missing entirely
or just predates the MLX submodules. No-op once both packages release
together; smooths the transitional window where unsloth/main has merged
but unsloth-zoo on PyPI has not.
---------
Co-authored-by: DoubleMathew <mmathew23@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* install: support STUDIO_HOME / UNSLOTH_STUDIO_HOME for custom install paths
Currently install.sh and install.ps1 hardcode all install paths off
$HOME / $env:USERPROFILE with no env-var fallback. This blocks
workspace-isolated installs (CI sandboxes, per-PR test environments,
multi-tenant boxes) unless the entire HOME / USERPROFILE is faked,
which also relocates ~/.gitconfig, ~/.ssh, and other unrelated state.
Add an opt-in env-var override that does only what is needed.
Resolution priority (highest first):
1. HOME / USERPROFILE explicitly redirected vs the password-database
default. Detected via getent (Linux), dscl (macOS), or
[Environment]::GetFolderPath (Windows). Best-effort: when the
detection mechanism is unavailable the check is skipped and we
fall through to step 2.
2. UNSLOTH_STUDIO_HOME, if set.
3. STUDIO_HOME, if set (alias for convenience; the variable name
already matches the internal var install.sh sets).
4. Default: legacy $HOME/.unsloth/studio (or
$USERPROFILE\.unsloth\studio on Windows). Identical to today's
behavior when no env var is set.
When an env var override fires:
* DATA_DIR is nested inside ($STUDIO_HOME/share, or $StudioHome\share
on Windows) so the runtime launcher and shortcuts find studio.conf
in the same place install-time wrote it.
* The unsloth CLI shim lands at $STUDIO_HOME/bin/unsloth (Unix) or
$StudioHome\bin\unsloth.exe (Windows). On Windows the shim already
lives under $StudioHome; the change only redirects DATA_DIR and
skips the persistent registry PATH update.
* Persistent shell PATH modifications are skipped (no .bashrc /
.zshrc / .profile append on Unix; no Add-ToUserPath on Windows).
Caller is expected to invoke via absolute path or add the bin dir
to PATH explicitly. Avoids polluting the user's profile with a
workspace-scoped path that may be deleted.
The Unix launcher script is the only piece that must read DATA_DIR
at runtime (it sources studio.conf from there). The hardcoded
DATA_DIR inside the LAUNCHER_EOF heredoc is replaced with an
@@DATA_DIR@@ placeholder substituted via sed at install time, using
the same approach the script already uses for other install-time
substitutions.
Default path behavior is unchanged: when no env var is set and HOME
is not redirected, install.sh / install.ps1 produce exactly the same
file layout as today.
Test scenarios verified locally on install.sh:
* Default (no env vars) -> $HOME/.unsloth/studio (legacy)
* HOME=/tmp/x -> /tmp/x/.unsloth/studio
* UNSLOTH_STUDIO_HOME=/tmp/y -> /tmp/y as STUDIO_HOME root
* STUDIO_HOME=/tmp/z (alias) -> /tmp/z as STUDIO_HOME root
* HOME redirect + env var (HOME wins) -> install follows HOME
* Unwritable override -> exits with clear ERROR message
* install: priority change -- env vars now win over HOME redirect
Flip the resolution order so explicit env vars take precedence over
HOME / USERPROFILE redirection.
New priority (highest first):
1. UNSLOTH_STUDIO_HOME, if set.
2. STUDIO_HOME, if set.
3. HOME / USERPROFILE explicitly redirected.
4. Default.
Rationale: the env vars are explicit single-purpose signals (the user
typed UNSLOTH_STUDIO_HOME=... specifically to redirect Studio). HOME
redirection is broader and incidental -- the user may have redirected
HOME for unrelated reasons (workspace tools, container builds) without
wanting Studio to follow it. When both are set, the more specific
signal should win.
When only HOME is redirected (no env var), behavior is unchanged from
the previous commit: install follows $HOME.
* install: address review feedback (sed escape, downstream propagation, edge cases)
Fixes from gemini-code-assist + chatgpt-codex-connector + reviewer.py
20-parallel run on the open PR.
install.sh:
* Escape sed replacement metacharacters before substituting @@DATA_DIR@@.
Two-stage escape: ' -> '\'' for safe single-quote shell embedding,
then \, &, | for sed replacement string + chosen delimiter. Heredoc
switched to single-quoted DATA_DIR='@@DATA_DIR@@' so we only need
single-quote escaping at runtime. Verified end-to-end with paths
containing & and | (the sed delimiter).
* Pass UNSLOTH_STUDIO_HOME into both setup.sh invocations
(--local and PyPI paths) so the downstream install resolves the
same Studio root install.sh picked.
* macOS .app stub: replace hardcoded
exec "$HOME/.local/share/unsloth/launch-studio.sh" with
exec "$_css_data_dir/launch-studio.sh" so the .app launches the
resolved launcher even in env-override mode.
* Use mkdir -p -- and cd -- when validating the env override so
paths starting with - cannot be misread as flags.
install.ps1:
* Drop .Guid from [guid]::NewGuid().Guid: the property does not
exist; the probe filename was always identical and not unique.
Default ToString() on System.Guid produces the canonical UUID
string we want.
* Guard LOCALAPPDATA before Join-Path to avoid aborting the
installer in service / CI contexts where LOCALAPPDATA is unset
(Join-Path under $ErrorActionPreference='Stop' would otherwise
throw). Computed once into $defaultDataDir; both 'profile' and
'default' branches reuse it.
* Set $env:UNSLOTH_STUDIO_HOME for the duration of the
'unsloth studio setup' subprocess so studio/setup.ps1 and
unsloth_cli see the same install root install.ps1 picked.
Restored in a finally block.
studio/setup.sh:
* Honor UNSLOTH_STUDIO_HOME / STUDIO_HOME (alias) when resolving
STUDIO_HOME, VENV_DIR, VENV_T5_*_DIR. Falls back to the legacy
$HOME/.unsloth/studio when no override is set.
studio/setup.ps1:
* Same change in PowerShell: honor $env:UNSLOTH_STUDIO_HOME /
$env:STUDIO_HOME for $StudioHome / $VenvDir resolution.
unsloth_cli/commands/studio.py:
* Replace the module-level constant
STUDIO_HOME = Path.home() / ".unsloth" / "studio"
with a resolver that honors UNSLOTH_STUDIO_HOME / STUDIO_HOME
before falling through to the legacy default. Same precedence
the installers use.
Verified locally: 6 install.sh scenarios still produce correct paths
(default, HOME redirect, env var, alias, both, bad override). New
sed-escape unit tests pass for paths containing & and |. Python
resolver matches priority: UNSLOTH_STUDIO_HOME > STUDIO_HOME > default.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install.sh: portable sed (no -i.bak) per gemini review feedback
GNU sed -i.bak vs BSD/macOS sed -i.bak vs BusyBox sed have subtly
different semantics. Use the POSIX-portable redirect-then-mv pattern
instead. Functionally identical, runs everywhere.
* studio: persist UNSLOTH_STUDIO_HOME so fresh shells find custom installs
Without this, a custom-root install (UNSLOTH_STUDIO_HOME=/work/studio
bash install.sh --local) only worked in the same shell that ran the
installer. Closing the terminal and reopening lost the env var, the
PATH was deliberately not persisted, and the Python CLI fell back to
~/.unsloth/studio. Result: 'Studio not set up' or quietly operating on
a stale legacy install.
Three persistence layers, all backwards-compatible (default installs
emit zero changes):
1. Unix studio.conf
install.sh now writes 'export UNSLOTH_STUDIO_HOME=...' next to
UNSLOTH_EXE in studio.conf when in env-override mode. The launcher
sources studio.conf at startup so the exec'd binary gets the var.
Default installs do not write this line; studio.conf stays
byte-identical to before.
2. Windows launch-studio.ps1
install.ps1 prepends '$env:UNSLOTH_STUDIO_HOME = ...' to the
generated launcher when in env-override mode. Default installs
produce the same launcher content as before.
3. Python sys.prefix inference
storage_roots.studio_root() and unsloth_cli/commands/studio.py
now infer the install root from sys.prefix when no env var is
set (Path(sys.prefix).parent for unsloth_studio venvs). Catches
direct invocations of <STUDIO_HOME>/bin/unsloth that bypass the
launcher entirely.
unsloth_cli/commands/studio.py also re-exports the resolved
UNSLOTH_STUDIO_HOME via os.environ.setdefault so child processes
(setup script, backend run.py) inherit it.
Backend storage roots (storage_roots.studio_root, cache_root) now
respect the env var via the shared resolver. run.py PID file,
transformers_version.py T5 venvs, and model_config.py vision-check
venv all switch to studio_root() so custom installs are
self-contained.
studio/setup.ps1: T5 sidecar venvs now resolve under $StudioHome
(was $env:USERPROFILE\.unsloth\studio\.venv_t5_*).
studio/setup.sh + studio/setup.ps1: llama.cpp build dir nests under
$STUDIO_HOME / $StudioHome when env-override is active, otherwise
keeps the legacy ~/.unsloth/llama.cpp.
Verified locally:
* studio.conf write block: env-override mode emits the export line;
default mode does not (byte-identical to today).
* PowerShell heredoc interpolation: correct output for both modes.
* studio_root() resolver: default, UNSLOTH_STUDIO_HOME, STUDIO_HOME
alias, and sys.prefix-based inference all return correct paths.
* cache_root() now derives from studio_root().
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: tilde expansion + macOS .app stub safe-quoting
Two fixes from running a 25-scenario simulation sweep against install.sh
across path edge cases (spaces, apostrophes, ampersands, pipes,
backslashes, dollar signs, Unicode, trailing slash, relative paths).
1. UNSLOTH_STUDIO_HOME=~/foo was landing as literal '~/foo' (env vars
are not subject to tilde expansion). Added a POSIX-portable case
block in install.sh, install.ps1, studio/setup.sh, studio/setup.ps1
that expands a leading ~ or ~/ to $HOME / $env:USERPROFILE.
The prefix-removal pattern is single-quoted ('${var#'~/'}') so the
shell does not tilde-expand the pattern back to $HOME/ before
matching -- a subtle dash/bash gotcha.
2. macOS .app stub used an unquoted heredoc ('<< STUB_EOF'), so any
$VAR / backtick / etc in the path would expand at .app launch time.
Switched to single-quoted heredoc ('<< 'STUB_EOF'') with a
placeholder + sed substitution + single-quoted shell embedding,
matching the @@DATA_DIR@@ pattern already used for launch-studio.sh.
Verified: 25/25 simulation scenarios pass on Linux dash + bash,
including paths with $VAR, &, |, \\, ', spaces, and Unicode. End-to-end
install in env-mode + fresh-shell launcher invocation confirmed: studio
binds to /api/health from a clean env, and sys.prefix-based inference
correctly returns the workspace root.
* install: stop accidentally treating default installs as env-override
Reviewer.py 20-runs cycle 1 found a unanimous P1 regression: a default
'unsloth studio update' relocates llama.cpp from ~/.unsloth/llama.cpp
to ~/.unsloth/studio/llama.cpp, because the CLI was re-exporting
UNSLOTH_STUDIO_HOME unconditionally and install.sh / install.ps1 were
passing it into setup.{sh,ps1} unconditionally. The setup scripts
treated the var's mere presence as "env-override mode" and relocated
the llama.cpp build dir away from the legacy path, breaking the
runtime backend's _find_llama_server_binary lookup on default installs.
Fixes:
* unsloth_cli/commands/studio.py: _resolve_studio_home now returns
(path, is_custom). Re-export only when is_custom -- a real env
override or a sys.prefix inference that resolves to a non-legacy
path. Default installs leave UNSLOTH_STUDIO_HOME unset.
* install.sh: gate UNSLOTH_STUDIO_HOME on $_STUDIO_HOME_REDIRECT == env
before calling setup.sh. Use 'env $VARS bash setup.sh' so the var
is set only for the subprocess, never leaked.
* install.ps1: gate $env:UNSLOTH_STUDIO_HOME on $StudioRedirectMode
-eq 'env' before invoking 'unsloth studio setup'. Restore prior
value in finally block (unset if it wasn't set).
* studio/setup.sh + setup.ps1: decide llama.cpp install root from
the resolved $STUDIO_HOME (not from env-var presence). If the
resolved path equals the legacy default ($HOME/.unsloth/studio),
fall back to ~/.unsloth/llama.cpp. This makes setup robust against
a stale UNSLOTH_STUDIO_HOME inherited from a parent process that
happens to point at the legacy default.
* studio/backend/core/inference/llama_cpp.py:
- _find_llama_server_binary() now searches studio_root() / llama.cpp
AND the legacy ~/.unsloth/llama.cpp (de-duped). Custom-root
installs become discoverable; default installs unaffected.
- kill_orphaned_servers ownership allowlist also includes
studio_root() / llama.cpp so custom-root processes are cleanable.
Verified locally:
* 25/25 sim scenarios still pass (path edge cases unchanged).
* setup.sh unit test: default-mode lands UNSLOTH_HOME at $HOME/.unsloth;
env-mode lands at $STUDIO_HOME.
* Python CLI unit test: default-mode returns is_custom=False and does
NOT setdefault UNSLOTH_STUDIO_HOME; env-mode sets is_custom=True.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install: || exit 1 on STUDIO_HOME subshell (dash set -e gap)
Gemini review feedback: in dash, set -e does not trigger on subshell
failures inside variable assignments. If 'cd -- "$_override" && pwd'
fails, STUDIO_HOME stays empty and DATA_DIR collapses to /share. Add
explicit '|| exit 1' on both install.sh:187 and setup.sh:413.
* install.sh: argv-safe setup invocation for paths with spaces
Cycle 2 reviewer.py 20-runs found a unanimous P1: passing the env-var
through 'env $_STUDIO_ENV_FOR_SETUP' word-splits on whitespace, so a
custom root like '/tmp/Unsloth Studio' becomes 'UNSLOTH_STUDIO_HOME=
/tmp/Unsloth' followed by env trying to exec 'Studio'.
Replaced with a tiny helper that prepends the env-var directly to the
argv (no string-form intermediary), so spaces are preserved as a
single argument. Default-mode invocation skips the env-var entirely.
Verified: 'UNSLOTH_STUDIO_HOME=/tmp/test space/studio' now reaches
setup.sh as a single value.
* studio: tighten sys.prefix inference + Tauri env handling + llama.cpp env
Cycle 3 reviewer.py findings (3 P1s converging):
* sys.prefix inference too broad: a developer venv named 'unsloth_studio'
was being treated as a custom Studio root. Narrow with an installer-
sentinel check (presence of share/studio.conf or bin/unsloth shim
inside the parent dir) in both unsloth_cli/commands/studio.py and
studio/backend/utils/paths/storage_roots.py.
* Tauri studio/src-tauri/src/process.rs::find_unsloth_binary() hardcoded
~/.unsloth/studio. Honor UNSLOTH_STUDIO_HOME / STUDIO_HOME (in that
priority order) before falling back to legacy.
* unsloth-zoo's GGUF export binds LLAMA_CPP_DEFAULT_DIR at import time
from UNSLOTH_LLAMA_CPP_PATH. For env-override installs, persist
UNSLOTH_LLAMA_CPP_PATH alongside UNSLOTH_STUDIO_HOME in studio.conf
(Unix), in the generated PowerShell launcher (Windows), and via
os.environ.setdefault in the Python CLI when running on a custom
root, so GGUF export uses the custom-root llama.cpp build instead
of the legacy ~/.unsloth/llama.cpp.
Default behaviour unchanged: no env vars are written to studio.conf
in default mode, no LLAMA_CPP_PATH is set, and the dev-venv inference
falls through to legacy when no installer sentinels are present.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: desktop_auth env-aware + legacy-root llama.cpp consistency
- desktop_auth.rs: honor UNSLOTH_STUDIO_HOME / STUDIO_HOME for the
.desktop_secret path so Tauri desktop login works against custom-root
installs instead of always reading ~/.unsloth/studio/auth/.
- install.sh / install.ps1 / unsloth_cli/commands/studio.py: when an env
override resolves to the legacy default ($HOME/.unsloth/studio), set
UNSLOTH_LLAMA_CPP_PATH to ~/.unsloth/llama.cpp (matching setup.sh /
setup.ps1's legacy-equality branch). Previously the persisted value
pointed at $STUDIO_HOME/llama.cpp, which was a non-existent location
and broke unsloth-zoo's import-time GGUF binding for that edge case.
* studio: tauri studio_root helper + marker-file persistence + ~ expansion
Address cycle-5 reviewer findings:
- Add studio/src-tauri/src/studio_root.rs: shared resolver with
UNSLOTH_STUDIO_HOME / STUDIO_HOME (priority order), tilde expansion
(~, ~/..., ~\...), installer-written marker fallback, then
~/.unsloth/studio. 5 unit tests cover the expansion paths.
- Tauri lookups now go through the shared resolver:
- process.rs::find_unsloth_binary
- desktop_auth.rs::desktop_secret_path
- main.rs::setup_logging (tauri.log under custom root)
- commands.rs::open_logs_dir (opens custom root dir)
- install.rs work_dir uses parent of resolved root (avoids creating
a stray ~/.unsloth on a custom-root install)
- install.sh / install.ps1 (env-mode only): write
~/.unsloth/studio-home marker so the desktop app launched from
Finder/Start Menu (no shell env inheritance) still resolves the
custom root.
- install.sh / install.ps1 non-interactive completion: when
StudioRedirectMode=env, print the absolute custom-root shim path
since the persistent rc/registry PATH update is intentionally
skipped in env-override mode.
- unsloth_cli/commands/studio.py: replace setdefault() with
truthy-check so a blank UNSLOTH_STUDIO_HOME / UNSLOTH_LLAMA_CPP_PATH
in the parent env doesn't suppress the inferred custom root.
40/40 cargo test --bins pass.
* studio: validate marker file + write in --tauri mode + propagate to subprocess
Cycle-6 reviewer follow-ups:
- studio_root.rs marker resolver now validates the persisted path before
using it. A stale ~/.unsloth/studio-home pointing at a deleted/moved
workspace is ignored (resolution falls back to the legacy default
rather than hijacking it). Validation accepts share/studio.conf
sentinel or bin/unsloth shim. Trailing newline strip uses
trim_end_matches(['\n','\r']) so paths whose content legitimately has
leading/trailing spaces survive.
- install.sh / install.ps1: marker write moved out of the launcher
generation path so it runs before the Tauri-mode early exit. Both
shell-launcher and Tauri-installed env-mode roots now persist the
marker. Removed the duplicate marker write that was previously inside
install.ps1's $studioHomeExport block.
- studio/src-tauri/src/install.rs: pass UNSLOTH_STUDIO_HOME to the
installer subprocess (when not already in scope) so app-initiated
repair / update flows reach the same root the running app uses.
cargo test --bins -- --test-threads=1: 44/44 pass (4 new tests for
marker validation: sentinel accepted, bin shim accepted, empty dir
rejected, missing path rejected).
* studio: fix Tauri legacy-fallback regression + stale marker cleanup
Cycle-7 reviewer follow-ups (regression I introduced in cycle 6):
- studio_root.rs: add StudioRootSource enum + resolve_studio_root_with_source().
Lets callers distinguish a real custom override (Env / Marker) from the
legacy fallback (Default).
- studio/src-tauri/src/install.rs: only forward UNSLOTH_STUDIO_HOME to the
installer subprocess when the resolution source is Env or Marker. The
Default fallback must NOT be passed -- install.sh / install.ps1 treat
any non-empty UNSLOTH_STUDIO_HOME as env-override mode and would
relocate DATA_DIR to $STUDIO_HOME/share and _LOCAL_BIN to $STUDIO_HOME/bin
(regressing default Tauri repair / update flows from the legacy
~/.local/share/unsloth and ~/.local/bin).
- install.sh / install.ps1: clear stale marker on default / HOME-redirect
installs. A user who first installed with UNSLOTH_STUDIO_HOME=/work/studio
then later reinstalls without env vars no longer has the desktop app
hijacked by ~/.unsloth/studio-home pointing at the old custom root.
- install.sh / install.ps1: when env mode wins over a redirected
HOME / USERPROFILE, write the marker into the OS-reported real profile
home (getent / dscl on Unix; [Environment]::GetFolderPath on Windows)
so a later desktop launch from the user's normal session still finds
it. Falls back to the current HOME / USERPROFILE.
cargo test --bins -- --test-threads=1: 45/45 pass (1 new for the source
enum invariants).
* install: scrub stale marker from real-home on HOME-redirect cleanup
Cycle-8 reviewer follow-up: the previous cleanup branch only removed
\$HOME/.unsloth/studio-home, leaving a stale marker in the real
password-database home after a prior env-mode install. A later default
install with redirected HOME / USERPROFILE would still see the desktop
app resolving the old custom root.
- install.sh: compute the real password-database home (via getent /
dscl) unconditionally, and scrub markers from BOTH \$HOME and the
real-home in the default / HOME-redirect cleanup branch.
- install.ps1: build a profile-candidate list (current USERPROFILE
+ OS-reported real profile) and remove markers from EVERY candidate
in the default / profile-redirect cleanup branch.
bash -n + cleanup smoke verified.
* revert: drop Tauri env-var support + marker file mechanism
Keep this PR scoped to shell installer + Python backend env-var support.
Tauri desktop integration with custom Studio roots is deferred to a
separate, focused PR.
Reverts to pre-PR state:
- studio/src-tauri/src/process.rs (find_unsloth_binary)
- studio/src-tauri/src/desktop_auth.rs (auth_secret_path)
- studio/src-tauri/src/main.rs (setup_logging tauri.log path)
- studio/src-tauri/src/commands.rs (open_logs_dir)
- studio/src-tauri/src/install.rs (work_dir + subprocess env)
- studio/src-tauri/src/studio_root.rs DELETED
Removes from install.sh / install.ps1:
- ~/.unsloth/studio-home marker write/read/cleanup
- HOME-redirect-aware marker location logic
What this PR keeps (the original scope):
- install.sh / install.ps1: UNSLOTH_STUDIO_HOME / STUDIO_HOME env-var
resolver with HOME-redirect detection, tilde expansion, legacy
fallback. Default installs are byte-identical to pre-PR.
- studio/setup.sh / studio/setup.ps1: legacy-equality llama.cpp path.
- studio.conf / launcher persists UNSLOTH_STUDIO_HOME +
UNSLOTH_LLAMA_CPP_PATH for fresh shells (env-mode only).
- unsloth_cli/commands/studio.py: env > sys.prefix sentinel > legacy
resolver, conditional re-export.
- studio/backend/utils/paths/storage_roots.py: same resolver.
- Backend modules use storage_roots (run.py, model_config.py,
transformers_version.py, llama_cpp.py).
cargo test --bins -- --test-threads=1: 34/34 pass (pre-PR baseline).
bash -n install.sh: clean.
* install: cycle-10 fixes (default launcher, --tauri guard, env-mode shortcuts, win PATH)
- install.sh launcher: default and HOME-redirect installs keep the
legacy DATA_DIR=\"\$HOME/.local/share/unsloth\" runtime form so a
later shell with a different \$HOME still resolves DATA_DIR. Only
env-mode bakes the resolved absolute path. Restores byte-identical
default behavior.
- install.sh / install.ps1: fail fast when --tauri is combined with
UNSLOTH_STUDIO_HOME / STUDIO_HOME. The desktop app still resolves
the legacy ~/.unsloth/studio root, so a custom-root --tauri install
would yield a desktop app that cannot find its binary or auth
secret. Print the right alternative.
- install.sh / install.ps1: skip persistent desktop / Start-Menu
shortcuts in env-override mode. Workspace-scoped installs would
otherwise leave launchers pointing at a path the user may delete.
Default and HOME/profile-redirect installs keep the shortcut.
- install.ps1: re-prepend env-override \$ShimDir AFTER
Refresh-SessionPath. Refresh rebuilds PATH as Machine > User >
current \$env:Path, so a previously-installed legacy User PATH
entry would otherwise win precedence over the current-session
env-override shim.
bash -n install.sh, pwsh parser install.ps1 + setup.ps1: clean.
cargo test --bins -- --test-threads=1: 34/34 (Tauri unchanged).
* install: cycle-11 fixes (env-mode launcher writes, --tauri legacy passthrough, run.py llama path)
- install.sh / install.ps1: env-mode no longer skips the entire
create_studio_shortcuts / New-StudioShortcuts function. Move the
early-return INSIDE those functions, just before the persistent
desktop / Start-Menu shortcut creation. The runtime launcher
(launch-studio.sh / launch-studio.ps1), studio.conf with
UNSLOTH_STUDIO_HOME / UNSLOTH_LLAMA_CPP_PATH exports, and the icon
ARE always written so env-mode shims can resolve via fresh shells.
- install.sh / install.ps1: --tauri guard passes through when the
override resolves to the legacy default ($HOME/.unsloth/studio /
%USERPROFILE%\.unsloth\studio). The desktop app already uses that
path, so explicit-equality is a supported edge case (matches the
llama.cpp legacy-equality branch).
- studio/backend/run.py: when launched directly (bypassing the
unsloth CLI), set UNSLOTH_STUDIO_HOME and UNSLOTH_LLAMA_CPP_PATH
before the rest of import chain runs so unsloth-zoo's import-time
LLAMA_CPP_DEFAULT_DIR binding picks up the custom-root build. Only
set when STUDIO_ROOT is a real custom override; legacy default
installs leave them unset.
bash -n install.sh, pwsh parser install.ps1: clean.
python ast parse studio/backend/run.py: clean.
cargo test --bins -- --test-threads=1: 34/34 pass (Tauri unchanged).
* install: cycle-12 fixes (--tauri trailing slash + main.py uvicorn env)
- install.sh / install.ps1 --tauri legacy passthrough: strip trailing
separators before comparing the override to the legacy default.
Previously UNSLOTH_STUDIO_HOME=\"\$HOME/.unsloth/studio/\" (with
trailing slash) was rejected even though it resolves to the
supported legacy root.
- studio/backend/main.py: when launched directly via
\`uvicorn main:app\` from a custom-root venv (bypassing both
unsloth_cli and run.py), export UNSLOTH_STUDIO_HOME and
UNSLOTH_LLAMA_CPP_PATH before any unsloth-zoo import so its
import-time LLAMA_CPP_DEFAULT_DIR binding picks up the custom-root
build. Only sets when STUDIO_ROOT is a real custom override.
bash -n install.sh, pwsh parser install.ps1, python ast main.py: clean.
Smoke probe: UNSLOTH_STUDIO_HOME=\$HOME/.unsloth/studio/ install.sh --tauri
no longer exits with the unsupported-custom-root error.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install.ps1: skip CWD-relative venv migration in env-override mode
The legacy ~/unsloth_studio venv migration path on Windows reads
%USERPROFILE%\unsloth_studio\Scripts\python.exe (a fixed home-relative
path). Under env-override mode this would Move-Item the user's
pre-existing default-install venv into $StudioHome\unsloth_studio,
breaking the default install and contaminating the workspace root.
Gate the migration on $StudioRedirectMode -ne 'env' so workspace-scoped
installs leave the user's default-install venv untouched.
No Linux equivalent: install.sh migrates from \$STUDIO_HOME/.venv which
is already env-mode-aware (points at the workspace root, not \$HOME).
* install: cycle-14 fixes (Tauri env scrub + setup.ps1 missing-root error)
Tauri does not honor UNSLOTH_STUDIO_HOME / STUDIO_HOME / UNSLOTH_LLAMA_CPP_PATH
yet -- the desktop app's Rust paths use the legacy ~/.unsloth/studio root.
If the user's shell has these env vars set, spawned Python subprocesses would
diverge from the Rust paths (custom-root Python <-> legacy-root Rust).
Scrub the three env vars at all Tauri subprocess spawn sites:
- process.rs: backend launch
- desktop_auth.rs: provision-desktop-auth subprocess
- install.rs: install.sh / install.ps1 invoked from the desktop app
(also prevents the --tauri guard from rejecting an inherited override).
setup.ps1: when UNSLOTH_STUDIO_HOME points at a non-existent directory,
'Resolve-Path -LiteralPath' threw a confusing PSObject error under
$ErrorActionPreference = "Stop". Test-Path the override first and emit a
friendly "run install.ps1 to create the install root" message instead.
* install: cycle-15 fixes (preserve UNSLOTH_LLAMA_CPP_PATH + add update.rs scrub)
UNSLOTH_LLAMA_CPP_PATH is a pre-existing custom-llama.cpp-directory override
the Python backend (studio/backend/core/inference/llama_cpp.py) and unsloth-zoo
intentionally support. It is unrelated to the Studio install root. Cycle 14
over-scrubbed it from the Tauri spawn sites, regressing desktop GGUF/llama.cpp
workflows for users who set it in their shell.
- process.rs / desktop_auth.rs / install.rs: stop scrubbing
UNSLOTH_LLAMA_CPP_PATH; only scrub UNSLOTH_STUDIO_HOME and STUDIO_HOME.
- update.rs: missed Tauri spawn site -- add the same UNSLOTH_STUDIO_HOME /
STUDIO_HOME scrub so 'unsloth studio update' from the desktop app updates
the legacy-root install Tauri actually manages.
Verified: cargo test --bins -- --test-threads=1 -> 34/34 pass.
* install.sh: document apostrophe-escape derivation inline
The shell quoting at install.sh:642 / 659 / 679 / 680 / 823 has been
flagged as broken across multiple review cycles, but every end-to-end
verification (DATA_DIR=\"a b's&c|d\$e\" -> generated launcher -> source ->
recovered exact input) passes. The proposed "8 backslash" fix would
double the escape and actually break what currently works.
Strengthen the inline comments to spell out the derivation:
- shell pattern \"s/'/'\\\\''/g\" passes \"s/'/'\\''/g\" to sed (\\\\ -> \\)
- sed replacement '\\'' yields close-quote / escaped-quote / open-quote
- stage 2 (\\, &, |) only needed where the value is then sed-replaced
into a launcher template via s|@@DATA_DIR@@|VALUE|g
studio.conf is written via printf, not sed, so it only needs stage 1.
No behavior change, only inline doc to head off future false positives.
* install/setup .ps1: use -LiteralPath for $StudioHome-derived paths
Pre-PR, $StudioHome was hardcoded to %USERPROFILE%\.unsloth\studio --
no wildcard characters possible. The PR introduces UNSLOTH_STUDIO_HOME /
STUDIO_HOME, so $StudioHome (and every path derived from it: $VenvDir,
$VenvPyExe, $UnslothExe, $UnslothHome, $LlamaCppDir, $VenvT5_*, etc.)
can now contain bracket characters that PowerShell would interpret as
wildcards.
Reproducer (from cycle 17 review 20):
pwsh> Test-Path 'studio[abc]/Scripts/python.exe'
False
pwsh> Test-Path -LiteralPath 'studio[abc]/Scripts/python.exe'
True
Switch the relevant Test-Path / Remove-Item / New-Item / Move-Item calls
in install.ps1 and studio/setup.ps1 to -LiteralPath. Sites where the
path is fixed (the shim under %LOCALAPPDATA%\Microsoft\WindowsApps,
$RepoRoot from -PSCommandPath) keep the wildcard-aware form.
* install/setup .ps1: fix New-Item -LiteralPath regression from cycle 17
Cycle 17 added -LiteralPath to all $StudioHome-derived path operations,
but New-Item has no -LiteralPath parameter (verified pwsh 7.6 syntax:
"New-Item [-Path] <string[]> [-ItemType <string>] ..."). Every directory-
creation site would throw "A parameter cannot be found that matches
parameter name 'LiteralPath'" at runtime, blocking T5 sidecar setup,
llama.cpp parent creation, and StudioHome creation.
Likewise, "Split-Path -LiteralPath $X -Parent" cannot mix LiteralPath
with -Parent (separate parameter sets). The default LiteralPath mode
already returns the parent.
Switch to [System.IO.Directory]::CreateDirectory($X), which natively
takes a literal path, and drop the trailing -Parent on Split-Path.
Verified end-to-end on a bracketed path "/tmp/...[abc]":
- CreateDirectory: created
- Test-Path -LiteralPath: detects
- nested CreateDirectory(Split-Path -LiteralPath ...): works
* install/setup .ps1: extend -LiteralPath sweep to remaining \$StudioHome paths
Cycle 17/18 missed several wildcard-aware operations on user-controlled
\$StudioHome-derived paths. Reviewers identified remaining sites:
install.ps1:
- \$UnslothExePath (Test-Path / Resolve-Path) at the shortcut creator
- \$VenvDir (Get-ChildItem) at the no-torch-runtime resolver
- \$ShimDir (New-Item Directory -- replaced with .NET CreateDirectory)
- \$ShimExe (Test-Path / Remove-Item / re-prepend guards) -- the shim
lives at \$StudioHome\\bin\\unsloth.exe in env-override mode, so it
inherits bracket sensitivity from \$StudioHome.
- \$UnslothExe (Copy-Item fallback) when HardLink fails.
studio/setup.ps1:
- \$LlamaServerBin (Test-Path) at the prebuilt-bundle / source-build
validation gates (3 sites). \$LlamaServerBin lives under \$BuildDir
under \$LlamaCppDir under \$UnslothHome under \$StudioHome.
New-Item HardLink keeps -Path because creating a non-existent target
with brackets succeeds (verified via direct pwsh smoke test).
* install: cycle-20 fixes (more setup.ps1 -LiteralPath + shell-quote launch hints)
setup.ps1: extend -LiteralPath sweep to remaining \$BuildDir-derived paths
that the cycle-19 commit missed:
- \$CmakeCacheFile (Test-Path + Select-String -Path)
- \$buildTmp (10 Test-Path / Remove-Item sites in source-build cleanup)
- \$QuantizeBin (Test-Path)
- \$altBin (Test-Path)
These all live under \$BuildDir -> \$LlamaCppDir -> \$UnslothHome ->
\$StudioHome, which is now user-controlled via UNSLOTH_STUDIO_HOME.
Bracket characters in the override would silently skip rebuild
detection or leave stale build artifacts.
install.sh: shell-quote the launch-instruction substep lines for env-
override mode. UNSLOTH_STUDIO_HOME values containing spaces or
apostrophes (e.g. "/tmp/O'Brien Studio") would print copy-paste-
unsafe commands -- the install succeeded but the printed launch
instructions split at the space. Now wraps with the canonical
'\\''-style escape so the printed lines parse with bash -n.
Verified end-to-end:
- printed shim line: '/tmp/O'\''Brien Studio/bin/unsloth' studio ...
- bash -n on the printed line passes.
* install.ps1: -LiteralPath for macOS-stub-launcher \$appDir-derived paths
The shortcut/launcher generator at install.ps1:418-693 writes the
stub launcher, .vbs, and icon under \$appDir = \$StudioDataDir, which in
env-override mode is \$StudioHome\share. Cycle 17/19/20 missed the
following wildcard-aware ops on these paths:
- Test-Path \$appDir (with New-Item Directory swap to .NET CreateDirectory)
- Set-Content -Path \$launcherVbs (for the WSH .vbs stub)
- Test-Path / Copy-Item \$bundledIcon (bundled icon copy)
- Test-Path / Remove-Item \$iconPath (icon header validation)
In env-override mode \$StudioHome can contain bracket characters;
without -LiteralPath the .vbs write fails outright and the icon
validation can either skip a present icon or fail to delete a
malformed one. (The COM shortcut creation downstream returns early
in env-override mode, so its path values don't need this treatment.)
* install: don't override pre-existing UNSLOTH_LLAMA_CPP_PATH in launchers
Cycle 14/15 established UNSLOTH_LLAMA_CPP_PATH as a pre-existing
custom-llama.cpp-directory override the Python backend and unsloth-zoo
intentionally support, independent of the Studio install root.
The launchers (studio.conf sourced by Unix launch-studio.sh, and the
PowerShell launch-studio.ps1) were unconditionally re-exporting it,
which silently overrides a user's pre-existing value when they invoke
the launcher from a shell where UNSLOTH_LLAMA_CPP_PATH is already set.
Make the assignment conditional in both launchers:
install.sh studio.conf:
if [ -z "\${UNSLOTH_LLAMA_CPP_PATH:-}" ]; then
export UNSLOTH_LLAMA_CPP_PATH='...'
fi
install.ps1 launch-studio.ps1:
if (-not \$env:UNSLOTH_LLAMA_CPP_PATH) {
\$env:UNSLOTH_LLAMA_CPP_PATH = '...'
}
UNSLOTH_STUDIO_HOME stays unconditional: the launcher is bound to a
specific install, so its STUDIO_HOME must always match that install.
* install.sh: harden --tauri legacy resolver against CDPATH and symlinks
Reviewer cycle 23 (inst 19) noted that the bare \`cd -- ... && pwd\` form
in the --tauri legacy comparison can echo a CDPATH-prefixed path when the
user has CDPATH set in their environment, contaminating the resolved
absolute path used in the legacy-equality check.
Switch to \`CDPATH= cd -P -- ... && pwd -P\` so:
- CDPATH= clears the cd-prefix-echo behavior
- -P / pwd -P resolves any symlinks to a canonical path
No behavior change for users without CDPATH set; correctness fix for
users who have it set in their shell.
* install + llama_cpp backend: cycle-24 hardening
Three real findings from cycle 24 reviewers:
1. install.sh:231 + studio/setup.sh:413 -- main \$STUDIO_HOME
resolvers used the same bare \`cd -- ... && pwd\` form that cycle 23
only fixed for the --tauri guard. Switch both to:
\$(CDPATH= cd -P -- "\$override" && pwd -P)
so relative custom-root values don't get CDPATH-prefixed or have
the cd-on-CDPATH stdout newline contaminate the captured value.
2. install.sh --tauri legacy root used logical \$HOME/.unsloth/studio
while the override side was canonicalized via pwd -P. A symlinked
\$HOME (e.g. /home/alice -> /u/alice) made the comparison fail even
when both sides pointed at the same directory. Canonicalize the
legacy side too when the dir exists.
3. studio/backend/core/inference/llama_cpp.py:_find_llama_server_binary
searched \$STUDIO_HOME/llama.cpp first then ~/.unsloth/llama.cpp
in default-mode installs. setup.sh / setup.ps1 only install llama.cpp
under \$STUDIO_HOME/llama.cpp in env-override mode; in default mode
it always lives at ~/.unsloth/llama.cpp. The post-PR search would
pick up a stale partial install at ~/.unsloth/studio/llama.cpp over
the real legacy binary.
Mirror setup's legacy-equality check: when studio_root() resolves
equal to ~/.unsloth/studio, search ONLY the legacy ~/.unsloth/llama.cpp.
Otherwise (env-override custom root), search custom first, legacy
fallback.
* install + setup: canonicalize legacy-equality comparison sites
Cycle 24 made \$STUDIO_HOME canonical via 'CDPATH= cd -P -- ... && pwd -P',
but the legacy-equality comparison sites still used the bare logical
"\$HOME/.unsloth/studio" string. With a symlinked \$HOME (e.g.
/home/alice -> /u/alice), the comparison fails even when both sides
point at the same dir, and llama.cpp ends up under a custom-root path
the Python backend's legacy comparison cannot find.
Reviewer cycle 25 inst 2 reproduced this with HOME=/tmp/link -> /tmp/real
and UNSLOTH_STUDIO_HOME=\$HOME/.unsloth/studio: setup.sh resolves
UNSLOTH_HOME to /tmp/real/.unsloth/studio while the backend search
resolves both physically equal and looks at /tmp/link/.unsloth/llama.cpp.
Canonicalize the legacy side at all four sites:
- install.sh:695 (create_studio_shortcuts llama.cpp path)
- studio/setup.sh:577 (UNSLOTH_HOME selection)
- install.ps1:462 (launcher UNSLOTH_LLAMA_CPP_PATH path)
- studio/setup.ps1:1829 (UnslothHome selection)
Apply CDPATH= cd -P -- ... && pwd -P (Unix) or Resolve-Path -LiteralPath
(Windows) when the legacy dir exists. unsloth_cli/commands/studio.py
already does this via Path.resolve().
* llama_cpp: gate _kill_orphaned_servers studio-root allowlist on env-override
Cycle 24 fixed _find_llama_server_binary to only search
\$STUDIO_HOME/llama.cpp when STUDIO_HOME is a real env override (not
the legacy default), but the symmetric _kill_orphaned_servers
allowlist still appended _sr() / "llama.cpp" unconditionally.
In default mode _sr() resolves to ~/.unsloth/studio, so
~/.unsloth/studio/llama.cpp would be treated as a Studio-owned install
root for the orphan-kill scan even though the default installer does
not own that path. A llama-server process running there from a
different tool or a stale partial install would be killed.
Apply the same legacy-equality check used in _find_llama_server_binary
and the install/setup scripts: only add _sr()/"llama.cpp" to the
allowlist when STUDIO_HOME != legacy default.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* setup.sh + setup.ps1: canonicalize both sides of legacy-equality check
Proactive audit pass found one real asymmetry the cycle-by-cycle
review process had not yet flagged:
- install.sh:704 / install.ps1:469 are gated on env-mode and only
run when STUDIO_HOME has already been canonicalized (cycle 24).
Symmetric.
- studio/setup.sh:577 / studio/setup.ps1:1829 run UNCONDITIONALLY,
including in default mode. In default mode STUDIO_HOME is set to
the bare logical \$HOME/.unsloth/studio (setup.sh:416) or
Join-Path \$env:USERPROFILE ".unsloth\\studio" (setup.ps1:1480).
Cycle 25 canonicalized only the legacy side, creating an
asymmetry under symlinked \$HOME / junctioned %USERPROFILE%.
Result of the asymmetry: a default-mode install on a host with
\$HOME=/tmp/link -> /tmp/real treats the legacy default as a custom
root, putting llama.cpp at \$STUDIO_HOME/llama.cpp instead of
~/.unsloth/llama.cpp -- and the Python backend's _find_llama_server_binary
(which uses .resolve() on both sides) then can't find the install.
Fix: canonicalize STUDIO_HOME on the fly at the comparison site, in
both setup.sh and setup.ps1. Symmetric with the now-canonicalized
legacy side from cycle 25, regardless of which mode set STUDIO_HOME.
The other two comparison sites (install.sh:704, install.ps1:469) are
already symmetric because they only run when STUDIO_HOME comes from
the env-override resolution path that already does pwd -P / Resolve-Path.
unsloth_cli/commands/studio.py + studio/backend/run.py + main.py +
llama_cpp.py already use .resolve() on both sides -- symmetric.
* install.ps1: env-override resolution uses .NET API for literal paths
Gemini code-review (review 4177641398, commit 2ea2c91) caught two
remaining New-Item -Path sites in the env-override resolution block
that the cycle 18 sweep missed:
- Line 123: New-Item -ItemType Directory -Path \$envOverride
- Line 132: New-Item -ItemType File -Path \$probe (writability test)
Both use -Path which interprets square brackets as wildcards. For a
user with UNSLOTH_STUDIO_HOME=C:\\workspaces\\studio[abc], both calls
would fail before the install starts. New-Item also has no
-LiteralPath in PowerShell 5.1.
Replace both with the .NET API:
- [System.IO.Directory]::CreateDirectory(\$envOverride)
- [System.IO.File]::WriteAllText(\$probe, "") -- closes the file
handle before the Remove-Item below.
End-to-end verified with /tmp/test-envoverride-[abc]-* path:
CreateDirectory + WriteAllText + Test-Path -LiteralPath all work.
* comments: condense multiline blocks added by this PR
Across the 27-cycle review process, comments accumulated as multiline
blocks explaining each fix's history (cycle numbers, prior bugs,
reviewer rationale). Compress every block to 1-2 lines that capture
just the WHY, dropping cycle references and history that belongs in
the PR description / commit log instead.
Net: 268 deletions / 124 insertions (-144 lines) of comments only.
Behavior unchanged. Verified: bash -n, pwsh parser, python ast.parse,
cargo check all pass.
* install.ps1: use 'return' over 'exit 1' for Install-UnslothStudio bail-outs
Per Gemini review #4177659001: when users run install.ps1 via
'irm ... | iex', 'exit 1' inside the function terminates the entire
PowerShell process and closes the user's terminal. 'return' bails out
of the function while keeping the shell open, matching existing error
sites at lines 34, 50, 57.
Three sites fixed: --tauri+env-override guard, env-override mkdir/access
failure, and write-probe failure. The 'exit' calls at lines 591/611
are inside a generated launcher here-string (a separate top-level .ps1
that runs as its own process), so they correctly stay as 'exit'.
* install.{sh,ps1}: address Gemini review #4177680451
Three medium fixes:
1. install.sh redirection detection: canonicalize both sides of the
$HOME vs passwd-DB comparison via 'CDPATH= cd -P -- ... && pwd -P'
so a trailing slash on $HOME (or symlink-vs-realpath mismatch with
getent/dscl output) doesn't misfire the redirection branch.
2. install.sh shim symlink: 'ln -sf' into an existing directory creates
the link INSIDE it ($_LOCAL_BIN/unsloth/unsloth instead of the
intended file). Pre-strip a real (non-symlink) directory at
$_LOCAL_BIN/unsloth before linking.
3. install.ps1 ShimExe: add -Recurse to Remove-Item so the launcher
refresh recovers if $ShimExe somehow exists as a directory rather
than a file (would otherwise drop into the catch and skip the
shim update).
* install.ps1: use 'throw' over 'return' for fatal validation failures
Cycle 28 reviewer.py (12/8 RC/APPROVE) caught a regression introduced
by the previous Gemini-review fix (#4177659001 -> commit 393e676b).
'return' inside Install-UnslothStudio kept iex'd terminals alive but
made 'pwsh -File install.ps1' exit with code 0 on fatal validation
failures (--tauri+custom-root rejected, STUDIO_HOME unwritable, etc.),
so CI / wrapper scripts treated failed installs as successful.
'throw' satisfies both constraints:
- pwsh -File install.ps1: exits with code 1 (CI sees failure)
- irm | iex: shows error to user, does NOT close the host terminal
Three sites: --tauri+env-override guard, mkdir/access failure,
write-probe failure. Verified throw -> exit code 1 under pwsh -File.
* install.ps1 launcher: single-quote child -Command path
Cycle 28 P2 finding: the generated launch-studio.ps1 builds the child
PowerShell -Command string with the executable path inside double
quotes, so a custom Studio root containing PowerShell metacharacters
(\$, backtick) re-expands in the child shell. Example:
D:\work\\\$job\studio -> child reparses \$job and runs the wrong path.
Fix: single-quote the path inside the child command and double any
apostrophes (PowerShell's literal-quote-escape form) so paths like
"O'Brien Studio & x|y" or "C:\work\\\$bad\studio" survive verbatim.
* install: harden custom Studio root handling
- install.sh shim refresh: refuse to recursively delete a real directory
at $_LOCAL_BIN/unsloth before creating the symlink. The previous rm -rf
could destroy unrelated user data living at that path.
- install.ps1 shim refresh: drop -Recurse from Remove-Item on $ShimExe and
refuse early when the shim path is a directory; mirrors the install.sh
guard so a directory at $StudioHome\bin\unsloth.exe is not blown away.
- install.ps1 PATH wiring: remove the redundant first $ShimDir prepend in
env-override mode; the post-Refresh-SessionPath prepend is the one that
takes effect, and the duplicate left $ShimDir in $env:Path twice.
- install.ps1 manual launch instructions: single-quote the printed shim
and Activate.ps1 paths so '$' / backtick metacharacters in custom roots
do not reparse when the user copies and pastes the command.
- studio/setup.sh: validate writability of UNSLOTH_STUDIO_HOME with the
same [ -w ] check install.sh already has, so a read-only override fails
with a clear message instead of an obscure uv pip permission error.
- Drop the STUDIO_HOME alias everywhere (storage_roots.py, studio.py,
install.sh, studio/setup.sh, install.ps1, studio/setup.ps1). The name
is too generic and an ambient STUDIO_HOME from unrelated tooling could
silently redirect the install. Only UNSLOTH_STUDIO_HOME is honored.
- unsloth_cli/commands/studio.py: defer UNSLOTH_STUDIO_HOME / UNSLOTH_LLAMA_CPP_PATH
re-export from import time into a helper invoked by the studio app
callback. Importing the module no longer mutates os.environ as a side
effect, so test runners and CLI introspection stop leaking those vars
into unrelated subprocesses.
- studio/backend/core/inference/llama_cpp.py: replace set-mutation inside
list comprehension with an explicit dedup loop for readability.
* install: harden custom Studio root edge cases
- install.ps1 shim refresh: move the directory-collision preflight outside
the lock-handling try/catch. The previous throw inside the try block was
swallowed by the surrounding catch and downgraded to a "Continuing with
the existing launcher" warning, leaving the install in a broken state
with no usable shim on disk.
- storage_roots.py / unsloth_cli/commands/studio.py: tighten the bin-shim
sentinel from .exists() to .is_file(). A directory at the candidate
bin/unsloth (or bin/unsloth.exe) path would otherwise false-positive
the venv inference and pick the wrong Studio root.
- storage_roots.py / unsloth_cli/commands/studio.py: wrap the env-var
override Path(...).expanduser().resolve() in try/except (OSError, ValueError),
matching the defensive pattern already used in studio/backend/main.py
and studio/backend/run.py. An invalid override (unresolvable network
drive, bad characters) now falls back to the un-resolved path instead
of crashing at import time.
* install: fail fast on missing custom root, allow brackets in shim path
- install.ps1 shim hardlink: switch the New-Item -ItemType HardLink call
from -Path to -LiteralPath so a custom Studio root containing bracket
characters does not fail under PowerShell's wildcard-aware -Path
parameter. Matches the -LiteralPath usage on every other Test-Path /
Remove-Item / Copy-Item call against the same shim path.
- studio/setup.sh override branch: replace the silent mkdir -p of the
override directory with an existence check that exits 1 with a clear
message. setup.sh runs against an existing install (via 'unsloth
studio update'), so a typo in UNSLOTH_STUDIO_HOME must not materialize
an empty workspace dir. Brings the Unix flow in line with setup.ps1,
which already errors on a missing override root.
* llama_cpp: scope orphan-server kill to the active install root
_kill_orphaned_servers used to unconditionally include the legacy
~/.unsloth/llama.cpp tree in install_roots, even when the running
Studio is in env-override mode and operates out of a custom root.
On a single OS user running both a default-install Studio and a
custom-root Studio concurrently, the custom Studio would kill the
default Studio's llama-server during startup orphan cleanup.
Hoist _is_custom_root out of the import try/catch so the legacy-
append decision sees it (default to False on ImportError so default
mode behaviour is unchanged), and gate the legacy ~/.unsloth/llama.cpp
append on `not _is_custom_root`.
* install: harden custom-root .venv migration and shim hardlink
- install.sh / install.ps1 OLD-layout .venv migration: gate on
default-mode only. Without the guard, pointing UNSLOTH_STUDIO_HOME at a
workspace that already has .venv (e.g. an unrelated Python project)
caused the torch validation to fail and the installer to recursively
remove the user's project venv. Mirrors the existing env-mode skip on
the CWD-relative venv migration immediately below.
- install.ps1 shim hardlink: revert to New-Item -ItemType HardLink -Path.
-LiteralPath is not accepted on the HardLink ItemType in any PowerShell
version, so the previous form always threw and silently fell back to
Copy-Item, breaking hardlink-update propagation. Bracket characters in
$ShimExe are still defended by the directory-collision preflight added
earlier.
- storage_roots.py / unsloth_cli/commands/studio.py: strip whitespace
from the UNSLOTH_STUDIO_HOME env var before the truthy check so a
blank " " override does not become a real path with trailing spaces
(which would silently break every downstream Studio path operation).
* Studio paths: tolerate stat / resolve failures during root inference
- storage_roots._infer_studio_home_from_venv: wrap the share/studio.conf
and bin/shim is_file() sentinel checks in try/except OSError. A
PermissionError on a restricted candidate dir would otherwise propagate
out of studio_root() and crash module import in run.py / main.py /
transformers_version.py / model_config.py at server startup.
- llama_cpp._kill_orphaned_servers: broaden the studio_root() guard from
ImportError-only to (ImportError, OSError, ValueError) so transient
resolve / sentinel failures do not crash the orphan-killer at server
startup. Matches _find_llama_server_binary's existing pattern.
- llama_cpp._find_llama_server_binary: nest the inner resolve() in its
own try/except and fall back to unresolved-path comparison instead of
dropping the custom search root entirely. A transient resolve() error
on the legacy path no longer loses the custom-root llama.cpp lookup.
* Add Studio install-root resilience tests
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: isolate custom-root installs from default-install state
- llama.cpp discovery in env-override mode no longer falls back to the
legacy ~/.unsloth/llama.cpp tree. The orphan-cleanup path already
excludes that root in custom mode; aligning discovery prevents a
custom-root Studio from launching a sibling install's binary it then
refuses to manage. Users who want a shared build set
UNSLOTH_LLAMA_CPP_PATH explicitly.
- Generated POSIX launcher (install.sh heredoc) namespaces LOCK_DIR with
a hash of DATA_DIR and persists the launched port to
$DATA_DIR/studio.port; in env-override mode the fast-path attaches only
to a port we ourselves wrote, never to a sibling Studio that happens
to be healthy on 8888..8908.
- Generated Windows launcher (install.ps1 heredoc) bakes a per-install
$portFile and SHA-256-suffixed mutex name, mirroring the POSIX side;
Find-HealthyStudioPort uses the port file in env-override mode.
- studio/setup.sh and studio/setup.ps1 require an .unsloth-studio-owned
marker before deleting $STUDIO_HOME/.venv_t5*, $STUDIO_HOME/llama.cpp,
and the sidecar T5 venvs in env-override mode. The marker is dropped
after fresh creation so subsequent runs of 'unsloth studio update'
proceed cleanly. Mirrors the existing .venv guard in install.sh.
- Wrap bare Path.resolve() calls on the legacy STUDIO_HOME constant in
studio/backend/main.py, studio/backend/run.py, and
unsloth_cli/commands/studio.py in the same try/except (OSError,
ValueError) used adjacently, so a restricted parent or recursive
symlink on $HOME does not crash module import / CLI startup.
* Studio: guard env-mode workspace against destructive cleanup
- install.sh and install.ps1 unconditionally rm -rf / Remove-Item the
new-layout $STUDIO_HOME/unsloth_studio when it has a python; in
env-override mode that path is a user-chosen workspace, mirroring
the .venv migration concern the .venv branch already guards. Refuse
to remove an existing $STUDIO_HOME/unsloth_studio that lacks Studio
sentinels (share/studio.conf or bin/unsloth).
- studio/setup.ps1 only checked Test-Path -PathType Container on the
custom root; setup.sh and install.ps1 both also write-probe via
WriteAllText / Remove-Item. Add the matching probe so 'unsloth
studio update' against an ACL-restricted root fails fast with a
clear message instead of erroring later while creating sidecar
venvs.
* Add Studio install/setup workspace-isolation tests
* Studio: tighten installer rationale comments
- install.sh: collapse a 5-line restatement into 3 lines, naming
env-mode behavior up front and the byte-identical pre-override
fallback after.
- install.ps1: correct misleading hardlink comment that claimed the
directory-collision preflight guards against wildcard expansion;
bracket characters in $ShimExe still glob-expand here, with the
Copy-Item -LiteralPath fallback handling them.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Split: keep only 2 file(s)
* Studio: harden env-mode workspace guards across installers and update path
Tightens the UNSLOTH_STUDIO_HOME custom-root protections so destructive
installer paths cannot displace unrelated user data when the override
points at a workspace.
install.sh / install.ps1: env-mode sentinel that gates rm -rf $VENV_DIR /
Remove-Item $VenvDir now requires share/studio.conf or the bin/unsloth(.exe)
shim to be a real file or symlink. Previously a directory at bin/unsloth or
bin\unsloth.exe satisfied the check (-e and bare Test-Path accept any path
type), so a workspace with unrelated content under unsloth_studio plus a
sibling directory at bin/unsloth could be wiped.
studio/setup.ps1: stale-venv rebuild branch now mirrors install.ps1's
env-mode guard before Remove-Item -LiteralPath $VenvDir -Recurse -Force.
Without this, "unsloth studio update" pointed at a custom workspace whose
unsloth_studio venv fails torch validation deletes the venv even when the
root carries no Studio sentinels.
studio/setup.sh / studio/setup.ps1: prebuilt llama.cpp install path now
calls _assert_studio_owned_or_absent / Assert-StudioOwnedOrAbsent before
invoking install_llama_prebuilt.py, and writes the .unsloth-studio-owned
marker on success. install_llama_prebuilt.py uses os.replace() to move
any existing install_dir aside before staging, so an unrelated
$STUDIO_HOME/llama.cpp could otherwise be displaced before the existing
source-build ownership guard ever ran.
* Studio: gate ownership guards on canonical custom-root and add venv marker
Tightens UNSLOTH_STUDIO_HOME ownership semantics so they fire only for a
genuinely custom root, never for an explicit override that resolves to the
legacy default. Adds an in-VENV marker that lets a partial install be
repaired and provides a strong primary sentinel for the deletion guard.
studio/setup.sh + studio/setup.ps1: hoist the canonical $STUDIO_HOME vs
legacy-default comparison so it sits next to the marker definition, derive
_STUDIO_HOME_IS_CUSTOM / $StudioHomeIsCustom once, and gate the
_assert_studio_owned_or_absent / Assert-StudioOwnedOrAbsent helpers and the
prebuilt llama.cpp marker writes on that flag instead of raw env-var
presence. UNSLOTH_STUDIO_HOME=$HOME/.unsloth/studio (legacy override) no
longer trips the guard for pre-PR T5 sidecar venvs or llama.cpp dirs that
predate the .unsloth-studio-owned marker. The duplicate canonical block
inside the llama.cpp section is removed; the new flag is reused.
studio/setup.ps1: Assert-StudioOwnedOrAbsent's marker check now requires
-PathType Leaf so a directory at .unsloth-studio-owned cannot satisfy it.
The in-place git-sync branch in the source-build path now calls
Mark-StudioOwned after a successful sync so a later prebuilt-update path
does not fail Assert-StudioOwnedOrAbsent on the same root.
install.sh + install.ps1: write $VENV_DIR/.unsloth-studio-owned right after
uv venv succeeds and accept it as the primary sentinel in the env-mode
deletion guard. This recovers from a partial install that was previously
unrepairable, and is a stronger sentinel than sibling shim files (the
marker is inside the venv that is about to be wiped, so an unrelated
workspace cannot accidentally satisfy it).
install.sh: drop the standalone -L test on $STUDIO_HOME/bin/unsloth in the
deletion guard. -L returns true for any symlink including symlinks to
directories and broken symlinks; -f already accepts the legitimate
file-targeted symlink shape created by ln -s at install.sh:1864.
* Studio: close residual workspace-isolation gaps for custom roots
Four follow-on hardenings that close the remaining cross-root leaks the
custom-root install plumbing still left open.
studio/setup.ps1 in-place git-sync: when the source-build path finds an
existing $LlamaCppDir/.git, it ran git remote set-url, checkout -B, and
clean -fdx in place before any ownership check. The previous fix marked
the tree as Studio-owned AFTER the sync but did not guard the BEFORE
case, so an unrelated workspace .git could be silently rewritten on the
first source-build under a custom UNSLOTH_STUDIO_HOME. Add the same
Assert-StudioOwnedOrAbsent guard already used by the prebuilt path and
the temp-dir swap path (gated on $StudioHomeIsCustom for parity).
Launcher port-file workspace isolation: the env-mode launchers' fast
path attached to any backend listening on the cached port that returned
a healthy /api/health, even when that backend belonged to a different
install root. studio/backend/main.py /api/health now returns the
resolved studio_root; install.sh _check_health and install.ps1
Test-StudioHealth verify it against UNSLOTH_STUDIO_HOME when set, so a
stale studio.port pointing at a sibling Studio is rejected instead of
opening the wrong UI.
studio/src-tauri preflight + commands: the Tauri desktop app stays on
the legacy root by design. process.rs / install.rs / desktop_auth.rs /
update.rs already strip UNSLOTH_STUDIO_HOME and STUDIO_HOME from their
CLI subprocesses, but preflight.rs run_cli_probe / probe_cli_capability
and commands.rs check_install_status did not, so a desktop launch from
a shell carrying those env vars produced status reflecting a different
root than the desktop manages. Mirror the existing scrub.
install.sh shim install: the previous `rm -f -- $_shim_path; ln -s ...`
pair leaves a window with no shim if interrupted. Use ln -sfn for an
atomic replace; the -n flag prevents descent into a symlink-to-directory
target (the existing directory guard above already rejects a real dir).
* Studio: replace launcher root verify with hex digest baked at install time
The previous launcher identity check returned the absolute resolved Studio
install root from /api/health and matched it against $UNSLOTH_STUDIO_HOME
in the launcher. Three problems that this commit closes:
- POSIX launcher used a raw bash `case` against the JSON-encoded value, so
paths containing characters that JSON escapes (e.g. /tmp/back\slash,
/tmp/O"Brien) caused the launcher to reject its own healthy backend.
- /api/health is unauthenticated and Studio supports `-H 0.0.0.0`, so any
reachable client could read the absolute install path (username, home
dir, workspace name, CI checkout path).
- The verification was gated on $UNSLOTH_STUDIO_HOME being set at runtime,
so a default-mode launcher would attach to a sibling env-mode Studio
listening on the same port instead of starting its own.
The fix replaces the raw path with a SHA-256 hex digest computed at install
time and baked into the generated launcher (mirroring how @@DATA_DIR@@ is
substituted today):
studio/backend/main.py: /api/health now returns `studio_root_id =
sha256(str(_studio_root()))` instead of the raw `studio_root` path.
install.sh: computes `_css_studio_root_id` once from $STUDIO_HOME using
python3, bakes `_EXPECTED_STUDIO_ROOT_ID='@@STUDIO_ROOT_ID@@'` into the
launcher heredoc, and adds `s|@@STUDIO_ROOT_ID@@|...|g` to the existing
sed pipeline for ALL modes (env / home / default). _check_health verifies
the baked id substring-matches the JSON response. Hex-only so no shell or
sed escape corner cases.
install.ps1: same shape on Windows. SHA256 the $StudioHome bytes, lower
hex, bake `$_ExpectedStudioRootId = '...'` into the launcher heredoc.
Test-StudioHealth now compares `$resp.studio_root_id -eq
$_ExpectedStudioRootId` unconditionally (no special-case for env-mode).
Default-mode launchers also bake their expected id, so two coexisting
Studio installs on the same machine can no longer cross-attach.
* Studio: harden launcher root-id and split install-time mode from runtime env
- install.sh launcher: compute studio_root_id with the venv Python (uv-managed
systems may not have system python3) and canonicalize STUDIO_HOME with
cd -P/pwd -P so default and home-redirect modes match the backend's
Path(sys.prefix).resolve() canonicalization. Fail fast instead of silently
baking an empty discriminator.
- install.sh launcher heredoc: gate PORT_FILE / namespaced LOCK_DIR on a baked
install-time mode flag (@@INSTALLED_IS_ENV_MODE@@) instead of the runtime
UNSLOTH_STUDIO_HOME variable so a sourced custom-root studio.conf cannot flip
a default-mode launcher into env-mode behavior with stale state.
- studio/backend/main.py: cache the studio_root_id digest at module load so
/api/health does not recompute hashlib + filesystem probes on every poll.
- studio/backend/core/inference/llama_cpp.py: widen the studio_root() probe
except clause from ImportError to (ImportError, OSError, ValueError) so it
matches the sibling _kill_orphaned_servers handler and tolerates Path.resolve
failures from broken symlinks or odd codecs.
* Studio: align launcher root-id digest with backend canonicalization
- studio/backend/main.py: hash the already-resolved _STUDIO_ROOT_RESOLVED
instead of recomputing str(_studio_root()); the default fallback in
storage_roots returns Path.home()/.unsloth/studio without .resolve(), so
on systems where $HOME is a symlink (NFS / AFS / Docker) the cached
digest now matches install.sh's cd -P/pwd -P canonicalization and the
launcher no longer rejects its own healthy backend.
- install.ps1: canonicalize $StudioHome via Resolve-Path before the SHA256
compute (env-mode already resolves at line 121, only default and profile
branches were raw); a junctioned USERPROFILE now produces the same digest
the backend computes via Path.resolve() for the same install.
- install.sh launcher template: substitute the non-user-controlled
@@STUDIO_ROOT_ID@@ and @@INSTALLED_IS_ENV_MODE@@ placeholders before the
user-controlled @@DATA_DIR@@ pass so a $DATA_DIR that contains the
literal placeholder text cannot be mutated by the second sed.
* Studio: tighten installer rationale comments
* Studio install: extend workspace-guard test coverage
Add behavioral coverage for env-mode workspace guards across install.sh,
install.ps1, studio/setup.sh, studio/setup.ps1, the launcher root-id
discriminator, and the backend's /api/health response. Also refresh the
custom-mode llama.cpp resilience assertion so it matches the implementation
that intentionally excludes the legacy tree from search_roots.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Honor STUDIO_HOME alias, fix workspace-guard test harness, harden rollback
The PR title and description promise STUDIO_HOME as a priority-2 alias
to UNSLOTH_STUDIO_HOME, but the implementation only read the longer name
in all six resolution sites. Wire the alias through install.sh,
install.ps1, studio/setup.sh, studio/setup.ps1, the Python storage_roots
resolver, and the unsloth_cli studio resolver. UNSLOTH_STUDIO_HOME wins
when both are set (more specific signal beats the generic alias).
Whitespace-only values are now treated as unset to match the Python
resolvers' .strip() semantics, preventing install/runtime layout drift
where the installer would create a literal " " directory while the
backend fell through to the legacy default.
Error messages and the substep status line report the env-var name the
user actually set ("UNSLOTH_STUDIO_HOME=..." vs "STUDIO_HOME=...") so
diagnostics stay accurate under either spelling.
Test harness fix: tests/test_studio_install_workspace_guard.py extracted
the install.sh venv-replacement block, but after the merge that block
delegates to _start_studio_venv_replacement (defined further up in
install.sh, not in the extracted snippet). Five sentinel-positive tests
echoed RESULT=ok but never moved $VENV_DIR. Add a single
_INSTALL_GUARD_STUBS constant that stands in a minimal mv-based stub
plus a no-op substep, and route every inline test script through a new
_build_install_guard_script() helper. All 50 tests now pass (was 45/50).
Rollback hardening: Start-StudioVenvRollback / Restore-StudioVenvRollback
/ Complete-StudioVenvRollback in install.ps1 used plain Test-Path,
Move-Item, Remove-Item against paths derived from $StudioHome. With a
custom UNSLOTH_STUDIO_HOME containing brackets (the very motivation for
the broader -LiteralPath sweep this PR set out to do), rollback would
silently misbehave under wildcard interpretation, turning a recoverable
install error into a destroyed env. Same fix for the --local Tauri
overlay block (Test-Path / Copy-Item / Get-FileHash on $VenvDir-derived
paths).
* Replace studio_root_id path-hash with per-install opaque id
The previous design computed studio_root_id as sha256 of the resolved
$STUDIO_HOME path, both at install time (baked into the launcher) and
at backend startup (returned via /api/health). This worked but had
three weaknesses:
1. Information disclosure on -H 0.0.0.0: anyone reaching /api/health
could confirm a guessed install path (username, workspace name,
etc.) by replaying the same hash.
2. Canonicalization brittleness: launcher (cd -P/pwd -P) and backend
(Path.resolve()) had to produce identical strings, which required
careful symlink/junction handling on every site (cycles 17-27 of
the PR review history were entirely about closing this drift).
3. Stale-launcher attach: an uninstall + reinstall at the same path
produced the same hash, so a launcher from the previous install
would silently attach to the new (incompatible) backend.
Replace the path-hash with a per-install opaque id:
- install.sh and install.ps1 generate 32 bytes from the platform CSPRNG
(/dev/urandom on POSIX with a python3 secrets fallback;
RandomNumberGenerator.Create().GetBytes on Windows) and persist it to
$STUDIO_HOME/share/studio_install_id with mode 0600. Atomic
temp-file-rename so a crash mid-install can't leave a half-written id.
The check 'if [ ! -s "$_css_id_file" ]' / Test-Path makes generation
idempotent across re-runs (so re-running install.sh doesn't invalidate
previously-baked launchers in the same install root).
- studio/backend/main.py replaces hashlib.sha256 with
_read_studio_install_id(), which reads $STUDIO_HOME/share/studio_install_id
once at module load. Validates the content against ^[0-9a-f]{64}$ so
malformed/truncated/uppercase/wrong-length content returns "" and
triggers the launcher's existing "no baked id, accept any healthy
Unsloth backend" fallback path.
- /api/health field name (studio_root_id) and wire format (64 hex chars)
preserved for compatibility with launchers already shipped via earlier
PR iterations.
Tests:
- Drop test_install_sh_root_id_matches_backend_resolved_under_symlinked_home
and test_install_ps1_canonicalizes_studio_home_before_root_id_hash --
the entire reason these existed (cd -P/Resolve-Path/Path.resolve()
digest agreement under symlinks/junctions) is moot when the id comes
from a file rather than from the path.
- Drop test_main_py_studio_root_id_hashes_resolved_root_not_unresolved
(no more hashing).
- Rewrite test_main_py_studio_root_id_caches_at_module_load to assert
the file-read pattern; add test_main_py_read_studio_install_id_validates_hex_and_handles_missing
to pin the exact rejection rules (empty / non-hex / wrong case /
wrong length all -> "").
- Rewrite test_install_sh_create_shortcuts_uses_venv_python_first as
test_install_sh_create_shortcuts_seeds_id_from_csprng_with_python_fallback
with a behavioral subprocess check that re-invocation is idempotent.
- Rename test_check_health_handles_path_with_backslash_via_hash to
test_check_health_handles_arbitrary_id_token (the JSON-escape concern
it pinned is preserved -- ids are hex-only by construction -- but the
test no longer derives the id from a path).
- Add test_install_sh_install_id_survives_symlinked_studio_home as a
regression test pinning that the new design has zero canonicalization
drift across symlinked parents.
- Update test_install_sh_bakes_studio_root_id_into_launcher and
test_install_ps1_bakes_studio_root_id_into_launcher to assert the
CSPRNG seed and the file location.
49/49 tests pass. Behavioral verification: install.sh-style generation
is idempotent across runs, three parallel installs at different roots
get distinct ids, reinstall at the same path produces a new id (so
stale launchers correctly fail to attach to the new backend), and
symlinked-\$HOME no longer causes launcher/backend disagreement.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <unslothai@gmail.com>
* Update VRAM estimator to cater to broader model configs
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix attn backend check, better support for MoE etc
* Studio: tighten VRAM estimator structured-shape and attention paths
- Conservative attention fallback: when resolve_attention_implementation
fails, charge the quadratic non-flash activation path instead of
silently keeping the optimistic flash_attention_2 default.
- Resolve attention on a shallow config copy so _set_attn_impl does not
mutate the cached config returned by _load_config_for_gpu_estimate.
- Use getattr for AutoModelForCausalLM._model_mapping to avoid raising
on private-attribute renames in transformers.
- Treat sdpa as O(n) linear attention; PyTorch SDPA dispatches to flash
or memory-efficient backends, only eager needs the quadratic term.
- Per-layer activation accounting: structured archs (head_dim,
layer_types, attention_k_eq_v, num_kv_shared_layers, double-wide MLP)
now flow into compute_activation_bytes via _text_linear_dims, instead
of using the legacy hidden_size//num_attention_heads KV/MLP shape.
- Exclude MLA configs (q_lora_rank set) from the structured-shape path
so q_lora low-rank projection formulas keep applying when head_dim is
also present.
- _build_text_module_elements emits a single MLA self_attn aggregate
using _compute_attn_elements when q_lora_rank is set, avoiding the
~10% overcount that fed into _compute_skipped_quantizable_elements.
- Restrict _module_path_matches to known text-tower prefixes so VLM
skip names like vision_tower.model.layers.<i>.self_attn.q_proj no
longer falsely shadow the text alias model.layers.<i>.self_attn.q_proj.
- Pick up enable_moe_block from the config and add the per-layer dense
MLP alongside the MoE experts in compute_total_params and
compute_lora_params (Gemma4-style parallel dense + MoE block).
- Single-pass structured layer accounting in _compute_layer_elements,
removing the duplicate _text_linear_dims walks.
- Drop the now-zero (activations - activations_computed) shard term in
VramBreakdown.min_gpu_vram and the stale comment that referred to it.
- attention_implementation typed as Optional[str] to match call sites
that pass None.
- Inline rationale comments on DOUBLE_QUANT_4BIT_FACTOR and
NON_FLASH_ATTENTION_FACTOR pointing at VRAM_ESTIMATION.md.
* Studio: extend parallel-MoE accounting + non-prefix dense layer support
- Apply enable_moe_block / moe_has_dense_mlp symmetrically: activation
per-layer MLP size in _layer_qkv_mlp_sizes now adds the parallel dense
MLP for MoE layers, matching the weight and LoRA accounting added in
the prior commit. Skip-quantizable mapping in _build_text_module_elements
now registers both mlp.experts and per-projection mlp.{name} entries
for MoE layers when the parallel dense block is present, so an
llm_int8_skip_modules entry like "model.layers.N.mlp" covers both.
- Track dense layer indices as a tuple (dense_layer_indices) extracted
from first_k_dense_replace or decoder_sparse_step + mlp_only_layers,
and dispatch dense-vs-MoE accounting through _is_dense_mlp_layer. The
prior count-based path silently mis-bucketed layers when mlp_only_layers
was non-prefix (e.g. [3, 5] on an 8-layer model). num_dense_layers is
derived from len(dense_layer_indices) for backward compatibility.
- Drop the redundant ">0" check in _is_kv_shared_layer so configs with
num_kv_shared_layers == num_hidden_layers (every layer shared) are
correctly recognized as shared.
- Refresh VRAM_ESTIMATION.md section 5 to note that sdpa joins
flash_attention_2 in the linear activation path; refresh the
VramBreakdown.activations_computed comment now that the activation
floor is gone.
* Studio: Gemma4 PLE accounting, flex_attention, KV-share guard restore
- Add flex_attention to LINEAR_ATTENTION_IMPLS. Unsloth's
resolve_attention_implementation returns "flex_attention" when
HAS_FLASH_ATTENTION is False and the model class supports flex; PyTorch
FlexAttention is a memory-efficient kernel, not a quadratic eager
attention path. Without this, activation estimates over-charge ~36x.
- Restore the `> 0` guard in _is_kv_shared_layer. Transformers Gemma4
(modeling_gemma4.py:1031, modular_gemma4.py:863, :926) uses
`layer_idx >= first_kv_shared_layer_idx > 0`, so configs that mark
every layer as KV-shared raise on construction. Reverting the
unconditional acceptance avoids producing a detailed estimate for a
shape the actual model code rejects.
- Extend the parallel dense MLP path (`enable_moe_block`) in
_build_text_module_elements: when the arch is non-structured, use
arch.intermediate_size for the dense gate/up/down dims instead of
_text_linear_dims (which returns moe_intermediate_size via
_get_mlp_size). Prior code under-counted skipped quantizable elements
for the parallel dense block by up to 8x on GLM-style configs.
- Add Gemma4 per-layer-input (PLE) module accounting:
per_layer_model_projection (one global Linear) plus per-layer
per_layer_input_gate and per_layer_projection are added to the
quantizable text-linear total in _compute_layer_elements;
post_per_layer_input_norm and per_layer_projection_norm flow into
the non-quantizable bucket. compute_lora_params adds the same three
Linear modules to the all-linear total. References:
transformers_versions/5.7.0/.../gemma4/modular_gemma4.py:1077-1083,
:1247-1253.
- VRAM_ESTIMATION.md section 5 now lists flex_attention alongside sdpa
and flash_attention_2 as linear-memory backends.
* Studio: shared-expert variants, mlp_layer_types dispatch, PLE skip, all-linear str, deepcopy resolver
Five targeted estimator corrections:
- _compute_dense_layer_indices now reads `mlp_layer_types` ahead of
`first_k_dense_replace` / `decoder_sparse_step`. Transformers Exaone-MoE,
Laguna, Hy_v3, GLM-MoE-DSA, GLM4-MoE-Lite, Ernie4_5_VL_MoE etc. ship the
per-position list and may omit the prefix-style fields entirely.
- _build_text_module_elements registers per_layer_input_gate /
per_layer_projection (per layer) and per_layer_model_projection (global)
in the canonical element map and alias map. The PLE element count was
added to total_quantizable in a prior commit but skip-module matching
against names like model.layers.0.per_layer_input_gate produced 0-byte
delta. Layer aggregate text.layers.<i> now sums all layer modules so
prefix skip names cover the PLE pieces too.
- _targets_all_linear coerces a bare string `"all-linear"` to `["all-linear"]`
before set comparison; the previous set comprehension iterated chars.
PEFT LoraConfig.target_modules accepts the bare-string convention.
- ModelArchConfig gains `shared_expert_intermediate_size`. extract_arch_config
reads `n_shared_experts` / `num_shared_experts` aliases and infers
`n_shared_experts=1` when only `shared_expert_intermediate_size` is set.
_compute_moe_mlp_elements and the structured + non-structured LoRA paths
size the shared expert with its own intermediate (Qwen3.5-MoE: 512 vs
routed moe_intermediate_size).
- _determine_attention_impl_for_gpu_estimate uses copy.deepcopy so the
resolver does not mutate nested text_config on the cached source.
PreTrainedConfig._attn_implementation setter walks `sub_configs` and the
prior shallow copy still touched the inner objects.
* Studio: extend MoE/PLE/KV-share accounting to activation and skip-alias paths
Five activation-path corrections plus two LoRA / skip-alias corrections so
that shared-expert, per-layer-input, and KV-shared-layer support is symmetric
across weights, LoRA, skip-quantizable, and activation paths.
- _layer_qkv_mlp_sizes: include shared-expert FFN in mlp_size (live shared
expert per token alongside routed experts) and keep K/V activation memory
for KV-shared layers; only the WEIGHT path uses has_k/has_v from
_layer_attention_dims.
- _per_layer_activation_bytes / compute_activation_bytes: account for
per_layer_input_gate (hd-sized) and per_layer_projection (pli-sized) per
layer plus the global per_layer_model_projection [B,S,L,PLI] tensor when
hidden_size_per_layer_input is set.
- _build_text_module_elements: split mlp.experts into routed and
mlp.shared_expert canonical entries; register layers.<i>.experts alias for
Gemma4 enable_moe_block layouts and mlp.shared_experts (plural) alias for
Exaone-MoE / Laguna / GLM4-MoE-Lite shared-expert variants.
- _compute_moe_mlp_elements: split into _compute_routed_moe_elements and
_compute_shared_moe_elements; only count shared_expert_gate (hd->1 Linear
per shared expert) when shared_expert_intermediate_size is set, which is
the Qwen2-MoE / Qwen3.5-MoE discriminator. Other shared-expert families
(Exaone-MoE, HY-V3, GLM4-MoE-Lite, Laguna) lack the gate.
- compute_lora_params: when target_modules='all-linear' bare keyword, drop
routed and shared MoE expert LoRA contributions. PEFT's all-linear targets
nn.Linear only; Unsloth's get_moe_target_parameters expands MoE expert
nn.Parameter LoRA only when target_modules contains explicit
gate_proj/up_proj/down_proj/gate_up_proj names.
- _per_layer_input_lora_params: thread target_modules through and add the
per-PLE-module contribution when the corresponding name appears, not only
under all-linear.
* Studio: top-k MoE activations, ERNIE list configs, suffix skips, multimodal full bytes
Six estimator corrections aligning the detailed accounting paths with real
training behavior:
- _layer_qkv_mlp_sizes scales the MoE-layer mlp_size by num_experts_per_tok
so the active routed-expert intermediate tensors are charged for activations.
Adds num_experts_per_tok to ModelArchConfig and extracts it from
num_experts_per_tok / top_k_experts (Gemma4 alias) in extract_arch_config.
- compute_lora_params splits routed and shared MoE LoRA contributions so that
bare target_modules='all-linear' zeroes routed (nn.Parameter expert tensors,
which Unsloth's get_moe_target_parameters does NOT enable for the bare
keyword) but keeps shared-expert LoRA (regular nn.Linear MLPs that
Unsloth's get_peft_regex DOES match).
- extract_arch_config gains a _first_scalar helper for ERNIE-style
moe_intermediate_size = [routed, shared] lists, plus moe_num_experts and
moe_num_shared_experts attribute aliases. When moe_intermediate_size is a
pair and shared_expert_intermediate_size is unset, the second element is
treated as the shared-expert intermediate.
- estimate_required_model_memory_gb's detailed branch retains
max(0, model_size_bytes - compute_total_params(arch) * 2) on top of the
arch-derived breakdown.model_weights so multimodal models (vision/audio
towers) and partially-modeled families (Gemma3n AltUp/Laurel etc.) do not
silently drop bytes that the safetensors total includes.
- _module_path_matches accepts a tail-only match when the skip entry is
shorter than the alias path. Transformers' BNB quantizer suffix-matches
short skip entries like ['q_proj'] / ['lm_head'] against full module
paths; the previous len(skip) < len(alias) early-return missed those.
- _per_layer_input_lora_params drops the all_linear branch and only counts
PLE LoRA when the user explicitly names per_layer_input_gate /
per_layer_projection / per_layer_model_projection. Unsloth's
get_peft_regex requires module names to contain a component tag
(mlp/attn/...); PLE module names lack any tag, so all-linear training
does not attach LoRA to them.
* Studio: full-FT extra optimizer/gradient inflation, MoE top-k aliases, ERNIE position dispatch, sibling experts aggregate
When the safetensors total exceeds the text-arch fp16 estimate (multimodal
vision/audio towers, partially-modeled families), only inflate the model
weights line for adapter methods but extend optimizer + gradient bytes
under full fine-tuning, where the extra params are trainable.
DBRX exposes top-k routing as moe_top_k and Hunyuan-V1-MoE as moe_topk;
neither is aliased to num_experts_per_tok via attribute_map, so probe both
when extracting arch config.
ERNIE 4.5 MoE / VL MoE configs declare MoE layers via
moe_layer_start_index / moe_layer_end_index / moe_layer_interval (with -1
meaning the last layer); add the position-style dispatch alongside the
existing mlp_layer_types / first_k_dense_replace / decoder_sparse_step
paths.
When moe_has_dense_mlp is set (Gemma4 enable_moe_block) the routed experts
live as a sibling of self.mlp at layers.<i>.experts in the actual model
layout; keep the layer mlp aggregate to the dense path and add a separate
experts aggregate so a skip module model.layers.<i>.mlp does not collapse
the routed experts as well.
* Studio: extend MoE family extraction (Llama4 / DBRX / Hunyuan / ERNIE) and align dense vs routed MLP widths
- Llama4: pick up `config.moe_layers` (auto-populated from
interleave_moe_layer_step) so dense layer indices reflect the actual
is_moe_layer dispatch.
- Llama4: add a separate `dense_intermediate_size` derived from
`intermediate_size_mlp` (used for the dense feed_forward path) and keep
`intermediate_size` for the routed/shared expert width. Auto-attach one
shared expert per MoE layer when the dense-vs-MoE width split is present.
- DBRX: walk the `ffn_config` sub-config when extracting MoE attrs
(moe_num_experts / moe_top_k / ffn_hidden_size). Without this DBRX is
misclassified as a dense arch.
- Hunyuan: normalize layer-wise `moe_topk` (and the canonical
`num_experts_per_tok` lookup it shadows via attribute_map) through a
worst-case scalar so the int(...) cast cannot crash on list values.
- ERNIE 4.5 MoE: switch the start/end/interval dispatch to the model's
`(layer_idx + 1) % interval == 0` modulo gate so MoE layers match the
decoder when interval > 1.
- ERNIE 4.5 VL MoE: drop the heuristic that read
`moe_intermediate_size[1]` as the shared expert width; in VL configs [1]
is the vision-routed width and shared experts are sized from [0].
- estimate_fp16_model_size_bytes: prefer the larger of config-derived and
local-weight bytes so the multimodal extra_bytes correction can fire
for local VLM directories.
* Add tests for VRAM estimator extensions
* Studio: trim verbose comments in VRAM estimator
Collapse multi-paragraph rationale blocks to 1-3 lines stating the single
load-bearing fact. Fix one inverted "fall through ... last" comment whose
claim disagreed with the surrounding code.
* Consolidate added tests into existing test_vram_estimation.py and test_gpu_selection.py
Move Llama4 / DBRX / ERNIE arch-extraction tests into test_vram_estimation.py
as TestLlama4ArchExtraction / TestDbrxFfnConfigExtraction /
TestErniePhaseModuloDispatch / TestErnieVlSharedExpertWidth classes. Move
estimate_fp16_model_size_bytes prefer-larger-of-config-or-local tests into
test_gpu_selection.py as TestEstimateFp16ModelSizeBytesPrefersLocalWeights.
Drop one redundant Llama4 num_dense_layers assertion already covered by the
moe_layers dispatch test.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Roland Tannous <115670425+rolandtannous@users.noreply.github.com>
* feat(studio): add Tauri native GGUF intake
* feat(studio): polish native GGUF intake
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio): load backend helpers during local setup
* fix(studio): acquire native load lease before unload
* Studio: harden native path lease verification and Tauri intake
- Wrap path.resolve(strict=True) and Path.stat() in NativePathLeaseError so a deleted or unmounted GGUF returns 400 instead of leaking the full filesystem path through the generic load_model/validate_model handler.
- Re-apply _reject_network_or_device_path to the resolved canonical path for defense in depth after symlink resolution.
- Replace try/except ValueError pattern in the device-path guard with Path.is_relative_to; the previous shape silently swallowed NativePathLeaseError (which subclasses ValueError) so /dev,/proc,/sys were never actually rejected.
- Broaden the lease redaction regex and dict-key check (Python and Rust diagnostics) to cover both native_path_lease and nativePathLease so the camelCase form emitted by Tauri/frontend payloads is also redacted.
- Hoist the redact_native_paths import to module top in loggers/handlers; the recursive filter no longer pays a per-record import lookup.
- Persist activeNativePathToken in the chat runtime store so the rollback branch can mint a fresh lease and reload the previous native GGUF when a new load fails after unload; clear it in clearCheckpoint and overwrite it on each successful load.
- use-native-drop: read options through a ref so the Tauri onDragDropEvent listener is registered once and stays attached across option changes; reject ambiguous multi-file drops up front instead of silently registering only the first GGUF.
- pick_native_model: use an async pick_file with a tokio oneshot channel instead of blocking_pick_file so the Tokio worker is not held for the duration of the OS dialog.
- registerNativeModelPath: drop the duplicate sourceKind argument; the Rust command parameter is source_kind.
- install_python_stack: insert the script directory (studio/) on sys.path; the previous insert pointed at studio/backend/ which does not satisfy `from backend.utils.wheel_utils import ...`.
* install_python_stack: keep _BACKEND_DIR on sys.path
Restore the studio/backend insertion. Although the immediately following `from backend.utils.wheel_utils import (...)` is satisfied by studio/ already being on sys.path[0] when invoked as `python studio/install_python_stack.py`, wheel_utils itself runs `from utils.native_path_leases import ...`, which requires studio/backend/ to be importable. Without the backend insertion, the existing tests/python/test_install_python_stack.py collection fails with ModuleNotFoundError: No module named 'utils'.
* Studio: tighten native path lease lifecycle and Tauri intake IPC
- register_native_model_path now hardcodes NativePathSourceKind::Drop on the Rust side and the frontend stops sending source_kind. The previous JS payload (source_kind only) never reached the Rust deserializer because Tauri's default ArgumentCase::Camel maps the Rust parameter source_kind to the JS key sourceKind, so drag/drop registration silently failed. Hardcoding the source kind also keeps audit metadata trustworthy on this command.
- Add native_path_secret_removed_for_child_start context manager and wrap multiprocessing.Process.start() at the inference, export, training, and data-recipe job spawn sites. The previous wrapper-only scrub left UNSLOTH_STUDIO_NATIVE_PATH_LEASE_SECRET visible to spawn-platform import-time worker code. The wrapper run_without_native_path_secret stays as defense-in-depth inside the child.
- Stop passing exc_info=True from the native-grant load/validate error logs in routes/inference.py. The structlog filter_sensitive_data processor runs before the renderer, so ConsoleRenderer formatted tracebacks bypassed redaction; the redacted str(e) preserves the message text.
- Replace the os.path.normcase string equality on the resolved canonical path with Path.samefile (with a normcase fallback) so Windows leases that differ only in extended-length \\?\ prefix or short-name spelling are accepted.
- Wrap consumeNativePathToken in its own try/catch in the chat runtime rollback. If the previous native-model token has aged out of TOKEN_TTL we now surface a clear modelsError instead of silently swallowing the rollback inside the outer catch.
- Reject non-ASCII lease strings in _split_lease and convert UnicodeEncodeError / binascii.Error / ValueError raised by _b64decode into NativePathLeaseError so verify_native_path_lease never escapes raw exceptions to the route handler.
- Tighten dropStateForPaths to mark multi-file payloads invalid so the overlay matches the post-fix drop handler that rejects the same payload.
- Replace the one-shot fetch in useNativePathLeasesSupported with a delayed-retry loop so the picker/drop becomes available once the backend is up rather than staying disabled for the rest of the session after a transient failure.
- Drop the unused setActiveNativePathToken setter; the value is set via setState directly in use-chat-model-runtime.
- Add a toast on auto-load failure in use-native-drop so a collapsed model selector does not hide the error.
- Burn the lease nonce before _validate_current_stat so a stat-failed lease is single-use even if a later state change happens to match the original size/mtime.
* Studio: cache lease secret, harden native path stat checks, polish intake UX
- Cache the decoded UNSLOTH_STUDIO_NATIVE_PATH_LEASE_SECRET on first verify and validate that it is base64-decodable and at least 32 bytes. Subsequent _decode_secret calls return from the cache and never touch os.environ, so concurrent /api/inference/load and /api/health requests no longer race with native_path_secret_removed_for_child_start scrubbing the env. native_path_leases_supported now wraps _decode_secret so the health flag matches what verify_native_path_lease actually accepts.
- Replace path.is_file()/is_dir() + path.stat() with os.lstat() in _validate_current_stat and explicitly reject S_ISLNK; size and mtime checks now refer to the link itself, closing the same-size+same-mtime symlink-swap window that the prior follow-symlink stat() left open.
- Add an issued_at_ms < expires_at_ms sanity check in _validate_payload to reject internally inconsistent (HMAC-protected) lease payloads.
- Sort _NATIVE_PATH_REDACTIONS by length (descending) before iterating in redact_native_paths so a longer registered path is replaced before a shorter prefix path; otherwise logs containing /foo/X.gguf.bak after only /foo/X.gguf was registered would leak the .bak suffix.
- classify_existing_path now re-checks the canonical path with symlink_metadata after canonicalize, so a regular file that is replaced with a symlink in the small canonicalize window is rejected at registration.
- ModelSelector renders the local file picker as its own block (not in the eject ternary), so a user with an active model can still replace it via the picker rather than only via drag/drop.
- useNativePathLeasesSupported caps the readiness probe at MAX_READINESS_POLLS (60 = ~5 minutes) and aborts the in-flight fetch on unmount via AbortController, so a permanently-disabled backend stops generating sustained traffic and hot-reload no longer leaks open connections.
- useChooseNativeModel returns a stable useCallback closure and guards the OS dialog with a useRef so rapid double-clicks cannot open multiple dialogs and orphan Rust tokens.
- Branch the multi-file drop toast: if no GGUF was present we say "Only .gguf model files can be dropped here." and otherwise "Drop a single .gguf model file." so users dropping non-GGUF attachments get an accurate explanation.
* native_path_leases: lstat the signed canonical path before resolving
The earlier change to lstat inside _validate_current_stat operates on grant.canonical_path, which is the post-resolve target. If the user atomically replaces the originally-signed file with a symlink to a different file of identical size and mtime, path.resolve(strict=True) follows the symlink, samefile returns True (both ends share the new inode), and the lstat in _validate_current_stat sees the regular target file rather than the symlink, so the swap goes undetected.
Add an os.lstat on the signed canonical path before path.resolve(strict=True), and reject S_ISLNK there. The lstat in _validate_current_stat stays as defense-in-depth for swaps that occur strictly between resolve and stat.
* Studio: scrub native lease secret before mp.Queue spawn and tighten lease lifecycle
- Move _CTX.Queue / _CTX.Event / _CTX.Process construction inside native_path_secret_removed_for_child_start at the inference, export, training and data-recipe spawn sites. The first Queue creation lazily spawns Python's multiprocessing.resource_tracker child, so when it ran outside the scrub context the tracker process inherited the lease secret. Reproduced via the proc filesystem environ entry; the wrapped order keeps the tracker clean.
- native_path_secret_removed_for_child_start now refcounts entries: the env var is popped on the first entry and restored only when the last context exits. Concurrent training/inference/export starts no longer serialize on the env lock across the entire proc.start yield, while still guaranteeing the env stays empty for the duration of every overlapping spawn.
- run_without_native_path_secret now also nulls the module-level cached lease secret. With the existing spawn-only multiprocessing context the cache is irrelevant in practice, but a future fork caller would otherwise inherit the in-memory secret even though the env var was scrubbed.
- filter_sensitive_data now applies the native lease key check on the top-level event_dict, not only on nested dicts, so a logger call that includes a lease value as a top-level keyword field actually redacts it (the bare value does not match the prefix-anchored regex).
- chat-page loadNativeModelIntent now passes intent.id to clearModelIntent so a second drag-drop during an in-flight first auto-load is not wiped from the chip area when the first resolves.
- Bump useNativePathLeasesSupported's MAX_READINESS_POLLS from 60 to 720 so first-run installs that compile llama.cpp from source or download large CUDA wheels (well past 5 minutes) don't permanently disable the native picker.
* native_path_leases: serialize first-decode against scrub context
_decode_secret used a separate _SECRET_INIT_LOCK from the env scrub's _NATIVE_PATH_ENV_LOCK, so the very first decode (before the cache is populated) could race a concurrent native_path_secret_removed_for_child_start and read os.environ during the env-empty window, raising "Native path grants require the managed desktop backend." Subsequent calls hit the cache and were already safe.
Acquire _NATIVE_PATH_ENV_LOCK around the env read inside _SECRET_INIT_LOCK and fall back to _SCRUB_SAVED_SECRET when the scrub has temporarily popped the env var. Lock ordering (init then env) is consistent with no other caller, so no deadlock.
* Studio: surface native model load errors and harden native path label cache
- Native model load and validate now bubble up the actual exception (with
paths redacted) and apply the same friendly-error rewrite the non-native
path uses, so users see "CUDA OOM", "trust_remote_code required", etc.
instead of a generic "Failed to load native model: <label>".
- run_without_native_path_secret now also nulls _SCRUB_SAVED_SECRET so a
forked grandchild that imports native_path_leases cannot recover the
secret via the scrub-aware fallback in _decode_secret.
- _NATIVE_PATH_LABELS now has its own 10000-entry cap independent of the
100-entry redaction list, so display_label_for_native_path no longer
falls back to returning the raw canonical path after 101 native paths
in one session. Redaction list keeps the 100-entry cap for log-scan
performance.
- _validate_payload now also rejects null bytes in display_label, which
is echoed back in HTTP responses and log lines.
* Studio: harden native path lease validation and chained native rollback
- child_env_without_native_path_secret now copies os.environ under
_NATIVE_PATH_ENV_LOCK so a concurrent scrub-context env pop cannot
raise RuntimeError: dictionary changed size during iteration in a
background hardware scan or other env reader.
- _validate_payload and grant construction route every signed numeric
field (version, issued_at_ms, expires_at_ms, size_bytes, modified_ms)
through new _required_int / _optional_int helpers that wrap raw int()
ValueError into NativePathLeaseError. The single upstream catcher
produces 400 instead of 500 for malformed signed payloads.
- verify_native_path_lease now runs _validate_current_stat before
_consume_nonce, so a transient stat error on the canonical path no
longer permanently burns the nonce. Concurrent verifies still
serialize through _consume_nonce, so single-use is preserved.
- Chained native model rollback now restores activeNativePathToken in
the chat runtime store after a successful rollback loadModel. Without
this, a second consecutive failed switch could not re-roll-back
because the store token had been overwritten by the failed attempt.
- validate_model now applies the same not_supported_hints friendly
rewrite to native model errors that load_model already does, so a
native .gguf that fails validation with an upstream "is not supported"
message gets the same actionable wording as the non-native branch.
* Studio: harden native path log redaction, status disclosure, and chip lifecycle
- structlog processor chain now runs format_exc_info before
filter_sensitive_data so traceback strings are produced (and then
redacted) rather than passed through as untouched (type, value, tb)
tuples that the JSON or console renderer formats after the redaction
filter has already finished.
- native_path_secret_removed_for_child_start clears _CACHED_LEASE_SECRET
in addition to popping the env var, so a fork during the scrub window
cannot inherit the cached bytes via the parent's heap. Parent verify
calls during the window keep working through the existing scrub-aware
fallback in _decode_secret.
- load_model's except ValueError handler now redacts native paths and
uses the native model log label when native_grant_backed is true.
Previously a ValueError raised after lease verification (e.g. from
ModelConfig.from_identifier or downstream GGUF parsing) returned the
raw exception string in the HTTP response body.
- llama_cpp_backend now records the native display label at GGUF load
time, and /api/inference/status prefers it over the redaction store.
After a Python backend restart the redaction store is empty; the
attribute keeps the friendly label, and an absolute model_identifier
with no other label source falls back to the basename so the canonical
path no longer appears in active_model.
- reveal_path_token uses native "reveal and select" commands on macOS
(open -R) and Windows (explorer /select,) so the file is highlighted
in the file manager. Linux keeps the existing parent-directory open.
- Native model rollback that fails because the previous token cannot be
consumed now throws a rollback-specific Error, and the outer empty
catch was replaced with one that re-throws the rollback error. The
rollback-specific message now reaches the user instead of being
overwritten by the original load error message.
- NativeModelChip tracks the Rust token's expiresAtMs on a single
setTimeout, disables the Load button at expiry, and relabels it
"Select again" with an explanatory tooltip so users do not click into
a guaranteed-failure path after the 15-minute TTL elapses.
* Studio: tighten native artifact policy, mmproj sibling check, and intake UX
- is_open_safe_artifact no longer grants Open for directories. Reveal
already handles directory navigation, so the change closes the
attack surface where a macOS .app artifact could be launched via
open_path_token + open::that_detached.
- Display labels are sanitized in classify_existing_path. Control
characters in filenames (newlines, tabs, NUL et al.) are replaced
with spaces and the label is trimmed and capped, so a file named
with embedded newlines cannot inject forged log lines or scramble
the UI status panel.
- validate_entry_path skips the size_bytes/modified_ms equality check
when the operation is Reveal or Open. Cloud-sync agents (Dropbox,
iCloud Drive, OneDrive) routinely rewrite extended-attribute
metadata which bumps mtime, and the user expects Reveal/Open to
remain available for files in synced folders.
- llama_cpp_backend gains a _native_grant_backed flag at GGUF load
success. /api/inference/status only applies the absolute-path
basename fallback when that flag is true, so a non-native absolute
local GGUF still reports its canonical model_identifier and unload
by identifier keeps working.
- Native vision GGUFs now run through _validate_native_mmproj_companion
before llama-server starts: the companion mmproj must be a regular
file, not a symlink, and must live in the same resolved directory as
the granted GGUF. This stops a hostile sibling or symlinked mmproj
from being loaded under a single-file lease.
- Chained native rollback restructured: the rollback loadModel + state
+ refresh runs inside its own try/catch that swallows so the outer
throw error surfaces the ORIGINAL load failure. The native-token
consume-failure case still throws the rollback-specific message
early, before the inner block runs, so its actionable guidance is
preserved.
- Loading-model state and the duplicate-load guard in the chat runtime
hook now compare both the model id and the native path token. Two
drops or picks with the same basename in different folders no longer
silently dedup; the second token is honored.
- chat-page loadNativeModelIntent awaits selectModel before clearing
the pending intent. If selectModel returns early via dedup or
throws, the chip and its token stay so the user can retry instead
of losing the selection.
- NativeModelChip's Reveal button is disabled when the lease has
expired (Rust would reject it anyway), and the Load button label
reads "Expired" instead of "Select again" so the disabled element
no longer promises an action it cannot perform.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* feat: enable deleting fine-tuned chat models
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: tighten fine-tuned model delete guards
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: harden fine-tuned model deletion edge cases
* fix: reject gguf export delete without variant; surface 503 on backend probe failure
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: narrow loaded-model delete guard by gguf_variant
* fix: clear inference state when cancelling model load
* fix: verify deletion completed and prune empty parent dirs
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Roland Tannous <rolandtannous@gravityq.ai>
* qwen3.6 unsloth studio support
* Add qwen3.6 causal-conv1d detection
* Update model_mappings.py
moved qwen3.6-27B to thinking train on completion template
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* add unsloth studio desktop app
* Fix review findings
- studio/src-tauri/tauri.conf.json: retarget updater to staging repo
(danielhanchen/unsloth-staging-2); switch to unslothai/unsloth on upstream merge.
- studio/src-tauri/linux/postremove.sh: drop the interactive read loop and the
/home/* iteration. Package maintainer scripts must stay non-interactive and
must not touch other users' data.
- studio/frontend/src/app/auth-guards.ts: honor tauriAutoAuth() boolean. Failed
auto-auth now redirects to /login; requireGuest/requirePasswordChangeFlow
only redirect to /chat when auth succeeds. The new early-return on failed
auth is intentional so the login / change-password flows remain reachable
when desktop auth is not yet established.
- studio/frontend/src/config/env.ts: keep fetched=false on health failure so
later calls retry instead of caching the client-side platform guess.
- studio/src-tauri/src/install.rs: pick the available system package manager
(apt-get, dnf, zypper, pacman); AppImage bundles run on non-Debian distros.
- studio/frontend/src/lib/open-link.ts + markdown-text/sources callers: return
boolean from openLink so callers only preventDefault on handled URLs; relative
hrefs now navigate natively.
- studio/frontend/src/features/settings/tabs/about-tab.tsx: fetch(apiUrl(...))
so the version request targets the backend port in desktop mode. The bare
/api/health predates the Tauri webview (blame: the earlier onboarding commit,
which ran with same-origin frontend/backend); in desktop mode the webview
origin is tauri://localhost so the bare path fails.
- install.ps1: gate the install_python_stack.py hotfix on a sentinel comment
instead of a content regex; append the sentinel after applying so reruns
are unambiguous.
- unsloth_cli/commands/studio.py _write_auth_secret: use the atomic mkstemp +
os.replace path on Windows too; chmod calls are wrapped in try/except OSError.
- studio/src-tauri/src/preflight.rs probe_existing_backends: fan out the health
probes concurrently; desktop-auth status still runs sequentially per candidate.
reqwest::Client is internally Arc-wrapped so the in-loop .clone() is a
refcount bump, not a deep clone; annotated inline.
- studio/src-tauri/src/preflight.rs run_cli_probe: wait() after kill() to reap
the child, matching probe_cli_capability.
- studio/src-tauri/src/process.rs + main.rs: add stop_backend_detached and use
it from the tray quit handler so the 5s graceful-wait does not block the
Tauri main loop. RunEvent::Exit keeps the synchronous safety-net call.
- studio/backend/main.py: drop the permissive localhost CORS regex in
api-only mode; the explicit allow_origins list is sufficient.
- .github/workflows/release-desktop.yml: drop max-parallel: 1 so platform
builds run in parallel, and lift releaseBody to an env var so the three
tauri-action invocations share one source of truth.
* Fix review findings (loop 2)
- studio/backend/auth/storage.py update_password: clear_desktop_secret()
alongside clear_bootstrap_password() so rotating the admin password
also revokes any previously provisioned .desktop_secret. Without this,
an old local desktop credential keeps minting fresh admin tokens via
/api/auth/desktop-login after a password rotation.
- studio/src-tauri/src/desktop_auth.rs provision_desktop_auth: wrap
cmd.output().await in tokio::time::timeout(30s). DESKTOP_AUTH_LOCK is
held across the whole desktop_auth flow, and previously a hanging
`unsloth studio provision-desktop-auth` subprocess would pin the lock
indefinitely and freeze every subsequent desktop_auth call.
* Add review tests
* Consolidate review tests
Merge review-added tests into the existing studio/backend/tests/test_desktop_auth.py
(the PR's authoritative desktop-auth test file). Drops three scaffolding files under
tests/python/ in favor of five focused tests next to the tests they extend:
- test_update_password_clears_desktop_secret (runtime)
- test_update_password_on_unknown_user_leaves_desktop_secret_intact (runtime)
- test_cli_provisioning_delegates_to_storage_create_desktop_secret (source-level)
- test_cli_connect_auth_db_reads_storage_db_path (source-level)
- test_desktop_auth_provision_has_bounded_timeout (Rust source-level)
* Revert auth-guards.ts Tauri branches to unconditional form
The review loop on PR 5144 introduced a regression: the isTauri branch of
requireAuth redirected to /login when tauriAutoAuth() returned false, and
requireGuest / requirePasswordChangeFlow silently fell through on the same
condition. The Tauri desktop app authenticates via a local auto-generated
secret; it must never surface /login or /change-password to the user. A
failed auto-auth should let the startup layer retry, not expose a password
form.
Restore the three Tauri branches to the author's original unconditional
form (requireAuth: return; requireGuest / requirePasswordChangeFlow: throw
redirect({to: '/chat'})). Keep the rest of the review fixes -- the
apiUrl() fetch wrapping, authRedirect helper, and fetchAuthStatus refactor
are all legitimate improvements and are preserved.
* Revert release-desktop.yml to author's version
The review loop's workflow-file tweaks (drop max-parallel: 1, lift releaseBody
to an env var) are cosmetic. OAuth tokens cannot push workflow-file changes,
and fine-grained PATs cannot honor maintainerCanModify on a third-party fork.
Reverting the workflow file to wasimysaid's version lets the push go through
without needing a classic PAT with both repo and workflow scopes.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Daniel Han <unslothai@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: Ollama support, recommended folders, Custom Folders UX polish
Backend:
- Add _scan_ollama_dir that reads manifests/registry.ollama.ai/library/*
and creates .gguf symlinks under <ollama_dir>/.studio_links/ pointing
at the content-addressable blobs, so detect_gguf_model and llama-server
-m work unchanged for Ollama models
- Filter entries under .studio_links from the generic models/hf/lmstudio
scanners to avoid duplicate rows and leaked internal paths in the UI
- New GET /api/models/recommended-folders endpoint returning LM Studio
and Ollama model directories that currently exist on the machine
(OLLAMA_MODELS env var + standard paths, ~/.lmstudio/models, legacy
LM Studio cache), used by the Custom Folders quick-add chips
- detect_gguf_model now uses os.path.abspath instead of Path.resolve so
the readable symlink name is preserved as display_name (e.g.
qwen2.5-0.5b-Q4_K_M.gguf instead of sha256-abc...)
- llama-server failure with a path under .studio_links or .cache/ollama
surfaces a friendlier message ("Some Ollama models do not work with
llama.cpp. Try a different model, or use this model directly through
Ollama instead.") instead of the generic validation error
Frontend:
- ListLabel supports an optional leading icon and collapse toggle; used
for Downloaded (download icon), Custom Folders (folder icon), and
Recommended (star icon)
- Custom Folders header gets folder icon on the left, and +, search,
and chevron buttons on the right; chevron uses ml-auto so it aligns
with the Downloaded and Recommended chevrons
- New recommended folder chips render below the registered scan folders
when there are unregistered well-known paths; one click adds them as
a scan folder
- Custom folder rows that are direct .gguf files (Ollama symlinks) load
immediately via onSelect instead of opening the GGUF variant expander
(which is for repos containing multiple quants, not single files)
- When loading a direct .gguf file path, send max_seq_length = 0 so the
backend uses the model's native context instead of the 4096 chat
default (qwen2.5:0.5b now loads at 32768 instead of 4096)
- New listRecommendedFolders() helper on the chat API
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: log silent exceptions and support read-only Ollama dirs
Replace silent except blocks in _scan_ollama_dir and the
recommended-folders endpoint with narrower exception types plus debug
or warning logs, so failures are diagnosable without hiding signal.
Add _ollama_links_dir helper that falls back to a per-ollama-dir hashed
namespace under Studio's own cache (~/.unsloth/studio/cache/ollama_links)
when the Ollama models directory is read-only. Common for system installs
at /usr/share/ollama/.ollama/models and /var/lib/ollama/.ollama/models
where the Studio process has read but not write access. Previously the
scanner returned an empty list in that case and Ollama models would
silently not appear.
The fallback preserves the .gguf suffix on symlink names so
detect_gguf_model keeps recognising them. The prior "raw sha256 blob
path" fallback would have missed the suffix check and failed to load.
* Address review: detect mmproj next to symlink target for vision GGUFs
Codex P1 on model_config.py:1012: when detect_gguf_model returns the
symlink path (to preserve readable display names), detect_mmproj_file
searched the symlink's parent directory instead of the target's. For
vision GGUFs surfaced via Ollama's .studio_links/ -- where the weight
file is symlinked but any mmproj sidecar lives next to the real blob
-- mmproj was no longer detected, so the model was misclassified as
text-only and llama-server would start without --mmproj.
detect_mmproj_file now adds the resolved target's parent to the scan
order when path is a symlink. Direct (non-symlink) .gguf paths are
unchanged, so LM Studio and HF cache layouts keep working exactly as
before. Verified with a fake layout reproducing the bug plus a
regression check on a non-symlink LM Studio model.
* Address review: support all Ollama namespaces and vision projector layers
- Iterate over all directories under registry.ollama.ai/ instead of
hardcoding the "library" namespace. Custom namespaces like
"mradermacher/llama3" now get scanned and include the namespace
prefix in display names, model IDs, and symlink names to avoid
collisions.
- Create companion -mmproj.gguf symlinks for Ollama vision models
that have an "application/vnd.ollama.image.projector" layer, so
detect_mmproj_file can find the projector alongside the model.
- Extract symlink creation into _make_symlink helper to reduce
duplication between model and projector paths.
* Address review: move imports to top level and add scan limit
- Move hashlib and json imports to the top of the file (PEP 8).
- Remove inline `import json as _json` and `import hashlib` from
function bodies, use the top-level imports directly.
- Add `limit` parameter to `_scan_ollama_dir()` with early exit
when the threshold is reached.
- Pass `_MAX_MODELS_PER_FOLDER` into the scanner so it stops
traversing once enough models are found.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: Windows fallback, all registry hosts, collision safety
_make_link (formerly _make_symlink):
- Falls back to os.link() hardlink when symlink_to() fails (Windows
without Developer Mode), then to shutil.copy2 as last resort
- Uses atomic os.replace via tmp file to avoid race window where the
.gguf path is missing during rescan
Scanner now handles all Ollama registry layouts:
- Uses rglob over manifests/ instead of hardcoding registry.ollama.ai
- Discovers hf.co/org/repo:tag and any other host, not just library/
- Filenames include a stable sha1 hash of the manifest path to prevent
collisions between models that normalize to the same stem
Per-model subdirectories under .studio_links/:
- Each model's links live in their own hash-keyed subdirectory
- detect_mmproj_file only sees the projector for that specific model,
not siblings from other Ollama models
Friendly Ollama error detection:
- Now also matches ollama_links/ (the read-only fallback cache path)
and model_identifier starting with "ollama/"
Recommended folders:
- Added os.access(R_OK | X_OK) check so unreadable system directories
like /var/lib/ollama/.ollama/models are not advertised as chips
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: filter ollama_links from generic scanners
The generic scanners (models_dir, hf_cache, lmstudio) already filter
out .studio_links to avoid duplicate Ollama entries, but missed the
ollama_links fallback cache directory used for read-only Ollama
installs. Add it to the filter.
* Address review: idempotent link creation and path-component filter
_make_link:
- Skip recreation when a valid link/copy already exists (samefile or
matching size check). Prevents blocking the model-list API with
multi-GB copies on repeated scans.
- Use uuid4 instead of os.getpid() for tmp file names to avoid race
conditions from concurrent scans.
- Log cleanup errors instead of silently swallowing them.
Path filter:
- Use os.sep-bounded checks instead of bare substring match to avoid
false positives on paths like "my.studio_links.backup/model.gguf".
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: drop copy fallback, targeted glob, robust path filter
_make_link:
- Drop shutil.copy2 fallback -- copying multi-GB GGUFs inside a sync
API request would block the backend. Log a warning and skip the
model when both symlink and hardlink fail.
Scanner:
- Replace rglob("*") with targeted glob patterns (*/*/* and */*/*/*)
to avoid traversing unrelated subdirectories in large custom folders.
Path filter:
- Use Path.parts membership check instead of os.sep substring matching
for robustness across platforms.
Scan limit:
- Skip _scan_ollama_dir when _generic already fills the per-folder cap.
* Address review: sha256, top-level uuid import, Path.absolute()
- Switch hashlib.sha1 to hashlib.sha256 for path hashing consistency.
- Move uuid import to the top of the file instead of inside _make_link.
- Replace os.path.abspath with Path.absolute() in detect_gguf_model
to match the pathlib style used throughout the codebase.
* Address review: fix stale comments (sha1, rglob, copy fallback)
Update three docstrings/comments that still referenced the old
implementation after recent changes:
- sha1 comment now says "not a security boundary" (no hash name)
- "rglob" -> "targeted glob patterns"
- "file copies as a last resort" -> removed (copy fallback was dropped)
* Address review: fix stale links, support all manifest depths, scope error
_make_link:
- Drop size-based idempotency shortcut that kept stale links after
ollama pull updates a tag to a same-sized blob. Only samefile()
is used now -- if the link doesn't point at the exact same inode,
it gets replaced.
Scanner:
- Revert targeted glob back to rglob so deeper OCI-style repo names
(5+ path segments) are not silently skipped.
Ollama error:
- Only show "Some Ollama models do not work with llama.cpp" when the
server output contains GGUF compatibility hints (key not found,
unknown architecture, failed to load). Unrelated failures like
OOM or missing binaries now show the generic error instead of
being misdiagnosed.
---------
Co-authored-by: Daniel Han <info@unsloth.ai>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
* Studio: add folder browser modal for Custom Folders
The Custom Folders row in the model picker currently only accepts a
typed path. On a remote-served Studio (Colab, shared workstation) that
means the user has to guess or paste the exact server-side absolute
path. A native browser folder picker can't solve this: HTML
`<input type="file" webkitdirectory>` hides the absolute path for
security, and the File System Access API (Chrome/Edge only) returns
handles rather than strings, neither of which the server can act on.
This PR adds a small in-app directory browser that lists paths on the
server and hands the chosen string back to the existing
`POST /api/models/scan-folders` flow.
## Backend
* New endpoint `GET /api/models/browse-folders`:
* `path` query param (expands `~`, accepts relative or absolute; empty
defaults to the user's home directory).
* `show_hidden` boolean to include dotfiles/dotdirs.
* Returns `{current, parent, entries[], suggestions[]}`. `parent` is
null at the filesystem root.
* Immediate subdirectories only (no recursion); files are never
returned.
* `entries[].has_models` is a cheap hint: the directory looks like it
holds models if it is named `models--*` (HF hub cache layout) or
one of the first 64 children is a .gguf/.safetensors/config.json/
adapter_config.json or another `models--*` subfolder.
* Sort order: model-bearing dirs, then plain, then hidden; case-
insensitive alphabetical within each bucket.
* Suggestions auto-populate from HOME, the HF cache root, and any
already-registered scan folders, deduplicated.
* Error surface: 404 for missing path, 400 for non-directory, 403 on
permission errors. Auth-required like the other models routes.
* New Pydantic schemas `BrowseEntry` and `BrowseFoldersResponse` in
`studio/backend/models/models.py`.
## Frontend
* New `FolderBrowser` component
(`studio/frontend/src/components/assistant-ui/model-selector/folder-browser.tsx`)
using the existing `Dialog` primitive. Features:
* Clickable breadcrumb with a `..` row for parent navigation.
* Quick-pick chips for the server-provided suggestions.
* `Show hidden` checkbox.
* In-flight fetch cancellation via AbortController so rapid
navigation doesn't flash stale results.
* Badges model-bearing directories inline.
* `chat-api.ts` gains `browseFolders(path?, showHidden?)` and matching
types.
* `pickers.tsx` adds a folder-magnifier icon next to the existing `Add`
button. Opening the browser seeds it with whatever the user has
already typed; confirming fills the text input, leaving the existing
validation and save flow unchanged.
## What it does NOT change
* The existing text-input flow still works; the browser is additive.
* No new permissions or escalation; the endpoint reads only directories
the server process is already allowed to read.
* No model scanning or filesystem mutation happens from the browser
itself -- it just returns basenames for render.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: cap folder-browser entries and expose truncated flag
Pointing the folder browser at a huge directory (``/usr/lib``,
``/proc``, or a synthetic tree with thousands of subfolders) previously
walked the whole listing and stat-probed every child via
``_looks_like_model_dir``. That is both a DoS shape for the server
process and a large-payload surprise for the client.
Introduce a hard cap of 2000 subdirectory entries and a
``truncated: bool`` field on the response. The frontend renders a small
hint below the list when it fires, prompting the user to narrow the
path. Below-cap directories are unchanged.
Verified end-to-end against the live backend with a synthetic tree of
2050 directories: response lands at 2000 entries, ``truncated=true``,
listing finishes in sub-second time (versus tens of seconds if we were
stat-storming).
* Studio: suggest LM Studio / Ollama dirs + 2-level model probe
Three improvements to the folder-browser, driven by actually dropping
an LM Studio-style install (publisher/model/weights.gguf) into the
sandbox and walking the UX:
## 1. Quick-pick chips for other local-LLM tools
`well_known_model_dirs()` (new) returns paths commonly used by
adjacent tools. Only paths that exist are returned so the UI never
shows dead chips.
* LM Studio current + legacy roots + user-configured
`downloadsFolder` from its `settings.json` (reuses the existing
`lmstudio_model_dirs()` helper).
* Ollama: `$OLLAMA_MODELS` env override, then `~/.ollama/models`,
`/usr/share/ollama/.ollama/models`, and `/var/lib/ollama/.ollama/models`
(the systemd-service install path surfaced in the upstream "where is
everything?" issue).
* Generic user-choice locations: `~/models`, `~/Models`.
Dedup is stable across all sources.
## 2. Two-level model-bearing probe
LM Studio and Ollama both use `root/publisher/model/weights.gguf`.
The previous `has_models` heuristic only probed one level, so the
publisher dir (whose immediate children are model dirs, not weight
files) was always marked as non-model-bearing. Pulled the direct-
signal logic into `_has_direct_model_signal` and added a grandchild
probe so the classic layout is now recognised.
Still O(PROBE^2) worst-case, still returns immediately for
`models--*` names (HF cache layout) and for any direct weight file.
## 3. model_files_here hint on response body
A leaf model dir (just GGUFs, no subdirs) previously rendered as
`(empty directory)` in the modal, confusing users into thinking the
folder wasn't scannable. Added a `model_files_here` count on the
response (capped at 200) and a small hint row in the modal: `N model
files in this folder. Click "Use this folder" to scan it.`
## Verification
Simulated an LM Studio install by downloading the real 84 MB
`unsloth/SmolLM2-135M-Instruct-Q2_K.gguf` into
`~/.lmstudio/models/unsloth/SmolLM2-135M-Instruct-GGUF/`. Confirmed
end-to-end:
* Home listing suggests `~/.lmstudio/models` as a chip.
* Browsing `~/.lmstudio/models` flags `unsloth` (publisher) as
`has_models=true` via the 2-level probe.
* Browsing the publisher flags `SmolLM2-135M-Instruct-GGUF` (model
dir) as `has_models=true`.
* Browsing the model dir returns empty entries but
`model_files_here=1`, and the frontend renders a hint telling the
user it is a valid target.
* Studio: one-click scan-folder add + prominent remove + plain search icon
Three small Custom Folders UX fixes after real-use walkthrough:
* **One-click add from the folder browser**. Confirming `Use this
folder` now submits the path directly to
`POST /api/models/scan-folders` instead of just populating the text
input. `handleAddFolder` takes an optional explicit path so the
submit lands in the same tick as `setFolderInput`, avoiding a
state-flush race. The typed-path + `Add` button flow is unchanged.
* **Prominent remove X on scan folders**. The per-folder delete
button was `text-muted-foreground/40` and hidden entirely on
desktop until hovered (`md:opacity-0 md:group-hover:opacity-100`).
Dropped the hover-only cloak, bumped color to `text-foreground/70`,
added a red hover/focus background, and sized the icon up from
`size-2.5` to `size-3`. Always visible on every viewport.
* **Plain search icon for the Browse button**. `FolderSearchIcon`
replaced with `Search01Icon` so it reads as a simple "find a
folder" action alongside the existing `Add01Icon`.
* Studio: align Custom Folders + and X buttons on the same right edge
The Custom Folders header used `px-2.5` with a `p-0.5` icon button,
while each folder row used `px-3` with a `p-1` button. That put the
X icon 4px further from the right edge than the +. Normalised both
rows to `px-2.5` with `p-1` so the two icons share a column.
* Studio: empty-state button opens the folder browser directly
The first-run empty state for Custom Folders was a text link reading
"+ Add a folder to scan for local models" whose click toggled the
text input. That's the wrong default: a user hitting the empty state
usually doesn't know what absolute path to type, which is exactly
what the folder browser is for.
* Reword to "Browse for a models folder" with a search-icon
affordance so the label matches what the click does.
* Click opens the folder browser modal directly. The typed-path +
Add button flow is still available via the + icon in the
section header, so users who know their path keep that option.
* Slightly bump the muted foreground opacity (70 -> hover:foreground)
so the button reads as a primary empty-state action rather than a
throwaway hint.
* Studio: Custom Folders header gets a dedicated search + add button pair
The Custom Folders section header had a single toggle button that
flipped between + and X. That put the folder-browser entry point
behind the separate empty-state link. Cleaner layout: two buttons in
the header, search first, then add.
* Search icon (left) opens the folder browser modal directly.
* Plus icon (right) toggles the text-path input (unchanged).
* The first-run empty-state link is removed -- the two header icons
cover both flows on every state.
Both buttons share the same padding / icon size so they line up with
each other and with the per-folder remove X.
* Studio: sandbox folder browser + bound caps + UX recoveries
PR review fixes for the Custom Folders folder browser. Closes the
high-severity CodeQL path-traversal alert and addresses the codex /
gemini P2 findings.
Backend (studio/backend/routes/models.py):
* New _build_browse_allowlist + _is_path_inside_allowlist sandbox.
browse_folders now refuses any target that doesn't resolve under
HOME, HF cache, Studio dirs, registered scan folders, or the
well-known third-party model dirs. realpath() is used so symlink
traversal cannot escape the sandbox. Also gates the parent crumb
so the up-row hides instead of 403'ing.
* _BROWSE_ENTRY_CAP now bounds *visited* iterdir entries, not
*appended* entries. Dirs full of files (or hidden subdirs when
show_hidden is False) used to defeat the cap.
* _count_model_files gets the same visited-count fix.
* PermissionError no longer swallowed silently inside the
enumeration / counter loops -- now logged at debug.
Frontend (folder-browser.tsx, pickers.tsx, chat-api.ts):
* splitBreadcrumb stops mangling literal backslashes inside POSIX
filenames; only Windows-style absolute paths trigger separator
normalization. The Windows drive crumb value is now C:/ (drive
root) instead of C: (drive-relative CWD-on-C).
* browseFolders accepts and forwards an AbortSignal so cancelled
navigations actually cancel the in-flight backend enumeration.
* On initial-path fetch error, FolderBrowser now falls back to HOME
instead of leaving the modal as an empty dead end.
* When the auto-add path (one-click "Use this folder") fails, the
failure now surfaces via toast in addition to the inline
paragraph (which is hidden when the typed-input panel is closed).
* Studio: rebuild browse target from trusted root for CodeQL clean dataflow
CodeQL's py/path-injection rule kept flagging the post-validation
filesystem operations because the sandbox check lived inside a
helper function (_is_path_inside_allowlist) and CodeQL only does
intra-procedural taint tracking by default. The user-derived
``target`` was still flowing into ``target.exists`` /
``target.is_dir`` / ``target.iterdir``.
The fix: after resolving the user-supplied ``candidate_path``,
locate the matching trusted root from the allowlist and rebuild
``target`` by appending each individually-validated segment to
that trusted root. Each segment is rejected if it isn't a single
safe path component (no separators, no ``..``, no empty/dot).
The downstream filesystem ops now operate on a Path constructed
entirely from ``allowed_roots`` (trusted) plus those validated
segments, so CodeQL's dataflow no longer sees a tainted source.
Behavior is unchanged for all valid inputs -- only the
construction of ``target`` is restructured. Live + unit tests
all pass (58 selected, 7 deselected for Playwright env).
* Studio: walk browse paths from trusted roots for CodeQL
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Ubuntu <ubuntu@h100-8-cheapest.us-east5-a.c.unsloth.internal>
* Reapply "updated models template mappers. added lfm2.5vl450m to transformers 5…" (#4945)
This reverts commit 33503ea248.
* Add missing gemma-4-31B-it bnb-4bit mapper entry and LFM2.5 upstream namespace for PR #4950
- Add unsloth/gemma-4-31B-it-unsloth-bnb-4bit to __INT_TO_FLOAT_MAPPER so
the int-to-float resolution works for this model (already listed in
TEMPLATE_TO_MODEL_MAPPER but had no mapper entry).
- Add LiquidAI/LFM2.5-1.2B-Instruct to lfm-2.5 TEMPLATE_TO_MODEL_MAPPER
entry so the canonical upstream namespace is mapped consistently with lfm-2.
* Add missing gemma-4-31B-it bnb-4bit Ollama mapping and lfm-2.5 chat template alias
- Add unsloth/gemma-4-31B-it-unsloth-bnb-4bit to OLLAMA_TEMPLATE_TO_MODEL_MAPPER
so Ollama export works for this model (E2B-it and E4B-it bnb-4bit variants were
already present, 31B-it was inconsistently omitted)
- Register CHAT_TEMPLATES["lfm-2.5"] as alias of the lfm-2 template to prevent
KeyError when Studio resolves LFM2.5 models through MODEL_TO_TEMPLATE_MAPPER
* Add missing LFM2 bnb-4bit INT_TO_FLOAT_MAPPER entry
unsloth/LFM2-1.2B-unsloth-bnb-4bit is referenced in model_mappings.py
but had no mapper.py entry, so model resolution would fail when users
load that variant with load_in_4bit=False or when the float name is
used with load_in_4bit=True.
* Fix review findings for PR #16
1. ollama_template_mappers.py: Restore dropped Gemma-4 base model IDs
(E2B, E4B, 31B, 26B-A4B) and add missing google/ upstream IDs to
the gemma4 Ollama mapper for consistency with other gemma entries.
2. mapper.py: Remove self-mapping non-bnb-4bit entries from
__INT_TO_FLOAT_MAPPER that were polluting FLOAT_TO_INT_MAPPER with
lowercase 16-bit names, causing load_in_4bit=True to return bad
model names. Add direct MAP_TO_UNSLOTH_16bit entries to preserve
the google->unsloth 16-bit redirects.
3. mapper.py: Add LFM2.5 MAP_TO_UNSLOTH_16bit redirect so
LiquidAI/LFM2.5-1.2B-Instruct resolves to its unsloth mirror.
* Add review tests for PR #4950
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Remove top-level test files
These test_*.py files were added at the repo root rather than under tests/.
Removing them from this PR; the production mapper changes remain.
* Add gemma-4-26B-A4B-it mapping
Adds unsloth/gemma-4-26B-A4B-it to __INT_TO_FLOAT_MAPPER as a 2-tuple so
google/gemma-4-26B-A4B-it routes to unsloth/gemma-4-26B-A4B-it across
INT_TO_FLOAT_MAPPER, FLOAT_TO_INT_MAPPER, and MAP_TO_UNSLOTH_16bit.
The 26B-A4B (MoE) model has no bnb-4bit variant, so the key uses the
plain unsloth name rather than the -unsloth-bnb-4bit suffix.
Removes the now-redundant standalone _add_with_lower call for the -it
variant; the 16bit mapping is registered via the dict loop.
* Add unsloth-bnb-4bit mappings for gemma-4 base (non-it) models
Adds E2B, E4B, 31B base unsloth-bnb-4bit entries to __INT_TO_FLOAT_MAPPER.
The 26B-A4B (MoE) base has no bnb-4bit variant on HF, so it stays on the
standalone _add_with_lower line for the 16bit-only routing.
Removes the redundant _add_with_lower lines for E2B, E4B, 31B base since
the dict loop now registers the same google->unsloth route through the
2-tuple entries, plus full FLOAT_TO_INT and INT_TO_FLOAT coverage.
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: refresh Downloaded GGUF list and recurse into variant subdirs
Two fixes for the model picker's "Downloaded" section.
Frontend (`pickers.tsx`):
* `HubModelPicker`'s mount effect short-circuited the cached-gguf and
cached-models refetch whenever the module-level cache already had
entries (`if (alreadyCached) return;`). After downloading a new repo
in the same session, reopening the picker rendered the stale cache
and the new repo never appeared in "Downloaded" until a full page
reload. The early return is removed so the lists are always refreshed
on mount; the module cache still drives the initial render so there
is no spinner flash when we already had data.
Backend (`utils/models/model_config.py`):
* `list_local_gguf_variants` and `_find_local_gguf_by_variant` used a
non-recursive `Path.glob("*.gguf")`. Some HF GGUF repos (e.g.
`unsloth/gemma-4-26B-A4B-it-GGUF`) place the largest quants under a
variant-named subdirectory such as `BF16/...gguf`, which the
top-level glob missed. Both helpers now use `rglob` and the variant
filename is stored as a path relative to the scan root so the
locator can still find the file.
The flat-layout case (variants directly in the snapshot root) is
unchanged: verified against `unsloth/gemma-4-E2B-it-GGUF` which still
returns its UD-Q4_K_XL variant correctly.
* Studio: emit posix-style relative filenames for local GGUF subdirs
`list_local_gguf_variants` was doing `str(f.relative_to(p))`, which on
Windows produces backslash-separated paths like `BF16\foo.gguf`. The
remote `list_gguf_variants` (HF API path) always returns forward-slash
filenames such as `BF16/foo.gguf`, so the two would diverge on Windows.
Switch to `.as_posix()` so the local and remote variant filenames stay
identical across Linux, macOS, and Windows. Verified by simulating with
`PureWindowsPath` in the test suite.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: detect mmproj at snapshot root for nested-variant layouts
When _find_local_gguf_by_variant returns a weight file inside a
quant-named subdir (e.g. snapshot/BF16/foo.gguf), detect_mmproj_file
was scanning only the immediate parent and missing the mmproj file
sitting at the snapshot root. The model was then loaded without
--mmproj, silently breaking vision support for repos that ship
nested variants.
detect_mmproj_file now takes an optional search_root and walks up
from the weight file to that root, in order, so the mmproj at the
snapshot root is picked up. Sibling quant subdirs are not scanned,
so an unrelated variant's mmproj does not leak in.
Also apply the suggested micro-optimization on relative_to in
list_local_gguf_variants -- only build the posix path when storing
the first file for a quant.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix: support GGUF variant selection for non-suffixed repos
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: harden GGUF detection across cached models and picker flows
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* chore: use shared GGUF picker helper for search rows
* fix: avoid mixed cache duplication and preserve GGUF fallback detection
* fix: unify GGUF cache matching and merge picker hints
* fix: normalize local GGUF matching across picker and model config
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: robust cached-gguf classification + hint-aware click routing
- _repo_gguf_size_bytes: treat size_on_disk=None as 0 and dedupe fallback
by commit_hash so partial/interrupted downloads don't TypeError out of
sum() and wipe the entire cached list.
- list_cached_gguf / list_cached_models: narrow per-repo try/except so
one malformed repo no longer poisons the whole response.
- handleModelClick: route through isKnownGgufRepo instead of the
suffix-only isGgufRepo, so non-suffixed GGUF repos still open the
variant expander from every call site.
- Replace the modelIsGgufById/resultIsGgufById Maps with Sets of known
GGUF ids to stop conflating "no hint" with "known not-GGUF".
- Make HfModelResult.isGguf required (it is always set in makeMapModel).
- Add regression tests for the None size case, mixed-repo inclusion in
cached-gguf, and per-repo error isolation.
* fix: exclude mmproj from GGUF classification and case-normalize hint lookups
- _repo_gguf_size_bytes now filters mmproj vision-adapter files so
safetensors+mmproj.gguf repos stay on the cached-models path and
non-GGUF rows no longer show zero pickable variants. A vision-capable
GGUF repo (main weight + mmproj adapter) still classifies as GGUF and
reports the main weight size.
- modelGgufIds / resultGgufIds now key on lowercased ids and
isKnownGgufRepo lowercases its lookup, so store and HF-search ids
that differ only by casing still match the same GGUF hint.
- New regression tests: mmproj-only repo excluded from cached-gguf,
same repo included in cached-models, vision-capable repo still
classified as GGUF with correct size.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Roland Tannous <rolandtannous@gravityq.ai>
Co-authored-by: Roland Tannous <115670425+rolandtannous@users.noreply.github.com>
* [Studio] Install flash attn at setup time for linux
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* cleanup changes
Signed-off-by: Datta Nimmaturi <venkatadattasainimmaturi@gmail.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Test cases
* wheel_utils: narrow url_exists exceptions and log at debug level
---------
Signed-off-by: Datta Nimmaturi <venkatadattasainimmaturi@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Roland Tannous <115670425+rolandtannous@users.noreply.github.com>
Co-authored-by: Roland Tannous <rolandtannous@gravityq.ai>
* Show non exported models in chat UI
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Distinguish b/w LoRa and full fine tune saves. Cleanup
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Roland Tannous <115670425+rolandtannous@users.noreply.github.com>
* Add ROCm detection to install.sh and expand shell tests
Add AMD ROCm GPU detection to get_torch_index_url() in install.sh.
When nvidia-smi is not found, probe for ROCm via amd-smi, /opt/rocm
version file, hipconfig, dpkg-query, and rpm.
Includes validation guard for malformed _rocm_tag, Debian epoch prefix
stripping, ROCm 7.2+ cap to rocm7.1 index, bitsandbytes AMD install,
and status messaging. Shell tests expanded to 23 cases.
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Add ROCm torch reinstall support to install_python_stack.py
Add _detect_rocm_version() and _ensure_rocm_torch() to detect when a
Linux host has ROCm but the venv received CPU-only torch, and reinstall
with the correct ROCm wheels. Covers ROCm 6.0 through 7.1 with a
30-second timeout on the torch GPU probe subprocess.
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Add ROCm support to llama.cpp prebuilt installer
Add has_rocm field to HostInfo, extend detect_host() to probe for ROCm
via hipcc/amd-smi/rocm-smi/ROCM_PATH, and route ROCm hosts to upstream
prebuilts (Linux ROCm 7.2 prebuilt with source fallback, Windows HIP
prebuilt with CPU fallback). Add linux-rocm and windows-hip install
kinds to runtime_patterns_for_choice().
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Add IS_ROCM hardware flag and fix AMD error message
Add IS_ROCM flag to hardware.py detect_hardware() (set when
torch.version.hip is present, DeviceType stays CUDA). Export IS_ROCM
from __init__.py. Add "rocm" key to get_package_versions().
Replace "We do not support AMD" error in tokenizer_utils.py with a
helpful message pointing to ROCm installation docs.
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Add comprehensive ROCm support test suite (68 tests)
Add tests/studio/install/test_rocm_support.py covering all ROCm code
paths across install_llama_prebuilt.py, install_python_stack.py,
hardware.py, tokenizer_utils.py, and install.sh. All tests use mocks
and run without AMD hardware.
Covers: asset selection (11), runtime patterns (5), HostInfo (4),
ROCm version detection (9), torch reinstall (9), index mapping (8),
hardware flag (8), tokenizer message (2), install.sh structure (10),
and live regression (1).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Harden ROCm support: probe error handling, version cap, validation
Address review findings from 8 independent reviewers:
- Wrap _ensure_rocm_torch() torch probe in try/except for
TimeoutExpired and OSError so a hung or broken torch import does not
crash the installer (8/8 reviewers flagged this)
- Add torch>=2.4,<2.11.0 version cap to the ROCm reinstall path to
prevent installing unsupported torch 2.11.0 from the rocm7.1 index
- Use with-statement for file reads in _detect_rocm_version() to avoid
resource leaks
- Handle ROCM_PATH="" correctly (use `or "/opt/rocm"` instead of
default parameter to avoid relative path resolution)
- Strengthen shell validation guard from rocm[0-9] to rocm[1-9] to
reject rocm0.x tags that would produce nonexistent PyTorch index URLs
- Switch shell version cap from blocklist to allowlist (rocm6.*|rocm7.0*
|rocm7.1* pass through, everything else caps to rocm7.1) so future
ROCm 10+ does not fall through to a nonexistent index
- Add sorted() to _ROCM_TORCH_INDEX lookup for defensive ordering
- Fix test_probe_timeout_handled: replace zero-assertion test with
proper assertions verifying reinstall proceeds after timeout
* Clean up rocm_paths list construction in detect_host()
Filter None from the ROCM_PATH env var lookup at list construction time
instead of relying on the inline `if p` guard in the any() call.
* Require actual AMD GPU presence before selecting ROCm paths
All 8 reviewers across 2 cycles independently flagged that ROCm
detection used toolkit/filesystem hints (hipcc, /opt/rocm, rocm-core)
as a proxy for GPU presence, which would misroute CPU-only or NVIDIA
hosts that happen to have ROCm tools installed.
Now all 3 detection points (install.sh, install_python_stack.py,
install_llama_prebuilt.py) probe for an actual AMD GPU before
entering the ROCm path:
- install.sh: check rocminfo for gfx* GPU names, or amd-smi list
for device rows, before version detection
- install_python_stack.py: new _has_rocm_gpu() function probes
rocminfo and amd-smi list before _ensure_rocm_torch() proceeds
- install_llama_prebuilt.py: detect_host() probes rocminfo/amd-smi
list instead of just checking tool existence or directory paths
Also:
- Shell test mock amd-smi now handles "list" subcommand
- Python tests updated to mock _has_rocm_gpu where needed
- Added test_no_gpu_with_rocm_tools_skips to verify the new guard
- Test index lookups now use sorted() to match production code
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Harden hipconfig version parsing and torch probe compatibility
- Add parts[1].isdigit() check in hipconfig version parsing to handle
versions like "6.3-HIP" where the minor component has non-numeric
suffix (strip "-" prefix before int() conversion)
- Use getattr() in torch probe subprocess to safely handle old or
custom torch builds that may lack torch.version.hip/cuda attributes
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Strengthen AMD GPU detection and add NVIDIA precedence guard
- Change amd-smi list detection from any-non-empty-output to requiring
"gpu" marker in output, matching the shell-side NR>1 check. Prevents
false positives from header-only amd-smi list output.
- Add nvidia-smi check at the top of _ensure_rocm_torch() so mixed
AMD+NVIDIA hosts preserve NVIDIA precedence (matching install.sh and
install_llama_prebuilt.py behavior).
- Apply the same amd-smi marker fix to install_llama_prebuilt.py
detect_host() for consistency.
* Add Windows-specific ROCm/HIP detection in detect_host()
The previous detect_host() ROCm check used rocminfo and amd-smi list
which are Linux-only tools. On Windows, has_rocm would always be False,
making the Windows HIP prebuilt path at line 1794 unreachable.
Now detect_host() uses platform-specific detection:
- Linux: rocminfo (check for gfx GPU names) or amd-smi list
- Windows: hipinfo.exe, amd-smi, or amdhip64.dll on PATH
This allows Windows AMD users to get the HIP prebuilt binary instead
of silently falling through to the CPU prebuilt.
* Add AMD ROCm gaps: Mamba/SSM source builds, GPU monitoring, Windows messaging, RDNA expansion
- worker.py: Add HIP detection to causal-conv1d/mamba-ssm probe, check
for hipcc before ROCm source builds, improve status messages and error
reporting, add timeout and uv support for the source build fallback
- amd.py: New AMD GPU monitoring module via amd-smi metric --json,
mirroring nvidia.py structure (utilization, temperature, power, VRAM)
- hardware.py: Branch to amd.py when IS_ROCM is True for GPU utilization,
visible GPU queries, and physical GPU count
- install_python_stack.py: Detect AMD GPUs on Windows and warn that
ROCm-enabled PyTorch must be installed manually
- kernels/utils.py: Expand is_rdna() to cover RDNA2 (gfx1030-1032),
RDNA3 (gfx1102-1103), RDNA3.5 (gfx1150-1152) alongside existing entries
- tests: Add 32 new tests covering all changes (95/95 pass)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Harden ROCm detection, fix VRAM heuristic, and expand RDNA2 coverage
- Windows ROCm detection: validate actual GPU presence via hipinfo/amd-smi
output markers instead of just checking tool existence on PATH
- _ensure_rocm_torch: validate nvidia-smi actually reports a GPU before
giving NVIDIA precedence (fixes AMD-only hosts with stale NVIDIA tools)
- amd.py _parse_numeric: handle dict-shaped metric objects from newer
amd-smi versions ({"value": 10, "unit": "W"}) and strip MiB/GiB units
- amd.py VRAM heuristic: raise threshold from 100k to 10M to correctly
handle MI300X (192 GB = 196608 MB) and other high-VRAM GPUs
- amd.py visible GPU: use AMD-reported GPU IDs instead of enumerate index
so non-dense sets like CUDA_VISIBLE_DEVICES=1,3 report correctly
- install.sh: add ROCm <6.0 minimum version guard (no PyTorch wheels
exist for older versions); fix rocm7.1* glob to not match rocm7.10+
- is_rdna: add gfx1033-1036 for RDNA2 mobile GPUs (RX 6600M etc.)
- worker.py: increase ROCm source build timeout from 600s to 1800s;
fix success log message for ROCm source builds
- Tests: update mocks for _has_usable_nvidia_gpu, add RDNA2 target asserts
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add HIP_VISIBLE_DEVICES support, unit-aware VRAM parsing, Windows GPU validation
- hardware.py: check HIP_VISIBLE_DEVICES and ROCR_VISIBLE_DEVICES on ROCm
before falling back to CUDA_VISIBLE_DEVICES, so multi-GPU AMD setups with
HIP-specific env vars report the correct visible device set
- amd.py: add _parse_memory_mb() that reads "unit" from dict-shaped amd-smi
JSON (e.g. {"value": 192, "unit": "GiB"}) and converts to MB correctly;
fixes MI300X VRAM misreported as 0.19 GB instead of 192 GB
- install_python_stack.py: Windows AMD warning now validates actual GPU
presence via hipinfo/amd-smi output markers before printing
- install_llama_prebuilt.py: restore amdhip64.dll fallback for Windows HIP
detection after tool-based checks, so Windows HIP installs without CLI
tools on PATH are still detected
- hardware.py: fix IS_ROCM comment to accurately describe its role
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix HIP_VISIBLE_DEVICES empty-string handling in GPU visibility spec
Use explicit None checks instead of Python `or` operator when reading
HIP_VISIBLE_DEVICES / ROCR_VISIBLE_DEVICES, so that an empty string
("") is correctly honored as "no visible GPUs" rather than silently
falling through to CUDA_VISIBLE_DEVICES on mixed ROCm+CUDA systems.
* Fix IS_ROCM test assertion for multi-line formatting
* Cap torchvision/torchaudio versions, remove amdhip64.dll fallback, fix visible GPU count
- Cap torchvision<0.26.0 and torchaudio<2.11.0 alongside torch<2.11.0 in
both install.sh and install_python_stack.py to prevent resolver from
selecting incompatible companion packages from ROCm wheel index
- Remove amdhip64.dll fallback in Windows ROCm detection (DLL presence
without hipinfo/amd-smi is not proof of GPU existence)
- Fix get_visible_gpu_count() to use _get_parent_visible_gpu_spec() which
respects HIP_VISIBLE_DEVICES/ROCR_VISIBLE_DEVICES on ROCm hosts
* Attribute is_rdna() RDNA2/3/3.5/4 expansion to PR #4428
The is_rdna() expansion to cover RDNA2 (gfx1030-1036), RDNA3
(gfx1100-1103), RDNA3.5 (gfx1150-1152), and RDNA4 (gfx1200-1201)
architectures is based on the original work from PR #4428.
Co-authored-by: GoldenGrapeGentleman <yueyuan@amd.com>
Co-authored-by: billishyahao <bill.he@amd.com>
* Support AMD Radeon for studio (#4770)
Co-authored-by: Iswarya Alex <iswarya.alex@amd.com>
* Remove ROCm test files from main PR
Move test_rocm_support.py and shell test additions to a separate PR
to keep the main ROCm support PR focused on implementation changes.
* Fix installer and hardware detection issues for PR #4720
- Fix empty _tri_arg passed to uv pip install in Radeon path (causes
"Empty field is not allowed for PEP508" error)
- Fix Radeon fallback: use ROCm index instead of CPU-only when
repo.radeon.com is unreachable (TORCH_INDEX_URL already has ROCm)
- Use $TORCH_CONSTRAINT in fallback paths instead of hardcoded strings
- Fix _pick_radeon_wheel: relax suffix to match manylinux_2_28_x86_64
wheels (AMD Radeon repo does not use bare linux_x86_64 platform tag)
- Fix IS_ROCM export: use __getattr__ so callers always see the live
value after detect_hardware() runs
- Fix apply_gpu_ids: set HIP_VISIBLE_DEVICES and ROCR_VISIBLE_DEVICES
on ROCm so _get_parent_visible_gpu_spec picks up narrowed GPU set
- Fix _parse_memory_mb: distinguish GB (1000 MB) from GiB (1024 MiB)
- Add amd-smi version as a fallback in _detect_rocm_version
- Fix trailing whitespace and missing newline at EOF in install.sh
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix GPU detection false positives and add missing health groups
- Fix _has_rocm_gpu() false positive: require "GPU: <number>" data rows
from amd-smi list, not just header containing "gpu"
- Apply same fix in detect_host() in install_llama_prebuilt.py
- Add runtime_payload_health_groups for linux-rocm and windows-hip so
partial/corrupt ROCm/HIP prebuilt installs are properly detected
- Add bitsandbytes install to Radeon fallback paths (was only in the
success path, skipped when repo.radeon.com was unreachable)
- Keep DEVICE/CHAT_ONLY as direct imports in __init__.py (matching main)
and only use __getattr__ for IS_ROCM
* Fix _ensure_rocm_torch and Windows AMD warning false positives
- _ensure_rocm_torch: only skip when HIP is already present, not for
CUDA builds (which are unusable on AMD-only hosts). Fixes the case
where a venv has a stale CUDA wheel and the repair step is skipped.
- Windows AMD warning: use GPU data row check (same as Linux fix) to
avoid false positives from amd-smi list header-only output.
* Fix amd-smi GPU detection for GPU[N] output format
Older amd-smi versions output "GPU[0] : Card series: ..." instead of
"GPU: 0". The regex now matches both "GPU: <digit>" and "GPU[<digit>"
formats to detect actual GPU data rows.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Harden AMD GPU detection against false positives
- install.sh: replace weak amd-smi list check (awk 'NR>1 && NF') with
strict pattern matching GPU data rows (/^GPU[[:space:]]*[:\[]/)
- All files: reject rocminfo gfx000 (CPU HSA agent) by requiring
gfx[1-9] instead of gfx[0-9] in the rocminfo GPU probe
- Fixes false positives on hosts with ROCm tools but no AMD GPU
* Remove duplicate comment from pre-commit merge
* Refactor: deduplicate AMD detection, consolidate bitsandbytes, clean up imports
- Extract _has_amd_rocm_gpu() shell function to avoid duplicating the
rocminfo/amd-smi GPU detection logic in get_torch_index_url and
the Radeon auto-detect block
- Consolidate bitsandbytes install into a single case block after torch
install (was duplicated 4 times across Radeon success/fallback paths)
- Move math and re imports to top of amd.py (were inline in functions)
- Add _smi_query() helper in hardware.py to centralize IS_ROCM backend
selection for get_gpu_utilization and get_visible_gpu_utilization
Addresses Gemini code review suggestions.
* Fix VRAM parsing for string values and GB/GiB consistency
- Extract unit from string-valued VRAM fields (e.g. "192 GiB") so
_parse_memory_mb correctly applies the unit multiplier instead of
treating the value as bare MB
- Treat GB and GiB identically (both as binary x1024) since GPU tools
including amd-smi use binary units even when labeling them "GB"
- Fixes incorrect VRAM reporting on MI300-class cards (was showing
~0.19 GB instead of 192 GB for string-valued outputs)
* Add --no-cache to uv for ROCm HIP source builds
Avoid stale cache artifacts from partial HIP source builds when
uv is used for causal-conv1d/mamba-ssm compilation on ROCm.
The pip path already uses --no-cache-dir; this adds the uv equivalent
(--no-cache) only when is_hip is True.
* Fix critical: initialize _amd_gpu_radeon before case block
_amd_gpu_radeon was only set inside the */rocm*) case arm, so on
NVIDIA/CPU/macOS paths where TORCH_INDEX_URL does not contain "rocm",
the variable was unbound. With set -u (nounset) enabled, this crashes
the installer for every non-AMD user.
Move initialization to before the case block so it is always defined.
* Fix Windows AMD: route has_rocm hosts to HIP prebuilt path
resolve_release_asset_choice was selecting windows-cpu for all Windows
x86_64 hosts including those with has_rocm=True. Windows AMD users
should fall through to resolve_upstream_asset_choice which tries the
HIP prebuilt first. Add "not host.has_rocm" guard to the published
windows-cpu selection.
* Harden ROCm detection, Radeon wheel fallback, and HIP visibility
Addresses review findings from parallel reviewers on PR #4720:
- install.sh: add _has_usable_nvidia_gpu() helper requiring nvidia-smi -L
to actually list a GPU before treating the host as NVIDIA. Fixes the
stale-nvidia-smi-on-PATH regression where AMD-only hosts fell into the
CUDA branch.
- install.sh: fix hipconfig awk blocks to propagate a non-zero exit code
when the output is not a recognisable version string, so the ||-chain
continues to dpkg-query / rpm instead of terminating early.
- install.sh: fail-closed on Radeon wheel fallback. When torch,
torchvision or torchaudio is missing from the Radeon repo for the
active Python tag, fall back to the standard ROCm index instead of
silently mixing Radeon wheels with PyPI defaults. Quote all wheel
arguments individually so wheel filenames cannot be word-split or
glob-expanded.
- install_llama_prebuilt.py: detect_host() now requires nvidia-smi -L to
list a GPU before setting has_physical_nvidia. Routes AMD ROCm hosts
with a broken leftover nvidia-smi to the ROCm path instead of
misclassifying them as NVIDIA.
- install_llama_prebuilt.py: scan upstream assets for any rocm-<version>
prebuilt instead of hard-coding rocm-7.2, so ROCm 6.x / 7.0 / 7.1 / 7.3+
users pick up a matching upstream prebuilt when one exists.
- install_llama_prebuilt.py: validate_server() adds --n-gpu-layers 1 for
linux-rocm and windows-hip hosts, so new HIP prebuilts are preflighted
on the GPU path instead of passing validation on CPU only.
- install_llama_prebuilt.py: restore the published windows-cpu fallback
for AMD Windows hosts without a HIP prebuilt so hash-approved bundles
are still preferred over the raw upstream CPU asset.
- install_python_stack.py: drop the /opt/rocm / hipcc gate in
_ensure_rocm_torch() and rely on _has_rocm_gpu(). Runtime-only ROCm
installs (package-managed minimal installs, Radeon software) that ship
amd-smi / rocminfo without hipcc can now repair a CPU-only venv via
"unsloth studio update". Adds an explicit IS_WINDOWS / IS_MACOS guard.
- studio/backend/utils/hardware/amd.py: honour HIP_VISIBLE_DEVICES /
ROCR_VISIBLE_DEVICES / CUDA_VISIBLE_DEVICES in
get_primary_gpu_utilization(). A process restricted to GPU 2 now
reports metrics for GPU 2 instead of physical GPU 0. Tighten the plain
bytes unit detection to an explicit allowlist.
- studio/backend/utils/hardware/hardware.py: route
get_backend_visible_gpu_info()'s backend_cuda_visible_devices field
through a helper that reads HIP_VISIBLE_DEVICES on ROCm. Drop the
unconditional "(rocm=False)" suffix in apply_gpu_ids() logs.
* Fix round 2 regressions: ROCm validate_server and Windows HIP routing
Follow-up to 810b833b addressing review findings on the first round of
hardening commits:
- install_llama_prebuilt.py validate_server: gate --n-gpu-layers on the
resolved install_kind instead of host.has_rocm. AMD Windows hosts
without a HIP prebuilt fall back to windows-cpu and must not be
validated with GPU layers; thread install_kind through from the
caller.
- install_llama_prebuilt.py resolve_release_asset_choice: reinstate the
"not has_rocm" guard on the published windows-cpu bundle so AMD
Windows hosts reach resolve_upstream_asset_choice() where the new
HIP prebuilt path lives. Prefer a published windows-hip bundle first
when one exists, fall through to upstream HIP + upstream CPU
otherwise.
- install_llama_prebuilt.py detect_host: also set has_physical_nvidia
when the secondary --query-gpu block confirms a working NVIDIA GPU,
so older nvidia-smi versions without -L support do not silently skip
the Linux diagnostics that key off has_physical_nvidia.
- install_llama_prebuilt.py: drop redundant "import re as _re" /
"import re as _re_rocm" local aliases in favour of the existing
top-level "import re".
- install_python_stack.py _ensure_rocm_torch: run the AMD
bitsandbytes install unconditionally after the HIP-torch probe so
"unsloth studio update" on venvs that already have ROCm torch still
gains the AMD bitsandbytes build.
- install.sh: add a non-x86_64 early-exit to get_torch_index_url() so
aarch64 / arm64 Linux hosts do not hit the ROCm wheel index
(PyTorch only publishes ROCm wheels for linux_x86_64).
- install.sh: add bitsandbytes install to the migrated-environment
branch so upgrades pick it up for ROCm hosts instead of only the
fresh-install path.
- install.sh: in the Radeon wheel path, pass version constraints +
--no-index --find-links to uv instead of explicit wheel URLs so a
version-compatible torch / torchvision / torchaudio triple is
resolved, rather than picking the highest-version wheel for each
package independently.
- studio/backend/utils/hardware/amd.py _first_visible_amd_gpu_id: fall
through to lower-priority visibility env vars when the first entry
is malformed (leading comma, all-whitespace first token) instead of
silently returning GPU 0.
* Fix round 3 findings: x86_64 guard, ROCm version clip, Radeon deps
Address issues surfaced by the round 3 reviewers on top of 8636fa63:
- install_python_stack.py _ensure_rocm_torch: add the same `x86_64`
guard that install.sh already has. Linux aarch64 / arm64 ROCm hosts
must skip the repair path entirely; PyTorch only publishes ROCm
wheels for linux_x86_64, and without this guard
`unsloth studio update` aborts with a missing-wheel error on non
x86_64 hosts.
- install_llama_prebuilt.py resolve_upstream_asset_choice: add a
best-effort _detect_host_rocm_version() helper (reading
/opt/rocm/.info/version, amd-smi version, hipconfig --version) and
filter rocm_candidates to entries whose major.minor is <= host
version. Falls back to the newest candidate only when no compatible
one exists, so a ROCm 6.4 host downloads rocm-6.4 instead of being
handed the numerically newest rocm-7.2 bundle (which fails preflight
and forces a source build).
- install.sh: remove the round 2 --no-index switch from the Radeon
wheel branch. --no-index forced uv to ignore PyPI entirely, which
broke transitive dependency resolution (filelock, sympy, networkx,
jinja2, fsspec, setuptools, typing-extensions, ...) on a fresh venv.
Restore the round 1 explicit wheel URL invocation but add a
torch / torchvision / torchaudio version-pair sanity check so a
mismatched trio (e.g. torch 2.9.1 + torchvision 0.23.0 + torchaudio
2.9.0) falls back to the standard ROCm index instead of installing a
broken combination.
- install_python_stack.py _ensure_rocm_torch: restructure the
"tag is None" path so it no longer short-circuits the bitsandbytes
install. On a ROCm runtime older than anything in
_ROCM_TORCH_INDEX, print the "no wheel" warning but still run the
AMD bitsandbytes install.
- studio/backend/core/training/worker.py: restore the pre-PR
"no timeout" behaviour for non-HIP causal-conv1d / mamba-ssm source
builds. The round 2 "timeout = 1800 if is_hip else 300" cap aborts
slow non-HIP builds (Linux aarch64, unsupported torch/CUDA combos)
after 5 minutes; omit timeout for the non-HIP branch so the cap
only applies to ROCm source builds.
* Fix round 4 findings: apply_gpu_ids env inheritance, Radeon X.Y, bitsandbytes gate
Address remaining issues surfaced by the round 4 reviewers:
- studio/backend/utils/hardware/hardware.py apply_gpu_ids: mirror the
selection into HIP_VISIBLE_DEVICES / ROCR_VISIBLE_DEVICES whenever
the caller already had a ROCm visibility env var set, not only when
IS_ROCM has already been set by detect_hardware(). Training and
inference workers call apply_gpu_ids() before detect_hardware()
runs, so the old guard would leave a forked ROCm worker with a
stale HIP_VISIBLE_DEVICES mask that no longer matched the
narrowed CUDA_VISIBLE_DEVICES selection.
- install.sh get_radeon_wheel_url: accept X.Y ROCm versions in
addition to X.Y.Z. The `/opt/rocm/.info/version` file and some
hipconfig versions report only two components, and the Radeon
repository publishes both rocm-rel-X.Y.Z/ and rocm-rel-X.Y/
directories, so treating X.Y as invalid caused Radeon hosts to fall
back to the generic ROCm index even when a matching AMD wheel set
existed.
- install_python_stack.py _ensure_rocm_torch: only install the AMD
bitsandbytes build when the venv actually has a ROCm-compatible
torch (either already present or just installed by this function).
Previously the bitsandbytes install ran unconditionally, which
could leave an AMD bitsandbytes layered on top of a CPU/CUDA torch
on hosts where the ROCm runtime is older than any entry in
_ROCM_TORCH_INDEX. Also add --force-reinstall so an existing
CPU/CUDA bitsandbytes is replaced by the AMD build during upgrades.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix gemini findings: amd-smi metric envelope validation and dict-wrapped GPU id
Two medium-severity defensive fixes from the gemini-code-assist review on
the AMD monitoring backend:
1. _extract_gpu_metrics may return a dict where every value is None when
amd-smi succeeds (zero exit) but the JSON envelope contains no usable
fields (error response, unsupported card). The new _has_real_metrics
helper lets get_primary_gpu_utilization surface available:False and
lets get_visible_gpu_utilization skip ghost device rows so the UI
does not render placeholder cards with empty numbers.
2. Newer amd-smi versions wrap scalar fields as {"value": 0, "unit":
"none"}, including the per-GPU id. The previous int(raw_id) call
silently fell back to the enumeration index in that case, losing the
real GPU id. Routing raw_id through the existing _parse_numeric
helper handles bare ints, floats, strings, and the dict shape
uniformly, with a debug log on parse failure.
* Fix gemini round 2 findings: explicit length guard on ROCm version file parser
Both _detect_rocm_version (install_python_stack.py) and
_detect_host_rocm_version (install_llama_prebuilt.py) read /opt/rocm/.info/version
or $ROCM_PATH/lib/rocm_version, split on "." and unconditionally accessed
parts[1]. The surrounding broad `except Exception: pass` already swallowed
the resulting IndexError, so a one-component file like "6\n" did fall
through to the next detection source -- but the control flow relied on
exception handling instead of an explicit check.
Add `if len(parts) >= 2:` guards in both helpers so the loop falls through
on its own without raising. Behaviour is unchanged for the common multi-
component case; the previously-silent IndexError path becomes an explicit
no-op.
* Fix gemini round 3: include has_rocm in validate_server fallback path
When validate_server is called without an explicit install_kind (older
call sites that have not been updated), the fallback was only enabling
--n-gpu-layers for NVIDIA and macOS arm64 hosts. AMD ROCm Linux hosts
fell through to the CPU validation path even though the prebuilt being
exercised was a HIP binary.
Add host.has_rocm to the fallback expression so the GPU offload flag is
applied consistently with the install_kind=='linux-rocm' / 'windows-hip'
branches above.
* Fix gemini round 4: remove risky bytes-vs-MB heuristic in _parse_memory_mb
The previous heuristic divided any bare number above 10_000_000 by
1024*1024 on the assumption that large unit-less values were bytes.
This misclassified small VRAM allocations: 5 MB of used VRAM reported
as 5_242_880 bytes without a unit would be taken at face value and
render as 5_242_880 MB (~5 TB) in the monitoring UI.
Modern amd-smi always provides explicit units (MiB/GiB dict form),
and legacy amd-smi returns bare numbers in MB -- the heuristic never
had a real workload to handle. Drop it and default to MB for bare
numeric input, keeping the existing unit-aware branches for dict /
string inputs unchanged.
The unrelated gemini suggestion to "default minor to 0" in the
amd-smi version awk parser was intentionally NOT applied: rocm7.0
and rocm7.1 ship different wheel sets, so silently substituting 0
for a missing minor could install the wrong wheels. The existing
reject-and-fall-through behaviour is safer.
* Fix gemini round 5: POSIX compliance and leading-comma visibility parsing
Three medium findings from gemini-code-assist addressed in this commit:
1. _pick_radeon_wheel used grep -o and sort -V, both GNU extensions
that are not in POSIX and break on BSD/BusyBox coreutils. install.sh
has a #!/bin/sh shebang so the whole pipeline was rewritten as a
single awk script that extracts all href="..." hits on each line,
filters to wheels matching the package prefix and python tag, and
picks the newest version via zero-padded lexical comparison. No
external sort or grep is needed.
2. _first_visible_amd_gpu_id in the AMD monitoring backend treated a
leading comma (e.g. HIP_VISIBLE_DEVICES=",1") as "fall through to
the next env var", which is surprising given the clear intent to
narrow to device 1. Filter empty tokens after the split and return
the first real one. An all-commas value ("," / ",,,") still falls
through because no real tokens exist; the empty-string and "-1"
explicit-zero cases are unchanged.
The unrelated amd-smi version awk parser suggestion was not applied
(see round 4 commit message for rationale: defaulting a missing minor
to 0 could silently install the wrong ROCm wheel set).
* Fix 20-reviewer.py findings: base drift, Radeon %2B, dpkg/rpm fallback, bnb, backend label
Consolidated fix batch from a 20-parallel reviewer.py run on the current
head. Each fix is drawn from a high-consensus finding and addresses a
real bug or feature gap, not a stylistic preference.
1. install.sh: bump `unsloth>=2026.4.2` -> `unsloth>=2026.4.4` at five
call sites so this branch no longer regresses main's version floor
(main bumped to 2026.4.4 in #4876). Without this, merging 4720 would
silently downgrade the minimum version pin for fresh installs.
2. install.sh: URL-decode Radeon wheel names before extracting the
torch / torchvision / torchaudio version strings. Real wheel URLs
from repo.radeon.com are percent-encoded ("torch-2.10.0%2Brocm7.2.0...")
so the previous `[+-]` terminator in the sed regex never matched,
`_torch_ver` stayed empty, `_radeon_versions_match` stayed false,
and every Radeon consumer install silently fell back to the generic
ROCm index. Now decode %2B -> + first, then extract, then validate.
3. install.sh: the two AMD bitsandbytes install lines were running
`uv pip install "bitsandbytes>=0.49.1"` without `--force-reinstall`,
so upgrades where the venv already has a CPU/CUDA bitsandbytes
satisfying the constraint would keep the stale non-AMD wheel. Add
`--force-reinstall --no-cache-dir` to both call sites, matching the
pattern already used in install_python_stack.py::_ensure_rocm_torch.
4. install_python_stack.py and install_llama_prebuilt.py: add
`dpkg-query -W rocm-core` and `rpm -q rocm-core` fallbacks to the
Python-side ROCm version detectors so they match the chain in
install.sh::get_torch_index_url. Package-managed ROCm installs
(Debian/Ubuntu/RHEL/Fedora distro packages) can expose GPUs via
rocminfo/amd-smi but still lack /opt/rocm/.info/version, hipconfig,
or amd-smi `version` output -- without these fallbacks, `unsloth
studio update` on such hosts returned None and skipped the ROCm
torch repair. Also strip the dpkg epoch prefix ("1:6.3.0-1") before
parsing so epoch-annotated packages parse correctly.
5. hardware.py: add a `_backend_label(device)` helper that returns
"rocm" when IS_ROCM is set and the device is DeviceType.CUDA, and
use it for every `"backend": ...` emission in JSON responses served
to the Studio frontend. Internally we still represent ROCm hosts as
DeviceType.CUDA (ROCm torch reuses the whole torch.cuda.* API
surface), but the user-facing API now correctly reports "rocm" on
AMD boxes instead of labeling them as "cuda".
All 250 simulation scenarios pass (was 233 before this batch: added 17
new regression tests covering the version pin, %2B decoding, bnb
force-reinstall flags, dpkg/rpm fallback presence, and the
_backend_label helper's four-way truth table).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix gemini round 6 + URL audit: amd.py defensive checks, rocm6.5+ clip to 6.4
Two rounds of fixes in one commit, plus a full URL audit of every PyPI /
download.pytorch.org / repo.radeon.com reference the PR introduces.
amd.py (4 medium gemini findings on commit b3627bc2):
1. _extract_gpu_metrics used `and vram_total_mb` as part of the vram_util
gate. The follow-up `vram_total_mb > 0` already handles the division
guard, but the truthiness check was redundant and slightly surprising
for a 0.0 valid value. Replace with explicit `is not None and > 0`
for both vram_util and power_util.
2. get_physical_gpu_count called `data.get("gpu", ...)` without guarding
for non-dict envelopes. A scalar / string JSON response from amd-smi
would raise AttributeError. Add an isinstance(data, dict) check and
return None for unexpected shapes.
3. get_visible_gpu_utilization had the same .get() exposure on the outer
envelope. Rewrite the gpu_list extraction as an explicit
list/dict/else cascade so a malformed scalar envelope produces
gpu_list=[data] and continues without raising.
4. The same function's per-entry loop also called gpu_data.get() on
whatever was inside gpu_list. If a scalar ever leaks into the list
(directly or via the previous fix's fallback), _extract_gpu_metrics
would raise on the first .get() inside the helper. Skip non-dict
entries in the loop before extracting metrics.
install.sh (URL audit finding, previously flagged by 20-reviewer as #13):
5. get_torch_index_url used `rocm6.*` in the rocm tag case statement,
which matched rocm6.5 and rocm6.6 and emitted
download.pytorch.org/whl/rocm6.5 -- which returns HTTP 403 because
PyTorch only publishes rocm 5.7, 6.0-6.4, 7.0-7.2. Enumerate the
supported 6.x minors explicitly and add a rocm6.* fallback branch
that clips to rocm6.4 (the last supported 6.x wheel set).
URL audit results (all URLs PR 4720 references):
- 14/14 download.pytorch.org/whl/{cpu,cu118,cu124,cu126,cu128,cu130,
rocm6.0..6.4,rocm7.0..7.2} return HTTP 200.
- 9/9 repo.radeon.com/rocm/manylinux/rocm-rel-{5.7,6.0,6.1,6.2,6.3,
6.4,7.0,7.1,7.2}/ return HTTP 200.
- X.Y.Z patch directories exist for 7.0.2, 7.1.1, 7.2.1 but NOT for
6.3.0, 6.4.0, 6.2.1 -- install.sh already handles this via the X.Y.Z
-> X.Y fallback sed in the Radeon wheel install block.
- Docs links (rocm.docs.amd.com, docs.unsloth.ai AMD guide) and the
llama.cpp GitHub releases API endpoint all return 200.
Test suite: 255 -> 258. New regression coverage:
- U17: get_physical_gpu_count tolerates scalar amd-smi envelope
- U18: get_visible_gpu_utilization tolerates scalar envelope
- U19a-c: vram_util / power_util return None on zero total, but
vram_total_gb still echoes 0.0 (not None)
- A_rocm{6.5,6.6,6.9}_clips_to_rocm64: install.sh clips unsupported
6.x minors to rocm6.4 instead of producing a 403 index URL
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix reviewer.py round 2: tokenizer AMD multi-GPU, --no-torch bnb, main.py backend label
Three high-confidence findings from a second 20-parallel reviewer.py run
on commit 7effb3ae. Triaged 15 total findings and applied the three that
were confirmed as real bugs; the rest were either false positives (e.g.
"migrated AMD venv not repaired" -- _ensure_rocm_torch runs downstream
via setup.sh regardless), design decisions (e.g. visibility mask env
vars not consulted in installer detection), or edge cases the existing
fallback logic already handles.
1. unsloth/tokenizer_utils.py [6/20]: the multi-GPU guard's shell probe
runs `nvidia-smi --query-gpu=memory.used`, catches the failure, then
only raises if `torch.cuda.is_available()` is False. On ROCm torch,
torch.cuda.is_available() returns True (ROCm reuses the torch.cuda.*
API), so the guard becomes dead code on AMD hosts and multi-GPU AMD
setups slip through even though unsloth does not support them yet.
Add a torch.cuda.device_count() > 1 fallback inside the except so
AMD multi-visible-device setups are flagged consistently with the
original CUDA memory check.
2. install.sh [1/20]: the fresh-install bitsandbytes block for AMD ROCm
ran unconditionally when TORCH_INDEX_URL matched `*/rocm*`, even when
SKIP_TORCH=true (from --no-torch or Intel Mac auto-detect). A user
running `install.sh --no-torch` on an AMD host would still pull in
bitsandbytes despite explicitly asking for GGUF-only mode. Wrap the
case block in an outer `[ "$SKIP_TORCH" = false ]` guard.
3. studio/backend/main.py [3/20]: the /api/system endpoint returned
`"device_backend": get_device().value`, which is "cuda" on ROCm
hosts (because ROCm torch piggybacks on torch.cuda). Other endpoints
(hardware.py) already use the _backend_label helper which swaps
"cuda" -> "rocm" when IS_ROCM. Route /api/system through the same
helper so the Studio UI reports the backend consistently across all
endpoints.
4. studio/backend/tests/test_utils.py: update test_backend_matches_device
to call _backend_label(get_device()) instead of raw get_device().value
so the test matches the new contract and still passes on CUDA hosts.
Tests: 258 -> 261. New regression coverage:
- X08 main.py /api/system uses _backend_label
- X09 tokenizer multi-GPU guard has device_count() fallback
- X10 fresh-install bnb case block gated on SKIP_TORCH=false
* fix: prevent bitsandbytes from overwriting ROCm torch with CUDA wheels
During install, bitsandbytes was installed without --no-deps, causing
uv to resolve torch from PyPI (CUDA build) and silently overwrite the
ROCm wheels that were just installed in the previous step.
This happened in three places:
- install.sh: bitsandbytes install in both migrated and fresh paths
- install_python_stack.py: bitsandbytes install inside _ensure_rocm_torch()
Additionally, multiple install steps in install_python_stack.py (extras,
overrides, studio deps) can pull in CUDA torch via transitive
dependencies. A final _ensure_rocm_torch() call at the end of the
install sequence ensures ROCm torch is always in place at runtime.
All changes are gated behind ROCm-specific conditions and do not affect
NVIDIA, CPU-only, macOS, or Windows install paths.
Tested on AMD Instinct MI300X VF with ROCm 7.2.0 -- confirms
torch==2.10.0+rocm7.1 with HIP 7.1.25424 after install.
* fix: ROCm inference fallback -- skip Unsloth patching and bnb 4-bit on HIP
On AMD ROCm (HIP), two issues prevent the normal Unsloth inference path:
1. Unsloth's global monkey-patching of transformers model classes
(LlamaRotaryEmbedding, attention modules) triggers
_assert_async_cuda_kernel crashes on HIP during generation.
Training uses different code paths and works fine.
2. bitsandbytes 4-bit matmul kernels also trigger HIP assertion
failures on MI300X (CDNA3 / gfx942), even without Unsloth patching.
This commit adds a ROCm-specific inference fallback that:
- Skips importing Unsloth at module level (prevents global patching)
- Loads models in 16-bit with plain transformers + PEFT instead
- Resolves pre-quantized model names (e.g. "xxx-bnb-4bit" -> "xxx")
since pre-quantized HF repos still trigger bnb codepaths
- Guards get_chat_template calls (unavailable without Unsloth import)
- Fixes max_seq_length=0 being passed to from_pretrained (GGUF
semantics don't apply to transformers path)
The NVIDIA path is completely unchanged -- Unsloth import and
for_inference() optimization remain active. GGUF inference (via
llama-server/HIP) is unaffected since it never imports Python model
classes. AMD GPUs typically have large VRAM (e.g. 192GB on MI300X)
so 16-bit loading is practical for inference.
Tested on AMD Instinct MI300X VF (ROCm 7.2, HIP 7.1.25424):
- Simple generation: PASS
- Compare mode (base vs finetuned): PASS
- GGUF inference + tool calling: PASS (unaffected by this change)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: guard audio/vision inference on ROCm, remove unused import
- Add clear RuntimeError for audio/vision model inference on ROCm
(these paths use Unsloth's FastModel/FastVisionModel which would
crash on HIP; GGUF inference is the supported path on AMD)
- Remove unused `import os as _os` from the ROCm changes
* fix: amd-smi parsing for newer output format (gpu_data wrapper, mem_usage, temperature)
amd-smi on recent ROCm versions (7.x) wraps metric output in a
{"gpu_data": [...]} envelope instead of returning a raw list. This
caused get_primary_gpu_utilization() and get_visible_gpu_utilization()
to fail silently (returning available=False) because the GPU data
dict was never unwrapped.
Additionally:
- VRAM data moved from "vram" to "mem_usage" with "total_vram" /
"used_vram" keys. Added fallback key lookup.
- Temperature "edge" sensor returns "N/A" on MI300X VF; the previous
dict.get() chain returned the "N/A" string instead of falling
through to "hotspot". Changed to a loop that checks each key until
a parseable value is found.
Tested on AMD Instinct MI300X VF (ROCm 7.2, amd-smi 24.x):
- GPU utilization: 0% (idle), up to 100% during training
- Temperature: 40-44C (from hotspot sensor)
- VRAM: 0.28/191.69 GB (idle)
- Power: 158-211W draw
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Bug fix detecting radeon (#4940)
* Bug fix detecting radeon
* Expanding GPU target for gfx1100*
* Generalize gfx family-prefix filter to cover gfx10/gfx12 as well
rocminfo on ROCm 6.1+ emits LLVM generic-family ISA lines alongside the
specific GPU (e.g. gfx11-generic next to gfx1100). The outer grep captures
the bare family prefix from the generic line, and passing that to
-DGPU_TARGETS breaks the HIP build because clang only accepts specific
gfxNNN ids.
The previous filter only special-cased gfx11. Generalize it so any bare
2-digit family prefix (gfx10, gfx11, gfx12, ...) is dropped whenever a
specific sibling target is present in the same list. No real AMD GPU has
a 2-digit gfx id, so the filter can only ever drop family prefixes and
never a real target.
Covers the existing gfx11 cases unchanged, and extends the same fix to
gfx10-1-generic / gfx10-3-generic (RDNA1/2) and gfx12-generic (RDNA4),
which would otherwise hit the same build failure on newer rocminfo.
---------
Co-authored-by: Iswarya Alex <iswarya.alex@amd.com>
Co-authored-by: Daniel Han <danielhanchen@users.noreply.github.com>
---------
Co-authored-by: Eda Z <eda.zhou@amd.com>
Co-authored-by: GoldenGrapeGentleman <yueyuan@amd.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: billishyahao <bill.he@amd.com>
Co-authored-by: Iswarya Alex <47045679+iswaryaalex@users.noreply.github.com>
Co-authored-by: Iswarya Alex <iswarya.alex@amd.com>
Co-authored-by: Daniel Han <danielhanchen@users.noreply.github.com>
* updated models template mappers. added lfm2.5vl450m to transformers 5.3.0 whitelist
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* split venv_t5 into venv_t5_530 and venv_t5_550 for tiered transformers 5.x support
* fix bfloat16 crash on T4 for FORCE_FLOAT32 models and disable trust_remote_code auto-enable for native t5 models
* revert FORCE_FLOAT32 dtype change
* restrict trust_remote_code auto-enable to Nemotron models only
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* use config.json model_type for tier detection, add unsloth/nvidia namespace guard
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Revert "[pre-commit.ci] auto fixes from pre-commit.com hooks"
This reverts commit fb43d468e2.
* Revert "use config.json model_type for tier detection, add unsloth/nvidia namespace guard"
This reverts commit fc49ae2453.
* add unsloth/nvidia namespace guard to Nemotron trust_remote_code auto-enable
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* reorder tier checks: all substring matches before config.json fetches
* extract shared activate_transformers_for_subprocess into transformers_version.py
* narrow Nemotron trust_remote_code to nemotron_h/nemotron-3-nano, add to export worker
* clean venv_t5 dirs before re-install in setup.sh, clarify version alias comment
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* run venv_t5 migration outside deps fast-path gate in both setup scripts
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Fix custom folder scanning when pointing directly at a model directory.
When a user adds a custom scan folder that points directly at a model
directory (e.g. /path/to/gemma-4-e2b-it-gguf/ containing config.json
and gemma-4-E2B-it-BF16.gguf), the model list previously showed
individual .gguf files as separate entries instead of recognizing the
directory as a single model. Clicking any entry showed "No GGUF
variants found" because list_local_gguf_variants received a file path
and immediately returned empty.
Changes:
- Add _is_model_directory() helper that detects directories with both
config metadata and actual model weight files (excludes mmproj GGUFs
and non-weight .bin files like tokenizer.bin)
- _scan_models_dir: detect self-model and return single directory entry
- _scan_lmstudio_dir: surface model directories directly instead of
descending into them as publisher folders; handle both root and child
model directories
- Add _resolve_gguf_dir() helper for GGUF path resolution that only
falls back to parent directory when parent has model metadata
- list_local_gguf_variants / _find_local_gguf_by_variant: use resolver
so .gguf file paths inside model directories work correctly
* Add vision detection cache to is_vision_model() to avoid redundant subprocess spawns
is_vision_model() is called 4-5 times per training run for the same model
with zero caching. For transformers 5.x models, each call spawns a full
subprocess (~6s each). This adds a module-level _vision_detection_cache dict
following the same pattern as the existing _audio_detection_cache used by
detect_audio_type(). The function is refactored into a thin cache wrapper
around _is_vision_model_uncached(), saving ~12s per training run.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Include hf_token in vision cache key for gated model correctness
Cache key is now (model_name, hf_token) instead of just model_name.
This prevents stale False results when an unauthenticated probe for a
gated model is followed by an authenticated call.
* Remove test file from main PR - will be submitted separately
* Fix vision cache: normalize model names and skip caching transient failures
- Normalize model names in cache key using resolve_cached_repo_id_case()
to avoid duplicate entries for different casings of the same HF repo
(aligns with case normalization from #4822)
- Return None instead of False on transient failures (network errors,
subprocess timeouts, HF API issues) so the cache layer can distinguish
"definitely not a vision model" from "failed to check"
- Only cache definitive True/False results; transient failures are retried
on the next call instead of being permanently locked in as False
* Refine failure handling: cache deterministic failures, guard normalization
- Subprocess non-zero exit, JSON errors, and general exceptions return
False (deterministic, cached) instead of None (retryable). Only
subprocess.TimeoutExpired returns None since timeouts are transient.
- Wrap cache key normalization in try/except so resolve_cached_repo_id_case
or normalize_path failures fall back to raw model_name instead of
crashing callers.
* Harden vision detection cache: fix transient failure handling, thread safety, token security
- All subprocess failure paths now return None (transient) instead of False,
preventing permanent misclassification of VLMs after temporary HF/auth/network errors
- Use SHA256 fingerprint for hf_token in cache key instead of raw bearer token
- Add threading.Lock with double-checked locking to prevent thundering herd
of concurrent subprocess spawns for the same uncached model
- Distinguish permanent failures (RepositoryNotFoundError, GatedRepoError,
ValueError) from transient ones in _is_vision_model_uncached
- Pass resolved/normalized model name to detection (not just cache key)
- Log normalization fallback at debug level instead of silent swallow
- Thread hf_token through callers in routes/models.py and trainer.py
that previously omitted it
* Refine lock strategy and token fingerprint
- Move detection computation outside the lock to avoid serializing
long-running subprocess spawns (60s timeout) and HF API calls across
all concurrent model checks. Lock is now only held for cache writes.
- Use full SHA256 digest for token fingerprint instead of truncated
16-char prefix to eliminate collision risk.
* Fix huggingface_hub import fallback and use atomic cache read
- Add fallback import path for RepositoryNotFoundError/GatedRepoError
from huggingface_hub.utils (older hub versions) when .errors is
not available
- Use sentinel-based dict.get() for single atomic cache read instead
of two-step in/[] pattern (future-proof for no-GIL runtimes)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* fix(studio): reuse HF cached repo casing to prevent duplicate downloads
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Move cache case resolution tests to separate PR
Tests for resolve_cached_repo_id_case and get_model_config case resolution
belong in their own PR to keep this change focused on the runtime fix.
* fix(studio): debug-log HF_HUB_CACHE fallback in path_utils
* Fix stale memoization in resolve_cached_repo_id_case
- Check exact-case path before memo to ensure a newly-appeared exact
match always wins over a previously memoized variant
- Validate memoized entries still exist on disk before returning them
to prevent stale results when cache dirs are deleted/recreated
* Minor cleanups for cache case resolution
- Use .is_dir() instead of .exists() for exact-case cache check
(cache entries are always directories)
- Remove redundant fallback in _detect_audio_from_tokenizer since
get_cache_path already handles case resolution and returns None
when the model is not cached
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* fix(studio): lazy-import AutoConfig in model_config.py to fix transformers 5.x version switch
Move `from transformers import AutoConfig` from module level to inside
load_model_config() where it is actually used.
model_config.py is transitively imported at module load time via:
core/inference/__init__ → llama_cpp → utils.models → model_config
In inference subprocesses (mp.spawn), this chain runs before
_activate_transformers_version() can prepend .venv_t5/ to sys.path.
The eager import caches transformers 4.57.6 in sys.modules, and the
subsequent sys.path change has no effect — Python always checks
sys.modules before sys.path.
Making the import lazy ensures transformers is not loaded until after
version activation, so the subprocess picks up the correct version.
* fix(studio): also lazy-import extract_model_size_b in llama_cpp.py
Belt-and-suspenders: make the import that originally triggered the
chain lazy as well, so future module-level AutoConfig additions in
utils.models cannot reintroduce the problem.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Gemma-4 support landed in transformers main
(huggingface/transformers#45192). Update the version pin from
5.5.0.dev0 to 5.5.0 across loader, Studio version switcher,
and the MLX installer. Also fix install_gemma4_mlx.sh which
referenced a non-existent v5.5-release branch -- pin it to
the correct commit (91b1ab1) instead.
* UI Changes
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Remove unrelated test file
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix(studio): prevent small models from stalling on tool-calling tasks
Small GGUF models (< 9B params) in "Think, Search, Code" mode would
often describe what they planned to do ("Let me create this dashboard")
and then stop generating without ever calling a tool.
Three changes:
1. Simplify web_tips for small models: remove the "fetch its full content
by calling web_search with the url parameter" guidance for models < 9B.
This multi-step instruction causes small models to plan elaborate
search-then-fetch-then-code sequences they cannot reliably execute.
2. Add "always call tools directly" imperative to the system prompt nudge
so models act immediately instead of narrating their intentions.
3. Add plan-without-action re-prompt in the agentic loop: when the model
emits planning text (matching patterns like "let me", "I'll", etc.)
without calling any tool, inject a nudge asking it to call the tool
and continue the loop. Capped at 2 re-prompts per request.
Benchmarked with Qwen3.5-4B-GGUF (N=5 trials per variant):
- Baseline: 40% of requests had any tool call
- Combined fix: 100% of requests had at least one tool call
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Daniel Han <danielhanchen@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix Windows "Non-relative patterns are unsupported" when loading local GGUF models
When a user loads a GGUF model from a local Windows path (e.g.
C:\Users\danie\.lmstudio\models\unsloth\functiongemma-270m-it-GGUF),
the model identifier contains backslashes and a drive letter. Both
load_model_defaults() and _has_specific_yaml() constructed a YAML
filename from the full absolute path and passed it to Path.rglob(),
which rejects non-relative patterns on Windows.
Fixed by detecting Windows-style paths (drive letters, UNC paths,
backslashes) in addition to Unix-style paths, and using only the
directory basename for the YAML filename lookup when the identifier
is a local filesystem path.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Refactor: reuse is_local_path helper, fix case-sensitive suffix lookup
- Replace inline local-path detection in model_config.py and
inference_config.py with the existing is_local_path() from utils.paths,
which already handles Unix, Windows drive-letter, UNC, and backslash paths
- Fix case-sensitive suffix lookup in load_model_defaults(): the
_REVERSE_MODEL_MAPPING is lowercase-keyed, so suffix comparisons must use
.lower() to match paths like /path/to/Spark-TTS-0.5B/LLM
* Fix WSL path parsing and _has_specific_yaml suffix lookup
- Use normalize_path() before Path() operations so backslash Windows
paths (e.g. C:\Users\...\model) are correctly split on POSIX/WSL hosts
where pathlib treats backslashes as literal characters
- Add suffix-based (2-component and 1-component) lookup to
_has_specific_yaml() so it matches the same resolution rules as
load_model_defaults(), fixing wrong inference params for local
suffix-mapped models like Spark-TTS-0.5B/LLM
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Revert to balanced for inference
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Remove unused for_inference parameter from get_device_map
Since inference and training both use "balanced" now, the for_inference
flag is dead code. Remove it from the function signature, the call site
in inference.py, and simplify the tests accordingly.
* Remove redundant TestDeviceMapForInference test class
TestGpuAutoSelection already covers the same multi-gpu and single-gpu
device_map assertions. The TestDeviceMapForInference class was left
over from when for_inference had distinct behavior.
* Remove redundant test_get_device_map_multi_gpu_uses_balanced
Its assertions ([0,1] -> balanced, [0] -> sequential) are already
covered by test_get_device_map_uses_explicit_gpu_selection.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* fix(studio): replace unicode emoji in print() to avoid cp1252 crash on Windows
On Windows the default console encoding is cp1252 which cannot encode
unicode emoji like U+2705 or U+26A0. bare print() calls with these
characters cause a UnicodeEncodeError at runtime.
- run.py: replace emoji with ASCII status prefixes [OK] and [WARNING]
- format_conversion.py: remove duplicate print() that mirrors the
logger.info() call on the next line, and drop the emoji from the
log message since loggers handle encoding separately
* fix(studio): apply same emoji/print cleanup to parallel VLM conversion path
The parallel URL-based conversion logic has the same duplicate print()
with emoji that was fixed in the sequential path. Remove the bare
print() and drop the emoji from the logger.info() call.
* Treat install_python_stack.py failure as fatal in setup.ps1
On Linux/Mac, setup.sh runs under set -euo pipefail so a non-zero
exit from install_python_stack.py aborts the installer. On Windows,
setup.ps1 had no exit code check -- if the Python script crashed
(eg from the cp1252 UnicodeEncodeError), the installer silently
continued past the dependency loop and reported success. Studio
would then fail at launch with ModuleNotFoundError for structlog,
fastapi, and other deps that were never installed.
Capture $LASTEXITCODE and exit 1 if the dependency installer fails,
matching the error handling pattern already used for PyTorch install.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* [WIP] balanced device map for studio
* gpus as a request parameter
* API for multi GPU stuff
* return multi gpu util in new API
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Use balanced_low0 instead of balanced
* Use balanced_low0 instead of balanced
* Fix device_map typo, UUID parsing crash, set() filter bug, and broken tests
- balanced_low0 -> balanced_low_0 (transformers/accelerate rejects the old string)
- get_parent_visible_gpu_ids() now handles UUID/MIG CUDA_VISIBLE_DEVICES
gracefully instead of crashing on int() parse
- _get_backend_visible_gpu_info() set() or None bug: empty set is falsy so
CUDA_VISIBLE_DEVICES=-1 would disable filtering and report all GPUs
- test_gpu_selection.py: add missing get_visible_gpu_utilization import and
add required job_id arg to start_training() calls
* Smart GPU determinism using estimates
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* disallow gpu selection for gguf for now
* cleanup
* Slightly larger baseline
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Treat empty list as auto
* Verbose logging/debug
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Cleanup and revert unnecessary deletions
* Cleanup excessive logs and guard against disk/cpu offload
* auth for visibility API. cleanup redundant imports. Adjust QLoRA estimate
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* support for non cuda gpus
* Fix multi-GPU auto-selection memory accounting
The multi_gpu_factor was applied uniformly to all GPUs including the
first one, which unfairly penalizes single-GPU capacity when
transitioning to multi-GPU. This created a discontinuity where a model
that barely fits 1 GPU would suddenly require 2 GPUs because the first
GPU's free memory was discounted by 20%.
Now the first GPU keeps its full free memory, and only additional GPUs
have an overhead factor (0.85) applied to account for inter-GPU
communication and sharding overhead. This gives more accurate
auto-selection and avoids unnecessary multi-GPU for models that
comfortably fit on one device.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add sandbox tests for multi-GPU selection logic
24 tests covering model size estimation, memory requirements, automatic
GPU selection, device map generation, GPU ID validation, and multi-GPU
overhead accounting. All tests use mocks so they run without GPUs on
Linux, macOS, and Windows.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix reviewer findings: 4bit inference estimate, fallback, GGUF gpu_ids, retry
1. 4-bit inference now uses reduced memory estimate (model_size/3 + buffer)
instead of the FP16 1.3x multiplier. This prevents over-sharding
quantized models across unnecessary GPUs.
2. When model size estimation fails, auto_select_gpu_ids now falls back to
all visible GPUs instead of returning None (which could default to
single-GPU loading for an unknown-size model).
3. GGUF inference route now treats gpu_ids=[] as auto-selection (same as
None) instead of rejecting it as an unsupported explicit request.
4. Training retry path for "could not get source code" now preserves the
gpu_ids parameter so the retry lands on the same GPUs.
5. Updated sandbox tests to cover the new 4-bit inference estimate branch.
* Remove accidentally added unsloth-zoo submodule
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix UUID/MIG visibility and update test expectations
1. nvidia.py: When CUDA_VISIBLE_DEVICES uses UUID/MIG tokens, the
visibility APIs now return "unresolved" with empty device lists instead
of exposing all physical GPUs. This prevents the UI from showing GPUs
that the backend process cannot actually use.
2. test_gpu_selection.py: Updated test expectations to match the new
multi-GPU overhead accounting (first GPU at full capacity, 0.85x for
additional GPUs) and 4-bit inference memory estimation formula.
All 60 tests now pass.
* Add CPU/disk offload guard to audio inference path
The audio model loading branch returned before the common
get_offloaded_device_map_entries() check, so audio models loaded with a
multi-GPU device_map that spilled layers to CPU/disk would be accepted
instead of rejected. Now audio loads also verify no modules are offloaded.
* Improve VRAM requirement estimates
* Replace balanced_low_0 with balanced
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* refine calculations for slightly easier nums
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* adjust estimates
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Use nums instead of obj to avoid seralisation error
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Harden nvidia-smi parsing and fix fallback GPU list
1. nvidia.py: Wrap int() casts for GPU index and memory in try/except
so MIG slices, N/A values, or unexpected nvidia-smi output skip the
unparseable row instead of aborting the entire GPU list.
2. nvidia.py: Handle GPU names containing commas by using the last
field as memory instead of a fixed positional index.
3. hardware.py: fallback_all now uses gpu_candidates (GPUs with verified
VRAM data) instead of raw devices list, which could include GPUs
with null VRAM that were excluded from the ranking.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* cleanup
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* consolidate raise_if_offload
* Improve MoE support. Guard against nvidia-smi failures
* Improve MoE support. Guard against nvidia-smi failures
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix shared-expert LoRA undercount, torch VRAM fallback, and apply_gpu_ids edge case
1. vram_estimation.py: compute_lora_params now includes shared experts
(n_shared_experts) alongside routed experts when computing MoE LoRA
adapter parameters. Previously only n_experts were counted, causing
the estimator to undercount adapter, optimizer, and gradient memory
for DeepSeek/GLM-style models with shared experts.
2. hardware.py: _torch_get_per_device_info now uses mem_get_info (which
reports system-wide VRAM usage) instead of memory_allocated (which
only reports this process's PyTorch allocations). This prevents
auto-selection from treating a GPU as mostly free when another
process is consuming VRAM. Falls back to memory_allocated when
mem_get_info is unavailable.
3. hardware.py: apply_gpu_ids([]) now returns early instead of setting
CUDA_VISIBLE_DEVICES="" which would disable CUDA entirely. Empty
list inherits the parent visibility, same as None.
4. hardware.py: Upgraded fallback_all GPU selection log from debug to
warning so operators are notified when the model likely will not fit
in available VRAM.
* Guard nvidia-smi subprocess calls against OSError and TimeoutExpired
get_visible_gpu_utilization and get_backend_visible_gpu_info now catch
OSError (nvidia-smi not found) and TimeoutExpired internally instead
of relying on callers to wrap every invocation. Returns the standard
available=False sentinel on failure so the torch-based fallback in
hardware.py can take over.
* Guard get_primary_gpu_utilization and reset GPU caches between tests
1. nvidia.py: get_primary_gpu_utilization now catches OSError and
TimeoutExpired internally, matching the pattern already used in
get_visible_gpu_utilization and get_backend_visible_gpu_info. All
three nvidia-smi callers are now self-contained.
2. test_gpu_selection.py: Added _GpuCacheResetMixin that resets the
module-level _physical_gpu_count and _visible_gpu_count caches in
tearDown. Applied to all test classes that exercise GPU selection,
device map, or visibility functions. This prevents stale cache
values from leaking between tests and causing flaky results on
machines with real GPUs.
* Fix nvidia-smi fallback regression and physical GPU count validation
1. hardware.py: get_gpu_utilization, get_visible_gpu_utilization, and
get_backend_visible_gpu_info now check result.get("available") before
returning the nvidia-smi result. When nvidia-smi is unavailable or
returns no data (e.g., containers without nvidia-smi, UUID/MIG masks),
the functions fall through to the torch-based fallback instead of
returning an empty result. This fixes a regression where the internal
exception handling in nvidia.py prevented the caller's except block
from triggering the fallback.
2. hardware.py: resolve_requested_gpu_ids now separates negative-ID
validation from physical upper-bound validation. The physical count
check is only enforced when it is plausibly a true physical count
(i.e., higher than the largest parent-visible ID), since
torch.cuda.device_count() under CUDA_VISIBLE_DEVICES returns the
visible count, not the physical total. The parent-visible-set check
remains authoritative in all cases. This prevents valid physical IDs
like [2, 3] from being rejected as "out of range" when nvidia-smi is
unavailable and CUDA_VISIBLE_DEVICES="2,3" makes torch report only
2 devices.
* Fix UUID/MIG torch fallback to enumerate devices by ordinal
When CUDA_VISIBLE_DEVICES uses UUID or MIG identifiers,
get_parent_visible_gpu_ids() returns [] because the tokens are
non-numeric. The torch fallback in get_visible_gpu_utilization() and
get_backend_visible_gpu_info() previously passed that empty list to
_torch_get_per_device_info(), getting nothing back.
Now both functions detect the empty-list case and fall back to
enumerating torch-visible ordinals (0..device_count-1) with
index_kind="relative". This means the UI and auto-selection still
see real device data in Kubernetes, MIG, and Slurm-style UUID
environments where nvidia-smi output cannot be mapped to physical
indices.
Updated test_uuid_parent_visibility to verify the new torch fallback
path returns available=True with relative ordinals.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add type hint for gpu_ids parameter in InferenceOrchestrator.load_model
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
normalize_path() unconditionally converted Windows paths like
C:\Users\... to WSL format /mnt/c/Users/..., which breaks path
resolution on native Windows. This caused LM Studio GGUF models
to fail detection (detect_gguf_model returned None for the invalid
path), falling through to the Unsloth import path which requires
a GPU.
Now only performs the /mnt/ mapping when actually running under WSL.
On native Windows, drive letters are preserved and backslashes are
normalized to forward slashes.
* fix: default HF cache to standard platform path instead of legacy Unsloth cache
* feat: show LM Studio and local models in chat Fine-tuned tab
* feat: show LM Studio models in Hub models tab
* fix: fetch local models after auth refresh completes
* Revert "fix: fetch local models after auth refresh completes"
This reverts commit cfd61f0ac7.
* fix: increase llama-server health check timeout to 600s for large models
* feat: expandable GGUF variant picker for LM Studio local models
* fix: show GGUF variant label for locally loaded LM Studio models
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: show publisher name in LM Studio model labels
* fix: set model_id for loose GGUF files in LM Studio publisher dirs
* fix: show publisher prefix in Fine-tuned tab LM Studio models
* fix: only use model_id for lmstudio source models
* fix: only show LM Studio models in Hub tab on Mac/chat-only mode
* fix: respect XDG_CACHE_HOME, handle Windows paths in isLocalPath, refresh LM Studio on remount
- _setup_cache_env now reads XDG_CACHE_HOME (falls back to ~/.cache)
instead of hard-coding ~/.cache/huggingface. This follows the standard
HF cache resolution chain and respects distro/container overrides.
- isLocalPath in GgufVariantExpander uses a regex that covers Windows
drive letters (C:\, D:/), UNC paths (\\server\share), relative paths
(./, ../), and tilde (~/) -- not just startsWith("/").
- HubModelPicker.useEffect now calls listLocalModels() before the
alreadyCached early-return gate so LM Studio models are always
refreshed on remount. Also seeds useState from _lmStudioCache for
instant display on re-open.
* fix: add comment explaining isLocalPath regex for Windows/cross-platform paths
* fix: prioritize unsloth publisher in LM Studio model list
* fix: scope unsloth-first sort to LM Studio models on all platforms
* fix: add missing _lmStudioCache module-level declaration
* fix: prioritize unsloth publisher before timestamp sort in LM Studio group
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* fix: install.sh Mac Intel compatibility + Studio no-torch support (#4621)
On Intel Macs (x86_64), PyTorch has no wheels for torch >= 2.3, so the
installer crashes. Even when torch is absent, Studio crashes on startup
because two files have bare top-level torch imports.
Studio's GGUF inference (llama.cpp) does not need PyTorch. Training and
HF-inference already isolate torch to subprocesses. Only 2 files in the
server startup chain had top-level torch imports preventing startup.
Changes:
- install.sh: detect architecture, default to Python 3.12 on Intel Mac,
skip torch install, add Python 3.13.8 guard for arm64, pass
UNSLOTH_NO_TORCH env var to setup.sh
- data_collators.py: remove unused `import torch` (no torch.* refs)
- chat_templates.py: lazy-import IterableDataset into function bodies
- install_python_stack.py: add IS_MACOS/NO_TORCH constants, skip
torch-dependent packages, skip overrides.txt, skip triton on macOS
No existing working flow changes. Linux/WSL and macOS arm64 behavior is
identical.
* tests: add test suite for Mac Intel compat + no-torch mode
Shell tests (test_mac_intel_compat.sh):
- version_ge edge cases (9 tests)
- Architecture detection for Darwin x86_64/arm64, Linux x86_64/aarch64
- get_torch_index_url returns cpu on simulated Darwin
- UNSLOTH_NO_TORCH propagation to both setup.sh branches
Python unit tests (test_no_torch_filtering.py):
- _filter_requirements with NO_TORCH_SKIP_PACKAGES
- NO_TORCH env var parsing (true/1/TRUE/false/0/unset)
- IS_MACOS constant check
- Overrides skip and triton macOS skip guards
Python import tests (test_studio_import_no_torch.py):
- data_collators.py loads in isolated no-torch venv
- chat_templates.py has no top-level torch imports
- Negative control confirms import torch fails without torch
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* tests: add E2E sandbox tests for Mac Intel no-torch mode
Replace static/synthetic test stubs with real sandbox tests:
- Shell: E2E uv venv creation at Python 3.12, mock uv shim to verify
torch install is skipped when MAC_INTEL=true, dynamic env propagation
test for UNSLOTH_NO_TORCH in both local and non-local install paths
- Python filtering: test real extras.txt and extras-no-deps.txt with
NO_TORCH_SKIP_PACKAGES, subprocess mock of install_python_stack() for
5 platform configs (NO_TORCH+macOS, Windows+NO_TORCH, normal Linux,
Windows-only, macOS-only), VCS URL and env marker edge cases
- Python imports: parametrized Python 3.12+3.13 venv fixture, dataclass
instantiation for all 3 collator classes, chat_templates.py exec with
stubs, negative controls proving import torch and torchao install fail
in no-torch venvs
91 total tests, all passing.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: address reviewer findings for Intel Mac no-torch mode
P1 fixes:
- Auto-infer NO_TORCH in install_python_stack.py via platform.machine()
so `unsloth studio update` preserves GGUF-only mode without needing
the UNSLOTH_NO_TORCH env var (6/10 reviewers)
- Add openai-whisper and transformers-cfg to NO_TORCH_SKIP_PACKAGES
since both have unconditional torch dependencies (4/10 reviewers)
- Skip unsloth-zoo on Intel Mac --local installs (depends on torch)
in both migrated and fresh install paths (1/10)
- Recreate stale 3.13 venvs as 3.12 on Intel Mac re-runs (1/10)
- Detect Apple Silicon under Rosetta via sysctl hw.optional.arm64
and warn user to use native arm64 terminal (1/10)
P2 fixes:
- Wire new test files into tests/run_all.sh (4/10 reviewers)
- Add update-path tests (skip_base=False) for Intel Mac
- Add _infer_no_torch tests for platform auto-detection
P3 fixes:
- Fix macOS progress bar total (triton step skipped but was counted)
- Fix temp file leak when Windows + NO_TORCH filters stack
All tests pass: 30 shell, 66 Python (96 total).
* feat: add --python override flag to install.sh
Lets users force a specific Python version, e.g. ./install.sh --python 3.12.
Addresses M2 Mac users whose systems resolve to a problematic 3.13.x patch.
When --python is set, the Intel Mac stale-venv guard and 3.13.8 auto-downgrade
are skipped so the user's choice is respected.
* tests: add comprehensive E2E sandbox tests for no-torch mode
Add test_e2e_no_torch_sandbox.py with 7 test groups (43 tests total)
covering the full no-torch import chain, edge cases, and install logic:
- Group 1: BEFORE vs AFTER import chain comparison (proves the bug
existed and the fix works by synthetically prepending top-level torch
imports)
- Group 2: Dataclass instantiation without torch
- Group 3: Edge cases with broken/fake torch modules on sys.path
- Group 4: Hardware detection fallback to CPU without torch
- Group 5: install.sh flag parsing, version resolution, arch detection
- Group 6: install_python_stack.py NO_TORCH filtering
- Group 7: Live server startup without torch (marked @server, skipped
when studio venv is unavailable)
All 43 tests pass on both Python 3.12 and 3.13 isolated venvs.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* feat: add --no-torch flag to install.sh/ps1, fix lazy import bug in dataset formatting
- Fix chat_templates.py: narrow torch IterableDataset import into inner
try/except ImportError so dataset.map() works without torch installed
- Fix format_conversion.py: same lazy import fix for convert_chatml_to_alpaca
and convert_alpaca_to_chatml
- Add --no-torch flag to install.sh with unified SKIP_TORCH variable
(driven by --no-torch flag OR MAC_INTEL auto-detection)
- Add --no-torch flag to install.ps1 with $SkipTorch variable
- Print CPU hint when no GPU detected and --no-torch not set
- Replace MAC_INTEL guards with SKIP_TORCH in torch install sections
- Update shell tests (40 pass) and Python tests (90 pass)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: address reviewer findings for --no-torch installer paths
- Fix migrated-env branch in install.sh and install.ps1: check
SKIP_TORCH first, then branch on STUDIO_LOCAL_INSTALL. Previously
SKIP_TORCH+non-local fell into else and installed unsloth-zoo (which
depends on torch), defeating --no-torch mode.
- Fix $env:UNSLOTH_NO_TORCH leak in install.ps1: always set to "true"
or "false" instead of only setting on the true branch. Prevents stale
no-torch state from leaking across runs in the same PS session.
- Fix install_python_stack.py update path: add NO_TORCH guard around
base.txt install so unsloth studio update does not reinstall
unsloth-zoo (which depends on torch) in no-torch mode.
* fix: install unsloth + unsloth-zoo with --no-deps in no-torch mode
Instead of skipping unsloth-zoo entirely (which breaks unsloth's
dependency on it), install both packages with --no-deps so they are
present but torch is not pulled in transitively. Applied consistently
across all no-torch paths: migrated-env, fresh-local, fresh-non-local
in install.sh, install.ps1, and install_python_stack.py.
* chore: temporarily remove test files (will be added in a follow-up)
* refactor: deduplicate SKIP_TORCH conditional branches in installers
Collapse if/else blocks that differ only by --no-deps into a single
branch with a conditional flag variable. Applied to migrated-env and
fresh-local paths in install.sh, install.ps1, and install_python_stack.py.
* fix: apply --no-deps to fresh non-local --no-torch install path
The non-local else branch was missing $_no_deps_arg/$noDepsArg, so
uv pip install unsloth would resolve torch from PyPI metadata (the
published unsloth package still declares torch as a hard dep). Now
--no-deps is applied consistently to all SKIP_TORCH code paths.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* feat: multi-source model discovery (HF default, legacy cache, LM Studio)
* Fix multi-source model discovery bugs
- Fix lmstudio_model_dirs: add ~/.lmstudio/models as default path,
remove dead sys.platform branch, add dedup via seen set
- Fix _setup_cache_env: preserve legacy HF cache env vars when the
legacy hub directory exists and is non-empty
- Fix _scan_lmstudio_dir: use absolute path for id field so
is_local_path() returns True
- Remove LM Studio dirs from allowed_roots (scanned unconditionally)
- Replace bare except passes with logger.warning in legacy cache blocks
- Fix delete_cached_model to search both default and legacy HF caches
- Make lmstudio_dirs non-optional in TS interface (matches Python schema)
- Exclude lmstudio source from trainable model filter
- Remove unused import sys
* Scan HF default cache alongside legacy and active caches
When _setup_cache_env overrides HF_HUB_CACHE to the legacy Unsloth
path, the standard HF default cache (~/.cache/huggingface/hub) was
never scanned, hiding models downloaded before Unsloth Studio was
installed.
Add hf_default_cache_dir() and _all_hf_cache_scans() helper that
deduplicates and scans all three HF cache locations (active, legacy,
default). Used in list_local_models, list_cached_gguf,
list_cached_models, and delete_cached_model.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* feat(db): add SQLite storage layer for training history
* feat(api): add training history endpoints and response models
* feat(training): integrate DB persistence into training event loop
* feat(ui): add training history views and card grid
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio): address review issues in training history persistence
- Strip hf_token/wandb_token from config before SQLite storage
- Add UUID suffix to job_id for collision resistance
- Use isfinite() for 0.0 metric handling throughout
- Respect _should_stop in error event finalization
- Run schema DDL once per process, not per connection
- Close connection on schema init failure
- Guard cleanup_orphaned_runs at startup
- Cap _metric_buffer at 500 entries
- Make FLUSH_THRESHOLD a class constant
- Map 'running' to 'training' phase in historical view
- Derive LR/GradNorm from history arrays in historical view
- Fix nested button with div[role=button] in history cards
- Guard String(value) against null/undefined in config popover
- Clear selectedHistoryRunId on auto tab switch
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio): address round-2 review findings across training backend and frontend
Backend (training.py):
- Move state mutation after proc.start() so a failed spawn does not wedge
the backend with is_training=True
- Create DB run row eagerly after proc.start() so runs appear in history
during model loading, not after first metric event
- Rewrite _flush_metrics_to_db() with snapshot-before-insert pattern to
preserve metrics arriving during the write and retain buffer on failure
- Guard eval_loss with float() coercion and math.isfinite(), matching the
existing grad_norm guard
- Increase pump thread join timeout from 3s to 8s to cover SQLite's
default 5s lock timeout
Frontend (studio-page.tsx):
- Fix history navigation: check isTrainingRunning instead of
showTrainingView in onSelectRun so completed runs are not misrouted
- Replace activeTab state + auto-switch useEffect with derived tab to
eliminate react-hooks/set-state-in-effect lint violation
Frontend (historical-training-view.tsx):
- Add explicit "running" branch to message ternary so running runs no
longer fall through to "Training errored"
- Derive loading from detail/error state and move cleanup to effect
return to eliminate react-hooks/set-state-in-effect lint violation
Frontend (progress-section.tsx):
- Derive stopRequested from isTrainingRunning && stopRequestedLocal to
eliminate react-hooks/set-state-in-effect lint violation and remove
unused useEffect import
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio): resolve 3 remaining bugs from round-2 review
1. Stuck on Current Run tab [12/20]: Only force "current-run" tab when
isTrainingRunning is true, not when stale completed-run data exists.
After training ends, users can freely navigate to Configure.
2. Incomplete metric sanitization [7/20]: Apply float() coercion and
isfinite() guards to loss and learning_rate, matching the existing
pattern used by grad_norm and eval_loss. Prevents TypeError from
string values and NaN leaks into history arrays.
3. Stop button state leak across runs [10/20]: Add key={runtime.jobId}
to ProgressSection so React remounts it when a new run starts,
resetting stopRequestedLocal state.
* fix(studio): deduplicate loss/lr sanitization in training event handler
Reuse _safe_loss/_safe_lr from the progress update block instead of
re-sanitizing the same raw event values for metric history.
* fix(studio): restore loss > 0 guard to prevent eval steps injecting 0.0 into metric histories
Round-2/3 fixes relaxed the history append guard from `loss > 0` to
`loss is not None`, which let eval-only log events (where loss defaults
to 0.0) append fake zeros into loss_history and lr_history. Restore the
`loss > 0` check to match the worker's own has_train_loss gate. The
float() coercion and isfinite() sanitization from round-3 remain intact.
* fix(studio): resolve training history bugs — nullable loss/lr, tab nav, sparkline
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* fix(studio): prevent ModuleNotFoundError in dataset.map() on Windows
On Windows, dataset.map() uses "spawn", which requires workers to
import compiled modules from disk. Previously, clear_unsloth_compiled_cache()
deleted the entire directory, causing workers to crash when looking for
UnslothSFTTrainer.py.
Changes:
1. Added `preserve_patterns` to cache cleanup to keep `Unsloth*Trainer.py`
on Windows while clearing model-specific files.
2. Added the cache directory to PYTHONPATH for spawn workers.
Linux/macOS behavior is unchanged.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix spawn-platform coverage, CWD path mismatch, and race condition for PR #4473
- Extend platform guard from win32-only to include macOS (also uses spawn
since Python 3.8, same ModuleNotFoundError would occur)
- Replace fragile CWD-based PYTHONPATH registration with centralized
register_compiled_cache_on_path() that uses the same __file__-relative
_CACHE_DIRS already used by cache_cleanup -- fixes path mismatch when
studio is launched from a directory other than the repo root
- Move PYTHONPATH registration to the top of _train_worker(), before any
dataset.map() call (previously it ran late in config assembly, after
dataset formatting which also calls dataset.map())
- Update inference.py model-unload to preserve trainer files on spawn
platforms, preventing a race where unloading a model via inference tab
would delete UnslothSFTTrainer.py while training workers are importing it
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix cache-dir precedence reversal in register_compiled_cache_on_path()
Iterating _CACHE_DIRS in forward order while calling insert(0) each time
reverses the declared priority: later entries shadow earlier ones. When
multiple compiled-cache directories exist, spawned workers could import a
stale trainer from the wrong cache.
Fix: iterate in reverse so that the highest-priority entry (first in
_CACHE_DIRS) is inserted last and ends up at position 0 in sys.path and
PYTHONPATH.
* fix: harden worker-count helpers against cpu_count=None and desired<=0
- safe_num_proc: guard os.cpu_count() with `or 1`, clamp multi-GPU
path with max(1, min(4, desired)), clamp return with max(1, desired)
- safe_thread_num_proc: same os.cpu_count() guard and return clamp
- Add regression tests (31 L1 unit + 10 sandbox edge-case tests)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* remove regression tests from PR
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* fix: handle Windows subprocess crash during dataset.map()
Windows uses spawn (not fork) for multiprocessing. Spawned workers
cannot resolve Unsloth's dynamically compiled cache modules from
unsloth_compiled_cache/, causing ModuleNotFoundError and RuntimeError
during dataset.map() tokenization.
Add two platform-guarded patches for sys.platform == "win32":
1. Force HF_DATASETS_MULTITHREADING_MAX_WORKERS=1 and set spawn method
2. Monkey-patch Dataset.map() to force num_proc=None
Fixes#4490
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* address review: extend spawn fix to macOS, add multiprocess fallback
- Change platform checks from sys.platform == "win32" to
sys.platform != "linux" so macOS (also spawn-based) is covered
- Wrap multiprocess import in try/except falling back to stdlib
multiprocessing when the multiprocess package isn't installed
- Rename _win32_safe_map to _spawn_safe_map to reflect broader scope
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: replace global Dataset.map monkey-patch with targeted num_proc routing
The previous approach had issues: Patch 1 set HF_DATASETS_MULTITHREADING_MAX_WORKERS
and forced set_start_method (dead code on platforms already using spawn), and Patch 2
globally monkey-patched Dataset.map() (too broad, missed Dataset.filter()).
Replace with a two-layer fix:
1. Studio layer: Add dataset_map_num_proc() that returns None on spawn platforms
(Windows, macOS). Unlike num_proc=1 which still creates Pool(1) and spawns a
worker, num_proc=None runs Dataset.map()/filter() truly in-process.
Update all dataset.map() callsites to use it. ThreadPoolExecutor callers
(format_conversion.py) keep using safe_num_proc() since threads are unaffected.
2. Root-cause layer: Propagate UNSLOTH_COMPILE_LOCATION via PYTHONPATH on spawn
platforms so spawned workers can import compiled modules. Mirrors the .venv_t5
pattern in worker.py. Does not import unsloth_zoo.compiler (heavy torch/triton
imports). Completely skipped on Linux.
Also extend safe_num_proc() to return 1 on macOS (was only guarding Windows),
and narrow the transformers 5.x dataloader guard from != "linux" to explicit
("win32", "darwin").
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: add safe_thread_num_proc() for ThreadPoolExecutor callsites
safe_num_proc() correctly caps to 1 on macOS/Windows for process-based
multiprocessing, but format_conversion.py reuses it for ThreadPoolExecutor
workers. Threads share address space and are unaffected by spawn, so
capping to 1 makes image URL downloads sequential -- a real regression.
Add safe_thread_num_proc() that skips the platform guard but keeps the
cpu_count heuristic, and switch both ThreadPoolExecutor callsites in
format_conversion.py to use it.
* fix: remove double-wrap in dataset_num_proc + fix num_proc=1 in datasets route
- trainer.py:3009: Replace safe_num_proc(max(1, os.cpu_count() // 4))
with max(1, (os.cpu_count() or 1) // 4) to avoid double-wrapping
inside dataset_map_num_proc which already calls safe_num_proc
- trainer.py:15-20: Clarify comment on PYTHONPATH propagation
- datasets.py:445: Change num_proc=1 to num_proc=None for 10-row
preview slice (avoids unnecessary multiprocessing overhead)
* fix: guard os.cpu_count() against None in worker-count helpers
os.cpu_count() can return None on some platforms. Use (os.cpu_count() or 1)
to prevent TypeError in safe_num_proc() and safe_thread_num_proc().
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* feat(chat): add server-side timings and context display for GGUF
Extract timings/usage metadata from llama-server SSE stream and forward
through the full stack. Replace client-side estimates with accurate
server-reported metrics (prompt eval, tok/s, token counts, cache hits).
Add context window usage bar to chat top nav.
* feat(chat): source badges with hover cards and 2-row collapse
- Add hover cards to source badges showing favicon, title, URL and
snippet description on hover
- Limit source badges to 2 rows with +X more expand/collapse
- Parse snippet from web search results for hover card descriptions
- Replace individual Source rendering with grouped SourcesGroup component
* fix(chat): add null guards for server timings edge cases
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(chat): reset contextUsage on thread switch, remove unused context-display
* fix(chat): stop double-counting completion tokens in tool-calling path
* fix(chat): skip metadata events in llm_assist consumers
* fix(chat): hide context usage bar in compare mode
* fix(chat): harden timings pipeline and context usage persistence
Accumulate prompt_ms, predicted_ms, and predicted_n from intermediate
tool-detection passes so the final metadata reflects total server work.
Persist contextUsage in message metadata (Dexie) and restore on thread
load. Add type guard in gguf_stream_chunks for unexpected dict events.
Clear contextUsage when entering compare mode.
* feat(chat): make GGUF stream metadata OpenAI-compatible
* fix(chat): address PR review feedback
* feat(chat): address PR review feedback
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix(recipe-studio): prevent fitView from zooming to wrong location on recipe load
* feat: add pymupdf/python-docx deps and unstructured uploads storage root
* feat: add POST /seed/upload-unstructured-file endpoint
* feat: add multi-file chunking with source_file column
* feat: update frontend types and API layer for multi-file upload
* feat: round-robin preview rows across source files
Ensures every uploaded file is represented in the preview table
by cycling through sources instead of just taking the first N rows.
* fix: disable OCR, fix auto-load timing, fix persistence on reload
- Disable pymupdf4llm OCR with write_images=False, show_progress=False
- Replace onAllUploaded callback with useEffect that detects uploading→done
transition (avoids stale closure reading empty file IDs)
- Fix importer to preserve file IDs from saved recipes instead of clearing
(clearing only happens at share time via sanitizeSeedForShare)
* fix: harden unstructured upload with input validation and state fixes
Validate block_id/file_id with alphanumeric regex to prevent path
traversal, use exact stem match for file deletion, add error handling
for metadata writes and empty files, fix React stale closures and
object mutations in upload loop, and correct validation logic for
unstructured seed resolved_paths.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: address PR review - legacy path import, share sanitizer, sync effect
Promote legacy source.path into resolved_paths for old unstructured
recipes, clear source.paths in share sanitizer to prevent leaking local
filesystem paths, and gate file sync effect to dialog open transition
so users can actually delete all uploaded files.
* fix: CSV column fix (BOM + whitespace + unnamed index re-save) for #4470
* fix: harden unstructured upload flow and polish dialog UX
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* feat: support full model GGUF export, disable incompatible methods in UI
* fix: resolve base model from config.json for venv_t5 export switching
* feat: detect BNB-quantized models and disable all export methods for quantized non-PEFT checkpoints
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: relocate Ollama Modelfile alongside GGUFs during non-PEFT export cleanup
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fixing Qwen3.5 bug and adding Outetts dependencies
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Apply suggestion from @danielhanchen
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Allow Windows setup to complete without NVIDIA GPU
setup.ps1 previously hard-exited if nvidia-smi was not found, blocking
setup entirely on CPU-only or non-NVIDIA machines. The backend already
supports CPU and MLX (Apple Silicon) in chat-only GGUF mode, and the
Linux/Mac setup.sh handles missing GPUs gracefully.
Changes:
- Convert the GPU check from a hard exit to a warning
- Guard CUDA toolkit installation behind $HasNvidiaSmi
- Install CPU-only PyTorch when no GPU is detected
- Build llama.cpp without CUDA flags when no GPU is present
- Update doc comment to reflect CPU support
* Cache frontend build across setup runs
Skip the frontend npm install + build if frontend/dist already exists.
Previously setup.ps1 nuked node_modules and package-lock.json on every
run, and both scripts always rebuilt even when dist/ was already present.
On a git clone editable install, the first setup run still builds the
frontend as before. Subsequent runs skip it, saving several minutes.
To force a rebuild, delete frontend/dist and re-run setup.
* Show pip progress for PyTorch download on Windows
The torch CUDA wheel is ~2.8 GB and the CPU wheel is ~300 MB. With
| Out-Null suppressing all output, the install appeared completely
frozen with no feedback. Remove | Out-Null for the torch install
lines so pip's download progress bar is visible. Add a size hint
so users know the download is expected to take a while.
Also moves the Triton success message inside the GPU branch so it
only prints when Triton was actually installed.
* Guard CUDA env re-sanitization behind GPU check in llama.cpp build
The CUDA_PATH re-sanitization block (lines 1020-1033) references
$CudaToolkitRoot which is only set when $HasNvidiaSmi is true and
the CUDA Toolkit section runs. On CPU-only machines, $CudaToolkitRoot
is null, causing Split-Path to throw:
Split-Path : Cannot bind argument to parameter 'Path' because it is null.
Wrap the entire block in `if ($HasNvidiaSmi -and $CudaToolkitRoot)`.
* Rebuild frontend when source files are newer than dist/
Instead of only checking if dist/ exists, compare source file timestamps
against the dist/ directory. If any file in frontend/src/ is newer than
dist/, trigger a rebuild. This handles the case where a developer pulls
new frontend changes and re-runs setup -- stale assets get rebuilt
automatically.
* Fix cmake not found on Windows after winget install
Two issues fixed:
1. After winget installs cmake, Refresh-Environment may not pick up the
new PATH entry (MSI PATH changes sometimes need a new shell). Added a
fallback that probes cmake's default install locations (Program Files,
LocalAppData) and adds the directory to PATH explicitly if found.
2. If cmake is still unavailable when the llama.cpp build starts (e.g.
winget failed silently or PATH was not updated), the build now skips
gracefully with a [SKIP] warning instead of crashing with
"cmake : The term 'cmake' is not recognized".
* Fix frontend rebuild detection and decouple oxc-validator install
Address review feedback:
- Check entire frontend/ directory for changes, not just src/.
The build also depends on package.json, vite.config.ts,
tailwind.config.ts, public/, and other config files. A change
to any of these now triggers a rebuild.
- Move oxc-validator npm install outside the frontend build gate
in setup.sh so it always runs on setup, matching setup.ps1
which already had it outside the gate.
* Show cmake errors on failure and retry CUDA VS integration with elevation
Two fixes for issue #4405 (Windows setup fails at cmake configure):
1. cmake configure: capture output and display it on failure instead of
piping to Out-Null. When the error mentions "No CUDA toolset found",
print a hint about the CUDA VS integration files.
2. CUDA VS integration copy: when the direct Copy-Item fails (needs
admin access to write to Program Files), retry with Start-Process
-Verb RunAs to prompt for elevation. This is the root cause of the
"No CUDA toolset found" cmake failure -- the .targets files that let
MSBuild compile .cu files are missing from the VS BuildCustomizations
directory.
* Address reviewer feedback: cmake PATH persistence, stale cache, torch error check
1. Persist cmake PATH to user registry so Refresh-Environment cannot
drop it later in the same setup run. Previously the process-only
PATH addition at phase 1 could vanish when Refresh-Environment
rebuilt PATH from registry during phase 2/3 installs.
2. Clean stale CMake cache before configure. If a previous run built
with CUDA and the user reruns without a GPU (or vice versa), the
cached GGML_CUDA value would persist. Now the build dir is removed
before configure.
3. Explicitly set -DGGML_CUDA=OFF for CPU-only builds instead of just
omitting CUDA flags. This prevents cmake from auto-detecting a
partial CUDA installation.
4. Fix CUDA cmake flag indentation -- was misaligned from the original
PR, now consistently indented inside the if/else block.
5. Fail hard if pip install torch returns a non-zero exit code instead
of silently continuing with a broken environment.
* Remove extra CUDA cmake flags to align Windows with Linux build
Drop GGML_CUDA_FA_ALL_QUANTS, GGML_CUDA_F16, GGML_CUDA_GRAPHS,
GGML_CUDA_FORCE_CUBLAS, and GGML_CUDA_PEER_MAX_BATCH_SIZE flags.
The Linux build in setup.sh only sets GGML_CUDA=ON and lets llama.cpp
use its defaults for everything else. Keep Windows consistent.
* Address reviewer round 2: GPU probe fallback, Triton check, stale binary rebuild
1. GPU detection: fallback to default nvidia-smi install locations
(Program Files\NVIDIA Corporation\NVSMI, System32) when nvidia-smi
is not on PATH. Prevents silent CPU-only provisioning on machines
that have a GPU but a broken PATH.
2. Triton: check $LASTEXITCODE after pip install and print [WARN]
on failure instead of unconditional [OK].
3. Stale llama-server: check CMakeCache.txt for GGML_CUDA setting
and rebuild if the existing binary does not match the current GPU
mode (e.g. CUDA binary on a now-CPU-only rerun, or vice versa).
* Fix frontend rebuild detection and npm dependency issues
Addresses reviewer feedback on the frontend caching logic:
1. setup.sh: Fix broken find command that caused exit under pipefail.
The piped `find | xargs find -newer` had paths after the expression
which GNU find rejects. Replaced with a simpler `find -maxdepth 1
-type f -newer dist/` that checks ALL top-level files (catches
index.html, bun.lock, etc. that the extension allowlist missed).
2. setup.sh: Guard oxc-validator npm install behind `command -v npm`
check. When the frontend build is skipped (dist/ is cached), Node
bootstrap is also skipped, so npm may not be available.
3. setup.ps1: Replace Get-ChildItem -Include with explicit path
probing for src/ and public/. PowerShell's -Include without a
trailing wildcard silently returns nothing, so src/public changes
were never detected. Also check ALL top-level files instead of
just .json/.ts/.js/.mjs extensions.
* Fix studio setup: venv isolation, centralized .venv_t5, uv targeting
- All platforms (including Colab) now create ~/.unsloth/studio/.venv
with --without-pip fallback for broken ensurepip environments
- Add --python sys.executable to uv pip install in install_python_stack.py
so uv targets the correct venv instead of system Python
- Centralize .venv_t5 bootstrap in transformers_version.py with proper
validation (checks required packages exist, not just non-empty dir)
- Replace ~150 lines of duplicated install code across 3 worker files
with calls to the shared _ensure_venv_t5_exists() helper
- Use uv-if-present with pip fallback; do not install uv at runtime
- Add site.addsitedir() shim in colab.py so notebook cells can import
studio packages from the venv without system-Python double-install
- Update .venv_t5 packages: huggingface_hub 1.3.0->1.7.1, add hf_xet
- Bump transformers pin 4.57.1->4.57.6 in requirements + constraints
- Add Fast-Install helper to setup.ps1 with uv+pip fallback
- Keep Colab-specific completion banner in setup.sh
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix nvidia-smi PATH persistence and cmake requirement for CPU-only
1. Store nvidia-smi as an absolute path ($NvidiaSmiExe) on first
detection. All later calls (Get-CudaComputeCapability,
Get-PytorchCudaTag, CUDA toolkit detection) use this absolute
path instead of relying on PATH. This survives Refresh-Environment
which rebuilds PATH from the registry and drops process-only
additions.
2. Make cmake fatal for CPU-only installs. CPU-only machines depend
entirely on llama-server for GGUF chat mode, so reporting "Setup
Complete!" without it is misleading. GPU machines can still skip
the llama-server build since they have other inference paths.
* Fix broken frontend freshness detection in setup scripts
- setup.sh: Replace broken `find | xargs find -newer` pipeline with
single `find ... -newer` call. The old pipeline produced "paths must
precede expression" errors (silently suppressed by 2>/dev/null),
causing top-level config changes to never trigger a rebuild.
- setup.sh: Add `command -v npm` guard to oxc-validator block so it
does not fail when Node was not installed (build-skip path).
- setup.ps1: Replace `Get-ChildItem -Include` (unreliable without
-Recurse on PS 5.1) with explicit directory paths for src/ and
public/ scanning.
- Both: Add *.html to tracked file patterns so index.html (Vite
entry point) changes trigger a rebuild.
- Both: Use -print -quit instead of piping to head -1 for efficiency.
* Fix bugs found during review of PRs #4404, #4400, #4399
- setup.sh: Add || true guard to find command that checks frontend/src
and frontend/public dirs, preventing script abort under set -euo
pipefail when either directory is missing
- colab.py: Use sys.path.insert(0, ...) instead of site.addsitedir()
so Studio venv packages take priority over system copies. Add warning
when venv is missing instead of silently failing.
- transformers_version.py: _venv_t5_is_valid() now checks installed
package versions via .dist-info metadata, not just directory presence.
Prevents false positives from stale or wrong-version packages.
- transformers_version.py: _install_to_venv_t5() now passes --upgrade
so pip replaces existing stale packages in the target directory.
- setup.ps1: CPU-only PyTorch install uses --index-url for cpu wheel
and all install commands use Fast-Install (uv with pip fallback).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix _venv_t5_is_valid dist-info loop exiting after first directory
Remove premature break that caused the loop over .dist-info directories
to exit after the first match even if it had no METADATA file. Now
continues iterating until a valid METADATA is found or all dirs are
exhausted.
* Capture error output on failure instead of discarding with Out-Null
setup.ps1: 6 locations changed from `| Out-Null` to `| Out-String` with
output shown on failure -- PyTorch GPU/CPU install, Triton install,
venv_t5 package loop, cmake llama-server and llama-quantize builds.
transformers_version.py: clean stale .venv_t5 directory before reinstall
when validation detects missing or version-mismatched packages.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix ModuleNotFoundError when CLI imports studio.backend.core
The backend uses bare "from utils.*" imports everywhere, relying on
backend/ being on sys.path. Workers and routes add it at startup, but
the CLI imports studio.backend.core as a package -- backend/ was never
added. Add sys.path setup at the top of core/__init__.py so lazy
imports resolve correctly regardless of entry point.
Fixes: unsloth inference unsloth/Qwen3-8B "who are you" crashing with
"No module named 'utils'"
* Fix frontend freshness check to detect all top-level file changes
The extension allowlist (*.json, *.ts, *.js, *.mjs, *.html) missed
files like bun.lock, so lockfile-only dependency changes could skip
the frontend rebuild. Check all top-level files instead.
* Add tiktoken to .venv_t5 for Qwen-family tokenizers
Qwen models use tiktoken-based tokenizers which fail when routed through
the transformers 5.x overlay without tiktoken installed. Add it to the
setup scripts (with deps for Windows) and runtime fallback list.
Integrates PR #4418.
* Fix tiktoken crash in _venv_t5_is_valid and stray brace in setup.ps1
_venv_t5_is_valid() crashed with ValueError on unpinned packages like
"tiktoken" (no ==version). Handle by splitting safely and skipping
version check for unpinned packages (existence check only).
Also remove stray closing brace in setup.ps1 tiktoken install block.
---------
Co-authored-by: Daniel Han <danielhanchen@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* studio: improve onboarding UX, tooltips, and training defaults
- Change splash text to "Train and run LLMs locally"
- Add "Chat Only" card with BubbleChatIcon to skip directly to chat
- Add Skip/Skip to Chat buttons in sidebar and footer
- Back button on step 1 returns to splash screen instead of being disabled
- Change "Watch video guide" to "Get started with our guide" with new URL
- Update intro text to mention all model types + chat
- Make all tooltips clickable (in addition to hover) via React context
- Strip surrounding quotes from pasted HF tokens
- Rename "Eval Split" to "Evaluation Split"
- Add SparklesIcon to "Auto Detect" format option
- Change step 4 heading to "Choose your training parameters"
- Default max_steps to 60
- Learning rate displayed in scientific notation with +/- stepper
- Context length options capped by model's max_position_embeddings (via AutoConfig)
- Fix "QLORA"/"LORA" to "QLoRA"/"LoRA" in summary step
- Backend: add max_position_embeddings to model config endpoint
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* compare for 2 diff models
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* resolving gemini comments
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: disable thinking for Qwen3.5 <9B and always for AI Assist
- Change Qwen3.5 thinking threshold from <=2B to <9B (0.8B, 2B, 4B
all disable thinking by default; 9B+ enables it)
- Always pass enable_thinking=False in AI Assist helper calls
(_run_with_helper and _generate_with_backend) regardless of chat
thinking settings
* studio: address PR review comments
- Extract _get_max_position_embeddings helper to DRY config extraction
- Fix "Skip to Chat" to navigate to /chat on step 1 (was /studio)
* fix: comment out debug print statements
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: skip Shiki highlighting for incomplete SVG code fences
While streaming SVG content, the syntax highlighter (Shiki) re-parses
the entire growing SVG on every token, blocking the main thread and
freezing the code area until the fence closes. Show a plain-text
preview for incomplete SVG fences instead, similar to how Mermaid
diagrams show a placeholder while streaming.
* studio: fix default top_k from 50/40 to 20 for chat inference
Per Qwen3.5 docs (unsloth.ai/docs/models/qwen3.5), top_k should be 20
for both thinking and non-thinking modes. The model-specific config in
inference_defaults.json already had top_k=20 for Qwen3.5, but the
generic fallback defaults were wrong:
- Frontend DEFAULT_INFERENCE_PARAMS.topK: 50 -> 20
- Backend generate_chat_completion top_k: 40 -> 20
- Backend generate_chat_completion_with_tools top_k: 40 -> 20
- Frontend title generation top_k: 40 -> 20
* studio: set universal inference defaults for unknown models
Default params for any model without specific config:
temperature=0.6, top_p=0.95, top_k=20, min_p=0.01,
presence_penalty=0.0, repetition_penalty=1.0
Models with entries in inference_defaults.json (Qwen3.5, Gemma-3,
Llama, etc.) override these with their recommended values.
Updated in: frontend DEFAULT_INFERENCE_PARAMS, backend Pydantic
request models, and backend generate_chat_completion defaults.
* studio: only trust_remote_code for unsloth/ models in AutoConfig
Only set trust_remote_code=True when the model name starts with
"unsloth/". All other models default to False for safety.
* studio: move Generating spinner above the composer
The "Generating" spinner was below the send message bar, causing
the bar to jump up and down. Move it above the composer in both
the regular thread view and the welcome/empty view.
* studio: adjust toast close button position away from edge
Move the X close button on toasts (like "Starting model...") from
top-1.5 to top-3 and add right-3, giving more breathing room from
the top-right corner.
* studio: make Think button smaller with tighter icon-text gap
Reduce gap from 1.5 to 0.5, padding from px-2.5/py-1 to px-2/py-0.5,
and icon from size-3.5 to size-3.
* studio: multiple onboarding and chat UX improvements
- Move Generating spinner above composer (fixes jumping send bar)
- Make Think button smaller with tighter icon-text gap
- Chat card now inside grid (same size as Audio/Embeddings cards)
- Rename "Chat Only" to "Chat"
- Chat card requires Continue to proceed (no auto-advance)
- Continue on Chat selection skips onboarding and goes to /chat
- Tooltip (i) click on Chat card doesn't trigger navigation
- Step 1 footer Back button goes back to splash (label is "Back")
- Splash "Skip Onboarding" renamed to "Skip to Chat", navigates to /chat
- Toast close button moved away from edge
* studio: align Skip to Chat button, add Skip to footer
- Sidebar "Skip to Chat" now uses primary (green) Button style with
arrow icon, full width, aligned like step items. Shows on all steps.
- Footer: added "Skip" outline button next to Continue that goes
directly to /studio with progress saved (markOnboardingDone)
* studio: change default max steps from 30 to 60 in toggle hook
The DEFAULT_MAX_STEPS in use-max-steps-epochs-toggle.ts was still 30,
used as fallback when toggling from epochs back to max steps.
* studio: extend context length options to 262K
CONTEXT_LENGTHS now includes 65536, 131072, 262144 in addition to
the existing 512-32768 range. The onboarding step filters these by
the model's max_position_embeddings (e.g. Nemotron-3-Nano-4B has
262144), showing powers of 2 up to the model's maximum.
* studio: auto-select LoRA vs QLoRA based on model size and GPU memory
After selecting a model in onboarding, detect the total model weight
file size from HF Hub (safetensors/bin files). Then estimate memory
needed: model_size_gb * 1.5 * context_scale, where context_scale is:
- <=8192 tokens: 1.0x
- >8192 tokens: 1.7x
- >=16384 tokens: 2.0x
- >=32768 tokens: 4.0x
If the estimate fits in free GPU VRAM, default to LoRA (16-bit).
Otherwise default to QLoRA (4-bit).
Backend changes:
- Add model_size_bytes to ModelDetails (models.py)
- Add _get_model_size_bytes() using HfApi.repo_info (routes/models.py)
- Add vram_free_gb to get_gpu_summary (hardware.py)
Frontend changes:
- Add autoSelectTrainingMethod() in training-config-store.ts
- Called after model defaults are loaded
- Add model_size_bytes to ModelConfigResponse type
- Add vramFreeGb to HardwareInfo hook
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: rename "Importing ML libraries..." to "Importing Unsloth..."
* studio: show model/dataset in training status, fix LoRA/QLoRA casing
- Training status now shows 'Training "model_name"' and 'Dataset = ...'
instead of generic "Starting training..."
- Fix Studio progress section to show QLoRA/LoRA instead of QLORA/LORA
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: rename 'Skip to Chat' to 'Skip Onboarding' on splash screen
* studio: add presence_penalty support for chat inference
Add presence_penalty as a parameter across the full stack:
- Backend: llama_cpp.py generate_chat_completion/with_tools, Pydantic
models (inference.py), routes/inference.py pass-through
- Frontend: InferenceParams type, DEFAULT_INFERENCE_PARAMS (0.0),
chat-adapter.ts payload, chat-settings-sheet.tsx slider (0-2),
model defaults loading from inference_defaults.json
- Set Qwen3.5 default presence_penalty to 1.5 per official docs
- Default for unknown models is 0.0 (off)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: fix Chat card deselecting Text and aligning with other cards
* studio: fix presence_penalty not loading from inference defaults
The inference_config.py load_inference_config() was not including
presence_penalty in the returned config dict, so the Qwen3.5
default of 1.5 from inference_defaults.json never reached the
frontend. Added it to the config builder.
* studio: add delete button for cached models in model selector
Add trash icon on each downloaded model row (GGUF and safetensors) with
confirmation dialog. Backend DELETE /api/models/delete-cached endpoint
uses huggingface_hub scan_cache_dir + delete_revisions to cleanly remove
cached repos, refusing if the model is currently loaded.
* studio: restore inference defaults, reasoning, and tools on page refresh
On page refresh with a model already loaded, the frontend was not
re-applying model-specific inference defaults (presence_penalty,
temperature, etc.) or restoring reasoning/tools support flags.
Backend: Add inference config, supports_reasoning, supports_tools,
and context_length to InferenceStatusResponse.
Frontend: In the refresh callback, when an active model is detected,
apply mergeRecommendedInference and restore reasoning/tools flags
with proper Qwen3.5 size-based defaults.
* studio: fix delete dialog closing before async completes
Prevent AlertDialogAction's default close behavior with
e.preventDefault() so the dialog stays open during deletion.
Also block onOpenChange dismiss while deleting is in progress.
* fix: add Dict and Any imports to inference models
* studio: fix Qwen3.5 reasoning threshold in frontend load path
The frontend loadModel handler had the old threshold (<=2) for
disabling reasoning on small Qwen3.5 models. Changed to <9 to
match the backend. This was causing 4B to not properly disable
thinking by default when auto-loaded.
* studio: move GGUF delete to per-variant level
For GGUF repos, the trash icon now appears on each downloaded variant
row inside the quantization expander instead of on the repo-level row.
Backend accepts optional variant param to delete specific GGUF files
(blob + symlink) rather than the entire repo cache.
* studio: restore ggufContextLength on page refresh
The Max Tokens slider was capped at 32768 on page refresh because
ggufContextLength was not restored from the status response.
Now set it from statusRes.context_length on reconnect.
* fix: remove <think> from Qwen3.5 response template marker
The train-on-responses-only feature uses template markers to find
where the assistant response starts. The Qwen3.5 response marker
included '<think>\n' which is only present when thinking mode is
enabled. With thinking disabled (default for <9B), the marker
never matched, causing 100% of samples to be dropped.
Changed response marker from '<|im_start|>assistant\n<think>\n'
to '<|im_start|>assistant\n' which works regardless of thinking mode.
* studio: fix sloth ASCII art alignment in training overlay
* fix: correct sloth ASCII art alignment to match Unsloth banner
* studio: add Python and terminal tool calling to chat
Register python and terminal tools alongside web search. Python
executor validates imports (stdlib only) via unsloth_zoo
rl_environments, runs code in a subprocess sandbox with 5-min
timeout and cancel support. Terminal executor blocks dangerous
commands (rm, sudo, etc.) and runs in a temp directory.
Update llama_cpp tool loop to show tool-specific status messages
and pass cancel_event through to executors. Rename composer
toggle from "Search" to "Tools" and show TerminalIcon for
execution status pills.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: fix Nemotron/transformers 5.x support, onboarding navigation, port binding
Backend:
- Dynamic transformers 5.x detection via tokenizer_config.json fetch
(checks for TokenizersBackend class, cached per-model)
- Bump transformers 5.x version from 5.2.0 to 5.3.0 across all workers,
setup scripts (setup.sh, setup.ps1)
- Auto-enable trust_remote_code for unsloth/* models needing transformers 5.x
(workaround for NemotronH config parsing bug in transformers)
- Auto-install mamba-ssm/causal-conv1d for SSM models (NemotronH, Falcon-H1)
with --no-build-isolation --no-deps to avoid torch version conflicts
- Add SO_REUSEADDR to port check in run.py (fixes Colab proxy stale connection
falsely reporting port as in-use)
Frontend:
- Fix "Skip to Chat" navigation: use window.location.href instead of React
Router navigate() to bypass useEffect redirect race
- Fix "Skip Onboarding" on splash: navigates to /studio (not /chat)
- Fix onboarding guard: only check isOnboardingDone() on initial mount
- Fix Chat card on step 1: add sr-only spacer for consistent alignment
- Fix Chat+Text both selected: clear RadioGroup value when Chat is selected
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: split tools toggle into Search and Code buttons
Replace the single "Tools" toggle with two independent toggles:
- "Search" (globe icon) enables web search only
- "Code" (terminal icon) enables Python and terminal execution
Add enabled_tools list field to the inference payload so the
backend only registers the tools the user has toggled on. Both
toggles appear in the main composer and the compare composer.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: fix tool calling import validation and error logging
Replace unsloth_zoo-dependent import checker with a standalone
ast-based validator using sys.stdlib_module_names. This properly
blocks non-stdlib imports (numpy, requests, etc.) and returns a
clear error message to the model so it can rewrite using only
stdlib.
Add full traceback to tool streaming error logs for debugging.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: parse gpt-oss harmony channels for clean safetensors chat output
gpt-oss models emit multi-channel output via harmony protocol tokens
(<|channel|>analysis<|message|>... and <|channel|>final<|message|>...).
TextIteratorStreamer with skip_special_tokens=True strips the special
tokens but leaves channel names concatenated with content, producing
garbled output like "analysisWe need to...assistantfinalHello!".
Add HarmonyTextStreamer that decodes with skip_special_tokens=False,
parses harmony markup via regex, and emits <think>analysis</think>
for the analysis channel and plain text for the final channel --
reusing the existing frontend reasoning UI.
Also expose supports_reasoning=True for non-GGUF gpt-oss models in
the /status endpoint so the frontend enables the Think toggle.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: use unsloth_zoo for Python sandbox validation
Set UNSLOTH_IS_PRESENT=1 and import check_python_modules and
check_signal_escape_patterns directly from unsloth_zoo instead
of a standalone fallback. This gives us the full Unsloth
validation including stdlib-only import checks and signal/timeout
escape pattern detection.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: allow all imports in Python tool sandbox
Remove stdlib-only import restriction. Keep signal escape
pattern detection via unsloth_zoo for safety.
* studio: fix ReadTimeout on tool streaming final pass
The 0.5s read timeout used for cancel-checking during streaming
also fires when waiting for the first response from llama-server
(e.g. reasoning model thinking for 15+ seconds). Add
_stream_with_retry() context manager that retries on ReadTimeout
while checking cancel_event, so the model has unlimited time to
think before producing the first token. Applied to both the
regular streaming path and the tool-calling final pass.
* fix: rewrite HarmonyTextStreamer with stateful incremental parsing
The delta-on-transformed approach had two critical bugs:
1. Before the full <|channel|>X<|message|> pattern was complete, the
strip-tokens fallback emitted "analysis" as plain text. Then when
the regex matched, _transform returned a completely different format
(<think>...</think>) and the delta was computed against the wrong
base string, producing fragments like "think>", "nk>", ">".
2. Even with full matches, the closing </think> tag shifted position
as content grew, so text[prev_len:] produced garbled deltas.
Replace with stateful incremental parsing that:
- Buffers until a complete channel+message pair is seen
- Emits <think> once when analysis channel first appears
- Streams analysis content deltas (computed on channel content directly)
- Emits </think> once when final channel first appears
- Streams final content deltas
- Closes open think tags in end()
Also skip the generic all_special_tokens stripping in
_clean_generated_text for gpt-oss since HarmonyTextStreamer already
produces clean output and the generic stripping was mangling <think>
tags.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: strip all <|...|> tokens in gpt-oss cleanup, not just harmony subset
The gpt-oss tokenizer has added tokens like <|return|> (id=200002) that
are not part of the harmony channel protocol but can leak into output.
The previous regex only stripped channel|message|start|end tokens.
Broaden the _clean_generated_text regex for gpt-oss to <\|[a-z_]+\|>
which catches all pipe-delimited tokens (return, constrain, reserved,
etc.) without matching <think>/<\/think> tags.
Verified: gpt-oss all_special_tokens are only <|return|>,
<|reserved_200017|>, <|startoftext|> -- none overlap with <think>.
The harmony tokens (channel, message, start, end) are added_tokens
but not in all_special_tokens.
* fix: hide config-only model repos from cached models list
Repos that only have metadata/config files cached (no .safetensors or
.bin weight files) were showing up in the Downloaded list with tiny
sizes like "1.8 KB" or "24 KB". These are just leftover config
snapshots from architecture checks, not usable models.
Filter the cached-models endpoint to only include repos that contain
actual model weight files (.safetensors or .bin).
* studio: fix toast description text contrast in dark mode
Add explicit !text-muted-foreground to toast description classNames
so secondary text (e.g. "Releases VRAM and resets inference state.")
is readable in dark mode.
* studio: fix Chat card icon alignment with size-4 spacer
Replace sr-only span (takes no space) with a size-4 shrink-0 div
matching the RadioGroupItem dimensions in other cards, so the Chat
icon aligns vertically with Text/Audio/Vision/Embeddings icons.
---------
Co-authored-by: workspace <user@workspace.local>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Manan17 <shahmanan170602@gmail.com>
Co-authored-by: Roland Tannous <rolandtannous@gravityq.ai>
* fix(llm_assist): disable thinking mode for helper model JSON output
Pass enable_thinking=False to generate_chat_completion() in both
_run_with_helper() and _generate_with_backend() so the Qwen3.5-4B
helper model produces clean JSON instead of wrapping responses in
<think> tags.
* fix(llm_assist): log per-request enable_thinking=False override
Add info-level log lines so the user can see that each helper/advisor
request overrides the server-level thinking default to False.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* studio: switch helper model to Qwen3.5-4B-GGUF
Replace Qwen3-4B-Instruct-2507-GGUF with Qwen3.5-4B-GGUF as the
default helper model for LLM-assisted dataset detection. Same
UD-Q4_K_XL variant.
* studio: fix stale GGUF metadata when switching models (#4347)
Reset _supports_reasoning, _supports_tools, _context_length, and
_chat_template at the start of _read_gguf_metadata() to prevent
stale settings from a previous model leaking into the next load.
Co-authored-by: Daniel Han <daniel@unsloth.ai>
* studio: change login error to "Incorrect password", add reset-password CLI
- Login error now says "Incorrect password" instead of the generic
"Incorrect username or password" since Studio only has one account.
- Add `unsloth studio reset-password` command that deletes the auth
database so a fresh admin account with a new random password is
created on the next server start.
* studio: include reset command in login error message
* studio: change password setup subtitle wording
Avaya Aggarwal