473 commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
ashzak
|
aefe904d66 |
feat(studio): implement S3 dataset loading (completes #5951) (#6222)
* feat(studio): add S3 dataset configuration foundation (#4539) Add foundational types and configuration for S3 bucket dataset loading: - Add S3Config type to frontend training types - Add S3Config Pydantic model to backend training models - Add "s3" as a DatasetSource option - Add s3Config state and setS3Config action to training config store - Add i18n translations for S3 configuration (English and Chinese) This provides the type definitions and UI text for S3 integration. Full implementation requires boto3 dependency and data loading logic. Refs: #4539 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Wire S3 config into training pipeline and prevent secrets persistence - Pass s3_config from request into training_kwargs so it flows to training subprocess - Add s3Config to NON_PERSISTED_STATE_KEYS to prevent AWS secrets from being saved to localStorage Addresses code review feedback on PR #5951. * Exclude S3 config from database persistence to protect secrets Filter out s3_config (which contains secret_access_key) from the config_json stored in training_runs table, preventing AWS credentials from being persisted to disk. Addresses P1 security feedback on PR #5951. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Re-raise HTTPException in start_training and defer s3 DatasetSource widening for PR #5951 * Redact s3_config from W&B run config and accept camelCase S3 credential aliases for PR #5951 * feat(studio): implement S3 dataset loading end-to-end Builds the actual S3 loader on top of the hardened #5951 foundation, turning the 501-gated scaffold into a working dataset source. Backend: - Add core/training/s3_dataset.py: lists and downloads supported dataset files (parquet/json/jsonl/csv) from an S3 bucket to a temp dir, using IAM-role or access-key credentials. boto3 is imported lazily (optional dep). - Wire s3_config into UnslothTrainer.load_and_format_dataset (downloads then reuses the existing local-file path) and thread it through worker.py. - Replace the 501 "not implemented" gate with a boto3-availability guard so S3 works when boto3 is present and fails clearly when it is not. - Add boto3 to studio.txt requirements. - Add tests/test_s3_dataset.py (8 tests) covering download/filtering, collisions, missing-boto3, and S3Config camelCase/IAM validation. Frontend: - Widen DatasetSource to include "s3"; add s3_config to the training payload type and mapper; add an S3 validation branch and selectS3Source store action. - Add s3-config-form.tsx (bucket/region/prefix/keys/IAM toggle) reusing the existing studio.dataset.s3.* i18n strings. - Add a Hugging Face / Local / Amazon S3 source toggle in dataset-section; the S3 config card replaces the dataset combobox when S3 is selected. - Fix DatasetPreviewDialog to accept the widened DatasetSource type. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix S3 dataset loader for PR #6222 * Fix S3 dataset edge cases for PR #6222 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix S3 IAM payload handling for PR #6222 * Block multimodal S3 datasets for PR #6222 --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com> Co-authored-by: Ash <ash@MacBook-Pro.local> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: wasimysaid <wasimysdev@gmail.com> |
||
|
Daniel Han
|
a70146df0f |
Studio: bundle Gemma 4 chat templates (E2B/E4B + larger) and auto-apply to unsloth/gemma-4-*-GGUF (#6245)
* Studio: override chat template for unsloth/gemma-4-*-GGUF with bundled gemma-4.jinja The chat templates baked into the shipped unsloth/gemma-4-*-GGUF quants predate Google's gemma-4 chat-template PR #118 and lack the preserve_thinking flag, so Studio cannot surface the "Preserve thinking" toggle for Gemma 4. Bundle the updated template and override the embedded one at llama-server launch via --chat-template-file, scoped to the gemma-4 GGUF family, so users do not need to re-download any quant. - Add studio/backend/assets/chat_templates/gemma-4.jinja (PR #118 based; preserve_thinking defaults false, the one deliberate divergence from upstream). - Add core/inference/chat_templates.py: gemma-4 GGUF matcher plus an effective-override resolver (explicit user template still wins). - Wire the resolver into routes/inference.py ahead of the reload-dedup check and both load_model calls so the live backend and the incoming request compare against the same template text (no spurious reloads). - Default preserve_thinking off in the launch-time chat_template_kwargs so direct API callers match the UI default. - Ship the asset via package-data and add unit tests. * Studio: ship E2B/E4B edge variant of the bundled Gemma 4 template Google ships two distinct gemma-4 chat templates: E2B and E4B omit the empty "<|channel>thought<channel|>" block on enable_thinking=false, while the 12b/26B-A4B/31B family emits it (confirmed against google/gemma-4-E2B-it, -E4B-it, -12b-it, -26B-A4B-it, -31B-it; the two families differ only in that one block). The single PR #118 based template followed the larger-model behavior, which is wrong for the E2B/E4B GGUFs this feature most targets. - Add studio/backend/assets/chat_templates/gemma-4-edge.jinja: identical to gemma-4.jinja minus the empty-thought-block, matching E2B/E4B behavior. - Route unsloth/gemma-4-E2B-it-GGUF and -E4B-it-GGUF to the edge template; 12b/26B-A4B/31B keep gemma-4.jinja. - Extend tests for the edge matcher, per-family routing, and the empty-thought block difference (off for edge, on for standard). * Studio: address review feedback on the gemma-4 template override - Normalize owner-less shorthand model ids in the template matcher: a bare "gemma-4-E2B-it-GGUF" is canonicalized to "unsloth/" the same way ModelConfig.from_identifier does, so shorthand loads still get the override (and the preserve_thinking capability) instead of falling back to the embedded template. - Scope the test's module stubs with unittest.mock.patch.dict instead of sys.modules.setdefault, and only stub deps that are missing, so the global module registry is not polluted for tests that run afterwards. - Guard the Jinja render tests with pytest.importorskip("jinja2") so the suite stays runnable in minimal Studio environments where jinja2 is not present. - Add tests for shorthand resolution. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: address 10-reviewer P1 findings on the gemma-4 template override - /status no longer surfaces Studio's auto-applied bundled template as a user-authored chat_template_override. The frontend adopts that field as editable state and would otherwise re-send the gemma-4 template as an explicit override for a later, unrelated model. /status now reports None when the live override equals the model's auto-resolved bundled template. - When a bundled family template is in effect, strip an inherited --chat-template-file from llama_extra_args too (not only when the raw request set chat_template_override). Otherwise a stale inherited template, appended last, shadows the bundled one while Studio reports the bundled template's capabilities. - Write the temp chat-template file as UTF-8 explicitly, and keep the bundled templates ASCII (replaced em dashes), so non-UTF-8 Windows locales cannot raise UnicodeEncodeError or emit a mis-encoded template. Added an ASCII guard test. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Daniel Han
|
90cb9499e8 |
Studio: serve DiffusionGemma with live in-place denoising and honest stats (#6250)
* Studio: serve DiffusionGemma GGUFs with the on-device visual decoder * Studio: render the DiffusionGemma denoising canvas live in chat with honest stats * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: harden DiffusionGemma runner resolution (Windows .exe, build/bin lookup, clear stale audio flag, safe PYTHONPATH, Linux-only pdeathsig) * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Daniel Han
|
240c0c3500 |
Studio: fix WSL Strix Halo GPU on reinstall (ROCDXG drop-in + system HIP before bundle) (#6227)
* install.sh: persist ROCm-on-WSL drop-in even when rocminfo already works
_maybe_bootstrap_rocm_wsl calls _ensure_rocm_probe_env (which exports a
transient HSA_ENABLE_DXG_DETECTION + adds /opt/rocm/bin to PATH on the
installer process) right before the "rocminfo enumerates gfx1151 -> already
set up, return early" gate. On any reinstall over an existing /opt/rocm --
the common case, since the uninstaller keeps shared ROCm userspace but
removes /etc/profile.d/unsloth-rocm-wsl.sh -- that probe env makes rocminfo
succeed, so the gate returns 0 WITHOUT ever persisting the drop-in. The
transient env dies with the installer, so the next login shell (Studio,
llama-server) sees no GPU: torch cuda_avail=False, rocminfo finds nothing,
the llama.cpp ROCm prebuilt segfaults on a GPU it can't reach.
Factor the drop-in writer into _persist_rocm_wsl_dropin() and call it before
the early return so the persistent env is restored whenever librocdxg is
present. Idempotent (only writes when the drop-in is missing), gated on
librocdxg so it never fires on non-WSL/non-ROCDXG hosts, root-writes or
sudo-tees like before. The fast-path branch now reuses the same helper.
Reproduced on gfx1151 (Radeon 8060S) under dash (the curl|sh shell):
before the fix a reinstall left the drop-in absent and torch cuda_avail
False; after, the drop-in is persisted and a fresh login shell reports
cuda_avail True. Verified under both dash and bash, and idempotent on
re-run.
* Studio WSL: load system HIP before a prebuilt's bundled runtime (gfx1151)
The lemonade / published llama.cpp ROCm prebuilts bundle their own HIP
runtime (libamdhip64) built for bare-metal Linux. In WSL the GPU is reached
through the system ROCm's librocdxg bridge over /dev/dxg, which the bundled
runtime cannot drive -- it segfaults on the first GPU call. So:
- install_llama_prebuilt.py: the prebuilt's llama-quantize/llama-server
validation runs with the bundle dir first on LD_LIBRARY_PATH, segfaults
(empty stderr), and the install silently falls back to a CPU source build
(which on this host can't even build for GPU -- hipcc absent). The Strix
Halo WSL user ends up on CPU despite a working GPU.
- llama_cpp.py: even if a GPU prebuilt were kept, the serve-time launcher
put the bundle dir first too, so it would crash at load.
Fix: on a ROCDXG WSL host (gated on /dev/dxg + "microsoft" /proc/version +
a librocdxg-providing /opt/rocm), prepend the system ROCm lib dir to
LD_LIBRARY_PATH so the WSL-capable libamdhip64 + librocdxg load first, while
the bundle still supplies libggml-hip / librocblas with the gfx1151 kernels.
Set HSA_ENABLE_DXG_DETECTION=1 alongside. Added _wsl_system_rocm_lib_dirs()
to both modules (kept identical so a prebuilt that passed install validation
runs the same way at serve time). Strict no-op on bare-metal Linux, NVIDIA,
macOS, and Windows.
Verified on gfx1151 (Radeon 8060S) in WSL (ROCm 7.2.1 + librocdxg, Adrenalin
ROCDXG): before, the lemonade gfx1151 prebuilt segfaulted and the install
fell back to a broken CPU build; after, install_llama_prebuilt validates and
keeps the GPU prebuilt (source=published, prebuilt_fallback_used=False), and
Studio serves Qwen3-1.7B-GGUF at 53 tok/s with the model resident in GPU
memory (llama-server device_info: ROCm0 = AMD Radeon 8060S).
* tests: cover the WSL ROCDXG drop-in + system-HIP-ordering fixes
- _wsl_system_rocm_lib_dirs: no-op without /dev/dxg, on bare-metal Linux,
and on WSL without librocdxg; returns the system lib dir on a ROCDXG WSL
host.
- binary_env: prepends the system ROCm lib dir ahead of the bundle and sets
HSA_ENABLE_DXG_DETECTION on WSL; unchanged on bare-metal Linux.
- install.sh: _persist_rocm_wsl_dropin exists, is gated on librocdxg, and the
rocminfo-already-works early return calls it before returning.
- llama_cpp.py: the serve-time launcher prepends the WSL rocm dirs before the
bundle dir (mirrors binary_env).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten WSL ROCDXG fix comments (no logic change)
Condense the drop-in / system-HIP-ordering comments and docstrings added in
this PR. Verified comment-only via AST parse + py_compile + sh/bash -n, the
308-test rocm_support suite, and a dash functional re-run of the bootstrap
(drop-in still persisted, env still set).
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
|
||
|
Tai An
|
b72cf0af24 |
fix(studio/responses): forward chat_template_kwargs enable_thinking to chat request (#6202)
* fix(studio/responses): forward chat_template_kwargs enable_thinking to chat request
The /v1/responses translation in _build_chat_request dropped
chat_template_kwargs (e.g. {"enable_thinking": true}) sent via the
Responses extra-body, so reasoning control was silently ignored.
Lift enable_thinking onto the typed ChatCompletionRequest field,
mirroring openai_chat_completions, so both the non-streaming and
streaming Responses pass-through paths honor it.
Fixes #6198
Signed-off-by: Tai An <antai12232931@outlook.com>
* Fix/adjust Responses reasoning for PR #6202
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix/adjust reasoning none for PR #6202
* Fix/adjust structured reasoning for PR #6202
* Fix/adjust responses reasoning review findings for PR #6202
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix/adjust responses reasoning follow-ups for PR #6202
* Fix/adjust think parsing gate for PR #6202
---------
Signed-off-by: Tai An <antai12232931@outlook.com>
Co-authored-by: Wasim Yousef Said <wasimysdev@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
|
||
|
Lee Jackson
|
7307fde839 |
Studio: Add custom provider option to Connections (#6112)
* feat: add custom connection * Fix custom provider handling for PR #6112 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix custom provider connection test for PR #6112 --------- Co-authored-by: wasimysaid <wasimysdev@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
oobabooga
|
72e67ae5a6 |
Studio: Add Tensor-Parallel llama.cpp support (#6040)
* Studio: Add Tensor-Parallel llama.cpp support * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: harden Tensor-Parallel fallback and GPU selection * Studio: reconcile split-mode extras and harden tensor-split planning * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: reconcile split-mode extras in backend duplicate-load guard * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: preserve inherited non-tensor split modes on reload * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: honor cancellation in tensor fallback, preserve tensor mode on rollback, and don't raise an explicit small context * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: reconcile split-mode in reload check and strip it on tensor downgrade * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Strip --tensor-split alongside --split-mode so inherited ratios don't override the tensor planner An inherited or stale --tensor-split in llama_extra_args was appended after Studio's computed --tensor-split and won last in llama.cpp, re-introducing the asymmetric-GPU OOM tensor mode is meant to prevent. Group -ts/--tensor-split into the split-mode shadow set so it is stripped on inherit and on the layer fallback; parse_split_mode_override still keys on the mode value only. * Drop quantized KV for the tensor attempt and report native max context Tensor mode aborts on a quantized KV cache, so a user with q8_0/q4_1 etc. who enabled Tensor Parallelism silently fell back to layer split. Clear the cache type (and strip inherited/explicit --cache-type) for the tensor attempt only; the layer fallback re-runs with tensor off and keeps the user's choice. Also report max_available_ctx from the native context, not an explicit small -c, so the context slider no longer warns too early in tensor mode. * Reconcile inherited split-mode extras in the already-loaded check When a same-model load omitted llama_extra_args, the tensor comparison resolved the raw (None) request and treated an inherited --split-mode tensor server as a mismatch, forcing a needless reload. Compare using the stored extras stripped the same way the reload strips them. * Pass tensor_parallel through compare-mode loads The generalized compare path loaded each GGUF without tensor_parallel, so compare ran layer split even with the toggle on and left the settings sheet stale. Send the toggle and hydrate the loaded state from the response, matching the main chat and recipe load paths. * Add --tensor-parallel flag to unsloth studio run The headless one-liner could only reach tensor mode by passing --split-mode tensor as a raw llama.cpp extra. Add a first-class --tensor-parallel/ --no-tensor-parallel option that sets the tensor_parallel field on the /api/inference/load payload, forwarded through the studio-venv re-exec like the other polarity flags. Matches the web UI toggle and the API field. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <michaelhan2050@gmail.com> |
||
|
Anmol Mishra
|
554c289538 |
fix: respect absolute export paths to prevent cross-drive copy failures (WinError 112) (#6088)
* fix: allow absolute save_directory in export paths to prevent cross-drive copy failures
The GGUF export pipeline (and all other export flows) forced every
save_directory through resolve_export_dir(), which always resolved
the path under exports_root() — typically ~/.unsloth/studio/exports/
on the system drive (C: on Windows).
When a user selected an output directory on a different drive (E:):
1. The absolute path was rejected at the Pydantic validator level.
2. Even if it got through, resolve_export_dir would re-resolve it
under C:\Users\.unsloth\studio\exports\.
3. After GGUF conversion completed on E:, the relocation step would
try to move/copy the finished files to C:, causing:
- WinError 17 (cross-drive move failure when shutil.move falls
through to a cross-filesystem copy)
- WinError 112 (disk full on C:)
Fix both layers:
- _validate_save_directory: accept absolute paths (they represent an
explicit user choice of output location).
- resolve_export_dir, resolve_output_dir, resolve_tensorboard_dir:
return absolute paths as-is instead of forcing them under the
default root. Keep the existing safety checks (null bytes, '..'
segments) and fall through to resolve_under_root for relative paths.
Fixes: https://github.com/unslothai/unsloth/issues/6082
* refactor: centralize user path validation into _resolve_user_path helper
Addresses code review feedback: the null-byte, '..', and absolute-path
checks were duplicated across resolve_output_dir, resolve_export_dir,
and resolve_tensorboard_dir. Extract a single _resolve_user_path helper
that all three delegate to.
No behavioral change — pure consolidation.
* fix: address code review — contain destructive cleanup and scope absolute paths
Address all review feedback from gemini-code-assist:
1. P1: destructive subdirectory cleanup (export_gguf)
The flattening loop in export_gguf previously rmtree'd every
subdirectory under abs_save_dir. When targeting an existing user
directory on a different drive (#6082), this could nuke unrelated
subdirectories. Now snapshot existing subdirectories before the
export and only clean up dirs created during this run.
2. P2: keep scan/read endpoints contained
Only resolve_export_dir accepts absolute paths (export is a write
path where user picks location). Reverted resolve_output_dir and
resolve_tensorboard_dir to use resolve_under_root directly — these
are used by scan/read/training endpoints that must stay contained
under their respective roots.
3. Centralization feedback
Removed the _resolve_user_path helper since it's no longer needed
with the narrowed scope. resolve_export_dir has the absolute path
logic inline with a clear docstring.
* fix: skip pre-existing subdirs in GGUF flatten loop and clean stale export intermediates
Two issues caught in code review (chatgpt-codex-connector):
1. The flattening loop moved ALL .gguf files from ALL subdirectories
into abs_save_dir, including pre-existing unrelated user subdirs.
Now skip pre-existing subdirs entirely unless they are known
export-owned intermediates (model/, model_gguf/).
2. After a failed export, known export-owned subdirectories (model/,
model_gguf/) were snapshotted as pre-existing on retry and never
cleaned up. These are now always cleaned up regardless, since they
are known intermediates created by the export pipeline.
* fix: separate write vs read export paths, guard same-dir rmtree
Three issues caught in code review (chatgpt-codex-connector):
1. P1: scan endpoint containment
resolve_export_dir was changed to accept absolute paths, but it's
also used by scan/read endpoints (routes/models.py) that must stay
contained under exports_root(). Split into:
- resolve_export_dir: contained, used by scans
- resolve_export_write_dir: accepts absolute paths, used by export
backend only
2. P1: same-directory rmtree
When a non-PEFT checkpoint's gguf_dir resolves to the same path as
abs_save_dir (user selected the checkpoint's gguf output as their
export directory), shutil.rmtree(gguf_dir) would delete the user's
chosen output directory. Now skip relocation when both paths resolve
to the same location.
3. P1: pre-existing subdir flatten loop
Reverted _EXPORT_OWNED_SUBDIRS logic — 'model/' and 'model_gguf/'
are common directory names in shared model folders and don't prove
export ownership. Now only clean up subdirs that didn't exist before
the export started.
* fix: remove dead _EXPORT_OWNED_SUBDIRS and fix _export_details for absolute paths
Two fixes from review comments:
1. Remove unused _EXPORT_OWNED_SUBDIRS declaration (leftover from
previous iteration that was intentionally removed).
2. _export_details now returns the full absolute path when the export
target is outside exports_root(), instead of truncating to basename.
Users who export to E:\ can now see the full destination path in
the success dialog.
* fix: use unique tmp dir for GGUF intermediates to avoid overwriting user dirs
When exporting to an absolute destination that already contains a
model/ subdirectory (e.g. a shared models folder), the hard-coded
model_save_path would overwrite files in that unrelated directory.
Use _tmp_model_<uuid> as the intermediate path instead, so user
directories are never touched. The tmp dir is created as a new subdir
of abs_save_dir and cleaned up by the flatten loop after GGUF files
are relocated.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix GGUF local export paths for PR #6088
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address GGUF export follow-ups for PR #6088
* Clean GGUF temp dirs on export failure for PR #6088
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix/adjust export path tests for PR #6088
* Fix/adjust export path review findings for PR #6088
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix/adjust home export path handling for PR #6088
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
|
||
|
Daniel Han
|
25ccfebc0b |
Studio: tune llama.cpp env for data-center GPUs (#6098)
* Studio: tune llama.cpp env for data-center GPUs Detect datacenter/professional NVIDIA GPUs at llama-server launch and set the llama.cpp env flags that help them, gated so consumer GeForce, AMD/ROCm, CPU and macOS are never touched. - GGML_CUDA_FORCE_CUBLAS_COMPUTE_32F=1 for any DC GPU (FP32 cuBLAS accumulation). On a B200 this is ~0% throughput cost with identical perplexity (7.3230 wikitext-2-raw, baseline and on), where on GeForce the same flag costs real throughput, hence the gate. - GGML_CUDA_P2P=1 and CUDA_SCALE_LAUNCH_QUEUES=4x for multi-GPU DC boxes. Benchmarked on 6x B200: +33-51% prompt processing on tensor (row) split and +8-16% on the default pipeline (layer) split, with no regression on the other split or on token generation. Detection uses torch device names (A100/A30/H100/H200/H800/GH200/B200/GB200/ GB300/L40/L4/RTX PRO 6000/RTX 6000 Ada). A mixed box with one consumer GPU in the selection is treated as non-DC. All writes are setdefault so a user value always wins, and UNSLOTH_DISABLE_DC_TUNING=1 turns the whole thing off. 37 unit tests cover detection, multi-GPU gating, user-override precedence, the disable flag and fail-open on error. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: fix data-center GPU detection false positives and physical-id mapping Two issues in the data-center llama.cpp env tuning gate: - _is_datacenter_gpu matched the marker allowlist as unbounded substrings, so workstation/laptop parts "NVIDIA RTX A1000" and "NVIDIA RTX A3000" matched "a100"/"a30" and were wrongly tuned as data-center GPUs (forcing FP32 cuBLAS accumulation and the multi-GPU env, which carry a real cost on those cards). Switch to a word-boundary regex. - gpu_indices carries physical GPU ids (translated from torch ordinals by _get_gpu_free_memory via CUDA_VISIBLE_DEVICES), but they were passed straight into torch.cuda.get_device_properties, which expects mask-relative ordinals. On a masked host (e.g. CUDA_VISIBLE_DEVICES=4,5,6,7) a selection like [4,5] fell out of range and silently dropped the tuning, and on a mixed mask it could probe the wrong GPU class. Build a physical-id to device-name map mirroring _get_gpu_free_memory, then look up the selection by physical id. Add regression tests for the A1000/A3000 false positives and for masked-host physical-id selection (reordered and mixed-class masks included). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: tighten data-center GPU tuning comments Comment-only pass over the DC tuning block and its tests: shorten verbose docstrings/comments, drop ones that restate the code, collapse multi-line blocks. Keep the load-bearing rationale (physical-id vs ordinal mapping, the word-boundary reason, the B200 benchmark numbers). No code change: verified with comment_tools.py check --strip-docstrings (code unchanged, comments only). --------- Co-authored-by: danielhanchen <michaelhan2050@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Irakli
|
95a2627bf6 |
Fix step count mismatch when sequence packing is enabled (#5967)
* Fix step count mismatch when sequence packing is enabled * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Emit a single step-0 progress event and guard applyStatus totalSteps Merge the two consecutive _update_progress calls before train() so the step-0 gate in _on_progress fires once instead of twice, avoiding a duplicate startup event and a null-metric step-0 row in training_metrics. Apply the same positive-number guard to applyStatus that applyProgress uses, so a stale or startup status poll can no longer overwrite the packed step count with 0 or replace it with a stale total. * Log debug message when train_dataset length is unavailable The TypeError fallback for length-less datasets (e.g. streaming IterableDataset) was silent, leaving no trace that the step estimate came from the raw dataset rather than the packed one. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Etherll <61019402+Etherll@users.noreply.github.com> |
||
|
oobabooga
|
7f2986a413 |
Studio: Add inline confirmation (Allow/Always allow/Deny) for tool calls (#5869)
* Studio: Add inline confirmation (Allow/Always allow/Deny) for tool calls * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix race in tool-call confirmation gate * Studio: gate built-in tool calls and harden the confirmation handshake The Allow / Always allow / Deny controls only lived in the fallback tool card, but the built-in tools (web search, python, terminal, code execution, image generation) render with their own components and so never showed the buttons. Those calls paused after tool_start with no way to approve them, hanging until the 1 hour timeout. Only MCP tools, which use the fallback renderer, actually worked. Render the controls for every tool card by wrapping each registered tool component (and the fallback) in thread.tsx with a shared ToolConfirmationControls, so the gate applies uniformly. Also make the handshake robust: - The gate keys on a per-call approval_id minted by the backend and echoed in tool_start, instead of session_id alone, so a stale or concurrent confirmation can no longer resolve the wrong call. - The approval slot is registered before tool_start is yielded, closing the race where a fast click or an auto "Always allow" could reach the backend before the waiter existed. - The frontend resolves with the same session id the request was sent with (plus the approval_id), fixing the new-thread mismatch where the confirmation targeted a different session than the blocked stream. - The confirm endpoint returns {resolved}; the UI keeps the buttons and shows a retry hint until the backend confirms a match, instead of hiding them on a failed or mistargeted post. - The gate runs after the disabled-tool and duplicate-call checks, so a call that will not execute is not put up for approval. A denied call is still excluded from duplicate detection, so re-issuing and approving it works. - "Always allow" is scoped per session to match the backend gate. Add backend tests for the approval registry, the SSE no-deadlock handshake, and the loop integration (allow, deny, disabled, duplicate, re-issue after deny). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Move "Confirm tool calls" to the Tools section * Studio: Keep tool group open while a tool call awaits confirmation * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix tool confirmation session scope for PR #5869 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix confirmation follow-ups for PR #5869 * Apply pre-commit formatting for PR #5869 * Fix confirmation cleanup for PR #5869 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Harden confirmation lookups for PR #5869 * Studio: make the tool-call confirmation decision immutable resolve_tool_decision accepted a second confirmation for the same approval_id and overwrote slot["decision"] in the window before the waiter reads it and pops the slot, so a duplicate or out-of-order POST could flip an Allow to Deny (and returned a misleading resolved:true). Reject once the slot's event is already set so the first decision wins. Adds a regression test. * Fix/adjust tool confirmations for PR #5869 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com> Co-authored-by: wasimysaid <wasimysdev@gmail.com> |
||
|
James Dawdy
|
f22e890ab8 |
fix(studio): inherit llama_extra_args and honor --no-mmproj (#5902)
* fix(studio): inherit llama_extra_args and honor --no-mmproj Reloading the same GGUF from the UI without gguf_variant no longer drops CLI pass-through args like --no-mmproj. Skip mmproj download and launch when --no-mmproj is present in llama_extra_args. Co-authored-by: Cursor <cursoragent@cursor.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix(studio): tighten GGUF llama_extra_args variant inheritance guard Reject inherited CLI args when the request changes gguf_variant or when omitted variant resolves differently from the stored extra_args source. Co-authored-by: Cursor <cursoragent@cursor.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Treat --no-mmproj-auto and --mmproj-auto with last-wins parsing for PR #5902 --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com> |
||
|
alkinun
|
672d8f0581 |
Expose runtime context length for hub models (#6154)
* expose runtime context length for hub models * runtime context helper review * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: Etherll <61019402+Etherll@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Viktor Ferenczi
|
120daf9d8b |
fix(studio/rocm): don't stack ROCR_VISIBLE_DEVICES on HIP_VISIBLE_DEVICES (#6176)
When pinning GPUs for the llama-server child, the ROCm path set both
HIP_VISIBLE_DEVICES and ROCR_VISIBLE_DEVICES to the same physical
indices. These masks filter at different layers and stack:
ROCR_VISIBLE_DEVICES reduces the visible set at the HSA/ROCr layer and
re-indexes from 0, then HIP_VISIBLE_DEVICES indexes into that reduced
set. _select_gpus ranks by free VRAM and picks the most-free card, so a
single non-zero pin (e.g. "1") becomes out of range at the HIP layer,
HIP enumerates 0 devices, and the model silently runs on CPU
("ggml_cuda_init: failed to initialize ROCm: no ROCm-capable device is
detected").
Set only HIP_VISIBLE_DEVICES (which narrows correctly on its own) and
clear any inherited ROCR mask so it can't double up.
Verified on a 2x Radeon AI PRO R9700 (gfx1201) host, ROCm 7.1.1: the
same selected=[1] load that fell back to CPU (~7.7 tok/s) now runs on
the GPU (~78 tok/s).
Fixes #6175
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
|
||
|
Nilay
|
004577c9cd |
studio: show MCP "Import config" on the add-server form (#6030)
* studio: import MCP servers from a config file * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * import config' on the add-server form * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: defensively handle MCP config imports * fix: address MCP import review follow-ups * fix: preserve apostrophes in Windows MCP commands * fix: preserve apostrophe-wrapped Windows MCP args * fix: align Windows MCP parsing with list2cmdline * fix: preserve explicit MCP remote transport intent * fix: trim MCP remote URLs before transport checks --------- Co-authored-by: Roland Tannous <rolandtannous@gravityq.ai> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Roland Tannous <115670425+rolandtannous@users.noreply.github.com> Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com> Co-authored-by: imagineer99 <samleejackson0@gmail.com> |
||
|
Daniel Han
|
3733e0b274 |
fix(studio): surface live step with null loss through the SSE progress stream (#6206)
* fix(studio): surface live step with null loss through the SSE progress stream The metric histories skip non-finite steps, so during a NaN stretch the SSE live loop and final complete event replayed the last finite step/loss pair. Follow the live progress step when it is ahead of the history tail and report its loss honestly (null until recovery). Completes the NaN honesty fix for the SSE consumer flagged in review. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Apply live-step handling to inactive streams and clear the UI loss on null for PR #6206 Fresh /progress connections after a finished run took the inactive branch which still replayed the last finite step and loss pair; apply the same live-step correction there. On the frontend, applyProgress kept the stale currentLoss when a payload advanced the step with a null loss; clear it so the display shows -- until the loss recovers. Widen the runtime state type to number | null, which the view layer already handles. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Daniel Han
|
bc85ecd145 |
Studio: report the real llama-server context window and add an opt-in overflow policy for OpenAI-compatible serving (#6164)
* Studio: report the real llama-server context window and add an opt-in overflow policy for OpenAI-compatible serving A community report showed OpenCode failing tool calls every few minutes against Studio's OpenAI-compatible API while the same GGUF was stable on LM Studio. Root cause: Studio advertises the requested context length, but llama-server can allocate less (memory-fit step on small GPUs, --parallel slot split), so clients budget against a window that does not exist. Their generations truncate mid tool call at the real wall (finish_reason=length with cut JSON arguments) and eventually the prompt itself exceeds the real window, returning a 400 that agentic clients treat as non-retryable. Changes: - After llama-server health, read default_generation_settings.n_ctx from /props and adopt it whenever it is below Studio's computed context, with a warning. The load response, status route, UI value, and the passthrough max_tokens ceiling all become honest automatically. - Expose context_length and max_context_length on /v1/models so clients can budget against the enforced window. - Accept empty role=tool content (commands with no output are routine in agentic loops; OpenAI and llama-server both accept it) instead of a 400. - Add context_overflow=truncate_middle (per request, or server-wide via UNSLOTH_CONTEXT_OVERFLOW=truncate_middle): on exceed_context_size_error the passthrough drops whole middle turn-groups (system prompt, first turn, and recent turns kept; tool calls stay paired with their results), clips oversized contents middle-out when group-dropping is not enough, clamps max_tokens to the generation headroom, and retries. Default stays 'error' with code=context_length_exceeded so clients running their own compaction keep full control. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: allocate the requested context for real (kv-unified, fit-ctx floor) Two launch-flag gaps caused the advertised vs allocated divergence at the source: - llama-server enables --kv-unified only when the slot count is auto; Studio always passes --parallel N, which silently splits -c into per-slot windows of -c/N. Pass --kv-unified when N > 1 so a single request can use the full advertised window (same total KV memory, shared pool). - with --fit on the fit step may set ctx as low as 4096; pass --fit-ctx <requested> for explicit requests so fit offloads or fails into the existing --fit off retry instead of silently shrinking the window. Both flags are gated on --help capability probing so older builds keep the current behavior, where the /props readback remains the backstop. Verified live: -c 98304 --parallel 4 now serves per-slot n_ctx 98304 (was 24576), 48k-token requests pass through the passthrough, and the readback warning no longer fires. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
BardiaKoopah
|
0ac051177a |
fix(mlx): forward resume_from_checkpoint to MLXTrainer.train() (#6173)
Studio's frontend exposes a Resume action and submits requests with resume_from_checkpoint set to a previous run's output_dir. The CUDA training paths in worker.py read this field from config and pass it to trainer.train() (see lines 2729-2787 and 3108-3229). The MLX path _run_mlx_training did neither: it never read config['resume_from_checkpoint'] and called trainer.train() with no args. The MLX trainer also did not accept the kwarg, so even threading it through would have been a no-op. With this PR + the unsloth-zoo companion PR adding the trainer-side support (saves optimizer_state + trainer_state, accepts and applies resume_from_checkpoint in MLXTrainer.train()), MLX Resume now works end-to-end. Verified on M2 16GB with Qwen3-0.6B + unsloth/LaTeX_OCR: loss at every post-resume step matches a fresh run bit for bit (2.168627977371216 == 2.168627977371216 at step 6, etc). Two lines: read the field near the other config.get() extractions in _run_mlx_training, pass it as a kwarg at the trainer.train() call site. Companion PR: unslothai/unsloth-zoo#751 |
||
|
BardiaKoopah
|
14f679da8a |
fix(studio): surface NaN loss honestly instead of laundering to last finite value (#6016)
When training produced a NaN or Inf loss event, the handler filtered the value to None but never updated progress.loss — clients kept seeing the last finite value as if everything were fine. Now: on non-finite loss, clear progress.loss to None and log a one-shot warning. Training continues (no phase=error, no _should_stop), matching the expected behavior for a non-fatal numerical event. Test: tests/test_training_nan_loss_handling.py with 6 cases covering finite, NaN, +/-Inf, idempotency of the one-shot warning, and recovery when a finite step follows a non-finite one. |
||
|
Daniel Han
|
73973435ad |
Studio: require an installed ROCm DLL before forcing BNB_ROCM_VERSION; drop shadowing shutil imports in save.py (#6194)
* Require a found ROCm DLL before forcing BNB_ROCM_VERSION in Studio paths main.py previously set BNB_ROCM_VERSION=72 whenever HIP_PATH or ROCM_PATH was set, and the training worker fell back to a blind 72 when DLL detection found nothing. On a Windows machine with the AMD HIP SDK installed but CUDA or CPU torch, that forces a ROCm backend onto a non-ROCm bitsandbytes wheel, which raises at import. Both paths now only write the override when a libbitsandbytes_rocm DLL actually exists (or a seeded value is already present), matching the strict gates in unsloth/import_fixes.py. Also removes four redundant local import shutil statements in unsloth/save.py that shadow the module-level import, the same pattern that caused the UnboundLocalError fixed in #6149. * Worker: gate the BNB override on a found ROCm DLL, preserving seeded marker Review follow-ups: track _found_rocm_bnb in the worker like main.py so a ROCm DLL with an unparsable name still gets the seeded or 72 fallback, and skip the env write entirely when no DLL exists so a seeded value keeps its sitecustomize marker and stays redetectable by later import fixes. |
||
|
Daniel Han
|
a5d6e6928d |
Studio: surface the llama.cpp update affordance when MTP is disabled (#6192)
* Studio: surface the llama.cpp update affordance when MTP is disabled When a model asks for MTP (auto on an MTP model, or forced mtp / mtp+ngram) but it gets disabled, the load already degrades gracefully and serves without speculative decoding. Until now the UI gave no hint why, or that an update would fix it. Record why MTP was dropped on the backend (spec_fallback_reason): the probe found no mtp token (binary_no_mtp), the spawn aborted with an outdated-arch / context-build error such as a prebuilt that predates the Gemma drafter (binary_outdated), or the current build could not run it, e.g. a CUDA kernel limit (runtime_error). Expose it in the inference status. In the chat Speculative Decoding section, show a short note and, for the two update-fixable reasons, an inline Update llama.cpp button that reuses the existing update flow. A runtime_error gets the note without an update push, since a newer build may not fix it. Backend tests cover the reason being set / cleared. Frontend typechecks. * Address review: tighten the update hint to genuinely outdated binaries Reserve binary_outdated (which surfaces the Update llama.cpp affordance) for an unknown-architecture abort, which proves the prebuilt predates the model; classify the generic memory/context build failures as runtime_error, where an update may not help. Frontend: only append the "Update llama.cpp to enable it" sentence when an update is actually available, so the text never points at an action the UI is not offering. |
||
|
Daniel Han
|
c3604d01f7 |
Studio: enable MTP for sub-3B Gemma separate-drafter GGUFs (#6191)
* Studio: enable MTP for sub-3B Gemma separate-drafter GGUFs The sub-3B auto-drop to ngram-mod was tuned for an embedded draft head (Qwen), whose per-token cost regresses below 3B. Gemma ships the head as a separate root mtp-*.gguf drafter, a tiny standalone model that is cheap enough to win below 3B: B200 Q4_K_XL bench, draft-mtp n=2 vs spec-off, gemma-4-E2B (2B) = 1.21x (accept ~0.65) while ngram-mod is 1.00x. Exempt a separate drafter from the sub-3B gate everywhere the threshold is applied: the resolver (_mtp_too_small), the auto-fit VRAM reserve, the drafter auto-download decision, and the reload-skip mirror via a has_separate_drafter flag on _auto_mode_drops_mtp. Embedded sub-3B heads (Qwen) still drop to ngram-mod. A drafter the binary cannot build (older prebuilt, or a CUDA kernel limit) still aborts the spawn and the load retries once without speculative decoding. Adds the full Qwen3.5 + Gemma-4 (regular and QAT) auto/off/forced resolver matrix, plus explicit sub-3B exemption tests. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Always compare the separate drafter in the reload-skip mirror The sub-3B wrapper around the drafter compare could skip it when the drafter was deleted out from under a running sub-3B server (detected None, stored set), leaving a stale launch. The resolved-path compare is cheap and already handles every case, so drop the _auto_mode_drops_mtp guard (and its now-unused imports) and always compare when the mode can use a drafter and the user does not own --spec-type. Addresses review feedback on #6191. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Lee Jackson
|
49024462f3 |
Studio: ignore unsupported env proxy during Studio startup (#6102)
* fix: ignore unsupported env proxy during Studio startup * fix: handle missing socksio env proxy at startup * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Match printf logging style and inline the proxy predicate for PR #6102 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com> |
||
|
Daniel Han
|
50e4e9c459 |
Studio: gracefully disable MTP when the model has no head or drafter (#6159)
* Studio: gracefully disable MTP when the model has no head or drafter
Selecting MTP or MTP+Ngram in Speculative Decoding on a GGUF with no nextn
head and no separate drafter aborted the whole load. llama-server does not
no-op an empty draft-mtp request: it exits with 'failed to measure MTP
context memory: failed to create llama_context', surfaced to the user as a
generic 'llama-server failed to start. Check that the GGUF file is valid
and you have enough memory.'
Build-time fix in _build_speculative_flags: when a forced mtp / mtp+ngram
mode targets a model with no MTP head and no drafter (is_mtp_model is
False), default back instead of emitting draft-mtp. mtp falls back to
--spec-default; mtp+ngram keeps the ngram-mod half, which needs no head.
Real MTP models (embedded head or separate drafter), sub-3B MTP overrides,
and the auto path are unchanged.
Runtime hardening: the existing post-launch MTP retry only fired for
separate-file drafters (--model-draft in spec_flags), so an embedded-head
model that the binary cannot build still hard-failed. Gate the retry on the
spec block requesting MTP, recognise the embedded-head abort strings
('failed to measure MTP context memory', 'failed to create llama_context'),
and make the drafter name None-safe in the warning.
Tests: extend the resolver matrix (forced mtp / mtp+ngram on a non-MTP
model) and add two cases asserting the default-back emission.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
|
||
|
Daniel Han
|
dab0b77673 |
Studio: in-app Update llama.cpp button to install the latest prebuilt (#6097)
Adds an in-app "Update llama.cpp" banner and button to Unsloth Studio. When the installed prebuilt is behind the latest published release, a non-invasive banner appears; clicking Update downloads the latest prebuilt for this host and swaps it in place in the background, with no restart. Detection reuses the freshness check from #5529. The update re-runs install_llama_prebuilt.py the same way setup.sh and setup.ps1 do after #5963: it forwards the published repo and the AMD gfx target derived from the install marker, and does not pass the removed --simple-policy or the arm64-only --cpu-fallback. While the installer swaps binaries the backend enters a maintenance state (flag set under the serial load lock, active server unloaded) so a concurrent load cannot start a server from a half-swapped binary; the next load uses the new build. The banner also handles refused responses and jobs started in another tab so it never sticks on "Updating...". Verified end to end on an NVIDIA B200: installed b9493, detected the update, applied it, and confirmed the binary at the same path advanced to b9585 in the same process. Hermetic backend tests and the frontend type-check pass. |
||
|
Daniel Han
|
b21c419a51 |
Studio: fall back to text-only when llama.cpp is too old for a model's vision projector (#6075)
* Studio: fall back to text-only when llama.cpp is too old for a model's vision projector Loading a GGUF vision model starts llama-server with --mmproj <projector>. When the installed llama.cpp prebuilt predates the model's projector format, llama-server aborts at startup with 'clip.cpp: Unknown projector type' (exit -6), and the whole load failed even though the base GGUF is a fine text/tools chat model. Seen with gemma-4 on a 3-day-old prebuilt (build b9496). load_model now retries the launch once without --mmproj when the captured startup output indicates a projector-format incompatibility. The retry runs the model text-only, marks the session non-vision (is_vision False, mmproj audio dropped) so the status/capabilities the frontend reads stay consistent, and warns the user to update llama.cpp. Detection is generic, not model-specific, and conservative: OOM, bad GGUF, port-bind, missing-file and other failures keep their existing handling and never retry. If the text-only retry also fails, it errors out with the real reason. Adds _is_projector_incompatibility and _strip_mmproj_args (unit-tested with the real gemma-4 abort plus negatives) and extracts _start_llama_process so both the initial start and the retry share one spawn path and each logs its argv. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Reformat mmproj fallback files to match main (ruff line-length 100 + kwarg spacing) --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: danielhanchen <michaelhan2050@gmail.com> |
||
|
oobabooga
|
2b319e8d3a |
Studio: support separate-file MTP GGUF drafters (Gemma 4) (#6125)
* Studio: support separate-file MTP GGUF drafters (Gemma 4) * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: fix review findings for separate-file MTP drafters * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: pair local MTP drafters by name and include them in reload dedup * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: manage --model-draft in extras and reject MTP/ copies as models * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Lee Jackson
|
0f00bc1e2a |
Studio: fix Gemma-4-12B-it not loading (#6054)
* Fix Studio Python, Gemma 4 Unified sidecar, and worker crash messages * Clean up Gemma 4 sidecar test patch contexts * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Polish inference worker crash message * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Address transformers tier review feedback * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Route Gemma 4 assistant models to transformers 5.10 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Daniel Han
|
6e057ffebe |
Studio: training survives a non-writable HF datasets cache (#6148)
* Studio: training survives a non-writable HF datasets cache A shared HF datasets cache can contain subtrees owned by another user (for example populated by an earlier root-run job). datasets then dies with "[Errno 13] Permission denied: ..._builder.lock" while locking the cached builder and the training run fails. load_dataset in the training worker and trainer now goes through a wrapper that catches the EACCES and rebuilds the dataset in a Studio-owned cache under cache_root()/hf-datasets, logging the fallback. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Scope the HF_DATASETS_CACHE override to the fallback load * Route non-streaming dataset preview loads through the cache-safe wrapper --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Long Yixing
|
bc8b9dc917 |
fix(studio): forward mlx custom format mapping (#6094) | ||
|
Long Yixing
|
154457dbf7 |
fix(studio): reject unsupported mlx training modes (#6091) | ||
|
Long Yixing
|
8e2a983f58 |
Fix Studio MLX VLM resized image layout (#6019)
* fix(studio): infer mlx vlm resized image layout * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Bubu
|
8af9fe63a3 |
fix: persist Windows ROCm BNB version (#6048)
* fix: persist Windows ROCm BNB version * style: apply kwarg spacing hook * fix: avoid persisting caller ROCm overrides * fix: redetect managed BNB ROCm defaults * style: apply ROCm guard test formatting --------- Co-authored-by: Daniel Han <danielhanchen@gmail.com> |
||
|
Daniel Han
|
62191c4765 |
Windows/WSL installer: fix winget msstore cert failure, amd-smi DiskPart prompt, and enable AMD GPU (Strix Halo gfx1151) (#5940)
* Fix Windows installer winget msstore certificate failure
`winget install` was invoked without `--source winget`, so winget also
queried the msstore source. When msstore fails certificate pinning
(error 0x8a15005e, "The server certificate did not match any of the
expected values") winget aborts and demands `--source`, so the Python
(and uv) install fails even though the package exists in the winget
source.
- Pass `--source winget` to all winget install calls (Python x2, uv).
Both packages live in the winget source, so this is strictly correct
and skips the failing msstore round-trip entirely.
- Add a python.org fallback (Install-PythonFromPythonOrg) that downloads
the official installer and runs it silently per-user (no admin/UAC)
when winget is unavailable or fails for any reason. Mirrors the
existing uv -> astral.sh fallback so Python installs without manual
steps. Resolves the latest 3.13.x from python.org with a pinned
fallback, and selects the amd64/arm64/x86 installer per architecture.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Pin remaining setup.ps1 winget calls to --source winget
Two winget invocations in studio/setup.ps1 still queried all sources and
could hit the same msstore certificate-pinning failure (0x8a15005e) that
broke the Python install in install.ps1:
- `winget show Nvidia.CUDA --versions` (CUDA Toolkit version probe)
- `winget install ... ShiningLight.OpenSSL.Dev` (OpenSSL dev for llama-server)
Every other winget call in this file already passes `--source winget`
(Git, CMake, VS Build Tools, CUDA install, Node.js, and setup.ps1's own
Python 3.12 install), so these two were stragglers. Both packages live in
the winget source; pinning it makes setup robust to an unhealthy msstore
source, matching the rest of the file.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Stop amd-smi GPU probe from popping a DiskPart UAC prompt
On Windows, AMD GPU detection in install.ps1 and studio/setup.ps1 runs
`amd-smi list` / `static --asic` / `version`. amd-smi (shipped in
System32 by the Adrenalin driver) auto-elevates to read GPU/APU memory
details, surfacing a confusing DiskPart UAC prompt mid-install. The
Studio backend already documents and circuit-breaks on this in
studio/backend/utils/hardware/amd.py, but the installers did not.
Add an Invoke-AmdSmiNoElevate helper (both scripts) that runs amd-smi via
Start-Process under __COMPAT_LAYER=RunAsInvoker so it cannot auto-elevate
(no prompt), with a 30s timeout (matching amd.py) so a flaky amd-smi
cannot stall the install for minutes. On failure/timeout the existing WMI
name -> gfx fallback still resolves the arch, so detection is unchanged on
working hosts.
Verified on a Strix Halo (Radeon 8060S / gfx1151) box: the prompt is gone
and the probe is bounded.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Add experimental ROCm-on-WSL setup helper for Strix Halo (gfx1151)
install.sh already routes gfx1151 (Radeon 8060S / Strix Halo) to the
repo.amd.com/rocm/whl/gfx1151 wheels once a ROCm runtime is present, but
it does not install AMD's driver/ROCm stack -- a large, admin-gated
prerequisite. scripts/install_rocm_wsl_strixhalo.sh automates the Linux
side on a dedicated Ubuntu 24.04 WSL2 distro: ROCm 7.2 (wsl usecase), the
rocr4wsl HSA runtime, a librocdxg build, env setup, and a PyTorch gfx1151
GPU smoke test. A hard preflight refuses to run until the Adrenalin
>=26.3.1 driver is actually present, so it cannot half-install.
Procedure adapted from AMD's ROCm-on-WSL docs and community gfx1151 notes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Detect AMD GPUs by name so native Windows gets a GPU llama.cpp
The gfx-arch inference from the WMI GPU name was gated behind $HasROCm,
which the hipinfo/amd-smi probe leaves false on the common Windows case
(Adrenalin driver only, no HIP SDK -- and amd-smi often cannot read the
arch without elevation). So an AMD GPU was detected by name but never
mapped to a gfx target, --rocm-gfx was not forwarded, and studio setup
fell back to a CPU llama.cpp build.
Un-gate the inference (install.ps1 + studio/setup.ps1) so it runs whenever
an AMD GPU name is available. The inferred gfx is forwarded as --rocm-gfx,
which makes install_llama_prebuilt.py download the matching lemonade-sdk
ROCm prebuilt (e.g. llama-bNNNN-windows-rocm-gfx1151-x64.zip) -- a
GPU-accelerated llama.cpp that bundles its own ROCm runtime, so it runs
with just the Adrenalin driver. PyTorch's ROCm wheels still require a
confirmed HIP SDK ($HasROCm), so this only affects llama.cpp / inference
and never pulls broken ROCm torch.
Also broaden the name->arch table to every family lemonade ships Windows
assets for: gfx120X (RDNA 4), gfx110X (RDNA 3), gfx1151/gfx1150
(RDNA 3.5), and gfx103X (RDNA 2). Unknown names still fall back to CPU.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Suppress amd-smi DiskPart UAC prompt in the Python install/runtime paths
The earlier PowerShell guard covered install.ps1 / setup.ps1, but the
Python installer (install_llama_prebuilt.py detect_host,
install_python_stack.py ROCm probes) and the Studio backend monitor
(amd.py) also shell out to amd-smi on Windows, where it auto-elevates and
pops the same DiskPart UAC prompt mid-install / at runtime.
Inject __COMPAT_LAYER=RunAsInvoker into the amd-smi subprocess env on
Windows so it runs un-elevated (no prompt). Callers already tolerate an
empty/failed result and fall back to WMI / name detection (installer) or
the existing circuit breaker (amd.py). Gated to Windows so Linux/macOS
amd-smi behaviour is unchanged.
- install_llama_prebuilt.py: handled centrally in run_capture (covers
detect_host's `amd-smi list` and the version probe).
- install_python_stack.py: new _amd_smi_env() helper on its 3 raw
subprocess.run amd-smi calls.
- amd.py: merge RunAsInvoker into the existing child env.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Tighten AMD GPU name->arch patterns to avoid mismatches
The W9[0-9]{3} and RX 90[0-9]{2} patterns added for RDNA 4 were
speculative and over-broad: W9xxx would also match old GCN FirePro
W9100/W9000 cards (wrong gfx1201 -> a lemonade gfx120X download that
fails validation), and RX 90[0-9]{2} was redundant with the explicit
9070/9060 entries. Drop both; keep only confirmed RDNA 4 SKUs. Unmatched
AMD names still fall back cleanly to CPU.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Fetch the llama.cpp validation model via huggingface_hub
The prebuilt validation downloads a tiny GGUF test model from huggingface
via bare urllib. On Windows / proxy setups where the server sends an
incomplete TLS chain, urllib cannot complete the Amazon CA chain (it does
no AIA intermediate fetching) and fails with CERTIFICATE_VERIFY_FAILED, so
a perfectly good GPU prebuilt is rejected and the installer falls back to a
CPU source build.
Route the validation-model download through huggingface_hub
(hf_hub_download) -- the same mechanism Studio uses for model downloads,
which completes the chain where urllib cannot -- keeping the direct URL as
a fallback. This lets the lemonade ROCm prebuilt validate and install on
cert-restricted machines (verified: hf_hub_download succeeds where urllib
returns CERTIFICATE_VERIFY_FAILED).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Guard the remaining raw amd-smi version probe via run_capture
A ROCm-version detector in install_llama_prebuilt.py called amd-smi version through a raw subprocess.run that bypassed run_capture's Windows RunAsInvoker guard, so it still triggered the DiskPart UAC prompt during setup. Route it through run_capture like the other amd-smi calls.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Forward --rocm-gfx even when the ROCm runtime is unconfirmed
setup.ps1 forwarded --rocm-gfx (and picked the windows-hip llama.cpp
prebuilt) only inside `if ($HasROCm)`. On Adrenalin-only hosts (amd-smi
present but no HIP SDK, so $HasROCm stays false) the gfx arch was
name-inferred but never forwarded, so install_llama_prebuilt.py saw
has_rocm=False and installed the CPU build -- even though the lemonade
gfx1151 GPU prebuilt runs fine there (it bundles its own ROCm runtime;
verified: llama-cli --list-devices -> ROCm0: AMD Radeon 8060S, 69 GB).
Forward --rocm-gfx whenever a gfx arch is known (it is authoritative and
implies ROCm in install_llama_prebuilt.py), and treat a known gfx arch as
windows-hip in the existing-install mismatch check. --has-rocm stays gated
on the confirmed-runtime signal.
Verified on Radeon 8060S / gfx1151: the installer now selects, validates,
and installs llama-b1286-windows-rocm-gfx1151-x64.zip (ROCm DLLs present)
instead of the CPU build.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Install AMD ROCm PyTorch on name-inferred gfx hosts (enables Train/Export)
setup.ps1 picked the AMD ROCm PyTorch wheels only inside `if ($HasROCm ...)`.
On Adrenalin-only hosts (amd-smi present but no HIP SDK, so $HasROCm is
false) the gfx arch was name-inferred but the ROCm-wheel branch never ran,
so the host got torch+cpu. With CPU torch, torch.cuda.is_available() is
False, so the Studio backend sets CHAT_ONLY=True and hides Train/Export.
Un-gate the ROCm PyTorch index resolution on a known gfx arch (mirrors the
llama.cpp --rocm-gfx fix). AMD's per-arch Windows wheels
(repo.amd.com/rocm/whl/<gfx>) bundle the ROCm runtime, so they work without
a HIP SDK; a failed install still falls back to CPU.
Verified on Radeon 8060S / gfx1151: torch 2.11.0+rocm7.13.0 installs and
torch.cuda.is_available() -> True, device "AMD Radeon(TM) 8060S Graphics",
GPU matmul OK -> CHAT_ONLY=False -> Train/Export enabled.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Force amd-smi un-elevated process-wide in the Python installers
Guarding individual amd-smi call sites kept missing some (install_python_stack.py's probe loop and its Windows GPU re-check), so the DiskPart UAC prompt kept reappearing. Set __COMPAT_LAYER=RunAsInvoker process-wide at the top of install_python_stack.py and install_llama_prebuilt.py on Windows so every amd-smi subprocess (current and future) runs un-elevated with no per-call guard. Safe: these scripts only spawn amd-smi/rocminfo/hipinfo probes and pip/uv. setup.ps1 keeps per-call guards because it also spawns winget installers that need elevation.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Fix Invoke-AmdSmiNoElevate exit code on PS 5.1 + RX 7700S arch match
Start-Process -PassThru leaves the returned process object's .ExitCode
$null after WaitForExit on Windows PowerShell 5.1, so the helper set
$LASTEXITCODE to $null and every caller's `if ($LASTEXITCODE -eq 0 ...)`
was always false -- the amd-smi GPU / gfx-token / ROCm-version detection
branch was effectively dead (masked only because the un-gated WMI
name->gfx inference still ran). Reproduced on PS 5.1.26100.
Rewrite the helper to use [System.Diagnostics.Process]::Start with a
ProcessStartInfo (UseShellExecute=false), whose .ExitCode is reliable,
with async stream reads (ReadToEndAsync) to avoid a pipe-buffer deadlock
and WaitForExit(timeout) to bound a flaky amd-smi. __COMPAT_LAYER=
RunAsInvoker (inherited via the process env) still suppresses the
auto-elevation / DiskPart prompt. Also drops the temp files and the
empty-ArgumentList edge case. Verified: exit code propagates
(7 -> $LASTEXITCODE=7), output captured, env restored.
Also fix the gfx1100 name pattern `RX 7700(?! S)` -> `RX 7700(?!S)` so the
spaceless retail name "RX 7700S" is correctly excluded (it belongs to the
gfx1102 row). Both found by PR review.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Address PR review follow-ups (install.sh table, update path, tests, WSL)
From the multi-agent PR review:
- install.sh: sync the AMD name->arch table with install.ps1 / setup.ps1
(the bash table had drifted to the old narrow patterns). Adds RDNA 2
(gfx103X), workstation PRO W SKUs, and more Strix Halo/Point names, and
orders gfx1102 before gfx1100 so the spaceless retail name "RX 7700S"
resolves correctly (bash case has no negative lookahead). AMD-ROCm-only:
the name inference stays gated behind _has_amd_rocm_gpu(), so NVIDIA /
CPU / macOS are unaffected.
- setup.ps1: the "dependencies up to date" fast path skipped the torch
reinstall, so an existing user who had CPU torch (installed before
ROCm-wheel support) stayed stuck in CHAT_ONLY. Now, when an AMD gfx arch
is known AND the installed torch is CPU-only, don't skip -- force the
dependency pass so the ROCm wheels install.
- scripts/install_rocm_wsl_strixhalo.sh: resolve the real /opt/rocm dir
instead of hardcoding ROCM_VER for LD_LIBRARY_PATH / the librocdxg
symlink (breaks if amdgpu-install lays ROCm under a patch-version dir);
add a LIBROCDXG_REF pin knob and a "verified against" freshness header.
- tests/studio/install/test_pr5940_followups.py: cover _hf_resolve_url_parts,
_fetch_validation_model_bytes (hf path + urllib fallback), run_capture's
Windows-only amd-smi RunAsInvoker injection, and install.ps1 vs setup.ps1
name-table parity (catches future drift). 14 tests, all passing.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix DiskPart UAC prompt: skip amd-smi on Windows without a HIP SDK
On Windows, amd-smi re-initialises the ROCm runtime on every invocation
(even `amd-smi version`) and, on hosts without a working HIP runtime
(consumer APUs/dGPUs with only the Adrenalin driver), elevates a child
process at runtime -- popping a UAC/DiskPart prompt. amd-smi's own
manifest is asInvoker, so __COMPAT_LAYER=RunAsInvoker cannot suppress
that runtime elevation (verified: even `amd-smi version` hangs and
times out with RunAsInvoker set).
Replace the ineffective RunAsInvoker-only approach with a real gate:
only spawn amd-smi on Windows when a HIP SDK is detectable (hipinfo
present, so amd-smi runs un-elevated) or the user opts in with
UNSLOTH_ENABLE_AMD_SMI=1. The gfx arch is already resolved from WMI
name inference (forwarded via --rocm-gfx), so ROCm wheel + lemonade
llama.cpp selection is unaffected. Linux/macOS amd-smi never elevates
and is untouched (no regression). RunAsInvoker is kept as harmless
belt-and-suspenders for tools that DO use manifest elevation.
Applied consistently across:
- studio/backend/utils/hardware/amd.py (runtime GPU polling)
- install.ps1, studio/setup.ps1 (install-time detection)
- studio/install_llama_prebuilt.py (prebuilt arch probe + version)
- studio/install_python_stack.py (ROCm version + arch probe)
Verified live on AMD Radeon 8060S (gfx1151), native Windows: fresh
install detects the GPU, installs ROCm torch (torch.cuda.is_available()
True), launches Studio with no DiskPart prompt, and inference, tool
calling, web search, LoRA finetuning, and GGUF export all run on the GPU.
Tests: add 6 _amd_smi_allowed() gating tests + PowerShell-installer gate
assertions; update the three amd-smi monitoring tests to opt in (they
mock amd-smi as available). Full suite: 267 passed, 2 skipped.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install.sh: helpful WSL message when the GPU isn't exposed to ROCm
In WSL, an AMD GPU's ROCm-on-WSL runtime is only available with a recent
Adrenalin driver AND a distro AMD supports (currently Ubuntu 24.04). When
neither is in place, GPU detection (rocminfo/_has_amd_rocm_gpu) finds
nothing and we silently fall back to CPU.
Add an actionable hint in the CPU-fallback path, shown only on WSL and
only AFTER detection has already failed -- so it is forward-compatible:
the moment a driver/distro DOES expose the GPU (e.g. if AMD later adds
Ubuntu 26.04 support), detection succeeds and the hint never fires. The
message:
- notes a GPU is plumbed in (/dev/dxg) but no ROCm runtime is exposed,
- lists the two prerequisites (Adrenalin driver + Ubuntu 24.04),
- if the distro is not 24.04, says AMD may not support it yet,
- tells the user to `wsl --install Ubuntu-24.04` and re-run,
- links AMD's ROCm-on-WSL guide + the experimental Strix Halo helper.
Verified live: on Ubuntu-24.04 the hint shows (version-warning omitted)
and the CPU install completes; on Ubuntu-26.04 the extra "this distro may
not be supported" line appears and points to 24.04.
Also fix the experimental scripts/install_rocm_wsl_strixhalo.sh: AMD's
repo.radeon.com/amdgpu-install/ is indexed by unified installer version
(30.30, 31.30, ...), NOT ROCm version, so the hard-coded
amdgpu-install/7.2.0/ path 404'd. Scan the installer dirs newest-first
for a noble .deb matching the target ROCm major.minor (ROCm 7.2 ->
30.30.x/amdgpu-install_7.2.x), falling back to the newest available.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* WSL: fix shortcut collision + pin ROCm-on-WSL driver reqs from AMD docs
Two WSL-related fixes informed by AMD's official ROCm-on-WSL docs and
field reports for Strix Halo / Ryzen AI Max+ (Radeon 8060S, gfx1151):
1. Shortcut collision (real bug). install.sh's WSL branch wrote
"Unsloth Studio.lnk" to the SAME Desktop / Start Menu folder as the
native-Windows installer (install.ps1 New-StudioShortcuts). Running
install.sh in WSL therefore silently retargeted the native shortcut at
the WSL launcher (wt.exe -> wsl.exe), so the desktop/start-menu icon
stopped launching native GPU Studio. Now the WSL shortcut uses a
DISTINCT name -- "Unsloth Studio (WSL - <distro>).lnk" -- and fetches
the Unsloth .ico to %LOCALAPPDATA%\Unsloth Studio so it shows the
proper icon. Native and WSL shortcuts now coexist.
2. Precise ROCm-on-WSL prerequisites. Research (AMD radeon-ryzen WSL
compatibility matrix, gianni.rosagallina.com Feb-2026 guide,
ROCm/ROCm#4952/#5509/#6022) confirms WSL GPU on Strix Halo requires
AMD Adrenalin Edition >= 26.1.1 (26.2.2+ is the first production
ROCDXG/WSL release) + ROCm 7.2.1 + Ubuntu 24.04; an older driver does
not inject the ROCm/DXG runtime into /usr/lib/wsl/lib, so rocminfo sees
only the CPU. install.sh's WSL hint and the experimental
install_rocm_wsl_strixhalo.sh header/preflight now state the exact
driver version (was a guessed ">=26.3.1"), bump ROCM_VER to 7.2.1, link
AMD's radeon-ryzen docs, and document the known librocdxg caveat that
usable VRAM is currently capped at the .wslconfig memory setting.
bash -n clean; install test suite 267 passed, 2 skipped.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* installer: hint when the AMD driver is too old for ROCm-on-WSL
Adds a detect-and-guide hook for the optional WSL-GPU path. An AMD GPU on
native Windows can also be used inside WSL2, but only with AMD Adrenalin
Edition >= 26.2.2 (the first production ROCDXG/WSL release). Native Windows
GPU works with any recent driver, so this is purely about enabling the WSL
path.
We intentionally do NOT auto-install the driver: AMD referrer-gates driver
downloads (scripted curl/Invoke-WebRequest are blocked) and does not publish
Adrenalin via winget, so no installer can reliably fetch it -- and silently
swapping a live display driver is risky. Instead we point the user at AMD's
official download page (one click), after which the existing WSL detection
lights up automatically.
- install.ps1: new Show-AmdWslDriverHint -- when an AMD GPU is present and the
installed driver predates the 26.2.2 release (DriverDate < 2026-02-01),
print a concise tip with the AMD download URL. Handles DriverDate as either
a CIM DateTime or a WMI string. Suppress with UNSLOTH_SKIP_AMD_DRIVER_HINT=1.
- install.sh (WSL hint): add the direct Adrenalin 26.2.2 download URL and note
that AMD downloads are referrer-gated (open in a browser).
Verified: hint fires on a Sept-2025 driver, auto-suppresses on >= 2026-02-01;
install.ps1 parses; install.sh bash -n clean; suite 267 passed, 2 skipped.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* install.ps1: refresh shell icon cache after creating the shortcut
After writing the Desktop / Start Menu .lnk, nudge Explorer to refresh
its icon (ie4uinit.exe -show). Without this, a stale icon cache can show
a blank shortcut icon until the next explorer restart -- most visible
when a shortcut of the same name was rewritten (e.g. a native install
followed by a WSL install, which previously shared the name; now they use
distinct names, but the cache nudge makes the icon appear immediately
regardless). Best-effort and wrapped in try/catch so it never fails the
install. The bundled unsloth.ico itself is valid (verified it renders).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* setup.ps1: don't silently CPU-build llama.cpp on an AMD GPU
For AMD, GPU acceleration comes from the lemonade ROCm prebuilt (it bundles
the ROCm runtime, no HIP SDK needed) and is the preferred/default path. The
source-build fallback is CPU-only -- a HIP/ROCm *source* build would need the
full HIP SDK + ROCm clang toolchain, which the prebuilt exists to avoid.
Previously, if an AMD-GPU host ever fell through to the source build (e.g. the
prebuilt could not be downloaded), it printed "building llama.cpp (CPU-only,
no NVIDIA GPU detected)" and quietly produced a CPU binary -- masking the lost
GPU acceleration. Now that case emits a loud [WARN] explaining the GPU prebuilt
is the AMD path and how to restore it (re-run / check network / set
UNSLOTH_LLAMA_RELEASE_TAG), so AMD never silently degrades to CPU.
No behavior change on the happy path: AMD still gets the GPU prebuilt (verified
on gfx1151: ggml-hip.dll bundled, ~80% GPU compute during inference). NVIDIA
(CUDA source build) and CPU-only hosts are unchanged.
setup.ps1 parses; install suite 267 passed, 2 skipped.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* uninstall: remove shared llama.cpp build, kill lock-holders, match WSL shortcut
Three gaps found by running a real uninstall on a native-Windows + WSL host;
all fixes are scoped to Unsloth-owned paths and no-op on the other pathways
(env/custom-root, NVIDIA/AMD/CPU, Mac) so nothing else regresses.
uninstall.ps1:
- Remove the default-mode SHARED llama.cpp build + cache. setup.ps1 installs
them at ~/.unsloth/llama.cpp and ~/.unsloth/.cache -- SIBLINGS of studio,
not under it -- so deleting <studio> left hundreds of MB behind. Now removed
explicitly, then ~/.unsloth is dropped ONLY if empty (never nukes unrelated
content). No-op in env/custom mode (llama.cpp nests under the custom root,
removed already) and when absent. UNSLOTH_LLAMA_CPP_PATH (user-owned) is kept.
- New _StopProcessesLockingRoots: _StopStudioProcesses only matched the venv
unsloth/python/studio exe, so it missed (a) llama-server.exe under llama.cpp
and (b) an orphaned multiprocessing python fork that ran from the SYSTEM
python but loaded a venv DLL (bitsandbytes) -- on Windows an open DLL handle
blocks the directory delete, leaving a half-removed install. The new helper
kills any process whose image path OR loaded module is under a target root
(module scan scoped to python/unsloth/llama-server names; vendor-agnostic).
- _RemovePath now retries (transient post-kill handle release).
uninstall.sh:
- Remove the default-mode ~/.unsloth/llama.cpp + ~/.unsloth/.cache; rmdir
~/.unsloth only if empty.
- WSL Windows-side shortcut cleanup now matches by TARGET (any
"Unsloth Studio*.lnk" whose target launches wsl.exe), covering both the
legacy "Unsloth Studio.lnk" and the new "Unsloth Studio (WSL - <distro>).lnk"
-- and never removes a native-Windows shortcut (which launches wscript.exe).
uninstall.ps1 parses; uninstall.sh passes sh -n and bash -n.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* install.ps1: invalidate Win11 Start Menu tile cache after creating shortcut
The Start Menu shortcut kept showing a blank/generic icon even after the
Explorer icon-cache rebuild, because Windows 11's StartMenuExperienceHost
keeps its OWN pre-rendered tile-icon cache
(%LOCALAPPDATA%\Packages\Microsoft.Windows.StartMenuExperienceHost_cw5n1h2txyewy\
TempState\TileCache_*.bin + StartUnifiedTileModelCache.dat), separate from
Explorer's iconcache_*.db. ie4uinit and an explorer.exe restart do not touch
it, and they don't recycle the host -- so a rewritten same-name shortcut keeps
showing the first-rendered (often the generic wscript ">") tile until the host
restarts on its own.
Fix: after creating the shortcut, drop only the Start Menu RENDER caches
(TileCache_* + StartUnifiedTileModelCache.dat) and stop StartMenuExperienceHost
(Windows auto-relaunches it), so the tile re-resolves the real icon via the
shell image factory. start2.bin (the user's pinned layout) is deliberately
preserved. Guarded by Test-Path (Windows 10 has no such host -> skipped) and
wrapped in try/catch so it can never fail the install. Windows-only
(install.ps1); no effect on Linux/macOS/Studio.
Verified live: rendering the shortcut via IShellItemImageFactory::GetImage (the
API StartMenuExperienceHost uses) returns the Unsloth sloth icon, color-matched,
after this invalidation -- previously it returned the generic script tile.
install.ps1 parses; install suite 267 passed, 2 skipped.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* ROCm-on-WSL for AMD Strix Halo (gfx1151): auto-setup + runtime enablement
Make Unsloth Studio set up ROCm-on-WSL automatically for AMD Strix Halo
(Radeon 8060S / gfx1151) and use the GPU at runtime, validated end-to-end
on a Ryzen AI Max+ PRO 395 (ROCm 7.2.1 + librocdxg + Adrenalin Apr-2026):
rocminfo enumerates gfx1151, torch.cuda True, ~85.8 GB UMA pool.
Every change is a strict no-op for all other configs (NVIDIA/CUDA,
discrete + native-Linux AMD ROCm, macOS/MLX, Windows, CPU-only, non-Strix
WSL) and can never abort the installer.
- scripts/install_rocm_wsl_strixhalo.sh: rewrite to the validated recipe.
Fixes that would have broken a working box: drop the /usr/lib/wsl/lib
preflight (a working ROCDXG host has only d3d12/dxcore there); remove the
obsolete rocr4wsl step (gone from the 7.2.1 repo; would hard-fail and also
rips out the standard hsa-rocr ROCDXG needs); dynamic librocdxg soname
(was hardcoded 1.1.0; build is 1.2.0); direct apt-repo install; Windows
SDK auto-discovery; persist env to /etc/profile.d + ~/.bashrc; idempotent.
- install.sh: _maybe_bootstrap_rocm_wsl auto-offers/runs the helper when it
detects a Strix Halo APU in WSL (/dev/dxg) with no ROCm runtime, then
loads the env so detection routes to the gfx1151 wheels. Fast-path when
already configured. Fix an inaccurate WSL hint line.
- studio/backend/main.py + worker.py: set HSA_ENABLE_DXG_DETECTION=1
in-process before torch (gated on /dev/dxg AND librocdxg.so), so the
worker uses the GPU even when launched outside a login shell. Mirrors the
existing BNB_ROCM_VERSION injection.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* uninstall: clean up ROCm-on-WSL artifacts + Start Menu tile cache
- uninstall.sh: remove the ROCm-on-WSL helper artifacts -- the librocdxg
build clone (~/.unsloth/librocdxg, which otherwise blocks the empty-dir
rmdir of ~/.unsloth), the throwaway smoke-test venv, the persisted env
(/etc/profile.d/unsloth-rocm-wsl.sh) and the ~/.bashrc block. The system
ROCm userspace is a shared prereq like CUDA and is kept by default;
UNSLOTH_UNINSTALL_ROCM=1 removes it too. No-ops on macOS / non-Strix Linux.
- uninstall.ps1: invalidate the Win11 Start Menu tile cache after removing
the shortcut so its tile disappears promptly (mirrors install.ps1),
preserving start2.bin.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* installer: accurate AMD ROCm messaging (HIP SDK optional, not required)
The Windows installer printed "HIP SDK not found - GPU-accelerated training
unavailable" / "ROCm wheels require the HIP SDK" whenever the HIP SDK was
absent. That is misleading: for a detected AMD GPU arch (gfx1151 etc.),
setup.ps1 installs AMD's bundled-runtime ROCm PyTorch wheels (repo.amd.com)
which ship their own ROCm runtime and do NOT need the HIP SDK -- verified
end-to-end (torch 2.11.0+rocm7.13.0, cuda True, QLoRA training on GPU) on a
Radeon 8060S with no HIP SDK installed.
Gate the GPU-detection + rocm-step messages on a detected gfx arch: when one
is known, state that GPU PyTorch uses bundled-runtime wheels and the HIP SDK
is optional; only when the arch is unknown fall back to the HIP-SDK hint.
Behavior (torch routing) is unchanged; this is messaging only. No-op for
NVIDIA/CUDA, HIP-SDK-present, and CPU paths (they hit earlier branches).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* installer: fix /opt/rocm data-loss + make WSL shortcut create/remove interop-robust
Two fixes from the 3-reviewer regression audit + live testing on a
systemd-enabled WSL distro (interop disabled):
F1 (data-loss, install_rocm_wsl_strixhalo.sh): the /opt/rocm symlink-repair
could force-delete a pre-existing REAL ROCm install. The guard only checked
that /opt/rocm is a real directory, not that it is the stray librocdxg stub.
Now it only touches /opt/rocm when it is NOT a real install (no bin/rocminfo,
bin/hipcc, or .info/version present), and MOVES it aside (rocm.unsloth-stub-bak)
instead of deleting it, so a wrong guess can never lose data.
WSL interop robustness (install.sh + uninstall.sh): both relied on
`command -v powershell.exe`, which is true even when WSL interop cannot EXECUTE
it (on systemd distros powershell.exe fails with "Exec format error"). Result:
the WSL shortcut silently failed to create (install) and to remove (uninstall).
- uninstall.sh: test that powershell.exe actually runs; if not, remove the
"Unsloth Studio (WSL...).lnk" files directly via drvfs (/mnt/<drive>), which
works without interop. The name is WSL-install-specific, so a native install's
"Unsloth Studio.lnk" is never touched.
- install.sh: when the shortcut cannot be created, warn with the manual launch
command + how to re-enable interop, instead of failing silently.
No behavior change on the interop-on path. The regression audit otherwise found
no regressions on Linux/Mac/Windows/CPU/NVIDIA install paths.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* install.sh: fast-path fully restores ROCm-on-WSL env when the drop-in is gone
Reinstall regression found by uninstall->reinstall testing: after a Studio
uninstall that removed /etc/profile.d/unsloth-rocm-wsl.sh but KEPT the shared
ROCm (the default), a non-login reinstall hit the bootstrap fast-path
(librocdxg present) and its else-branch only set HSA_ENABLE_DXG_DETECTION --
NOT PATH/LD_LIBRARY_PATH. So rocminfo was not on PATH, GPU detection failed,
and the installer fell back to CPU-only PyTorch.
Fix: when librocdxg is present but the env drop-in is missing, restore the
FULL env inline (HSA + TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL + PATH +
LD_LIBRARY_PATH) so rocminfo is found and detection routes to the GPU, and
recreate /etc/profile.d/unsloth-rocm-wsl.sh so future shells and the Studio
worker get it too. No change to the env-present fast-path or any other host.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* installer: clear Explorer icon cache so shortcut icons aren't blank
Root cause of the persistent blank Desktop + Start Menu icons: Explorer caches
each shortcut's icon in iconcache_*.db and does NOT re-read the .ico when a
same-name .lnk is recreated across reinstalls. The .ico and .lnk are correct
(the shell renders them non-blank via IShellItemImageFactory; the .ico has real
image data at 16/32/48/128 px), but the stale cache entry wins. The previous
fix only ran a weak `ie4uinit -show` + the Start Menu tile-cache clear -- it
never invalidated Explorer's icon cache, so the desktop icon stayed blank.
Fix (native install.ps1 New-StudioShortcuts AND the WSL shortcut path in
install.sh):
- ie4uinit -ClearIconCache (thorough; replaces -show as the primary refresh)
- SHChangeNotify(SHCNE_ASSOCCHANGED) to force a live desktop/taskbar refresh
WITHOUT restarting explorer
- keep the Win11 Start Menu tile-cache invalidation (and add it to the WSL
shortcut path too, preserving start2.bin)
Non-disruptive (no explorer restart). install.ps1 parses clean; install.sh
passes bash -n + dash -n; the heredoc-generated WSL PowerShell parses clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* installer: per-item SHChangeNotify(UPDATEITEM) reliably fixes blank icons
The blank Desktop/Start Menu shortcut icons are a stale Explorer PER-ITEM icon
cache: when a same-name .lnk is recreated across reinstalls, Explorer caches the
previously-resolved (often generic "white page") icon for that item and won't
re-extract the .ico on its own. The .ico and the .lnk's IconLocation are correct
(every icon API renders the sloth) -- only Explorer's cached display is stale.
The previous refresh (ie4uinit -ClearIconCache + a GLOBAL SHCNE_ASSOCCHANGED
broadcast) does NOT recover a stale item -- confirmed by reproduction. The
reliable, NON-disruptive fix (no explorer restart) is a PER-ITEM
SHChangeNotify(SHCNE_UPDATEITEM, SHCNF_PATHW, <lnk path>) for each created
shortcut, which forces Explorer to re-read that exact item's icon.
Verified end-to-end: deliberately staled a shortcut to the generic icon, ran the
installer's exact new refresh code, and the sloth icon recovered with NO explorer
restart (confirmed by capturing the live desktop via PrintWindow).
Applied to both native install.ps1 (New-StudioShortcuts) and the WSL shortcut
path in install.sh. Still clears the on-disk icon cache (ie4uinit) and the Win11
Start Menu tile cache (preserving start2.bin).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* uninstall: remove leftover llama.cpp .staging root so ~/.unsloth is cleaned
The llama.cpp atomic-install staging root (install_llama_prebuilt.py
INSTALL_STAGING_ROOT_NAME=.staging) is a sibling of the llama.cpp install
dir (~/.unsloth/.staging in default mode). It is normally pruned after a
successful activate, but an interrupted or retained build can leave a
<name>.staging-XXXX tree behind. The uninstallers removed llama.cpp and
.cache but not .staging, so the final empty-dir cleanup of ~/.unsloth failed
and the directory lingered. Reproduced on WSL (Ubuntu-24.04) where an empty
llama.cpp.staging-XXXX dir kept ~/.unsloth alive after uninstall.
Remove ~/.unsloth/.staging in both uninstall.sh and uninstall.ps1. No-op in
env/custom mode (staging nests under the custom root removed already) and
when absent. Cross-platform fix (the staging logic is platform-agnostic).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* installer: WSL-absent hint + fix here-string lint false positive
install.ps1: in the AMD WSL-ROCm driver hint, detect when wsl.exe is absent
and add a one-line "wsl --install -d Ubuntu-24.04" pointer so a Strix Halo
user with no WSL yet gets an actionable next step (the hint previously assumed
an Ubuntu-24.04 distro already existed). Best-effort, informational only.
test_rocm_support.py: test_no_here_strings did a crude substring check that
false-positived on the conda-style block marker
printf '# <<< Unsloth ROCm-on-WSL (gfx1151) <<<' -- a string literal written
into the /etc/profile.d drop-in, also used as a sed delimiter pair by
uninstall.sh, not a here-string. Strip quoted spans before the check so the
lint still catches a real here-string operator but ignores quoted literals.
install.sh remains POSIX-clean (sh -n / dash -n / bash -n all pass).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* installer: address PR review comments (gfx1150 mapping, amd-smi opt-out, WSL bootstrap, SDK path, make)
Apply the valid bot review findings on #5940; reject the ones that don't hold.
Fixed:
- AMD name->gfx table (setup.ps1 + install.ps1): Radeon 890M and Ryzen AI 9 HX
370/375 are Strix POINT (gfx1150), not Strix Halo (gfx1151). Move 890M / HX 37x
/ AI 9 HX to the gfx1150 row and drop the bogus HX 38x pattern (no such Strix
Halo SKU). Matches the runtime classifier in worker.py (890M/880M -> gfx1150;
8060S/8050S -> gfx1151). Prevents Strix Point hosts from getting the wrong ROCm
prebuilt/wheels.
- amd-smi opt-out (setup.ps1 + install.ps1): an explicit UNSLOTH_ENABLE_AMD_SMI=
0/false/no/off now wins over the HIP-SDK heuristic, so a host with a HIP SDK
binary but a broken runtime no longer gets the DiskPart/UAC prompt the opt-out
exists to avoid.
- amd-smi warning probes (install_python_stack.py): _has_rocm_gpu and
_detect_amd_gfx_codes now gate amd-smi behind _amd_smi_allowed() (and pass
_amd_smi_env()), closing the last unguarded amd-smi spawn on Windows.
- WSL ROCm bootstrap (install.sh): the "already-usable ROCm?" early return now
requires rocminfo to enumerate the real gfx1151 agent instead of the generic
_has_amd_rocm_gpu (whose broad gfx[1-9][0-9] match accepts a fallback
"gfx11-generic" ISA), so a Strix Halo box missing the ROCDXG bridge is no longer
skipped. The shared helper is untouched (no gfx90a regression).
- install_rocm_wsl_strixhalo.sh:
* Quote-safe Windows SDK discovery: the old for-in-$(ls -d "...Program Files
(x86)/...") word-split on the space and never matched; use find + read loop.
* Add `make` to apt prereqs (cmake only recommends it; minimal images lacked it
and the librocdxg `make -j` build failed).
* Verification requires gfx1151 exactly (not gfx1[0-9]) so a generic ISA or an
unrelated RDNA GPU can't pass while the real GPU is absent.
Reviewed but NOT changed:
- "Forward inferred ROCm arch without HasROCm" (setup.ps1): already correct --
--rocm-gfx is forwarded under `if ($script:ROCmGfxArch)`, not `if ($HasROCm)`.
- "Route inferred arch into install.ps1 torch path": not a bug -- install.ps1
installs CPU torch as a base by design and setup.ps1 swaps in the ROCm wheel for
the inferred arch (gate `($HasROCm -or $ROCmGfxArch) -and cpu`); verified live
the native install ends on torch 2.11.0+rocm7.13.0.
- "$p null guard after Start-Process" (install.ps1/setup.ps1): redundant -- the
amd-smi runner uses [Process]::Start wrapped in try/catch, so a null process
already returns "" with LASTEXITCODE=1 (no uncaught exception).
- "ls -> find for /usr/lib/wsl/lib" (gemini): stale -- that heuristic was removed;
only a comment about it remains.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* installer(rocm-wsl): auto-install the Windows 11 SDK via winget (fewer manual steps)
librocdxg's build needs the Windows SDK 'shared' headers on the Windows host.
Previously the helper just die()d with "install the Windows 11 SDK and re-run" if
they were missing -- a manual prerequisite that broke the otherwise-seamless
`curl ... install.sh | sh` one-liner on Strix Halo.
Now, when the headers aren't found, the helper installs the Windows 11 SDK on the
Windows host from inside WSL via winget (powershell.exe interop), then
re-discovers them. The SDK installer elevates -> ONE UAC prompt on the Windows
desktop; the headers appear under /mnt/c immediately (drvfs is live, no reboot).
The user already consented to the ROCm-on-WSL setup, so no extra prompt is added
beyond the OS UAC gate.
- New _find_win_sdk (space-safe find of the newest installed SDK 'shared' dir)
and _install_windows_sdk_via_winget helpers.
- winget IDs tried newest-stable first: Microsoft.WindowsSDK.10.0.26100, then
.22621. The presence of the headers (re-check) is the source of truth, not
winget's exit code. </dev/null so winget never consumes a piped `curl|sh` stdin.
- Best-effort + non-fatal: interop-off / no-winget / declined-UAC all fall
through to the existing clear manual-install die(). Opt out with
UNSLOTH_SKIP_WIN_SDK_INSTALL=1.
Removes the last avoidable manual step from the WSL Strix Halo path; only the AMD
Adrenalin driver (AMD referrer-gates the download) remains manual. Verified
_find_win_sdk resolves the spaced "Program Files (x86)" path; bash -n clean; all
winget flags validated against `winget install --help`.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* installer(amd): gate install-time amd-smi probe to fix DiskPart UAC prompt
install_python_stack.py's Windows "AMD GPU detected but ROCm torch missing"
warning probe ran `amd-smi list` whenever amd-smi was on PATH -- and amd-smi
ships in C:\Windows\System32 with the AMD Adrenalin driver -- without the
_amd_smi_allowed() gate that every other amd-smi call site in the file uses.
On Adrenalin-only hosts (no HIP SDK) amd-smi elevates a child at runtime and
pops a UAC/DiskPart prompt that __COMPAT_LAYER=RunAsInvoker cannot suppress
(amd-smi's manifest is asInvoker). The probe also ran before the
ROCm-torch-installed check, so it fired on every Windows AMD install.
Gate it behind _amd_smi_allowed() and pass _amd_smi_env(), matching
_has_rocm_gpu()/_detect_amd_gfx_codes(). When skipped, the only loss is the
best-effort "AMD GPU detected" note on HIP-SDK-less hosts.
Adds a per-function AST regression test asserting every function in
install_python_stack.py that names the amd-smi command and spawns a subprocess
also references _amd_smi_allowed() (flags the pre-fix code; passes after).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* studio(cli): fix `unsloth studio stop` crashing on Windows
`stop` used the POSIX `os.kill(pid, 0)` liveness probe, but on Windows
CPython raises OSError (WinError 87, "The parameter is incorrect") for
*every* pid -- alive or dead. `stop` only catches ProcessLookupError /
PermissionError, so the OSError propagated and the command crashed with
a traceback before ever reaching its (correct) `taskkill /F` path.
Add a cross-platform `_pid_alive(pid)` helper (tasklist on Windows,
signal-0 elsewhere) and use it for both the pre-check and the post-kill
wait loop. The actual kill path is unchanged.
Verified on Windows (Python 3.13): os.kill(pid,0) raises WinError 87 for
both a live and a dead pid; `_pid_alive` returns True/False correctly and
the full stop() flow (alive -> taskkill -> dead -> "stopped") passes
end-to-end against a throwaway process.
Adds tests/studio/test_cli_studio_stop_windows.py (AST guard against a
bare os.kill(pid,0) liveness probe + mock-only _pid_alive behaviour for
the win32 tasklist branch and the POSIX signal-0 branch).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* installer(amd): fix install.sh name->arch table misrouting Strix Point to gfx1151
The bash name->arch inference table in install.sh placed Strix Point
identifiers (Radeon 890M, "Ryzen AI 9 HX 370/375", "AI 9 HX") in the
gfx1151 (Strix Halo) row, diverging from the install.ps1 / setup.ps1
PowerShell tables which correctly map them to gfx1150. It also carried a
stray "HX 38" token absent from the PowerShell source-of-truth.
Align install.sh with the PowerShell tables:
gfx1151 row: 8060S|8050S|8040S|Strix Halo|Ryzen AI Max|AI Max
gfx1150 row: 890M|880M|860M|840M|Strix Point|Krackan|HX 37|AI 9 HX|...
Impact is low (the bash table only feeds the display label _gpu_disp_gfx
and the "set UNSLOTH_ROCM_GFX_ARCH=..." hint; wheel selection is driven
by the detected ROCm version, not this name string) but a Strix Point
user would otherwise see/copy the wrong gfx arch.
Add a parity test (test_install_sh_name_arch_agrees_with_ps_for_strix_and_non_amd)
that parses install.sh's case table and asserts Strix Halo->gfx1151,
Strix Point->gfx1150, RX 7700S->gfx1102, and NVIDIA/Intel->no match,
cross-checking against install.ps1 (the previous parity test only
compared install.ps1 <-> setup.ps1, missing install.sh).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* setup.ps1: keep prebuilt-llama ownership guard within the test's block window
The AMD additions to the prebuilt-llama.cpp block (the windows-hip vs
windows-cpu existing-install kind validation) pushed the
install_llama_prebuilt.py invocation to ~1999 chars after the
"installing prebuilt llama.cpp bundle (preferred path)" anchor, right at
the edge of the 2000-char window that
test_setup_ps1_prebuilt_llama_cpp_has_ownership_guard slices -- so the
helper string was truncated and the test failed with "substring not
found" (CI: Repo tests (CPU)).
The ownership-guard invariant (Assert-StudioOwnedOrAbsent precedes the
install_llama_prebuilt.py call) was already satisfied; only the proximity
to the anchor regressed. Move the "installing prebuilt..." substep to
immediately before the install (after the existing-install pre-cleanup),
which also reads better (validate/clean existing -> then "installing"),
shrinking anchor->helper from 1999 to 413 chars. Behaviour is unchanged
(console message ordering only).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* install.sh: auto-run Strix Halo ROCm-on-WSL setup by default
`curl -fsSL https://unsloth.ai/install.sh | sh` should make a Strix Halo
(gfx1151) GPU usable inside WSL with no extra commands. Previously the
ROCm-on-WSL bootstrap was opt-in: it required UNSLOTH_ROCM_WSL_AUTO=1 or an
interactive [Y/n] at a TTY, and silently skipped under a pipe (no /dev/tty),
so the piped one-liner never set the GPU up automatically.
Flip it to auto-by-default for the single narrow case the existing guards
allow (WSL + Strix Halo + /dev/dxg + no usable ROCm yet) -- exactly the GPU
setup the user ran the installer for. Opt out with
UNSLOTH_SKIP_ROCM_WSL_SETUP=1. The Tauri desktop app keeps its own consent UI
(only auto-runs when it passes UNSLOTH_ROCM_WSL_AUTO=1). All hardware/OS
guards are unchanged, so non-Strix / non-WSL / NVIDIA / native-Linux / macOS /
CPU paths are unaffected.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* PR comments: condense to be succinct (comments/docstrings only)
Shorten the verbose explanatory comments and docstrings this PR added across
the installer, scripts, backend shims, CLI, and tests -- tighter, fewer lines,
while preserving every non-obvious "why" (os.kill WinError 87, amd-smi
RunAsInvoker/UAC, /dev/dxg + librocdxg gating, the ROCm-on-WSL bootstrap guard
chain, ownership guards, etc.). No executable code, string literals, messages,
or behavior changed.
Verified comments-only: docstring-normalized AST equality (Python, 9 files),
non-comment token equality (PowerShell, 3 files), comment-stripped diff +
sh -n / bash -n (shell, 3 files). Behavior re-confirmed: get_torch_index_url +
gfx name->arch table 44/44 under dash & bash; rocm_support / pr5940_followups /
cli_studio_stop tests green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Installer: address PR review (amd-smi opt-out, pipefail, multi-distro, non-root)
Fixes valid findings from the Codex/Gemini PR review:
- install.ps1 / setup.ps1: gate the `amd-smi version` ROCm-version fallback with
$amdSmiAllowed so UNSLOTH_ENABLE_AMD_SMI=0 opt-out is honored (the device
probe was gated but this fallback wasn't), avoiding the DiskPart/UAC prompt.
- install_rocm_wsl_strixhalo.sh: make the post-verification rocminfo summary
best-effort (|| true) so head's early pipe-close under `set -o pipefail` can't
fail the bootstrap after gfx1151 was already enumerated; pin the Windows SDK
`winget install` to --source winget (matches the msstore-cert fix rationale).
- install.ps1: python.org fallback installs the py launcher per-user
(InstallLauncherAllUsers=0, avoids admin), and derives the fallback full
version from the requested minor so a non-default UNSLOTH_PYTHON (e.g. 3.12)
isn't silently replaced with 3.13 when the listing is unreachable.
- install.sh: recreate /etc/profile.d/unsloth-rocm-wsl.sh via `sudo tee` for a
non-root reinstall (a plain redirect failed silently, dropping the ROCm env).
- uninstall.sh: scope WSL Windows-side shortcut removal to the current
WSL_DISTRO_NAME (per-distro name or -d "<distro>" arg) so uninstalling one
distro no longer deletes other distros' launchers.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Studio ROCm Windows: fix field-reported issues from Strix Halo testers
Four fixes from PR #5940 field reports (Win11 native, gfx1151):
1. bitsandbytes arch-probe spam: bnb's get_rocm_gpu_arch() runs
hipinfo.exe via subprocess PATH at import; the AMD torch wheel ships
hipInfo.exe in the venv Scripts dir, which is only on PATH for
activated venvs. Every bnb import logged "Could not detect ROCm GPU
architecture: [WinError 2]" ERROR + WARNING (even with the HIP SDK
installed, whose bin dir is not on PATH either). Prepend the Scripts
dir to PATH before bnb imports in main.py, worker.py, and
install_python_stack.py, gated on the file existing (only AMD wheels
ship it). Verified on gfx1151: ROCM_GPU_ARCH now resolves to gfx1151
with zero errors.
2. OOM-guard double-tax on native Windows unified APUs: mem_get_info's
total is the WDDM budget the driver grants HIP (BIOS carve + ~half
of remaining RAM) -- the OS share is already outside it. The 0.80
unified cap on top denied loads that fit (field report: 48.49 GiB
budget -> "38.79 GiB allowed" OOM for a 47.29 GiB load with 48.08
free). Use 1.0 on win32 unified; Linux keeps 0.80, discrete 0.90.
3. "Missing VRAM" confusion: log the WDDM budget vs physical RAM with
the fix (BIOS UMA frame buffer / AMD Software Variable Graphics
Memory) when the grant is under 75% of RAM, so a 48 GiB cap on a
96 GiB box reads as policy, not a Studio bug.
4. llama-server fit-step crash (Qwen3.6-27B-MTP + mmproj, lemonade
gfx1151): --fit defaults to 'on' upstream, so the fit step runs even
when Studio already placed the model via -ngl -1, and aborts in
ggml-cuda.cu on some ROCm hosts. Retry the spawn once with --fit off
when the server crashes during startup and Studio's own VRAM math
had placed the model (never when use_fit or an explicit fit flag was
passed). Also keep the TAIL of crash output in the error log (the
diagnostic line prints last; head-truncation cut exactly that) and
reference the full on-disk log.
Verified live on Radeon 8060S: bnb import clean, Qwen3.5-4B-MTP loads
and generates through the new spawn loop, stub-crash retry appends
--fit off and recovers, fraction probes confirm WDDM overcommit and
sub-1.0-only enforcement on current AMD wheels.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio ROCm Windows: GPU-name fallbacks so nothing depends on amd-smi
amd-smi does not reliably exist on Windows: the HIP SDK never ships a
CLI, inbox Windows Update drivers do not, and only some full Adrenalin
packages drop amd-smi.exe into System32 (field report: fresh Win11 +
Adrenalin + HIP SDK, still no amd-smi anywhere). Make every consumer
work without it:
- install_python_stack._detect_windows_gfx_arch: two new probes after
hipinfo/amd-smi -- (2b) the venv Scripts hipInfo.exe shipped by AMD
torch wheels (drives `studio update` on driver-only hosts), and (4) a
last-resort GPU marketing-name -> gfx table via WMI
(Win32_VideoController), mirroring setup.ps1's $nameArchTable so a
standalone repair resolves the arch with zero AMD tooling installed.
- install_llama_prebuilt._resolve_exe: also probe the venv Scripts dir
so a standalone rerun finds hipInfo.exe without HIP_PATH.
- hardware/amd.py _run_amd_smi: which() guard before spawning --
absence now disables the poller in one step instead of burning the
3-strike circuit breaker on FileNotFoundError; corrected the stale
comment claiming Adrenalin ships amd-smi.
Simulated against the real detection functions on gfx1151: amd-smi
absent, present-but-crashing (exit 1), present-but-hanging (60s sleep
vs 5-10s probe timeouts), and hard opt-out -- all resolve gfx1151, no
exceptions, bounded time. Full adversarial install (broken amd-smi
stub first on PATH + UNSLOTH_ENABLE_AMD_SMI=1, fresh uninstall first):
exit 0, name-table arch inference, lemonade gfx1151 b1292 prebuilt,
torch 2.11.0+rocm7.13.0 cuda_avail=True on the 8060S, Studio boots
healthy and stops cleanly.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: per-attempt llama-server log names + amd-smi test portability
Found by cross-platform simulation of the --fit off retry (Windows +
Linux sandboxes, real load_model with stub servers):
- llama-server log filename now carries the spawn-attempt index. The
retry can respawn within the same epoch second; reusing the name
opened the same file with "w" and truncated the crash log the retry
warning had just pointed the user at (proven with a frozen
time.time: one file, crash evidence gone; with the suffix both
attempts keep their logs). Regression-pinned in
test_llama_cpp_wait_for_health.py.
- test_amd_primary_gpu_with_mock now mocks shutil.which alongside
subprocess.run: the amd-smi absence guard which()-checks before
spawning, so on hosts without a real amd-smi (Linux CI, driver-only
Windows) the subprocess mock was never reached and the test failed.
Surfaced by running the suite in a clean Linux sandbox.
Simulation coverage on both OSes: 67-case platform/edge matrix
(real shipped code blocks under win32/linux/darwin spoofs: OOM-guard
fractions + VGM-hint boundary, bnb PATH-prepend gates, retry
eligibility incl. equals-forms and decoy tokens, GPU-name table
adversarial set, WMI fallback without powershell, monitor absence
semantics), 6-scenario live retry matrix (crash-once/crash-always/
exit-zero/explicit-fit/hang/log-collision) against real llama-server
spawns on Windows and WSL (GPU success legs on the 8060S), and a
3-engine browser matrix (chromium/firefox/webkit) driving the live
backend's health + authed /v1 chat completion.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: classify unified-memory via props.is_integrated first
Align the ROCm OOM-guard classifier with PR #5988's UMA gate: consult
hipDeviceProp_t.integrated (props.is_integrated) before the hardcoded
arch set. Strictly additive -- truthy upgrades to unified; 0/absent
falls through to the existing gfx1150/gfx1151 + device-name logic, so
wheels that omit or zero the field cannot downgrade the known APU set.
Extends correct unified-cap treatment to APUs outside that set (e.g.
gfx1103 Phoenix iGPUs) and keeps Studio's two unified-memory consumers
on one driver signal. Verified live on gfx1151 (is_integrated == 1 on
the AMD Windows wheel -> ('gfx1151', True) via the new path).
* AMD detection: probe rocminfo with HSA_ENABLE_DXG_DETECTION and sync setup.sh gfx table
Fleet validation on a Strix Halo WSL2 box showed the system rocminfo
(HSA 1.18, ROCm 7.2.1) only enumerates the GPU over /dev/dxg when
HSA_ENABLE_DXG_DETECTION=1, and that rocminfo can sit at /opt/rocm/bin
off PATH outside login shells. Detection probes that miss either of
these report no GPU on a working ROCDXG host and select the CPU build
even though the lemonade bundle offloads fine (95.7 tok/s measured vs
64.5 CPU on the same laptop). Seed the env (a no-op on bare metal) and
the PATH fallback in install.sh, studio/setup.sh, and the installer's
Linux rocm probe, mirroring what main.py/worker.py already do for the
runtime.
Also sync studio/setup.sh's name->gfx table with install.sh: 890M and
the HX 37/AI 9 HX SKUs are Strix Point (gfx1150, not gfx1151), RX 7700S
must match gfx1102 before the gfx1100 row, and the RDNA2/workstation
rows were missing. New parity test pins the two bash tables together so
they cannot drift again.
* Studio: persist server session logs + native-crash stacks to disk
Field report (Strix Halo, 96 GB UMA carve, WSL and native Windows):
"the studio just terminates without a warning". A native crash in the
GPU runtime kills the process with no Python traceback, and a desktop-
shortcut console closes before anything can be read. The server only
ever logged to the console, so there was nothing to send back.
run_server now tees stdout/stderr to
~/.unsloth/studio/logs/server/server-<ts>-pid<n>.log (console behavior
unchanged; file copy is best-effort), arms faulthandler at the same
file so access violations / SIGSEGV leave a stack trace on disk, and
exports PYTHONFAULTHANDLER=1 so training workers inherit crash dumps
on their captured stderr. Armed before `from main import app` so even
import-time failures leave evidence. Keeps the newest 20 session logs;
opt out with UNSLOTH_STUDIO_NO_FILE_LOG=1. Prints "Session log: <path>"
at startup so users know what to attach.
Verified on this box: a forced real segfault (faulthandler._sigsegv)
leaves the full session output plus "Fatal Python error: Segmentation
fault" and the thread stack in the file while the console shows
nothing; a normal server boot captures the startup banner and serves
health as before.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* AMD probe: honor a pre-set HSA_ENABLE_DXG_DETECTION value
Match the shell helpers, which use the parameter-default form: a user
who exports HSA_ENABLE_DXG_DETECTION=0 to deliberately hide the GPU
from DXG detection should not have the probe override it.
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
|
||
|
Daniel Han
|
8848a310df |
Studio: clean-room compact RAG (knowledge bases, hybrid search, fast indexing) (#5910)
Adds a self-contained RAG stack to Studio: knowledge bases with chunked indexing, hybrid (dense + lexical) retrieval, and an automatic first-pass context inject into chat. Embeddings run through a local llama-server GGUF backend (default unsloth/bge-small-en-v1.5-GGUF) with a sentence-transformers fallback. The chat tool loop gains a search_knowledge_base tool, a per-turn re-search cap, and source citation, layered on top of the shared ToolLoopController. |
||
|
oobabooga
|
57be5868f9 |
Studio: improve OpenAI- and Anthropic-compatible API spec compliance (#6010)
* Studio: fix OpenAI- and Anthropic-compatible API spec compliance * Studio: fix API spec-compliance gaps on passthrough and streaming paths * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: carry context_length_exceeded through the OpenAI passthrough error path * Studio: count tool-schema tokens in the Anthropic server-tool stream, and small stream-handling guards * Studio: guard message_delta usage against None and normalize developer role before proxying * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: honor max_completion_tokens on the external-provider proxy path * Studio: forward llama-server cached_tokens into OpenAI prompt_tokens_details * Studio: sanitize messages in count_tokens to match the /v1/messages prompt * Studio: report max_tokens for truncated tool calls and guard null usage in metadata events * Studio: drop the request-id middleware (headers aren't declared in either spec) * Studio: include the required request_id field in Anthropic error bodies * Studio: honor max_completion_tokens on the audio (TTS / audio-input) paths * Studio: add the _effective_max_tokens helper and route all max-token sites through it * Studio: align API compatibility edge cases * Studio: clarify multi-choice chat support * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: clarify logprobs chat support * Studio: opt the local chat UI into the streaming usage chunk so the context bar and tok/s repopulate * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: forward seed to llama-server, and fix Anthropic server-tool stop_reason, tool_result id correlation, and parallel-tool execution cap * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: align OpenAI chat completion spec edge cases * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: align backend API compatibility tests * Studio: honor tool caps and internal stream usage * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: coerce nullable stream usage counts * Studio: preserve system prompts with developer messages --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: wasimysaid <wasimysdev@gmail.com> |
||
|
Wasim Yousef Said
|
ccb471f5bf |
Improve local chat tool call flow (#5962)
Unify the Studio local tool-call loop (GGUF + safetensors) behind a shared ToolLoopController: ordered preface-then-tool-card rendering, duplicate-call de-looping with a forced final answer, XML-leak containment, and a parser fix that accepts closed <function=...> calls followed by trailing prose. Includes backend tests for the controller, strict parser, and GGUF route cursor reset. |
||
|
Daniel Han
|
187144d4e7 |
Reduce and tighten code comments and docstrings repo-wide (#6095)
Trim and tighten code comments and docstrings across the repository. Comment-only: every changed file verified code-identical to main via AST/token comparison. |
||
|
Daniel Han
|
8292e699e4 |
Studio: make code comments and docstrings more succinct (#6029)
Trim and tighten code comments and docstrings across studio/ Python. Comment-only: every changed file verified code-identical to main via AST/token comparison. |
||
|
Daniel Han
|
3ce187da02 |
Formatting: ruff line-length 100, kwarg-spacing passes, drop blank after short local imports (#6079)
Raise ruff line-length to 100 and extend the local pre-commit format pipeline (def-signature magic-comma normalization, short multi-line assert collapse, kwarg '=' spacing, blank-line-after-short-import removal, adjacent string-literal / f-string+plain merge, redundant-pass pruning). Every transform re-checks the file AST and is dropped if it would differ; the whole-repo reformat is verified AST-identical per file and idempotent. |
||
|
Michael Han
|
1e811acd62 |
Studio: tag MLX loaded models as MLX instead of Base in chat (#6067)
* Studio: tag MLX loaded models as MLX instead of Base in chat * Studio: tag MLX named hub defaults via name heuristic |
||
|
Michael Han
|
1b588cd141 |
Studio: emit usage and timings for MLX generation speed stats (#6068)
* Studio: emit usage and timings for MLX generation speed stats * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: make MLX generation stats request scoped --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Daniel Han
|
4c06c1dcc7 |
Studio: enable audio input for Gemma 4 GGUFs; default chat model to Qwen3.5-4B-MTP (#6000)
* Studio: enable audio input for Gemma 4 GGUF models Audio file upload was disabled for Gemma 4 vision+audio GGUFs (e.g. gemma-4-12b-it-GGUF) even though their mmproj carries an audio encoder (clip.has_audio_encoder, gemma4ua). Two causes: - Audio-input detection only matched Gemma 3n's <audio_soft_token>; Gemma 4 uses <|audio|>, so audio_vlm was never detected. - The GGUF load/status responses hardcoded has_audio_input=False, so the flag was dropped even when audio_vlm was detected (affected Gemma 3n GGUFs too). Changes: - Recognize <|audio|> alongside <audio_soft_token> in the llama-server token probe and the tokenizer-config pattern. - Read clip.has_audio_encoder from the mmproj as an independent, model-agnostic signal (read_mmproj_audio_capability). - Emit the computed has_audio_input on the GGUF load/status responses. - Tests for the new pattern and the mmproj reader. * Studio: default chat model and dataset helper to Qwen3.5-4B-MTP Switch the auto-loaded chat default and the dataset-analysis helper GGUF from gemma-4-E2B-it to unsloth/Qwen3.5-4B-MTP-GGUF (UD-Q4_K_XL). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Long Yixing
|
63dc27f76e |
fix(studio): disable mlx gc for none (#5991) | ||
|
Lee Jackson
|
e0ff6a1404 |
Studio: manage chat history with projects (#5725)
* feat: align project sidebar UX with ChatGPT * feat: align project sidebar UX with ChatGPT * feat(chat): load stored project list * feat(chat): add project sidebar workflows * fix: stabilize project page navigation * fix: projects chat loading * fix: show project chat thread * style: sidebar project spacing and hover clipping * style: add expandable project chat history and move-to-project submenu * feat: polish project sidebar * feat: persist project sandbox paths * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: only create sandbox project workspace dir * feat: add optional project workspace deletion from delete dialog * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix: stabilize chat projects CI failures * fix: polish project chat navigation * Studio: manage chat history with projects Group chats into projects with a dedicated projects page and route. Sidebar shows recents with per-row actions and a vertical more-vertical menu, and the sidebar scrollbar stays hidden so rows never shift on hover. Includes chat settings and composer refinements. * Studio: projects sidebar and breadcrumb polish Sidebar: - Remove the Compare nav item. - Widen the sidebar to match the projects layout. - Replace the scroll-gated bottom fade with a static fade pinned above the profile box, so it no longer attaches to Recents or lags the collapse and expand animation. Topbar breadcrumb (chat-page): - On a project landing show "Projects" linking to the projects list. - Inside a project chat show the project name and chat title, with the project name linking back to that specific project page. - Drop the divider between the model selector and the breadcrumb. * Studio: make project workspace delete test cross-platform test_chat_project_delete_files_removes_workspace rooted the project under pytest tmp_path, which resolves to /private/tmp on macOS. The workspace delete guard refuses paths under the system denylist by design, so the test passed on Linux CI but failed on macOS. Add a workspace_projects_home fixture that keeps tmp_path on Linux and Windows (CI unchanged) and falls back to a home subdir only when the temp root is on the platform denylist. Derive the workspace path from the created project so it tracks the projects home. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: satisfy import-hoist check for new path re-exports documents_root and project_workspaces_root are re-exported from utils.paths but only referenced as __all__ string literals, which the import-hoist safety net does not count as a use. It flagged the two newly added re-exports as unused imports and failed Source lint. Name-load both via a module-level _REEXPORTED tuple so the check sees them used. No behaviour change; consumers still import them from utils.paths. * fix: avoid projects empty-state flash * fix: batch chat search indexing * Studio: polish chat sidebar, run settings, and search - Use the native OS scrollbar for the chat sidebar, Run settings panel, and chat search list instead of a custom scrollbar - Highlight the active run in the sidebar and keep chat search available during training - Stop the training log view from replaying when navigating back to a run - Rename the chat settings panel to Run settings and align its toggle icon and position - Tighten heading and sidebar letter spacing and lighten the Train and Recents labels - Match the search dialog corner style across light and dark and drop the stray border - Make the MCP Servers section header plain text instead of a link - Remove a stray .orig backup file * studio/frontend: restore Compare entry point in the sidebar The chat-projects sidebar redesign dropped the Compare nav item and moved it to thread-sidebar.tsx, which is not imported or rendered anywhere. That left no way for a user to start a new model comparison (enterCompare only fired from the guided tour and the training handoff), and broke the Compare/Recipes/Export UI smoke test that clicks [data-tour="chat-compare"]. Re-add the Compare NavItem to the New Chat / Search group, carrying data-tour="chat-compare" and the same new-comparison navigation as before. * studio/frontend: use Unsloth green for the fallback profile avatar Switch the initials-avatar background from blue to #14b789 so the sidebar and edit-profile avatar match the Unsloth brand colour. * studio/frontend: turn project breadcrumb into a project switcher dropdown * studio/frontend: stop project card kebab clicks from opening the project * studio/frontend: hide project switcher outside projects * studio/frontend: stabilize project switcher loading * style: project switcher alignment --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: shimmyshimmer <107991372+shimmyshimmer@users.noreply.github.com> Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local> Co-authored-by: Roland Tannous <rolandtannous@gravityq.ai> Co-authored-by: Roland Tannous <115670425+rolandtannous@users.noreply.github.com> |
||
|
Wasim Yousef Said
|
dfba4cc5ca |
Studio: add HTML artifacts to chat (#5772)
* Studio: add chat HTML artifact primitives * Studio: add local render_html tool support * Studio: wire render_html artifacts in chat UI * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: add chat artifact surface * Studio: mount chat artifact panel and overlay * Studio: fix chat artifact review regressions * Studio: fix chat artifact panel and sandbox previews * Studio: address chat artifact review follow-ups * Studio: polish chat artifact UI affordances * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: scope artifact IDs by message to prevent cross-turn collisions * Studio: fix artifact panel for local threads and surface tool errors * Studio: restrict artifact frame embedding to same-origin * Studio: stop local chat thread remount loop * Studio: fix chat artifact store cleanup regressions * Studio: shim artifact preview storage in sandbox * feat(chat): add artifact rendering controls * fix(chat): show artifact progress during tool calls * fix(chat): refine artifact preview behavior * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * fix(chat): ignore tool markers inside arguments * feat(chat): polish artifact preview panel * fix(chat): stabilize artifact panel behavior * fix(inference): merge duplicate Anthropic tool starts * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Daniel Han
|
4b6a733fc3 |
Studio: harden stdio MCP gating and fix transport edge cases (#5892)
* Studio: harden stdio MCP gating and fix transport edge cases - Gate the Data Recipe stdio path behind UNSLOTH_STUDIO_ALLOW_STDIO_MCP so a hosted deployment cannot spawn local processes through recipes - Enforce the gate inside _client() so the transport sink cannot spawn when disabled - keep_alive=False so stdio probes/calls do not leave orphan subprocesses - Force OAuth off for stdio servers on create and update - Drop stored headers when a server switches transport type - Reject a command whose first token is a URL scheme - Add MCP gate and improvement tests * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * tests: skip Data Recipe stdio tests when data_designer is absent The data_designer plugin is only installed in the Studio test job, so guard the two build_mcp_providers tests with importorskip so the core matrix skips them instead of failing on ModuleNotFoundError. --------- Co-authored-by: Daniel Han <michaelhan2050@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|
Daniel Han
|
6cc2220e78 |
Studio: clearer error for diffusion GGUFs loaded as chat models (#5857)
Classify llama-server startup failures so diffusion/image GGUFs (FLUX, Qwen-Image, LTX, ERNIE-Image, Z-Image, ...) point users to the Images page instead of a misleading out-of-memory error. Other unknown architectures get a precise unsupported message; Ollama and OOM fallbacks are preserved. Architecture is matched exactly against general.architecture, covering the arches Unsloth ships as GGUF: flux, qwen_image, ltxv, wan, lumina2. Fixes #5842. |
||
|
oobabooga
|
ff00fdd155 |
Studio: add stdio MCP server support (#5863)
* Studio: add stdio MCP server support * Fix stdio command validation and Windows quoting |
||
|
Daniel Han
|
8ec9a74fd3 |
studio: ROCm cleanups follow-up to #5301 (#5874)
Follow-up cleanups to the merged AMD ROCm support PR #5301: 1. De-duplicate the torchao Windows-ROCm import stub into a single shared module (studio/backend/core/_torchao_stub.py); both workers call one install_torchao_windows_rocm_stub() entrypoint. 2. Align the gfx name/arch comment columns in setup.sh and setup.ps1. 3. Isolate the float16 dtype fallback to AMD without native bf16; NVIDIA keeps dtype=None so unsloth's own bf16/fp16/FORCE_FLOAT32 detection is honored. 4. Hoist unconditional stdlib imports (gc, glob, re, subprocess, copy, types, sys, importlib.metadata) from function bodies to module top across the PR #5301-touched files; heavy/optional/relative imports stay lazy. 5. bitsandbytes Windows-ROCm install now uses plain pip (force_pip=True) instead of UV_SKIP_WHEEL_FILENAME_CHECK, per the AMD hackathon docs. Also adds scripts/verify_import_hoist.py (a scope-aware LEGB AST resolver that catches dangling-alias and rename-clash bugs in import-hoist refactors) and wires it into the Lint CI source-lint job as a self-test plus a pull_request compare gate. |