Commit graph

5,711 commits

Author SHA1 Message Date
Daniel Han
95b50ec57a Compress installer/Studio comments to essentials (comment-only)
Follow-up to the merged #5940. Trims verbose comments across the
Windows/WSL installer and Studio backend to their load-bearing content
(constraints, env-var names, issue refs, magic-value rationale),
removing restated-code narration and multi-sentence justifications.

Comment-only and machine-verified: every .py file is AST-dump-identical
(docstrings normalized), every .ps1 is non-comment-token identical, and
every .sh has zero non-comment-line changes with bash -n clean, all
checked against main. Install suites (340 passed) and backend suites
(282 passed) unchanged. Net -159 comment lines across 15 files.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-13 02:37:08 -07:00
alkinun
8e57c5ee34
Fix Responses tool output content arrays (#6287)
* Fix Responses tool output content arrays

* Fix/adjust Responses tool output content for PR #6287

* Fix/adjust original image detail for PR #6287

---------

Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
2026-06-13 02:23:34 -07:00
Daniel Han
985792a83b
Installer: drop redundant -WindowStyle Hidden from the Windows launcher VBS (#6284)
* Installer: drop redundant -WindowStyle Hidden from the Windows launcher VBS

The desktop / Start Menu shortcut launches Studio through a generated
launch-studio.vbs that runs:

  shell.Run "powershell ... -WindowStyle Hidden -File launch-studio.ps1", 0, False

The second argument to shell.Run is intWindowStyle 0 (hidden), so WScript
already launches the child windowless. The child -WindowStyle Hidden is
therefore redundant: dropping it keeps the launcher hidden and behaviour
identical, while removing the WScript-spawns-hidden-ExecutionPolicy-Bypass
PowerShell token combination that antivirus heuristics weight. That shape was
reported as a Kaspersky HEUR:Trojan.VBS.Agent false positive during install.

Adds tests/studio/install/test_launch_studio_launcher.py to stop the flag from
being reintroduced and to assert the launcher stays windowless via
shell.Run(cmd, 0, False).

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: Daniel Han <michaelhan2050@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-06-12 23:57:30 -07:00
Daniel Han
a8af0a1a4f
Fix llama.cpp prebuilt: skip already-installed same-release fallback (#6285)
* Fix llama.cpp prebuilt: skip the already-installed same-release fallback

install_prebuilt computes diffusion_visual_server_backfill_needed from the
newest candidate (plan.attempts[0]); when that is True it passed
existing_install_dir=None to validate_prebuilt_attempts, which disabled the
"existing install already matches this candidate" skip for the WHOLE plan. So
when the newest bundle failed validation the installer re-downloaded and
re-extracted an older fallback bundle that was already correctly installed.

Pass the real install dir always and gate the skip per-attempt: a matching
candidate is skipped unless that specific candidate still needs the
DiffusionGemma backfill re-extract.

Also make test_llama_cpp_search_roots_handles_studio_root_oserror read the full
_find_llama_server_binary / _kill_orphaned_servers method bodies instead of a
fixed 4000-char window. The except handler it asserts already exists, but the
function grew past the window so the guard silently failed; slicing to the next
sibling def keeps the check correct as the file grows.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: Daniel Han <michaelhan2050@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-06-12 23:56:55 -07:00
Michael Han
502d4ade1c
Tidy update banner and auth button spacing (#6279)
* Fix update banner button spacing

* Reduce auth submit button horizontal padding

* Wrap web update banner actions on narrow widths

---------

Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local>
2026-06-12 20:01:18 -07:00
Daniel Han
2b76c6855a
Bump install.sh / install.ps1 pin to unsloth>=2026.6.6 (#6270) 2026-06-12 11:31:07 -07:00
Daniel Han
cd270e2878
Studio: keep llama-server discovery from crashing on an access-denied candidate (#6268) v0.1.463-beta
* Studio: keep llama-server discovery from crashing on an access-denied candidate

_find_llama_server_binary probed candidates with Path.is_file(), which raises
PermissionError (WinError 5) when a path exists but is momentarily inaccessible
(antivirus lock, an install replace in flight, an elevated-install ACL),
aborting model validation. Treat a denied-but-present path as the real binary
so discovery returns it; absent paths still skip.

* Retry a transiently locked binary instead of returning a denied path

Returning a still-denied path only moved the PermissionError to the next
is_file() (probe_server_capabilities). Retry briefly so a transient lock
clears and discovery returns an accessible path; on a persistent lock return
nothing rather than a path downstream cannot stat.

* Studio: do not fall back to another llama-server when a pinned one is locked

A denied LLAMA_SERVER_PATH made discovery skip the explicit pin and run a
lower-priority managed or PATH binary, so a load could silently use a stale or
incompatible server. Split the probe into a file/absent/denied status: when the
pinned path exists but stays access-denied, warn and stop rather than falling
back to a different executable.

* Studio: never downgrade past a denied pinned or managed llama-server

Extend the no-fallback rule beyond LLAMA_SERVER_PATH: a present-but-denied
UNSLOTH_LLAMA_CPP_PATH or managed ($STUDIO_HOME/llama.cpp, ~/.unsloth/llama.cpp)
binary now reports temporarily-unavailable instead of silently launching a
lower-priority legacy or PATH server. Shared _scan_pinned/_unavailable helpers;
legacy in-tree and PATH stay genuine fallbacks (a denied candidate there just
continues).

* Studio: let diffusion asset lookup use a locked llama-server path for its dir

DiffusionGemma does not run llama-server; _find_diffusion_assets only needs the
install dir to find the adjacent llama-diffusion-gemma-visual-server. The
no-fallback rule returning None on a transiently locked llama-server therefore
hid an available visual-server and raised 'runner not found'. Add an
include_denied option so diffusion lookup gets the locked path (its dir is all
it needs), while inference keeps the no-denied-path, no-downgrade behavior.

* Studio: report a locked llama-server as temporarily unavailable, not missing

When the pinned/managed binary stays access-denied through the retries, discovery
returns None and load_model raised 'binary not found', a terminal error that
points users at reinstalling rather than retrying a transient AV/install lock.
Reuse include_denied to detect the locked path and raise a distinct
temporarily-unavailable, retry message instead.

* Studio: GGUF preflight treats a locked llama-server as present

The pre-download preflight (and so /api/inference/validate) used the default
discovery, which returns None for a transiently access-denied binary, so it
raised 'binary not found' for a binary that merely needs the lock to clear. Use
include_denied so the existence check counts a locked binary as present; the
load itself still reports a still-locked binary as temporarily unavailable.
2026-06-12 11:20:07 -07:00
Daniel Han
8c0e753673 Update _utils.py 2026-06-12 11:09:23 -07:00
Daniel Han
d7dd5556bb
Studio: backfill the DiffusionGemma visual-server on a tag-matching update (#6267)
* Studio: backfill the DiffusionGemma visual-server on a tag-matching update

An install made before the visual-server entered the copy allowlist (#6254)
matches on tag yet lacks the binary, so the update skip never backfills it and
DiffusionGemma fails with "runner not found". Re-extract the bundle when a
matching install is missing it, gated to the fork bundles that ship it so
upstream installs never thrash and so it self-limits once the binary lands.

* Studio: make the visual-server backfill bypass the existing-install short-circuit

The tag-matching skip is checked twice: once in install_prebuilt and again
inside validate_prebuilt_attempts via existing_install_matches_choice, which
only compares metadata and primary executables, not the DiffusionGemma
visual-server. So a stale install missing that binary still raised
ExistingInstallSatisfied and skipped the reinstall. Pass existing_install_dir
as None when a backfill is needed so the bundle actually re-extracts.
2026-06-12 10:56:09 -07:00
oobabooga
01d152b06f
Studio: only advertise a Cloudflare tunnel once it actually serves (#6264) 2026-06-12 14:36:35 -03:00
Michael Han
e017248616
Studio: polish update banner layout and sidebar icons (#6266)
Polishes the in-app update notification banners and the sidebar account menu.

Update banners (web and llama.cpp):
- Add a download icon next to the title, aligned with the title text.
- Even out the action row so the buttons share balanced padding and equal spacing.
- Let the row wrap on narrow viewports so Copy command is never clipped under overflow-hidden.

Sidebar:
- Use the setting-07 cog for the profile button and the Settings menu item.
- Use the sun-03 icon for the Light Mode menu item.
- Slightly enlarge the profile cog.

Frontend only. Build and typecheck pass.
2026-06-12 10:32:11 -07:00
Lee Jackson
31439d9eed
Studio: extend llama.cpp first-token timeout (#5841)
* fix: extend llama.cpp first-token timeout

* fix: timeout label pluralization

* studio: distinguish llama stream timeout phases

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix/adjust timeout handling for PR #5841

* Fix lint failure for PR #5841

* Fix/adjust stream timeout handling for PR #5841

* Fix/adjust first token timeout for PR #5841

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix/adjust passthrough timeouts for PR #5841

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix/adjust preheader stream cancellation for PR #5841

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix/adjust timeout PR diff for PR #5841

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix/adjust Python 3.9 stream iteration for PR #5841

* Fix first body timeout for PR #5841

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix first token timeout deadlines for PR #5841

---------

Co-authored-by: Roland Tannous <115670425+rolandtannous@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
2026-06-12 18:41:38 +02:00
Daniel Han
b91116cacc
studio: declare UNSLOTH_IS_PRESENT at backend startup (#6262)
The studio backend lazily imports unsloth_zoo submodules (export's
llama_cpp, hardware's vllm_utils, mlx, chat_templates). Each imports the
top-level unsloth_zoo, which raises 'Please install Unsloth via pip
install unsloth' unless UNSLOTH_IS_PRESENT is set (normally by import
unsloth). Only hardware.py set it per-site; the rest crash on a clean
install, and it surfaced on Windows. Set it once at backend startup (as
unsloth does on import) so every site, and the DiffusionGemma runner,
works across platforms. Follow-up to #6259 (which covered only the shim
subprocess env).
2026-06-12 08:16:01 -07:00
sqersters
cc32777f30
Studio: keep distinct bpw flavors of the same GGUF quant (#5729)
list_gguf_variants() keys files by _extract_quant_label(), which only captured the base quant token. Repos that ship the same base quant at multiple bits-per-weight (e.g. byteshape/Qwen3.6-35B-A3B-MTP-GGUF with three IQ4_XS files at 3.53/3.97/4.19 bpw) collapsed into a single row and Studio summed their sizes (~48 GB).

Extend the regex to capture an optional trailing -<N>(.<N>)?bpw modifier so each flavor produces a unique label. Round-trips through _find_local_gguf_by_variant and _download_gguf since both sides use the same extractor.

Fixes #5728.

Co-authored-by: Etherll <61019402+Etherll@users.noreply.github.com>
2026-06-12 17:57:25 +03:00
oobabooga
5300c047b6
Installer: drop the lemonade ROCm fallback now the fork ships identical per-gfx prebuilts (#6225)
---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2026-06-12 11:53:26 -03:00
Daniel Han
9d8daecf18
Bump install.sh / install.ps1 pin to unsloth>=2026.6.5 (#6260) 2026-06-12 07:41:25 -07:00
alkinun
c2b8da704d
studio: add keyboard navigation to model picker (#5628) v0.1.462-beta
* Fix model picker keyboard navigation

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Address model picker review feedback

* Fix model picker tab order with row actions

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Remove model picker keyboard contract test

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Etherll <61019402+Etherll@users.noreply.github.com>
2026-06-12 17:36:11 +03:00
Daniel Han
2a01ba8aad
DiffusionGemma: set UNSLOTH_IS_PRESENT for the shim subprocess (#6259)
A clean install could not run DiffusionGemma: the runner spawns
python -m unsloth_zoo.diffusion_studio.shim, and unsloth_zoo refuses to
import unless UNSLOTH_IS_PRESENT is set (normally by import unsloth). The
shim never imports unsloth, so the subprocess died with
'Please install Unsloth via pip install unsloth!' and the model load
failed with a 500. Set the flag in the runner child env, as unsloth does
on import.
2026-06-12 07:35:59 -07:00
Daniel Han
be2a122e21
install.sh: keep the studio launch from draining the curl | sh script (#6258)
When installed via 'curl -fsSL https://unsloth.ai/install.sh | sh', the
shell reads the script from the pipe on stdin. The optional 'start now'
step launches 'unsloth studio' in the foreground, and as a server it
inherits and drains the rest of the piped script from stdin. The shell
then hits EOF partway through the trailing if/case and dies with
'Syntax error: end of file unexpected (expecting fi)' (reported on WSL,
where /bin/sh is dash). Redirect the launch's stdin away from the pipe so
the script stays intact. The server does not read stdin and Ctrl+C still
works via the controlling terminal.
2026-06-12 07:35:17 -07:00
Daniel Han
4c4422893c Merge branch 'main' of https://github.com/unslothai/unsloth 2026-06-12 07:16:42 -07:00
Daniel Han
dd4afa64d3 Update _utils.py 2026-06-12 07:15:37 -07:00
alkinun
ac844b0be7
fix(studio): keep local GGUF vision on llama-server (#5770) v0.1.461-beta
* fix(studio): keep local GGUF vision on llama-server

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix(studio): lower local GGUF vision log level

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix(studio): find GGUF companions from variant dirs

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: imagineer99 <samleejackson0@gmail.com>
2026-06-12 15:09:31 +01:00
hoobnn
f033213c0b
Studio: account for mmproj VRAM in GGUF fit budget (#5825) (#5849)
* Studio: account for mmproj VRAM in GGUF fit budget (#5825)

Vision GGUFs load the mmproj projector onto the GPU via --mmproj
alongside the weights, but the context auto-sizing / GPU-selection
budget sized off _get_gguf_size_bytes(model_path), which counts only
the weight file(s). The projector was never added, so the budget was
too optimistic: context got mis-estimated and tight vision loads
spilled to system RAM / OOM'd.

Resolve the launch projector once before GPU selection and fold its
size into the fit budget. The same resolved path feeds both the budget
and the --mmproj launch flag, so the two cannot disagree. The summary
log now reports the projector size separately, keeping "GGUF size"
accurate.

Adds _mmproj_vram_bytes() + unit tests (no GPU / network / subprocess).

* Studio: simplify mmproj summary-log concatenation (#5825)

Address review: the summary log mixed explicit `+` with implicit
f-string concatenation. Extract the optional projector fragment into
`mmproj_note` so the logger.info uses uniform implicit concatenation.
No behavioral change.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: trim mmproj VRAM comments

---------

Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: imagineer99 <samleejackson0@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-06-12 15:04:08 +01:00
Daniel Han
7397c87843
Bump install.sh / install.ps1 pin to unsloth>=2026.6.4 (#6257) 2026-06-12 07:01:22 -07:00
Daniel Han
756265e3fd
DiffusionGemma: disable tools, enable artifacts canvas by default (#6255) v0.1.46-beta
DiffusionGemma serves via the visual runner, which streams per-step
canvas frames so the answer resolves live in the bubble. The agentic
tool loop (generate_chat_completion_with_tools) does not forward those
frames, so whenever a tool pill (Search/Code) was on the live canvas
silently vanished while text still streamed. DiffusionGemma is not a
tool-calling target anyway, so report supports_tools=False for it: the
chat always takes the frame-forwarding path, and the Search/Code pills
disable themselves (a local model has no builtin web search either).

Also turn the artifacts canvas on by default for DiffusionGemma so a
full-HTML answer (e.g. a playable game) renders as an interactive
sandboxed card without the user flipping the global artifacts toggle.
2026-06-12 06:55:14 -07:00
Michael Han
587e59ca83
Studio: clearer check mark and even plus-menu hover padding (#6253)
* Studio: use a clearer check mark for the app-wide tick

Route every check mark through a shared tick-icon module with a plain
tick geometry, sized slightly larger than the stock icon so it reads
clearly in menus. No Pro icon assets are vendored.

* Studio: even out plus-menu hover padding on all sides

Trim the plus-menu side gutter to 9px so it matches the 0.5rem top/bottom
padding, and drop the container radius to 21px to keep corners concentric.

* Studio: bake the tick fallback stroke at 1.5 to match the icon set

---------

Co-authored-by: shimmyshimmer <info@unsloth.ai>
2026-06-12 06:44:08 -07:00
Michael Han
99704ffe47
Studio: project sources backed by RAG (#6205)
* Studio: make project sources work with RAG and polish project UI

Projects had a disabled Sources tab with an Add sources placeholder.
This wires it up end to end on top of the RAG engine:

- Add a project scope to the RAG store, ingestion and retrieval
- New endpoints: POST/GET /api/rag/projects/{id}/documents
- search_knowledge_base resolves kb, project and thread scopes; an
  explicit KB stays exclusive, project and thread scopes combine
- Multi-scope search: FTS uses scope IN (...), vec0 KNN runs per
  scope and merges by cosine score
- Lazy ALTER TABLE adds documents.project_id on existing databases
- Deleting a project also removes its indexed sources
- Sources tab now uploads with progress chips and drag and drop
- Chats inside a project auto-enable retrieval over project sources
  when the project has indexed documents (cached probe, no Docs pill
  needed); external providers still never receive rag_scope

UI polish:
- Rounder project cards with folder icon chip and softer shadow
- Project header icon in a rounded chip
- Chats/Sources pills and Add sources button without borders

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: match Add sources button shadow to the chat composer in light mode

* Studio: round project switcher hover pill and pad the folder icon

* Studio: remove border from project sources box

* Studio: grey hover on project cards and menu, move search into header, widen page spacing

* Studio: shorten sources copy, white header pills with composer shadow, fixed-width search, hub-size page headings

* Studio: align project landing blocks to the composer width

* Studio: restore muted background and flat look on projects header controls

* Studio: darker grey hover on project cards in light mode

* Studio: soften project card hover grey

* Studio: keep project card menu button visible while its menu is open

* Studio: drop focus outlines and rings on buttons and clickable icons, keep input focus styles

* Studio: address review feedback on project sources

- Remove uploaded files from disk when a project is deleted, confined
  to the uploads root
- 404 project uploads when the project does not exist, matching the KB
  endpoint
- Guard lexical search against an empty scope list
- Re-invalidate the project sources probe after uploads and removals
  settle so a chat sent mid-upload cannot cache a stale negative
- Keep keyboard focus rings: only mouse focus drops the Tailwind ring,
  the browser default outline stays removed

* Studio: add a green New badge to the project Sources tab

* Studio: unify New pills, fully round with soft emerald fill and no border

* Studio: a touch more vertical padding on New pills

* Fix project RAG source edge cases for PR #6205

* Fix duplicate RAG upload cleanup for PR #6205

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
2026-06-12 15:42:51 +02:00
Daniel Han
ebe3bbd041 Versioning 2026-06-12 06:27:38 -07:00
Daniel Han
6c493b4076
Attach DiffusionGemma visual-server from the prebuilt bundle (#6254)
The prebuilt bundles ship llama-diffusion-gemma-visual-server, but
runtime_patterns_for_choice pruned it, so a fresh install never placed
it next to llama-server. ensure_diffusion_visual_server then found no
standalone release asset and skipped it, leaving Studio unable to serve
DiffusionGemma GGUFs natively (it required DG_VISUAL_BIN or a source
build). Keep the binary in the runtime allowlist on Linux, macOS and
Windows so it lands in build/bin and is activated automatically.
2026-06-12 06:26:35 -07:00
Wasim Yousef Said
ce34aba932
Fix Studio S3 dataset panel layout (#6252) 2026-06-12 15:19:52 +02:00
oobabooga
522f308be2
Studio: cache MCP tool discovery instead of re-probing every chat send (#5828)
* Studio: cache MCP tool discovery instead of re-probing every chat send

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Stop re-probing offline/down MCP servers every time

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Add tests for mid-probe delete and OAuth cool-off paths

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Don't cool-off a server edited or deleted mid-probe

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Guard MCP refresh cache writes

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Roland Tannous <115670425+rolandtannous@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: imagineer99 <samleejackson0@gmail.com>
2026-06-12 14:18:06 +01:00
ashzak
aefe904d66
feat(studio): implement S3 dataset loading (completes #5951) (#6222)
* feat(studio): add S3 dataset configuration foundation (#4539)

Add foundational types and configuration for S3 bucket dataset loading:

- Add S3Config type to frontend training types
- Add S3Config Pydantic model to backend training models
- Add "s3" as a DatasetSource option
- Add s3Config state and setS3Config action to training config store
- Add i18n translations for S3 configuration (English and Chinese)

This provides the type definitions and UI text for S3 integration.
Full implementation requires boto3 dependency and data loading logic.

Refs: #4539

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Wire S3 config into training pipeline and prevent secrets persistence

- Pass s3_config from request into training_kwargs so it flows to training subprocess
- Add s3Config to NON_PERSISTED_STATE_KEYS to prevent AWS secrets from being
  saved to localStorage

Addresses code review feedback on PR #5951.

* Exclude S3 config from database persistence to protect secrets

Filter out s3_config (which contains secret_access_key) from the
config_json stored in training_runs table, preventing AWS credentials
from being persisted to disk.

Addresses P1 security feedback on PR #5951.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Re-raise HTTPException in start_training and defer s3 DatasetSource widening for PR #5951

* Redact s3_config from W&B run config and accept camelCase S3 credential aliases for PR #5951

* feat(studio): implement S3 dataset loading end-to-end

Builds the actual S3 loader on top of the hardened #5951 foundation,
turning the 501-gated scaffold into a working dataset source.

Backend:
- Add core/training/s3_dataset.py: lists and downloads supported dataset
  files (parquet/json/jsonl/csv) from an S3 bucket to a temp dir, using
  IAM-role or access-key credentials. boto3 is imported lazily (optional dep).
- Wire s3_config into UnslothTrainer.load_and_format_dataset (downloads then
  reuses the existing local-file path) and thread it through worker.py.
- Replace the 501 "not implemented" gate with a boto3-availability guard so
  S3 works when boto3 is present and fails clearly when it is not.
- Add boto3 to studio.txt requirements.
- Add tests/test_s3_dataset.py (8 tests) covering download/filtering,
  collisions, missing-boto3, and S3Config camelCase/IAM validation.

Frontend:
- Widen DatasetSource to include "s3"; add s3_config to the training payload
  type and mapper; add an S3 validation branch and selectS3Source store action.
- Add s3-config-form.tsx (bucket/region/prefix/keys/IAM toggle) reusing the
  existing studio.dataset.s3.* i18n strings.
- Add a Hugging Face / Local / Amazon S3 source toggle in dataset-section;
  the S3 config card replaces the dataset combobox when S3 is selected.
- Fix DatasetPreviewDialog to accept the widened DatasetSource type.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix S3 dataset loader for PR #6222

* Fix S3 dataset edge cases for PR #6222

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix S3 IAM payload handling for PR #6222

* Block multimodal S3 datasets for PR #6222

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Ash <ash@MacBook-Pro.local>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
2026-06-12 14:52:04 +02:00
Daniel Han
a70146df0f
Studio: bundle Gemma 4 chat templates (E2B/E4B + larger) and auto-apply to unsloth/gemma-4-*-GGUF (#6245)
* Studio: override chat template for unsloth/gemma-4-*-GGUF with bundled gemma-4.jinja

The chat templates baked into the shipped unsloth/gemma-4-*-GGUF quants predate
Google's gemma-4 chat-template PR #118 and lack the preserve_thinking flag, so
Studio cannot surface the "Preserve thinking" toggle for Gemma 4. Bundle the updated
template and override the embedded one at llama-server launch via --chat-template-file,
scoped to the gemma-4 GGUF family, so users do not need to re-download any quant.

- Add studio/backend/assets/chat_templates/gemma-4.jinja (PR #118 based;
  preserve_thinking defaults false, the one deliberate divergence from upstream).
- Add core/inference/chat_templates.py: gemma-4 GGUF matcher plus an
  effective-override resolver (explicit user template still wins).
- Wire the resolver into routes/inference.py ahead of the reload-dedup check and
  both load_model calls so the live backend and the incoming request compare against
  the same template text (no spurious reloads).
- Default preserve_thinking off in the launch-time chat_template_kwargs so direct
  API callers match the UI default.
- Ship the asset via package-data and add unit tests.

* Studio: ship E2B/E4B edge variant of the bundled Gemma 4 template

Google ships two distinct gemma-4 chat templates: E2B and E4B omit the empty
"<|channel>thought<channel|>" block on enable_thinking=false, while the
12b/26B-A4B/31B family emits it (confirmed against google/gemma-4-E2B-it,
-E4B-it, -12b-it, -26B-A4B-it, -31B-it; the two families differ only in that
one block). The single PR #118 based template followed the larger-model
behavior, which is wrong for the E2B/E4B GGUFs this feature most targets.

- Add studio/backend/assets/chat_templates/gemma-4-edge.jinja: identical to
  gemma-4.jinja minus the empty-thought-block, matching E2B/E4B behavior.
- Route unsloth/gemma-4-E2B-it-GGUF and -E4B-it-GGUF to the edge template;
  12b/26B-A4B/31B keep gemma-4.jinja.
- Extend tests for the edge matcher, per-family routing, and the empty-thought
  block difference (off for edge, on for standard).

* Studio: address review feedback on the gemma-4 template override

- Normalize owner-less shorthand model ids in the template matcher: a bare
  "gemma-4-E2B-it-GGUF" is canonicalized to "unsloth/" the same way
  ModelConfig.from_identifier does, so shorthand loads still get the override
  (and the preserve_thinking capability) instead of falling back to the
  embedded template.
- Scope the test's module stubs with unittest.mock.patch.dict instead of
  sys.modules.setdefault, and only stub deps that are missing, so the global
  module registry is not polluted for tests that run afterwards.
- Guard the Jinja render tests with pytest.importorskip("jinja2") so the suite
  stays runnable in minimal Studio environments where jinja2 is not present.
- Add tests for shorthand resolution.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: address 10-reviewer P1 findings on the gemma-4 template override

- /status no longer surfaces Studio's auto-applied bundled template as a
  user-authored chat_template_override. The frontend adopts that field as
  editable state and would otherwise re-send the gemma-4 template as an explicit
  override for a later, unrelated model. /status now reports None when the live
  override equals the model's auto-resolved bundled template.
- When a bundled family template is in effect, strip an inherited
  --chat-template-file from llama_extra_args too (not only when the raw request
  set chat_template_override). Otherwise a stale inherited template, appended
  last, shadows the bundled one while Studio reports the bundled template's
  capabilities.
- Write the temp chat-template file as UTF-8 explicitly, and keep the bundled
  templates ASCII (replaced em dashes), so non-UTF-8 Windows locales cannot raise
  UnicodeEncodeError or emit a mis-encoded template. Added an ASCII guard test.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-06-12 05:49:39 -07:00
Daniel Han
90cb9499e8
Studio: serve DiffusionGemma with live in-place denoising and honest stats (#6250)
* Studio: serve DiffusionGemma GGUFs with the on-device visual decoder

* Studio: render the DiffusionGemma denoising canvas live in chat with honest stats

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: harden DiffusionGemma runner resolution (Windows .exe, build/bin lookup, clear stale audio flag, safe PYTHONPATH, Linux-only pdeathsig)

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-06-12 05:48:06 -07:00
Michael Han
6d22829ef7
Studio: refine menu chevron, tick icon, and one-line plus-menu shape (#6251)
* Studio: refine menu chevron, tick icon, and one-line plus-menu shape

- Shrink submenu > chevron to size-3.5 in the dropdown, context, and
  menubar sub-triggers so it stays centered but reads lighter.
- Use the tick-02 duotone-standard geometry for the app-wide check mark
  via a shared tick-icon module.
- Drop the fully rounded pill on one-line plus-menus so they match the
  standard menu shape.

* Studio: nudge submenu chevron down to 13px

* Studio: nudge submenu chevron down to 12px

* Studio: drop vendored Pro tick geometry, keep free check icon

The tick-02 duotone-standard path is a Hugeicons Pro asset and is not part
of the MIT free pack, so vendoring it into the source is a licensing risk.
Revert the check mark to the bundled free Tick02Icon and remove the shared
tick-icon module. Keeps the chevron sizing and one-line plus-menu changes.

---------

Co-authored-by: shimmyshimmer <info@unsloth.ai>
Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local>
2026-06-12 05:44:55 -07:00
oobabooga
f4fc06b5bb
Studio: persist speculative decoding preference across restart and model switch (#6169)
* Studio: persist speculative decoding preference across restart and model switch

* Studio: persist speculative preference on apply, not on edit

* Fix/adjust speculative decoding persistence for PR #6169

* Fix speculative ngram alias for PR #6169

* Fix compare speculative preference for PR #6169

---------

Co-authored-by: wasimysaid <wasimysdev@gmail.com>
2026-06-12 14:37:44 +02:00
Nilay
911ceba7fa
Studio: new-chat shortcut, composer draft autosave, archive threads (#5771)
* new-chat shortcut, composer draft autosave, archive threads

* fix

* Studio: harden chat UX additions for legacy threads and unavailable storage

Two robustness fixes on top of the new chat UX features:

- groupThreads: coerce archived to a boolean before comparing
  (Boolean(t.archived) !== archived). Threads from the older browser-only
  Studio, or any record predating the archived field, can carry
  archived === undefined/null. The raw `!== archived` comparison dropped
  those from BOTH the Recents and Archived sidebar groups, hiding existing
  chats. Treat missing as not-archived so legacy chats still appear in
  Recents.

- composer draft autosave: wrap the localStorage read and write in
  try/catch. When storage is unavailable (private mode, disabled cookies,
  blocked storage) or full (quota exceeded), getItem/setItem throw; the
  throw in the restore effect would surface to React and break the chat
  page. Draft persistence is best-effort, so degrade quietly.

Verified with bun unit tests on the real groupThreads (legacy undefined no
longer vanishes) and Playwright across chromium, firefox and webkit (draft
save/restore/isolation/clear, new-chat shortcut + crypto.randomUUID, and
localStorage blocked/quota throw handling). tsc clean; no new eslint findings.

* Fix composer draft bleed and orphan cleanup for PR #5771

Centralize composer draft storage in a small util and tighten two edge cases:

- New chat draft bleed: every new chat shared the chat-draft:__new__ slot, so
  starting a fresh chat could restore the previous one's half-typed text. Clear
  that slot at every new chat entry point (sidebar buttons and Cmd/Ctrl+Shift+O).

- Orphan drafts: deleting a thread left its chat-draft:<id> key behind. Clear
  the draft for every deleted thread id.

New util utils/composer-draft.ts owns the key format and wraps localStorage in
try/catch (private mode, blocked storage, quota), replacing the inline copy in
thread.tsx so reads and writes stay best effort everywhere.

* address review

* fix: remove unused chat sidebar binding

---------

Co-authored-by: Daniel Han <michaelhan2050@gmail.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: imagineer99 <samleejackson0@gmail.com>
2026-06-12 13:36:31 +01:00
Daniel Han
3427e3fd62
Studio: fix Downloaded model list disappearing and order it by last download (#6247)
* Studio: fix Downloaded model list disappearing and order it by last download

The chat model picker scan for cached GGUF and safetensors models aborted
whenever an auxiliary Hugging Face cache dir (such as ~/.cache/huggingface/hub)
was unreadable, returning an empty list. That hid the Downloaded section and
let already downloaded models appear under Recommended. Isolate each cache
probe so an inaccessible directory is skipped instead of failing the scan.

Also order Downloaded newest-first using cached blob mtimes (multi-quant repos
group by their most recent quant), keep the section visible while searching,
and make the per-quant downloaded check per-snapshot and mmproj aware so a
Recommended quant is never falsely marked downloaded.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: harden gguf-variants scan and dedupe by newest timestamp

Guard f.stat() per file so a broken symlink or unreadable file in a
snapshot no longer aborts the downloaded check early, and match quant
labels case-insensitively. When the same repo is present in multiple
caches with equal size, keep the newest last_modified so Downloaded
ordering reflects the most recent copy.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: apply cache-scan guards to sibling endpoints found in review

Extend the inaccessible-cache guard and mmproj/stat hardening to the
parallel HF cache code paths flagged in review:

- list_local_models and the Hub inventory scan now skip an unreadable
  auxiliary cache instead of returning 500.
- The GGUF download-progress endpoint excludes mmproj adapters and
  guards f.stat() so one bad file does not zero a repo's progress.
- The offline snapshot scanner guards its is_dir() probes.
- The chat-only picker no longer renders a blank list when a search
  matches only cached non-GGUF models.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-06-12 05:27:34 -07:00
Michael Han
cd8e6c1e77
Studio: use px-2.5 for Hub option menu padding (#6249)
Swaps the arbitrary px-[10px] on the Hub option menu surface for the
standard px-2.5 utility. Both resolve to 10px, so this is a no-op
visually and just keeps to the standard spacing scale.
2026-06-12 04:57:28 -07:00
Michael Han
9df607c0de
Studio: fully rounded Hub pills and refreshed menu icons (#6248)
Rounds out the Hub chrome and aligns a handful of icons with the rest
of the app. The header stat pills, HTTP/Xet toggle, HF token button,
info chips, status badges and the single-row download card are now full
pills, and the All formats style dropdown matches the composer + menu
geometry. Also widens the chat options menu so "Move to project" fits
on one line, and swaps a few icons to their HugeIcons equivalents.

Changes:
- hub.css: hub-tag-soft/hub-tag-meta pills go fully rounded; a single-
  row hub-download-card reads as a full pill via :has(), while the
  expanded card keeps its 22px radius, border and background.
- hub-option-menu.tsx: dropdown surface and items match the composer
  + menu (22px container, 10px side padding, 12px item radius).
- hf-token-indicator.tsx, transport-toggle.tsx: fully rounded with a
  fixed 26px height so they line up with the neighbouring pills.
- model-inspector.tsx, dot-tag.tsx, gguf-download-card.tsx: status and
  info chips go fully rounded.
- models-header.tsx: Eject uses RemoveCircleIcon; active model pill
  fully rounded.
- on-device-folders-dialog.tsx: folder rows use Folder02Icon.
- pickers.tsx: Recommended star uses the HugeIcons star.
- command.tsx, chat-search-dialog.tsx, prompt-storage-dialog.tsx: search
  fields use the HugeIcons Search01Icon.
- app-sidebar.tsx: chat options menu widened to w-56.

Pure styling and icon swaps, no behavior changes.
2026-06-12 04:51:19 -07:00
Daniel Han
240c0c3500
Studio: fix WSL Strix Halo GPU on reinstall (ROCDXG drop-in + system HIP before bundle) (#6227)
* install.sh: persist ROCm-on-WSL drop-in even when rocminfo already works

_maybe_bootstrap_rocm_wsl calls _ensure_rocm_probe_env (which exports a
transient HSA_ENABLE_DXG_DETECTION + adds /opt/rocm/bin to PATH on the
installer process) right before the "rocminfo enumerates gfx1151 -> already
set up, return early" gate. On any reinstall over an existing /opt/rocm --
the common case, since the uninstaller keeps shared ROCm userspace but
removes /etc/profile.d/unsloth-rocm-wsl.sh -- that probe env makes rocminfo
succeed, so the gate returns 0 WITHOUT ever persisting the drop-in. The
transient env dies with the installer, so the next login shell (Studio,
llama-server) sees no GPU: torch cuda_avail=False, rocminfo finds nothing,
the llama.cpp ROCm prebuilt segfaults on a GPU it can't reach.

Factor the drop-in writer into _persist_rocm_wsl_dropin() and call it before
the early return so the persistent env is restored whenever librocdxg is
present. Idempotent (only writes when the drop-in is missing), gated on
librocdxg so it never fires on non-WSL/non-ROCDXG hosts, root-writes or
sudo-tees like before. The fast-path branch now reuses the same helper.

Reproduced on gfx1151 (Radeon 8060S) under dash (the curl|sh shell):
before the fix a reinstall left the drop-in absent and torch cuda_avail
False; after, the drop-in is persisted and a fresh login shell reports
cuda_avail True. Verified under both dash and bash, and idempotent on
re-run.

* Studio WSL: load system HIP before a prebuilt's bundled runtime (gfx1151)

The lemonade / published llama.cpp ROCm prebuilts bundle their own HIP
runtime (libamdhip64) built for bare-metal Linux. In WSL the GPU is reached
through the system ROCm's librocdxg bridge over /dev/dxg, which the bundled
runtime cannot drive -- it segfaults on the first GPU call. So:

  - install_llama_prebuilt.py: the prebuilt's llama-quantize/llama-server
    validation runs with the bundle dir first on LD_LIBRARY_PATH, segfaults
    (empty stderr), and the install silently falls back to a CPU source build
    (which on this host can't even build for GPU -- hipcc absent). The Strix
    Halo WSL user ends up on CPU despite a working GPU.
  - llama_cpp.py: even if a GPU prebuilt were kept, the serve-time launcher
    put the bundle dir first too, so it would crash at load.

Fix: on a ROCDXG WSL host (gated on /dev/dxg + "microsoft" /proc/version +
a librocdxg-providing /opt/rocm), prepend the system ROCm lib dir to
LD_LIBRARY_PATH so the WSL-capable libamdhip64 + librocdxg load first, while
the bundle still supplies libggml-hip / librocblas with the gfx1151 kernels.
Set HSA_ENABLE_DXG_DETECTION=1 alongside. Added _wsl_system_rocm_lib_dirs()
to both modules (kept identical so a prebuilt that passed install validation
runs the same way at serve time). Strict no-op on bare-metal Linux, NVIDIA,
macOS, and Windows.

Verified on gfx1151 (Radeon 8060S) in WSL (ROCm 7.2.1 + librocdxg, Adrenalin
ROCDXG): before, the lemonade gfx1151 prebuilt segfaulted and the install
fell back to a broken CPU build; after, install_llama_prebuilt validates and
keeps the GPU prebuilt (source=published, prebuilt_fallback_used=False), and
Studio serves Qwen3-1.7B-GGUF at 53 tok/s with the model resident in GPU
memory (llama-server device_info: ROCm0 = AMD Radeon 8060S).

* tests: cover the WSL ROCDXG drop-in + system-HIP-ordering fixes

- _wsl_system_rocm_lib_dirs: no-op without /dev/dxg, on bare-metal Linux,
  and on WSL without librocdxg; returns the system lib dir on a ROCDXG WSL
  host.
- binary_env: prepends the system ROCm lib dir ahead of the bundle and sets
  HSA_ENABLE_DXG_DETECTION on WSL; unchanged on bare-metal Linux.
- install.sh: _persist_rocm_wsl_dropin exists, is gated on librocdxg, and the
  rocminfo-already-works early return calls it before returning.
- llama_cpp.py: the serve-time launcher prepends the WSL rocm dirs before the
  bundle dir (mirrors binary_env).

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Tighten WSL ROCDXG fix comments (no logic change)

Condense the drop-in / system-HIP-ordering comments and docstrings added in
this PR. Verified comment-only via AST parse + py_compile + sh/bash -n, the
308-test rocm_support suite, and a dash functional re-run of the bootstrap
(drop-in still persisted, env still set).

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-06-12 04:46:39 -07:00
Tai An
b72cf0af24
fix(studio/responses): forward chat_template_kwargs enable_thinking to chat request (#6202)
* fix(studio/responses): forward chat_template_kwargs enable_thinking to chat request

The /v1/responses translation in _build_chat_request dropped
chat_template_kwargs (e.g. {"enable_thinking": true}) sent via the
Responses extra-body, so reasoning control was silently ignored.
Lift enable_thinking onto the typed ChatCompletionRequest field,
mirroring openai_chat_completions, so both the non-streaming and
streaming Responses pass-through paths honor it.

Fixes #6198

Signed-off-by: Tai An <antai12232931@outlook.com>

* Fix/adjust Responses reasoning for PR #6202

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix/adjust reasoning none for PR #6202

* Fix/adjust structured reasoning for PR #6202

* Fix/adjust responses reasoning review findings for PR #6202

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix/adjust responses reasoning follow-ups for PR #6202

* Fix/adjust think parsing gate for PR #6202

---------

Signed-off-by: Tai An <antai12232931@outlook.com>
Co-authored-by: Wasim Yousef Said <wasimysdev@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-06-12 13:20:07 +02:00
Michael Han
aba21db466
Studio: bottom update banners, smooth llama.cpp progress, re-prompt after copy (#6233)
Web "New Unsloth version" banner now matches the llama.cpp banner design
(borderless rounded card, drop shadow, heading title) and moves to the
bottom-right, stacked with the llama.cpp banner in a shared container. Keeps
the Release notes link, and the copy command is platform aware (curl for
macOS/Linux/WSL, irm for Windows).

The web banner is checked once per launch only, with no polling; a re-check
just happens the next time the user opens the app. Copying the install
command no longer dismisses it for good: it hides and returns on the next
launch while the install is still behind. The X button still dismisses the
version permanently.

llama.cpp progress bar: poll the job every 500ms and smooth the displayed
value so it animates with the real download instead of snapping to the ~90%
post-download hold and finishing.
2026-06-12 04:18:57 -07:00
Michael Han
c836228de8
Studio: pinnable plus menu items and saved prompt pins (#6237)
Adjustable items (Chat with Files, MCP, Saved prompts, Compare chat,
Export chat, Canvas, Projects) can be pinned to the top level of the
composer plus menu from Settings -> Chat. Unpinned items move into the
More submenu, which hides itself when empty. Saved prompts can be
pinned individually so they surface in the Saved prompts submenu.
2026-06-12 04:17:45 -07:00
Michael Han
b3ffa266da
Studio: login card polish and sidebar label alignment (#6242)
Rounder corners and more padding on the login card, a submit button
that sizes to its label, and the sidebar chat and run rows nudged 2px
left with the section headers brought flush to the row text so the
labels line up like Gemini's sidebar.
2026-06-12 04:15:49 -07:00
Michael Han
40cb0a71b4
Studio: model selector and settings polish (#6240)
Removes the empty band under Eject loaded model, centers the trigger
chevrons, moves the model reload Apply and Reset buttons above Chat
Template so they sit under the settings they apply, matches the
settings nav divider to the sidebar line, and bumps the settings tab
headings slightly.
2026-06-12 04:13:22 -07:00
Lee Jackson
7307fde839
Studio: Add custom provider option to Connections (#6112)
* feat: add custom connection

* Fix custom provider handling for PR #6112

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix custom provider connection test for PR #6112

---------

Co-authored-by: wasimysaid <wasimysdev@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-06-12 13:09:35 +02:00
oobabooga
72e67ae5a6
Studio: Add Tensor-Parallel llama.cpp support (#6040)
* Studio: Add Tensor-Parallel llama.cpp support

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: harden Tensor-Parallel fallback and GPU selection

* Studio: reconcile split-mode extras and harden tensor-split planning

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: reconcile split-mode extras in backend duplicate-load guard

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: preserve inherited non-tensor split modes on reload

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: honor cancellation in tensor fallback, preserve tensor mode on rollback, and don't raise an explicit small context

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: reconcile split-mode in reload check and strip it on tensor downgrade

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Strip --tensor-split alongside --split-mode so inherited ratios don't override the tensor planner

An inherited or stale --tensor-split in llama_extra_args was appended after
Studio's computed --tensor-split and won last in llama.cpp, re-introducing the
asymmetric-GPU OOM tensor mode is meant to prevent. Group -ts/--tensor-split
into the split-mode shadow set so it is stripped on inherit and on the layer
fallback; parse_split_mode_override still keys on the mode value only.

* Drop quantized KV for the tensor attempt and report native max context

Tensor mode aborts on a quantized KV cache, so a user with q8_0/q4_1 etc. who
enabled Tensor Parallelism silently fell back to layer split. Clear the cache
type (and strip inherited/explicit --cache-type) for the tensor attempt only;
the layer fallback re-runs with tensor off and keeps the user's choice.

Also report max_available_ctx from the native context, not an explicit small
-c, so the context slider no longer warns too early in tensor mode.

* Reconcile inherited split-mode extras in the already-loaded check

When a same-model load omitted llama_extra_args, the tensor comparison resolved
the raw (None) request and treated an inherited --split-mode tensor server as a
mismatch, forcing a needless reload. Compare using the stored extras stripped
the same way the reload strips them.

* Pass tensor_parallel through compare-mode loads

The generalized compare path loaded each GGUF without tensor_parallel, so
compare ran layer split even with the toggle on and left the settings sheet
stale. Send the toggle and hydrate the loaded state from the response, matching
the main chat and recipe load paths.

* Add --tensor-parallel flag to unsloth studio run

The headless one-liner could only reach tensor mode by passing --split-mode
tensor as a raw llama.cpp extra. Add a first-class --tensor-parallel/
--no-tensor-parallel option that sets the tensor_parallel field on the
/api/inference/load payload, forwarded through the studio-venv re-exec like the
other polarity flags. Matches the web UI toggle and the API field.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
2026-06-12 04:00:52 -07:00
Anmol Mishra
554c289538
fix: respect absolute export paths to prevent cross-drive copy failures (WinError 112) (#6088)
* fix: allow absolute save_directory in export paths to prevent cross-drive copy failures

The GGUF export pipeline (and all other export flows) forced every
save_directory through resolve_export_dir(), which always resolved
the path under exports_root() — typically ~/.unsloth/studio/exports/
on the system drive (C: on Windows).

When a user selected an output directory on a different drive (E:):
1. The absolute path was rejected at the Pydantic validator level.
2. Even if it got through, resolve_export_dir would re-resolve it
   under C:\Users\.unsloth\studio\exports\.
3. After GGUF conversion completed on E:, the relocation step would
   try to move/copy the finished files to C:, causing:
   - WinError 17 (cross-drive move failure when shutil.move falls
     through to a cross-filesystem copy)
   - WinError 112 (disk full on C:)

Fix both layers:
- _validate_save_directory: accept absolute paths (they represent an
  explicit user choice of output location).
- resolve_export_dir, resolve_output_dir, resolve_tensorboard_dir:
  return absolute paths as-is instead of forcing them under the
  default root. Keep the existing safety checks (null bytes, '..'
  segments) and fall through to resolve_under_root for relative paths.

Fixes: https://github.com/unslothai/unsloth/issues/6082

* refactor: centralize user path validation into _resolve_user_path helper

Addresses code review feedback: the null-byte, '..', and absolute-path
checks were duplicated across resolve_output_dir, resolve_export_dir,
and resolve_tensorboard_dir. Extract a single _resolve_user_path helper
that all three delegate to.

No behavioral change — pure consolidation.

* fix: address code review — contain destructive cleanup and scope absolute paths

Address all review feedback from gemini-code-assist:

1. P1: destructive subdirectory cleanup (export_gguf)
   The flattening loop in export_gguf previously rmtree'd every
   subdirectory under abs_save_dir. When targeting an existing user
   directory on a different drive (#6082), this could nuke unrelated
   subdirectories. Now snapshot existing subdirectories before the
   export and only clean up dirs created during this run.

2. P2: keep scan/read endpoints contained
   Only resolve_export_dir accepts absolute paths (export is a write
   path where user picks location). Reverted resolve_output_dir and
   resolve_tensorboard_dir to use resolve_under_root directly — these
   are used by scan/read/training endpoints that must stay contained
   under their respective roots.

3. Centralization feedback
   Removed the _resolve_user_path helper since it's no longer needed
   with the narrowed scope. resolve_export_dir has the absolute path
   logic inline with a clear docstring.

* fix: skip pre-existing subdirs in GGUF flatten loop and clean stale export intermediates

Two issues caught in code review (chatgpt-codex-connector):

1. The flattening loop moved ALL .gguf files from ALL subdirectories
   into abs_save_dir, including pre-existing unrelated user subdirs.
   Now skip pre-existing subdirs entirely unless they are known
   export-owned intermediates (model/, model_gguf/).

2. After a failed export, known export-owned subdirectories (model/,
   model_gguf/) were snapshotted as pre-existing on retry and never
   cleaned up. These are now always cleaned up regardless, since they
   are known intermediates created by the export pipeline.

* fix: separate write vs read export paths, guard same-dir rmtree

Three issues caught in code review (chatgpt-codex-connector):

1. P1: scan endpoint containment
   resolve_export_dir was changed to accept absolute paths, but it's
   also used by scan/read endpoints (routes/models.py) that must stay
   contained under exports_root(). Split into:
   - resolve_export_dir: contained, used by scans
   - resolve_export_write_dir: accepts absolute paths, used by export
     backend only

2. P1: same-directory rmtree
   When a non-PEFT checkpoint's gguf_dir resolves to the same path as
   abs_save_dir (user selected the checkpoint's gguf output as their
   export directory), shutil.rmtree(gguf_dir) would delete the user's
   chosen output directory. Now skip relocation when both paths resolve
   to the same location.

3. P1: pre-existing subdir flatten loop
   Reverted _EXPORT_OWNED_SUBDIRS logic — 'model/' and 'model_gguf/'
   are common directory names in shared model folders and don't prove
   export ownership. Now only clean up subdirs that didn't exist before
   the export started.

* fix: remove dead _EXPORT_OWNED_SUBDIRS and fix _export_details for absolute paths

Two fixes from review comments:

1. Remove unused _EXPORT_OWNED_SUBDIRS declaration (leftover from
   previous iteration that was intentionally removed).

2. _export_details now returns the full absolute path when the export
   target is outside exports_root(), instead of truncating to basename.
   Users who export to E:\ can now see the full destination path in
   the success dialog.

* fix: use unique tmp dir for GGUF intermediates to avoid overwriting user dirs

When exporting to an absolute destination that already contains a
model/ subdirectory (e.g. a shared models folder), the hard-coded
model_save_path would overwrite files in that unrelated directory.

Use _tmp_model_<uuid> as the intermediate path instead, so user
directories are never touched. The tmp dir is created as a new subdir
of abs_save_dir and cleaned up by the flatten loop after GGUF files
are relocated.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix GGUF local export paths for PR #6088

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Address GGUF export follow-ups for PR #6088

* Clean GGUF temp dirs on export failure for PR #6088

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix/adjust export path tests for PR #6088

* Fix/adjust export path review findings for PR #6088

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix/adjust home export path handling for PR #6088

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
2026-06-12 12:52:57 +02:00
Daniel Han
6d206b488c
chore(studio/frontend): normalize line endings to LF (#6012)
* chore(studio/frontend): normalize line endings to LF

45 source files under studio/frontend/ were committed with CRLF or mixed
line endings while the rest of the repo and the JS/TS tooling assume LF.

Add a scoped `studio/frontend/** text=auto eol=lf` rule to .gitattributes
and run `git add --renormalize studio/frontend` so these files are stored
with LF in the index. The rule is scoped to the frontend tree (not a
repo-wide *.ts/*.tsx/... policy) so it cannot force LF on files elsewhere;
text=auto leaves binary assets (logos, fonts) untouched.

This commit is whitespace-only (CRLF -> LF) — no source content changed
(verified with `git diff --ignore-cr-at-eol`). It is intentionally
isolated so it can be listed in .git-blame-ignore-revs and skipped by
reviewers and `git blame`.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* chore: ignore the frontend LF-normalization commit in git blame

Add .git-blame-ignore-revs listing the whitespace-only line-ending
normalization commit so it doesn't pollute `git blame` output. GitHub
applies this file automatically; locally run
`git config blame.ignoreRevsFile .git-blame-ignore-revs`.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-12 03:51:59 -07:00