From b2b1dcd6ad72addde9a4324d15a2966fd0df6437 Mon Sep 17 00:00:00 2001
From: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
Date: Thu, 11 Jun 2026 09:27:34 -0700
Subject: [PATCH 01/50] Studio: llama.cpp update banner redesign, About tab
license info, UI polish (#6196)
* Studio: llama.cpp update banner redesign, About tab license info, inline system prompt editing, naming cleanup
- Redesign the llama.cpp update banner to match the chat composer surface
(borderless rounded card, composer shadow, Hellix Medium title), rename
actions to Update and add a 15 minute Remind me later snooze
- Keep the banner up until the user explicitly acts on it; drop the
outside click dismissal
- Add a Settings > General > Notifications toggle to disable the banner
for training-only setups (on by default)
- Rename the Help settings tab to About and add a License section
(Unsloth Studio AGPL-3.0, Unsloth Core Apache-2.0) linking to the
license files in this repo
- Make the run settings system prompt box an inline editable textarea;
the popup editor opens when the prompt overflows the box
- Pointer cursor on the preset dropdown chevron
- Dark mode toasts use the chat composer surface color
- Replace standalone Studio with Unsloth in user facing strings; keep
Unsloth Studio, LM Studio, Fine-tuning Studio, Recipe Studio and CLI
commands unchanged
* Studio: open the system prompt popup on box click, balance banner padding
- The system prompt box opens the Edit System Prompt dialog on click,
matching the pencil action
- Slightly more bottom padding on the llama.cpp update banner so the
spacing reads even next to the action pills
* Studio: replace unsloth studio update with the installer commands in update guidance
- The unsloth studio update command no longer works, so the About tab
update section now shows the one-line installer (curl or irm) for
PyPI and unknown installs, and git pull plus the local installer for
checkouts
- Add a short note that unsloth studio update is no longer supported
- Link the Installation, Updating and Windows install docs pages
- The package update banner now copies the platform installer command
instead of unsloth studio update
* Studio: rounder account menu, inline system prompt box with popup from the label
- Account menu corners go from 14px to 18px via a specific override,
since list menus pin border-radius globally
- llama.cpp banner bottom padding 22px
- System prompt is an inline editable textarea again; clicking the
System Prompt label opens the popup editor, and an overflowing
prompt opens it on box click
* Studio: show the standard install commands in the About update section
- Both one-line install commands (MacOS/Linux/WSL and Windows
PowerShell) are always shown, labeled like the docs, since running
them again updates an existing install
- Drop the unsloth studio update deprecation note
- Add the Mac install guide to the docs links
* Studio: clearer platform toggle and layout in the About update section
- Section heading is Update
- Platform picker is a pair of pill buttons, MacOS / Linux and Windows,
and only the selected platform's install command is shown
- Intro reads: To install or update Unsloth
- Local update heading separates checkout guidance from the standard
install command
* Studio: report GitHub branch instead of dev for source checkouts
A source checkout not on an exact release tag now shows
GitHub (e.g. GitHub main) as the Studio version in About.
Detached or unusual HEADs still fall back to dev.
* Studio: tighten the About update section copy and toggle styling
- Platform toggle buttons are borderless pills
- Shorter local update wording and restart note
- Docs links read Mac and Windows
* Studio: tighten line spacing in the sidebar account button
* Studio: fix vanishing compact MCP icon on hover, single line pill tooltips
- Compact caret pills (MCP, RAG) keep their icon on hover for inactive
pills too; the off switch hover rules hid the icon while compact mode
hid the X, leaving an empty slot
- Compact icon tooltips and single line compact tooltips render as full
pills; wrapped tooltips keep the 9px corners. TooltipContent measures
line count in a ref callback since Radix mounts portal content
without re-rendering the wrapper
- 1px gap between the name and Unsloth lines in the sidebar account
button
* Studio: Projects hover plus button, align recents with the label
- Hovering the Projects nav item reveals a plus button that opens the
New project dialog, with the same circular hover treatment as the
chat row actions
- Recent chat titles start at the same x as the Recents label
- The system prompt overflow lock only engages for a non-empty prompt
with a laid-out box, so a mis-measure cannot turn clicks into the
popup
* Clip system prompt overflow inside the rounded box
Wrap the inline system prompt textarea in a rounded overflow-hidden
surface so scrolled text and the scrollbar stay inside the box. The
focus ring moves to the wrapper via focus-within.
* Add updating progress bar to llama banner and shorten settings copy
While an update is applying, the banner action row becomes an
indeterminate progress bar that keeps animating under reduced motion,
matching the other loading indicators. Settings descriptions across
General, Profile, Appearance, Chat, Connections, API, and About are
trimmed without losing meaning.
* Address review: desktop update note, server platform detection, zh-CN keys
The About tab no longer shows terminal install commands in the desktop
app, where the bundled backend updates through the built-in updater;
it shows a short note and the docs links instead.
fetchDeviceType now sends the auth token to /api/health, which only
reports the server platform to authed callers, and caches only a
server-reported value. Copied install commands then match the host
platform rather than the browser when they differ (WSL, SSH).
zh-CN gains translations for the new notification and license keys,
the renamed About tab title, and the desktop update note.
* Real download progress for llama.cpp updates, prompt and sidebar polish
The update worker now streams the installer output and parses its
download percent lines into job progress, exposed via the update-status
API. The installer emits finer non-tty milestones when
UNSLOTH_PROGRESS_PERCENT_STEP is set; the worker requests 5 percent
steps. The banner renders a determinate bar from the reported fraction
and falls back to the sweep until the first percent arrives.
Also removes the focus ring on the inline system prompt box and
slightly shrinks the Projects hover plus icon.
---
studio/backend/routes/llama.py | 1 +
studio/backend/tests/test_llama_cpp_update.py | 130 +++++----
studio/backend/utils/llama_cpp_update.py | 53 +++-
studio/backend/utils/studio_version.py | 35 ++-
.../frontend/src/components/app-sidebar.tsx | 29 +-
.../src/components/llama-update-banner.tsx | 117 ++++----
.../src/components/tauri/startup-screen.tsx | 2 +-
studio/frontend/src/components/ui/tooltip.tsx | 18 ++
.../src/components/web/update-banner.tsx | 16 +-
studio/frontend/src/config/env.ts | 20 +-
studio/frontend/src/features/auth/api.ts | 2 +-
.../src/features/auth/tauri-auto-auth.ts | 2 +-
.../studio-web-speech-dictation-adapter.ts | 4 +-
.../features/chat/chat-providers-dialog.tsx | 2 +-
.../src/features/chat/chat-settings-sheet.tsx | 61 +++-
.../frontend/src/features/chat/tour/steps.tsx | 4 +-
.../chat/utils/chat-history-storage.ts | 2 +-
.../src/features/hub/download-manager/api.ts | 6 +-
.../download-manager/download-api-adapter.ts | 2 +-
.../executions/execution-data-tab.tsx | 2 +-
.../executions/execution-overview-tab.tsx | 2 +-
.../runtime/execution-progress-island.tsx | 2 +-
.../dialogs/seed/seed-dialog.tsx | 6 +-
.../components/update-studio-instructions.tsx | 265 +++++++++++-------
.../src/features/settings/tabs/about-tab.tsx | 33 ++-
.../features/settings/tabs/general-tab.tsx | 21 ++
.../src/features/studio/tour/steps/nav.tsx | 2 +-
.../src/hooks/use-llama-update-check.ts | 33 ++-
.../src/hooks/use-llama-update-pref.ts | 49 ++++
.../frontend/src/hooks/use-tauri-backend.ts | 22 +-
studio/frontend/src/i18n/locales/en.ts | 107 ++++---
studio/frontend/src/i18n/locales/zh-CN.ts | 65 +++--
studio/frontend/src/index.css | 41 ++-
studio/install_llama_prebuilt.py | 14 +-
34 files changed, 831 insertions(+), 339 deletions(-)
create mode 100644 studio/frontend/src/hooks/use-llama-update-pref.ts
diff --git a/studio/backend/routes/llama.py b/studio/backend/routes/llama.py
index b30c0f5aea..47a5bc1dd3 100644
--- a/studio/backend/routes/llama.py
+++ b/studio/backend/routes/llama.py
@@ -31,6 +31,7 @@ class LlamaUpdateJob(BaseModel):
from_tag: Optional[str] = None
to_tag: Optional[str] = None
error: Optional[str] = None
+ progress: Optional[float] = Field(None, description = "0..1 while running, 1 on success.")
started_at: Optional[str] = None
finished_at: Optional[str] = None
diff --git a/studio/backend/tests/test_llama_cpp_update.py b/studio/backend/tests/test_llama_cpp_update.py
index 05a107377c..d21d653b76 100644
--- a/studio/backend/tests/test_llama_cpp_update.py
+++ b/studio/backend/tests/test_llama_cpp_update.py
@@ -28,6 +28,55 @@ import utils.llama_cpp_update as upd # noqa: E402
MARKER = "UNSLOTH_PREBUILT_INFO.json"
+class _FakeInstallerPopen:
+ """Stands in for the streamed installer process in _run_update."""
+
+ def __init__(
+ self,
+ cmd,
+ *,
+ returncode = 0,
+ lines = None,
+ on_start = None,
+ captured_kwargs = None,
+ **kwargs,
+ ):
+ if captured_kwargs is not None:
+ captured_kwargs.update(kwargs)
+ if on_start is not None:
+ on_start(list(cmd))
+ self.returncode = returncode
+ self.stdout = iter(lines or [])
+
+ def wait(self):
+ return self.returncode
+
+ def kill(self):
+ pass
+
+
+def _patch_installer_popen(
+ monkeypatch,
+ *,
+ returncode = 0,
+ lines = None,
+ on_start = None,
+ captured_kwargs = None,
+):
+ monkeypatch.setattr(
+ upd.subprocess,
+ "Popen",
+ lambda cmd, **kw: _FakeInstallerPopen(
+ cmd,
+ returncode = returncode,
+ lines = lines,
+ on_start = on_start,
+ captured_kwargs = captured_kwargs,
+ **kw,
+ ),
+ )
+
+
def _write_install(
dir_: Path,
tag: str,
@@ -260,14 +309,15 @@ def test_start_update_source_build_installs_prebuilt(monkeypatch, tmp_path):
def _fake_run(cmd, **kwargs):
cmd = list(cmd)
- # Status polls probe `llama-server --version`; keep the installer argv.
- if "--version" in cmd:
- return _Proc()
- captured["cmd"] = cmd
- _write_install(install_dir, "b9585") # installer writes the marker
+ assert "--version" in cmd # only status polls still use run()
return _Proc()
+ def _on_start(cmd):
+ captured["cmd"] = cmd
+ _write_install(install_dir, "b9585") # installer writes the marker
+
monkeypatch.setattr(upd.subprocess, "run", _fake_run)
+ _patch_installer_popen(monkeypatch, on_start = _on_start)
res = upd.start_update()
assert res["started"] is True, res
@@ -298,21 +348,27 @@ def test_start_update_happy_path(monkeypatch, tmp_path):
stdout = "installed"
stderr = ""
- def _fake_run(cmd, **kwargs):
- cmd = list(cmd)
- # Status polls probe `llama-server --version`; keep the installer argv.
- if "--version" in cmd:
- return _Proc()
+ def _on_start(cmd):
captured["cmd"] = cmd
# Simulate the installer writing a new marker with the latest tag.
_write_install(install_dir, "b9518")
- return _Proc()
- monkeypatch.setattr(upd.subprocess, "run", _fake_run)
+ popen_kwargs: dict = {}
+ _patch_installer_popen(
+ monkeypatch,
+ lines = [
+ "[llama-prebuilt] resolving release\n",
+ "Downloading llama.zip: 35.0% (12.0 MiB/35.0 MiB) at 9.0 MiB/s\n",
+ "Downloading llama.zip: 80.0% (28.0 MiB/35.0 MiB) at 9.0 MiB/s\n",
+ ],
+ on_start = _on_start,
+ captured_kwargs = popen_kwargs,
+ )
res = upd.start_update()
assert res["started"] is True
assert res["job"]["from_tag"] == "b9493"
+ assert res["job"]["progress"] == 0.0
# Wait for the background worker.
deadline = time.time() + 10
@@ -328,6 +384,10 @@ def test_start_update_happy_path(monkeypatch, tmp_path):
assert str(install_dir) in captured["cmd"]
assert "--llama-tag" in captured["cmd"] and "latest" in captured["cmd"]
assert "unslothai/llama.cpp" in captured["cmd"]
+ # Progress lines were parsed and success pins progress at 1.0.
+ assert job["progress"] == 1.0
+ # The worker asks the installer for fine-grained progress milestones.
+ assert popen_kwargs["env"]["UNSLOTH_PROGRESS_PERCENT_STEP"] == "5"
def test_start_update_installer_failure_reports_error(monkeypatch, tmp_path):
@@ -336,12 +396,7 @@ def test_start_update_installer_failure_reports_error(monkeypatch, tmp_path):
monkeypatch.setattr(upd, "_find_binary", lambda: binary)
monkeypatch.setattr(upd, "_installer_script", lambda: tmp_path / "install_llama_prebuilt.py")
- class _Proc:
- returncode = 2
- stdout = ""
- stderr = "boom: network error"
-
- monkeypatch.setattr(upd.subprocess, "run", lambda cmd, **kw: _Proc())
+ _patch_installer_popen(monkeypatch, returncode = 2, lines = ["boom: network error\n"])
res = upd.start_update()
assert res["started"] is True
@@ -410,14 +465,15 @@ def _capture_install_cmd(
def _fake_run(cmd, **kwargs):
cmd = list(cmd)
- # Status polls probe `llama-server --version`; keep the installer argv.
- if "--version" in cmd:
- return _Proc()
- captured["cmd"] = cmd
- _write_install(install_dir, latest, repo = repo, asset = asset)
+ assert "--version" in cmd # only status polls still use run()
return _Proc()
+ def _on_start(cmd):
+ captured["cmd"] = cmd
+ _write_install(install_dir, latest, repo = repo, asset = asset)
+
monkeypatch.setattr(upd.subprocess, "run", _fake_run)
+ _patch_installer_popen(monkeypatch, on_start = _on_start)
res = upd.start_update()
assert res["started"] is True, res
@@ -535,18 +591,12 @@ def test_update_sets_maintenance_flag_and_unloads(monkeypatch, tmp_path):
seen = {}
- class _Proc:
- returncode = 0
- stdout = "ok"
- stderr = ""
-
- def _fake_run(cmd, **kwargs):
+ def _on_start(cmd):
# The maintenance flag must be set while the installer runs.
seen["flag_during_install"] = backend._llama_update_in_progress
_write_install(install_dir, "b9518")
- return _Proc()
- monkeypatch.setattr(upd.subprocess, "run", _fake_run)
+ _patch_installer_popen(monkeypatch, on_start = _on_start)
res = upd.start_update()
assert res["started"] is True
@@ -571,12 +621,7 @@ def test_update_clears_maintenance_flag_on_installer_failure(monkeypatch, tmp_pa
backend = _FakeBackend()
_inject_backend(monkeypatch, backend)
- class _Proc:
- returncode = 1
- stdout = ""
- stderr = "boom"
-
- monkeypatch.setattr(upd.subprocess, "run", lambda cmd, **kw: _Proc())
+ _patch_installer_popen(monkeypatch, returncode = 1, lines = ["boom\n"])
res = upd.start_update()
assert res["started"] is True
@@ -606,16 +651,7 @@ def test_update_fails_open_when_backend_unavailable(monkeypatch, tmp_path):
monkeypatch.setitem(sys.modules, "routes", routes_pkg)
monkeypatch.setitem(sys.modules, "routes.inference", inference_mod)
- class _Proc:
- returncode = 0
- stdout = "ok"
- stderr = ""
-
- def _fake_run(cmd, **kwargs):
- _write_install(install_dir, "b9518")
- return _Proc()
-
- monkeypatch.setattr(upd.subprocess, "run", _fake_run)
+ _patch_installer_popen(monkeypatch, on_start = lambda cmd: _write_install(install_dir, "b9518"))
res = upd.start_update()
assert res["started"] is True
diff --git a/studio/backend/utils/llama_cpp_update.py b/studio/backend/utils/llama_cpp_update.py
index b138b29af3..55bf4b3d84 100644
--- a/studio/backend/utils/llama_cpp_update.py
+++ b/studio/backend/utils/llama_cpp_update.py
@@ -59,10 +59,17 @@ _job: dict = {
"from_tag": None,
"to_tag": None,
"error": None,
+ "progress": None,
"started_at": None,
"finished_at": None,
}
+# Matches the installer's download progress lines, e.g.
+# "Downloading x.zip: 35.0% (12.3 MiB/35.1 MiB) at 8.2 MiB/s".
+_PROGRESS_LINE_RE = re.compile(r"(\d+(?:\.\d+)?)%\s*\(")
+# The download dominates the update; extract/validate fill the last slice.
+_DOWNLOAD_PROGRESS_CEILING = 0.95
+
def _utcnow() -> str:
return time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
@@ -357,15 +364,46 @@ def _run_update(install_dir: Path, repo: str, asset: Optional[str], script: Path
]
cmd.extend(_rocm_install_args(asset))
logger.info("llama update: installing", cmd = " ".join(cmd))
- proc = subprocess.run(
+ # Stream the installer output so download percent lines feed
+ # job["progress"]; finer milestones via UNSLOTH_PROGRESS_PERCENT_STEP.
+ env = dict(os.environ, UNSLOTH_PROGRESS_PERCENT_STEP = "5")
+ proc = subprocess.Popen(
cmd,
- capture_output = True,
+ stdout = subprocess.PIPE,
+ stderr = subprocess.STDOUT,
text = True,
- timeout = _INSTALL_TIMEOUT_SECONDS,
+ env = env,
)
- if proc.returncode != 0:
- tail = (proc.stderr or proc.stdout or "").strip()[-1500:]
- raise RuntimeError(f"installer exited {proc.returncode}: {tail or 'no output'}")
+ timed_out = threading.Event()
+
+ def _kill_on_timeout() -> None:
+ timed_out.set()
+ proc.kill()
+
+ watchdog = threading.Timer(_INSTALL_TIMEOUT_SECONDS, _kill_on_timeout)
+ watchdog.daemon = True
+ watchdog.start()
+ tail_lines: list[str] = []
+ try:
+ assert proc.stdout is not None
+ for line in proc.stdout:
+ tail_lines.append(line)
+ if len(tail_lines) > 80:
+ del tail_lines[0]
+ m = _PROGRESS_LINE_RE.search(line)
+ if m is None:
+ continue
+ fraction = min(float(m.group(1)) / 100.0, 1.0) * _DOWNLOAD_PROGRESS_CEILING
+ with _job_lock:
+ _job["progress"] = max(_job.get("progress") or 0.0, fraction)
+ returncode = proc.wait()
+ finally:
+ watchdog.cancel()
+ if timed_out.is_set():
+ raise RuntimeError(f"installer timed out after {_INSTALL_TIMEOUT_SECONDS}s")
+ if returncode != 0:
+ tail = "".join(tail_lines).strip()[-1500:]
+ raise RuntimeError(f"installer exited {returncode}: {tail or 'no output'}")
# New UNSLOTH_PREBUILT_INFO.json is on disk; drop caches so the next
# status read reflects the freshly installed tag.
@@ -382,6 +420,7 @@ def _run_update(install_dir: Path, repo: str, asset: Optional[str], script: Path
),
to_tag = new_tag,
error = None,
+ progress = 1.0,
finished_at = _utcnow(),
)
logger.info("llama update: success", to_tag = new_tag)
@@ -467,6 +506,7 @@ def start_update() -> dict:
from_tag = from_tag,
to_tag = None,
error = None,
+ progress = 0.0,
started_at = _utcnow(),
finished_at = None,
)
@@ -491,6 +531,7 @@ def _reset_job_for_tests() -> None:
from_tag = None,
to_tag = None,
error = None,
+ progress = None,
started_at = None,
finished_at = None,
)
diff --git a/studio/backend/utils/studio_version.py b/studio/backend/utils/studio_version.py
index 98c48fe45c..9c18070fbb 100644
--- a/studio/backend/utils/studio_version.py
+++ b/studio/backend/utils/studio_version.py
@@ -15,6 +15,7 @@ _DEV_VERSION = "dev"
_GIT_TIMEOUT_SECONDS = 1.0
_STUDIO_TAG_RE = re.compile(r"^v\d+\.\d+\.\d+(?:-[0-9A-Za-z.][0-9A-Za-z.-]*)?$")
_GIT_DESCRIBE_SUFFIX_RE = re.compile(r"-\d+-g[0-9A-Fa-f]+(?:-dirty)?$")
+_GIT_BRANCH_RE = re.compile(r"^[0-9A-Za-z._/-]+$")
_MAX_VERSION_LENGTH = 64
@@ -71,6 +72,35 @@ def _exact_git_studio_tag(repo_root: Path) -> str | None:
return tag if is_valid_studio_release_version(tag) else None
+def _git_branch(repo_root: Path) -> str | None:
+ try:
+ result = subprocess.run(
+ ["git", "rev-parse", "--abbrev-ref", "HEAD"],
+ cwd = repo_root,
+ check = False,
+ stdout = subprocess.PIPE,
+ stderr = subprocess.DEVNULL,
+ text = True,
+ timeout = _GIT_TIMEOUT_SECONDS,
+ )
+ except (OSError, subprocess.TimeoutExpired):
+ return None
+
+ if result.returncode != 0:
+ return None
+
+ branch = result.stdout.strip()
+ # "HEAD" means detached, e.g. a tag or commit checkout.
+ if (
+ not branch
+ or branch == "HEAD"
+ or len(branch) > _MAX_VERSION_LENGTH
+ or _GIT_BRANCH_RE.fullmatch(branch) is None
+ ):
+ return None
+ return branch
+
+
def get_studio_version(repo_root: Path | None = None) -> str:
"""Return the installed Studio release tag for display, or ``dev``.
@@ -81,7 +111,10 @@ def get_studio_version(repo_root: Path | None = None) -> str:
if _is_source_checkout(resolved_repo_root):
git_tag = _exact_git_studio_tag(resolved_repo_root)
- return git_tag if git_tag is not None else _DEV_VERSION
+ if git_tag is not None:
+ return git_tag
+ branch = _git_branch(resolved_repo_root)
+ return f"GitHub {branch}" if branch is not None else _DEV_VERSION
stamped_version = _studio_release_build.STUDIO_RELEASE_VERSION
if is_valid_studio_release_version(stamped_version):
diff --git a/studio/frontend/src/components/app-sidebar.tsx b/studio/frontend/src/components/app-sidebar.tsx
index 7baa69571c..f1a200b09b 100644
--- a/studio/frontend/src/components/app-sidebar.tsx
+++ b/studio/frontend/src/components/app-sidebar.tsx
@@ -60,6 +60,7 @@ import {
Logout05Icon,
MoreVerticalIcon,
Search01Icon,
+ PlusSignIcon,
PowerIcon,
PencilEdit02Icon,
LayoutAlignLeftIcon,
@@ -558,7 +559,8 @@ export function AppSidebar() {
: "sidebar-row-action group-hover/recent-item:opacity-100 group-hover/recent-item:pointer-events-auto focus-visible:opacity-100 focus-visible:pointer-events-auto";
const buttonClass = cn(
"sidebar-nav-btn h-[33px] cursor-pointer rounded-full pr-4 text-[14.5px] leading-[19px] tracking-nav font-medium",
- variant === "project" ? "pl-[39px]" : "pl-3",
+ // pl-3.5 starts the title at the same x as the Recents label text.
+ variant === "project" ? "pl-[39px]" : "pl-3.5",
variant === "project"
? "group-hover/project-chat-item:pr-8 group-has-[.sidebar-row-action[data-state=open]]/project-chat-item:pr-8"
: "group-hover/recent-item:pr-8 group-has-[.sidebar-row-action[data-state=open]]/recent-item:pr-8",
@@ -837,7 +839,28 @@ export function AppSidebar() {
navigate({ to: "/projects" });
closeMobileIfOpen();
}}
- />
+ className="group/projects-item relative"
+ >
+ {
+ e.stopPropagation();
+ setProjectCreateMoveTarget(null);
+ setProjectNameDraft("");
+ setCreatingProject(true);
+ }}
+ className="sidebar-row-action group-hover/projects-item:opacity-100 group-hover/projects-item:pointer-events-auto focus-visible:opacity-100 focus-visible:pointer-events-auto group-data-[collapsible=icon]:hidden"
+ >
+
+
+
+
+
-
+
{displayTitle}
Unsloth
diff --git a/studio/frontend/src/components/llama-update-banner.tsx b/studio/frontend/src/components/llama-update-banner.tsx
index c8a4dd1e40..83320f8e15 100644
--- a/studio/frontend/src/components/llama-update-banner.tsx
+++ b/studio/frontend/src/components/llama-update-banner.tsx
@@ -3,9 +3,10 @@
import { Button } from "@/components/ui/button";
import { useLlamaUpdateCheck } from "@/hooks/use-llama-update-check";
+import { useShowLlamaUpdateBanner } from "@/hooks/use-llama-update-pref";
import { toast } from "@/lib/toast";
import { AnimatePresence, motion } from "motion/react";
-import { type ReactElement, useEffect, useRef } from "react";
+import type { ReactElement } from "react";
const EASE_OUT_QUART: [number, number, number, number] = [0.165, 0.84, 0.44, 1];
@@ -15,15 +16,19 @@ interface LlamaUpdateBannerProps {
/**
* Non-invasive "Update llama.cpp" affordance. Appears bottom-right ~1s after a
- * newer prebuilt is detected and stays up until dismissed (click outside / X)
- * or updated. Clicking Update swaps the prebuilt in place via POST /api/llama/update.
+ * newer prebuilt is detected and stays up until the user explicitly acts on it
+ * (X, Update, or Remind me later). Clicking Update swaps the prebuilt in place
+ * via POST /api/llama/update. Can be turned off entirely in Settings ->
+ * General -> Notifications (on by default).
*/
export function LlamaUpdateBanner({
enabled = true,
}: LlamaUpdateBannerProps): ReactElement | null {
- const { status, visible, applying, apply, dismiss } = useLlamaUpdateCheck({
- enabled,
- });
+ const showBannerPref = useShowLlamaUpdateBanner();
+ const { status, visible, applying, apply, dismiss, snooze } =
+ useLlamaUpdateCheck({
+ enabled: enabled && showBannerPref,
+ });
async function handleUpdate() {
const result = await apply();
@@ -40,30 +45,12 @@ export function LlamaUpdateBanner({
const show =
visible && status != null && (status.update_available || applying);
- const bannerRef = useRef
(null);
-
- // Dismiss when the user clicks anything outside the banner. Kept off while an
- // update is applying so the progress stays visible.
- useEffect(() => {
- if (!show || applying) return;
- function onPointerDown(event: PointerEvent) {
- if (
- bannerRef.current &&
- !bannerRef.current.contains(event.target as Node)
- ) {
- dismiss();
- }
- }
- document.addEventListener("pointerdown", onPointerDown, true);
- return () =>
- document.removeEventListener("pointerdown", onPointerDown, true);
- }, [show, applying, dismiss]);
+ const updateProgress = status?.job.progress ?? null;
return (
{show ? (
-
+
{applying ? null : (
)}
-
-
- 🦥
-
-
- {applying ? "Updating llama.cpp..." : "New llama.cpp prebuilt"}
+
+
+ {applying ? "Updating llama.cpp..." : "New llama.cpp version"}
+
+
+ {status?.installed_tag ?? "unknown"} →{" "}
+
+ {status?.latest_tag ?? ""}
+
-
- {status?.installed_tag ?? "unknown"} →{" "}
-
- {status?.latest_tag ?? ""}
-
-
-
-
- {applying ? "Updating..." : "Update llama.cpp"}
-
-
+ {updateProgress != null && updateProgress > 0 ? (
+
+ ) : (
+ // No percent yet (resolving the release): sweep until the
+ // first download progress arrives.
+
+ )}
+
+ ) : (
+
+
+ Update
+
+
+ Remind me later
+
+
+ )}
) : null}
diff --git a/studio/frontend/src/components/tauri/startup-screen.tsx b/studio/frontend/src/components/tauri/startup-screen.tsx
index 1eb3f81d10..8e373c82a0 100644
--- a/studio/frontend/src/components/tauri/startup-screen.tsx
+++ b/studio/frontend/src/components/tauri/startup-screen.tsx
@@ -228,7 +228,7 @@ function RepairingContent({
-
Updating existing Studio install...
+
Updating existing Unsloth install...
{latest && (
{latest}
)}
diff --git a/studio/frontend/src/components/ui/tooltip.tsx b/studio/frontend/src/components/ui/tooltip.tsx
index 954b1871f3..cbc1a09a8f 100644
--- a/studio/frontend/src/components/ui/tooltip.tsx
+++ b/studio/frontend/src/components/ui/tooltip.tsx
@@ -98,9 +98,27 @@ function TooltipContent({
}: React.ComponentProps
& {
variant?: TooltipVariant;
}) {
+ // Single-line compact tooltips render as a full pill; wrapped ones keep
+ // the squarer corners so tall pills do not look like capsules. A ref
+ // callback measures on mount: Radix mounts the portal content without
+ // re-rendering this wrapper, so an effect here would never see the node.
+ const measureRef = useCallback(
+ (el: HTMLDivElement | null) => {
+ if (!el || variant !== "default") return;
+ const cs = getComputedStyle(el);
+ const lineHeight = Number.parseFloat(cs.lineHeight) || 16;
+ const innerHeight =
+ el.clientHeight -
+ Number.parseFloat(cs.paddingTop) -
+ Number.parseFloat(cs.paddingBottom);
+ el.classList.toggle("rounded-full!", innerHeight < lineHeight * 1.5);
+ },
+ [variant],
+ );
return (
s.deviceType);
+ const installCmd =
+ deviceType === "windows"
+ ? STUDIO_INSTALL_WINDOWS_CMD
+ : STUDIO_INSTALL_UNIX_CMD;
const [copiedVersion, setCopiedVersion] = useState(null);
const dismissTimerRef = useRef | null>(null);
@@ -36,7 +44,7 @@ export function WebUpdateBanner({
}
async function handleCopyCommand() {
- if (!(await copyToClipboard(STUDIO_UPDATE_CMD))) {
+ if (!(await copyToClipboard(installCmd))) {
return;
}
setCopiedVersion(status?.latestVersion ?? null);
@@ -89,8 +97,8 @@ export function WebUpdateBanner({
Package update available: {status.latestVersion}
- Installed package: {status.currentVersion}. To update Studio,
- run this in your terminal, then restart Studio.
+ Installed package: {status.currentVersion}. To update Unsloth,
+ run this in your terminal, then restart Unsloth.
diff --git a/studio/frontend/src/config/env.ts b/studio/frontend/src/config/env.ts
index 92dad2ae27..61ab111d45 100644
--- a/studio/frontend/src/config/env.ts
+++ b/studio/frontend/src/config/env.ts
@@ -46,12 +46,28 @@ export async function fetchDeviceType(): Promise {
if (fetched) return usePlatformStore.getState().deviceType;
try {
- const res = await fetch(apiUrl("/api/health"));
+ // /api/health only reports the server's device_type to authed callers.
+ // Read the token from storage directly: importing features/auth here
+ // would be an import cycle (auth/session imports this store).
+ const token =
+ typeof window === "undefined"
+ ? null
+ : localStorage.getItem("unsloth_auth_token");
+ const res = await fetch(apiUrl("/api/health"), {
+ headers: token ? { Authorization: `Bearer ${token}` } : undefined,
+ });
if (res.ok) {
const data = (await res.json()) as { device_type?: string; chat_only?: boolean };
const deviceType = data.device_type ?? detectLocalPlatform();
const chatOnly = data.chat_only ?? false;
- usePlatformStore.setState({ deviceType, chatOnly, fetched: true });
+ // Cache only a server-reported platform. Unauthenticated responses fall
+ // back to the browser platform, which can differ from the host (WSL,
+ // SSH); keeping fetched=false retries once a token exists.
+ usePlatformStore.setState({
+ deviceType,
+ chatOnly,
+ fetched: data.device_type !== undefined,
+ });
return deviceType;
}
} catch {
diff --git a/studio/frontend/src/features/auth/api.ts b/studio/frontend/src/features/auth/api.ts
index 6356f89378..144e85d40a 100644
--- a/studio/frontend/src/features/auth/api.ts
+++ b/studio/frontend/src/features/auth/api.ts
@@ -175,7 +175,7 @@ export async function authFetch(
"You appear to be offline. Check your network connection and try again.",
);
}
- throw new Error("Studio isn't running -- please relaunch it.");
+ throw new Error("Unsloth isn't running -- please relaunch it.");
}
throw err;
}
diff --git a/studio/frontend/src/features/auth/tauri-auto-auth.ts b/studio/frontend/src/features/auth/tauri-auto-auth.ts
index 760af26395..e825d5de31 100644
--- a/studio/frontend/src/features/auth/tauri-auto-auth.ts
+++ b/studio/frontend/src/features/auth/tauri-auto-auth.ts
@@ -26,7 +26,7 @@ let pending: { promise: Promise; force: boolean } | null = null;
let lastTauriAuthFailure: string | null = null;
const TAURI_AUTH_FAILURE_FALLBACK =
- "Desktop authentication failed. Update or repair the managed Studio install, then restart Studio.";
+ "Desktop authentication failed. Update or repair the managed Unsloth install, then restart Unsloth.";
const BACKEND_NOT_READY_MESSAGE = "Backend is not ready";
function authFailureMessage(error: unknown): string {
diff --git a/studio/frontend/src/features/chat/adapters/studio-web-speech-dictation-adapter.ts b/studio/frontend/src/features/chat/adapters/studio-web-speech-dictation-adapter.ts
index a679e8ed83..146a12bd40 100644
--- a/studio/frontend/src/features/chat/adapters/studio-web-speech-dictation-adapter.ts
+++ b/studio/frontend/src/features/chat/adapters/studio-web-speech-dictation-adapter.ts
@@ -18,7 +18,7 @@ const describeMediaError = (error: unknown): string => {
return "Dictation could not access the microphone.";
}
if (error.name === "NotAllowedError") {
- return "Microphone access is blocked. Allow microphone access for this Studio page, then try again.";
+ return "Microphone access is blocked. Allow microphone access for this Unsloth page, then try again.";
}
if (error.name === "NotFoundError") {
return "No microphone was found for dictation.";
@@ -31,7 +31,7 @@ const describeMediaError = (error: unknown): string => {
const describeSpeechError = (error: string, message?: string): string => {
if (error === "not-allowed") {
- return "Speech recognition was blocked by the browser. Check microphone permissions for this Studio page.";
+ return "Speech recognition was blocked by the browser. Check microphone permissions for this Unsloth page.";
}
if (error === "service-not-allowed") {
return "Speech recognition is blocked by the browser speech service.";
diff --git a/studio/frontend/src/features/chat/chat-providers-dialog.tsx b/studio/frontend/src/features/chat/chat-providers-dialog.tsx
index e35a8e35e0..0594b55083 100644
--- a/studio/frontend/src/features/chat/chat-providers-dialog.tsx
+++ b/studio/frontend/src/features/chat/chat-providers-dialog.tsx
@@ -1511,7 +1511,7 @@ export function ChatProvidersSettings({
Connections
- Manage model connections for chat through the Studio proxy.
+ Manage model connections for chat.
diff --git a/studio/frontend/src/features/chat/chat-settings-sheet.tsx b/studio/frontend/src/features/chat/chat-settings-sheet.tsx
index 03c183fe76..5751a1a426 100644
--- a/studio/frontend/src/features/chat/chat-settings-sheet.tsx
+++ b/studio/frontend/src/features/chat/chat-settings-sheet.tsx
@@ -325,6 +325,7 @@ function CollapsibleSection({
label,
labelHref,
headerAction,
+ onLabelClick,
children,
defaultOpen = false,
first = false,
@@ -342,6 +343,8 @@ function CollapsibleSection({
* nested in a button.
*/
headerAction?: ReactNode;
+ /** When set, clicking the label runs this instead of toggling collapse. */
+ onLabelClick?: () => void;
children?: ReactNode;
defaultOpen?: boolean;
first?: boolean;
@@ -395,7 +398,7 @@ function CollapsibleSection({
{label}
@@ -558,6 +561,9 @@ export function ChatSettingsPanel({
const [presetNameInput, setPresetNameInput] = useState(activePreset);
const [systemPromptEditorOpen, setSystemPromptEditorOpen] = useState(false);
const [systemPromptDraft, setSystemPromptDraft] = useState("");
+ // When the prompt overflows the inline box, clicking opens the popup editor.
+ const systemPromptBoxRef = useRef(null);
+ const [systemPromptOverflows, setSystemPromptOverflows] = useState(false);
const [activePresetBaseline, setActivePresetBaseline] = useState(params);
const presets = useMemo(() => {
return getOrderedPresets(customPresets);
@@ -762,6 +768,16 @@ export function ChatSettingsPanel({
}
}, [open]);
+ useEffect(() => {
+ const el = systemPromptBoxRef.current;
+ setSystemPromptOverflows(
+ params.systemPrompt.length > 0 &&
+ el != null &&
+ el.clientHeight > 0 &&
+ el.scrollHeight > el.clientHeight + 1,
+ );
+ }, [params.systemPrompt, open]);
+
const settingsScrollRef = useRef(null);
const settingsContent = (
@@ -1119,7 +1135,7 @@ export function ChatSettingsPanel({
/>
@@ -1326,22 +1343,36 @@ export function ChatSettingsPanel({
}
>
-
-
- {params.systemPrompt ||
- "Example: You are a helpful assistant..."}
-
-
+
diff --git a/studio/frontend/src/features/chat/tour/steps.tsx b/studio/frontend/src/features/chat/tour/steps.tsx
index de27e93c88..cbbb7bae4f 100644
--- a/studio/frontend/src/features/chat/tour/steps.tsx
+++ b/studio/frontend/src/features/chat/tour/steps.tsx
@@ -28,7 +28,7 @@ export function buildChatTourSteps({
body: (
<>
This selects what’s loaded for inference. Hub = base models. Fine-tuned
- = trained Studio outputs, including LoRA adapters and full finetunes.
+ = trained Unsloth outputs, including LoRA adapters and full finetunes.
>
),
},
@@ -38,7 +38,7 @@ export function buildChatTourSteps({
title: "Two tabs",
body: (
<>
- Hub: search Hugging Face models. Fine-tuned: local Studio outputs you’ve
+ Hub: search Hugging Face models. Fine-tuned: local Unsloth outputs you’ve
trained or exported. If results look off, compare base vs fine-tuned
outputs to see what changed.
>
diff --git a/studio/frontend/src/features/chat/utils/chat-history-storage.ts b/studio/frontend/src/features/chat/utils/chat-history-storage.ts
index 0e9b30a5cc..2023ae30b1 100644
--- a/studio/frontend/src/features/chat/utils/chat-history-storage.ts
+++ b/studio/frontend/src/features/chat/utils/chat-history-storage.ts
@@ -134,7 +134,7 @@ export function isExpectedBackgroundChatStorageError(error: unknown): boolean {
(error.message === "Invalid or expired token" ||
error.message === "Not authenticated" ||
error.message === "Request failed (401)" ||
- error.message === "Studio isn't running -- please relaunch it.")
+ error.message === "Unsloth isn't running -- please relaunch it.")
);
}
diff --git a/studio/frontend/src/features/hub/download-manager/api.ts b/studio/frontend/src/features/hub/download-manager/api.ts
index ca987f1643..2c55b90edd 100644
--- a/studio/frontend/src/features/hub/download-manager/api.ts
+++ b/studio/frontend/src/features/hub/download-manager/api.ts
@@ -12,14 +12,14 @@ function parseErrorText(status: number, body: unknown): string {
const detail = (body as { detail?: unknown }).detail;
const formatted = formatFastApiDetail(detail);
if (status === 405) {
- return `${formatted || "Method Not Allowed"} - the Studio backend did not accept this API method. Restart Studio so the frontend and backend are on the same build.`;
+ return `${formatted || "Method Not Allowed"} - the Unsloth backend did not accept this API method. Restart Unsloth so the frontend and backend are on the same build.`;
}
if (formatted) return formatted;
const message = (body as { message?: unknown }).message;
if (typeof message === "string" && message) return message;
}
if (status === 405) {
- return "Method Not Allowed - the Studio backend did not accept this API method. Restart Studio so the frontend and backend are on the same build.";
+ return "Method Not Allowed - the Unsloth backend did not accept this API method. Restart Unsloth so the frontend and backend are on the same build.";
}
return `Request failed (${status})`;
}
@@ -137,7 +137,7 @@ const DOWNLOAD_TRANSPORT_CAPABILITIES_FALLBACK: DownloadTransportCapabilities =
http: { available: true, reason: null },
xet: {
available: null,
- reason: "Couldn't verify Xet support with the Studio backend.",
+ reason: "Couldn't verify Xet support with the Unsloth backend.",
},
};
let downloadTransportCapabilitiesCache: DownloadTransportCapabilities | null =
diff --git a/studio/frontend/src/features/hub/download-manager/download-api-adapter.ts b/studio/frontend/src/features/hub/download-manager/download-api-adapter.ts
index 6942dfbbb9..a6518ec656 100644
--- a/studio/frontend/src/features/hub/download-manager/download-api-adapter.ts
+++ b/studio/frontend/src/features/hub/download-manager/download-api-adapter.ts
@@ -184,7 +184,7 @@ export async function effectiveTransportMode(
return preferred;
}
const reason =
- capabilities.xet.reason ?? "Studio will use HTTP downloads instead.";
+ capabilities.xet.reason ?? "Unsloth will use HTTP downloads instead.";
if (lastXetUnavailableWarningReason !== reason) {
lastXetUnavailableWarningReason = reason;
toast.warning("Xet download transport unavailable", {
diff --git a/studio/frontend/src/features/recipe-studio/components/executions/execution-data-tab.tsx b/studio/frontend/src/features/recipe-studio/components/executions/execution-data-tab.tsx
index a3e5dbe962..1616f6f77d 100644
--- a/studio/frontend/src/features/recipe-studio/components/executions/execution-data-tab.tsx
+++ b/studio/frontend/src/features/recipe-studio/components/executions/execution-data-tab.tsx
@@ -45,7 +45,7 @@ function formatGitHubSourceMessage(execution: RecipeExecutionRecord): string {
return "Collecting repository threads before rows are available.";
}
if (source.status === "rate_limited") {
- return source.message ?? "Waiting for GitHub rate limit. Studio will resume automatically.";
+ return source.message ?? "Waiting for GitHub rate limit. Unsloth will resume automatically.";
}
return source.message ?? "Collecting repository threads before rows are available.";
}
diff --git a/studio/frontend/src/features/recipe-studio/components/executions/execution-overview-tab.tsx b/studio/frontend/src/features/recipe-studio/components/executions/execution-overview-tab.tsx
index 72624738fc..5ff9d644e7 100644
--- a/studio/frontend/src/features/recipe-studio/components/executions/execution-overview-tab.tsx
+++ b/studio/frontend/src/features/recipe-studio/components/executions/execution-overview-tab.tsx
@@ -41,7 +41,7 @@ function formatSourceMessage(execution: RecipeExecutionRecord): string {
typeof source.retry_after_sec === "number" && source.retry_after_sec > 0
? ` Waiting ~${formatMetricValue(source.retry_after_sec)}s.`
: "";
- return `Waiting for GitHub rate limit. Studio will resume automatically.${wait}`;
+ return `Waiting for GitHub rate limit. Unsloth will resume automatically.${wait}`;
}
return source.message ?? "Crawling GitHub source.";
}
diff --git a/studio/frontend/src/features/recipe-studio/components/runtime/execution-progress-island.tsx b/studio/frontend/src/features/recipe-studio/components/runtime/execution-progress-island.tsx
index 563280c55f..1a194ba857 100644
--- a/studio/frontend/src/features/recipe-studio/components/runtime/execution-progress-island.tsx
+++ b/studio/frontend/src/features/recipe-studio/components/runtime/execution-progress-island.tsx
@@ -68,7 +68,7 @@ function formatGitHubSourceSummary(
typeof source.retry_after_sec === "number" && source.retry_after_sec > 0
? ` ~${formatMetricValue(source.retry_after_sec)}s`
: "";
- return `Waiting for GitHub rate limit${wait}. Studio will resume automatically.`;
+ return `Waiting for GitHub rate limit${wait}. Unsloth will resume automatically.`;
}
if (source.status === "retrying") {
return source.message ?? "GitHub request failed; retrying automatically.";
diff --git a/studio/frontend/src/features/recipe-studio/dialogs/seed/seed-dialog.tsx b/studio/frontend/src/features/recipe-studio/dialogs/seed/seed-dialog.tsx
index 878f9eba7c..0ed7eeed75 100644
--- a/studio/frontend/src/features/recipe-studio/dialogs/seed/seed-dialog.tsx
+++ b/studio/frontend/src/features/recipe-studio/dialogs/seed/seed-dialog.tsx
@@ -334,8 +334,8 @@ export function GithubRepoSeedForm({
/>
{usingEnvToken
- ? "Studio detected a server env token, so saved/shared recipes can leave this blank."
- : "Blank is safest for saved/shared recipes because Studio will read the server environment at run time."}
+ ? "Unsloth detected a server env token, so saved/shared recipes can leave this blank."
+ : "Blank is safest for saved/shared recipes because Unsloth will read the server environment at run time."}
{hasToken && (
@@ -450,7 +450,7 @@ export function GithubRepoSeedForm({
- Backed by Studio's built-in github_repo seed reader. Large
+ Backed by Unsloth's built-in github_repo seed reader. Large
repos can take minutes, so start with small limits for previews.
diff --git a/studio/frontend/src/features/settings/components/update-studio-instructions.tsx b/studio/frontend/src/features/settings/components/update-studio-instructions.tsx
index 90f66483b7..ef6bbf3c18 100644
--- a/studio/frontend/src/features/settings/components/update-studio-instructions.tsx
+++ b/studio/frontend/src/features/settings/components/update-studio-instructions.tsx
@@ -4,21 +4,29 @@
import { copyToClipboard } from "@/lib/copy-to-clipboard";
import { useT } from "@/i18n";
import { cn } from "@/lib/utils";
-import { Copy01Icon, Tick02Icon } from "@hugeicons/core-free-icons";
+import {
+ ArrowUpRight01Icon,
+ Copy01Icon,
+ Tick02Icon,
+} from "@hugeicons/core-free-icons";
import { HugeiconsIcon } from "@hugeicons/react";
import { AnimatePresence, motion, useReducedMotion } from "motion/react";
import type { ReactElement } from "react";
import { useEffect, useRef, useState } from "react";
-const STUDIO_UPDATE_CMD = "unsloth studio update";
-const STUDIO_UPDATE_FALLBACK_UNIX_CMD =
+const STUDIO_INSTALL_UNIX_CMD =
"curl -fsSL https://unsloth.ai/install.sh | sh";
-const STUDIO_UPDATE_FALLBACK_WINDOWS_CMD =
- "irm https://unsloth.ai/install.ps1 | iex";
+const STUDIO_INSTALL_WINDOWS_CMD = "irm https://unsloth.ai/install.ps1 | iex";
const STUDIO_LOCAL_PULL_CMD = "git pull --ff-only";
-const STUDIO_LOCAL_UPDATE_CMD = "unsloth studio update --local";
-const STUDIO_LOCAL_FALLBACK_UNIX_CMD = "./install.sh --local";
-const STUDIO_LOCAL_FALLBACK_WINDOWS_CMD = ".\\install.ps1 --local";
+const STUDIO_LOCAL_INSTALL_UNIX_CMD = "./install.sh --local";
+const STUDIO_LOCAL_INSTALL_WINDOWS_CMD = ".\\install.ps1 --local";
+
+const DOCS_INSTALL_URL = "https://unsloth.ai/docs/get-started/install";
+const DOCS_UPDATING_URL =
+ "https://unsloth.ai/docs/get-started/install/updating";
+const DOCS_MAC_URL = "https://unsloth.ai/docs/get-started/install/mac";
+const DOCS_WINDOWS_URL =
+ "https://unsloth.ai/docs/get-started/install/windows-installation";
export type UpdateShell = "windows" | "unix";
export type UpdateInstallSource =
@@ -30,15 +38,6 @@ export type UpdateInstallSource =
| "unknown";
type UpdateInstallSourceState = UpdateInstallSource | "loading";
-function getStudioUpdateInstructionLine(
- shell: UpdateShell,
- t: ReturnType,
-): string {
- return shell === "windows"
- ? t("settings.about.update.openPowerShell")
- : t("settings.about.update.openTerminal");
-}
-
function isLocalInstallSource(
installSource?: UpdateInstallSourceState | null,
): boolean {
@@ -128,6 +127,77 @@ function CopyableCommand({
);
}
+function DocsLink({
+ href,
+ label,
+}: {
+ href: string;
+ label: string;
+}): ReactElement {
+ return (
+
+ {label}
+
+
+ );
+}
+
+function UpdateDocsLinks(): ReactElement {
+ const t = useT();
+ return (
+
+ {t("settings.about.update.docs")}
+
+
+
+
+
+ );
+}
+
+function ShellToggleButton({
+ active,
+ label,
+ onClick,
+}: {
+ active: boolean;
+ label: string;
+ onClick: () => void;
+}): ReactElement {
+ return (
+
+ {label}
+
+ );
+}
+
// biome-ignore lint/complexity/noExcessiveCognitiveComplexity: keep source-specific update guidance in one component so the command matrix stays visible.
export function UpdateStudioInstructions({
className,
@@ -145,6 +215,9 @@ export function UpdateStudioInstructions({
const shell = shellOverride ?? defaultShell;
const prefersReducedMotion = useReducedMotion();
const windows = shell === "windows";
+ // null means the desktop app: its bundled backend updates through the
+ // built-in updater, so terminal commands would target the wrong install.
+ const desktopManaged = installSource === null;
const localInstallSource = isLocalInstallSource(installSource);
const checkoutInstallSource =
installSource === "editable" || installSource === "local_repo";
@@ -163,6 +236,22 @@ export function UpdateStudioInstructions({
? { opacity: 1 }
: { opacity: 0, y: -2 };
+ if (desktopManaged) {
+ return (
+
+ {showTitle ? (
+
+ {t("settings.about.update.title")}
+
+ ) : null}
+
+ {t("settings.about.update.desktopManaged")}
+
+
+
+ );
+ }
+
return (
) : null}
-
- setShellOverride("windows")}
- className={cn(
- "px-0.5 py-0.5 font-medium transition-colors",
- windows
- ? "text-foreground"
- : "text-muted-foreground hover:text-emerald-600",
- )}
- aria-pressed={windows}
- >
- Windows
-
- /
-
+ setShellOverride("unix")}
- className={cn(
- "px-0.5 py-0.5 font-medium transition-colors",
- windows
- ? "text-muted-foreground hover:text-emerald-600"
- : "text-foreground",
- )}
- aria-pressed={!windows}
- >
- macOS/Linux
-
+ />
+ setShellOverride("windows")}
+ />
{loadingInstallSource ? (
{t("settings.about.update.checkingInstall")}
- ) : localInstallSource ? (
+ ) : (
<>
+
+ {t("settings.about.update.installIntro")}
+
+
+
+
+
+
+ >
+ )}
+ {loadingInstallSource ? null : localInstallSource ? (
+ <>
+
+ {t("settings.about.update.localUpdateHeading")}
+
{t("settings.about.update.localInstallDetected")}
@@ -224,16 +326,9 @@ export function UpdateStudioInstructions({
command={STUDIO_LOCAL_PULL_CMD}
copyLabel={t("settings.about.update.gitPullCommand")}
/>
-
-
- {t("settings.about.update.localInstallerFallback")}
-
@@ -261,7 +356,7 @@ export function UpdateStudioInstructions({
@@ -282,54 +377,22 @@ export function UpdateStudioInstructions({
{t("settings.about.update.restartAfterUpdate")}
+
>
) : unknownInstallSource ? (
<>
{t("settings.about.update.unknownInstall")}
-
- {t("settings.about.update.curlOrPypi")}
+
+ {t("settings.about.update.localUpdateHeading")}
-
{t("settings.about.update.localCheckout")}
-
-
- {t("settings.about.update.restartAfterUpdate")}
-
- >
- ) : (
- <>
-
-
- {getStudioUpdateInstructionLine(shell, t)}
-
-
-
-
- {t("settings.about.update.fallbackInstruction")}
-
{t("settings.about.update.restartAfterUpdate")}
+
+ >
+ ) : (
+ <>
+
+ {t("settings.about.update.restartAfterUpdate")}
+
+
>
)}
diff --git a/studio/frontend/src/features/settings/tabs/about-tab.tsx b/studio/frontend/src/features/settings/tabs/about-tab.tsx
index e73ad802de..69c4aa6ef1 100644
--- a/studio/frontend/src/features/settings/tabs/about-tab.tsx
+++ b/studio/frontend/src/features/settings/tabs/about-tab.tsx
@@ -142,7 +142,7 @@ export function AboutTab() {
-
+
{studioVersion}
@@ -207,6 +207,37 @@ export function AboutTab() {
+
+
+
+ {t("settings.about.license.studioLicense")}
+
+
+
+
+
+ {t("settings.about.license.libraryLicense")}
+
+
+
+
+
s.autoTitle);
const setAutoTitle = useChatRuntimeStore((s) => s.setAutoTitle);
const chatOnly = usePlatformStore((s) => s.chatOnly);
+ const showLlamaUpdates = useShowLlamaUpdateBanner();
const redirectTo = `${pathname}${search}`;
const [draftToken, setDraftToken] = useState(hfToken ?? "");
@@ -315,6 +322,20 @@ export function GeneralTab() {
+
+
+
+
+
+
- Studio: pick base model, dataset, hyperparams, then start training. After
+ Unsloth: pick base model, dataset, hyperparams, then start training. After
you start, you’ll see a Training view with live loss/metrics. Chat is for
testing base vs LoRA adapters. Export packages checkpoints for deployment.{" "}
diff --git a/studio/frontend/src/hooks/use-llama-update-check.ts b/studio/frontend/src/hooks/use-llama-update-check.ts
index 1ac5d9768e..5c609558d6 100644
--- a/studio/frontend/src/hooks/use-llama-update-check.ts
+++ b/studio/frontend/src/hooks/use-llama-update-check.ts
@@ -5,11 +5,14 @@ import { authFetch, getAuthToken } from "@/features/auth";
import { useCallback, useEffect, useRef, useState } from "react";
// First check shortly after load, then re-surface as an hourly reminder. The
-// banner stays up until the user dismisses it (click outside / X) or updates.
+// banner stays up until the user explicitly acts on it (X, Update, or
+// Remind me later).
const FIRST_CHECK_DELAY_MS = 1000;
const REMINDER_INTERVAL_MS = 60 * 60 * 1000; // ~1 hour
+// "Remind me later" re-surfaces sooner than the hourly reminder.
+const SNOOZE_DELAY_MS = 15 * 60 * 1000; // ~15 minutes
// While an update is applying, poll the job state at this cadence.
-const JOB_POLL_INTERVAL_MS = 3000;
+const JOB_POLL_INTERVAL_MS = 1500;
export interface LlamaUpdateJob {
state: "idle" | "running" | "success" | "error";
@@ -17,6 +20,8 @@ export interface LlamaUpdateJob {
from_tag: string | null;
to_tag: string | null;
error: string | null;
+ // Download fraction (0..1) while running, 1 on success, null when unknown.
+ progress: number | null;
}
export interface LlamaUpdateStatus {
@@ -42,6 +47,7 @@ function parseStatus(value: unknown): LlamaUpdateStatus | null {
from_tag: typeof job.from_tag === "string" ? job.from_tag : null,
to_tag: typeof job.to_tag === "string" ? job.to_tag : null,
error: typeof job.error === "string" ? job.error : null,
+ progress: typeof job.progress === "number" ? job.progress : null,
},
};
}
@@ -73,9 +79,10 @@ export interface LlamaApplyResult {
/**
* Polls the backend for a newer llama.cpp prebuilt. When one exists, `visible`
- * becomes true ~1s after load and stays up until the user dismisses it (click
- * outside / X) or updates; it re-surfaces every ~hour as a reminder. `apply()`
- * triggers the in-place swap and tracks the job.
+ * becomes true ~1s after load and stays up until the user dismisses it (X),
+ * snoozes it ("Remind me later", ~15 min), or updates; it re-surfaces every
+ * ~hour as a reminder. `apply()` triggers the in-place swap and tracks the
+ * job.
*/
export function useLlamaUpdateCheck({
enabled = true,
@@ -84,6 +91,7 @@ export function useLlamaUpdateCheck({
const [visible, setVisible] = useState(false);
const [applying, setApplying] = useState(false);
const pollTimer = useRef | null>(null);
+ const snoozeTimer = useRef | null>(null);
const clearPollTimer = useCallback(() => {
if (pollTimer.current) {
@@ -161,6 +169,10 @@ export function useLlamaUpdateCheck({
clearTimeout(firstTimer);
clearInterval(reminder);
clearPollTimer();
+ if (snoozeTimer.current) {
+ clearTimeout(snoozeTimer.current);
+ snoozeTimer.current = null;
+ }
};
}, [enabled, surfaceIfAvailable, clearPollTimer]);
@@ -168,6 +180,16 @@ export function useLlamaUpdateCheck({
setVisible(false);
}, []);
+ // Hide now, re-check and re-surface after SNOOZE_DELAY_MS.
+ const snooze = useCallback(() => {
+ setVisible(false);
+ if (snoozeTimer.current) clearTimeout(snoozeTimer.current);
+ snoozeTimer.current = setTimeout(() => {
+ snoozeTimer.current = null;
+ fetchStatus().then(surfaceIfAvailable);
+ }, SNOOZE_DELAY_MS);
+ }, [surfaceIfAvailable]);
+
const apply = useCallback(async (): Promise => {
if (applying) return { ok: false, error: "already running" };
setApplying(true);
@@ -219,5 +241,6 @@ export function useLlamaUpdateCheck({
applying,
apply,
dismiss,
+ snooze,
};
}
diff --git a/studio/frontend/src/hooks/use-llama-update-pref.ts b/studio/frontend/src/hooks/use-llama-update-pref.ts
new file mode 100644
index 0000000000..33a21965c4
--- /dev/null
+++ b/studio/frontend/src/hooks/use-llama-update-pref.ts
@@ -0,0 +1,49 @@
+// SPDX-License-Identifier: AGPL-3.0-only
+// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
+
+import { useSyncExternalStore } from "react";
+
+// Whether the llama.cpp update banner may appear. On by default; only an
+// explicit "false" (Settings -> General -> Notifications) disables it.
+const STORAGE_KEY = "unsloth_show_llama_update_banner";
+
+const listeners = new Set<() => void>();
+
+export function getShowLlamaUpdateBanner(): boolean {
+ try {
+ return localStorage.getItem(STORAGE_KEY) !== "false";
+ } catch {
+ return true;
+ }
+}
+
+export function setShowLlamaUpdateBanner(show: boolean): void {
+ try {
+ if (show) {
+ // Remove rather than store "true" so the default stays on.
+ localStorage.removeItem(STORAGE_KEY);
+ } else {
+ localStorage.setItem(STORAGE_KEY, "false");
+ }
+ } catch {
+ // storage unavailable
+ }
+ for (const listener of listeners) listener();
+}
+
+function subscribe(listener: () => void): () => void {
+ listeners.add(listener);
+ // Sync toggles made in another tab.
+ const onStorage = (event: StorageEvent) => {
+ if (event.key === STORAGE_KEY) listener();
+ };
+ window.addEventListener("storage", onStorage);
+ return () => {
+ listeners.delete(listener);
+ window.removeEventListener("storage", onStorage);
+ };
+}
+
+export function useShowLlamaUpdateBanner(): boolean {
+ return useSyncExternalStore(subscribe, getShowLlamaUpdateBanner);
+}
diff --git a/studio/frontend/src/hooks/use-tauri-backend.ts b/studio/frontend/src/hooks/use-tauri-backend.ts
index b376cb3a37..086cfc8090 100644
--- a/studio/frontend/src/hooks/use-tauri-backend.ts
+++ b/studio/frontend/src/hooks/use-tauri-backend.ts
@@ -56,23 +56,23 @@ function wait(ms: number) {
function externalConflictMessage(preflight: DesktopPreflightResult) {
if (preflight.reason === "desktop_owned_backend_active") {
return preflight.port
- ? `A desktop-owned Studio server for this install is already running on port ${preflight.port}. Quit the other desktop app instance, then try again.`
- : "A desktop-owned Studio server for this install is already running. Quit the other desktop app instance, then try again.";
+ ? `A desktop-owned Unsloth server for this install is already running on port ${preflight.port}. Quit the other desktop app instance, then try again.`
+ : "A desktop-owned Unsloth server for this install is already running. Quit the other desktop app instance, then try again.";
}
if (preflight.reason === "desktop_owned_backend_starting") {
- return "The desktop-owned Studio backend is still starting. Wait a moment, then try again.";
+ return "The desktop-owned Unsloth backend is still starting. Wait a moment, then try again.";
}
if (preflight.reason?.startsWith("desktop_owned_backend_unmanageable:")) {
return preflight.port
- ? `A desktop-owned Studio backend on port ${preflight.port} cannot be safely controlled by this desktop app. Stop that backend, then reopen Studio.`
- : "A desktop-owned Studio backend cannot be safely controlled by this desktop app. Stop that backend, then reopen Studio.";
+ ? `A desktop-owned Unsloth backend on port ${preflight.port} cannot be safely controlled by this desktop app. Stop that backend, then reopen Unsloth.`
+ : "A desktop-owned Unsloth backend cannot be safely controlled by this desktop app. Stop that backend, then reopen Unsloth.";
}
return preflight.port
- ? `A Studio server for this install is already running from a terminal on port ${preflight.port}. Stop that server, or run \`unsloth studio update\` from that terminal before using the desktop app.`
- : "A Studio server for this install is already running from a terminal. Stop that server, or run `unsloth studio update` from that terminal before using the desktop app.";
+ ? `A Unsloth server for this install is already running from a terminal on port ${preflight.port}. Stop that server, or run \`unsloth studio update\` from that terminal before using the desktop app.`
+ : "A Unsloth server for this install is already running from a terminal. Stop that server, or run `unsloth studio update` from that terminal before using the desktop app.";
}
async function waitForManagedServerReady(
@@ -248,8 +248,8 @@ export function useTauriBackend() {
} else {
setBackendError(
preflight.disposition === "owned_stale"
- ? "Desktop-owned Studio backend is too old for this desktop app. Run `unsloth studio update`, then restart Studio."
- : "Managed Studio install is too old. Run `unsloth studio update`.",
+ ? "Desktop-owned Unsloth backend is too old for this desktop app. Run `unsloth studio update`, then restart Unsloth."
+ : "Managed Unsloth install is too old. Run `unsloth studio update`.",
);
}
return;
@@ -304,7 +304,7 @@ export function useTauriBackend() {
if (msg.includes("already running")) {
startingRef.current = false;
setBackendError(
- "Managed server is already running but did not report a port. Restart Studio and try again.",
+ "Managed server is already running but did not report a port. Restart Unsloth and try again.",
);
return;
}
@@ -596,7 +596,7 @@ export function useTauriBackend() {
const detail =
event instanceof CustomEvent && typeof event.detail === "string"
? event.detail
- : "Desktop authentication failed. Update or repair the managed Studio install, then restart Studio.";
+ : "Desktop authentication failed. Update or repair the managed Unsloth install, then restart Unsloth.";
setAuthFailure(detail);
};
window.addEventListener("tauri-auth-failed", onAuthFailed);
diff --git a/studio/frontend/src/i18n/locales/en.ts b/studio/frontend/src/i18n/locales/en.ts
index a0ee28fd86..19863d42bb 100644
--- a/studio/frontend/src/i18n/locales/en.ts
+++ b/studio/frontend/src/i18n/locales/en.ts
@@ -83,7 +83,7 @@ export const en = {
title: "Settings",
dialog: {
title: "Settings",
- description: "Manage your Unsloth Studio preferences.",
+ description: "Manage your Unsloth preferences.",
closeAriaLabel: "Close settings",
},
tabs: {
@@ -93,11 +93,11 @@ export const en = {
chat: "Chat",
connections: "Connections",
apiKeys: "API",
- about: "Help",
+ about: "About",
},
general: {
title: "General",
- description: "Global preferences for Unsloth Studio.",
+ description: "Global preferences for Unsloth.",
account: "Account",
huggingFaceToken: "Hugging Face token",
huggingFaceTokenDescription:
@@ -112,38 +112,44 @@ export const en = {
sectionTitle: "Helper LLM",
preloadOnStartup: "Pre-cache Helper LLM on startup",
preloadOnStartupDescription:
- "Download and cache the AI Assist helper model in the background when Studio starts. Off by default; AI Assist can still download it on demand when clicked.",
+ "Download the AI Assist helper model in the background on startup. Off by default; AI Assist can still fetch it on demand.",
disabledByEnv:
"Disabled by UNSLOTH_HELPER_MODEL_DISABLE in the backend environment.",
loadError: "Failed to load Helper LLM settings.",
saveError: "Failed to save Helper LLM settings.",
},
+ notifications: {
+ sectionTitle: "Notifications",
+ showLlamaUpdates: "llama.cpp update notifications",
+ showLlamaUpdatesDescription:
+ "Notify when a newer llama.cpp build is available. Turn off if you only train.",
+ },
gettingStarted: "Getting started",
startOnboarding: "Start onboarding",
startOnboardingDescription:
- "Open the setup wizard again without changing your account.",
+ "Reopen the setup wizard without changing your account.",
startOnboardingAction: "Start onboarding",
uploads: {
sectionTitle: "Uploads",
maxUploadSize: "Training dataset upload cap",
maxUploadSizeDescription:
- "Applies to training dataset uploads. Default is {defaultSize} MB.",
+ "Default is {defaultSize} MB.",
},
resetPreferences: {
sectionTitle: "Danger zone",
label: "Reset all local preferences",
description:
- "Clears local-only preferences. Chats, API access, and DB-backed settings are not affected.",
+ "Clears local-only preferences. Chats, API access, and DB-backed settings are kept.",
action: "Reset preferences",
confirmTitle: "Reset all local preferences?",
confirmDescription:
- "This clears local-only preferences, then reloads Studio. Chats, API access, and DB-backed settings are not affected.",
+ "Clears local-only preferences and reloads Unsloth. Chats, API access, and DB-backed settings are kept.",
confirmAction: "Reset and reload",
},
},
profile: {
title: "Profile",
- description: "Update how your profile appears in Studio.",
+ description: "How your profile appears in Unsloth.",
changePicture: "Change profile picture",
displayName: "Display name",
nameSaved: "Profile name saved",
@@ -163,7 +169,7 @@ export const en = {
theme: {
title: "Theme",
label: "Color scheme",
- description: "Choose light, dark, or follow your system.",
+ description: "Light, dark, or follow your system.",
system: "System",
light: "Light",
dark: "Dark",
@@ -171,7 +177,7 @@ export const en = {
language: {
title: "Language",
label: "Display language",
- description: "Choose the language used by Studio.",
+ description: "The language used by Unsloth.",
},
layout: {
title: "Layout",
@@ -182,25 +188,25 @@ export const en = {
},
chat: {
title: "Chat",
- description: "Manage your chat history stored on this device.",
+ description: "Manage chat history stored on this device.",
artifacts: {
title: "Artifacts",
collapseHtmlBlocks: "Collapse HTML blocks",
collapseHtmlBlocksDescription:
- "Artifacts mode collapses full HTML fallback automatically. Turn this on to also collapse full fenced HTML documents when Artifacts is off.",
+ "Artifacts mode collapses full HTML automatically. Turn on to also collapse fenced HTML documents when Artifacts is off.",
allowNetworkAccess: "Allow artifact network access",
allowNetworkAccessDescription:
- "Let artifact previews load scripts, styles, fonts, media, fetch, and WebSocket resources from HTTP(S) CDNs. Keep off for fully offline previews.",
+ "Let artifact previews load scripts, styles, fonts, media, and network resources from CDNs. Keep off for fully offline previews.",
},
data: "Data",
exportHistory: "Export chat history",
exportHistoryDescription:
- "Download all chats and messages as a JSON file.",
+ "Download all chats and messages as JSON.",
exportAction: "Export",
exportingAction: "Exporting...",
exportConversations: "Export Recents and Projects",
exportConversationsDescription:
- "Download Recents only, or Recents plus project chats, as Raw JSONL, CSV, or ShareGPT JSONL, combined or one file per chat.",
+ "Download Recents or Recents plus project chats as Raw JSONL, CSV, or ShareGPT JSONL, combined or per chat.",
exportConversationsAction: "Export",
exportScopeRecents: "Recents",
exportScopeAll: "Recents + Projects",
@@ -208,14 +214,14 @@ export const en = {
exportPerChatSuffix: "(per chat)",
importChats: "Import chats",
importChatsDescription:
- "Add conversations from a JSONL, NDJSON, or CSV export to Recents.",
+ "Import a JSONL, NDJSON, or CSV export into Recents.",
importChatsAction: "Import",
importNoConversations: "No conversations found in file.",
importedOneChat: "Imported 1 conversation to Recents.",
importedChatCount: "Imported {count} conversations to Recents.",
importFailed: "Import failed.",
clearHistory: "Clear chat history",
- clearHistoryDescription: "Delete local chat history from this device.",
+ clearHistoryDescription: "Delete chat history from this device.",
clearAction: "Clear",
clearAllChats: "Clear all chats",
clearAllChatsDescription: "Permanently delete every chat on this device.",
@@ -228,7 +234,7 @@ export const en = {
clearOneChatTitle: "Clear 1 chat?",
clearChatsTitle: "Clear {count} chats?",
clearChatsConfirmDescription:
- "This permanently deletes every chat and message stored on this device. This cannot be undone.",
+ "Permanently deletes every chat on this device. This cannot be undone.",
clearingAction: "Clearing...",
clearOneChatAction: "Clear 1 chat",
clearChatCountAction: "Clear {count} chats",
@@ -251,12 +257,12 @@ export const en = {
},
connections: {
title: "Connections",
- description: "Manage providers and external service connections.",
+ description: "Manage providers and external connections.",
},
apiKeys: {
title: "API",
description:
- "Access Unsloth programmatically via the OpenAI-compatible API.",
+ "Access Unsloth via the OpenAI-compatible API.",
readDocs: "Read the API docs",
noAccess: "No API access yet.",
newBadge: "New",
@@ -296,62 +302,71 @@ export const en = {
revokeToken: "Revoke token",
revokeTitle: 'Revoke access token "{name}"?',
revokeDescription:
- "Applications using this token will immediately lose access. This cannot be undone.",
+ "Apps using this token immediately lose access. This cannot be undone.",
revokeAction: 'Revoke "{name}"',
revoking: "Revoking...",
},
about: {
title: "About",
description:
- "Documentation, release notes, feedback, and Studio build info.",
- studioVersion: "Studio Version",
+ "Docs, release notes, feedback, and build info.",
+ studioVersion: "Unsloth Version",
packageVersion: "Package Version",
- updates: "Updates",
+ updates: "Update",
help: "Help",
documentation: "Documentation",
releaseNotes: "Release notes",
whatsNew: "What's new",
feedback: "Feedback",
reportIssue: "Report an issue",
+ license: {
+ sectionTitle: "License",
+ studioLabel: "Unsloth Studio",
+ studioLicense: "AGPL-3.0",
+ studioDescription:
+ "Open source under the GNU AGPL v3.0.",
+ libraryLabel: "Unsloth Core",
+ libraryLicense: "Apache-2.0",
+ libraryDescription: "Licensed under Apache 2.0.",
+ },
dangerZone: "Danger zone",
shutDownStudio: "Shut down Unsloth Studio",
shutDownStudioDescription:
- "Stops the Studio server process and ends your session.",
+ "Stops the Unsloth server and ends your session.",
shutDown: "Shut down",
update: {
title: "Update Unsloth Studio",
- openPowerShell: "Open PowerShell and run:",
- openTerminal: "Open Terminal and run:",
commandText: "{label} text",
copied: "Copied",
copyCommand: "Copy command",
commandCopied: "{label} copied",
copyNamedCommand: "Copy {label}",
- checkingInstall: "Checking how Studio was installed...",
+ checkingInstall: "Checking how Unsloth was installed...",
+ installIntro: "To install or update Unsloth:",
+ localUpdateHeading: "Local update",
+ installCommandUnix: "macOS/Linux install command",
+ installCommandWindows: "Windows install command",
localInstallDetected:
- "Source or local install detected. To avoid replacing it with PyPI, update from the checkout or source you originally installed from.",
- pullThenUpdate:
- "Pull latest changes from your Unsloth repo checkout, then update Studio locally:",
+ "Local install detected. Update from your original checkout to avoid replacing it with PyPI.",
+ pullThenUpdate: "Pull the latest changes, then run the local installer:",
gitPullCommand: "git pull command",
- localUpdateCommand: "local update command",
- localInstallerFallback:
- "If the Studio update command is unavailable, run the local installer from that checkout:",
localInstallerCommand: "local installer command",
sourceInstallDetected:
- "This looks like a source or VCS package install. Reinstall from the original local path or Git URL you used.",
+ "Source or VCS package install detected. Reinstall from the original local path or Git URL.",
repoCheckoutFallback:
- "If you still have the Unsloth repo checkout, run the local installer from that checkout:",
- restartAfterUpdate:
- "Restart Studio after updating for changes to take effect.",
+ "If you still have the repo checkout, run the local installer from it:",
+ restartAfterUpdate: "Restart Unsloth after updating.",
+ desktopManaged:
+ "The desktop app keeps its bundled backend updated and will prompt when a new version is available.",
unknownInstall:
- "Studio could not detect how it was installed. Check how you installed Studio first, then choose the matching update path.",
- curlOrPypi: "For curl or PyPI installs, run:",
- updateCommand: "update command",
+ "Could not detect how Unsloth was installed. For installer or PyPI installs, use the commands above.",
localCheckout:
- "For local checkout installs, update from that checkout instead and use the local update command:",
- fallbackInstruction:
- "If that fails or unsloth studio update is unavailable, run:",
- fallbackCommand: "fallback command",
+ "For local checkout installs, run the local installer from that checkout:",
+ docs: "Install docs:",
+ docsInstall: "Installation",
+ docsUpdating: "Updating",
+ docsMac: "Mac",
+ docsWindows: "Windows",
},
},
},
diff --git a/studio/frontend/src/i18n/locales/zh-CN.ts b/studio/frontend/src/i18n/locales/zh-CN.ts
index e7d3babfe9..5173f9fa74 100644
--- a/studio/frontend/src/i18n/locales/zh-CN.ts
+++ b/studio/frontend/src/i18n/locales/zh-CN.ts
@@ -93,7 +93,7 @@ export const zhCN = {
chat: "聊天",
connections: "连接",
apiKeys: "API",
- about: "帮助",
+ about: "关于",
},
general: {
title: "通用",
@@ -106,6 +106,12 @@ export const zhCN = {
chatDefaults: "聊天默认设置",
autoTitleNewChats: "自动为新聊天命名",
autoTitleNewChatsDescription: "根据第一条消息生成简短标题。",
+ notifications: {
+ sectionTitle: "通知",
+ showLlamaUpdates: "llama.cpp 更新通知",
+ showLlamaUpdatesDescription:
+ "有新的 llama.cpp 构建时提醒。如果只用于训练可以关闭。",
+ },
gettingStarted: "入门",
startOnboarding: "开始引导",
startOnboardingDescription: "重新打开设置向导,不会更改你的账号。",
@@ -124,13 +130,13 @@ export const zhCN = {
action: "重置偏好设置",
confirmTitle: "重置所有本地偏好设置?",
confirmDescription:
- "这会清除仅保存在本地的偏好设置,然后重新加载 Studio。聊天、API 访问权限和数据库中的设置不会受到影响。",
+ "这会清除仅保存在本地的偏好设置,然后重新加载 Unsloth。聊天、API 访问权限和数据库中的设置不会受到影响。",
confirmAction: "重置并重新加载",
},
},
profile: {
title: "个人资料",
- description: "更新你在 Studio 中显示的个人资料。",
+ description: "更新你在 Unsloth 中显示的个人资料。",
changePicture: "更换头像",
displayName: "显示名称",
nameSaved: "个人资料名称已保存",
@@ -150,7 +156,7 @@ export const zhCN = {
language: {
title: "语言",
label: "显示语言",
- description: "选择 Studio 使用的语言。",
+ description: "选择 Unsloth 使用的语言。",
},
theme: {
title: "主题",
@@ -274,9 +280,9 @@ export const zhCN = {
revoking: "撤销中...",
},
about: {
- title: "帮助",
- description: "文档、发布说明、反馈和 Studio 构建信息。",
- studioVersion: "Studio 版本",
+ title: "关于",
+ description: "文档、发布说明、反馈和 Unsloth 构建信息。",
+ studioVersion: "Unsloth 版本",
packageVersion: "包版本",
updates: "更新",
help: "帮助",
@@ -285,43 +291,52 @@ export const zhCN = {
whatsNew: "最新内容",
feedback: "反馈",
reportIssue: "报告问题",
+ license: {
+ sectionTitle: "许可证",
+ studioLabel: "Unsloth Studio",
+ studioLicense: "AGPL-3.0",
+ studioDescription: "基于 GNU AGPL v3.0 开源。",
+ libraryLabel: "Unsloth Core",
+ libraryLicense: "Apache-2.0",
+ libraryDescription: "基于 Apache License 2.0 许可。",
+ },
dangerZone: "危险区域",
shutDownStudio: "关闭 Unsloth Studio",
- shutDownStudioDescription: "停止 Studio 服务进程并结束你的会话。",
+ shutDownStudioDescription: "停止 Unsloth 服务进程并结束你的会话。",
shutDown: "关闭",
update: {
title: "更新 Unsloth Studio",
- openPowerShell: "打开 PowerShell 并运行:",
- openTerminal: "打开终端并运行:",
commandText: "{label} 文本",
copied: "已复制",
copyCommand: "复制命令",
commandCopied: "{label} 已复制",
copyNamedCommand: "复制 {label}",
- checkingInstall: "正在检查 Studio 的安装方式...",
+ checkingInstall: "正在检查 Unsloth 的安装方式...",
+ installIntro: "安装或更新 Unsloth:",
+ localUpdateHeading: "本地更新",
+ installCommandUnix: "macOS/Linux 安装命令",
+ installCommandWindows: "Windows 安装命令",
localInstallDetected:
- "检测到源码或本地安装。为避免替换为 PyPI 版本,请从最初安装时使用的 checkout 或源码位置更新。",
- pullThenUpdate:
- "从你的 Unsloth 仓库 checkout 拉取最新变更,然后本地更新 Studio:",
+ "检测到本地安装。请从最初的 checkout 更新,以免被 PyPI 版本替换。",
+ pullThenUpdate: "拉取最新变更,然后运行本地安装器:",
gitPullCommand: "git pull 命令",
- localUpdateCommand: "本地更新命令",
- localInstallerFallback:
- "如果 Studio 更新命令不可用,请从该 checkout 运行本地安装器:",
localInstallerCommand: "本地安装器命令",
sourceInstallDetected:
"这看起来是源码或 VCS 包安装。请从最初使用的本地路径或 Git URL 重新安装。",
repoCheckoutFallback:
"如果你仍保留 Unsloth 仓库 checkout,请从该 checkout 运行本地安装器:",
- restartAfterUpdate: "更新后重启 Studio,使变更生效。",
+ restartAfterUpdate: "更新后请重启 Unsloth。",
+ desktopManaged:
+ "桌面应用会自动更新其内置后端,有新版本时会提示。",
unknownInstall:
- "Studio 无法检测安装方式。请先确认你如何安装 Studio,然后选择匹配的更新方式。",
- curlOrPypi: "对于 curl 或 PyPI 安装,请运行:",
- updateCommand: "更新命令",
+ "Unsloth 无法检测安装方式。如果你使用一键安装器或 PyPI 安装,请使用上面的命令。",
localCheckout:
- "对于本地 checkout 安装,请改为从该 checkout 更新并使用本地更新命令:",
- fallbackInstruction:
- "如果失败,或 unsloth studio update 不可用,请运行:",
- fallbackCommand: "备用命令",
+ "对于本地 checkout 安装,请改为从该 checkout 运行本地安装器:",
+ docs: "安装文档:",
+ docsInstall: "安装",
+ docsUpdating: "更新",
+ docsMac: "Mac",
+ docsWindows: "Windows",
},
},
},
diff --git a/studio/frontend/src/index.css b/studio/frontend/src/index.css
index 0d3b1698e1..9e394a31e7 100644
--- a/studio/frontend/src/index.css
+++ b/studio/frontend/src/index.css
@@ -552,6 +552,7 @@
.sidebar-nav-btn[data-state="open"],
.group\/project-item:hover:not(:has(.sidebar-row-action:hover)) .sidebar-nav-btn,
.group\/project-chat-item:hover:not(:has(.sidebar-row-action:hover)) .sidebar-nav-btn,
+ .group\/projects-item:hover:not(:has(.sidebar-row-action:hover)) .sidebar-nav-btn,
.group\/recent-item:hover:not(:has(.sidebar-row-action:hover)) .sidebar-nav-btn,
.group\/run-item:hover:not(:has(.sidebar-row-action:hover)) .sidebar-nav-btn {
background-color: var(--nav-surface-hover) !important;
@@ -562,6 +563,7 @@
.dark .sidebar-nav-btn[data-state="open"],
.dark .group\/project-item:hover:not(:has(.sidebar-row-action:hover)) .sidebar-nav-btn,
.dark .group\/project-chat-item:hover:not(:has(.sidebar-row-action:hover)) .sidebar-nav-btn,
+ .dark .group\/projects-item:hover:not(:has(.sidebar-row-action:hover)) .sidebar-nav-btn,
.dark .group\/recent-item:hover:not(:has(.sidebar-row-action:hover)) .sidebar-nav-btn,
.dark .group\/run-item:hover:not(:has(.sidebar-row-action:hover)) .sidebar-nav-btn {
color: #fff !important;
@@ -957,6 +959,11 @@
border-radius: 14px !important;
}
+ /* Account menu: a touch rounder than standard list menus. */
+ [data-slot="dropdown-menu-content"].app-user-menu.menu-soft-surface-up {
+ border-radius: 18px !important;
+ }
+
/* Every dropdown/menu/select/popover: borderless, chatbox shadow in light,
none in dark. !important so it also overrides the bespoke menu shadows. */
[data-slot="dropdown-menu-content"],
@@ -1078,13 +1085,16 @@
[data-pill-compact="true"]
.composer-pill-btn:not([data-keep-label])[data-pill-label]:hover::after {
content: attr(data-pill-label);
- @apply pointer-events-none absolute bottom-[calc(100%+6px)] left-1/2 z-50 -translate-x-1/2 rounded-[9px] bg-black px-2.5 py-1.5 text-[11px] font-medium leading-snug whitespace-nowrap text-white shadow-md;
+ /* Always one nowrap line, so always a full pill. */
+ @apply pointer-events-none absolute bottom-[calc(100%+6px)] left-1/2 z-50 -translate-x-1/2 rounded-full bg-black px-2.5 py-1.5 text-[11px] font-medium leading-snug whitespace-nowrap text-white shadow-md;
}
/* Compact caret pills (RAG, MCP) open their menu on click instead of
- toggling off, so keep the icon and skip the X swap. */
+ toggling off, so keep the icon and skip the X swap. No data-active
+ requirement: the off-switch glyph rules below hide the icon on hover
+ even for inactive pills, which left a blank slot in compact mode. */
[data-pill-compact="true"]
- .composer-pill-btn:not([data-keep-label]):has(.composer-pill-caret)[data-active="true"]:hover
+ .composer-pill-btn:not([data-keep-label]):has(.composer-pill-caret):hover
.composer-pill-glyph
> :not(.composer-pill-x) {
opacity: 1;
@@ -1512,6 +1522,20 @@
}
}
+ /* Indeterminate loading bar: a 1/3-width segment sweeping the track. */
+ .loading-bar-slide {
+ animation: loading-bar-slide 1.3s ease-in-out infinite;
+ }
+
+ @keyframes loading-bar-slide {
+ from {
+ transform: translateX(-110%);
+ }
+ to {
+ transform: translateX(420%);
+ }
+ }
+
@keyframes artifact-loading-line {
0% {
transform: translate3d(-125%, 0, 0) scaleX(0.78);
@@ -1750,10 +1774,10 @@
margin: 0 !important;
}
-/* Composer shadow on the dark background color so toasts
- do not merge into card-colored surfaces behind them. */
+/* Dark toasts share the chatbox surface color (.chat-composer-surface
+ uses var(--card) in dark); the composer shadow lifts them off the page. */
.dark [data-sonner-toast][data-styled='true'] {
- background-color: var(--background) !important;
+ background-color: var(--card) !important;
box-shadow: 0 2px 8px -2px rgba(0, 0, 0, 0.16) !important;
}
@@ -2099,6 +2123,11 @@
animation-iteration-count: infinite !important;
}
+ .loading-bar-slide {
+ animation-duration: 1.3s !important;
+ animation-iteration-count: infinite !important;
+ }
+
/* Keep the plus/x morph animating under reduced motion (small rotation,
like the spinners above). */
.unsloth-composer-plus svg {
diff --git a/studio/install_llama_prebuilt.py b/studio/install_llama_prebuilt.py
index 2b42a7a058..bd8c90dc9c 100644
--- a/studio/install_llama_prebuilt.py
+++ b/studio/install_llama_prebuilt.py
@@ -856,6 +856,16 @@ def format_byte_count(num_bytes: float) -> str:
return f"{num_bytes:.1f} B"
+def _progress_percent_step() -> int:
+ """Non-tty milestone granularity. The in-app updater sets
+ UNSLOTH_PROGRESS_PERCENT_STEP=5 to stream finer progress lines."""
+ try:
+ step = int(os.environ.get("UNSLOTH_PROGRESS_PERCENT_STEP", "25"))
+ except ValueError:
+ return 25
+ return min(max(step, 1), 50)
+
+
class DownloadProgress:
def __init__(self, label: str, total_bytes: int | None) -> None:
self.label = label
@@ -868,6 +878,7 @@ class DownloadProgress:
)
self.is_tty = term_ok and self.stream.isatty()
self.completed = False
+ self.milestone_step = _progress_percent_step()
self.last_milestone_percent = -1
self.last_milestone_bytes = 0
self.has_rendered_tty_progress = False
@@ -918,7 +929,8 @@ class DownloadProgress:
should_emit = False
if self.total_bytes is not None:
percent = int((downloaded_bytes * 100) / max(self.total_bytes, 1))
- milestone_percent = min((percent // 25) * 25, 100)
+ step = self.milestone_step
+ milestone_percent = min((percent // step) * step, 100)
if milestone_percent > self.last_milestone_percent and milestone_percent < 100:
self.last_milestone_percent = milestone_percent
should_emit = True
From 983c1f0616b65e3c26a3dc3eeee389e8c9a28ae7 Mon Sep 17 00:00:00 2001
From: Daniel Han
Date: Thu, 11 Jun 2026 09:33:28 -0700
Subject: [PATCH 02/50] Bump install.sh / install.ps1 pin to unsloth>=2026.6.3
(#6212)
PyPI release unsloth 2026.6.3 is now live. Bump the pinned floor in
install.sh and install.ps1 from unsloth>=2026.6.2 to unsloth>=2026.6.3
so fresh installs resolve to the new wheel.
---
install.ps1 | 10 +++++-----
install.sh | 10 +++++-----
2 files changed, 10 insertions(+), 10 deletions(-)
diff --git a/install.ps1 b/install.ps1
index fd7d3f4a81..cf7bb63cdf 100644
--- a/install.ps1
+++ b/install.ps1
@@ -1876,7 +1876,7 @@ shell.Run cmd, 0, False
if ($SkipTorch) {
# No-torch: install unsloth + unsloth-zoo with --no-deps, then
# runtime deps (typer, safetensors, transformers, etc.) with --no-deps.
- $baseInstallExit = Invoke-InstallCommand { uv pip install --python $VenvPython --no-deps --reinstall-package unsloth --reinstall-package unsloth-zoo "unsloth>=2026.6.2" unsloth-zoo }
+ $baseInstallExit = Invoke-InstallCommand { uv pip install --python $VenvPython --no-deps --reinstall-package unsloth --reinstall-package unsloth-zoo "unsloth>=2026.6.3" unsloth-zoo }
if ($baseInstallExit -eq 0) {
# Resolve pydantic WITH deps so pip pins pydantic-core
# to the matching version (no-torch-runtime.txt below
@@ -1890,7 +1890,7 @@ shell.Run cmd, 0, False
}
}
} else {
- $baseInstallExit = Invoke-InstallCommand { uv pip install --python $VenvPython --reinstall-package unsloth --reinstall-package unsloth-zoo "unsloth>=2026.6.2" unsloth-zoo }
+ $baseInstallExit = Invoke-InstallCommand { uv pip install --python $VenvPython --reinstall-package unsloth --reinstall-package unsloth-zoo "unsloth>=2026.6.3" unsloth-zoo }
}
if ($baseInstallExit -ne 0) {
Write-Host "[ERROR] Failed to install unsloth (exit code $baseInstallExit)" -ForegroundColor Red
@@ -1937,7 +1937,7 @@ shell.Run cmd, 0, False
if ($SkipTorch) {
# No-torch: install unsloth + unsloth-zoo with --no-deps, then
# runtime deps (typer, safetensors, transformers, etc.) with --no-deps.
- $baseInstallExit = Invoke-InstallCommand { uv pip install --python $VenvPython --no-deps --upgrade-package unsloth --upgrade-package unsloth-zoo "unsloth>=2026.6.2" unsloth-zoo }
+ $baseInstallExit = Invoke-InstallCommand { uv pip install --python $VenvPython --no-deps --upgrade-package unsloth --upgrade-package unsloth-zoo "unsloth>=2026.6.3" unsloth-zoo }
if ($baseInstallExit -eq 0) {
# Same pydantic-with-deps trick as the migrated branch.
$baseInstallExit = Invoke-InstallCommand { uv pip install --python $VenvPython pydantic }
@@ -1949,7 +1949,7 @@ shell.Run cmd, 0, False
}
}
} elseif ($StudioLocalInstall) {
- $baseInstallExit = Invoke-InstallCommand { uv pip install --python $VenvPython --upgrade-package unsloth "unsloth>=2026.6.2" unsloth-zoo }
+ $baseInstallExit = Invoke-InstallCommand { uv pip install --python $VenvPython --upgrade-package unsloth "unsloth>=2026.6.3" unsloth-zoo }
} else {
$baseInstallExit = Invoke-InstallCommand { uv pip install --python $VenvPython --upgrade-package unsloth -- "$PackageName" }
}
@@ -1977,7 +1977,7 @@ shell.Run cmd, 0, False
Write-TauriLog "STEP" "Installing unsloth"
substep "installing unsloth (this may take a few minutes)..."
if ($StudioLocalInstall) {
- $baseInstallExit = Invoke-InstallCommand { uv pip install --python $VenvPython unsloth-zoo "unsloth>=2026.6.2" --torch-backend=auto }
+ $baseInstallExit = Invoke-InstallCommand { uv pip install --python $VenvPython unsloth-zoo "unsloth>=2026.6.3" --torch-backend=auto }
if ($baseInstallExit -ne 0) {
Write-Host "[ERROR] Failed to install unsloth (exit code $baseInstallExit)" -ForegroundColor Red
return (Exit-InstallFailure "Failed to install unsloth (exit code $baseInstallExit)" $baseInstallExit)
diff --git a/install.sh b/install.sh
index df9eca65c7..532ac61bc0 100755
--- a/install.sh
+++ b/install.sh
@@ -2405,7 +2405,7 @@ if [ "$_MIGRATED" = true ]; then
# to prevent transitive torch resolution.
run_install_cmd "install unsloth (migrated no-torch)" uv pip install --python "$_VENV_PY" --no-deps \
--reinstall-package unsloth --reinstall-package unsloth-zoo \
- "unsloth>=2026.6.2" unsloth-zoo
+ "unsloth>=2026.6.3" unsloth-zoo
# Resolve pydantic WITH deps so pip pins pydantic-core to the
# matching version (no-torch-runtime.txt below is --no-deps).
# All transitive deps are torch-free.
@@ -2418,7 +2418,7 @@ if [ "$_MIGRATED" = true ]; then
else
run_install_cmd "install unsloth (migrated)" uv pip install --python "$_VENV_PY" \
--reinstall-package unsloth --reinstall-package unsloth-zoo \
- "unsloth>=2026.6.2" unsloth-zoo
+ "unsloth>=2026.6.3" unsloth-zoo
fi
if [ "$STUDIO_LOCAL_INSTALL" = true ]; then
substep "overlaying local repo (editable)..."
@@ -2622,7 +2622,7 @@ elif [ -n "$TORCH_INDEX_URL" ]; then
# runtime deps (typer, safetensors, transformers, etc.) with --no-deps.
run_install_cmd "install unsloth (no-torch)" uv pip install --python "$_VENV_PY" --no-deps \
--upgrade-package unsloth --upgrade-package unsloth-zoo \
- "unsloth>=2026.6.2" unsloth-zoo
+ "unsloth>=2026.6.3" unsloth-zoo
# Same pydantic-with-deps trick as the migrated branch.
run_install_cmd "install pydantic (with deps for compatible core)" \
uv pip install --python "$_VENV_PY" pydantic
@@ -2640,7 +2640,7 @@ elif [ -n "$TORCH_INDEX_URL" ]; then
fi
elif [ "$STUDIO_LOCAL_INSTALL" = true ]; then
run_install_cmd "install unsloth (local)" uv pip install --python "$_VENV_PY" \
- --upgrade-package unsloth "unsloth>=2026.6.2" unsloth-zoo
+ --upgrade-package unsloth "unsloth>=2026.6.3" unsloth-zoo
substep "overlaying local repo (editable)..."
run_install_cmd "overlay local repo" uv pip install --python "$_VENV_PY" -e "$_REPO_ROOT" --no-deps
substep "overlaying unsloth-zoo from git main..."
@@ -2672,7 +2672,7 @@ else
tauri_log "STEP" "Installing Unsloth"
substep "installing unsloth (this may take a few minutes)..."
if [ "$STUDIO_LOCAL_INSTALL" = true ]; then
- run_install_cmd "install unsloth (auto torch backend)" uv pip install --python "$_VENV_PY" unsloth-zoo "unsloth>=2026.6.2" --torch-backend=auto
+ run_install_cmd "install unsloth (auto torch backend)" uv pip install --python "$_VENV_PY" unsloth-zoo "unsloth>=2026.6.3" --torch-backend=auto
substep "overlaying local repo (editable)..."
run_install_cmd "overlay local repo" uv pip install --python "$_VENV_PY" -e "$_REPO_ROOT" --no-deps
substep "overlaying unsloth-zoo from git main..."
From 672d8f058199978d06e434ad509f2354f9b8989c Mon Sep 17 00:00:00 2001
From: alkinun
Date: Thu, 11 Jun 2026 22:13:53 +0300
Subject: [PATCH 03/50] Expose runtime context length for hub models (#6154)
* expose runtime context length for hub models
* runtime context helper review
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Etherll <61019402+Etherll@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
---
studio/backend/core/inference/inference.py | 9 ++++
.../backend/core/inference/mlx_inference.py | 2 +
studio/backend/core/inference/orchestrator.py | 1 +
.../backend/core/inference/runtime_context.py | 22 +++++++++
studio/backend/core/inference/worker.py | 13 ++++++
studio/backend/models/inference.py | 2 +-
studio/backend/routes/inference.py | 45 ++++++++++++++-----
.../tests/test_native_context_length.py | 39 ++++++++++++++++
unsloth_cli/commands/studio.py | 23 +++++++++-
.../tests/test_studio_run_parallel_flag.py | 37 +++++++++++++++
10 files changed, 180 insertions(+), 13 deletions(-)
create mode 100644 studio/backend/core/inference/runtime_context.py
diff --git a/studio/backend/core/inference/inference.py b/studio/backend/core/inference/inference.py
index 55c2c551a3..2b9517692f 100644
--- a/studio/backend/core/inference/inference.py
+++ b/studio/backend/core/inference/inference.py
@@ -25,6 +25,7 @@ from utils.hardware import (
get_visible_gpu_count,
)
from core.inference.audio_codecs import AudioCodecManager
+from core.inference.runtime_context import runtime_context_length
from io import StringIO
import structlog
from loggers import get_logger
@@ -405,6 +406,10 @@ class InferenceBackend:
# Reject CPU/disk offload for audio models too
raise_if_offloaded(self.models[model_name]["model"], device_map, "Inference")
+ self.models[model_name]["context_length"] = runtime_context_length(
+ self.models[model_name].get("model"),
+ max_seq_length,
+ )
self.active_model_name = model_name
self.loading_models.discard(model_name)
@@ -485,6 +490,10 @@ class InferenceBackend:
self.models[model_name]["tokenizer"] = tokenizer
raise_if_offloaded(self.models[model_name]["model"], device_map, "Inference")
+ self.models[model_name]["context_length"] = runtime_context_length(
+ self.models[model_name].get("model"),
+ max_seq_length,
+ )
self._load_chat_template_info(model_name)
diff --git a/studio/backend/core/inference/mlx_inference.py b/studio/backend/core/inference/mlx_inference.py
index fbbdb9a773..5c7799152f 100644
--- a/studio/backend/core/inference/mlx_inference.py
+++ b/studio/backend/core/inference/mlx_inference.py
@@ -7,6 +7,7 @@ instead of torch/transformers for model loading and generation.
import threading
from typing import Optional, Generator
+from core.inference.runtime_context import runtime_context_length
from loggers import get_logger
logger = get_logger(__name__)
@@ -175,6 +176,7 @@ class MLXInferenceBackend:
"is_audio": False,
"audio_type": None,
"has_audio_input": False,
+ "context_length": runtime_context_length(self._model, max_seq_length),
}
# Capture chat_template_info so the worker IPC reply ships it back and
# the route layer classifies capabilities like the other paths.
diff --git a/studio/backend/core/inference/orchestrator.py b/studio/backend/core/inference/orchestrator.py
index 32bb25c976..4e4788ace9 100644
--- a/studio/backend/core/inference/orchestrator.py
+++ b/studio/backend/core/inference/orchestrator.py
@@ -727,6 +727,7 @@ class InferenceOrchestrator:
"is_audio": model_info.get("is_audio", False),
"audio_type": model_info.get("audio_type"),
"has_audio_input": model_info.get("has_audio_input", False),
+ "context_length": model_info.get("context_length"),
}
# Mirror chat_template_info so routes can classify caps
# without re-entering the subprocess.
diff --git a/studio/backend/core/inference/runtime_context.py b/studio/backend/core/inference/runtime_context.py
new file mode 100644
index 0000000000..8b881628be
--- /dev/null
+++ b/studio/backend/core/inference/runtime_context.py
@@ -0,0 +1,22 @@
+# SPDX-License-Identifier: AGPL-3.0-only
+# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
+
+"""Runtime context length helpers shared by inference backends."""
+
+from __future__ import annotations
+
+from typing import Any, Optional
+
+
+def runtime_context_length(model: Any, fallback: Optional[int] = None) -> Optional[int]:
+ """Return the effective context length Unsloth attached to a loaded model."""
+ for value in (getattr(model, "max_seq_length", None), fallback):
+ if isinstance(value, bool):
+ continue
+ try:
+ value_int = int(value)
+ except (TypeError, ValueError):
+ continue
+ if value_int > 0:
+ return value_int
+ return None
diff --git a/studio/backend/core/inference/worker.py b/studio/backend/core/inference/worker.py
index 4f5b985234..cc79654087 100644
--- a/studio/backend/core/inference/worker.py
+++ b/studio/backend/core/inference/worker.py
@@ -315,6 +315,18 @@ def _handle_load(backend, config: dict, resp_queue: Any) -> None:
"audio_type": getattr(mc, "audio_type", None),
"has_audio_input": getattr(mc, "has_audio_input", False),
}
+ try:
+ _bm = getattr(backend, "models", {}) or {}
+ _entry = (
+ _bm.get(mc.identifier)
+ or _bm.get(getattr(backend, "active_model_name", None))
+ or {}
+ )
+ _context_length = _entry.get("context_length")
+ if _context_length is not None:
+ model_info["context_length"] = int(_context_length)
+ except Exception as _ctx_exc:
+ logger.warning("context_length forward failed: %s", _ctx_exc)
# Forward chat_template_info so the parent can classify capabilities.
try:
_bm = getattr(backend, "models", {}) or {}
@@ -881,6 +893,7 @@ def run_inference_process(*, cmd_queue: Any, resp_queue: Any, cancel_event, conf
name: {
"is_vision": info.get("is_vision", False),
"is_lora": info.get("is_lora", False),
+ "context_length": info.get("context_length"),
}
for name, info in backend.models.items()
},
diff --git a/studio/backend/models/inference.py b/studio/backend/models/inference.py
index f94d5366eb..b70202d6ff 100644
--- a/studio/backend/models/inference.py
+++ b/studio/backend/models/inference.py
@@ -171,7 +171,7 @@ class LoadResponse(BaseModel):
description = "Whether the model defaults require trust_remote_code to be enabled for loading.",
)
context_length: Optional[int] = Field(
- None, description = "Model's native context length (from GGUF metadata)"
+ None, description = "Runtime context length in tokens for the loaded model"
)
max_context_length: Optional[int] = Field(
None, description = "Maximum context length currently available on this hardware"
diff --git a/studio/backend/routes/inference.py b/studio/backend/routes/inference.py
index 3333db9d07..3850d0cbb2 100644
--- a/studio/backend/routes/inference.py
+++ b/studio/backend/routes/inference.py
@@ -29,6 +29,16 @@ from utils.models import extract_model_size_b as _extract_model_size_b
from utils.api_errors import openai_error_body, anthropic_error_body
+def _positive_int_or_none(value: Any) -> Optional[int]:
+ if isinstance(value, bool):
+ return None
+ try:
+ value_int = int(value)
+ except (TypeError, ValueError):
+ return None
+ return value_int if value_int > 0 else None
+
+
def _install_httpcore_asyncgen_silencer() -> None:
"""Silence benign httpx/httpcore asyncgen GC noise on Python 3.13.
@@ -1390,6 +1400,7 @@ async def load_model(
reasoning_always_on = _sf_flags["reasoning_always_on"],
supports_preserve_thinking = _sf_flags["supports_preserve_thinking"],
supports_tools = _sf_flags["supports_tools"],
+ context_length = _positive_int_or_none(_model_info.get("context_length")),
chat_template = _chat_template,
)
@@ -1731,6 +1742,7 @@ async def load_model(
reasoning_always_on = _sf_flags["reasoning_always_on"],
supports_preserve_thinking = _sf_flags["supports_preserve_thinking"],
supports_tools = _sf_flags["supports_tools"],
+ context_length = _positive_int_or_none(_model_info.get("context_length")),
chat_template = _chat_template,
)
@@ -2119,6 +2131,7 @@ async def get_status(current_subject: str = Depends(get_current_subject)):
reasoning_always_on = _sf_flags["reasoning_always_on"],
supports_preserve_thinking = _sf_flags["supports_preserve_thinking"],
supports_tools = _sf_flags["supports_tools"],
+ context_length = _positive_int_or_none(model_info.get("context_length")),
chat_template = chat_template,
llama_cpp_supports_mtp = _supports_mtp,
llama_cpp_prebuilt_stale = _stale,
@@ -4625,28 +4638,38 @@ def _openai_model_objects() -> list[dict]:
"created": _created,
"owned_by": "local",
}
- # Extension fields: the real per-request window (post /props readback)
- # so clients can budget/compact against the enforced limit.
- if llama_backend.context_length:
- entry["context_length"] = llama_backend.context_length
- if llama_backend.max_context_length:
- entry["max_context_length"] = llama_backend.max_context_length
+ _ctx = _positive_int_or_none(getattr(llama_backend, "context_length", None))
+ if _ctx is not None:
+ entry["context_length"] = _ctx
+ _max_ctx = _positive_int_or_none(getattr(llama_backend, "max_context_length", None))
+ if _max_ctx is not None:
+ entry["max_context_length"] = _max_ctx
+ _native_ctx = _positive_int_or_none(getattr(llama_backend, "native_context_length", None))
+ if _native_ctx is not None:
+ entry["native_context_length"] = _native_ctx
models.append(entry)
# Check Unsloth backend
backend = get_inference_backend()
if backend.active_model_name:
+ model_info = backend.models.get(backend.active_model_name, {})
entry = {
"id": backend.active_model_name,
"object": "model",
"created": _created,
"owned_by": "local",
}
- _sf_ctx = getattr(backend, "context_length", None) or getattr(
- backend, "max_seq_length", None
- )
- if _sf_ctx:
- entry["context_length"] = _sf_ctx
+ _ctx = _positive_int_or_none(model_info.get("context_length"))
+ if _ctx is None:
+ for _candidate in (
+ getattr(backend, "context_length", None),
+ getattr(backend, "max_seq_length", None),
+ ):
+ _ctx = _positive_int_or_none(_candidate)
+ if _ctx is not None:
+ break
+ if _ctx is not None:
+ entry["context_length"] = _ctx
models.append(entry)
return models
diff --git a/studio/backend/tests/test_native_context_length.py b/studio/backend/tests/test_native_context_length.py
index 0290bb1308..de1ca0649e 100644
--- a/studio/backend/tests/test_native_context_length.py
+++ b/studio/backend/tests/test_native_context_length.py
@@ -271,6 +271,7 @@ class TestPydanticModels:
def test_load_response_has_field(self):
"""Field exists in LoadResponse.model_fields."""
assert "native_context_length" in LoadResponse.model_fields
+ assert "context_length" in LoadResponse.model_fields
def test_load_response_defaults_none(self):
"""Omitting native_context_length defaults to None."""
@@ -319,6 +320,7 @@ class TestPydanticModels:
def test_status_response_has_field(self):
"""Field exists in InferenceStatusResponse.model_fields."""
assert "native_context_length" in InferenceStatusResponse.model_fields
+ assert "context_length" in InferenceStatusResponse.model_fields
def test_status_response_has_chat_template_field(self):
"""Status includes chat_template so the UI can rehydrate after refresh."""
@@ -347,6 +349,18 @@ class TestPydanticModels:
roundtripped = LoadResponse.model_validate_json(resp.model_dump_json())
assert roundtripped.native_context_length == 131072
+ def test_context_length_roundtrip(self):
+ """Runtime context_length serializes for non-GGUF/hub models."""
+ resp = LoadResponse(
+ status = "loaded",
+ model = "test",
+ display_name = "Test",
+ inference = {},
+ context_length = 8192,
+ )
+ roundtripped = LoadResponse.model_validate_json(resp.model_dump_json())
+ assert roundtripped.context_length == 8192
+
# =====================================================================
# D. TestRouteCompleteness -- source-level verification
@@ -408,6 +422,16 @@ class TestRouteCompleteness:
"native_context_length" not in block
), f"Non-GGUF LoadResponse should not set native_context_length:\n{block[:200]}"
+ def test_non_gguf_load_responses_set_runtime_context_length(self):
+ """Non-GGUF LoadResponse blocks report runtime context_length."""
+ blocks = self._find_construction_blocks("LoadResponse")
+ non_gguf = [b for b in blocks if "is_gguf = True" not in b and "is_gguf=True" not in b]
+ assert non_gguf, "Expected at least one non-GGUF LoadResponse block"
+ for block in non_gguf:
+ assert (
+ "context_length" in block
+ ), f"Non-GGUF LoadResponse should set context_length:\n{block[:200]}"
+
def test_status_path(self):
"""InferenceStatusResponse construction with llama_backend has the field."""
blocks = self._find_construction_blocks("InferenceStatusResponse")
@@ -420,6 +444,21 @@ class TestRouteCompleteness:
found
), "No InferenceStatusResponse block with llama_backend has native_context_length"
+ def test_non_gguf_status_path_reports_runtime_context_length(self):
+ """Non-GGUF InferenceStatusResponse reports context_length from model_info."""
+ blocks = self._find_construction_blocks("InferenceStatusResponse")
+ found = False
+ for block in blocks:
+ if "is_gguf = False" in block and "context_length" in block:
+ found = True
+ break
+ assert found, "No non-GGUF InferenceStatusResponse block with context_length"
+
+ def test_openai_models_listing_reports_context_length(self):
+ """/v1/models includes context_length when the backend knows it."""
+ assert 'entry["context_length"]' in self._source
+ assert 'model_info.get("context_length")' in self._source
+
# =====================================================================
# E. TestEdgeCases
diff --git a/unsloth_cli/commands/studio.py b/unsloth_cli/commands/studio.py
index 7493b87eee..fb63fbf208 100644
--- a/unsloth_cli/commands/studio.py
+++ b/unsloth_cli/commands/studio.py
@@ -570,6 +570,19 @@ def _load_model_via_http(
raise RuntimeError(f"Model load failed (HTTP {exc.code}): {body}") from exc
+def _format_context_length_line(load_result: dict) -> Optional[str]:
+ value = load_result.get("context_length")
+ if isinstance(value, bool):
+ return None
+ try:
+ value_int = int(value)
+ except (TypeError, ValueError):
+ return None
+ if value_int <= 0:
+ return None
+ return f" Context length: {value_int} tokens"
+
+
# ── unsloth studio (server) ──────────────────────────────────────────
@@ -841,7 +854,10 @@ def run(
None, "--gguf-variant", help = "GGUF quant variant (e.g. UD-Q4_K_XL)"
),
max_seq_length: int = typer.Option(
- 0, "--max-seq-length", help = "Max sequence length (0 = model default)"
+ 0,
+ "--max-seq-length",
+ "--context-length",
+ help = "Runtime context length in tokens (0 = model default for GGUF; 2048 for hub models)",
),
load_in_4bit: bool = typer.Option(True, "--load-in-4bit/--no-load-in-4bit"),
api_key_name: str = typer.Option(
@@ -1080,6 +1096,7 @@ def run(
loaded_model = result.get("model", model)
display_variant = f" ({gguf_variant})" if gguf_variant else ""
+ context_length_line = _format_context_length_line(result)
# 6. Print banner.
display_host = _resolve_external_ip() if host == "0.0.0.0" else host
@@ -1119,6 +1136,8 @@ def run(
if _cf_url:
typer.echo(f" Secure link access via Cloudflare: {_cf_url}")
typer.echo(f" Model loaded: {loaded_model}{display_variant}")
+ if context_length_line:
+ typer.echo(context_length_line)
typer.echo(f" API Key: {api_key}")
typer.echo("")
typer.echo(" OpenAI / Anthropic SDK base URL:")
@@ -1153,6 +1172,8 @@ def run(
typer.echo(f"URL: {base_url}")
if _cf_url:
typer.echo(f"Secure link access via Cloudflare: {_cf_url}")
+ if context_length_line:
+ typer.echo(context_length_line.strip())
typer.echo(f"API Key: {api_key}")
typer.secho(_tool_notice, fg = _tool_notice_fg, bold = True)
diff --git a/unsloth_cli/tests/test_studio_run_parallel_flag.py b/unsloth_cli/tests/test_studio_run_parallel_flag.py
index abc0b3c7c4..8870fa348d 100644
--- a/unsloth_cli/tests/test_studio_run_parallel_flag.py
+++ b/unsloth_cli/tests/test_studio_run_parallel_flag.py
@@ -51,6 +51,18 @@ def test_parallel_option_is_registered():
assert required in flags, f"flag {required!r} missing from --parallel option"
+def test_context_length_alias_is_registered():
+ """`--context-length` is an operator-facing alias for --max-seq-length."""
+ studio_mod = _load_run_command()
+ import inspect
+
+ sig = inspect.signature(studio_mod.run)
+ opt = sig.parameters["max_seq_length"].default
+ flags = set(getattr(opt, "param_decls", None) or [])
+ assert "--max-seq-length" in flags
+ assert "--context-length" in flags
+
+
def test_parallel_default_is_four():
"""Default must stay at 4 so plain `unsloth studio run` is unchanged."""
studio_mod = _load_run_command()
@@ -249,6 +261,15 @@ def test_reexec_np_is_first_class_alias(monkeypatch):
assert _value_after(argv, "--port") == "8888", argv
+def test_reexec_forwards_context_length_alias(monkeypatch):
+ """Alias should normalize to the existing child --max-seq-length flag."""
+ result, captured = _invoke_run(monkeypatch, _BASE + ["--context-length", "8192"])
+ assert len(captured) == 1, result.output
+ argv = captured[0]["argv"]
+ assert _value_after(argv, "--max-seq-length") == "8192", argv
+ assert "--context-length" not in argv, argv
+
+
def test_reexec_mixed_parallel_with_passthrough(monkeypatch):
"""--parallel + llama-server pass-through flags must all reach the child."""
result, captured = _invoke_run(
@@ -262,6 +283,22 @@ def test_reexec_mixed_parallel_with_passthrough(monkeypatch):
assert _value_after(argv, "--temp") == "0.7", argv
+def test_context_length_banner_line_formats_ints():
+ studio_mod = _load_run_command()
+ assert studio_mod._format_context_length_line({"context_length": 4096}) == (
+ " Context length: 4096 tokens"
+ )
+ assert studio_mod._format_context_length_line({"context_length": "8192"}) == (
+ " Context length: 8192 tokens"
+ )
+
+
+@pytest.mark.parametrize("value", [None, 0, -1, True, ""])
+def test_context_length_banner_line_omits_unknown_values(value):
+ studio_mod = _load_run_command()
+ assert studio_mod._format_context_length_line({"context_length": value}) is None
+
+
@pytest.mark.parametrize(
"user_flag,expected_in_child",
[
From fb56b82a383469dd707f92b8ee35e0fdc0d6fb07 Mon Sep 17 00:00:00 2001
From: oobabooga
Date: Thu, 11 Jun 2026 23:23:14 -0300
Subject: [PATCH 04/50] Studio: fix llama.cpp update banner offering a
downgrade / sticking on mix releases (#6219)
---
studio/backend/routes/llama.py | 4 +-
.../backend/tests/test_llama_cpp_freshness.py | 134 +++++++++++++++++-
studio/backend/tests/test_llama_cpp_update.py | 52 ++++++-
studio/backend/utils/llama_cpp_freshness.py | 87 ++++++++++--
studio/backend/utils/llama_cpp_update.py | 33 ++++-
5 files changed, 290 insertions(+), 20 deletions(-)
diff --git a/studio/backend/routes/llama.py b/studio/backend/routes/llama.py
index 47a5bc1dd3..3aae6f4209 100644
--- a/studio/backend/routes/llama.py
+++ b/studio/backend/routes/llama.py
@@ -41,7 +41,9 @@ class LlamaUpdateStatusResponse(BaseModel):
False,
description = "True when the install came from an Unsloth prebuilt (has a marker).",
)
- update_available: bool = Field(False, description = "True when installed_tag != latest_tag.")
+ update_available: bool = Field(
+ False, description = "True when the latest release is genuinely newer than the install."
+ )
stale: bool = Field(
False, description = "Update available AND install older than the staleness threshold."
)
diff --git a/studio/backend/tests/test_llama_cpp_freshness.py b/studio/backend/tests/test_llama_cpp_freshness.py
index cb17e0d5e7..f8e4619ded 100644
--- a/studio/backend/tests/test_llama_cpp_freshness.py
+++ b/studio/backend/tests/test_llama_cpp_freshness.py
@@ -21,12 +21,25 @@ _BACKEND_DIR = str(Path(__file__).resolve().parent.parent)
if _BACKEND_DIR not in sys.path:
sys.path.insert(0, _BACKEND_DIR)
+
+class _NoopLogger:
+ """structlog-style logger: every method swallows positional + kwargs.
+
+ A stdlib logging.Logger rejects structlog's keyword fields (e.g.
+ ``logger.warning(msg, error=...)``), which leaked into the update module's
+ error path and failed only when this file's stub loaded first.
+ """
+
+ def __getattr__(self, _name):
+ return lambda *a, **k: None
+
+
_loggers_stub = _types.ModuleType("loggers")
-_loggers_stub.get_logger = lambda name: __import__("logging").getLogger(name)
+_loggers_stub.get_logger = lambda *a, **k: _NoopLogger()
sys.modules.setdefault("loggers", _loggers_stub)
_structlog_stub = _types.ModuleType("structlog")
-_structlog_stub.get_logger = lambda *a, **k: __import__("logging").getLogger("stub")
+_structlog_stub.get_logger = lambda *a, **k: _NoopLogger()
sys.modules.setdefault("structlog", _structlog_stub)
import pytest
@@ -51,6 +64,11 @@ def _write_marker(install_dir: Path, **overrides) -> Path:
.replace("+00:00", "Z"),
}
payload.update(overrides)
+ # The installer always writes `tag` and `release_tag` from the same release
+ # (a normalized base vs the full release tag), so keep the pair consistent
+ # when a test overrides only `tag`.
+ if "tag" in overrides and "release_tag" not in overrides:
+ payload["release_tag"] = overrides["tag"]
install_dir.mkdir(parents = True, exist_ok = True)
(install_dir / "UNSLOTH_PREBUILT_INFO.json").write_text(json.dumps(payload))
return install_dir / "UNSLOTH_PREBUILT_INFO.json"
@@ -303,3 +321,115 @@ def test_format_stale_warning_singular_day():
msg = fr.format_stale_warning({"installed_tag": "b9190", "latest_tag": "b9300", "age_days": 1})
assert "1 day" in msg
assert "1 days" not in msg
+
+
+# parse_base_build / is_behind.
+
+
+def test_parse_base_build():
+ assert fr.parse_base_build("b9596") == 9596
+ assert fr.parse_base_build(" b9596 ") == 9596
+ assert fr.parse_base_build("b9596-mix-e6f2453") == 9596 # mix suffix doesn't defeat it
+ assert fr.parse_base_build("9596") is None
+ assert fr.parse_base_build("master-abc") is None
+ assert fr.parse_base_build("") is None
+ assert fr.parse_base_build(None) is None
+
+
+@pytest.mark.parametrize(
+ "installed, latest, expected",
+ [
+ (
+ "b9596-mix-e6f2453",
+ "b9596-mix-e6f2453",
+ False,
+ ), # already on the mix latest -> not behind
+ ("b9596", "b9594", False), # latest is an older build -> downgrade guard
+ ("b9596", "b9594-mix-xxx", False), # older mix latest -> still guarded
+ ("b9500", "b9596-mix-e6f2453", True), # newer base -> behind
+ ("b9596-mix-aaa", "b9596-mix-bbb", True), # new mix at same base -> behind
+ ("b9596", "b9596-mix-bbb", True), # clean -> mix at same base -> behind
+ ("b9596-mix-aaa", "b9596", False), # bare base never supersedes a mix install
+ ("b9596", "b9596", False), # identical -> not behind
+ (" b9596 ", "b9596", False), # whitespace-only diff -> not behind
+ ("master-abc", "master-def", True), # non-bNNNN both -> plain inequality
+ ("master-abc", "master-abc", False),
+ (None, "b9596", False),
+ ("b9596", None, False),
+ ],
+)
+def test_is_behind(installed, latest, expected):
+ assert fr.is_behind(installed, latest) is expected
+
+
+def test_check_prebuilt_freshness_not_behind_on_mix_latest(monkeypatch, tmp_path):
+ # Installed the mix latest: marker base tag b9596, full release_tag with sha,
+ # GitHub latest is that same full tag. Must not report behind (sticky bug).
+ install_dir = tmp_path / "llama.cpp"
+ _write_marker(install_dir, tag = "b9596", release_tag = "b9596-mix-e6f2453")
+ bin_path = _fake_binary(install_dir, layout = "root")
+ monkeypatch.setattr(
+ fr, "_fetch_latest_release_tag", lambda repo, timeout = 5.0: "b9596-mix-e6f2453"
+ )
+ info = fr.check_prebuilt_freshness(str(bin_path))
+ assert info["behind"] is False
+ assert info["stale"] is False
+
+
+def test_check_prebuilt_freshness_downgrade_guard(monkeypatch, tmp_path):
+ # A lagging latest (older build than installed) must never read as behind/stale.
+ install_dir = tmp_path / "llama.cpp"
+ _write_marker(
+ install_dir,
+ tag = "b9585",
+ installed_at_utc = (datetime.now(tz = timezone.utc) - timedelta(days = 30))
+ .isoformat()
+ .replace("+00:00", "Z"),
+ )
+ bin_path = _fake_binary(install_dir, layout = "root")
+ monkeypatch.setattr(fr, "_fetch_latest_release_tag", lambda repo, timeout = 5.0: "b9518")
+ info = fr.check_prebuilt_freshness(str(bin_path))
+ assert info["behind"] is False
+ assert info["stale"] is False
+
+
+def test_fetch_latest_release_tag_uses_publish_time(monkeypatch):
+ # Resolves newest by published_at (like the installer), skips drafts/prereleases,
+ # and does NOT just take GitHub's first/`/releases/latest` item.
+ import urllib.request
+
+ class _Resp:
+ def __init__(self, payload):
+ self._p = json.dumps(payload).encode()
+
+ def read(self):
+ return self._p
+
+ def __enter__(self):
+ return self
+
+ def __exit__(self, *a):
+ return False
+
+ payload = [
+ {
+ "tag_name": "b9518",
+ "draft": False,
+ "prerelease": False,
+ "published_at": "2026-06-04T21:11:19Z",
+ },
+ {
+ "tag_name": "b9596-mix-e6f2453",
+ "draft": False,
+ "prerelease": False,
+ "published_at": "2026-06-11T22:50:41Z",
+ },
+ {
+ "tag_name": "b9999-draft",
+ "draft": True,
+ "prerelease": False,
+ "published_at": "2026-06-12T00:00:00Z",
+ },
+ ]
+ monkeypatch.setattr(urllib.request, "urlopen", lambda req, timeout = 5.0: _Resp(payload))
+ assert fr._fetch_latest_release_tag("unslothai/llama.cpp") == "b9596-mix-e6f2453"
diff --git a/studio/backend/tests/test_llama_cpp_update.py b/studio/backend/tests/test_llama_cpp_update.py
index d21d653b76..3b8f511a7c 100644
--- a/studio/backend/tests/test_llama_cpp_update.py
+++ b/studio/backend/tests/test_llama_cpp_update.py
@@ -82,18 +82,21 @@ def _write_install(
tag: str,
repo: str = "unslothai/llama.cpp",
asset: str | None = None,
+ release_tag: str | None = None,
) -> str:
"""Create a fake prebuilt install tree and return the llama-server path.
``asset`` is the bundle filename recorded in the marker; omit it to model an
- older marker that predates asset-based ROCm forwarding (backward compat)."""
+ older marker that predates asset-based ROCm forwarding (backward compat).
+ ``release_tag`` is the full release tag (e.g. a ``b9596-mix-`` mix
+ build); defaults to ``tag`` for a plain prebuilt."""
bin_dir = dir_ / "build" / "bin"
bin_dir.mkdir(parents = True, exist_ok = True)
binary = bin_dir / "llama-server"
binary.write_text("#!/bin/sh\necho stub\n")
marker = {
"tag": tag,
- "release_tag": tag,
+ "release_tag": release_tag or tag,
"published_repo": repo,
"installed_at_utc": "2020-01-01T00:00:00Z",
"bundle_profile": "cuda13-newer",
@@ -106,10 +109,13 @@ def _write_install(
@pytest.fixture(autouse = True)
-def _clean_state(monkeypatch):
+def _clean_state(monkeypatch, tmp_path):
freshness.reset_caches()
upd._reset_job_for_tests()
upd._resolve_memo.clear()
+ # Isolate the freshness disk cache so the suite never writes the real
+ # ~/.unsloth cache (the default when storage_roots can't be imported).
+ monkeypatch.setattr(freshness, "_cache_dir", lambda: tmp_path / ".freshness_cache")
# Deterministic markerless paths: no host-pinned binary, no custom dir.
monkeypatch.delenv("LLAMA_SERVER_PATH", raising = False)
monkeypatch.delenv("UNSLOTH_LLAMA_CPP_PATH", raising = False)
@@ -395,6 +401,7 @@ def test_start_update_installer_failure_reports_error(monkeypatch, tmp_path):
binary = _write_install(install_dir, "b9493")
monkeypatch.setattr(upd, "_find_binary", lambda: binary)
monkeypatch.setattr(upd, "_installer_script", lambda: tmp_path / "install_llama_prebuilt.py")
+ monkeypatch.setattr(freshness, "_fetch_latest_release_tag", lambda repo, timeout = 5.0: "b9518")
_patch_installer_popen(monkeypatch, returncode = 2, lines = ["boom: network error\n"])
@@ -617,6 +624,7 @@ def test_update_clears_maintenance_flag_on_installer_failure(monkeypatch, tmp_pa
binary = _write_install(install_dir, "b9493")
monkeypatch.setattr(upd, "_find_binary", lambda: binary)
monkeypatch.setattr(upd, "_installer_script", lambda: tmp_path / "install_llama_prebuilt.py")
+ monkeypatch.setattr(freshness, "_fetch_latest_release_tag", lambda repo, timeout = 5.0: "b9518")
backend = _FakeBackend()
_inject_backend(monkeypatch, backend)
@@ -795,3 +803,41 @@ def test_start_update_source_build_refuses_when_newer(monkeypatch, tmp_path):
res = upd.start_update()
assert res["started"] is False
assert res["reason"] == "up_to_date"
+
+
+# --- mix-tag detection + apply guard (the reported banner bug) ---
+
+
+def test_status_not_offered_on_mix_latest(monkeypatch, tmp_path):
+ # Installed the mix latest; GitHub latest is that same full tag -> no banner.
+ binary = _write_install(tmp_path / "llama.cpp", "b9596", release_tag = "b9596-mix-e6f2453")
+ monkeypatch.setattr(upd, "_find_binary", lambda: binary)
+ monkeypatch.setattr(
+ freshness, "_fetch_latest_release_tag", lambda repo, timeout = 5.0: "b9596-mix-e6f2453"
+ )
+ st = upd.get_update_status()
+ assert st["update_available"] is False
+ assert st["installed_tag"] == "b9596"
+ assert st["latest_tag"] == "b9596-mix-e6f2453"
+
+
+def test_status_not_offered_when_latest_lags(monkeypatch, tmp_path):
+ # A lagging latest (older build than installed) must never be offered.
+ binary = _write_install(tmp_path / "llama.cpp", "b9585")
+ monkeypatch.setattr(upd, "_find_binary", lambda: binary)
+ monkeypatch.setattr(freshness, "_fetch_latest_release_tag", lambda repo, timeout = 5.0: "b9518")
+ st = upd.get_update_status()
+ assert st["update_available"] is False
+
+
+def test_start_update_marked_refuses_when_not_behind(monkeypatch, tmp_path):
+ # A direct POST / stale banner must not reinstall when already on the latest.
+ binary = _write_install(tmp_path / "llama.cpp", "b9596", release_tag = "b9596-mix-e6f2453")
+ monkeypatch.setattr(upd, "_find_binary", lambda: binary)
+ monkeypatch.setattr(upd, "_installer_script", lambda: tmp_path / "install_llama_prebuilt.py")
+ monkeypatch.setattr(
+ freshness, "_fetch_latest_release_tag", lambda repo, timeout = 5.0: "b9596-mix-e6f2453"
+ )
+ res = upd.start_update()
+ assert res["started"] is False
+ assert res["reason"] == "up_to_date"
diff --git a/studio/backend/utils/llama_cpp_freshness.py b/studio/backend/utils/llama_cpp_freshness.py
index 3e5066ca2d..f5fd745334 100644
--- a/studio/backend/utils/llama_cpp_freshness.py
+++ b/studio/backend/utils/llama_cpp_freshness.py
@@ -13,6 +13,7 @@ from __future__ import annotations
import json
import os
+import re
import time
from datetime import datetime, timezone
from pathlib import Path
@@ -104,11 +105,18 @@ def _save_disk_cache(repo: str, latest_tag: Optional[str]) -> None:
def _fetch_latest_release_tag(repo: str, timeout: float = 5.0) -> Optional[str]:
- """GitHub API call. None on any failure (offline, rate-limited, etc)."""
+ """Newest published release tag for `repo`, by publish time.
+
+ Resolves "latest" the way install_llama_prebuilt.py does (newest
+ non-draft/non-prerelease by ``published_at``), NOT via GitHub's
+ ``/releases/latest`` pointer. That pointer sorts by commit date and can lag
+ behind the build the installer actually installs, so detection and apply
+ disagreed -- the cause of the downgrade/sticky banner. None on any failure
+ (offline, rate-limited, etc)."""
import urllib.error
import urllib.request
- url = f"https://api.github.com/repos/{repo}/releases/latest"
+ url = f"https://api.github.com/repos/{repo}/releases?per_page=30"
headers = {
"Accept": "application/vnd.github+json",
"User-Agent": "unsloth-studio-freshness-check",
@@ -128,8 +136,21 @@ def _fetch_latest_release_tag(repo: str, timeout: float = 5.0) -> Optional[str]:
) as exc:
logger.debug("freshness fetch failed", repo = repo, error = str(exc))
return None
- tag = data.get("tag_name")
- return tag if isinstance(tag, str) and tag else None
+ if not isinstance(data, list):
+ return None
+ published = [
+ r
+ for r in data
+ if isinstance(r, dict)
+ and not r.get("draft")
+ and not r.get("prerelease")
+ and isinstance(r.get("tag_name"), str)
+ and r.get("tag_name")
+ ]
+ if not published:
+ return None
+ newest = max(published, key = lambda r: r.get("published_at") or "")
+ return newest["tag_name"]
def latest_published_release(repo: str, *, force_refresh: bool = False) -> Optional[str]:
@@ -172,19 +193,58 @@ def _parse_installed_at(value: object) -> Optional[datetime]:
return dt
+def parse_base_build(tag: object) -> Optional[int]:
+ """Numeric base build from a release tag. Handles both a plain ``bNNNN`` and
+ a mix-build tag like ``b9596-mix-`` (anchored at the start, so the mix
+ suffix doesn't defeat it). None for anything not starting with ``bNNNN``."""
+ if not isinstance(tag, str):
+ return None
+ m = re.match(r"b(\d+)", tag.strip())
+ return int(m.group(1)) if m else None
+
+
+def is_behind(installed: Optional[str], latest: Optional[str]) -> bool:
+ """Whether `installed` is genuinely behind `latest`, comparing the FULL
+ release identity (so a mix build can legitimately be the latest) with a
+ base-build guard so a lagging GitHub /releases/latest can never read as an
+ update or a downgrade.
+
+ - identical tags -> not behind (clears the sticky banner post-update)
+ - higher base build on `latest` -> behind; lower -> NOT behind (downgrade guard)
+ - same base build: a different/new mix -> behind, but a bare ``bNNNN`` never
+ supersedes a mix build (extra PRs) at that base -> not behind
+ - non-bNNNN tags -> behind (plain inequality, since they already differ)
+ """
+ if not installed or not latest:
+ return False
+ installed, latest = installed.strip(), latest.strip()
+ if installed == latest:
+ return False
+ ib, lb = parse_base_build(installed), parse_base_build(latest)
+ if ib is None or lb is None:
+ return True
+ if lb != ib:
+ return lb > ib
+ # Same base build, different tags: offer a mix (latest carries a suffix), but
+ # never offer a bare base over a mix install at the same base.
+ return latest != f"b{lb}"
+
+
def check_prebuilt_freshness(
binary_path: Optional[str],
*,
threshold_days: int = STALENESS_THRESHOLD_DAYS,
now: Optional[datetime] = None,
) -> dict:
- """Returns {has_marker, stale, installed_tag, latest_tag,
+ """Returns {has_marker, stale, behind, installed_tag, latest_tag,
installed_at_utc, age_days, published_repo, threshold_days}.
- stale = True iff installed != latest AND age >= threshold.
- Fails open on missing data (stale stays False)."""
+ behind = installed genuinely older than latest (see is_behind).
+ stale = behind AND age >= threshold.
+ Fails open on missing data (behind/stale stay False)."""
out: dict = {
"has_marker": False,
"stale": False,
+ "behind": False,
"installed_tag": None,
"latest_tag": None,
"installed_at_utc": None,
@@ -196,16 +256,25 @@ def check_prebuilt_freshness(
if not marker:
return out
out["has_marker"] = True
+ # Display prefers the normalized base ("tag"); comparison below prefers the
+ # full "release_tag" -- deliberately opposite fallbacks.
out["installed_tag"] = marker.get("tag") or marker.get("release_tag")
out["installed_at_utc"] = marker.get("installed_at_utc")
out["published_repo"] = marker.get("published_repo")
+ # The marker records both a normalized base tag ("tag", e.g. b9596) and the
+ # full release tag ("release_tag", e.g. b9596-mix-). Compare against the
+ # FULL identity, since GitHub /releases/latest returns the full tag_name --
+ # comparing the normalized base against the full latest is what produced the
+ # permanent "downgrade" banner on every mix release.
+ installed_full = marker.get("release_tag") or marker.get("tag")
repo = out["published_repo"]
- if not repo or not out["installed_tag"]:
+ if not repo or not installed_full:
return out
latest = latest_published_release(repo)
out["latest_tag"] = latest
- if not latest or latest == out["installed_tag"]:
+ out["behind"] = is_behind(installed_full, latest)
+ if not out["behind"]:
return out
installed_at = _parse_installed_at(out["installed_at_utc"])
diff --git a/studio/backend/utils/llama_cpp_update.py b/studio/backend/utils/llama_cpp_update.py
index 55bf4b3d84..654ade6cd4 100644
--- a/studio/backend/utils/llama_cpp_update.py
+++ b/studio/backend/utils/llama_cpp_update.py
@@ -286,9 +286,10 @@ def get_update_status(*, force_refresh: bool = False) -> dict:
freshness = check_prebuilt_freshness(binary)
installed = freshness.get("installed_tag")
latest = freshness.get("latest_tag")
- update_available = bool(
- freshness.get("has_marker") and installed and latest and installed != latest
- )
+ # `behind` compares the full release identity with a base-build guard, so a
+ # lagging /releases/latest or a mix-tagged latest can't show a false update
+ # (see llama_cpp_freshness.is_behind).
+ update_available = bool(freshness.get("has_marker") and freshness.get("behind"))
with _job_lock:
job = dict(_job)
@@ -405,9 +406,14 @@ def _run_update(install_dir: Path, repo: str, asset: Optional[str], script: Path
tail = "".join(tail_lines).strip()[-1500:]
raise RuntimeError(f"installer exited {returncode}: {tail or 'no output'}")
- # New UNSLOTH_PREBUILT_INFO.json is on disk; drop caches so the next
- # status read reflects the freshly installed tag.
+ # New UNSLOTH_PREBUILT_INFO.json is on disk; drop in-memory caches and
+ # re-prime the 24h disk freshness cache with the true newest, so the
+ # banner can't linger on a stale same-base value after the swap.
reset_caches()
+ try:
+ latest_published_release(repo, force_refresh = True)
+ except Exception as exc: # pragma: no cover - network defensive
+ logger.debug("llama update: post-install freshness refresh failed", error = str(exc))
new_marker = read_install_marker(_find_binary())
new_tag = (new_marker or {}).get("tag") or (new_marker or {}).get("release_tag")
@@ -456,7 +462,24 @@ def start_update() -> dict:
"job": get_update_status()["job"],
}
+ # A job already in flight wins over any freshness re-check below (and skips
+ # its network call). The final lock block re-checks to close the TOCTOU.
+ with _job_lock:
+ if _job["state"] == _JOB_RUNNING:
+ return {"started": False, "reason": "already_running", "job": dict(_job)}
+
if marker:
+ # Mirror the detection guard: a direct POST or a stale banner must not
+ # start an install when the latest is not actually newer (force a fresh
+ # check so a stale 24h cache can't wrongly block a real update either).
+ status = get_update_status(force_refresh = True)
+ if not status.get("update_available"):
+ return {
+ "started": False,
+ "reason": "up_to_date",
+ "message": "The installed llama.cpp build is already at the latest prebuilt.",
+ "job": status["job"],
+ }
install_dir = _install_dir_for(binary)
repo = marker.get("published_repo") or DEFAULT_PUBLISHED_REPO
from_tag = marker.get("tag") or marker.get("release_tag")
From 0793775c01c24099cd4f31432d89ba0e9635931c Mon Sep 17 00:00:00 2001
From: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
Date: Thu, 11 Jun 2026 20:07:37 -0700
Subject: [PATCH 05/50] Fix kwarg spacing in training files to satisfy
pre-commit (#6209)
The ruff-format-with-kwargs hook reformats these three files, so
pre-commit.ci fails on every PR. Formatting only, no behavior change.
From 11d5f64eeb81a3dacd5a4c0c8eb1b45268be00e1 Mon Sep 17 00:00:00 2001
From: Daniel Han
Date: Thu, 11 Jun 2026 20:07:51 -0700
Subject: [PATCH 06/50] Studio: reword the Cloudflare line when the public
probe fails (#6217)
On a 0.0.0.0 bind whose public ip:port is not reachable (cloud firewall),
the banner still printed "Secure link access via Cloudflare: " right
after "is NOT reachable from the public internet", which reads as if the
tunnel might also be blocked. The Cloudflare quick-tunnel works regardless.
Thread the reachability probe result through a module-level _public_reachable
tri-state and, when the public probe definitively failed but the tunnel is
up, print "Also, the secure link access via Cloudflare works: ".
Reachable or undecided cases keep the existing wording.
---
studio/backend/run.py | 20 ++++++-
.../backend/tests/test_cloudflare_tunnel.py | 54 +++++++++++++++++++
2 files changed, 72 insertions(+), 2 deletions(-)
diff --git a/studio/backend/run.py b/studio/backend/run.py
index 32dfe06a18..a883154c4a 100644
--- a/studio/backend/run.py
+++ b/studio/backend/run.py
@@ -232,6 +232,9 @@ def _verify_global_reachability(display_host: str, port: int) -> None:
public internet. Synchronous so output lands between the banner URLs and the
stop hint. Bounded at ~15s; failures swallowed (verifier failing != Studio
failing). Only meaningful for a wildcard bind."""
+ global _public_reachable
+ # Reset to "unknown" each run; set True/False only when the probe decides.
+ _public_reachable = None
import ipaddress
import json
import time
@@ -324,12 +327,14 @@ def _verify_global_reachability(display_host: str, port: int) -> None:
print("", flush = True)
if ok_nodes:
+ _public_reachable = True
print(
f"{ok_c} Reachability check: {url}/ is reachable from the "
f"public internet ({ok_nodes}/{total} probe nodes connected).{reset}",
flush = True,
)
elif err_nodes:
+ _public_reachable = False
print(
f"{err_c} Reachability check: {url}/ is NOT reachable from "
f"the public internet ({err_nodes}/{total} probe nodes failed).{reset}",
@@ -422,7 +427,9 @@ def _print_cloudflare_line() -> None:
"""Print the Cloudflare quick-tunnel URL for 0.0.0.0 binds, if one is up.
Reads the module-level URL set by ``run_server``. Prints nothing when the
- tunnel is disabled or failed -- failures are silently ignored.
+ tunnel is disabled or failed -- failures are silently ignored. When the public
+ reachability probe just failed (``_public_reachable is False``) but the tunnel
+ is up, reword to point the user at the Cloudflare link as the way in.
"""
if not _cloudflare_url:
return
@@ -430,7 +437,10 @@ def _print_cloudflare_line() -> None:
accent = "\033[38;5;150;1m"
reset = "\033[0m"
- line = f" Secure link access via Cloudflare: {_cloudflare_url}"
+ if _public_reachable is False:
+ line = f" Use the secure link access via Cloudflare instead: {_cloudflare_url}"
+ else:
+ line = f" Secure link access via Cloudflare: {_cloudflare_url}"
print(f"{accent}{line}{reset}" if stdout_supports_color() else line)
@@ -622,6 +632,12 @@ _shutdown_event = None
# None when there is no tunnel (loopback, disabled, or a silently-ignored failure).
_cloudflare_url = None
+# Public reachability from the last _verify_global_reachability run, read by the
+# Cloudflare banner line. True when the public ip:port probe confirmed reachable,
+# False when it confirmed NOT reachable, None when the probe did not run or could
+# not decide (timeout, blocked, private address).
+_public_reachable = None
+
_DEFAULT_FRONTEND_PATH = Path(__file__).resolve().parent.parent / "frontend" / "dist"
diff --git a/studio/backend/tests/test_cloudflare_tunnel.py b/studio/backend/tests/test_cloudflare_tunnel.py
index 8208f7f83f..873547631d 100644
--- a/studio/backend/tests/test_cloudflare_tunnel.py
+++ b/studio/backend/tests/test_cloudflare_tunnel.py
@@ -13,6 +13,7 @@ import importlib.util
import io
import sys
import tarfile
+import types
from pathlib import Path
import pytest
@@ -423,3 +424,56 @@ def test_run_server_gates_tunnel_on_wildcard():
source = _RUN_PY.read_text()
assert "_cloudflare_enabled" in source
assert 'host == "0.0.0.0"' in source
+
+
+def _run_print_cloudflare_line(monkeypatch, *, cloudflare_url, public_reachable):
+ """Exec the real _print_cloudflare_line source in isolation (run.py has heavy
+ deps), with the two module globals injected and startup_banner stubbed."""
+ src = _RUN_PY.read_text()
+ tree = ast.parse(src)
+ func_src = next(
+ ast.get_source_segment(src, n)
+ for n in ast.walk(tree)
+ if isinstance(n, ast.FunctionDef) and n.name == "_print_cloudflare_line"
+ )
+ stub = types.ModuleType("startup_banner")
+ stub.stdout_supports_color = lambda: False
+ monkeypatch.setitem(sys.modules, "startup_banner", stub)
+ captured: list[str] = []
+ ns = {
+ "_cloudflare_url": cloudflare_url,
+ "_public_reachable": public_reachable,
+ "print": lambda *a, **k: captured.append(" ".join(str(x) for x in a)),
+ }
+ exec(compile(func_src, "", "exec"), ns)
+ ns["_print_cloudflare_line"]()
+ return "\n".join(captured)
+
+
+def test_cloudflare_line_reworded_when_public_unreachable(monkeypatch):
+ out = _run_print_cloudflare_line(
+ monkeypatch, cloudflare_url = "https://x.trycloudflare.com", public_reachable = False
+ )
+ assert "Use the secure link access via Cloudflare instead: https://x.trycloudflare.com" in out
+
+
+def test_cloudflare_line_default_wording_when_reachable(monkeypatch):
+ out = _run_print_cloudflare_line(
+ monkeypatch, cloudflare_url = "https://x.trycloudflare.com", public_reachable = True
+ )
+ assert "Secure link access via Cloudflare: https://x.trycloudflare.com" in out
+ assert "Use the secure link" not in out
+
+
+def test_cloudflare_line_default_wording_when_unknown(monkeypatch):
+ # Probe did not run / could not decide -> keep the existing wording.
+ out = _run_print_cloudflare_line(
+ monkeypatch, cloudflare_url = "https://x.trycloudflare.com", public_reachable = None
+ )
+ assert "Secure link access via Cloudflare: https://x.trycloudflare.com" in out
+ assert "Use the secure link" not in out
+
+
+def test_cloudflare_line_prints_nothing_without_tunnel(monkeypatch):
+ out = _run_print_cloudflare_line(monkeypatch, cloudflare_url = None, public_reachable = False)
+ assert out == ""
From 84b42c92830bef8914f2bb12fe99cba208addb23 Mon Sep 17 00:00:00 2001
From: Leo Borcherding
Date: Thu, 11 Jun 2026 22:14:22 -0500
Subject: [PATCH 07/50] fix: deduplicate lemonade ROCm prebuilt selection log
(#6021)
* fix: deduplicate lemonade ROCm prebuilt selection log
resolve_lemonade_rocm_choice() is called twice per install (direct
planner + resolve_upstream_asset_choice). The API fetch is already
memoised via _fetch_lemonade_release_cached but the selection log
lines were still emitted on both calls, printing the 'trying
lemonade-sdk ROCm prebuilt' banner and hash-manifest NOTE twice.
Add _lemonade_selection_logged set keyed on (gfx_target, asset_name)
and guard the two log() calls behind a membership check so they print
exactly once per process regardless of call count.
Also extend the _clear_lemonade_release_cache test fixture to clear
the new set between tests to prevent cross-test state bleed.
Fixes #6020
* fix: write log() output to stdout to avoid PowerShell NativeCommandError
On Windows, PowerShell treats any stderr output from a native process as
an error record and prefixes it with 'python.exe :' and sets the
ErrorId to NativeCommandError. Since log() wrote to sys.stderr, every
[llama-prebuilt] status line triggered this, making normal progress
output look like errors in the installer console.
Switch log() to sys.stdout. The download progress bar (DownloadProgress)
retains its stderr/tty logic unchanged -- that path is for interactive
terminal rendering, not status logging.
* fix: remove redundant 'or ""' in lemonade log_key
host.rocm_gfx_target is already guaranteed truthy by the early
return at the top of resolve_lemonade_rocm_choice. The fallback
was dead code.
* Keep resolver stdout machine-readable, route install logs to stdout
log() sending everything to stdout breaks the resolver modes: setup.sh
json.load()s the whole stdout, so one helper log line (network retry,
release-tag scan) corrupts the parse and silently drops back to building
"latest". Default log() to stderr and flip to stdout only on the install
path, where PowerShell otherwise renders stderr as NativeCommandError
noise. Also tighten the lemonade dedup comments.
---------
Co-authored-by: Daniel Han
---
.../test_lemonade_llamacpp_rocm_bins_mock.py | 9 +++-
studio/install_llama_prebuilt.py | 45 +++++++++++++------
2 files changed, 39 insertions(+), 15 deletions(-)
diff --git a/studio/backend/tests/test_lemonade_llamacpp_rocm_bins_mock.py b/studio/backend/tests/test_lemonade_llamacpp_rocm_bins_mock.py
index 57bbc84fb0..7c6c514b8f 100644
--- a/studio/backend/tests/test_lemonade_llamacpp_rocm_bins_mock.py
+++ b/studio/backend/tests/test_lemonade_llamacpp_rocm_bins_mock.py
@@ -32,14 +32,19 @@ if resolve_lemonade_rocm_choice is None or _LEMONADE_GFX_FAMILIES is None:
@pytest.fixture(autouse = True)
def _clear_lemonade_release_cache():
- """Prevent cross-test pollution of the lemonade release lru_cache when
- future tests vary the fetch_json mock return value."""
+ """Prevent cross-test pollution of the lemonade release lru_cache and
+ selection-log dedup set when tests vary the fetch_json mock return value."""
_cache = getattr(_mod, "_fetch_lemonade_release_cached", None)
+ _logged: set | None = getattr(_mod, "_lemonade_selection_logged", None)
if _cache is not None and hasattr(_cache, "cache_clear"):
_cache.cache_clear()
+ if _logged is not None:
+ _logged.clear()
yield
if _cache is not None and hasattr(_cache, "cache_clear"):
_cache.cache_clear()
+ if _logged is not None:
+ _logged.clear()
_STUB_TAG = "b1262"
diff --git a/studio/install_llama_prebuilt.py b/studio/install_llama_prebuilt.py
index bd8c90dc9c..bed7a27a63 100644
--- a/studio/install_llama_prebuilt.py
+++ b/studio/install_llama_prebuilt.py
@@ -460,8 +460,14 @@ def is_busy_lock_error(exc: BaseException) -> bool:
return False
+# Status logs default to stderr so resolver modes keep stdout machine-readable
+# (setup.sh json.load()s the whole stdout). main() flips this for the install
+# path, where PowerShell otherwise renders stderr as NativeCommandError noise.
+_LOG_TO_STDOUT = False
+
+
def log(message: str) -> None:
- print(f"[llama-prebuilt] {message}", file = sys.stderr)
+ print(f"[llama-prebuilt] {message}", file = sys.stdout if _LOG_TO_STDOUT else sys.stderr)
def log_lines(lines: Iterable[str]) -> None:
@@ -3928,6 +3934,12 @@ def _is_trusted_github_release_url(url: str, expected_repo: str) -> bool:
return False
+# (gfx_target, asset_name) pairs already logged. resolve_lemonade_rocm_choice()
+# runs twice per install (direct planner + resolve_upstream_asset_choice), so
+# this stops its selection banner and hash-manifest NOTE printing twice.
+_lemonade_selection_logged: "set[tuple[str, str]]" = set()
+
+
@functools.lru_cache(maxsize = 8)
def _fetch_lemonade_release_cached(api_url: str, llama_tag: str) -> "dict | None":
"""Cached wrapper around fetch_json for lemonade release lookups.
@@ -4034,18 +4046,22 @@ def resolve_lemonade_rocm_choice(
# Note: lemonade tags Linux assets with "ubuntu" but the binary is a
# generic glibc build that runs on any distro (Arch, Fedora, ...), so
# this attempt is selected for all Linux ROCm hosts, not just Ubuntu.
- log(
- f"AMD GPU {host.rocm_gfx_target!r} ({gfx_family}) -- "
- f"trying lemonade-sdk ROCm prebuilt {asset_name} "
- f"(works on any glibc Linux, not just Ubuntu)"
- )
- log(
- f"NOTE: lemonade-sdk/llamacpp-rocm releases are not covered by the "
- f"Unsloth approved-hash manifest; download integrity relies on "
- f"functional validation (llama-bench / llama-server smoke tests) "
- f"after extraction. Set UNSLOTH_DISABLE_LEMONADE_ROCM=1 to skip "
- f"lemonade and fall back to the upstream HIP build path."
- )
+ # Log once per (gfx_target, asset); see _lemonade_selection_logged.
+ log_key = (host.rocm_gfx_target, asset_name)
+ if log_key not in _lemonade_selection_logged:
+ _lemonade_selection_logged.add(log_key)
+ log(
+ f"AMD GPU {host.rocm_gfx_target!r} ({gfx_family}) -- "
+ f"trying lemonade-sdk ROCm prebuilt {asset_name} "
+ f"(works on any glibc Linux, not just Ubuntu)"
+ )
+ log(
+ f"NOTE: lemonade-sdk/llamacpp-rocm releases are not covered by the "
+ f"Unsloth approved-hash manifest; download integrity relies on "
+ f"functional validation (llama-bench / llama-server smoke tests) "
+ f"after extraction. Set UNSLOTH_DISABLE_LEMONADE_ROCM=1 to skip "
+ f"lemonade and fall back to the upstream HIP build path."
+ )
return AssetChoice(
repo = LEMONADE_ROCM_REPO,
tag = release_tag,
@@ -6963,6 +6979,9 @@ def main() -> int:
raise SystemExit(
"install_llama_prebuilt.py: --install-dir is required unless --resolve-llama-tag, --resolve-install-tag, or --resolve-source-build is used"
)
+ # Install path only: route status logs to stdout (see _LOG_TO_STDOUT note).
+ global _LOG_TO_STDOUT
+ _LOG_TO_STDOUT = True
install_prebuilt(
install_dir = Path(args.install_dir).expanduser().resolve(),
llama_tag = args.llama_tag,
From 6dae2f525b5218143f85cc8f401313e182bccac3 Mon Sep 17 00:00:00 2001
From: Daniel Han
Date: Thu, 11 Jun 2026 20:37:01 -0700
Subject: [PATCH 08/50] Stop false RoPE 'default' warning and fix rope drift
gate on transformers 5 (#6223)
* Handle rope_type 'default' on transformers 5 to stop false RoPE warning
transformers 5 reports rope_type="default" for every plain (unscaled) config
and dropped "default" from ROPE_INIT_FUNCTIONS. _compute_config_rope_inv_freq
then did ROPE_INIT_FUNCTIONS["default"], hit KeyError, returned None and logged
"Could not apply RoPE scaling 'default'; long-context generation may degrade"
on every model load. The inv_freq was still correct (the constructor recomputes
vanilla on None), but the warning is a false alarm for unscaled models.
Compute the unscaled inv_freq directly for rope_type "default"/None instead of
going through ROPE_INIT_FUNCTIONS, so plain configs return the right value with
no warning. Scaled types (llama3/linear/yarn/...) are unchanged.
Also skip test_object_style_rope_scaling_on_config_delegates_correctly when
transformers strict-validates rope_scaling (5.x): it rejects a non-dict object
on config.rope_scaling, so the object-style delegation path cannot be set up
there. The test still runs and asserts on transformers <5.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
---
tests/utils/test_rope_scaling_drift.py | 9 ++++++++-
unsloth/models/llama.py | 14 ++++++++++++++
2 files changed, 22 insertions(+), 1 deletion(-)
diff --git a/tests/utils/test_rope_scaling_drift.py b/tests/utils/test_rope_scaling_drift.py
index fae5a7d8a8..193cb830f5 100644
--- a/tests/utils/test_rope_scaling_drift.py
+++ b/tests/utils/test_rope_scaling_drift.py
@@ -300,7 +300,14 @@ def test_object_style_rope_scaling_on_config_delegates_correctly():
expected = _reference_inv_freq(dict_config, "linear")
object_config = _make_config({"rope_type": "linear", "factor": 4.0})
- object_config.rope_scaling = FakeLinearRopeScalingConfig()
+ try:
+ object_config.rope_scaling = FakeLinearRopeScalingConfig()
+ except Exception:
+ pytest.skip(
+ "transformers strict-validates rope_scaling to dict/RopeParameters/None, "
+ "so object-style config.rope_scaling (and the delegation retry it "
+ "exercises) is unreachable on this version."
+ )
inv_freq, attention_scaling = _compute_config_rope_inv_freq(
object_config, object_config.rope_scaling
)
diff --git a/unsloth/models/llama.py b/unsloth/models/llama.py
index 2d31f71ab1..a60edf3fbe 100644
--- a/unsloth/models/llama.py
+++ b/unsloth/models/llama.py
@@ -1673,12 +1673,26 @@ def _llama3_inv_freq_from_config(
return torch.where(is_medium, smoothed, scaled)
+def _vanilla_inv_freq_from_config(config, device = "cpu"):
+ """Unscaled RoPE inv_freq (rope_type 'default'/None), matching the constructor's fallback."""
+ base = _get_rope_theta(config, default = 10000.0)
+ dim = getattr(config, "head_dim", None)
+ if dim is None:
+ dim = int(config.hidden_size // config.num_attention_heads)
+ return 1.0 / (base ** (torch.arange(0, dim, 2, dtype = torch.int64, device = device).float() / dim))
+
+
def _compute_config_rope_inv_freq(config, rope_scaling):
"""(inv_freq, attention_scaling) per config.rope_scaling via transformers'
ROPE_INIT_FUNCTIONS, with an inline llama3 fallback; (None, 1.0) on failure."""
original_rope_scaling = rope_scaling
rope_scaling = _rope_scaling_as_dict(rope_scaling)
rope_type = rope_scaling.get("rope_type", None) or rope_scaling.get("type", None)
+ # "default"/unset means unscaled RoPE. transformers >=5 reports
+ # rope_type="default" for every plain config and dropped "default" from
+ # ROPE_INIT_FUNCTIONS, so compute it directly instead of warning per load.
+ if rope_type in (None, "default"):
+ return _vanilla_inv_freq_from_config(config).to(dtype = torch.float32, device = "cpu"), 1.0
try:
from transformers.modeling_rope_utils import ROPE_INIT_FUNCTIONS
From 3e6920627c3157e0967e29e41aa615fd793eb51c Mon Sep 17 00:00:00 2001
From: James Dawdy
Date: Fri, 12 Jun 2026 02:11:05 -0500
Subject: [PATCH 09/50] fix(studio): load run.py by path for editable installs
(#5909)
* fix(studio): load run.py by path for editable installs
`studio update` can leave a partial site-packages/studio/backend/ tree
(plugin build artefacts only). That shadowed tree wins over an editable
install and breaks `from studio.backend.run import ...`. Loading run.py
by file path via importlib sidesteps the conflict.
The module is cached in _RUN_MODULE so repeated calls are cheap.
If exec_module fails, the module is removed from sys.modules before
re-raising so a subsequent retry starts clean.
Co-Authored-By: Claude Sonnet 4.6
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Handle None __file__ when checking cached run module for PR #5909
* Harden _load_backend_auth_storage against None __file__ and resolve cache-key path (PR #5909)
* Adapt studio run/cloudflare in-venv tests to _load_run_module loader (PR #5909)
---------
Co-authored-by: Jim Dawdy
Co-authored-by: Claude Sonnet 4.6
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han
---
unsloth_cli/commands/studio.py | 81 ++++++++++++++-----
.../tests/test_studio_cloudflare_flag.py | 6 ++
.../tests/test_studio_run_parallel_flag.py | 4 +
3 files changed, 70 insertions(+), 21 deletions(-)
diff --git a/unsloth_cli/commands/studio.py b/unsloth_cli/commands/studio.py
index fb63fbf208..ffdf36bedf 100644
--- a/unsloth_cli/commands/studio.py
+++ b/unsloth_cli/commands/studio.py
@@ -184,6 +184,47 @@ def _find_run_py() -> Optional[Path]:
return None
+_RUN_MODULE = None
+
+
+def _load_run_module():
+ """Import studio.backend.run without relying on package resolution.
+
+ `studio update` can leave a partial ``site-packages/studio/backend/``
+ tree (plugin build artefacts only). That shadowed tree wins over an
+ editable install and breaks ``from studio.backend.run import ...``.
+ Loading by file path sidesteps the conflict.
+ """
+ global _RUN_MODULE
+ if _RUN_MODULE is not None:
+ return _RUN_MODULE
+
+ run_py = _find_run_py()
+ if run_py is None:
+ raise ImportError("Could not find studio/backend/run.py. Re-run: unsloth studio setup")
+
+ loaded = sys.modules.get("studio.backend.run")
+ if loaded is not None:
+ # __file__ can be None for namespace packages from partial trees.
+ loaded_path = Path(getattr(loaded, "__file__", None) or "").resolve()
+ if loaded_path == run_py.resolve():
+ _RUN_MODULE = loaded
+ return _RUN_MODULE
+
+ spec = importlib.util.spec_from_file_location("studio.backend.run", run_py)
+ if spec is None or spec.loader is None:
+ raise ImportError(f"Could not load studio backend from {run_py}")
+ module = importlib.util.module_from_spec(spec)
+ sys.modules["studio.backend.run"] = module
+ try:
+ spec.loader.exec_module(module)
+ except Exception:
+ sys.modules.pop("studio.backend.run", None)
+ raise
+ _RUN_MODULE = module
+ return _RUN_MODULE
+
+
def _find_setup_script() -> Optional[Path]:
"""Find studio/setup.sh or studio/setup.ps1.
@@ -329,9 +370,11 @@ def _load_backend_auth_storage():
auth_dir = backend_dir / "auth"
storage_py = auth_dir / "storage.py"
loaded = sys.modules.get("auth.storage")
- loaded_path = Path(getattr(loaded, "__file__", "")).resolve()
- if loaded is not None and loaded_path == storage_py:
- return loaded
+ if loaded is not None:
+ # __file__ can be None for namespace packages from partial trees.
+ loaded_path = Path(getattr(loaded, "__file__", None) or "").resolve()
+ if loaded_path == storage_py.resolve():
+ return loaded
package = sys.modules.get("auth")
package_paths = [Path(path).resolve() for path in getattr(package, "__path__", [])]
@@ -706,11 +749,11 @@ def studio_default(
typer.echo("Studio not set up. Run install.sh first.")
raise typer.Exit(1)
- from studio.backend.run import run_server
+ run_mod = _load_run_module()
+ run_server = run_mod.run_server
if not silent:
- from studio.backend.run import _resolve_external_ip
- display_host = _resolve_external_ip() if host == "0.0.0.0" else host
+ display_host = run_mod._resolve_external_ip() if host == "0.0.0.0" else host
typer.echo(f"Starting Unsloth Studio on http://{display_host}:{port}")
run_kwargs = dict(
@@ -725,20 +768,17 @@ def studio_default(
run_kwargs["frontend_path"] = frontend
run_server(**run_kwargs)
- from studio.backend.run import _shutdown_event
-
try:
- if _shutdown_event is not None:
+ if run_mod._shutdown_event is not None:
# Event.wait() with no timeout blocks at C-level on Linux
# and swallows SIGINT; loop with a 1s timeout instead.
- while not _shutdown_event.is_set():
- _shutdown_event.wait(timeout = 1)
+ while not run_mod._shutdown_event.is_set():
+ run_mod._shutdown_event.wait(timeout = 1)
else:
while True:
time.sleep(1)
except KeyboardInterrupt:
- from studio.backend.run import _graceful_shutdown, _server
- _graceful_shutdown(_server)
+ run_mod._graceful_shutdown(run_mod._server)
typer.echo("\nShutting down...")
@@ -1036,7 +1076,8 @@ def run(
os.execvp(str(studio_bin), args)
# ── 2. Start server (always suppress built-in banner) ─────────────
- from studio.backend.run import run_server, _resolve_external_ip
+ run_mod = _load_run_module()
+ run_server = run_mod.run_server
run_kwargs = dict(
host = host,
@@ -1099,7 +1140,7 @@ def run(
context_length_line = _format_context_length_line(result)
# 6. Print banner.
- display_host = _resolve_external_ip() if host == "0.0.0.0" else host
+ display_host = run_mod._resolve_external_ip() if host == "0.0.0.0" else host
base_url = f"http://{display_host}:{actual_port}"
sdk_base_url = f"{base_url}/v1"
# run_server started the tunnel during the silent run above (0.0.0.0 only).
@@ -1178,17 +1219,15 @@ def run(
typer.secho(_tool_notice, fg = _tool_notice_fg, bold = True)
# 7. Wait for Ctrl+C.
- from studio.backend.run import _shutdown_event, _graceful_shutdown, _server
-
try:
- if _shutdown_event is not None:
- while not _shutdown_event.is_set():
- _shutdown_event.wait(timeout = 1)
+ if run_mod._shutdown_event is not None:
+ while not run_mod._shutdown_event.is_set():
+ run_mod._shutdown_event.wait(timeout = 1)
else:
while True:
time.sleep(1)
except KeyboardInterrupt:
- _graceful_shutdown(_server)
+ run_mod._graceful_shutdown(run_mod._server)
typer.echo("\nShutting down...")
diff --git a/unsloth_cli/tests/test_studio_cloudflare_flag.py b/unsloth_cli/tests/test_studio_cloudflare_flag.py
index 5c2a039547..6ab1ce21ff 100644
--- a/unsloth_cli/tests/test_studio_cloudflare_flag.py
+++ b/unsloth_cli/tests/test_studio_cloudflare_flag.py
@@ -210,6 +210,9 @@ def test_run_in_venv_passes_cloudflare_to_run_server(monkeypatch, user_flag, exp
)
fake_backend_run.run_server = fake_run_server
fake_backend_run._resolve_external_ip = lambda: "127.0.0.1"
+ # run() loads the backend via _load_run_module() (by file path); inject the
+ # mock as the cached run module so the stubbed run_server is used.
+ monkeypatch.setattr(studio_mod, "_RUN_MODULE", fake_backend_run)
import typer as _typer
@@ -270,6 +273,9 @@ def test_run_in_venv_shuts_down_on_startup_abort(monkeypatch):
backend._server = object()
backend._shutdown_event = None
backend._graceful_shutdown = lambda server: shutdown_calls.append(server)
+ # run() loads the backend via _load_run_module() (by file path); inject the
+ # mock as the cached run module so the stubbed symbols are used.
+ monkeypatch.setattr(studio_mod, "_RUN_MODULE", backend)
# set_tool_policy is imported as `from state.tool_policy import set_tool_policy`.
state_mod = sys.modules.setdefault("state", types.ModuleType("state"))
diff --git a/unsloth_cli/tests/test_studio_run_parallel_flag.py b/unsloth_cli/tests/test_studio_run_parallel_flag.py
index 8870fa348d..8caf31a432 100644
--- a/unsloth_cli/tests/test_studio_run_parallel_flag.py
+++ b/unsloth_cli/tests/test_studio_run_parallel_flag.py
@@ -426,6 +426,10 @@ def test_in_venv_path_passes_parallel_to_run_server(monkeypatch, value):
)
fake_backend_run.run_server = fake_run_server
fake_backend_run._resolve_external_ip = lambda: "127.0.0.1"
+ # run() loads the backend via _load_run_module() (by file path), which
+ # ignores a sys.modules mock with no matching __file__; inject it as the
+ # cached run module so the stubbed run_server is used.
+ monkeypatch.setattr(studio_mod, "_RUN_MODULE", fake_backend_run)
import typer as _typer
From f22e890ab81ebe3fd22505c954fe1df7c61e152d Mon Sep 17 00:00:00 2001
From: James Dawdy
Date: Fri, 12 Jun 2026 02:27:04 -0500
Subject: [PATCH 10/50] fix(studio): inherit llama_extra_args and honor
--no-mmproj (#5902)
* fix(studio): inherit llama_extra_args and honor --no-mmproj
Reloading the same GGUF from the UI without gguf_variant no longer drops
CLI pass-through args like --no-mmproj. Skip mmproj download and launch
when --no-mmproj is present in llama_extra_args.
Co-authored-by: Cursor
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio): tighten GGUF llama_extra_args variant inheritance guard
Reject inherited CLI args when the request changes gguf_variant or when
omitted variant resolves differently from the stored extra_args source.
Co-authored-by: Cursor
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Treat --no-mmproj-auto and --mmproj-auto with last-wins parsing for PR #5902
---------
Co-authored-by: Cursor
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han
---
studio/backend/core/inference/llama_cpp.py | 15 ++++++-----
.../core/inference/llama_server_args.py | 22 +++++++++++++++
studio/backend/routes/inference.py | 27 +++++++++++--------
.../backend/tests/test_llama_server_args.py | 17 ++++++++++++
4 files changed, 64 insertions(+), 17 deletions(-)
diff --git a/studio/backend/core/inference/llama_cpp.py b/studio/backend/core/inference/llama_cpp.py
index b83e4e6961..80b4862aa8 100644
--- a/studio/backend/core/inference/llama_cpp.py
+++ b/studio/backend/core/inference/llama_cpp.py
@@ -28,6 +28,7 @@ from typing import Callable, Generator, Iterable, List, Optional
import httpx
from core.inference.llama_server_args import (
+ extra_args_disable_mmproj,
parse_cache_override,
parse_ctx_override,
resolve_cache_type_kv,
@@ -2929,8 +2930,8 @@ class LlamaCppBackend:
hf_variant = hf_variant,
hf_token = hf_token,
)
- # Auto-download mmproj for vision models
- if is_vision and not mmproj_path:
+ # Auto-download mmproj for vision models unless opted out.
+ if is_vision and not mmproj_path and not extra_args_disable_mmproj(extra_args):
mmproj_path = self._download_mmproj(
hf_repo = hf_repo,
hf_token = hf_token,
@@ -3191,10 +3192,12 @@ class LlamaCppBackend:
gpu_indices, use_fit = None, True
effective_ctx = requested_ctx # fall back to original
- launch_mmproj_path = self._resolve_launch_mmproj_path(
- model_path = model_path,
- mmproj_path = mmproj_path,
- )
+ launch_mmproj_path = None
+ if not extra_args_disable_mmproj(extra_args):
+ launch_mmproj_path = self._resolve_launch_mmproj_path(
+ model_path = model_path,
+ mmproj_path = mmproj_path,
+ )
# Need both a resolved mmproj AND the config vision flag; a stray
# mmproj passing the family-name heuristic must not flip a non-VLM
# GGUF into vision mode.
diff --git a/studio/backend/core/inference/llama_server_args.py b/studio/backend/core/inference/llama_server_args.py
index 1862c9c5de..00f8c66d5c 100644
--- a/studio/backend/core/inference/llama_server_args.py
+++ b/studio/backend/core/inference/llama_server_args.py
@@ -260,6 +260,28 @@ def resolve_cache_type_kv(
return override if override is not None else fallback_cache_type_kv
+_MMPROJ_DISABLE_FLAGS: frozenset[str] = frozenset({"--no-mmproj", "--no-mmproj-auto"})
+_MMPROJ_ENABLE_FLAGS: frozenset[str] = frozenset({"--mmproj-auto"})
+
+
+def extra_args_disable_mmproj(args: Optional[Iterable[str]]) -> bool:
+ """True when pass-through args opt out of vision mmproj loading.
+
+ llama-server parses --mmproj-auto / --no-mmproj / --no-mmproj-auto as one
+ boolean with last-wins semantics; mirror that here.
+ """
+ if not args:
+ return False
+ disabled = False
+ for raw in args:
+ flag = _flag_name(str(raw))
+ if flag in _MMPROJ_DISABLE_FLAGS:
+ disabled = True
+ elif flag in _MMPROJ_ENABLE_FLAGS:
+ disabled = False
+ return disabled
+
+
def strip_shadowing_flags(
args: Iterable[str],
*,
diff --git a/studio/backend/routes/inference.py b/studio/backend/routes/inference.py
index 3850d0cbb2..856d9bacf8 100644
--- a/studio/backend/routes/inference.py
+++ b/studio/backend/routes/inference.py
@@ -1449,18 +1449,23 @@ async def load_model(
# parse against a freshly-supplied first-class field.
if request.llama_extra_args is None and llama_backend.extra_args:
source = llama_backend.extra_args_source
- # Compare against the resolved variant, not the request field:
- # callers commonly omit gguf_variant for local ``.gguf`` paths
- # and HF auto-pick flows. ``config.gguf_variant`` is the variant
- # load_model was actually invoked with (see HF / local branches
- # below), so both sides key off the same string.
- resolved_variant = config.gguf_variant
- same_source = bool(
- source
- and source[0]
- and source[0].lower() == model_identifier.lower()
- and (source[1] or "").lower() == (resolved_variant or "").lower()
+ # Compare against the resolved variant, not the request
+ # field: callers commonly omit gguf_variant for local
+ # ``.gguf`` paths and HF auto-pick flows. ``config.gguf_
+ # variant`` is the variant load_model was actually
+ # invoked with (see the HF / local branches below), so
+ # both sides of the comparison key off the same string.
+ resolved_variant = (config.gguf_variant or "").lower()
+ request_variant = (request.gguf_variant or "").lower()
+ stored_variant = (source[1] or "").lower() if source else ""
+ same_model = bool(
+ source and source[0] and source[0].lower() == model_identifier.lower()
)
+ if request.gguf_variant:
+ variant_mismatch = request_variant != stored_variant
+ else:
+ variant_mismatch = bool(stored_variant and resolved_variant != stored_variant)
+ same_source = same_model and not variant_mismatch
if not same_source:
logger.info(
"Not inheriting llama_extra_args: stored args came from %s, loading %s",
diff --git a/studio/backend/tests/test_llama_server_args.py b/studio/backend/tests/test_llama_server_args.py
index 09775707ee..6ae9d21e47 100644
--- a/studio/backend/tests/test_llama_server_args.py
+++ b/studio/backend/tests/test_llama_server_args.py
@@ -27,6 +27,7 @@ parse_cache_override = _lsa.parse_cache_override
parse_ctx_override = _lsa.parse_ctx_override
resolve_cache_type_kv = _lsa.resolve_cache_type_kv
strip_shadowing_flags = _lsa.strip_shadowing_flags
+extra_args_disable_mmproj = _lsa.extra_args_disable_mmproj
validate_extra_args = _lsa.validate_extra_args
@@ -509,6 +510,22 @@ def test_strip_shadowing_flags_defaults_strip_everything():
assert out == []
+def test_extra_args_disable_mmproj_detects_flag():
+ assert extra_args_disable_mmproj(["--no-mmproj"]) is True
+ assert extra_args_disable_mmproj(["--threads", "12", "--no-mmproj"]) is True
+ assert extra_args_disable_mmproj(["--no-mmproj-auto"]) is True
+
+
+def test_extra_args_disable_mmproj_false_when_absent():
+ assert extra_args_disable_mmproj(None) is False
+ assert extra_args_disable_mmproj(["--threads", "12"]) is False
+
+
+def test_extra_args_disable_mmproj_last_wins():
+ assert extra_args_disable_mmproj(["--no-mmproj", "--mmproj-auto"]) is False
+ assert extra_args_disable_mmproj(["--mmproj-auto", "--no-mmproj-auto"]) is True
+
+
def test_strip_shadowing_flags_drops_model_draft_with_spec():
# --model-draft (and aliases) are Studio-managed since the separate
# MTP drafter support: an inherited copy must not last-wins-override
From 515abca84efaf0e85723249f7a67fef6c634c2a1 Mon Sep 17 00:00:00 2001
From: James Dawdy
Date: Fri, 12 Jun 2026 02:27:18 -0500
Subject: [PATCH 11/50] fix(studio): adopt server-loaded model before chat
auto-load (#5900)
* fix(studio): adopt server-loaded model before chat auto-load
When the user starts Studio via `studio run -m`, the web UI could still
auto-load a different cached GGUF on the first message because the chat
checkpoint was empty. Sync from /api/inference/status before falling back
to autoLoadSmallestModel so CLI-loaded models are not replaced.
Co-authored-by: Cursor
* fix(studio): hydrate adopted CLI model and harden auto-load errors
Extract shared inference-status hydration for refresh() and CLI adopt
paths so the first chat turn gets reasoning/tools flags. Wrap auto-load
(including adopt) in try/catch for image-edit cleanup, and drop the
redundant adopt call in run().
Co-authored-by: Cursor
* Guard model adoption against status failures and mid-flight selection for PR #5900
* ci: trigger pre-commit.ci after main merge
Co-authored-by: Cursor
---------
Co-authored-by: Cursor
Co-authored-by: Daniel Han
---
.../src/features/chat/api/chat-adapter.ts | 6 +
.../chat/hooks/use-chat-model-runtime.ts | 177 ++-----------
.../lib/apply-inference-status-to-store.ts | 236 ++++++++++++++++++
3 files changed, 257 insertions(+), 162 deletions(-)
create mode 100644 studio/frontend/src/features/chat/lib/apply-inference-status-to-store.ts
diff --git a/studio/frontend/src/features/chat/api/chat-adapter.ts b/studio/frontend/src/features/chat/api/chat-adapter.ts
index d944c7ab74..983d4f5aa2 100644
--- a/studio/frontend/src/features/chat/api/chat-adapter.ts
+++ b/studio/frontend/src/features/chat/api/chat-adapter.ts
@@ -19,6 +19,7 @@ import {
toExternalBackendProviderType,
} from "../external-providers";
import { pickFriendlyContainerName } from "../lib/friendly-names";
+import { tryAdoptServerActiveModel } from "../lib/apply-inference-status-to-store";
import {
clampReasoningEffortToLevels,
getExternalMaxOutputTokens,
@@ -1129,6 +1130,10 @@ async function autoLoadSmallestModel(): Promise<{
loaded: boolean;
blockedByTrustRemoteCode: boolean;
}> {
+ if (await tryAdoptServerActiveModel()) {
+ return { loaded: true, blockedByTrustRemoteCode: false };
+ }
+
const store = useChatRuntimeStore.getState();
const hfToken = store.hfToken || null;
const trustRemoteCode = store.params.trustRemoteCode ?? false;
@@ -1472,6 +1477,7 @@ export function createOpenAIStreamAdapter(): ChatModelAdapter {
}
if (!useChatRuntimeStore.getState().params.checkpoint) {
+ // Prefer a model already loaded by the CLI/API before auto-loading.
let loaded: boolean;
let blockedByTrustRemoteCode: boolean;
try {
diff --git a/studio/frontend/src/features/chat/hooks/use-chat-model-runtime.ts b/studio/frontend/src/features/chat/hooks/use-chat-model-runtime.ts
index 4bd36e00d4..23ae1d80aa 100644
--- a/studio/frontend/src/features/chat/hooks/use-chat-model-runtime.ts
+++ b/studio/frontend/src/features/chat/hooks/use-chat-model-runtime.ts
@@ -24,12 +24,15 @@ import {
} from "../api/chat-api";
import { formatEta, formatRate } from "../utils/format-transfer";
import {
- CHAT_REASONING_ENABLED_KEY,
- loadOptionalBool,
- type ReasoningEffort,
resolveToolsEnabledOnLoad,
useChatRuntimeStore,
} from "../stores/chat-runtime-store";
+import {
+ applyActiveModelStatusToStore,
+ clampLocalReasoningEffort,
+ normalizeSpeculativeType,
+ resolveInferenceCheckpointId,
+} from "../lib/apply-inference-status-to-store";
import {
mergeBackendRecommendedInference,
resolveLoadMaxSeqLength,
@@ -211,39 +214,6 @@ function getTrustRemoteCodeRequiredMessage(modelName: string): string {
return `${modelName} needs custom code enabled to load. Turn on "Enable custom code" in Chat Settings, then try again.`;
}
-// Canonicalises any backend/persisted value onto the Speculative Decoding
-// dropdown's modes ("auto"/"mtp"/"ngram"/"mtp+ngram"/"off"/null). Mirrors
-// backend _canonicalize_spec_mode so legacy persisted values round-trip.
-function normalizeSpeculativeType(v: string | null | undefined): string | null {
- if (v == null) return null;
- const s = String(v).trim().toLowerCase();
- if (!s) return null;
- if (s === "auto" || s === "default") return "auto";
- if (s === "off") return "off";
- if (s === "ngram-simple") return "ngram-simple";
- if (s === "mtp" || s === "draft-mtp") return "mtp";
- if (s === "ngram" || s === "ngram-mod") return "ngram";
- if (s === "mtp+ngram") return "mtp+ngram";
- // Comma-chained legacy values (e.g. from older persisted state).
- const parts = s.split(",").map((p) => p.trim()).filter(Boolean);
- const hasMtp = parts.some((p) => p === "mtp" || p === "draft-mtp");
- const hasNgram = parts.some((p) => p === "ngram" || p === "ngram-mod");
- if (hasMtp && hasNgram) return "mtp+ngram";
- if (hasMtp) return "mtp";
- if (hasNgram) return "ngram";
- // Unknown -> safe fallback to Auto so the dropdown stays controlled.
- return "auto";
-}
-
-type LocalReasoningEffort = Extract;
-
-function clampLocalReasoningEffort(value: ReasoningEffort): LocalReasoningEffort {
- if (value === "low" || value === "medium" || value === "high") {
- return value;
- }
- return "low";
-}
-
export function useChatModelRuntime() {
const params = useChatRuntimeStore((state) => state.params);
const models = useChatRuntimeStore((state) => state.models);
@@ -328,132 +298,15 @@ export function useChatModelRuntime() {
const selectedCheckpoint = useChatRuntimeStore.getState().params.checkpoint;
const isExternalSelectionActive = isExternalModelId(selectedCheckpoint);
if (statusRes.active_model && !isExternalSelectionActive) {
- setCheckpoint(statusRes.active_model, statusRes.gguf_variant);
-
- // Apply inference defaults on reconnect (page refresh with model already loaded)
- if (statusRes.inference) {
- const currentParams = useChatRuntimeStore.getState().params;
- setParams(
- mergeBackendRecommendedInference({
- current: currentParams,
- response: statusRes,
- modelId: statusRes.active_model,
- presetSource: useChatRuntimeStore.getState().activePresetSource,
- }),
- );
- }
-
- // Restore reasoning/tools support flags and context length
- const hydratingExistingModel =
- selectedCheckpoint !== statusRes.active_model ||
- useChatRuntimeStore.getState().activeGgufVariant !==
- (statusRes.gguf_variant ?? null);
- const supportsReasoning = statusRes.supports_reasoning ?? false;
- const reasoningAlwaysOn = statusRes.reasoning_always_on ?? false;
- const reasoningStyle = statusRes.reasoning_style ?? "enable_thinking";
- const reasoningEffortLevels =
- reasoningStyle === "reasoning_effort"
- ? (["low", "medium", "high"] as const)
- : (["low", "medium", "high"] as const);
- const supportsPreserveThinking = statusRes.supports_preserve_thinking ?? false;
- const supportsTools = statusRes.supports_tools ?? false;
- const storedReasoningEnabled = loadOptionalBool(
- CHAT_REASONING_ENABLED_KEY,
- );
- const currentGgufContextLength = statusRes.is_gguf
- ? (statusRes.context_length ?? null)
- : null;
- const ggufMaxContextLength = statusRes.is_gguf
- ? (statusRes.max_context_length ?? null)
- : null;
- const ggufNativeContextLength = statusRes.is_gguf
- ? (statusRes.native_context_length ?? null)
- : null;
- const currentSpecType = normalizeSpeculativeType(
- statusRes.speculative_type,
- );
- // Refresh runs on F5 (needs hydration) and right after a load (store
- // already set). For user-configurable params, only hydrate when the
- // shadow `loaded*` field is null ("not yet hydrated"); otherwise we'd
- // clobber what the load path just applied and revert the user.
- const prevState = useChatRuntimeStore.getState();
- const clampedReasoningEffort = clampLocalReasoningEffort(
- prevState.reasoningEffort,
- );
- const nextDefaultChatTemplate =
- statusRes.chat_template === undefined
- ? prevState.defaultChatTemplate
- : statusRes.chat_template;
- useChatRuntimeStore.setState({
- supportsReasoning,
- reasoningAlwaysOn,
- reasoningStyle,
- supportsReasoningOff: reasoningStyle !== "reasoning_effort",
- reasoningEffortLevels,
- reasoningEffort: clampedReasoningEffort,
- supportsPreserveThinking,
- supportsTools,
- // Reset per-turn reasoning flag so:
- // 1. non-reasoning models don't inherit a stale off state, and
- // 2. local reasoning-effort models (Off hidden via
- // supportsReasoningOff=false) don't carry reasoningEnabled=false
- // from an external model where Off was selected -- the composer
- // would still show "Think: " but the adapter would omit
- // the kwarg, so Harmony falls back to its default effort.
- reasoningEnabled: supportsReasoning
- ? reasoningStyle === "reasoning_effort"
- ? true
- : useChatRuntimeStore.getState().reasoningEnabled
- : true,
- ggufContextLength: currentGgufContextLength,
- ggufMaxContextLength,
- ggufNativeContextLength,
- modelRequiresTrustRemoteCode:
- statusRes.requires_trust_remote_code ?? false,
- defaultChatTemplate: nextDefaultChatTemplate,
- loadedIsMultimodal: isMultimodalResponse(statusRes),
- specFallbackReason: statusRes.spec_fallback_reason ?? null,
- ...(prevState.loadedSpeculativeType === null && {
- speculativeType: currentSpecType,
- loadedSpeculativeType: currentSpecType,
- }),
- ...(statusRes.spec_draft_n_max !== undefined &&
- prevState.loadedSpecDraftNMax === null &&
- prevState.specDraftNMax === null && {
- specDraftNMax: statusRes.spec_draft_n_max ?? null,
- loadedSpecDraftNMax: statusRes.spec_draft_n_max ?? null,
- }),
- ...(statusRes.cache_type_kv !== undefined &&
- prevState.loadedKvCacheDtype === null && {
- kvCacheDtype: statusRes.cache_type_kv,
- loadedKvCacheDtype: statusRes.cache_type_kv,
- }),
- ...(statusRes.chat_template_override !== undefined &&
- prevState.loadedChatTemplateOverride === null &&
- prevState.chatTemplateOverride === null && {
- chatTemplateOverride: statusRes.chat_template_override,
- loadedChatTemplateOverride: statusRes.chat_template_override,
- }),
- });
- // setModels(listRes...) above used catalog data, which omits audio
- // capability. Re-apply live status so attach gates survive a refresh.
- syncModelCapabilities(statusRes.active_model, statusRes);
-
- // Set reasoning default for Qwen3.5/3.6 small models
- if (
- supportsReasoning &&
- hydratingExistingModel &&
- storedReasoningEnabled === null
- ) {
- let reasoningDefault = true;
- const mid = statusRes.active_model.toLowerCase();
- if (mid.includes("qwen3.5") || mid.includes("qwen3.6")) {
- const sizeMatch = mid.match(/(\d+\.?\d*)\s*b/);
- if (sizeMatch && parseFloat(sizeMatch[1]) < 9) {
- reasoningDefault = false;
- }
- }
- useChatRuntimeStore.setState({ reasoningEnabled: reasoningDefault });
+ const checkpointId = resolveInferenceCheckpointId(statusRes);
+ if (checkpointId) {
+ setCheckpoint(checkpointId, statusRes.gguf_variant);
+ applyActiveModelStatusToStore(statusRes, {
+ previousCheckpoint: selectedCheckpoint,
+ });
+ // setModels(listRes...) above used catalog data, which omits audio
+ // capability. Re-apply live status so attach gates survive a refresh.
+ syncModelCapabilities(checkpointId, statusRes);
}
} else if (!statusRes.active_model && !isExternalSelectionActive) {
useChatRuntimeStore.setState({
diff --git a/studio/frontend/src/features/chat/lib/apply-inference-status-to-store.ts b/studio/frontend/src/features/chat/lib/apply-inference-status-to-store.ts
new file mode 100644
index 0000000000..c02d304dcb
--- /dev/null
+++ b/studio/frontend/src/features/chat/lib/apply-inference-status-to-store.ts
@@ -0,0 +1,236 @@
+// SPDX-License-Identifier: AGPL-3.0-only
+// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
+
+import { getInferenceStatus } from "../api/chat-api";
+import { mergeBackendRecommendedInference } from "../presets/preset-policy";
+import {
+ CHAT_REASONING_ENABLED_KEY,
+ loadOptionalBool,
+ type ReasoningEffort,
+ resolveToolsEnabledOnLoad,
+ useChatRuntimeStore,
+} from "../stores/chat-runtime-store";
+import { isMultimodalResponse, type InferenceStatusResponse } from "../types/api";
+import type { ChatModelSummary } from "../types/runtime";
+
+type LocalReasoningEffort = Extract;
+
+// Canonicalises backend / persisted speculative mode values onto the UI modes.
+export function normalizeSpeculativeType(
+ v: string | null | undefined,
+): string | null {
+ if (v == null) return null;
+ const s = String(v).trim().toLowerCase();
+ if (!s) return null;
+ if (s === "auto" || s === "default") return "auto";
+ if (s === "off") return "off";
+ if (s === "ngram-simple") return "ngram-simple";
+ if (s === "mtp" || s === "draft-mtp") return "mtp";
+ if (s === "ngram" || s === "ngram-mod") return "ngram";
+ if (s === "mtp+ngram") return "mtp+ngram";
+ const parts = s.split(",").map((p) => p.trim()).filter(Boolean);
+ const hasMtp = parts.some((p) => p === "mtp" || p === "draft-mtp");
+ const hasNgram = parts.some((p) => p === "ngram" || p === "ngram-mod");
+ if (hasMtp && hasNgram) return "mtp+ngram";
+ if (hasMtp) return "mtp";
+ if (hasNgram) return "ngram";
+ return "auto";
+}
+
+export function clampLocalReasoningEffort(
+ value: ReasoningEffort,
+): LocalReasoningEffort {
+ if (value === "low" || value === "medium" || value === "high") {
+ return value;
+ }
+ return "low";
+}
+
+export function resolveInferenceCheckpointId(
+ status: InferenceStatusResponse,
+): string | null {
+ if (!status.active_model) return null;
+ return status.model_identifier ?? status.active_model;
+}
+
+function ensureActiveModelInStoreList(
+ status: InferenceStatusResponse,
+ checkpointId: string,
+): void {
+ const store = useChatRuntimeStore.getState();
+ if (store.models.some((model) => model.id === checkpointId)) {
+ return;
+ }
+ const summary: ChatModelSummary = {
+ id: checkpointId,
+ name: status.active_model ?? checkpointId,
+ isVision: status.is_vision ?? false,
+ isLora: false,
+ isGguf: status.is_gguf ?? false,
+ isAudio: status.is_audio ?? false,
+ audioType: status.audio_type ?? null,
+ hasAudioInput: status.has_audio_input ?? false,
+ };
+ store.setModels([...store.models, summary]);
+}
+
+export type ApplyInferenceStatusOptions = {
+ previousCheckpoint?: string;
+};
+
+/** Mirror refresh() hydration so adopted CLI models get reasoning/tools flags. */
+export function applyActiveModelStatusToStore(
+ status: InferenceStatusResponse,
+ options: ApplyInferenceStatusOptions = {},
+): void {
+ const checkpointId = resolveInferenceCheckpointId(status);
+ if (!checkpointId) return;
+
+ const store = useChatRuntimeStore.getState();
+ const previousCheckpoint =
+ options.previousCheckpoint ?? store.params.checkpoint;
+
+ if (status.inference) {
+ store.setParams(
+ mergeBackendRecommendedInference({
+ current: store.params,
+ response: status,
+ modelId: checkpointId,
+ presetSource: store.activePresetSource,
+ }),
+ );
+ }
+
+ const hydratingExistingModel =
+ previousCheckpoint !== checkpointId ||
+ store.activeGgufVariant !== (status.gguf_variant ?? null);
+ const supportsReasoning = status.supports_reasoning ?? false;
+ const reasoningAlwaysOn = status.reasoning_always_on ?? false;
+ const reasoningStyle = status.reasoning_style ?? "enable_thinking";
+ const reasoningEffortLevels =
+ reasoningStyle === "reasoning_effort"
+ ? (["low", "medium", "high"] as const)
+ : (["low", "medium", "high"] as const);
+ const supportsPreserveThinking = status.supports_preserve_thinking ?? false;
+ const supportsTools = status.supports_tools ?? false;
+ const storedReasoningEnabled = loadOptionalBool(CHAT_REASONING_ENABLED_KEY);
+ const currentGgufContextLength = status.is_gguf
+ ? (status.context_length ?? null)
+ : null;
+ const ggufMaxContextLength = status.is_gguf
+ ? (status.max_context_length ?? null)
+ : null;
+ const ggufNativeContextLength = status.is_gguf
+ ? (status.native_context_length ?? null)
+ : null;
+ const currentSpecType = normalizeSpeculativeType(status.speculative_type);
+ const prevState = useChatRuntimeStore.getState();
+ const clampedReasoningEffort = clampLocalReasoningEffort(
+ prevState.reasoningEffort,
+ );
+ const nextDefaultChatTemplate =
+ status.chat_template === undefined
+ ? prevState.defaultChatTemplate
+ : status.chat_template;
+
+ useChatRuntimeStore.setState({
+ supportsReasoning,
+ reasoningAlwaysOn,
+ reasoningStyle,
+ supportsReasoningOff: reasoningStyle !== "reasoning_effort",
+ reasoningEffortLevels,
+ reasoningEffort: clampedReasoningEffort,
+ supportsPreserveThinking,
+ supportsTools,
+ ...resolveToolsEnabledOnLoad(supportsTools),
+ reasoningEnabled: supportsReasoning
+ ? reasoningStyle === "reasoning_effort"
+ ? true
+ : useChatRuntimeStore.getState().reasoningEnabled
+ : true,
+ ggufContextLength: currentGgufContextLength,
+ ggufMaxContextLength,
+ ggufNativeContextLength,
+ modelRequiresTrustRemoteCode: status.requires_trust_remote_code ?? false,
+ defaultChatTemplate: nextDefaultChatTemplate,
+ loadedIsMultimodal: isMultimodalResponse(status),
+ specFallbackReason: status.spec_fallback_reason ?? null,
+ ...(prevState.loadedSpeculativeType === null && {
+ speculativeType: currentSpecType,
+ loadedSpeculativeType: currentSpecType,
+ }),
+ ...(status.spec_draft_n_max !== undefined &&
+ prevState.loadedSpecDraftNMax === null &&
+ prevState.specDraftNMax === null && {
+ specDraftNMax: status.spec_draft_n_max ?? null,
+ loadedSpecDraftNMax: status.spec_draft_n_max ?? null,
+ }),
+ ...(status.cache_type_kv !== undefined &&
+ prevState.loadedKvCacheDtype === null && {
+ kvCacheDtype: status.cache_type_kv,
+ loadedKvCacheDtype: status.cache_type_kv,
+ }),
+ ...(status.chat_template_override !== undefined &&
+ prevState.loadedChatTemplateOverride === null &&
+ prevState.chatTemplateOverride === null && {
+ chatTemplateOverride: status.chat_template_override,
+ loadedChatTemplateOverride: status.chat_template_override,
+ }),
+ });
+
+ ensureActiveModelInStoreList(status, checkpointId);
+
+ if (
+ supportsReasoning &&
+ hydratingExistingModel &&
+ storedReasoningEnabled === null
+ ) {
+ let reasoningDefault = true;
+ const mid = checkpointId.toLowerCase();
+ if (mid.includes("qwen3.5") || mid.includes("qwen3.6")) {
+ const sizeMatch = mid.match(/(\d+\.?\d*)\s*b/);
+ if (sizeMatch && parseFloat(sizeMatch[1]) < 9) {
+ reasoningDefault = false;
+ }
+ }
+ useChatRuntimeStore.setState({ reasoningEnabled: reasoningDefault });
+ }
+}
+
+/**
+ * Adopt the model already loaded on the inference server (e.g. via
+ * ``unsloth studio run -m``) into the chat UI checkpoint without
+ * triggering a new /api/inference/load.
+ */
+export async function tryAdoptServerActiveModel(): Promise {
+ const store = useChatRuntimeStore.getState();
+ if (store.params.checkpoint) {
+ return true;
+ }
+
+ let status: InferenceStatusResponse;
+ try {
+ status = await getInferenceStatus();
+ } catch {
+ // Status endpoint unavailable: fall back to the normal auto-load path.
+ return false;
+ }
+ if (!status.active_model) {
+ return false;
+ }
+
+ const checkpointId = resolveInferenceCheckpointId(status);
+ if (!checkpointId) {
+ return false;
+ }
+
+ // Re-check after the await: keep a checkpoint the user picked meanwhile.
+ const previousCheckpoint =
+ useChatRuntimeStore.getState().params.checkpoint;
+ if (previousCheckpoint) {
+ return true;
+ }
+ store.setCheckpoint(checkpointId, status.gguf_variant);
+ applyActiveModelStatusToStore(status, { previousCheckpoint });
+ return true;
+}
From 2fadc7b22c72863f91e9a79a91ecbdce0aae8959 Mon Sep 17 00:00:00 2001
From: Daniel Han
Date: Fri, 12 Jun 2026 00:53:13 -0700
Subject: [PATCH 12/50] Fix stale sidebar regression test to match the gap-px
markup (#6232)
test_sidebar_account_block_uses_leading_tight hardcoded gap-0.5 in its selector, but the sidebar account-block div moved to gap-px during UI polish (#6196), so the regex stopped matching and the test failed across every studio PR's Repo tests (CPU). Match the gap utility loosely (gap-\S+) since this guard is about the leading-* class for descender clipping, not the spacing.
---
tests/studio/test_studio_text_descender_clipping.py | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/tests/studio/test_studio_text_descender_clipping.py b/tests/studio/test_studio_text_descender_clipping.py
index 0b805cbd6f..ac26a10f06 100644
--- a/tests/studio/test_studio_text_descender_clipping.py
+++ b/tests/studio/test_studio_text_descender_clipping.py
@@ -36,8 +36,10 @@ def test_model_selector_trigger_label_uses_leading_tight():
def test_sidebar_account_block_uses_leading_tight():
src = _read(APP_SIDEBAR)
+ # Match the account-block parent div regardless of its gap utility (gap-0.5,
+ # gap-px, ...); this guard is about the leading-* class, not the spacing.
pattern = re.compile(
- r'',
+ r'
',
)
matches = pattern.findall(src)
assert matches, "could not find sidebar account-block parent div"
From a24c9987ca6a10537e59cadff8cc1bac9e38e19a Mon Sep 17 00:00:00 2001
From: Daniel Han
Date: Fri, 12 Jun 2026 01:12:20 -0700
Subject: [PATCH 13/50] Studio: gate the staged prebuilt runtime validation
behind a flag (off by default) (#6216)
The post-download llama-quantize / llama-server smoke test JIT-compiles CUDA kernels on the first GPU forward pass and stalls every install and update by minutes on Blackwell (sm_100). Gate it behind _RUN_STAGED_PREBUILT_VALIDATION, disabled for now, keeping the smoke test and the source-build fallback it triggers fully intact so it can be restored by flipping the flag to True.
Hashless external prebuilts (e.g. lemonade) are not in the approved-sha256 manifest and rely on the functional smoke test as their only integrity gate, so they are always validated regardless of the flag; only approved bundles, already proven by the sha256 manifest, skip it.
The sha256 archive verification and the static Linux/macOS preflights are unchanged and still run for every install.
---
studio/install_llama_prebuilt.py | 48 +++++---
.../test_install_llama_prebuilt_logic.py | 106 ++++++++++++++++++
2 files changed, 137 insertions(+), 17 deletions(-)
diff --git a/studio/install_llama_prebuilt.py b/studio/install_llama_prebuilt.py
index bed7a27a63..e3b2d493fb 100644
--- a/studio/install_llama_prebuilt.py
+++ b/studio/install_llama_prebuilt.py
@@ -174,6 +174,12 @@ TEST_MODEL_URL = "https://huggingface.co/ggml-org/models/resolve/main/tinyllamas
TEST_MODEL_SHA256 = "270cba1bd5109f42d03350f60406024560464db173c0e387d91f0426d3bd256d"
VALIDATION_MODEL_CACHE_DIRNAME = ".cache"
VALIDATION_MODEL_CACHE_FILENAME = "stories260K.gguf"
+# Master switch for the staged runtime smoke test (llama-quantize + llama-server)
+# in validate_prebuilt_choice. Disabled for now: the llama-server GPU forward pass
+# JIT-compiles CUDA kernels on first load and stalls every install and update by
+# minutes on Blackwell (sm_100). The check and the source-build fallback it triggers
+# are kept intact -- set this to True to re-enable them.
+_RUN_STAGED_PREBUILT_VALIDATION = False
INSTALL_LOCK_TIMEOUT_SECONDS = 300
INSTALL_STAGING_ROOT_NAME = ".staging"
GITHUB_AUTH_HOSTS = {"api.github.com", "github.com"}
@@ -6551,23 +6557,31 @@ def validate_prebuilt_choice(
approved_checksums = approved_checksums,
prebuilt_fallback_used = prebuilt_fallback_used,
)
- validate_quantize(
- quantize_path,
- probe_path,
- quantized_path,
- install_dir,
- host,
- runtime_line = choice.runtime_line,
- )
- validate_server(
- server_path,
- probe_path,
- host,
- install_dir,
- runtime_line = choice.runtime_line,
- install_kind = choice.install_kind,
- )
- log(f"staged prebuilt validation succeeded for {choice.name}")
+ # Hashless external prebuilts (e.g. lemonade) are not in the approved-sha256
+ # manifest and rely on the functional smoke test as their only integrity gate,
+ # so they are always validated. For an approved bundle the sha256 manifest
+ # already proves integrity, so its runtime smoke test -- a cold CUDA-JIT pass
+ # costing minutes on Blackwell sm_100 -- is gated behind
+ # _RUN_STAGED_PREBUILT_VALIDATION, disabled for now. The check and the
+ # source-build fallback it triggers are kept intact; flip the flag to restore it.
+ if choice.expected_sha256 is None or _RUN_STAGED_PREBUILT_VALIDATION:
+ validate_quantize(
+ quantize_path,
+ probe_path,
+ quantized_path,
+ install_dir,
+ host,
+ runtime_line = choice.runtime_line,
+ )
+ validate_server(
+ server_path,
+ probe_path,
+ host,
+ install_dir,
+ runtime_line = choice.runtime_line,
+ install_kind = choice.install_kind,
+ )
+ log(f"staged prebuilt validation succeeded for {choice.name}")
return server_path, quantize_path
diff --git a/tests/studio/install/test_install_llama_prebuilt_logic.py b/tests/studio/install/test_install_llama_prebuilt_logic.py
index 000c5a7ace..313cfa9225 100644
--- a/tests/studio/install/test_install_llama_prebuilt_logic.py
+++ b/tests/studio/install/test_install_llama_prebuilt_logic.py
@@ -2761,3 +2761,109 @@ def test_python_runtime_dirs_covers_cu13_and_library_bin(monkeypatch, tmp_path:
assert str(cu13_arch) in dirs
assert str(library_bin) in dirs
assert str(torch_lib) in dirs
+
+
+def _nvidia_linux_host():
+ return HostInfo(
+ system = "Linux",
+ machine = "x86_64",
+ is_windows = False,
+ is_linux = True,
+ is_macos = False,
+ is_x86_64 = True,
+ is_arm64 = False,
+ nvidia_smi = None,
+ driver_cuda_version = None,
+ compute_caps = ["10.0"],
+ visible_cuda_devices = None,
+ has_physical_nvidia = True,
+ has_usable_nvidia = True,
+ )
+
+
+def _run_validate_prebuilt_choice(monkeypatch, tmp_path, *, expected_sha256):
+ """Drive validate_prebuilt_choice with every heavy install step stubbed and
+ return how many times the functional quantize/server smoke tests ran."""
+ calls = {"quantize": 0, "server": 0}
+ server_path = tmp_path / "install" / "build" / "bin" / "llama-server"
+ quantize_path = tmp_path / "install" / "build" / "bin" / "llama-quantize"
+
+ src = INSTALL_LLAMA_PREBUILT
+ monkeypatch.setattr(
+ src, "preferred_source_archive", lambda *a, **k: ("repo", "ref", None, False)
+ )
+ monkeypatch.setattr(src, "hydrate_source_tree", lambda *a, **k: None)
+ monkeypatch.setattr(src, "install_from_archives", lambda *a, **k: (server_path, quantize_path))
+ monkeypatch.setattr(src, "preflight_linux_installed_binaries", lambda *a, **k: None)
+ monkeypatch.setattr(src, "preflight_macos_installed_binaries", lambda *a, **k: None)
+ monkeypatch.setattr(src, "ensure_repo_shape", lambda *a, **k: None)
+ monkeypatch.setattr(src, "write_prebuilt_metadata", lambda *a, **k: None)
+ monkeypatch.setattr(
+ src,
+ "validate_quantize",
+ lambda *a, **k: calls.__setitem__("quantize", calls["quantize"] + 1),
+ )
+ monkeypatch.setattr(
+ src, "validate_server", lambda *a, **k: calls.__setitem__("server", calls["server"] + 1)
+ )
+
+ bundle_name = "app-b9998-linux-x64-cuda13-newer.tar.gz"
+ source_archive = tmp_path / "source.tar.gz"
+ bundle_archive = tmp_path / "bundle.tar.gz"
+ source_archive.write_bytes(b"source")
+ bundle_archive.write_bytes(b"bundle")
+
+ choice = AssetChoice(
+ repo = "local",
+ tag = "b9998",
+ name = bundle_name,
+ url = "file://bundle",
+ source_label = "local",
+ is_ready_bundle = True,
+ install_kind = "linux-cuda",
+ bundle_profile = "cuda13-newer",
+ runtime_line = "cuda13",
+ expected_sha256 = expected_sha256,
+ )
+ src.validate_prebuilt_choice(
+ choice,
+ _nvidia_linux_host(),
+ tmp_path / "install",
+ tmp_path / "work",
+ tmp_path / "stories260K.gguf",
+ requested_tag = "b9998",
+ llama_tag = "b9998",
+ release_tag = "b9998",
+ approved_checksums = approved_checksums_for(
+ "b9998",
+ source_archive = source_archive,
+ bundle_archive = bundle_archive,
+ bundle_name = bundle_name,
+ ),
+ prebuilt_fallback_used = False,
+ quantized_path = tmp_path / "stories260K-q4.gguf",
+ )
+ return calls
+
+
+def test_validate_prebuilt_choice_approved_validation_skipped_when_flag_off(tmp_path, monkeypatch):
+ # An approved (sha256-verified) bundle skips the staged smoke test while the
+ # flag is off: the manifest hash is its integrity gate.
+ calls = _run_validate_prebuilt_choice(monkeypatch, tmp_path, expected_sha256 = "ab" * 32)
+ assert calls == {"quantize": 0, "server": 0}
+
+
+def test_validate_prebuilt_choice_hashless_build_always_validated(tmp_path, monkeypatch):
+ # A hashless external build (e.g. lemonade) has no approved sha256, so the
+ # functional smoke test is its only integrity gate and must run even while the
+ # flag is off -- otherwise a corrupted/replaced archive could be activated.
+ calls = _run_validate_prebuilt_choice(monkeypatch, tmp_path, expected_sha256 = None)
+ assert calls == {"quantize": 1, "server": 1}
+
+
+def test_validate_prebuilt_choice_approved_validation_runs_when_flag_enabled(tmp_path, monkeypatch):
+ # Flipping _RUN_STAGED_PREBUILT_VALIDATION back on restores the full smoke test
+ # for approved bundles too, proving the check is kept intact, only gated off.
+ monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "_RUN_STAGED_PREBUILT_VALIDATION", True)
+ calls = _run_validate_prebuilt_choice(monkeypatch, tmp_path, expected_sha256 = "ab" * 32)
+ assert calls == {"quantize": 1, "server": 1}
From 14ed91e39a30e013928983bde3409ef1847ec36e Mon Sep 17 00:00:00 2001
From: alkinun
Date: Fri, 12 Jun 2026 11:15:37 +0300
Subject: [PATCH 14/50] Fix FastModel config passthrough for sequence
classification (#6203)
* add FastModel config passthrough
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix fastmodel config passthrough for task configs
* fix config-driven FastModel task model selection
* fix text only fastmodel task config selection
* fix fastmodel task config inference from user configs
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix fastmodel problem_type config passthrough
* fix fastlanguagemodel config passthrough: FastLlamaModel owns user config
* fix fastlanguagemodel config passthrough: forward user config to causal loads and keep checkpoint quantization_config
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Etherll <61019402+Etherll@users.noreply.github.com>
---
.../test_fast_model_config_passthrough.py | 215 ++++++++++++++++++
unsloth/models/_utils.py | 23 ++
unsloth/models/llama.py | 74 ++++--
unsloth/models/loader.py | 98 ++++++--
unsloth/models/vision.py | 14 +-
5 files changed, 389 insertions(+), 35 deletions(-)
create mode 100644 tests/python/test_fast_model_config_passthrough.py
diff --git a/tests/python/test_fast_model_config_passthrough.py b/tests/python/test_fast_model_config_passthrough.py
new file mode 100644
index 0000000000..b2ba3d2eef
--- /dev/null
+++ b/tests/python/test_fast_model_config_passthrough.py
@@ -0,0 +1,215 @@
+"""FastModel config passthrough and nested task config handling."""
+
+import ast
+from pathlib import Path
+
+
+REPO_ROOT = Path(__file__).resolve().parents[2]
+LOADER_PATH = REPO_ROOT / "unsloth" / "models" / "loader.py"
+VISION_PATH = REPO_ROOT / "unsloth" / "models" / "vision.py"
+UTILS_PATH = REPO_ROOT / "unsloth" / "models" / "_utils.py"
+LLAMA_PATH = REPO_ROOT / "unsloth" / "models" / "llama.py"
+
+
+def _source(path):
+ return path.read_text()
+
+
+def _class_method(tree, class_name, method_name):
+ for node in tree.body:
+ if isinstance(node, ast.ClassDef) and node.name == class_name:
+ for item in node.body:
+ if isinstance(item, ast.FunctionDef) and item.name == method_name:
+ return item
+ raise AssertionError(f"{class_name}.{method_name} not found")
+
+
+def _assigns_from_kwargs_pop(method, target_name, key_name):
+ for node in ast.walk(method):
+ if not isinstance(node, ast.Assign):
+ continue
+ if not any(
+ isinstance(target, ast.Name) and target.id == target_name for target in node.targets
+ ):
+ continue
+ value = node.value
+ if not (
+ isinstance(value, ast.Call)
+ and isinstance(value.func, ast.Attribute)
+ and value.func.attr == "pop"
+ and isinstance(value.func.value, ast.Name)
+ and value.func.value.id == "kwargs"
+ and value.args
+ and isinstance(value.args[0], ast.Constant)
+ and value.args[0].value == key_name
+ ):
+ continue
+ return True
+ return False
+
+
+def _calls_name(method, name):
+ return any(
+ isinstance(node, ast.Call) and isinstance(node.func, ast.Name) and node.func.id == name
+ for node in ast.walk(method)
+ )
+
+
+def _load_task_attr_helper():
+ source = _source(UTILS_PATH)
+ funcs = {
+ node.name: ast.get_source_segment(source, node)
+ for node in ast.parse(source).body
+ if isinstance(node, ast.FunctionDef)
+ }
+ ns = {}
+ for name in ("_config_set", "set_task_config_attr"):
+ exec(funcs[name], ns)
+ return ns["set_task_config_attr"]
+
+
+def _load_loader_task_helpers():
+ source = _source(LOADER_PATH)
+ funcs = {
+ node.name: ast.get_source_segment(source, node)
+ for node in ast.parse(source).body
+ if isinstance(node, ast.FunctionDef)
+ }
+ ns = {}
+ for name in (
+ "_config_get",
+ "_config_diff",
+ "_has_sequence_classification_architecture",
+ "_get_user_task_config_attrs",
+ ):
+ exec(funcs[name], ns)
+ return ns["_get_user_task_config_attrs"]
+
+
+def test_fast_model_consumes_user_config_kwarg():
+ tree = ast.parse(_source(LOADER_PATH))
+ method = _class_method(tree, "FastModel", "from_pretrained")
+
+ assert _assigns_from_kwargs_pop(method, "user_config", "config")
+
+
+def test_fast_base_model_consumes_user_config_kwarg():
+ tree = ast.parse(_source(VISION_PATH))
+ method = _class_method(tree, "FastBaseModel", "from_pretrained")
+
+ assert _assigns_from_kwargs_pop(method, "user_config", "config")
+
+
+def test_fast_llama_model_consumes_user_config_kwarg():
+ tree = ast.parse(_source(LLAMA_PATH))
+ method = _class_method(tree, "FastLlamaModel", "from_pretrained")
+
+ assert _assigns_from_kwargs_pop(method, "user_config", "config")
+
+
+def test_fast_base_model_sets_task_attrs_on_nested_text_config():
+ tree = ast.parse(_source(VISION_PATH))
+ method = _class_method(tree, "FastBaseModel", "from_pretrained")
+
+ assert _calls_name(method, "set_task_config_attr")
+
+
+def test_fast_base_model_pops_problem_type_as_config_attr():
+ source = _source(VISION_PATH)
+
+ assert '("id2label", "label2id", "problem_type")' in source
+
+
+def test_fast_model_uses_user_config_num_labels_for_task_model_selection():
+ tree = ast.parse(_source(LOADER_PATH))
+ method = _class_method(tree, "FastModel", "from_pretrained")
+
+ assert _calls_name(method, "_get_user_task_config_attrs")
+
+
+def test_fast_model_captures_user_config_num_labels_before_text_only_switch():
+ source = _source(LOADER_PATH)
+
+ fallback = source.index("task_config_attrs = _get_user_task_config_attrs(user_config)")
+ text_only_switch = source.index("model_config = text_config")
+
+ assert fallback < text_only_switch
+
+
+def test_user_task_config_attrs_ignore_default_num_labels():
+ get_user_task_config_attrs = _load_loader_task_helpers()
+
+ class Config:
+ num_labels = 2
+ id2label = {0: "LABEL_0", 1: "LABEL_1"}
+ label2id = {"LABEL_0": 0, "LABEL_1": 1}
+
+ def to_diff_dict(self):
+ return {}
+
+ assert get_user_task_config_attrs(Config()) == {}
+
+
+def test_user_task_config_attrs_preserve_custom_label_maps():
+ get_user_task_config_attrs = _load_loader_task_helpers()
+
+ class Config:
+ num_labels = 2
+ id2label = {0: "negative", 1: "positive"}
+ label2id = {"negative": 0, "positive": 1}
+
+ def to_diff_dict(self):
+ return {"id2label": self.id2label, "label2id": self.label2id}
+
+ attrs = get_user_task_config_attrs(Config())
+
+ assert attrs["num_labels"] == 2
+ assert attrs["id2label"] == {0: "negative", 1: "positive"}
+ assert attrs["label2id"] == {"negative": 0, "positive": 1}
+
+
+def test_user_task_config_attrs_preserve_explicit_dict_num_labels():
+ get_user_task_config_attrs = _load_loader_task_helpers()
+
+ assert get_user_task_config_attrs({"num_labels": 2}) == {"num_labels": 2}
+
+
+def test_task_config_attr_updates_parent_and_text_config_objects():
+ set_task_config_attr = _load_task_attr_helper()
+
+ class TextConfig:
+ pass
+
+ class ParentConfig:
+ def __init__(self):
+ self.text_config = TextConfig()
+
+ def get_text_config(self):
+ return self.text_config
+
+ config = ParentConfig()
+
+ set_task_config_attr(config, "num_labels", 3)
+
+ assert config.num_labels == 3
+ assert config.text_config.num_labels == 3
+
+
+def test_task_config_attr_updates_parent_and_text_config_dicts():
+ set_task_config_attr = _load_task_attr_helper()
+ config = {"text_config": {}}
+
+ set_task_config_attr(config, "label2id", {"negative": 0, "positive": 1})
+
+ assert config["label2id"] == {"negative": 0, "positive": 1}
+ assert config["text_config"]["label2id"] == {"negative": 0, "positive": 1}
+
+
+def test_task_config_attr_ignores_primitive_text_config():
+ set_task_config_attr = _load_task_attr_helper()
+ config = {"text_config": "not-a-config"}
+
+ set_task_config_attr(config, "num_labels", 2)
+
+ assert config["num_labels"] == 2
+ assert config["text_config"] == "not-a-config"
diff --git a/unsloth/models/_utils.py b/unsloth/models/_utils.py
index 6baa9a1398..2f4e3a069e 100644
--- a/unsloth/models/_utils.py
+++ b/unsloth/models/_utils.py
@@ -69,6 +69,7 @@ __all__ = [
"resolve_attention_implementation",
"resolve_encoder_attention_implementation",
"_set_attn_impl",
+ "set_task_config_attr",
"patch_fast_lora",
"validate_loftq_config",
"RaiseUninitialized",
@@ -306,6 +307,28 @@ def _config_set(config, field_name, value):
setattr(config, field_name, value)
+def set_task_config_attr(config, field_name, value):
+ _config_set(config, field_name, value)
+ text_config = None
+ if isinstance(config, dict):
+ text_config = config.get("text_config", None)
+ elif config is not None:
+ get_text_config = getattr(config, "get_text_config", None)
+ if callable(get_text_config):
+ try:
+ text_config = get_text_config()
+ except Exception:
+ text_config = None
+ if text_config is None:
+ text_config = getattr(config, "text_config", None)
+ if (
+ text_config is not None
+ and text_config is not config
+ and (isinstance(text_config, dict) or hasattr(text_config, "__dict__"))
+ ):
+ _config_set(text_config, field_name, value)
+
+
def _iter_attention_configs(config, seen = None):
if config is None or (not isinstance(config, dict) and not hasattr(config, "__dict__")):
return
diff --git a/unsloth/models/llama.py b/unsloth/models/llama.py
index a60edf3fbe..08802f030e 100644
--- a/unsloth/models/llama.py
+++ b/unsloth/models/llama.py
@@ -2383,11 +2383,30 @@ class FastLlamaModel:
assert dtype == torch.float16 or dtype == torch.bfloat16 or dtype == torch.float32
# RoPE Scaling
- model_config = AutoConfig.from_pretrained(
- model_name,
- token = token,
- attn_implementation = "sdpa",
- )
+ # Respect a user-provided config so it is the single config object used
+ # everywhere below; otherwise HF would receive it again through **kwargs
+ # alongside our own config= and fail with a duplicate-kwarg TypeError.
+ user_config = kwargs.pop("config", None)
+ if user_config is not None:
+ model_config = user_config
+ # model_name may have been remapped to a prequantized repo whose
+ # checkpoint needs its quantization_config; graft it onto the user
+ # config or the 4bit weights load without their quant state.
+ if getattr(model_config, "quantization_config", None) is None:
+ _checkpoint_config = AutoConfig.from_pretrained(
+ model_name,
+ token = token,
+ attn_implementation = "sdpa",
+ )
+ _checkpoint_quant = getattr(_checkpoint_config, "quantization_config", None)
+ if _checkpoint_quant is not None:
+ model_config.quantization_config = _checkpoint_quant
+ else:
+ model_config = AutoConfig.from_pretrained(
+ model_name,
+ token = token,
+ attn_implementation = "sdpa",
+ )
model_config.model_name = model_name
model_max_seq_length = model_config.max_position_embeddings
@@ -2504,14 +2523,17 @@ class FastLlamaModel:
# Transformers 5.x @strict config classes reject unexpected kwargs
# like num_labels and max_position_embeddings. Set on the config
# object directly and pass config= instead.
- model_config.num_labels = num_labels
+ set_task_config_attr(model_config, "num_labels", num_labels)
if max_position_embeddings is not None:
model_config.max_position_embeddings = max_position_embeddings
# Pop config-level attrs that would be rejected by @strict model init
for _cfg_key in ("id2label", "label2id", "rope_scaling"):
_cfg_val = kwargs.pop(_cfg_key, None)
if _cfg_val is not None:
- setattr(model_config, _cfg_key, _cfg_val)
+ if _cfg_key in ("id2label", "label2id"):
+ set_task_config_attr(model_config, _cfg_key, _cfg_val)
+ else:
+ setattr(model_config, _cfg_key, _cfg_val)
model = AutoModelForSequenceClassification.from_pretrained(
model_name,
config = model_config,
@@ -2544,17 +2566,33 @@ class FastLlamaModel:
fast_inference = fast_inference,
)
elif not fast_inference:
- model = AutoModelForCausalLM.from_pretrained(
- model_name,
- device_map = device_map,
- # torch_dtype = dtype, # transformers changed torch_dtype to dtype
- # quantization_config = bnb_config,
- token = token,
- max_position_embeddings = max_position_embeddings,
- trust_remote_code = trust_remote_code,
- attn_implementation = preferred_attn_impl,
- **kwargs,
- )
+ if user_config is not None:
+ # Transformers 5.x @strict model init rejects extra kwargs next
+ # to config=; set the override on the config and pass the single
+ # config object through so user overrides reach the actual load.
+ if max_position_embeddings is not None:
+ model_config.max_position_embeddings = max_position_embeddings
+ model = AutoModelForCausalLM.from_pretrained(
+ model_name,
+ config = model_config,
+ device_map = device_map,
+ token = token,
+ trust_remote_code = trust_remote_code,
+ attn_implementation = preferred_attn_impl,
+ **kwargs,
+ )
+ else:
+ model = AutoModelForCausalLM.from_pretrained(
+ model_name,
+ device_map = device_map,
+ # torch_dtype = dtype, # transformers changed torch_dtype to dtype
+ # quantization_config = bnb_config,
+ token = token,
+ max_position_embeddings = max_position_embeddings,
+ trust_remote_code = trust_remote_code,
+ attn_implementation = preferred_attn_impl,
+ **kwargs,
+ )
# Attach dispatch hooks for bnb multi-device loads.
from unsloth.models.vision import _attach_bnb_multidevice_hooks
diff --git a/unsloth/models/loader.py b/unsloth/models/loader.py
index 4dc3046928..cfcfcae505 100644
--- a/unsloth/models/loader.py
+++ b/unsloth/models/loader.py
@@ -101,6 +101,7 @@ from ._utils import (
resolve_model_class,
_is_family_text_decoder,
_apply_text_only_key_mapping,
+ set_task_config_attr,
)
# Single source of truth is unsloth_zoo.model_lists. Re-exported so callers
@@ -133,6 +134,57 @@ def _strip_unsloth_bnb_4bit_suffix(model_name: str) -> str:
return s
+def _config_get(
+ config,
+ field_name,
+ default = None,
+):
+ if isinstance(config, dict):
+ return config.get(field_name, default)
+ return getattr(config, field_name, default)
+
+
+def _config_diff(config):
+ if isinstance(config, dict):
+ return config
+ to_diff_dict = getattr(config, "to_diff_dict", None)
+ if callable(to_diff_dict):
+ try:
+ diff = to_diff_dict()
+ if isinstance(diff, dict):
+ return diff
+ except Exception:
+ pass
+ return {}
+
+
+def _has_sequence_classification_architecture(config):
+ architectures = _config_get(config, "architectures", None) or []
+ return any(str(arch).endswith("ForSequenceClassification") for arch in architectures)
+
+
+def _get_user_task_config_attrs(user_config):
+ if user_config is None:
+ return {}
+ diff = _config_diff(user_config)
+ attrs = {}
+ for key in ("id2label", "label2id", "problem_type"):
+ if key in diff:
+ attrs[key] = _config_get(user_config, key, diff.get(key))
+ if isinstance(user_config, dict) and "num_labels" in user_config:
+ attrs["num_labels"] = user_config["num_labels"]
+ elif _has_sequence_classification_architecture(user_config):
+ num_labels = _config_get(user_config, "num_labels", None)
+ if num_labels is not None:
+ attrs["num_labels"] = num_labels
+ elif "id2label" in attrs:
+ try:
+ attrs["num_labels"] = len(attrs["id2label"])
+ except TypeError:
+ pass
+ return attrs
+
+
DISABLE_COMPILE_MODEL_NAMES = [
"aya_vision",
"modernbert",
@@ -907,6 +959,7 @@ class FastModel(FastBaseModel):
*args,
**kwargs,
):
+ user_config = kwargs.pop("config", None)
# Respect user-provided quantization_config (e.g. BitsAndBytesConfig)
quantization_config = kwargs.get("quantization_config", None)
if quantization_config is not None:
@@ -1104,13 +1157,15 @@ class FastModel(FastBaseModel):
)
try:
- model_config = AutoConfig.from_pretrained(
- model_name,
- token = token,
- revision = revision,
- trust_remote_code = trust_remote_code,
- local_files_only = local_files_only,
- )
+ model_config = user_config
+ if model_config is None:
+ model_config = AutoConfig.from_pretrained(
+ model_name,
+ token = token,
+ revision = revision,
+ trust_remote_code = trust_remote_code,
+ local_files_only = local_files_only,
+ )
is_model = True
except ImportError:
raise
@@ -1384,12 +1439,15 @@ class FastModel(FastBaseModel):
load_in_fp8 = False
load_in_16bit = True
- model_config = AutoConfig.from_pretrained(
- model_name,
- token = token,
- trust_remote_code = trust_remote_code,
- local_files_only = local_files_only,
- )
+ if user_config is not None:
+ model_config = user_config
+ else:
+ model_config = AutoConfig.from_pretrained(
+ model_name,
+ token = token,
+ trust_remote_code = trust_remote_code,
+ local_files_only = local_files_only,
+ )
if not was_disabled:
enable_progress_bars()
@@ -1469,6 +1527,17 @@ class FastModel(FastBaseModel):
else:
tokenizer_name = kwargs.pop("tokenizer_name", None)
+ # Capture task intent before text_only can replace a parent VLM config
+ # with its nested text config.
+ task_config_attrs = _get_user_task_config_attrs(user_config)
+ for _cfg_key in ("num_labels", "id2label", "label2id", "problem_type"):
+ _cfg_val = kwargs.get(_cfg_key, None)
+ if _cfg_val is not None:
+ task_config_attrs[_cfg_key] = _cfg_val
+ _num_labels = task_config_attrs.get("num_labels", None)
+ for _cfg_key, _cfg_val in task_config_attrs.items():
+ set_task_config_attr(model_config, _cfg_key, _cfg_val)
+
# Check if VLM
architectures = getattr(model_config, "architectures", None)
if architectures is None:
@@ -1499,7 +1568,8 @@ class FastModel(FastBaseModel):
else:
is_vlm = False
# If num_labels is set, use AutoModelForSequenceClassification
- _num_labels = kwargs.get("num_labels", None)
+ for _cfg_key, _cfg_val in task_config_attrs.items():
+ set_task_config_attr(model_config, _cfg_key, _cfg_val)
if auto_model is None:
if _num_labels is not None:
from transformers import AutoModelForSequenceClassification
diff --git a/unsloth/models/vision.py b/unsloth/models/vision.py
index a52ab359bd..e8161427d5 100644
--- a/unsloth/models/vision.py
+++ b/unsloth/models/vision.py
@@ -37,6 +37,7 @@ from ._utils import (
_get_text_only_config,
_is_family_text_decoder,
_apply_text_only_key_mapping,
+ set_task_config_attr,
)
from ._utils import *
from .loader_utils import _get_fp8_mode_and_check_settings
@@ -592,6 +593,10 @@ class FastBaseModel:
text_only = False,
**kwargs,
):
+ user_config = kwargs.pop("config", None)
+ if auto_config is None and user_config is not None:
+ auto_config = user_config
+
if unsloth_vllm_standby and os.environ.get("UNSLOTH_VLLM_STANDBY", "0") != "1":
raise RuntimeError(
"Unsloth: UNSLOTH_VLLM_STANDBY is True, but UNSLOTH_VLLM_STANDBY is not set to 1!"
@@ -950,11 +955,14 @@ class FastBaseModel:
# Move config-level attributes onto the config object directly.
_num_labels = kwargs.pop("num_labels", None)
if _num_labels is not None:
- model_config.num_labels = _num_labels
- for _cfg_key in ("id2label", "label2id", "max_position_embeddings"):
+ set_task_config_attr(model_config, "num_labels", _num_labels)
+ for _cfg_key in ("id2label", "label2id", "problem_type"):
_cfg_val = kwargs.pop(_cfg_key, None)
if _cfg_val is not None:
- setattr(model_config, _cfg_key, _cfg_val)
+ set_task_config_attr(model_config, _cfg_key, _cfg_val)
+ _cfg_val = kwargs.pop("max_position_embeddings", None)
+ if _cfg_val is not None:
+ setattr(model_config, "max_position_embeddings", _cfg_val)
model = auto_model.from_pretrained(
model_name,
config = model_config,
From 51f1c8732dba3cb6076a5929142fc2f933e6b353 Mon Sep 17 00:00:00 2001
From: dylanschroers <60888108+dylanschroers@users.noreply.github.com>
Date: Fri, 12 Jun 2026 04:31:31 -0400
Subject: [PATCH 15/50] fix: decode subprocess output as UTF-8 in save.py on
Windows (#6218)
* Fix UnicodeDecodeError on Windows reading subprocess output in save path
On Windows the default text encoding is the locale code page (cp1252), not
UTF-8. The text-mode subprocess calls in save.py (text=True /
universal_newlines=True) set no explicit encoding, so they decode
llama.cpp / Ollama output with cp1252. When a child process emits a byte
undefined in cp1252 -- e.g. 0x9d, which appears inside the UTF-8 encoding
of common punctuation / box-drawing glyphs and in non-ASCII file paths --
the read raises UnicodeDecodeError and aborts GGUF export.
Add encoding="utf-8", errors="replace" to all 8 text-mode subprocess calls.
errors="replace" also avoids silent mojibake for inputs whose bytes happen
to be valid-but-wrong in cp1252.
Add tests/saving/test_save_subprocess_utf8_encoding.py:
- an AST drift detector asserting every text-mode subprocess call in
save.py pins encoding="utf-8" (runs without importing torch/unsloth_zoo)
- a behavioural test reproducing the cp1252 failure and the utf-8 fix
Relates-to: #2660
Co-Authored-By: Claude Opus 4.8
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Claude Opus 4.8
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
---
.../test_save_subprocess_utf8_encoding.py | 148 ++++++++++++++++++
unsloth/save.py | 16 ++
2 files changed, 164 insertions(+)
create mode 100644 tests/saving/test_save_subprocess_utf8_encoding.py
diff --git a/tests/saving/test_save_subprocess_utf8_encoding.py b/tests/saving/test_save_subprocess_utf8_encoding.py
new file mode 100644
index 0000000000..4a609cd7b7
--- /dev/null
+++ b/tests/saving/test_save_subprocess_utf8_encoding.py
@@ -0,0 +1,148 @@
+# Unsloth - 2x faster, 60% less VRAM LLM training and finetuning
+# Copyright 2023-present Daniel Han-Chen, Michael Han-Chen & the Unsloth team. All rights reserved.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Lesser General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU Lesser General Public License for more details.
+
+"""Regression tests for unslothai/unsloth#2660.
+
+On Windows the default text encoding is the locale code page (e.g. cp1252),
+not UTF-8. ``subprocess.Popen`` / ``subprocess.run`` opened in text mode
+(``text=True`` / ``universal_newlines=True``) without an explicit
+``encoding`` therefore decode child-process output with cp1252. When
+llama.cpp / Ollama emit a byte that is undefined in cp1252 (e.g. ``0x9d``,
+which appears inside the UTF-8 encoding of common punctuation and box-drawing
+glyphs), the read raises ``UnicodeDecodeError`` and aborts the GGUF export.
+
+Two checks:
+
+* ``test_save_subprocess_text_calls_declare_utf8_encoding`` -- a source-level
+ drift detector. It parses ``unsloth/save.py`` (no import, so it runs under
+ the GPU/torch-free harness) and fails if any text-mode subprocess call is
+ missing ``encoding="utf-8"``. This is the regression guard: it is red
+ before the fix and green after.
+* ``test_utf8_replace_decodes_non_cp1252_subprocess_output`` -- a behavioural
+ check that documents the bug and the fix deterministically on any platform:
+ raw child output that is invalid under cp1252 raises, while the
+ ``encoding="utf-8", errors="replace"`` kwargs used by the fix read it
+ cleanly.
+"""
+
+from __future__ import annotations
+
+import ast
+import subprocess
+import sys
+from pathlib import Path
+
+import pytest
+
+SAVE_PY = Path(__file__).resolve().parents[2] / "unsloth" / "save.py"
+
+
+def _is_subprocess_call(node: ast.Call) -> bool:
+ """True for ``subprocess.Popen(...)`` / ``subprocess.run(...)``."""
+ func = node.func
+ return (
+ isinstance(func, ast.Attribute)
+ and func.attr in {"Popen", "run"}
+ and isinstance(func.value, ast.Name)
+ and func.value.id == "subprocess"
+ )
+
+
+def _kw(node: ast.Call, name: str):
+ for kw in node.keywords:
+ if kw.arg == name:
+ return kw.value
+ return None
+
+
+def _is_true(value) -> bool:
+ return isinstance(value, ast.Constant) and value.value is True
+
+
+def _is_text_mode(node: ast.Call) -> bool:
+ """Text mode = ``text=True`` or ``universal_newlines=True``."""
+ return _is_true(_kw(node, "text")) or _is_true(_kw(node, "universal_newlines"))
+
+
+def _collect_text_mode_subprocess_calls() -> list[ast.Call]:
+ tree = ast.parse(SAVE_PY.read_text(encoding = "utf-8"), filename = str(SAVE_PY))
+ return [
+ node
+ for node in ast.walk(tree)
+ if isinstance(node, ast.Call) and _is_subprocess_call(node) and _is_text_mode(node)
+ ]
+
+
+def test_text_mode_subprocess_calls_exist():
+ """Guard the guard: if save.py stops using text-mode subprocess calls the
+ drift test below would vacuously pass, so make sure we are actually
+ inspecting something."""
+ calls = _collect_text_mode_subprocess_calls()
+ assert len(calls) >= 6, (
+ f"Expected several text-mode subprocess calls in {SAVE_PY.name}, "
+ f"found {len(calls)} -- has the file been restructured?"
+ )
+
+
+def test_save_subprocess_text_calls_declare_utf8_encoding():
+ """Every text-mode subprocess call in save.py must pin encoding='utf-8'.
+
+ Without it, reading llama.cpp/Ollama output crashes on Windows (cp1252).
+ Fails before the #2660 fix, passes after.
+ """
+ offenders = []
+ for node in _collect_text_mode_subprocess_calls():
+ enc = _kw(node, "encoding")
+ ok = isinstance(enc, ast.Constant) and enc.value == "utf-8"
+ if not ok:
+ offenders.append(node.lineno)
+
+ assert not offenders, (
+ "Text-mode subprocess call(s) in unsloth/save.py missing "
+ 'encoding="utf-8" (UnicodeDecodeError on Windows, #2660) at line(s): '
+ + ", ".join(map(str, sorted(offenders)))
+ )
+
+
+def test_utf8_replace_decodes_non_cp1252_subprocess_output():
+ """Document the failure and the fix with a real subprocess.
+
+ The child emits U+201D (right double quote), whose UTF-8 encoding
+ ``E2 80 9D`` contains byte 0x9D -- undefined in cp1252. Decoding the raw
+ bytes as cp1252 raises (the bug); the fix's kwargs read it cleanly.
+ """
+ # All-ASCII argv; the child builds the non-ASCII char itself so this is
+ # deterministic regardless of the parent's locale.
+ child = (
+ "import sys; "
+ "sys.stdout.buffer.write(('tensor ' + chr(0x201D) + ' x\\n').encode('utf-8'))"
+ )
+
+ raw = subprocess.run([sys.executable, "-c", child], capture_output = True).stdout
+ assert b"\x9d" in raw # precondition: output carries the cp1252-undefined byte
+
+ # Failing behaviour before the fix: cp1252 (the Windows default) cannot
+ # decode this output.
+ with pytest.raises(UnicodeDecodeError):
+ raw.decode("cp1252")
+
+ # Correct behaviour after the fix: the exact kwargs save.py now uses.
+ result = subprocess.run(
+ [sys.executable, "-c", child],
+ capture_output = True,
+ text = True,
+ encoding = "utf-8",
+ errors = "replace",
+ )
+ assert result.stdout.startswith("tensor ")
+ assert "”" in result.stdout
diff --git a/unsloth/save.py b/unsloth/save.py
index 629cbb9548..a6cc665d3e 100644
--- a/unsloth/save.py
+++ b/unsloth/save.py
@@ -186,6 +186,8 @@ def _quantize_q2_k_l(
command,
shell = False,
text = True,
+ encoding = "utf-8",
+ errors = "replace",
stdout = subprocess.PIPE,
stderr = subprocess.STDOUT,
bufsize = 1,
@@ -206,6 +208,8 @@ def _quantize_q2_k_l(
check = True,
capture_output = True,
text = True,
+ encoding = "utf-8",
+ errors = "replace",
)
except subprocess.CalledProcessError as e:
if print_output and hasattr(e, "stdout") and e.stdout:
@@ -1989,6 +1993,8 @@ def create_ollama_model(username: str, model_name: str, tag: str, modelfile_path
["curl", "http://localhost:11434"],
capture_output = True,
text = True,
+ encoding = "utf-8",
+ errors = "replace",
timeout = 3,
)
if init_check.returncode == 0:
@@ -2011,6 +2017,8 @@ def create_ollama_model(username: str, model_name: str, tag: str, modelfile_path
text = True,
bufsize = 1,
universal_newlines = True,
+ encoding = "utf-8",
+ errors = "replace",
)
for line in iter(process.stdout.readline, ""):
@@ -2031,6 +2039,8 @@ def push_to_ollama_hub(username: str, model_name: str, tag: str):
["curl", "http://localhost:11434"],
capture_output = True,
text = True,
+ encoding = "utf-8",
+ errors = "replace",
timeout = 3,
)
if init_check.returncode == 0:
@@ -2047,6 +2057,8 @@ def push_to_ollama_hub(username: str, model_name: str, tag: str):
text = True,
bufsize = 1,
universal_newlines = True,
+ encoding = "utf-8",
+ errors = "replace",
)
for line in iter(process.stdout.readline, ""):
@@ -2758,6 +2770,8 @@ def unsloth_convert_lora_to_ggml_and_push_to_hub(
stderr = subprocess.PIPE,
bufsize = 1,
universal_newlines = True,
+ encoding = "utf-8",
+ errors = "replace",
) as sp:
for line in sp.stdout:
print(line, end = "", flush = True)
@@ -2838,6 +2852,8 @@ def unsloth_convert_lora_to_ggml_and_save_locally(
stderr = subprocess.PIPE,
bufsize = 1,
universal_newlines = True,
+ encoding = "utf-8",
+ errors = "replace",
) as sp:
for line in sp.stdout:
print(line, end = "", flush = True)
From 514850fb327330448a285fa20e88834e1b9d035a Mon Sep 17 00:00:00 2001
From: Mohammad Hussian
Date: Fri, 12 Jun 2026 14:15:19 +0530
Subject: [PATCH 16/50] patch: fix EmptyLogits gathering in nested payloads and
Accelerate recursively_apply (#6092)
* Fix EmptyLogits gathering in nested structure and patch recursively_apply on accelerator module
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Wire EmptyLogits Accelerate patch into startup and fix find_device, pickling, tests for PR #6092
- Call patch_accelerate_recursively_apply() in _gpu_init.py so real imports
install it; previously it was only invoked by the tests
- Make both wrappers idempotent so repeated calls do not stack
- Rework find_device: skip EmptyLogits while still finding real tensors in any
order, keep returning None for tensor-free payloads (AlignDevicesHook relies
on None), fall back to PartialState().device only for sentinel-only payloads
- Give EmptyLogits stateless __reduce__ and drop the stomped pickle stubs on
EMPTY_LOGITS so debug mode gather_object works in real distributed runs
- Put test tensors on PartialState().device so the debug mode test also passes
on GPU machines, and add drift tests for startup wiring, idempotency and
find_device ordering
Verified on 2x B200: ACCELERATE_DEBUG_MODE=1 torchrun gather/broadcast/pad of
sentinel and mixed payloads all pass, training losses unchanged, full drift
suite 25/25.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Define EmptyLogits equality on the class for PR #6092
Gathered sentinel copies must compare equal in accelerate debug mode
regardless of whether the patched recursively_apply saw the sentinel first
in that process. Class body __eq__ requires restoring __hash__ explicitly.
Verified: 123 case simulation battery on accelerate 0.34.2 through latest,
2 process gloo CPU and NCCL GPU debug mode runs, drift suite 25/25.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han
---
tests/test_import_fixes_drift.py | 158 +++++++++++++++++++++++++++++++
unsloth/_gpu_init.py | 3 +
unsloth/import_fixes.py | 90 ++++++++++++++++++
unsloth/models/_utils.py | 17 ++++
4 files changed, 268 insertions(+)
diff --git a/tests/test_import_fixes_drift.py b/tests/test_import_fixes_drift.py
index 6c75a4b886..c9f971c891 100644
--- a/tests/test_import_fixes_drift.py
+++ b/tests/test_import_fixes_drift.py
@@ -586,6 +586,164 @@ def test_accelerate_utils_imports_module_present():
)
+def test_accelerate_recursively_apply_empty_logits_patch():
+ """Verify patch_accelerate_recursively_apply overrides recursively_apply to bypass EmptyLogits."""
+ pytest.importorskip("accelerate")
+
+ import accelerate.utils.operations as acc_ops
+ from unsloth.import_fixes import patch_accelerate_recursively_apply
+
+ class EmptyLogits:
+ pass
+
+ e = EmptyLogits()
+ patch_accelerate_recursively_apply()
+
+ res = acc_ops.recursively_apply(lambda x: x, e, error_on_other_type = True)
+ assert res is e
+
+
+def test_accelerate_gather_empty_logits_debug_mode_patch():
+ """Verify gather and broadcast bypass EmptyLogits when debug mode is enabled."""
+ pytest.importorskip("accelerate")
+ from accelerate.state import PartialState, DistributedType
+ import accelerate.utils.operations as acc_ops
+ from unsloth.import_fixes import patch_accelerate_recursively_apply
+ import unittest.mock as mock
+ import torch
+
+ class EmptyLogits:
+ pass
+
+ e = EmptyLogits()
+ patch_accelerate_recursively_apply()
+
+ # Enable debug mode and mock distributed state
+ state = PartialState()
+ orig_debug = state.debug
+ orig_dist_type = state.distributed_type
+ orig_num_processes = state.num_processes
+
+ state.debug = True
+ state.distributed_type = DistributedType.MULTI_GPU
+ state.num_processes = 2
+
+ # Mock gather_object to return [obj] * num_processes
+ def mock_gather_object(obj, *args, **kwargs):
+ return [obj] * state.num_processes
+
+ # Mock _gpu_gather to recursively apply replication of tensors
+ def mock_gpu_gather(tensor, *args, **kwargs):
+ def _gather_one(t):
+ if t.ndim == 0:
+ t = t.clone()[None]
+ return torch.cat([t] * state.num_processes, dim = 0)
+
+ return acc_ops.recursively_apply(_gather_one, tensor, error_on_other_type = True)
+
+ # Mock _gpu_broadcast to return data unchanged
+ def mock_gpu_broadcast(data, *args, **kwargs):
+ return data
+
+ try:
+ with (
+ mock.patch(
+ "accelerate.utils.operations.gather_object",
+ side_effect = mock_gather_object,
+ ),
+ mock.patch("accelerate.utils.operations._gpu_gather", side_effect = mock_gpu_gather),
+ mock.patch(
+ "accelerate.utils.operations._gpu_broadcast",
+ side_effect = mock_gpu_broadcast,
+ ),
+ ):
+ # 1. Top-level EmptyLogits should gather correctly (returns e)
+ res = acc_ops.gather(e)
+ assert res is e
+
+ # 2. Nested EmptyLogits alone
+ res_nested = acc_ops.gather([e])
+ assert isinstance(res_nested, list) and res_nested[0] is e
+
+ # 3. Mixed payload with real tensor and EmptyLogits
+ # Real tensor should be gathered (concatenated across processes).
+ # Tensors must live on state.device or the debug-mode device
+ # check fails on GPU machines.
+ real_tensor = torch.tensor([42], device = state.device)
+ payload = {"labels": real_tensor, "logits": e}
+ res_mixed = acc_ops.gather(payload)
+
+ assert isinstance(res_mixed, dict)
+ assert res_mixed["logits"] is e
+ # Since num_processes = 2, it should be gathered to [42, 42]
+ assert torch.equal(res_mixed["labels"], torch.tensor([42, 42], device = state.device))
+
+ # 4. Broadcast with EmptyLogits
+ res_broadcast = acc_ops.broadcast(e)
+ assert res_broadcast is e
+
+ # 5. Mixed payload with broadcast
+ res_broadcast_mixed = acc_ops.broadcast(payload)
+ assert isinstance(res_broadcast_mixed, dict)
+ assert res_broadcast_mixed["logits"] is e
+ assert torch.equal(res_broadcast_mixed["labels"], real_tensor)
+ finally:
+ state.debug = orig_debug
+ state.distributed_type = orig_dist_type
+ state.num_processes = orig_num_processes
+
+
+def test_accelerate_patch_is_idempotent():
+ """Calling patch_accelerate_recursively_apply twice must not stack wrappers."""
+ pytest.importorskip("accelerate")
+ import accelerate.utils.operations as acc_ops
+ from unsloth.import_fixes import patch_accelerate_recursively_apply
+
+ patch_accelerate_recursively_apply()
+ recursively_apply = acc_ops.recursively_apply
+ find_device = acc_ops.find_device
+ patch_accelerate_recursively_apply()
+ assert (
+ acc_ops.recursively_apply is recursively_apply
+ ), "DRIFT DETECTED: recursively_apply was wrapped twice."
+ assert acc_ops.find_device is find_device, "DRIFT DETECTED: find_device was wrapped twice."
+
+
+def test_accelerate_find_device_skips_empty_logits():
+ """find_device must search past EmptyLogits and keep None for tensor-free data."""
+ pytest.importorskip("accelerate")
+ import torch
+ import accelerate.utils.operations as acc_ops
+ from accelerate.state import PartialState
+ from unsloth.import_fixes import patch_accelerate_recursively_apply
+
+ class EmptyLogits:
+ pass
+
+ patch_accelerate_recursively_apply()
+ tensor = torch.tensor([1.0])
+ # Sentinel first must not stop the search before the real tensor
+ assert acc_ops.find_device({"logits": EmptyLogits(), "labels": tensor}) == tensor.device
+ # Tensor-free payloads without the sentinel keep returning None
+ # (AlignDevicesHook relies on None to skip output device moves)
+ assert acc_ops.find_device({"a": 1}) is None
+ # Sentinel-only payloads fall back to the current device so that
+ # debug mode find_device(...).type does not raise AttributeError
+ assert acc_ops.find_device(EmptyLogits()) == PartialState().device
+
+
+def test_accelerate_patch_wired_into_gpu_init():
+ """The patch must be installed at startup, not only importable."""
+ import pathlib
+ import unsloth.import_fixes as import_fixes
+
+ source = pathlib.Path(import_fixes.__file__).with_name("_gpu_init.py").read_text()
+ assert "patch_accelerate_recursively_apply()" in source, (
+ "DRIFT DETECTED: patch_accelerate_recursively_apply is defined but "
+ "never called in _gpu_init.py, so real imports never install it."
+ )
+
+
# ===========================================================================
# bitsandbytes -- ROCm arch / warp-size detection shape
# ===========================================================================
diff --git a/unsloth/_gpu_init.py b/unsloth/_gpu_init.py
index 5da1e27a9d..86f675c281 100644
--- a/unsloth/_gpu_init.py
+++ b/unsloth/_gpu_init.py
@@ -189,6 +189,7 @@ from .import_fixes import (
fix_trl_vllm_ascend,
fix_peft_transformers_weight_conversion_import,
patch_peft_weight_converter_compatibility,
+ patch_accelerate_recursively_apply,
)
fix_xformers_performance_issue()
@@ -217,6 +218,7 @@ disable_broken_wandb()
# build_peft_weight_mapping instead of being swallowed by its ImportError.
fix_peft_transformers_weight_conversion_import()
patch_peft_weight_converter_compatibility()
+patch_accelerate_recursively_apply()
del fix_xformers_performance_issue
del fix_vllm_aimv2_issue
@@ -240,6 +242,7 @@ del disable_torchcodec_if_broken
del disable_broken_wandb
del fix_peft_transformers_weight_conversion_import
del patch_peft_weight_converter_compatibility
+del patch_accelerate_recursively_apply
# Torch 2.4 has including_emulation
if DEVICE_TYPE == "cuda":
diff --git a/unsloth/import_fixes.py b/unsloth/import_fixes.py
index 695a6a577a..a100e11f0c 100644
--- a/unsloth/import_fixes.py
+++ b/unsloth/import_fixes.py
@@ -2584,3 +2584,93 @@ def maybe_set_windows_rocm_bnb_version():
"(detected from the installed bitsandbytes ROCm wheel on Windows)."
)
return version
+
+
+def patch_accelerate_recursively_apply():
+ """
+ Make Accelerate's recursive utilities tolerate Unsloth's EmptyLogits
+ sentinel. recursively_apply returns the sentinel unchanged instead of
+ raising TypeError, and find_device skips it while still finding real
+ tensors, falling back to PartialState().device only for sentinel-only
+ payloads. Both wrappers are idempotent and are propagated to every
+ already imported accelerate namespace.
+ """
+ try:
+ import accelerate.utils.operations as acc_ops
+ except Exception:
+ return
+
+ original_recursively_apply = getattr(acc_ops, "recursively_apply", None)
+ if original_recursively_apply is not None and not getattr(
+ original_recursively_apply, "__unsloth_patched__", False
+ ):
+
+ @functools.wraps(original_recursively_apply)
+ def _patched_recursively_apply(func, data, *args, **kwargs):
+ if type(data).__name__ == "EmptyLogits":
+ cls = type(data)
+ if cls.__eq__ is object.__eq__:
+ # Debug mode compares gathered metadata across ranks with ==
+ cls.__eq__ = lambda self, other: type(other).__name__ == "EmptyLogits"
+ return data
+ return original_recursively_apply(func, data, *args, **kwargs)
+
+ _patched_recursively_apply.__unsloth_patched__ = True
+
+ for mod_name, mod in tuple(sys.modules.items()):
+ if mod_name.startswith("accelerate") and mod is not None:
+ if getattr(mod, "recursively_apply", None) is original_recursively_apply:
+ try:
+ setattr(mod, "recursively_apply", _patched_recursively_apply)
+ except Exception:
+ pass
+
+ original_find_device = getattr(acc_ops, "find_device", None)
+ if original_find_device is not None and not getattr(
+ original_find_device, "__unsloth_patched__", False
+ ):
+ from collections.abc import Mapping
+
+ @functools.wraps(original_find_device)
+ def _patched_find_device(data):
+ import torch
+
+ found_sentinel = False
+
+ def _search(obj):
+ nonlocal found_sentinel
+ if type(obj).__name__ == "EmptyLogits":
+ found_sentinel = True
+ elif isinstance(obj, Mapping):
+ for value in obj.values():
+ device = _search(value)
+ if device is not None:
+ return device
+ elif isinstance(obj, (tuple, list)):
+ for value in obj:
+ device = _search(value)
+ if device is not None:
+ return device
+ elif isinstance(obj, torch.Tensor):
+ return obj.device
+ return None
+
+ device = _search(data)
+ if device is None and found_sentinel:
+ # Debug mode calls find_device(...).type on gather/broadcast inputs
+ try:
+ from accelerate.state import PartialState
+ return PartialState().device
+ except Exception:
+ pass
+ return device
+
+ _patched_find_device.__unsloth_patched__ = True
+
+ for mod_name, mod in tuple(sys.modules.items()):
+ if mod_name.startswith("accelerate") and mod is not None:
+ if getattr(mod, "find_device", None) is original_find_device:
+ try:
+ setattr(mod, "find_device", _patched_find_device)
+ except Exception:
+ pass
diff --git a/unsloth/models/_utils.py b/unsloth/models/_utils.py
index 2f4e3a069e..6024a02c2c 100644
--- a/unsloth/models/_utils.py
+++ b/unsloth/models/_utils.py
@@ -2703,6 +2703,16 @@ class EmptyLogits:
def __str__(self):
return LOGITS_ERROR_STRING
+ def __reduce__(self):
+ # Stateless pickling so gather_object works on the sentinel
+ return (type(self), ())
+
+ def __eq__(self, other):
+ # Gathered copies must compare equal in accelerate debug mode
+ return type(other).__name__ == "EmptyLogits"
+
+ __hash__ = object.__hash__
+
EMPTY_LOGITS = EmptyLogits()
functions = dir(torch.Tensor)
@@ -2713,6 +2723,13 @@ for j, function in enumerate(functions):
exec(f"EMPTY_LOGITS.{function} = raise_{j}", globals(), locals())
except:
continue
+# The loop above stomps pickle hooks with stubs returning None, which breaks
+# gather_object on EMPTY_LOGITS in distributed runs. Restore default pickling.
+for function in ("__reduce__", "__reduce_ex__", "__getstate__", "__setstate__"):
+ try:
+ delattr(EMPTY_LOGITS, function)
+ except Exception:
+ pass
def validate_loftq_config(loftq_config, lora_dropout, bias, init_lora_weights, model):
From de0c5a2f09ba8230987dc6a580fe77c1a6da9bba Mon Sep 17 00:00:00 2001
From: Ban <3637117+Ban921@users.noreply.github.com>
Date: Fri, 12 Jun 2026 16:50:45 +0800
Subject: [PATCH 17/50] Studio: show Apple GPU temperature and power in the GPU
monitor (macOS) (#6187)
* Studio: show Apple GPU temperature and power in the GPU monitor (macOS)
The GPU monitor on Apple Silicon always showed -- for Temperature and
Power: the MLX branch of get_gpu_utilization() hardcoded None because
ioreg's AGXAccelerator PerformanceStatistics carries neither metric.
Add utils/hardware/apple.py, mirroring macmon's no-sudo approach:
- Temperature: average of the AppleSMC "Tg*" float keys via the
AppleSMCKeysEndpoint user client (ctypes/IOKit, macOS 14+).
- Power: IOReport "Energy Model" group, "GPU Energy" channels; each
poll diffs the energy counter against the previous poll's sample, so
the value is the average wattage over the polling window. The first
poll only sets the baseline and returns None.
Both readers latch to None on first failure and never raise, so
non-Mac platforms and locked-down hosts keep the previous behavior.
* Sample IOReport with the subscribed channels descriptor for PR #6187
IOReportCreateSubscription writes the channel descriptor that later samples
must use; sampling with the original requested group can return no Energy
Model entries on hosts that normalize the channel set, leaving power_draw_w
null after the baseline. Use the subscribed descriptor (matching macmon) and
fall back to the requested channels if the OS leaves it unset.
---------
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: danielhanchen <23090290+danielhanchen@users.noreply.github.com>
---
.../backend/tests/test_apple_gpu_sensors.py | 74 +++
studio/backend/utils/hardware/apple.py | 430 ++++++++++++++++++
studio/backend/utils/hardware/hardware.py | 6 +-
3 files changed, 508 insertions(+), 2 deletions(-)
create mode 100644 studio/backend/tests/test_apple_gpu_sensors.py
create mode 100644 studio/backend/utils/hardware/apple.py
diff --git a/studio/backend/tests/test_apple_gpu_sensors.py b/studio/backend/tests/test_apple_gpu_sensors.py
new file mode 100644
index 0000000000..50947d1380
--- /dev/null
+++ b/studio/backend/tests/test_apple_gpu_sensors.py
@@ -0,0 +1,74 @@
+# SPDX-License-Identifier: AGPL-3.0-only
+# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
+
+"""Tests for Apple Silicon GPU sensors (SMC temperature + IOReport power)."""
+
+import ctypes
+import platform
+import time
+
+import pytest
+
+from utils.hardware import apple
+
+_IS_APPLE_SILICON = platform.system() == "Darwin" and platform.machine() == "arm64"
+
+
+class TestFourcc:
+ def test_roundtrip(self):
+ for key in ("#KEY", "Tg0D", "flt "):
+ assert apple._fourcc_str(apple._fourcc(key)) == key
+
+ def test_known_value(self):
+ # "flt " FourCC, same constant macmon uses.
+ assert apple._fourcc("flt ") == 1718383648
+
+
+class TestWatts:
+ def test_millijoules(self):
+ assert apple._watts(2000, "mJ", 2.0) == pytest.approx(1.0)
+
+ def test_microjoules(self):
+ assert apple._watts(5_000_000, "uJ", 1.0) == pytest.approx(5.0)
+
+ def test_nanojoules(self):
+ assert apple._watts(1_500_000_000, "nJ", 1.0) == pytest.approx(1.5)
+
+ def test_unknown_unit_returns_none(self):
+ assert apple._watts(1000, "J", 1.0) is None
+
+ def test_zero_elapsed_returns_none(self):
+ assert apple._watts(1000, "mJ", 0.0) is None
+
+
+class TestAverageValidTemps:
+ def test_averages_and_rounds(self):
+ assert apple._average_valid_temps([40.0, 50.0, 60.05]) == 50.0
+
+ def test_filters_invalid(self):
+ assert apple._average_valid_temps([-1.0, 0.0, 151.0, 42.0]) == 42.0
+
+ def test_empty_returns_none(self):
+ assert apple._average_valid_temps([]) is None
+ assert apple._average_valid_temps([0.0, 200.0]) is None
+
+
+class TestSmcStructLayout:
+ def test_key_data_matches_smc_protocol_size(self):
+ # The AppleSMC user client rejects calls whose struct size differs.
+ assert ctypes.sizeof(apple._SMCKeyData) == 80
+
+
+@pytest.mark.skipif(not _IS_APPLE_SILICON, reason = "requires Apple Silicon")
+class TestLiveSensors:
+ def test_gpu_temperature_in_plausible_range(self):
+ temp = apple.read_gpu_temperature_c()
+ assert temp is not None
+ assert 0.0 < temp <= 150.0
+
+ def test_gpu_power_after_baseline(self):
+ apple.read_gpu_power_w() # first call only sets the baseline
+ time.sleep(0.3)
+ power = apple.read_gpu_power_w()
+ assert power is not None
+ assert power >= 0.0
diff --git a/studio/backend/utils/hardware/apple.py b/studio/backend/utils/hardware/apple.py
new file mode 100644
index 0000000000..3f14af60ad
--- /dev/null
+++ b/studio/backend/utils/hardware/apple.py
@@ -0,0 +1,430 @@
+# SPDX-License-Identifier: AGPL-3.0-only
+# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
+
+"""Apple Silicon GPU temperature and power -- no sudo required.
+
+Mirrors macmon's approach (https://github.com/vladkens/macmon):
+ * Temperature: average of the AppleSMC "Tg*" float keys (available since
+ macOS 14; on older systems the keys are absent and this returns None).
+ * Power: IOReport "Energy Model" group, "GPU Energy" channel. Each poll
+ diffs the energy counter against the previous poll's sample, so the
+ result is the average wattage over the polling window. The first poll
+ only sets the baseline and returns None.
+
+Public API (never raises; returns None when sensors are unavailable):
+ read_gpu_temperature_c()
+ read_gpu_power_w()
+"""
+
+import ctypes
+import struct
+import time
+from typing import Iterable, Optional
+
+from loggers import get_logger
+
+logger = get_logger(__name__)
+
+_IOKIT_PATH = "/System/Library/Frameworks/IOKit.framework/IOKit"
+_CF_PATH = "/System/Library/Frameworks/CoreFoundation.framework/CoreFoundation"
+_IOREPORT_PATH = "/usr/lib/libIOReport.dylib"
+
+# AppleSMC user-client protocol (same constants as macmon / SMCKit).
+_SMC_SELECTOR_HANDLE_EVENT = 2
+_SMC_CMD_READ_BYTES = 5
+_SMC_CMD_KEY_AT_INDEX = 8
+_SMC_CMD_KEY_INFO = 9
+
+_MAX_VALID_TEMP_C = 150.0
+_CF_STRING_ENCODING_UTF8 = 0x08000100
+_ENERGY_UNIT_DIVISORS = {"mJ": 1e3, "uJ": 1e6, "nJ": 1e9}
+
+
+# ========== Pure helpers ==========
+
+
+def _fourcc(key: str) -> int:
+ """Encode a 4-char SMC key/type name as a big-endian integer."""
+ return int.from_bytes(key.encode("ascii"), "big")
+
+
+def _fourcc_str(value: int) -> str:
+ return value.to_bytes(4, "big").decode("ascii", errors = "replace")
+
+
+def _watts(energy: int, unit: str, elapsed_s: float) -> Optional[float]:
+ """Convert an IOReport energy counter delta into average watts."""
+ divisor = _ENERGY_UNIT_DIVISORS.get(unit.strip())
+ if divisor is None or elapsed_s <= 0:
+ return None
+ return energy / divisor / elapsed_s
+
+
+def _average_valid_temps(values: Iterable[float]) -> Optional[float]:
+ valid = [v for v in values if 0.0 < v <= _MAX_VALID_TEMP_C]
+ if not valid:
+ return None
+ return round(sum(valid) / len(valid), 1)
+
+
+def _is_gpu_energy_channel(name: str) -> bool:
+ # Exact "GPU Energy" plus "DIE_N_GPU Energy" on Ultra chips; the separate
+ # "GPU SRAM*" channels are not GPU core power.
+ return name.endswith("GPU Energy") and "SRAM" not in name
+
+
+# ========== AppleSMC structs (layout must match the kernel exactly) ==========
+
+
+class _SMCKeyDataVers(ctypes.Structure):
+ _fields_ = [
+ ("major", ctypes.c_uint8),
+ ("minor", ctypes.c_uint8),
+ ("build", ctypes.c_uint8),
+ ("reserved", ctypes.c_uint8),
+ ("release", ctypes.c_uint16),
+ ]
+
+
+class _SMCPLimitData(ctypes.Structure):
+ _fields_ = [
+ ("version", ctypes.c_uint16),
+ ("length", ctypes.c_uint16),
+ ("cpu_p_limit", ctypes.c_uint32),
+ ("gpu_p_limit", ctypes.c_uint32),
+ ("mem_p_limit", ctypes.c_uint32),
+ ]
+
+
+class _SMCKeyInfo(ctypes.Structure):
+ _fields_ = [
+ ("data_size", ctypes.c_uint32),
+ ("data_type", ctypes.c_uint32),
+ ("data_attributes", ctypes.c_uint8),
+ ]
+
+
+class _SMCKeyData(ctypes.Structure):
+ _fields_ = [
+ ("key", ctypes.c_uint32),
+ ("vers", _SMCKeyDataVers),
+ ("p_limit_data", _SMCPLimitData),
+ ("key_info", _SMCKeyInfo),
+ ("result", ctypes.c_uint8),
+ ("status", ctypes.c_uint8),
+ ("data8", ctypes.c_uint8),
+ ("data32", ctypes.c_uint32),
+ ("bytes", ctypes.c_uint8 * 32),
+ ]
+
+
+# ========== Library loaders ==========
+
+
+def _load_iokit() -> ctypes.CDLL:
+ iokit = ctypes.CDLL(_IOKIT_PATH)
+ iokit.IOServiceMatching.restype = ctypes.c_void_p
+ iokit.IOServiceMatching.argtypes = [ctypes.c_char_p]
+ iokit.IOServiceGetMatchingServices.argtypes = [
+ ctypes.c_uint32,
+ ctypes.c_void_p,
+ ctypes.POINTER(ctypes.c_uint32),
+ ]
+ iokit.IOIteratorNext.restype = ctypes.c_uint32
+ iokit.IOIteratorNext.argtypes = [ctypes.c_uint32]
+ iokit.IORegistryEntryGetName.argtypes = [ctypes.c_uint32, ctypes.c_char_p]
+ iokit.IOServiceOpen.argtypes = [
+ ctypes.c_uint32,
+ ctypes.c_uint32,
+ ctypes.c_uint32,
+ ctypes.POINTER(ctypes.c_uint32),
+ ]
+ iokit.IOObjectRelease.argtypes = [ctypes.c_uint32]
+ iokit.IOConnectCallStructMethod.argtypes = [
+ ctypes.c_uint32,
+ ctypes.c_uint32,
+ ctypes.c_void_p,
+ ctypes.c_size_t,
+ ctypes.c_void_p,
+ ctypes.POINTER(ctypes.c_size_t),
+ ]
+ return iokit
+
+
+def _load_cf() -> ctypes.CDLL:
+ cf = ctypes.CDLL(_CF_PATH)
+ cf.CFStringCreateWithCString.restype = ctypes.c_void_p
+ cf.CFStringCreateWithCString.argtypes = [ctypes.c_void_p, ctypes.c_char_p, ctypes.c_uint32]
+ cf.CFStringGetCString.restype = ctypes.c_bool
+ cf.CFStringGetCString.argtypes = [
+ ctypes.c_void_p,
+ ctypes.c_char_p,
+ ctypes.c_long,
+ ctypes.c_uint32,
+ ]
+ cf.CFRelease.argtypes = [ctypes.c_void_p]
+ cf.CFDictionaryGetValue.restype = ctypes.c_void_p
+ cf.CFDictionaryGetValue.argtypes = [ctypes.c_void_p, ctypes.c_void_p]
+ cf.CFArrayGetCount.restype = ctypes.c_long
+ cf.CFArrayGetCount.argtypes = [ctypes.c_void_p]
+ cf.CFArrayGetValueAtIndex.restype = ctypes.c_void_p
+ cf.CFArrayGetValueAtIndex.argtypes = [ctypes.c_void_p, ctypes.c_long]
+ return cf
+
+
+def _load_ioreport() -> ctypes.CDLL:
+ ior = ctypes.CDLL(_IOREPORT_PATH)
+ ior.IOReportCopyChannelsInGroup.restype = ctypes.c_void_p
+ ior.IOReportCopyChannelsInGroup.argtypes = [
+ ctypes.c_void_p,
+ ctypes.c_void_p,
+ ctypes.c_uint64,
+ ctypes.c_uint64,
+ ctypes.c_uint64,
+ ]
+ ior.IOReportCreateSubscription.restype = ctypes.c_void_p
+ ior.IOReportCreateSubscription.argtypes = [
+ ctypes.c_void_p,
+ ctypes.c_void_p,
+ ctypes.POINTER(ctypes.c_void_p),
+ ctypes.c_uint64,
+ ctypes.c_void_p,
+ ]
+ ior.IOReportCreateSamples.restype = ctypes.c_void_p
+ ior.IOReportCreateSamples.argtypes = [ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p]
+ ior.IOReportCreateSamplesDelta.restype = ctypes.c_void_p
+ ior.IOReportCreateSamplesDelta.argtypes = [ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p]
+ ior.IOReportChannelGetChannelName.restype = ctypes.c_void_p
+ ior.IOReportChannelGetChannelName.argtypes = [ctypes.c_void_p]
+ ior.IOReportChannelGetUnitLabel.restype = ctypes.c_void_p
+ ior.IOReportChannelGetUnitLabel.argtypes = [ctypes.c_void_p]
+ ior.IOReportSimpleGetIntegerValue.restype = ctypes.c_int64
+ ior.IOReportSimpleGetIntegerValue.argtypes = [ctypes.c_void_p, ctypes.c_int32]
+ return ior
+
+
+def _cfstr(cf: ctypes.CDLL, text: str) -> int:
+ return cf.CFStringCreateWithCString(None, text.encode("utf-8"), _CF_STRING_ENCODING_UTF8)
+
+
+def _from_cfstr(cf: ctypes.CDLL, ref: Optional[int]) -> str:
+ if not ref:
+ return ""
+ buf = ctypes.create_string_buffer(128)
+ if not cf.CFStringGetCString(ref, buf, len(buf), _CF_STRING_ENCODING_UTF8):
+ return ""
+ return buf.value.decode("utf-8", errors = "replace").strip()
+
+
+# ========== SMC connection (GPU temperature) ==========
+
+
+class _SMCConnection:
+ """Connection to AppleSMCKeysEndpoint; discovers "Tg*" GPU temp keys once."""
+
+ def __init__(self):
+ self._iokit = _load_iokit()
+ self._conn = self._open()
+ self._key_info_cache: dict[int, _SMCKeyInfo] = {}
+ self.gpu_keys = [
+ key
+ for key in self._all_keys()
+ if key.startswith("Tg") and self.read_float(key) is not None
+ ]
+
+ def _open(self) -> int:
+ iterator = ctypes.c_uint32(0)
+ matching = self._iokit.IOServiceMatching(b"AppleSMC")
+ if self._iokit.IOServiceGetMatchingServices(0, matching, ctypes.byref(iterator)) != 0:
+ raise OSError("AppleSMC service not found")
+ try:
+ conn = self._open_keys_endpoint(iterator.value)
+ finally:
+ self._iokit.IOObjectRelease(iterator.value)
+ if conn is None:
+ raise OSError("AppleSMCKeysEndpoint not found")
+ return conn
+
+ def _open_keys_endpoint(self, iterator: int) -> Optional[int]:
+ task = ctypes.CDLL(None).mach_task_self()
+ while device := self._iokit.IOIteratorNext(iterator):
+ name = ctypes.create_string_buffer(128)
+ self._iokit.IORegistryEntryGetName(device, name)
+ if name.value != b"AppleSMCKeysEndpoint":
+ self._iokit.IOObjectRelease(device)
+ continue
+ conn = ctypes.c_uint32(0)
+ status = self._iokit.IOServiceOpen(device, task, 0, ctypes.byref(conn))
+ self._iokit.IOObjectRelease(device)
+ if status != 0:
+ raise OSError(f"IOServiceOpen(AppleSMCKeysEndpoint) failed: {status}")
+ return conn.value
+ return None
+
+ def _call(self, ival: _SMCKeyData) -> _SMCKeyData:
+ oval = _SMCKeyData()
+ olen = ctypes.c_size_t(ctypes.sizeof(_SMCKeyData))
+ status = self._iokit.IOConnectCallStructMethod(
+ self._conn,
+ _SMC_SELECTOR_HANDLE_EVENT,
+ ctypes.byref(ival),
+ ctypes.sizeof(_SMCKeyData),
+ ctypes.byref(oval),
+ ctypes.byref(olen),
+ )
+ if status != 0:
+ raise OSError(f"IOConnectCallStructMethod failed: {status}")
+ if oval.result != 0:
+ raise OSError(f"SMC result code: {oval.result}")
+ return oval
+
+ def _read_key_info(self, key_id: int) -> _SMCKeyInfo:
+ cached = self._key_info_cache.get(key_id)
+ if cached is not None:
+ return cached
+ oval = self._call(_SMCKeyData(key = key_id, data8 = _SMC_CMD_KEY_INFO))
+ self._key_info_cache[key_id] = oval.key_info
+ return oval.key_info
+
+ def _read_bytes(self, key: str) -> Optional[bytes]:
+ try:
+ key_id = _fourcc(key)
+ info = self._read_key_info(key_id)
+ oval = self._call(_SMCKeyData(key = key_id, data8 = _SMC_CMD_READ_BYTES, key_info = info))
+ return bytes(oval.bytes[: info.data_size])
+ except OSError:
+ return None
+
+ def read_float(self, key: str) -> Optional[float]:
+ try:
+ info = self._read_key_info(_fourcc(key))
+ except OSError:
+ return None
+ if info.data_size != 4 or info.data_type != _fourcc("flt "):
+ return None
+ data = self._read_bytes(key)
+ if data is None or len(data) != 4:
+ return None
+ return struct.unpack(" Optional[str]:
+ try:
+ oval = self._call(_SMCKeyData(data8 = _SMC_CMD_KEY_AT_INDEX, data32 = index))
+ return oval.key.to_bytes(4, "big").decode("ascii")
+ except (OSError, UnicodeDecodeError):
+ return None
+
+ def _all_keys(self) -> list[str]:
+ count_bytes = self._read_bytes("#KEY")
+ if count_bytes is None or len(count_bytes) != 4:
+ return []
+ count = int.from_bytes(count_bytes, "big")
+ names = (self._key_name_at(i) for i in range(count))
+ return [name for name in names if name is not None]
+
+ def gpu_temperature_c(self) -> Optional[float]:
+ readings = (self.read_float(key) for key in self.gpu_keys)
+ return _average_valid_temps(value for value in readings if value is not None)
+
+
+# ========== IOReport subscription (GPU power) ==========
+
+
+class _IOReportEnergy:
+ """Persistent subscription to the "Energy Model" group for GPU wattage."""
+
+ def __init__(self):
+ self._cf = _load_cf()
+ self._ior = _load_ioreport()
+ self._channels = self._ior.IOReportCopyChannelsInGroup(
+ _cfstr(self._cf, "Energy Model"), None, 0, 0, 0
+ )
+ if not self._channels:
+ raise OSError("IOReport 'Energy Model' channel group unavailable")
+ subscribed = ctypes.c_void_p()
+ self._sub = self._ior.IOReportCreateSubscription(
+ None, self._channels, ctypes.byref(subscribed), 0, None
+ )
+ if not self._sub:
+ raise OSError("IOReportCreateSubscription failed")
+ # Sample with the channels IOReport subscribes us to, not the requested
+ # group (matches macmon); fall back if the OS leaves it unset.
+ self._sample_channels = subscribed if subscribed else self._channels
+ self._channels_key = _cfstr(self._cf, "IOReportChannels")
+ self._prev: Optional[tuple[int, float]] = None # (sample ref, monotonic s)
+
+ def gpu_power_w(self) -> Optional[float]:
+ sample = self._ior.IOReportCreateSamples(self._sub, self._sample_channels, None)
+ if not sample:
+ return None
+ now = time.monotonic()
+ prev, self._prev = self._prev, (sample, now)
+ if prev is None:
+ return None
+ prev_sample, prev_time = prev
+ delta = self._ior.IOReportCreateSamplesDelta(prev_sample, sample, None)
+ self._cf.CFRelease(prev_sample)
+ if not delta:
+ return None
+ try:
+ return self._gpu_watts_from_delta(delta, now - prev_time)
+ finally:
+ self._cf.CFRelease(delta)
+
+ def _gpu_watts_from_delta(self, delta: int, elapsed_s: float) -> Optional[float]:
+ items = self._cf.CFDictionaryGetValue(delta, self._channels_key)
+ if not items:
+ return None
+ total: Optional[float] = None
+ for i in range(self._cf.CFArrayGetCount(items)):
+ item = self._cf.CFArrayGetValueAtIndex(items, i)
+ name = _from_cfstr(self._cf, self._ior.IOReportChannelGetChannelName(item))
+ if not _is_gpu_energy_channel(name):
+ continue
+ unit = _from_cfstr(self._cf, self._ior.IOReportChannelGetUnitLabel(item))
+ energy = self._ior.IOReportSimpleGetIntegerValue(item, 0)
+ watts = _watts(energy, unit, elapsed_s)
+ if watts is not None:
+ total = (total or 0.0) + watts
+ return round(total, 1) if total is not None else None
+
+
+# ========== Public API (module singletons, failure-latched) ==========
+
+_smc: Optional[_SMCConnection] = None
+_smc_failed = False
+_energy: Optional[_IOReportEnergy] = None
+_energy_failed = False
+
+
+def read_gpu_temperature_c() -> Optional[float]:
+ """Average Apple GPU die temperature in degrees C, or None if unavailable."""
+ global _smc, _smc_failed
+ if _smc_failed:
+ return None
+ try:
+ if _smc is None:
+ _smc = _SMCConnection()
+ return _smc.gpu_temperature_c()
+ except Exception as e:
+ _smc_failed = True
+ logger.warning("Apple SMC GPU temperature unavailable: %s", e)
+ return None
+
+
+def read_gpu_power_w() -> Optional[float]:
+ """Average GPU power in watts since the previous call, or None.
+
+ The first call establishes the baseline sample and returns None.
+ """
+ global _energy, _energy_failed
+ if _energy_failed:
+ return None
+ try:
+ if _energy is None:
+ _energy = _IOReportEnergy()
+ return _energy.gpu_power_w()
+ except Exception as e:
+ _energy_failed = True
+ logger.warning("Apple IOReport GPU power unavailable: %s", e)
+ return None
diff --git a/studio/backend/utils/hardware/hardware.py b/studio/backend/utils/hardware/hardware.py
index 893d0364f7..86dfa8a93f 100644
--- a/studio/backend/utils/hardware/hardware.py
+++ b/studio/backend/utils/hardware/hardware.py
@@ -714,17 +714,19 @@ def get_gpu_utilization() -> Dict[str, Any]:
except Exception:
pass
+ from . import apple
+
return {
"available": True,
"backend": device.value,
"gpu_utilization_pct": agx.get("utilization_pct") if agx else None,
- "temperature_c": None,
+ "temperature_c": apple.read_gpu_temperature_c(),
"vram_used_gb": round(vram_used_gb, 2),
"vram_total_gb": round(total_gb, 2),
"vram_utilization_pct": (
round((vram_used_gb / total_gb) * 100, 1) if total_gb > 0 else None
),
- "power_draw_w": None,
+ "power_draw_w": apple.read_gpu_power_w(),
"power_limit_w": None,
"power_utilization_pct": None,
}
From 7f2986a413c0ddd2bbc077b58dbd82a750337e45 Mon Sep 17 00:00:00 2001
From: oobabooga
Date: Fri, 12 Jun 2026 05:55:26 -0300
Subject: [PATCH 18/50] Studio: Add inline confirmation (Allow/Always
allow/Deny) for tool calls (#5869)
* Studio: Add inline confirmation (Allow/Always allow/Deny) for tool calls
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix race in tool-call confirmation gate
* Studio: gate built-in tool calls and harden the confirmation handshake
The Allow / Always allow / Deny controls only lived in the fallback tool
card, but the built-in tools (web search, python, terminal, code
execution, image generation) render with their own components and so
never showed the buttons. Those calls paused after tool_start with no way
to approve them, hanging until the 1 hour timeout. Only MCP tools, which
use the fallback renderer, actually worked.
Render the controls for every tool card by wrapping each registered tool
component (and the fallback) in thread.tsx with a shared
ToolConfirmationControls, so the gate applies uniformly.
Also make the handshake robust:
- The gate keys on a per-call approval_id minted by the backend and
echoed in tool_start, instead of session_id alone, so a stale or
concurrent confirmation can no longer resolve the wrong call.
- The approval slot is registered before tool_start is yielded, closing
the race where a fast click or an auto "Always allow" could reach the
backend before the waiter existed.
- The frontend resolves with the same session id the request was sent
with (plus the approval_id), fixing the new-thread mismatch where the
confirmation targeted a different session than the blocked stream.
- The confirm endpoint returns {resolved}; the UI keeps the buttons and
shows a retry hint until the backend confirms a match, instead of
hiding them on a failed or mistargeted post.
- The gate runs after the disabled-tool and duplicate-call checks, so a
call that will not execute is not put up for approval. A denied call is
still excluded from duplicate detection, so re-issuing and approving it
works.
- "Always allow" is scoped per session to match the backend gate.
Add backend tests for the approval registry, the SSE no-deadlock
handshake, and the loop integration (allow, deny, disabled, duplicate,
re-issue after deny).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Move "Confirm tool calls" to the Tools section
* Studio: Keep tool group open while a tool call awaits confirmation
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix tool confirmation session scope for PR #5869
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix confirmation follow-ups for PR #5869
* Apply pre-commit formatting for PR #5869
* Fix confirmation cleanup for PR #5869
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Harden confirmation lookups for PR #5869
* Studio: make the tool-call confirmation decision immutable
resolve_tool_decision accepted a second confirmation for the same approval_id
and overwrote slot["decision"] in the window before the waiter reads it and
pops the slot, so a duplicate or out-of-order POST could flip an Allow to Deny
(and returned a misleading resolved:true). Reject once the slot's event is
already set so the first decision wins. Adds a regression test.
* Fix/adjust tool confirmations for PR #5869
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han
Co-authored-by: wasimysaid
---
studio/backend/core/inference/llama_cpp.py | 71 ++++-
studio/backend/core/inference/orchestrator.py | 2 +
.../core/inference/safetensors_agentic.py | 53 +++-
studio/backend/models/inference.py | 10 +
studio/backend/routes/inference.py | 78 +++++-
studio/backend/state/tool_approvals.py | 139 ++++++++++
.../backend/tests/test_anthropic_messages.py | 13 +
.../backend/tests/test_llama_cpp_tool_loop.py | 171 ++++++++++++
.../tests/test_openai_tool_passthrough.py | 101 +++++++
.../tests/test_safetensors_tool_loop.py | 49 +++-
studio/backend/tests/test_tool_approvals.py | 261 ++++++++++++++++++
.../backend/tests/test_tool_confirm_loop.py | 170 ++++++++++++
.../backend/tests/test_tool_confirm_stream.py | 219 +++++++++++++++
.../src/components/assistant-ui/thread.tsx | 30 +-
.../tool-confirmation-controls.tsx | 157 +++++++++++
.../components/assistant-ui/tool-fallback.tsx | 3 +
.../components/assistant-ui/tool-group.tsx | 28 +-
.../src/features/chat/api/chat-adapter.ts | 46 ++-
.../src/features/chat/api/chat-api.ts | 25 ++
.../src/features/chat/chat-settings-sheet.tsx | 25 ++
.../chat/stores/chat-runtime-store.ts | 70 +++++
.../frontend/src/features/chat/types/api.ts | 2 +
22 files changed, 1694 insertions(+), 29 deletions(-)
create mode 100644 studio/backend/state/tool_approvals.py
create mode 100644 studio/backend/tests/test_tool_approvals.py
create mode 100644 studio/backend/tests/test_tool_confirm_loop.py
create mode 100644 studio/backend/tests/test_tool_confirm_stream.py
create mode 100644 studio/frontend/src/components/assistant-ui/tool-confirmation-controls.tsx
diff --git a/studio/backend/core/inference/llama_cpp.py b/studio/backend/core/inference/llama_cpp.py
index 80b4862aa8..8cf37ed9ec 100644
--- a/studio/backend/core/inference/llama_cpp.py
+++ b/studio/backend/core/inference/llama_cpp.py
@@ -60,6 +60,13 @@ from core.inference.tool_loop_controller import (
ToolLoopController,
tool_event_provenance,
)
+from state.tool_approvals import (
+ TOOL_REJECTED_MESSAGE,
+ abort_tool_decision,
+ begin_tool_decision,
+ new_approval_id,
+ wait_tool_decision,
+)
logger = get_logger(__name__)
@@ -2192,8 +2199,9 @@ class LlamaCppBackend:
else None
),
(
- f"{general['general.organization']}/"
- f"{general['general.basename']}".replace(" ", "-")
+ f"{general['general.organization']}/{general['general.basename']}".replace(
+ " ", "-"
+ )
if general.get("general.organization") and general.get("general.basename")
else None
),
@@ -3748,7 +3756,7 @@ class LlamaCppBackend:
)
logger.info(
- f"llama-server ready on port {self._port} " f"for model '{model_identifier}'"
+ f"llama-server ready on port {self._port} for model '{model_identifier}'"
)
# Probe outside _lock (interruptible by /unload); init inside.
@@ -4373,7 +4381,7 @@ class LlamaCppBackend:
proc.kill()
logger.info(
- f"Killed orphaned llama-server process " f"(pid={proc.info['pid']})"
+ f"Killed orphaned llama-server process (pid={proc.info['pid']})"
)
except (
psutil.NoSuchProcess,
@@ -4910,6 +4918,7 @@ class LlamaCppBackend:
rag_scope: Optional[dict] = None,
seed: Optional[int] = None,
disable_parallel_tool_use: bool = False,
+ confirm_tool_calls: bool = False,
) -> Generator[dict, None, None]:
"""
Agentic loop: let the model call tools, execute them, and continue.
@@ -4928,7 +4937,7 @@ class LlamaCppBackend:
# Forced first-pass RAG so a doc question doesn't lose to web_search. Emits
# the same tool card + citations a real call would.
- _auto = build_rag_autoinject(conversation, rag_scope)
+ _auto = None if confirm_tool_calls else build_rag_autoinject(conversation, rag_scope)
if _auto:
for _ev in _auto["events"]:
yield _ev
@@ -5077,7 +5086,7 @@ class LlamaCppBackend:
if response.status_code != 200:
error_body = response.read().decode()
raise RuntimeError(
- f"llama-server returned {response.status_code}: " f"{error_body}"
+ f"llama-server returned {response.status_code}: {error_body}"
)
raw_buf = ""
@@ -5488,8 +5497,7 @@ class LlamaCppBackend:
force = True,
)
logger.info(
- f"Safety net: parsed {len(tool_calls)} tool call(s) "
- f"from streamed content"
+ f"Safety net: parsed {len(tool_calls)} tool call(s) from streamed content"
)
else:
# ── DRAINING path: assemble tool_calls ──
@@ -5609,8 +5617,51 @@ class LlamaCppBackend:
decision.as_assistant_tool_call()
)
- yield {"type": "status", "text": decision.status_text}
- yield decision.tool_start_event()
+ needs_confirm = bool(confirm_tool_calls)
+ approval_id = new_approval_id() if needs_confirm else ""
+ decision_slot = (
+ begin_tool_decision(session_id, approval_id) if needs_confirm else None
+ )
+ start_event = decision.tool_start_event()
+ start_event["approval_id"] = approval_id
+ start_event["awaiting_confirmation"] = needs_confirm
+
+ try:
+ yield {"type": "status", "text": decision.status_text}
+ yield start_event
+
+ if (
+ decision_slot is not None
+ and wait_tool_decision(
+ decision_slot,
+ approval_id,
+ cancel_event = cancel_event,
+ )
+ == "deny"
+ ):
+ decision_slot = None
+ yield {
+ "type": "tool_end",
+ "tool_name": decision.tool_name,
+ "tool_call_id": decision.tool_call_id,
+ "result": TOOL_REJECTED_MESSAGE,
+ "provenance": decision.provenance,
+ }
+ denied_message = {
+ "role": "tool",
+ "name": decision.tool_name,
+ "content": TOOL_REJECTED_MESSAGE,
+ }
+ if decision.tool_call_id:
+ denied_message["tool_call_id"] = decision.tool_call_id
+ conversation.append(denied_message)
+ if _forced_tool_call_pending:
+ _forced_tool_call_pending = False
+ continue
+ decision_slot = None
+ finally:
+ if decision_slot is not None:
+ abort_tool_decision(decision_slot, approval_id)
_effective_timeout = None if tool_call_timeout >= 9999 else tool_call_timeout
# RAG: cap paraphrased KB re-searches that slip past the dup guard.
diff --git a/studio/backend/core/inference/orchestrator.py b/studio/backend/core/inference/orchestrator.py
index 4e4788ace9..4ccac2912e 100644
--- a/studio/backend/core/inference/orchestrator.py
+++ b/studio/backend/core/inference/orchestrator.py
@@ -861,6 +861,7 @@ class InferenceOrchestrator:
tool_call_timeout: int = 300,
session_id: Optional[str] = None,
rag_scope: Optional[dict] = None,
+ confirm_tool_calls: bool = False,
use_adapter: Optional[Union[bool, str]] = None,
stats_holder: Optional[dict] = None,
**_unused,
@@ -922,6 +923,7 @@ class InferenceOrchestrator:
tool_call_timeout = tool_call_timeout,
session_id = session_id,
rag_scope = rag_scope,
+ confirm_tool_calls = confirm_tool_calls,
)
def generate_with_adapter_control(
diff --git a/studio/backend/core/inference/safetensors_agentic.py b/studio/backend/core/inference/safetensors_agentic.py
index 7942edb6d7..3b6a393f3d 100644
--- a/studio/backend/core/inference/safetensors_agentic.py
+++ b/studio/backend/core/inference/safetensors_agentic.py
@@ -35,6 +35,13 @@ from core.inference.tool_loop_controller import (
status_for_tool,
tool_event_provenance,
)
+from state.tool_approvals import (
+ TOOL_REJECTED_MESSAGE,
+ abort_tool_decision,
+ begin_tool_decision,
+ new_approval_id,
+ wait_tool_decision,
+)
logger = get_logger(__name__)
@@ -146,6 +153,7 @@ def run_safetensors_tool_loop(
tool_call_timeout: int = 300,
session_id: Optional[str] = None,
rag_scope: Optional[dict] = None,
+ confirm_tool_calls: bool = False,
) -> Generator[dict, None, None]:
"""Drive an agentic tool loop on top of a cumulative-text generator.
@@ -174,7 +182,7 @@ def run_safetensors_tool_loop(
# Forced first-pass RAG (mirrors the GGUF loop) so doc Qs don't lose to web_search.
from core.inference.tools import build_rag_autoinject
- _auto = build_rag_autoinject(conversation, rag_scope)
+ _auto = None if confirm_tool_calls else build_rag_autoinject(conversation, rag_scope)
if _auto:
for _ev in _auto["events"]:
yield _ev
@@ -509,8 +517,47 @@ def run_safetensors_tool_loop(
else:
assistant_msg.setdefault("tool_calls", []).append(decision.as_assistant_tool_call())
- yield {"type": "status", "text": decision.status_text}
- yield decision.tool_start_event()
+ needs_confirm = bool(confirm_tool_calls)
+ approval_id = new_approval_id() if needs_confirm else ""
+ decision_slot = begin_tool_decision(session_id, approval_id) if needs_confirm else None
+ start_event = decision.tool_start_event()
+ start_event["approval_id"] = approval_id
+ start_event["awaiting_confirmation"] = needs_confirm
+
+ try:
+ yield {"type": "status", "text": decision.status_text}
+ yield start_event
+
+ if (
+ decision_slot is not None
+ and wait_tool_decision(
+ decision_slot,
+ approval_id,
+ cancel_event = cancel_event,
+ )
+ == "deny"
+ ):
+ decision_slot = None
+ yield {
+ "type": "tool_end",
+ "tool_name": decision.tool_name,
+ "tool_call_id": decision.tool_call_id,
+ "result": TOOL_REJECTED_MESSAGE,
+ "provenance": decision.provenance,
+ }
+ denied_message = {
+ "role": "tool",
+ "name": decision.tool_name,
+ "content": TOOL_REJECTED_MESSAGE,
+ }
+ if decision.tool_call_id:
+ denied_message["tool_call_id"] = decision.tool_call_id
+ conversation.append(denied_message)
+ continue
+ decision_slot = None
+ finally:
+ if decision_slot is not None:
+ abort_tool_decision(decision_slot, approval_id)
eff_timeout = None if tool_call_timeout >= 9999 else tool_call_timeout
# RAG: cap paraphrased KB re-searches that slip past the dup guard.
diff --git a/studio/backend/models/inference.py b/studio/backend/models/inference.py
index b70202d6ff..039a5d75dd 100644
--- a/studio/backend/models/inference.py
+++ b/studio/backend/models/inference.py
@@ -690,6 +690,10 @@ class ChatCompletionRequest(BaseModel):
None,
description = "[x-unsloth] When true, append tools from every enabled MCP server to this request's tool list.",
)
+ confirm_tool_calls: Optional[bool] = Field(
+ None,
+ description = "[x-unsloth] When true, pause before each tool call and wait for the user to allow/deny it via POST /api/inference/tool-confirm.",
+ )
auto_heal_tool_calls: Optional[bool] = Field(
True,
description = "[x-unsloth] Auto-detect and fix malformed tool calls from model output.",
@@ -926,6 +930,12 @@ class ChatCompletionRequest(BaseModel):
return self
+class ToolConfirmRequest(BaseModel):
+ session_id: Optional[str] = None
+ approval_id: Optional[str] = None
+ decision: Literal["allow", "deny"] = "deny"
+
+
# ── OpenAI shell-tool container management ─────────────────────
diff --git a/studio/backend/routes/inference.py b/studio/backend/routes/inference.py
index 856d9bacf8..9ff30cd6db 100644
--- a/studio/backend/routes/inference.py
+++ b/studio/backend/routes/inference.py
@@ -655,6 +655,7 @@ from models.inference import (
ChatCompletionRequest,
ChatCompletionChunk,
ChatCompletion,
+ ToolConfirmRequest,
ChatMessage,
ChunkChoice,
ChoiceDelta,
@@ -702,6 +703,7 @@ from core.inference.anthropic_compat import (
AnthropicPassthroughEmitter,
)
from auth.authentication import get_current_subject
+from state.tool_approvals import resolve_tool_decision
from core.inference.key_exchange import decrypt_api_key
from core.inference.providers import get_provider_info, get_base_url
@@ -1495,8 +1497,7 @@ async def load_model(
# Shouldn't happen on already-validated args; degrade to
# no-extras rather than 400 if managed flags changed.
logger.warning(
- "Stored llama_extra_args failed revalidation; "
- "loading without them: %s",
+ "Stored llama_extra_args failed revalidation; loading without them: %s",
stripped,
)
extra_llama_args = []
@@ -1938,6 +1939,20 @@ async def cancel_inference(request: Request, current_subject: str = Depends(get_
return {"cancelled": n}
+@studio_router.post("/tool-confirm")
+async def confirm_tool_call(
+ request: ToolConfirmRequest, current_subject: str = Depends(get_current_subject)
+):
+ matched = resolve_tool_decision(
+ request.approval_id,
+ request.decision,
+ session_id = request.session_id,
+ )
+ if not matched:
+ raise HTTPException(status_code = 404, detail = "No pending tool call confirmation")
+ return {"resolved": True}
+
+
@router.post("/generate/stream")
async def generate_stream(
request: GenerateRequest, current_subject: str = Depends(get_current_subject)
@@ -3196,6 +3211,22 @@ async def openai_chat_completions(
# ── External provider routing ────────────────────────────────
# encrypted_api_key is optional -- local providers (llama.cpp / vLLM / Ollama) may run without auth.
if payload.provider_id or payload.provider_type:
+ if payload.confirm_tool_calls and (
+ payload.enable_tools is True
+ or bool(payload.enabled_tools)
+ or bool(payload.tools)
+ or bool(payload.openai_code_exec_container_id)
+ or bool(payload.anthropic_code_exec_container_id)
+ ):
+ raise HTTPException(
+ status_code = 400,
+ detail = openai_error_body(
+ "confirm_tool_calls is only supported for local streaming tools.",
+ status = 400,
+ code = "invalid_request_error",
+ param = "confirm_tool_calls",
+ ),
+ )
if _wants_multiple_choices(payload):
_raise_unsupported_n("external provider chat completions")
return await _proxy_to_external_provider(payload, request)
@@ -3567,6 +3598,16 @@ async def openai_chat_completions(
use_tools = False
if use_tools:
+ if payload.confirm_tool_calls and not payload.stream:
+ raise HTTPException(
+ status_code = 400,
+ detail = openai_error_body(
+ "confirm_tool_calls requires stream=true for local tool execution.",
+ status = 400,
+ code = "invalid_request_error",
+ param = "confirm_tool_calls",
+ ),
+ )
if _wants_multiple_choices(payload):
_raise_unsupported_n("GGUF tool chat completions")
# ── Tool-use system prompt nudge ──────────────────────
@@ -3637,6 +3678,7 @@ async def openai_chat_completions(
session_id = payload.session_id,
rag_scope = payload.rag_scope,
disable_parallel_tool_use = payload.parallel_tool_calls is False,
+ confirm_tool_calls = bool(payload.confirm_tool_calls),
)
_tool_sentinel = object()
@@ -3646,6 +3688,7 @@ async def openai_chat_completions(
_tracker.__enter__()
async def gguf_tool_stream():
+ gen = None
try:
first_chunk = ChatCompletionChunk(
id = completion_id,
@@ -3768,6 +3811,11 @@ async def openai_chat_completions(
error_chunk = _openai_stream_error_chunk(e)
yield f"data: {json.dumps(error_chunk)}\n\n"
finally:
+ if gen is not None:
+ try:
+ gen.close()
+ except (RuntimeError, ValueError):
+ pass
_tracker.__exit__(None, None, None)
return StreamingResponse(
@@ -4091,6 +4139,16 @@ async def openai_chat_completions(
_sf_use_tools = False
if _sf_use_tools:
+ if payload.confirm_tool_calls and not payload.stream:
+ raise HTTPException(
+ status_code = 400,
+ detail = openai_error_body(
+ "confirm_tool_calls requires stream=true for local tool execution.",
+ status = 400,
+ code = "invalid_request_error",
+ param = "confirm_tool_calls",
+ ),
+ )
_sf_nudge = _build_tool_action_nudge(
tools = _sf_tools_to_use,
model_name = model_name,
@@ -4167,6 +4225,7 @@ async def openai_chat_completions(
else 300,
session_id = payload.session_id,
rag_scope = payload.rag_scope,
+ confirm_tool_calls = bool(payload.confirm_tool_calls),
use_adapter = payload.use_adapter,
stats_holder = _sf_stats_holder,
)
@@ -4177,6 +4236,7 @@ async def openai_chat_completions(
_sf_tracker.__enter__()
async def sf_tool_stream():
+ gen = None
try:
first_chunk = ChatCompletionChunk(
id = completion_id,
@@ -4293,6 +4353,11 @@ async def openai_chat_completions(
}
yield f"data: {json.dumps(error_chunk)}\n\n"
finally:
+ if gen is not None:
+ try:
+ gen.close()
+ except (RuntimeError, ValueError):
+ pass
_sf_tracker.__exit__(None, None, None)
if payload.stream:
@@ -5934,6 +5999,15 @@ async def anthropic_messages(
)
if server_tools:
+ if bool(getattr(payload, "confirm_tool_calls", False)):
+ raise HTTPException(
+ status_code = 400,
+ detail = anthropic_error_body(
+ "confirm_tool_calls is not supported for Anthropic Messages server tools.",
+ status = 400,
+ err_type = "invalid_request_error",
+ ),
+ )
from core.inference.tools import ALL_TOOLS
openai_tools = _select_anthropic_server_tools(
diff --git a/studio/backend/state/tool_approvals.py b/studio/backend/state/tool_approvals.py
new file mode 100644
index 0000000000..f66226b61d
--- /dev/null
+++ b/studio/backend/state/tool_approvals.py
@@ -0,0 +1,139 @@
+# SPDX-License-Identifier: AGPL-3.0-only
+# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.
+
+"""Per-call tool-call confirmation gate.
+
+When a chat request sets ``confirm_tool_calls``, the agentic loop pauses
+before executing each tool and waits here for the user's decision, which
+arrives via ``POST /api/inference/tool-confirm`` on a separate connection.
+
+Each gated call is identified by a unique ``approval_id`` (minted with
+``new_approval_id``) that the loop both registers here and echoes in the
+``tool_start`` stream event. The frontend sends that exact id back, so a
+stale or duplicate confirmation -- or a second tool awaiting a decision in
+the same session -- can never resolve the wrong call. ``session_id`` is
+kept alongside purely as a scope check.
+
+The slot is registered with ``begin_tool_decision`` *before* the loop
+yields ``tool_start``, closing the race where a fast confirmation (or an
+auto "Always allow") could otherwise arrive before the waiter exists.
+``wait_tool_decision`` then blocks and cleans up its own slot.
+"""
+
+import secrets
+import threading
+from typing import Optional
+
+# Generous ceiling so a user can deliberate; cancellation (stop button /
+# disconnect) still breaks the wait early via ``cancel_event``.
+_DECISION_TIMEOUT = 3600.0
+
+# Fed to the model as the tool result when the user denies a call, so it
+# can adapt and keep responding instead of the turn ending abruptly.
+TOOL_REJECTED_MESSAGE = "The user declined to run this tool call."
+
+_lock = threading.Lock()
+# approval_id -> {"event": threading.Event, "decision": str|None, "session": str}
+_pending: dict[str, dict] = {}
+
+
+def new_approval_id() -> str:
+ """Mint an unguessable id for one pending tool-call confirmation."""
+ return secrets.token_urlsafe(16)
+
+
+def begin_tool_decision(session_id, approval_id) -> dict:
+ """Register a pending decision slot and return it.
+
+ Call this *before* yielding the ``tool_start`` event so the waiter
+ always exists by the time the user's confirmation can arrive.
+ """
+ slot = {
+ "event": threading.Event(),
+ "decision": None,
+ "session": session_id or "",
+ }
+ with _lock:
+ _pending[approval_id] = slot
+ return slot
+
+
+def wait_tool_decision(
+ slot,
+ approval_id,
+ cancel_event = None,
+ timeout = _DECISION_TIMEOUT,
+):
+ """Block on a slot from ``begin_tool_decision`` until the user decides.
+
+ Returns ``"allow"`` or ``"deny"``. Falls back to ``"deny"`` if the wait
+ times out or generation is cancelled before the user decides. Always
+ removes its own slot on exit.
+ """
+ try:
+ waited = 0.0
+ while not slot["event"].wait(timeout = 0.5):
+ if cancel_event is not None and cancel_event.is_set():
+ return "deny"
+ waited += 0.5
+ if waited >= timeout:
+ return "deny"
+ return slot["decision"] or "deny"
+ finally:
+ with _lock:
+ if _pending.get(approval_id) is slot:
+ _pending.pop(approval_id, None)
+
+
+def abort_tool_decision(slot, approval_id) -> None:
+ """Remove a slot that was announced but never entered ``wait_tool_decision``.
+
+ Streaming wrappers may stop after ``tool_start`` is yielded and before
+ the loop resumes into ``wait_tool_decision``. In that case there is no
+ waiter to run the normal cleanup path, so the generator close path calls
+ this explicitly.
+ """
+ with _lock:
+ if _pending.get(approval_id) is slot:
+ _pending.pop(approval_id, None)
+
+
+def request_tool_decision(
+ session_id,
+ approval_id,
+ cancel_event = None,
+ timeout = _DECISION_TIMEOUT,
+):
+ """Register and wait in one call (when the slot is not needed early)."""
+ slot = begin_tool_decision(session_id, approval_id)
+ return wait_tool_decision(slot, approval_id, cancel_event = cancel_event, timeout = timeout)
+
+
+def resolve_tool_decision(
+ approval_id,
+ decision,
+ session_id = None,
+) -> bool:
+ """Record the user's "allow"/"deny" decision and unblock the loop.
+
+ Returns ``True`` if a pending call matched, ``False`` otherwise (e.g. a
+ stale or duplicate confirmation, or a session-scope mismatch).
+
+ The first decision wins: once a slot's event is set, a later (duplicate or
+ out-of-order) confirmation for the same id is rejected without mutating the
+ recorded decision, so an Allow can never be flipped to Deny in the window
+ before the waiter reads ``slot["decision"]`` and pops the slot.
+ """
+ if not approval_id:
+ return False
+ with _lock:
+ slot = _pending.get(approval_id)
+ if not slot:
+ return False
+ if session_id is not None and slot["session"] != (session_id or ""):
+ return False
+ if slot["event"].is_set():
+ return False
+ slot["decision"] = decision
+ slot["event"].set()
+ return True
diff --git a/studio/backend/tests/test_anthropic_messages.py b/studio/backend/tests/test_anthropic_messages.py
index e1230ae113..f8d3f44d4b 100644
--- a/studio/backend/tests/test_anthropic_messages.py
+++ b/studio/backend/tests/test_anthropic_messages.py
@@ -1543,6 +1543,19 @@ class TestAnthropicMessagesToolRouting:
_drive(anthropic_messages(payload, request = None, current_subject = "t"))
assert backend.calls[0][0] == "tools"
+ def test_confirm_tool_calls_rejected_for_server_tools(self, monkeypatch):
+ backend = _mock_backend(monkeypatch)
+ payload = _basic_payload(
+ confirm_tool_calls = True,
+ tools = [{"type": "web_search_20250305", "name": "web_search"}],
+ )
+
+ with pytest.raises(HTTPException) as exc:
+ _drive(anthropic_messages(payload, request = None, current_subject = "t"))
+ assert exc.value.status_code == 400
+ assert "confirm_tool_calls is not supported" in exc.value.detail["error"]["message"]
+ assert backend.calls == []
+
def test_per_request_enable_tools_false_blocks_server_tool_alias(self, monkeypatch):
backend = _mock_backend(monkeypatch)
payload = _basic_payload(
diff --git a/studio/backend/tests/test_llama_cpp_tool_loop.py b/studio/backend/tests/test_llama_cpp_tool_loop.py
index fa583ef53d..3c121c281d 100644
--- a/studio/backend/tests/test_llama_cpp_tool_loop.py
+++ b/studio/backend/tests/test_llama_cpp_tool_loop.py
@@ -21,6 +21,8 @@ if _BACKEND_DIR not in sys.path:
sys.path.insert(0, _BACKEND_DIR)
from core.inference.llama_cpp import LlamaCppBackend
+from state import tool_approvals
+from state.tool_approvals import TOOL_REJECTED_MESSAGE, resolve_tool_decision
def _sse(delta: dict) -> str:
@@ -70,6 +72,27 @@ def _tool_names(payload: dict) -> list[str]:
]
+def _structured_tool_call(tool_name: str, arguments: dict, call_id: str) -> list[str]:
+ return [
+ _sse(
+ {
+ "tool_calls": [
+ {
+ "index": 0,
+ "id": call_id,
+ "type": "function",
+ "function": {
+ "name": tool_name,
+ "arguments": json.dumps(arguments),
+ },
+ }
+ ]
+ }
+ ),
+ _done(),
+ ]
+
+
def test_structured_tool_call_after_visible_preface_is_executed(monkeypatch):
"""llama-server may emit content first and then native delta.tool_calls.
@@ -1149,3 +1172,151 @@ def test_reprompted_tool_call_still_streams_final_answer(monkeypatch):
content_texts = [event.get("text", "") for event in events if event.get("type") == "content"]
assert content_texts == ["I will use render_html now.", "Final note after tool."]
assert len(payloads) == 3
+
+
+def test_confirm_tool_calls_allow_executes_gguf_tool(monkeypatch):
+ streams = [
+ _structured_tool_call("python", {"code": "print(1)"}, "call_py"),
+ [_sse({"content": "Done."}), _done()],
+ ]
+ payloads: list[dict] = []
+ backend = _make_backend(monkeypatch, streams, payloads)
+
+ calls: list[tuple[str, dict]] = []
+
+ def fake_execute_tool(name, arguments, **_kwargs):
+ calls.append((name, arguments))
+ return "OK"
+
+ monkeypatch.setattr("core.inference.tools.execute_tool", fake_execute_tool)
+ monkeypatch.setattr("core.inference.llama_cpp.new_approval_id", lambda: "approval-1")
+ monkeypatch.setattr(
+ "core.inference.llama_cpp.begin_tool_decision",
+ lambda *_a, **_k: object(),
+ )
+ monkeypatch.setattr("core.inference.llama_cpp.wait_tool_decision", lambda *_a, **_k: "allow")
+
+ events = list(
+ backend.generate_chat_completion_with_tools(
+ messages = [{"role": "user", "content": "run python"}],
+ tools = [{"type": "function", "function": {"name": "python"}}],
+ max_tool_iterations = 1,
+ confirm_tool_calls = True,
+ session_id = "sess",
+ )
+ )
+
+ starts = [event for event in events if event.get("type") == "tool_start"]
+ assert len(starts) == 1
+ assert starts[0]["approval_id"]
+ assert starts[0]["awaiting_confirmation"] is True
+ assert calls == [("python", {"code": "print(1)"})]
+ assert any(event.get("type") == "tool_end" and event.get("result") == "OK" for event in events)
+
+
+def test_confirm_tool_calls_close_after_prompt_cleans_gguf_slot(monkeypatch):
+ approval_id = "approval-close"
+ streams = [_structured_tool_call("python", {"code": "print(1)"}, "call_py")]
+ payloads: list[dict] = []
+ backend = _make_backend(monkeypatch, streams, payloads)
+
+ monkeypatch.setattr(
+ "core.inference.tools.execute_tool",
+ lambda *_a, **_k: (_ for _ in ()).throw(AssertionError("tool should not run")),
+ )
+ monkeypatch.setattr("core.inference.llama_cpp.new_approval_id", lambda: approval_id)
+
+ with tool_approvals._lock:
+ tool_approvals._pending.clear()
+
+ gen = backend.generate_chat_completion_with_tools(
+ messages = [{"role": "user", "content": "run python"}],
+ tools = [{"type": "function", "function": {"name": "python"}}],
+ max_tool_iterations = 1,
+ confirm_tool_calls = True,
+ session_id = "sess",
+ )
+ try:
+ assert next(gen)["type"] == "status"
+ start = next(gen)
+ assert start["type"] == "tool_start"
+ assert start["approval_id"] == approval_id
+ with tool_approvals._lock:
+ assert approval_id in tool_approvals._pending
+ finally:
+ gen.close()
+
+ with tool_approvals._lock:
+ assert approval_id not in tool_approvals._pending
+ assert resolve_tool_decision(approval_id, "allow", session_id = "sess") is False
+
+
+def test_confirm_tool_calls_skips_gguf_rag_autoinject(monkeypatch):
+ streams = [[_sse({"content": "Done."}), _done()]]
+ payloads: list[dict] = []
+ backend = _make_backend(monkeypatch, streams, payloads)
+
+ def fail_autoinject(*_args, **_kwargs):
+ raise AssertionError("RAG autoinject must not run before approval")
+
+ monkeypatch.setattr("core.inference.tools.build_rag_autoinject", fail_autoinject)
+
+ events = list(
+ backend.generate_chat_completion_with_tools(
+ messages = [{"role": "user", "content": "use docs"}],
+ tools = [{"type": "function", "function": {"name": "search_knowledge_base"}}],
+ max_tool_iterations = 1,
+ confirm_tool_calls = True,
+ session_id = "sess",
+ rag_scope = {"thread_id": "t1"},
+ )
+ )
+
+ assert any(event.get("type") == "content" and event.get("text") == "Done." for event in events)
+
+
+def test_confirm_tool_calls_deny_skips_gguf_tool_and_retry_can_execute(monkeypatch):
+ same_call = _structured_tool_call("python", {"code": "print(1)"}, "call_py")
+ streams = [
+ same_call,
+ _structured_tool_call("python", {"code": "print(1)"}, "call_py_retry"),
+ [_sse({"content": "Done."}), _done()],
+ ]
+ payloads: list[dict] = []
+ backend = _make_backend(monkeypatch, streams, payloads)
+
+ calls: list[tuple[str, dict]] = []
+
+ def fake_execute_tool(name, arguments, **_kwargs):
+ calls.append((name, arguments))
+ return "OK"
+
+ decisions = iter(["deny", "allow"])
+ approvals = iter(["approval-1", "approval-2"])
+
+ monkeypatch.setattr("core.inference.tools.execute_tool", fake_execute_tool)
+ monkeypatch.setattr("core.inference.llama_cpp.new_approval_id", lambda: next(approvals))
+ monkeypatch.setattr(
+ "core.inference.llama_cpp.begin_tool_decision",
+ lambda *_a, **_k: object(),
+ )
+ monkeypatch.setattr(
+ "core.inference.llama_cpp.wait_tool_decision",
+ lambda *_a, **_k: next(decisions),
+ )
+
+ events = list(
+ backend.generate_chat_completion_with_tools(
+ messages = [{"role": "user", "content": "run python"}],
+ tools = [{"type": "function", "function": {"name": "python"}}],
+ max_tool_iterations = 2,
+ confirm_tool_calls = True,
+ session_id = "sess",
+ )
+ )
+
+ starts = [event for event in events if event.get("type") == "tool_start"]
+ ends = [event for event in events if event.get("type") == "tool_end"]
+ assert len(starts) == 2
+ assert [event["result"] for event in ends] == [TOOL_REJECTED_MESSAGE, "OK"]
+ assert calls == [("python", {"code": "print(1)"})]
diff --git a/studio/backend/tests/test_openai_tool_passthrough.py b/studio/backend/tests/test_openai_tool_passthrough.py
index 1d994b46c0..79e8c9ae49 100644
--- a/studio/backend/tests/test_openai_tool_passthrough.py
+++ b/studio/backend/tests/test_openai_tool_passthrough.py
@@ -401,6 +401,28 @@ class TestChatCompletionRequestToolFields:
)
self._assert_unsupported_n(resp)
+ def test_confirm_tool_calls_rejected_for_provider_tools(self, monkeypatch):
+ class _UnusedBackend:
+ is_loaded = False
+
+ client = self._v1_client(monkeypatch, _UnusedBackend())
+ resp = client.post(
+ "/v1/chat/completions",
+ json = {
+ "messages": [{"role": "user", "content": "hi"}],
+ "provider_type": "openai",
+ "external_model": "gpt-4.1",
+ "enable_tools": True,
+ "enabled_tools": ["web_search"],
+ "confirm_tool_calls": True,
+ },
+ )
+
+ assert resp.status_code == 400
+ body = resp.json()
+ assert body["error"]["param"] == "confirm_tool_calls"
+ assert "only supported for local streaming tools" in body["error"]["message"]
+
def test_logprobs_rejected_until_supported(self, monkeypatch):
class _UnusedBackend:
is_loaded = False
@@ -480,6 +502,7 @@ class TestChatCompletionRequestToolFields:
def test_n_rejected_for_non_gguf_path(self, monkeypatch):
class _NoGGUFBackend:
is_loaded = False
+ supports_tools = False
class _InferenceBackend:
active_model_name = "test-model"
@@ -495,6 +518,45 @@ class TestChatCompletionRequestToolFields:
)
self._assert_unsupported_n(resp)
+ def test_confirm_tool_calls_requires_streaming_for_safetensors_tools(self, monkeypatch):
+ import routes.inference as inference_route
+
+ class _NoGGUFBackend:
+ is_loaded = False
+ supports_tools = False
+
+ class _InferenceBackend:
+ active_model_name = "test-model"
+ models = {"test-model": {"chat_template_info": {"template": "chatml"}}}
+
+ def generate_chat_completion_with_tools(self, **kwargs):
+ raise AssertionError("tool loop should be rejected before starting")
+
+ def generate_chat_completion(self, **kwargs):
+ raise AssertionError("plain path should not be used")
+
+ monkeypatch.setattr(
+ inference_route,
+ "_detect_safetensors_features",
+ lambda backend, chat_template: {"supports_tools": True},
+ )
+ client = self._v1_client(monkeypatch, _NoGGUFBackend(), _InferenceBackend())
+ resp = client.post(
+ "/v1/chat/completions",
+ json = {
+ "messages": [{"role": "user", "content": "hi"}],
+ "enable_tools": True,
+ "enabled_tools": ["web_search"],
+ "confirm_tool_calls": True,
+ "stream": False,
+ },
+ )
+
+ assert resp.status_code == 400
+ body = resp.json()
+ assert body["error"]["param"] == "confirm_tool_calls"
+ assert "requires stream=true" in body["error"]["message"]
+
def test_multiturn_tool_loop_messages(self):
req = ChatCompletionRequest(
messages = [
@@ -1206,6 +1268,45 @@ class TestGgufVisionToolRouting:
assert captured["kwargs"]["disable_parallel_tool_use"] is True
+ def test_confirm_tool_calls_requires_streaming_for_gguf_tools(self, monkeypatch):
+ import routes.inference as inf_mod
+
+ def _plain(**kwargs):
+ raise AssertionError("plain GGUF path should not be used")
+
+ def _tools(**kwargs):
+ raise AssertionError("tool loop should be rejected before starting")
+
+ backend = SimpleNamespace(
+ is_loaded = True,
+ is_vision = False,
+ supports_tools = True,
+ model_identifier = "test-gguf",
+ generate_chat_completion = _plain,
+ generate_chat_completion_with_tools = _tools,
+ )
+ monkeypatch.setattr(inf_mod, "get_llama_cpp_backend", lambda: backend)
+
+ payload = ChatCompletionRequest(
+ model = "default",
+ enable_tools = True,
+ enabled_tools = ["web_search"],
+ confirm_tool_calls = True,
+ stream = False,
+ messages = [{"role": "user", "content": "search once"}],
+ )
+
+ with pytest.raises(HTTPException) as exc:
+ self._drive(
+ openai_chat_completions(
+ payload,
+ request = self._Request(),
+ current_subject = "test",
+ )
+ )
+ assert exc.value.status_code == 400
+ assert "requires stream=true" in exc.value.detail["error"]["message"]
+
def test_standard_gguf_merges_system_and_developer_messages(self, monkeypatch):
import routes.inference as inf_mod
diff --git a/studio/backend/tests/test_safetensors_tool_loop.py b/studio/backend/tests/test_safetensors_tool_loop.py
index 8aa6e5df4e..12731783a0 100644
--- a/studio/backend/tests/test_safetensors_tool_loop.py
+++ b/studio/backend/tests/test_safetensors_tool_loop.py
@@ -28,6 +28,8 @@ from core.inference.tool_call_parser import (
parse_tool_calls_from_text,
strip_tool_markup,
)
+from state import tool_approvals
+from state.tool_approvals import resolve_tool_decision
from utils.datasets import is_gpt_oss_model_name
@@ -84,8 +86,7 @@ class TestParser:
# A code parameter with a literal must not truncate: the
# parser uses end-of-body as the only boundary for single-param calls.
text = (
- "html = ' '\n"
- "print('hi') "
+ "html = ' '\nprint('hi') "
)
result = parse_tool_calls_from_text(text)
assert len(result) == 1
@@ -1033,6 +1034,50 @@ class TestGuardrails:
_collect_events(loop)
assert exec_fn.calls == [("web_search", {"query": "x"})]
+ def test_confirm_tool_calls_close_after_prompt_cleans_slot(self, monkeypatch):
+ approval_id = "approval-close-sf"
+ monkeypatch.setattr(safetensors_agentic, "new_approval_id", lambda: approval_id)
+
+ loop, exec_fn = _make_loop(
+ turns = [['{"name":"python","arguments":{"code":"print(1)"}} ']],
+ exec_results = ["OK"],
+ confirm_tool_calls = True,
+ session_id = "sess",
+ max_tool_iterations = 1,
+ )
+
+ with tool_approvals._lock:
+ tool_approvals._pending.clear()
+
+ try:
+ assert next(loop)["type"] == "status"
+ start = next(loop)
+ assert start["type"] == "tool_start"
+ assert start["approval_id"] == approval_id
+ with tool_approvals._lock:
+ assert approval_id in tool_approvals._pending
+ finally:
+ loop.close()
+
+ with tool_approvals._lock:
+ assert approval_id not in tool_approvals._pending
+ assert resolve_tool_decision(approval_id, "allow", session_id = "sess") is False
+ assert exec_fn.calls == []
+
+ def test_confirm_tool_calls_skips_rag_autoinject(self, monkeypatch):
+ def fail_autoinject(*_args, **_kwargs):
+ raise AssertionError("RAG autoinject must not run before approval")
+
+ monkeypatch.setattr("core.inference.tools.build_rag_autoinject", fail_autoinject)
+ loop, exec_fn = _make_loop(
+ turns = [["plain answer"]],
+ confirm_tool_calls = True,
+ rag_scope = {"thread_id": "t1"},
+ )
+ events = _collect_events(loop)
+ assert any(e.get("type") == "content" and e.get("text") == "plain answer" for e in events)
+ assert exec_fn.calls == []
+
def test_auto_heal_disabled_preserves_xml_on_final_no_tools_pass(self):
turns = iter(
[
diff --git a/studio/backend/tests/test_tool_approvals.py b/studio/backend/tests/test_tool_approvals.py
new file mode 100644
index 0000000000..af792e652c
--- /dev/null
+++ b/studio/backend/tests/test_tool_approvals.py
@@ -0,0 +1,261 @@
+# SPDX-License-Identifier: AGPL-3.0-only
+# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
+
+"""Concurrency tests for the per-call tool-call confirmation gate.
+
+``state.tool_approvals`` coordinates two threads: the agentic loop thread
+blocked in ``wait_tool_decision`` and the request thread that delivers the
+user's choice through ``resolve_tool_decision``. Each gated call carries a
+unique ``approval_id`` so a stale or concurrent confirmation can never
+resolve the wrong call. These tests exercise that handshake directly --
+no model, no server -- so the race windows are fast and deterministic.
+"""
+
+import threading
+import time
+
+import pytest
+
+from state import tool_approvals
+from state.tool_approvals import (
+ TOOL_REJECTED_MESSAGE,
+ abort_tool_decision,
+ begin_tool_decision,
+ new_approval_id,
+ request_tool_decision,
+ resolve_tool_decision,
+ wait_tool_decision,
+)
+
+
+@pytest.fixture(autouse = True)
+def _clear_pending():
+ """Each test starts and ends with an empty ``_pending`` map."""
+ with tool_approvals._lock:
+ tool_approvals._pending.clear()
+ yield
+ with tool_approvals._lock:
+ tool_approvals._pending.clear()
+
+
+class _Waiter:
+ """Run ``request_tool_decision`` in a thread and capture its result."""
+
+ def __init__(
+ self,
+ session_id,
+ approval_id,
+ cancel_event = None,
+ timeout = None,
+ ):
+ self.session_id = session_id
+ self.approval_id = approval_id
+ self.cancel_event = cancel_event
+ self.timeout = timeout
+ self.result = None
+ self._thread = threading.Thread(target = self._run, daemon = True)
+
+ def _run(self):
+ kwargs = {"cancel_event": self.cancel_event}
+ if self.timeout is not None:
+ kwargs["timeout"] = self.timeout
+ self.result = request_tool_decision(self.session_id, self.approval_id, **kwargs)
+
+ def start(self):
+ self._thread.start()
+ _wait_until(lambda: _has_pending(self.approval_id))
+ return self
+
+ def join(self, timeout = 5.0):
+ self._thread.join(timeout = timeout)
+ assert not self._thread.is_alive(), "waiter thread did not finish"
+ return self.result
+
+
+def _has_pending(approval_id) -> bool:
+ with tool_approvals._lock:
+ return approval_id in tool_approvals._pending
+
+
+def _wait_until(
+ pred,
+ timeout = 2.0,
+ interval = 0.005,
+) -> bool:
+ deadline = time.monotonic() + timeout
+ while time.monotonic() < deadline:
+ if pred():
+ return True
+ time.sleep(interval)
+ return False
+
+
+# ── Basic allow / deny ───────────────────────────────────────────────
+
+
+def test_allow_decision():
+ aid = new_approval_id()
+ w = _Waiter("sess", aid).start()
+ assert resolve_tool_decision(aid, "allow", session_id = "sess") is True
+ assert w.join() == "allow"
+
+
+def test_deny_decision():
+ aid = new_approval_id()
+ w = _Waiter("sess", aid).start()
+ assert resolve_tool_decision(aid, "deny", session_id = "sess") is True
+ assert w.join() == "deny"
+
+
+def test_slot_cleaned_up_after_decision():
+ aid = new_approval_id()
+ w = _Waiter("sess", aid).start()
+ resolve_tool_decision(aid, "allow")
+ w.join()
+ assert _wait_until(lambda: not _has_pending(aid))
+
+
+def test_abort_tool_decision_removes_unwaited_slot():
+ aid = new_approval_id()
+ slot = begin_tool_decision("sess", aid)
+ abort_tool_decision(slot, aid)
+ assert not _has_pending(aid)
+ assert resolve_tool_decision(aid, "allow", session_id = "sess") is False
+
+
+def test_approval_ids_are_unique():
+ ids = {new_approval_id() for _ in range(1000)}
+ assert len(ids) == 1000
+
+
+# ── Pre-registration race (begin before wait) ────────────────────────
+
+
+def test_resolve_before_wait_is_not_lost():
+ """A decision delivered after ``begin`` but before ``wait`` survives.
+
+ The loop registers the slot before it yields ``tool_start``, so even a
+ confirmation that races ahead of the blocking ``wait`` is recorded on
+ the slot and returned -- never dropped.
+ """
+ aid = new_approval_id()
+ slot = begin_tool_decision("sess", aid)
+ assert resolve_tool_decision(aid, "allow", session_id = "sess") is True
+ # wait() is only entered now, after the decision already landed.
+ assert wait_tool_decision(slot, aid) == "allow"
+ assert not _has_pending(aid)
+
+
+# ── Resolver edge cases ──────────────────────────────────────────────
+
+
+def test_resolve_unknown_approval_returns_false():
+ assert resolve_tool_decision(new_approval_id(), "allow") is False
+
+
+def test_resolve_empty_approval_returns_false():
+ assert resolve_tool_decision("", "allow") is False
+ assert resolve_tool_decision(None, "allow") is False
+
+
+def test_resolve_wrong_session_scope_returns_false():
+ aid = new_approval_id()
+ w = _Waiter("sess-a", aid).start()
+ # Correct approval_id but the wrong session must not resolve it.
+ assert resolve_tool_decision(aid, "allow", session_id = "sess-b") is False
+ assert _has_pending(aid)
+ # The right session still works.
+ assert resolve_tool_decision(aid, "allow", session_id = "sess-a") is True
+ assert w.join() == "allow"
+
+
+def test_duplicate_resolve_after_completion_returns_false():
+ aid = new_approval_id()
+ w = _Waiter("sess", aid).start()
+ assert resolve_tool_decision(aid, "allow") is True
+ w.join()
+ assert _wait_until(lambda: not _has_pending(aid))
+ assert resolve_tool_decision(aid, "deny") is False
+
+
+def test_first_decision_is_immutable():
+ """A second confirmation cannot flip an already-recorded decision.
+
+ The waiter reads ``slot["decision"]`` outside the lock and then cleans up,
+ so a duplicate or out-of-order POST that lands in that window must be
+ rejected and must not overwrite the first decision -- an Allow can never
+ become a Deny. Distinct from the after-completion case above: here the slot
+ is still pending (no waiter has consumed it yet).
+ """
+ aid = new_approval_id()
+ slot = begin_tool_decision("sess", aid)
+ assert resolve_tool_decision(aid, "allow", session_id = "sess") is True
+ # Second decision, same id, before any waiter consumes/cleans the slot.
+ assert resolve_tool_decision(aid, "deny", session_id = "sess") is False
+ assert slot["decision"] == "allow"
+ # The waiter still observes the first (immutable) decision.
+ assert wait_tool_decision(slot, aid) == "allow"
+ assert not _has_pending(aid)
+
+
+# ── Cancellation and timeout ─────────────────────────────────────────
+
+
+def test_cancel_event_breaks_wait_as_deny():
+ cancel = threading.Event()
+ aid = new_approval_id()
+ w = _Waiter("sess", aid, cancel_event = cancel).start()
+ cancel.set()
+ assert w.join(timeout = 3.0) == "deny"
+ assert _wait_until(lambda: not _has_pending(aid))
+
+
+def test_timeout_returns_deny():
+ aid = new_approval_id()
+ start = time.monotonic()
+ result = request_tool_decision("sess", aid, timeout = 0.1)
+ assert result == "deny"
+ assert time.monotonic() - start < 2.0
+ assert not _has_pending(aid)
+
+
+# ── Independence across concurrent calls ─────────────────────────────
+
+
+def test_two_pending_calls_same_session_are_independent():
+ """Keying on approval_id, not session, keeps concurrent calls distinct.
+
+ Resolving the first call's id must not unblock or alter the second
+ call pending in the same session.
+ """
+ a1, a2 = new_approval_id(), new_approval_id()
+ w1 = _Waiter("sess", a1).start()
+ w2 = _Waiter("sess", a2).start()
+
+ assert resolve_tool_decision(a1, "deny", session_id = "sess") is True
+ assert w1.join() == "deny"
+ # w2 is still waiting on its own id.
+ assert _has_pending(a2)
+ assert resolve_tool_decision(a2, "allow", session_id = "sess") is True
+ assert w2.join() == "allow"
+
+
+def test_concurrent_distinct_calls_route_their_own_decisions():
+ n = 25
+ waiters = {}
+ for i in range(n):
+ aid = new_approval_id()
+ waiters[aid] = _Waiter(f"s{i}", aid).start()
+ expected = {aid: ("allow" if i % 2 == 0 else "deny") for i, aid in enumerate(waiters)}
+ for aid, decision in expected.items():
+ assert resolve_tool_decision(aid, decision) is True
+ for aid, w in waiters.items():
+ assert w.join() == expected[aid]
+
+
+# ── Constants ────────────────────────────────────────────────────────
+
+
+def test_rejected_message_is_user_facing_text():
+ assert isinstance(TOOL_REJECTED_MESSAGE, str)
+ assert TOOL_REJECTED_MESSAGE.strip()
diff --git a/studio/backend/tests/test_tool_confirm_loop.py b/studio/backend/tests/test_tool_confirm_loop.py
new file mode 100644
index 0000000000..ce7852c95f
--- /dev/null
+++ b/studio/backend/tests/test_tool_confirm_loop.py
@@ -0,0 +1,170 @@
+# SPDX-License-Identifier: AGPL-3.0-only
+# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
+
+"""Integration tests for the confirmation gate inside the real tool loop.
+
+These drive ``run_safetensors_tool_loop`` (no model -- hand-crafted fake
+generators) with ``confirm_tool_calls=True`` and resolve each pending
+decision inline. The slot is registered before ``tool_start`` is yielded,
+so resolving right after receiving that event always lands before the
+loop blocks. Covers: allow executes once, deny skips execution and feeds
+back the rejection, disabled/duplicate calls are not prompted, and a
+denied call does not pollute duplicate detection.
+"""
+
+import pytest
+
+from core.inference.safetensors_agentic import run_safetensors_tool_loop
+from state import tool_approvals
+from state.tool_approvals import TOOL_REJECTED_MESSAGE, resolve_tool_decision
+
+_SESSION = "loop-session"
+
+
+@pytest.fixture(autouse = True)
+def _clear_pending():
+ with tool_approvals._lock:
+ tool_approvals._pending.clear()
+ yield
+ with tool_approvals._lock:
+ tool_approvals._pending.clear()
+
+
+class _FakeExecuteTool:
+ def __init__(self):
+ self.calls = []
+
+ def __call__(
+ self,
+ name,
+ arguments,
+ *,
+ cancel_event = None,
+ timeout = None,
+ session_id = None,
+ rag_scope = None,
+ ):
+ self.calls.append((name, arguments))
+ return f"RESULT[{name}]"
+
+
+def _tool_call(name, args_json):
+ return f'{{"name": "{name}", "arguments": {args_json}}} '
+
+
+def _multi_turn(turns):
+ """A single_turn generator that yields one full snapshot per turn."""
+ turn_iter = iter(turns)
+
+ def _gen(_messages):
+ try:
+ yield next(turn_iter)
+ except StopIteration:
+ return
+
+ return _gen
+
+
+_DEFAULT_TOOLS = [
+ {"type": "function", "function": {"name": "python"}},
+ {"type": "function", "function": {"name": "web_search"}},
+]
+
+
+def _drive(
+ turns,
+ decisions,
+ *,
+ tools = None,
+):
+ """Run the loop, resolving each gated tool_start with the next decision.
+
+ The advertised ``tools`` list drives the loop's enabled-tool filter
+ (pass a list omitting a tool to make a call to it "disabled").
+ Returns (events, execute_calls).
+ """
+ decision_iter = iter(decisions)
+ exec_fn = _FakeExecuteTool()
+ gen = run_safetensors_tool_loop(
+ single_turn = _multi_turn(turns),
+ messages = [{"role": "user", "content": "hi"}],
+ tools = _DEFAULT_TOOLS if tools is None else tools,
+ execute_tool = exec_fn,
+ session_id = _SESSION,
+ confirm_tool_calls = True,
+ )
+ events = []
+ for ev in gen:
+ events.append(ev)
+ if ev["type"] == "tool_start" and ev.get("awaiting_confirmation"):
+ # Slot is already registered (begin ran before this yield), so
+ # the decision lands before the loop enters its blocking wait.
+ resolve_tool_decision(ev["approval_id"], next(decision_iter), session_id = _SESSION)
+ return events, exec_fn.calls
+
+
+def _tool_starts(events):
+ return [e for e in events if e["type"] == "tool_start"]
+
+
+def _tool_ends(events):
+ return [e for e in events if e["type"] == "tool_end"]
+
+
+def test_allow_executes_the_tool_once():
+ events, calls = _drive(
+ [_tool_call("python", '{"code": "print(1)"}'), "final answer"],
+ ["allow"],
+ )
+ starts = _tool_starts(events)
+ assert len(starts) == 1
+ assert starts[0]["awaiting_confirmation"] is True
+ assert starts[0]["approval_id"]
+ assert calls == [("python", {"code": "print(1)"})]
+ assert _tool_ends(events)[0]["result"] == "RESULT[python]"
+
+
+def test_deny_skips_execution_and_feeds_rejection():
+ events, calls = _drive(
+ [_tool_call("python", '{"code": "print(1)"}'), "final answer"],
+ ["deny"],
+ )
+ assert calls == [] # tool never ran
+ assert _tool_ends(events)[0]["result"] == TOOL_REJECTED_MESSAGE
+
+
+def test_disabled_tool_is_not_prompted():
+ events, calls = _drive(
+ [_tool_call("python", '{"code": "print(1)"}'), "final answer"],
+ [],
+ tools = [{"type": "function", "function": {"name": "web_search"}}],
+ )
+ assert _tool_starts(events) == []
+ assert _tool_ends(events) == []
+ assert calls == []
+
+
+def test_duplicate_call_is_not_prompted():
+ same = _tool_call("python", '{"code": "print(1)"}')
+ events, calls = _drive([same, same, "final answer"], ["allow"])
+ starts = _tool_starts(events)
+ assert len(starts) == 1
+ assert starts[0]["awaiting_confirmation"] is True
+ assert calls == [("python", {"code": "print(1)"})]
+ assert len(_tool_ends(events)) == 1
+
+
+def test_denied_call_can_be_reissued_and_approved():
+ # Deny, then the model re-issues the identical call -> approving it must
+ # execute, not get suppressed as a duplicate (denied calls are not added
+ # to the duplicate-detection history).
+ same = _tool_call("python", '{"code": "print(1)"}')
+ events, calls = _drive([same, same, "final answer"], ["deny", "allow"])
+ starts = _tool_starts(events)
+ assert len(starts) == 2
+ assert starts[0]["awaiting_confirmation"] is True
+ assert starts[1]["awaiting_confirmation"] is True # not treated as dup
+ assert calls == [("python", {"code": "print(1)"})] # ran once, on approve
+ ends = _tool_ends(events)
+ assert ends[0]["result"] == TOOL_REJECTED_MESSAGE
+ assert ends[1]["result"] == "RESULT[python]"
diff --git a/studio/backend/tests/test_tool_confirm_stream.py b/studio/backend/tests/test_tool_confirm_stream.py
new file mode 100644
index 0000000000..b8e0472e12
--- /dev/null
+++ b/studio/backend/tests/test_tool_confirm_stream.py
@@ -0,0 +1,219 @@
+# SPDX-License-Identifier: AGPL-3.0-only
+# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
+
+"""End-to-end handshake test for the tool-confirmation gate, no model.
+
+The real Studio stream wrappers in ``routes/inference.py`` drive the
+synchronous agentic generator with ``await asyncio.to_thread(next, gen,
+...)`` so the blocking ``threading.Event`` wait runs off the event loop.
+This test rebuilds that exact pattern around the real
+``state.tool_approvals`` functions, served by a real uvicorn process on
+loopback (the same server Studio uses), and proves the load-bearing
+property:
+
+* ``tool_start`` reaches the client before the gate blocks, and
+* the separate ``/tool-confirm`` POST is served *while* the stream
+ connection is blocked, after which the stream resumes with the executed
+ (allow) or rejected (deny) result -- i.e. no deadlock.
+
+Each scenario runs under a socket-level timeout, so a regression that
+reintroduces a deadlock fails fast instead of hanging the suite.
+"""
+
+import asyncio
+import json
+import socket
+import threading
+import time
+
+import httpx
+import pytest
+import uvicorn
+from fastapi import FastAPI, Request
+from fastapi.responses import StreamingResponse
+
+from state import tool_approvals
+from state.tool_approvals import (
+ TOOL_REJECTED_MESSAGE,
+ begin_tool_decision,
+ new_approval_id,
+ resolve_tool_decision,
+ wait_tool_decision,
+)
+
+_EXECUTED_RESULT = "tool executed: 2"
+
+
+@pytest.fixture(autouse = True)
+def _clear_pending():
+ with tool_approvals._lock:
+ tool_approvals._pending.clear()
+ yield
+ with tool_approvals._lock:
+ tool_approvals._pending.clear()
+
+
+def _build_app() -> FastAPI:
+ """Minimal app mirroring the real stream/confirm wiring."""
+ app = FastAPI()
+
+ def agentic_gen(session_id, cancel_event):
+ # Same shape as the real loops: register the approval slot, announce
+ # the call (echoing approval_id), gate on the decision, then either
+ # execute or feed back the rejection.
+ approval_id = new_approval_id()
+ slot = begin_tool_decision(session_id, approval_id)
+ yield {
+ "type": "tool_start",
+ "tool_name": "python",
+ "approval_id": approval_id,
+ "awaiting_confirmation": True,
+ }
+ denied = wait_tool_decision(slot, approval_id, cancel_event = cancel_event) == "deny"
+ result = TOOL_REJECTED_MESSAGE if denied else _EXECUTED_RESULT
+ yield {"type": "tool_end", "tool_name": "python", "result": result}
+
+ @app.post("/stream")
+ async def stream(req: Request):
+ body = await req.json()
+ session_id = body.get("session_id")
+ cancel_event = threading.Event()
+ sentinel = object()
+
+ async def wrapper():
+ gen = agentic_gen(session_id, cancel_event)
+ while True:
+ event = await asyncio.to_thread(next, gen, sentinel)
+ if event is sentinel:
+ break
+ yield f"data: {json.dumps(event)}\n\n"
+
+ return StreamingResponse(wrapper(), media_type = "text/event-stream")
+
+ @app.post("/tool-confirm")
+ async def tool_confirm(req: Request):
+ body = await req.json()
+ resolved = resolve_tool_decision(
+ body.get("approval_id"),
+ body.get("decision"),
+ session_id = body.get("session_id"),
+ )
+ return {"resolved": resolved}
+
+ return app
+
+
+def _free_port() -> int:
+ s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
+ s.bind(("127.0.0.1", 0))
+ port = s.getsockname()[1]
+ s.close()
+ return port
+
+
+class _Server:
+ """Run a uvicorn server in a background thread for the test's lifetime."""
+
+ def __init__(self, app):
+ self.port = _free_port()
+ config = uvicorn.Config(app, host = "127.0.0.1", port = self.port, log_level = "warning")
+ self.server = uvicorn.Server(config)
+ self._thread = threading.Thread(target = self.server.run, daemon = True)
+
+ def __enter__(self):
+ self._thread.start()
+ deadline = time.monotonic() + 10.0
+ while time.monotonic() < deadline:
+ if self.server.started:
+ return self
+ time.sleep(0.02)
+ raise AssertionError("uvicorn did not start in time")
+
+ def __exit__(self, *exc):
+ self.server.should_exit = True
+ self._thread.join(timeout = 10.0)
+
+ @property
+ def base_url(self) -> str:
+ return f"http://127.0.0.1:{self.port}"
+
+
+async def _gate_is_blocking(approval_id) -> None:
+ """Wait until the stream thread is parked on this approval's slot.
+
+ The slot is registered before ``tool_start`` is yielded, so it exists
+ by the time the client receives the event -- exactly as in reality,
+ where the confirm POST only arrives after the card renders.
+ """
+ for _ in range(400):
+ with tool_approvals._lock:
+ slot = tool_approvals._pending.get(approval_id)
+ if slot is not None and not slot["event"].is_set():
+ return
+ await asyncio.sleep(0.005)
+ raise AssertionError("gate never started waiting")
+
+
+async def _drive(base_url, session_id, decision):
+ events = []
+ resolved = None
+ timeout = httpx.Timeout(10.0)
+ async with httpx.AsyncClient(base_url = base_url, timeout = timeout) as client:
+ async with client.stream("POST", "/stream", json = {"session_id": session_id}) as resp:
+ assert resp.status_code == 200
+ async for line in resp.aiter_lines():
+ if not line.startswith("data: "):
+ continue
+ event = json.loads(line[len("data: ") :])
+ events.append(event)
+ if event["type"] == "tool_start":
+ # The stream is now blocked on the gate; the confirm
+ # POST (echoing approval_id) must still be served over a
+ # second connection.
+ approval_id = event["approval_id"]
+ await _gate_is_blocking(approval_id)
+ r = await client.post(
+ "/tool-confirm",
+ json = {
+ "session_id": session_id,
+ "approval_id": approval_id,
+ "decision": decision,
+ },
+ )
+ resolved = r.json()["resolved"]
+ return events, resolved
+
+
+def _run(session_id, decision):
+ with _Server(_build_app()) as srv:
+ return asyncio.run(
+ asyncio.wait_for(_drive(srv.base_url, session_id, decision), timeout = 15.0)
+ )
+
+
+def _types(events):
+ return [e["type"] for e in events]
+
+
+def test_allow_resumes_stream_with_executed_result():
+ events, resolved = _run("sess-allow", "allow")
+ assert resolved is True
+ assert _types(events) == ["tool_start", "tool_end"]
+ assert events[-1]["result"] == _EXECUTED_RESULT
+
+
+def test_deny_resumes_stream_with_rejection_result():
+ events, resolved = _run("sess-deny", "deny")
+ assert resolved is True
+ assert _types(events) == ["tool_start", "tool_end"]
+ assert events[-1]["result"] == TOOL_REJECTED_MESSAGE
+
+
+def test_tool_start_precedes_the_block_and_carries_approval_id():
+ # The first streamed event is always tool_start, proving the buttons
+ # can render before the backend pauses for the decision -- and it
+ # carries the approval_id / awaiting_confirmation the UI needs.
+ events, _ = _run("sess-order", "allow")
+ assert events[0]["type"] == "tool_start"
+ assert events[0]["awaiting_confirmation"] is True
+ assert events[0]["approval_id"]
diff --git a/studio/frontend/src/components/assistant-ui/thread.tsx b/studio/frontend/src/components/assistant-ui/thread.tsx
index 84e1654079..100143ed33 100644
--- a/studio/frontend/src/components/assistant-ui/thread.tsx
+++ b/studio/frontend/src/components/assistant-ui/thread.tsx
@@ -19,6 +19,7 @@ import {
thinkEffortAriaLabel,
thinkToggleAriaLabel,
} from "@/components/assistant-ui/think-aria-label";
+import { withToolConfirmation } from "@/components/assistant-ui/tool-confirmation-controls";
import { ToolFallback } from "@/components/assistant-ui/tool-fallback";
import { ToolGroup } from "@/components/assistant-ui/tool-group";
import { CodeExecutionToolUI } from "@/components/assistant-ui/tool-ui-code-execution";
@@ -2520,6 +2521,19 @@ const CancelledIndicator: FC = () => {
);
};
+const WebSearchToolUIConfirmable = withToolConfirmation(WebSearchToolUI);
+const KnowledgeBaseToolUIConfirmable =
+ withToolConfirmation(KnowledgeBaseToolUI);
+const PythonToolUIConfirmable = withToolConfirmation(PythonToolUI);
+const TerminalToolUIConfirmable = withToolConfirmation(TerminalToolUI);
+const CodeExecutionToolUIConfirmable =
+ withToolConfirmation(CodeExecutionToolUI);
+const ImageGenerationToolUIConfirmable = withToolConfirmation(
+ ImageGenerationToolUI,
+);
+const RenderHtmlToolUIConfirmable = withToolConfirmation(RenderHtmlToolUI);
+const ToolFallbackConfirmable = withToolConfirmation(ToolFallback);
+
const AssistantMessage: FC = () => {
return (
{
ToolGroup: ToolGroup,
tools: {
by_name: {
- web_search: WebSearchToolUI,
- search_knowledge_base: KnowledgeBaseToolUI,
- python: PythonToolUI,
- terminal: TerminalToolUI,
- code_execution: CodeExecutionToolUI,
- image_generation: ImageGenerationToolUI,
- render_html: RenderHtmlToolUI,
+ web_search: WebSearchToolUIConfirmable,
+ search_knowledge_base: KnowledgeBaseToolUIConfirmable,
+ python: PythonToolUIConfirmable,
+ terminal: TerminalToolUIConfirmable,
+ code_execution: CodeExecutionToolUIConfirmable,
+ image_generation: ImageGenerationToolUIConfirmable,
+ render_html: RenderHtmlToolUIConfirmable,
},
- Fallback: ToolFallback,
+ Fallback: ToolFallbackConfirmable,
},
}}
/>
diff --git a/studio/frontend/src/components/assistant-ui/tool-confirmation-controls.tsx b/studio/frontend/src/components/assistant-ui/tool-confirmation-controls.tsx
new file mode 100644
index 0000000000..89a15c61ab
--- /dev/null
+++ b/studio/frontend/src/components/assistant-ui/tool-confirmation-controls.tsx
@@ -0,0 +1,157 @@
+// SPDX-License-Identifier: AGPL-3.0-only
+// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
+
+"use client";
+
+import { Button } from "@/components/ui/button";
+import { resolveToolConfirmation } from "@/features/chat/api/chat-api";
+import { useChatRuntimeStore } from "@/features/chat/stores/chat-runtime-store";
+import type {
+ ToolCallMessagePartComponent,
+ ToolCallMessagePartStatus,
+} from "@assistant-ui/react";
+import { useCallback, useEffect, useState } from "react";
+
+/**
+ * Allow / Always allow / Deny controls for a tool call paused awaiting the
+ * user's confirmation. Rendered alongside every tool card (built-in and
+ * MCP) so the gate works for all tools, not just the ones using the
+ * fallback renderer.
+ *
+ * A card is "awaiting" only when the adapter registered a backend-gated
+ * pending call for it (see `toolConfirmations` in the runtime store), so
+ * non-gated cards -- toggle off, or external-provider tools that already
+ * ran -- never show controls.
+ */
+export function ToolConfirmationControls({
+ toolCallId,
+ toolName,
+ result,
+ status,
+}: {
+ toolCallId?: string;
+ toolName: string;
+ result: unknown;
+ status?: ToolCallMessagePartStatus;
+}) {
+ const confirmation = useChatRuntimeStore((s) =>
+ toolCallId &&
+ Object.prototype.hasOwnProperty.call(s.toolConfirmations, toolCallId)
+ ? s.toolConfirmations[toolCallId]
+ : undefined,
+ );
+ const allowToolAlways = useChatRuntimeStore((s) => s.allowToolAlways);
+ const clearToolConfirmation = useChatRuntimeStore(
+ (s) => s.clearToolConfirmation,
+ );
+ const autoAllowKey = confirmation?.autoAllowKey ?? "";
+ const autoAllowed = useChatRuntimeStore(
+ (s) =>
+ s.alwaysAllowToolsBySession.get(autoAllowKey)?.has(toolName) ?? false,
+ );
+
+ const [decided, setDecided] = useState(false);
+ const [pending, setPending] = useState<"allow" | "deny" | null>(null);
+ const [failed, setFailed] = useState(false);
+
+ // Still awaiting our decision: a gated pending entry exists, the tool has
+ // not produced a result, and the card is in its running state.
+ const awaiting =
+ confirmation !== undefined &&
+ result === undefined &&
+ status?.type === "running";
+ const showControls = awaiting && !decided;
+
+ const resolve = useCallback(
+ async (decision: "allow" | "deny") => {
+ if (!toolCallId || !confirmation) return;
+ setPending(decision);
+ setFailed(false);
+ try {
+ const ok = await resolveToolConfirmation(
+ confirmation.sessionId,
+ confirmation.approvalId,
+ decision,
+ );
+ if (ok) {
+ // Only hide the controls once the backend confirms it matched the
+ // pending call -- otherwise the generation would stay blocked with
+ // no way to retry.
+ setDecided(true);
+ clearToolConfirmation(toolCallId);
+ } else {
+ setFailed(true);
+ }
+ } catch {
+ setFailed(true);
+ } finally {
+ setPending(null);
+ }
+ },
+ [toolCallId, confirmation, clearToolConfirmation],
+ );
+
+ // Tools the user marked "Always allow" (this session) approve themselves.
+ useEffect(() => {
+ if (showControls && autoAllowed && pending === null && !failed) {
+ void resolve("allow");
+ }
+ }, [showControls, autoAllowed, pending, failed, resolve]);
+
+ if (!showControls) return null;
+ // Auto-approved tools resolve silently unless the post fails.
+ if (autoAllowed && !failed) return null;
+
+ return (
+
+ void resolve("allow")}
+ >
+ Allow
+
+ {
+ if (autoAllowKey) allowToolAlways(autoAllowKey, toolName);
+ void resolve("allow");
+ }}
+ >
+ Always allow
+
+ void resolve("deny")}
+ >
+ Deny
+
+ {failed ? (
+
+ Could not send your decision. Try again.
+
+ ) : null}
+
+ );
+}
+
+export function withToolConfirmation(
+ Component: ToolCallMessagePartComponent,
+): ToolCallMessagePartComponent {
+ const WithToolConfirmation: ToolCallMessagePartComponent = (props) => (
+ <>
+
+
+ >
+ );
+ return WithToolConfirmation;
+}
diff --git a/studio/frontend/src/components/assistant-ui/tool-fallback.tsx b/studio/frontend/src/components/assistant-ui/tool-fallback.tsx
index eeb4c2059f..8930b6386f 100644
--- a/studio/frontend/src/components/assistant-ui/tool-fallback.tsx
+++ b/studio/frontend/src/components/assistant-ui/tool-fallback.tsx
@@ -325,6 +325,9 @@ const ToolFallbackImpl: ToolCallMessagePartComponent = ({
result,
status,
}) => {
+ // Allow/Deny confirmation controls are rendered uniformly for every tool
+ // card (built-in and fallback) by the `withToolConfirmation` wrapper in
+ // thread.tsx, so this renderer stays purely presentational.
const isCancelled =
status?.type === "incomplete" && status.reason === "cancelled";
diff --git a/studio/frontend/src/components/assistant-ui/tool-group.tsx b/studio/frontend/src/components/assistant-ui/tool-group.tsx
index 569e673da3..3ce65627e2 100644
--- a/studio/frontend/src/components/assistant-ui/tool-group.tsx
+++ b/studio/frontend/src/components/assistant-ui/tool-group.tsx
@@ -9,6 +9,7 @@ import {
type PropsWithChildren,
} from "react";
import { useAuiState } from "@assistant-ui/react";
+import { useChatRuntimeStore } from "@/features/chat/stores/chat-runtime-store";
import { ChevronDownIcon } from "lucide-react";
import { Wrench01Icon } from "@hugeicons/core-free-icons";
import { HugeiconsIcon } from "@hugeicons/react";
@@ -216,6 +217,31 @@ const ToolGroupImpl: FC<
(part) => part.type === "tool-call" && part.toolName === "render_html",
),
);
+ // A blocking allow/deny prompt must never be hidden inside a collapsed
+ // group, so force the group open while any of its calls awaits confirmation.
+ const toolConfirmations = useChatRuntimeStore((s) => s.toolConfirmations);
+ const hasPendingConfirmation = useAuiState(({ message }) =>
+ message.parts
+ .slice(startIndex, endIndex + 1)
+ .some(
+ (part) =>
+ part.type === "tool-call" &&
+ Object.prototype.hasOwnProperty.call(
+ toolConfirmations,
+ part.toolCallId,
+ ),
+ ),
+ );
+ const messageRunning = useAuiState(
+ ({ message }) => message.status?.type === "running",
+ );
+ // Keep the group open once a confirmation forced it open, so answering an
+ // allow/deny doesn't snap it shut between sequential tool calls. It reverts
+ // to the default collapsed state once the turn finishes.
+ const forcedOpenRef = useRef(false);
+ if (hasPendingConfirmation) forcedOpenRef.current = true;
+ const forceOpen =
+ hasPendingConfirmation || (forcedOpenRef.current && messageRunning);
// Render single tool calls and artifacts directly so cards never hide in a
// collapsed group.
@@ -224,7 +250,7 @@ const ToolGroupImpl: FC<
}
return (
-
+
{children}
diff --git a/studio/frontend/src/features/chat/api/chat-adapter.ts b/studio/frontend/src/features/chat/api/chat-adapter.ts
index 983d4f5aa2..ce9f1c6544 100644
--- a/studio/frontend/src/features/chat/api/chat-adapter.ts
+++ b/studio/frontend/src/features/chat/api/chat-adapter.ts
@@ -1440,6 +1440,10 @@ export function createOpenAIStreamAdapter(): ChatModelAdapter {
const resolvedThreadId =
(unstable_threadId ?? runtime.activeThreadId) || undefined;
const sandboxSessionId = await resolveSandboxSessionId(resolvedThreadId);
+ const toolConfirmationScopeId = resolvedThreadId
+ ? `${sandboxSessionId || "_default"}:${resolvedThreadId}`
+ : sandboxSessionId || "_default";
+ const toolConfirmationIdsByBackendId = new Map();
const resolvedThreadKey = resolvedThreadId ?? null;
const pendingImageEditReferenceForRun = runtime.pendingImageEditReference;
const selectedImageEditReference =
@@ -1513,6 +1517,7 @@ export function createOpenAIStreamAdapter(): ChatModelAdapter {
imageToolsEnabled,
artifactsEnabled,
mcpEnabledForChat,
+ confirmToolCalls,
webFetchToolsEnabled,
ragEnabled,
ragSource,
@@ -2435,6 +2440,7 @@ export function createOpenAIStreamAdapter(): ChatModelAdapter {
: []),
],
mcp_enabled: mcpEnabledForChat,
+ confirm_tool_calls: confirmToolCalls,
// Scope: thread_id = this thread's docs, kb_id = a KB.
...(ragEnabled
? {
@@ -2560,9 +2566,20 @@ export function createOpenAIStreamAdapter(): ChatModelAdapter {
toolEvent.provenance,
);
if (toolEvent.type === "tool_start") {
+ const backendToolCallId =
+ (toolEvent.tool_call_id as string) || "";
+ const approvalId = (toolEvent.approval_id as string) || "";
+ const awaitingConfirmation =
+ toolEvent.awaiting_confirmation === true;
const id =
- (toolEvent.tool_call_id as string) ||
- `${toolEvent.tool_name}_${Date.now()}`;
+ awaitingConfirmation && approvalId
+ ? `${toolConfirmationScopeId}:${approvalId}`
+ : backendToolCallId ||
+ approvalId ||
+ `${toolEvent.tool_name}_${Date.now()}`;
+ if (awaitingConfirmation && backendToolCallId) {
+ toolConfirmationIdsByBackendId.set(backendToolCallId, id);
+ }
const toolArgs = (toolEvent.arguments ??
{}) as ToolCallMessagePart["args"];
const idx = toolCallParts.findIndex(
@@ -2593,11 +2610,30 @@ export function createOpenAIStreamAdapter(): ChatModelAdapter {
...(toolProvenance ? { provenance: toolProvenance } : {}),
} as PositionedToolCallPart);
}
+ if (awaitingConfirmation) {
+ useChatRuntimeStore
+ .getState()
+ .setToolConfirmation(
+ id,
+ approvalId,
+ sandboxSessionId ?? "",
+ toolConfirmationScopeId,
+ );
+ }
} else if (toolEvent.type === "tool_end") {
+ const backendToolCallId =
+ (toolEvent.tool_call_id as string) || "";
const id =
- (toolEvent.tool_call_id as string) ||
+ (backendToolCallId
+ ? toolConfirmationIdsByBackendId.get(backendToolCallId)
+ : undefined) ||
+ backendToolCallId ||
toolCallParts[toolCallParts.length - 1]?.toolCallId ||
"";
+ if (backendToolCallId) {
+ toolConfirmationIdsByBackendId.delete(backendToolCallId);
+ }
+ useChatRuntimeStore.getState().clearToolConfirmation(id);
const idx = toolCallParts.findIndex(
(p) => p.toolCallId === id,
);
@@ -3149,6 +3185,10 @@ export function createOpenAIStreamAdapter(): ChatModelAdapter {
throw err;
} finally {
abortSignal.removeEventListener("abort", onAbortCancel);
+ const confirmStore = useChatRuntimeStore.getState();
+ for (const part of toolCallParts) {
+ confirmStore.clearToolConfirmation(part.toolCallId);
+ }
runtime.setGeneratingStatus(null);
runtime.setToolStatus(null);
clearTimeout(warmupTimer);
diff --git a/studio/frontend/src/features/chat/api/chat-api.ts b/studio/frontend/src/features/chat/api/chat-api.ts
index 9d0939c37d..110d573571 100644
--- a/studio/frontend/src/features/chat/api/chat-api.ts
+++ b/studio/frontend/src/features/chat/api/chat-api.ts
@@ -110,6 +110,31 @@ export async function unloadModel(payload: UnloadModelRequest): Promise {
await parseJsonOrThrow(response);
}
+/**
+ * Allow or deny a tool call that is paused awaiting user confirmation
+ * (when the "Confirm tool calls" toggle is on). The call is identified by
+ * the backend ``approvalId`` echoed in the tool_start event; ``sessionId``
+ * is a scope check. Resolves to ``true`` only when the backend matched a
+ * pending call, so the caller can surface a retry on a stale/failed post.
+ */
+export async function resolveToolConfirmation(
+ sessionId: string,
+ approvalId: string,
+ decision: "allow" | "deny",
+): Promise {
+ const response = await authFetch("/api/inference/tool-confirm", {
+ method: "POST",
+ headers: { "Content-Type": "application/json" },
+ body: JSON.stringify({
+ session_id: sessionId,
+ approval_id: approvalId,
+ decision,
+ }),
+ });
+ const parsed = await parseJsonOrThrow<{ resolved?: boolean }>(response);
+ return parsed.resolved === true;
+}
+
export interface CachedGgufRepo {
repo_id: string;
size_bytes: number;
diff --git a/studio/frontend/src/features/chat/chat-settings-sheet.tsx b/studio/frontend/src/features/chat/chat-settings-sheet.tsx
index 5751a1a426..59ac2ae6b4 100644
--- a/studio/frontend/src/features/chat/chat-settings-sheet.tsx
+++ b/studio/frontend/src/features/chat/chat-settings-sheet.tsx
@@ -1496,6 +1496,7 @@ export function ChatSettingsPanel({
@@ -1682,6 +1683,30 @@ function AutoHealToolCallsToggle() {
);
}
+function ConfirmToolCallsToggle() {
+ const confirmToolCalls = useChatRuntimeStore((s) => s.confirmToolCalls);
+ const setConfirmToolCalls = useChatRuntimeStore((s) => s.setConfirmToolCalls);
+
+ return (
+
+
+
+ Confirm tool calls
+
+
+ When on, local Studio tool calls pause for your approval before they
+ run. Provider-hosted tools are not gated here.
+
+
+
+
+ );
+}
+
function ChatTemplateFields() {
const defaultTemplate = useChatRuntimeStore((s) => s.defaultChatTemplate);
const override = useChatRuntimeStore((s) => s.chatTemplateOverride);
diff --git a/studio/frontend/src/features/chat/stores/chat-runtime-store.ts b/studio/frontend/src/features/chat/stores/chat-runtime-store.ts
index 3ef63d2945..2f0e3027a8 100644
--- a/studio/frontend/src/features/chat/stores/chat-runtime-store.ts
+++ b/studio/frontend/src/features/chat/stores/chat-runtime-store.ts
@@ -34,6 +34,7 @@ export const CHAT_COLLAPSE_HTML_ARTIFACTS_KEY =
export const CHAT_ALLOW_ARTIFACT_NETWORK_ACCESS_KEY =
"unsloth_chat_allow_artifact_network_access";
export const CHAT_MCP_ENABLED_KEY = "unsloth_chat_mcp_enabled";
+export const CHAT_CONFIRM_TOOL_CALLS_KEY = "unsloth_chat_confirm_tool_calls";
export const CHAT_WEB_FETCH_TOOLS_ENABLED_KEY =
"unsloth_chat_web_fetch_tools_enabled";
export const CHAT_RAG_SOURCE_KEY = "unsloth_chat_rag_source";
@@ -402,6 +403,29 @@ type ChatRuntimeStore = {
// autoInject = forced first-pass retrieval before answering.
ragAutoInject: RagAutoInject;
ragAutoInjectMinScore: number;
+ /**
+ * When on, local Studio tool calls pause for an explicit allow/deny in the
+ * chat before they run.
+ */
+ confirmToolCalls: boolean;
+ /**
+ * Per-chat set of tool names the user chose to auto-approve via "Always
+ * allow". Keyed by UI confirmation scope, not necessarily the backend
+ * sandbox session id. Not persisted across reloads.
+ */
+ alwaysAllowToolsBySession: Map>;
+ /**
+ * Tool calls currently paused awaiting the user's allow/deny decision,
+ * keyed by the scoped frontend tool-call id. Each entry carries the backend
+ * ``approvalId`` to echo back and the ``sessionId`` the generation runs
+ * under, so the confirmation always resolves the exact pending call. The
+ * ``autoAllowKey`` scopes the UI-only "Always allow" bucket per chat.
+ * Only backend-gated local tool calls are added here.
+ */
+ toolConfirmations: Record<
+ string,
+ { approvalId: string; sessionId: string; autoAllowKey: string }
+ >;
/**
* Fetch pill state, independent of `toolsEnabled` (Search). Only
* consulted when `providerSupportsBuiltinWebFetch` is true.
@@ -483,6 +507,15 @@ type ChatRuntimeStore = {
setCollapseHtmlArtifacts: (enabled: boolean) => void;
setAllowArtifactNetworkAccess: (enabled: boolean) => void;
setMcpEnabledForChat: (enabled: boolean) => void;
+ setConfirmToolCalls: (enabled: boolean) => void;
+ allowToolAlways: (sessionId: string, toolName: string) => void;
+ setToolConfirmation: (
+ toolCallId: string,
+ approvalId: string,
+ sessionId: string,
+ autoAllowKey: string,
+ ) => void;
+ clearToolConfirmation: (toolCallId: string) => void;
setWebFetchToolsEnabled: (enabled: boolean) => void;
setRagEnabled: (enabled: boolean) => void;
setRagSource: (source: RagSource) => void;
@@ -749,6 +782,9 @@ export const useChatRuntimeStore = create((set, get) => ({
false,
),
mcpEnabledForChat: loadBool(CHAT_MCP_ENABLED_KEY, false),
+ confirmToolCalls: loadBool(CHAT_CONFIRM_TOOL_CALLS_KEY, false),
+ alwaysAllowToolsBySession: new Map>(),
+ toolConfirmations: {},
webFetchToolsEnabled: loadBool(CHAT_WEB_FETCH_TOOLS_ENABLED_KEY, false),
// RAG is opt-in per session: always starts off, never restored from storage.
ragEnabled: false,
@@ -1074,6 +1110,40 @@ export const useChatRuntimeStore = create((set, get) => ({
saveBool(CHAT_MCP_ENABLED_KEY, mcpEnabledForChat);
return { mcpEnabledForChat };
}),
+ setConfirmToolCalls: (confirmToolCalls) =>
+ set(() => {
+ saveBool(CHAT_CONFIRM_TOOL_CALLS_KEY, confirmToolCalls);
+ return { confirmToolCalls };
+ }),
+ allowToolAlways: (sessionId, toolName) =>
+ set((state) => {
+ const current = state.alwaysAllowToolsBySession.get(sessionId);
+ if (current?.has(toolName)) return state;
+ const next = new Map(state.alwaysAllowToolsBySession);
+ next.set(sessionId, new Set(current ?? []).add(toolName));
+ return { alwaysAllowToolsBySession: next };
+ }),
+ setToolConfirmation: (toolCallId, approvalId, sessionId, autoAllowKey) =>
+ set((state) => ({
+ toolConfirmations: {
+ ...state.toolConfirmations,
+ [toolCallId]: { approvalId, sessionId, autoAllowKey },
+ },
+ })),
+ clearToolConfirmation: (toolCallId) =>
+ set((state) => {
+ if (
+ !Object.prototype.hasOwnProperty.call(
+ state.toolConfirmations,
+ toolCallId,
+ )
+ ) {
+ return state;
+ }
+ const next = { ...state.toolConfirmations };
+ delete next[toolCallId];
+ return { toolConfirmations: next };
+ }),
setWebFetchToolsEnabled: (webFetchToolsEnabled) =>
set(() => {
saveBool(CHAT_WEB_FETCH_TOOLS_ENABLED_KEY, webFetchToolsEnabled);
diff --git a/studio/frontend/src/features/chat/types/api.ts b/studio/frontend/src/features/chat/types/api.ts
index a0e5d5355e..96caa38bd4 100644
--- a/studio/frontend/src/features/chat/types/api.ts
+++ b/studio/frontend/src/features/chat/types/api.ts
@@ -282,6 +282,8 @@ export interface OpenAIChatCompletionsRequest {
enabled_tools?: string[];
/** Local models + enable_tools only. */
mcp_enabled?: boolean;
+ /** Local models + enable_tools only. */
+ confirm_tool_calls?: boolean;
/** Exactly one of `kb_id` (a KB) or `thread_id` (thread docs). */
rag_scope?: {
kb_id?: string;
From 6b62b2b5c04c640271b25746ac191cff37c66189 Mon Sep 17 00:00:00 2001
From: Daniel Han
Date: Fri, 12 Jun 2026 01:56:05 -0700
Subject: [PATCH 19/50] Guard Apple GPU power against negative counter-reset
readings (#6235)
IOReport energy counters can reset (sleep/wake, power gating), making a poll
delta negative. Return None for a negative total so the monitor shows -- for
that poll instead of a bogus negative wattage; it self-corrects next poll.
---
studio/backend/utils/hardware/apple.py | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/studio/backend/utils/hardware/apple.py b/studio/backend/utils/hardware/apple.py
index 3f14af60ad..62dbd10b8d 100644
--- a/studio/backend/utils/hardware/apple.py
+++ b/studio/backend/utils/hardware/apple.py
@@ -386,7 +386,9 @@ class _IOReportEnergy:
watts = _watts(energy, unit, elapsed_s)
if watts is not None:
total = (total or 0.0) + watts
- return round(total, 1) if total is not None else None
+ if total is None or total < 0: # negative = counter reset; show -- not a bogus draw
+ return None
+ return round(total, 1)
# ========== Public API (module singletons, failure-latched) ==========
From 95a2627bf6d96efebd64681a555bf8b9ab90e022 Mon Sep 17 00:00:00 2001
From: Irakli <39024518+IrakliXYZ@users.noreply.github.com>
Date: Fri, 12 Jun 2026 13:30:53 +0400
Subject: [PATCH 20/50] Fix step count mismatch when sequence packing is
enabled (#5967)
* Fix step count mismatch when sequence packing is enabled
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Emit a single step-0 progress event and guard applyStatus totalSteps
Merge the two consecutive _update_progress calls before train() so the
step-0 gate in _on_progress fires once instead of twice, avoiding a
duplicate startup event and a null-metric step-0 row in training_metrics.
Apply the same positive-number guard to applyStatus that applyProgress
uses, so a stale or startup status poll can no longer overwrite the
packed step count with 0 or replace it with a stale total.
* Log debug message when train_dataset length is unavailable
The TypeError fallback for length-less datasets (e.g. streaming
IterableDataset) was silent, leaving no trace that the step estimate
came from the raw dataset rather than the packed one.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Etherll <61019402+Etherll@users.noreply.github.com>
---
studio/backend/core/training/trainer.py | 18 ++++++++++++++----
studio/backend/core/training/worker.py | 2 +-
.../training/stores/training-runtime-store.ts | 9 ++++++---
3 files changed, 21 insertions(+), 8 deletions(-)
diff --git a/studio/backend/core/training/trainer.py b/studio/backend/core/training/trainer.py
index 57342b2453..085f999dd6 100644
--- a/studio/backend/core/training/trainer.py
+++ b/studio/backend/core/training/trainer.py
@@ -3367,7 +3367,19 @@ class UnslothTrainer:
# ========== PROGRESS TRACKING ==========
self.trainer.add_callback(self._create_progress_callback())
- num_samples = len(dataset["dataset"] if isinstance(dataset, dict) else dataset)
+ num_samples = None
+ if hasattr(self.trainer, "train_dataset") and self.trainer.train_dataset is not None:
+ try:
+ num_samples = len(self.trainer.train_dataset)
+ except TypeError:
+ logger.debug(
+ "train_dataset does not support len(); falling back to "
+ "raw dataset size for step estimation."
+ )
+
+ if num_samples is None:
+ num_samples = len(dataset["dataset"] if isinstance(dataset, dict) else dataset)
+
batch_size = training_args.get("batch_size", 2)
total_steps = self._calculate_total_steps(
num_samples,
@@ -3376,10 +3388,8 @@ class UnslothTrainer:
training_args.get("num_epochs", 3),
training_args.get("max_steps", 0),
)
- self._update_progress(total_steps = total_steps)
-
# ========== START TRAINING ==========
- self._update_progress(status_message = "Starting training...")
+ self._update_progress(total_steps = total_steps, status_message = "Starting training...")
logger.info("Starting training...\n")
self.trainer.train(resume_from_checkpoint = training_args.get("resume_from_checkpoint"))
diff --git a/studio/backend/core/training/worker.py b/studio/backend/core/training/worker.py
index 18b25cb4fe..c7c9003a04 100644
--- a/studio/backend/core/training/worker.py
+++ b/studio/backend/core/training/worker.py
@@ -2462,7 +2462,7 @@ def run_training_process(*, event_queue: Any, stop_queue: Any, config: dict) ->
def _on_progress(progress: TrainingProgress):
has_train_loss = progress.step > 0 and progress.loss is not None
has_eval_loss = progress.eval_loss is not None
- if has_train_loss or has_eval_loss:
+ if (progress.step == 0 and progress.total_steps > 0) or has_train_loss or has_eval_loss:
event_queue.put(
{
"type": "progress",
diff --git a/studio/frontend/src/features/training/stores/training-runtime-store.ts b/studio/frontend/src/features/training/stores/training-runtime-store.ts
index 97fbd32d57..9eaaa98c0e 100644
--- a/studio/frontend/src/features/training/stores/training-runtime-store.ts
+++ b/studio/frontend/src/features/training/stores/training-runtime-store.ts
@@ -209,8 +209,8 @@ export const useTrainingRuntimeStore = create()((set) => (
currentStep:
typeof detailStep === "number" ? Math.max(detailStep, 0) : state.currentStep,
totalSteps:
- typeof detailTotal === "number"
- ? Math.max(detailTotal, 0)
+ typeof detailTotal === "number" && detailTotal > 0
+ ? detailTotal
: state.totalSteps,
currentLoss:
typeof detailLoss === "number" ? detailLoss : state.currentLoss,
@@ -273,7 +273,10 @@ export const useTrainingRuntimeStore = create()((set) => (
...state,
jobId: payload.job_id || state.jobId,
currentStep: step,
- totalSteps: Math.max(payload.total_steps, state.totalSteps),
+ totalSteps:
+ typeof payload.total_steps === "number" && payload.total_steps > 0
+ ? payload.total_steps
+ : state.totalSteps,
// A null loss at a new step means the backend reported a non-finite
// loss; clear the display instead of keeping the stale value.
currentLoss:
From e59ce0db0477b4b3161be2ce63f199b5e270b726 Mon Sep 17 00:00:00 2001
From: alkinun
Date: Fri, 12 Jun 2026 12:37:51 +0300
Subject: [PATCH 21/50] fix/uv-bytecode-timeout (#6166)
* fix/uv-bytecode-timeout
* make sure that win installer upgrades uv for bytecode timeout
* Clarify uv bytecode timeout comment in install.sh and install.ps1
* Read installer scripts as UTF-8 in parity test so it runs on Windows
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Prefer freshly installed uv when an older one shadows it on PATH
---------
Co-authored-by: Daniel Han
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
---
install.ps1 | 57 +++++++++++++++++++---
install.sh | 6 ++-
tests/python/test_cross_platform_parity.py | 56 ++++++++++++++++++---
3 files changed, 104 insertions(+), 15 deletions(-)
diff --git a/install.ps1 b/install.ps1
index cf7bb63cdf..9abddc9ce2 100644
--- a/install.ps1
+++ b/install.ps1
@@ -1177,14 +1177,38 @@ shell.Run cmd, 0, False
if ($SkipTorch) { $InitialGpuBranch = "no_torch" }
Write-TauriDiag -GpuBranch $InitialGpuBranch -TorchIndexFamily "none" -PythonVersionForDiag $DiagPythonVersion
- # ── Install uv if not present ──
+ # ── Install uv ──
Write-TauriLog "STEP" "Installing uv package manager"
- if (-not (Get-Command uv -ErrorAction SilentlyContinue)) {
- substep "installing uv package manager..."
+ $UvMinVersion = "0.7.22"
+ function Test-UvVersionOk {
+ $cmd = Get-Command uv -ErrorAction SilentlyContinue
+ if (-not $cmd) { return $false }
+ try {
+ $raw = (& uv --version 2>$null | Select-Object -First 1)
+ } catch {
+ return $false
+ }
+ if ($raw -notmatch 'uv\s+([0-9]+(?:\.[0-9]+)+)') { return $false }
+ try {
+ return ([version]$Matches[1] -ge [version]$UvMinVersion)
+ } catch {
+ return $false
+ }
+ }
+
+ if (-not (Test-UvVersionOk)) {
+ if (Get-Command uv -ErrorAction SilentlyContinue) {
+ substep "updating uv package manager..."
+ } else {
+ substep "installing uv package manager..."
+ }
if ($script:WingetAvailable) {
$prevEAP = $ErrorActionPreference
$ErrorActionPreference = "Continue"
- try { winget install --id=astral-sh.uv -e --source winget --accept-package-agreements --accept-source-agreements } catch {}
+ try { winget upgrade --id=astral-sh.uv -e --source winget --accept-package-agreements --accept-source-agreements } catch {}
+ if (-not (Test-UvVersionOk)) {
+ try { winget install --id=astral-sh.uv -e --source winget --accept-package-agreements --accept-source-agreements } catch {}
+ }
$ErrorActionPreference = $prevEAP
Refresh-SessionPath
}
@@ -1192,19 +1216,40 @@ shell.Run cmd, 0, False
# use Astral's official PowerShell installer. This is the only
# supported path on hosts without winget (Windows ARM64 runners,
# corporate machines without the Store, etc.).
- if (-not (Get-Command uv -ErrorAction SilentlyContinue)) {
+ if (-not (Test-UvVersionOk)) {
substep "installing uv via https://astral.sh/uv/install.ps1..." "Yellow"
Invoke-Expression (Invoke-RestMethod -Uri "https://astral.sh/uv/install.ps1")
Refresh-SessionPath
}
}
- if (-not (Get-Command uv -ErrorAction SilentlyContinue)) {
+ # A freshly installed uv can sit later on PATH than an older one (active
+ # venv, Scoop/pipx shim). Prefer a just-installed uv from a known location.
+ if (-not (Test-UvVersionOk)) {
+ $origPath = $env:PATH
+ foreach ($d in @($env:UV_INSTALL_DIR, $env:XDG_BIN_HOME,
+ (Join-Path $env:USERPROFILE ".local\bin"),
+ (Join-Path $env:LOCALAPPDATA "Microsoft\WinGet\Links"))) {
+ if ($d -and (Test-Path $d)) {
+ $env:PATH = "$d;$origPath"
+ if (Test-UvVersionOk) { break }
+ $env:PATH = $origPath
+ }
+ }
+ }
+
+ if (-not (Test-UvVersionOk)) {
step "uv" "could not be installed" "Red"
substep "Install it from https://docs.astral.sh/uv/" "Yellow"
return (Exit-InstallFailure "uv could not be installed")
}
+ # When bytecode compilation is enabled, large installs can exceed uv's 60s
+ # default on slow machines. Default to 180s, preserving overrides ("0" disables).
+ if (-not $env:UV_COMPILE_BYTECODE_TIMEOUT) {
+ $env:UV_COMPILE_BYTECODE_TIMEOUT = "180"
+ }
+
# ── Create venv (migrate old layout if possible, otherwise fresh) ──
# Pass the resolved executable path to uv so it does not re-resolve
# a version string back to a conda interpreter.
diff --git a/install.sh b/install.sh
index 532ac61bc0..eba92d0746 100755
--- a/install.sh
+++ b/install.sh
@@ -1456,7 +1456,11 @@ fi
# ── Install uv ──
tauri_log "STEP" "Installing uv package manager"
-UV_MIN_VERSION="0.7.14"
+UV_MIN_VERSION="0.7.22"
+
+# When bytecode compilation is enabled, large installs can exceed uv's 60s default on slow machines. Default to 180s, preserving overrides ("0" disables).
+: "${UV_COMPILE_BYTECODE_TIMEOUT:=180}"
+export UV_COMPILE_BYTECODE_TIMEOUT
version_ge() {
# returns 0 if $1 >= $2
diff --git a/tests/python/test_cross_platform_parity.py b/tests/python/test_cross_platform_parity.py
index 0f2e73257a..34f984714e 100644
--- a/tests/python/test_cross_platform_parity.py
+++ b/tests/python/test_cross_platform_parity.py
@@ -20,7 +20,7 @@ class TestNoTorchBackendAutoInInstallSh:
"""
def test_no_torch_backend_auto_outside_fallback(self):
- lines = INSTALL_SH.read_text().splitlines()
+ lines = INSTALL_SH.read_text(encoding = "utf-8").splitlines()
# Fallback block: from "GPU detection failed" to the next "fi".
fallback_start = None
fallback_end = None
@@ -48,7 +48,7 @@ class TestNoTorchBackendAutoInInstallSh:
def test_fallback_uses_torch_backend_auto(self):
"""The fallback branch should use --torch-backend=auto as recovery."""
- text = INSTALL_SH.read_text()
+ text = INSTALL_SH.read_text(encoding = "utf-8")
assert (
"GPU detection failed" in text
), "install.sh should have a fallback branch for when GPU detection fails"
@@ -58,13 +58,13 @@ class TestInstallShHasGpuDetection:
"""install.sh must contain the get_torch_index_url function."""
def test_function_exists(self):
- text = INSTALL_SH.read_text()
+ text = INSTALL_SH.read_text(encoding = "utf-8")
assert (
"get_torch_index_url()" in text
), "install.sh is missing the get_torch_index_url() function"
def test_torch_index_url_assigned(self):
- text = INSTALL_SH.read_text()
+ text = INSTALL_SH.read_text(encoding = "utf-8")
assert (
"TORCH_INDEX_URL=$(get_torch_index_url)" in text
), "install.sh should assign TORCH_INDEX_URL from get_torch_index_url()"
@@ -115,8 +115,8 @@ class TestCudaMappingParity:
def test_same_cuda_suffixes(self):
"""Both scripts should produce the same ordered list of CUDA index suffixes."""
- sh_text = INSTALL_SH.read_text()
- ps1_text = INSTALL_PS1.read_text()
+ sh_text = INSTALL_SH.read_text(encoding = "utf-8")
+ ps1_text = INSTALL_PS1.read_text(encoding = "utf-8")
sh_thresholds = self._extract_cuda_thresholds_sh(sh_text)
ps1_thresholds = self._extract_cuda_thresholds_ps1(ps1_text)
@@ -134,13 +134,53 @@ class TestPyTorchMirrorEnvVar:
"""Both install scripts must support the UNSLOTH_PYTORCH_MIRROR env var."""
def test_install_sh_has_mirror_var(self):
- text = INSTALL_SH.read_text()
+ text = INSTALL_SH.read_text(encoding = "utf-8")
assert (
"UNSLOTH_PYTORCH_MIRROR" in text
), "install.sh should reference UNSLOTH_PYTORCH_MIRROR"
def test_install_ps1_has_mirror_var(self):
- text = INSTALL_PS1.read_text()
+ text = INSTALL_PS1.read_text(encoding = "utf-8")
assert (
"UNSLOTH_PYTORCH_MIRROR" in text
), "install.ps1 should reference UNSLOTH_PYTORCH_MIRROR"
+
+
+class TestUvBytecodeCompileTimeout:
+ """Installers should relax uv bytecode compilation timeout by default."""
+
+ @staticmethod
+ def _version_tuple(version: str) -> tuple[int, ...]:
+ return tuple(int(part) for part in version.split("."))
+
+ def test_install_sh_uses_uv_version_with_timeout_env(self):
+ text = INSTALL_SH.read_text(encoding = "utf-8")
+ match = re.search(r'^UV_MIN_VERSION="([^"]+)"$', text, re.MULTILINE)
+ assert match, "install.sh should declare UV_MIN_VERSION"
+ assert self._version_tuple(match.group(1)) >= self._version_tuple("0.7.22")
+
+ def test_install_ps1_uses_uv_version_with_timeout_env(self):
+ text = INSTALL_PS1.read_text(encoding = "utf-8")
+ match = re.search(r'^\s*\$UvMinVersion = "([^"]+)"$', text, re.MULTILINE)
+ assert match, "install.ps1 should declare $UvMinVersion"
+ assert self._version_tuple(match.group(1)) >= self._version_tuple("0.7.22")
+ assert "function Test-UvVersionOk" in text
+ assert "if (-not (Test-UvVersionOk))" in text
+
+ def test_install_sh_preserves_timeout_override(self):
+ text = INSTALL_SH.read_text(encoding = "utf-8")
+ assert (
+ ': "${UV_COMPILE_BYTECODE_TIMEOUT:=180}"' in text
+ ), "install.sh should default UV_COMPILE_BYTECODE_TIMEOUT without overwriting callers"
+ assert (
+ "export UV_COMPILE_BYTECODE_TIMEOUT" in text
+ ), "install.sh should export UV_COMPILE_BYTECODE_TIMEOUT for uv subprocesses"
+
+ def test_install_ps1_preserves_timeout_override(self):
+ text = INSTALL_PS1.read_text(encoding = "utf-8")
+ assert (
+ "if (-not $env:UV_COMPILE_BYTECODE_TIMEOUT)" in text
+ ), "install.ps1 should preserve caller UV_COMPILE_BYTECODE_TIMEOUT overrides"
+ assert (
+ '$env:UV_COMPILE_BYTECODE_TIMEOUT = "180"' in text
+ ), "install.ps1 should default UV_COMPILE_BYTECODE_TIMEOUT"
From 25ccfebc0b218fe8321c023d25f7363aad8b7016 Mon Sep 17 00:00:00 2001
From: Daniel Han
Date: Fri, 12 Jun 2026 02:39:01 -0700
Subject: [PATCH 22/50] Studio: tune llama.cpp env for data-center GPUs (#6098)
* Studio: tune llama.cpp env for data-center GPUs
Detect datacenter/professional NVIDIA GPUs at llama-server launch and set
the llama.cpp env flags that help them, gated so consumer GeForce, AMD/ROCm,
CPU and macOS are never touched.
- GGML_CUDA_FORCE_CUBLAS_COMPUTE_32F=1 for any DC GPU (FP32 cuBLAS
accumulation). On a B200 this is ~0% throughput cost with identical
perplexity (7.3230 wikitext-2-raw, baseline and on), where on GeForce the
same flag costs real throughput, hence the gate.
- GGML_CUDA_P2P=1 and CUDA_SCALE_LAUNCH_QUEUES=4x for multi-GPU DC boxes.
Benchmarked on 6x B200: +33-51% prompt processing on tensor (row) split and
+8-16% on the default pipeline (layer) split, with no regression on the
other split or on token generation.
Detection uses torch device names (A100/A30/H100/H200/H800/GH200/B200/GB200/
GB300/L40/L4/RTX PRO 6000/RTX 6000 Ada). A mixed box with one consumer GPU in
the selection is treated as non-DC. All writes are setdefault so a user value
always wins, and UNSLOTH_DISABLE_DC_TUNING=1 turns the whole thing off.
37 unit tests cover detection, multi-GPU gating, user-override precedence, the
disable flag and fail-open on error.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: fix data-center GPU detection false positives and physical-id mapping
Two issues in the data-center llama.cpp env tuning gate:
- _is_datacenter_gpu matched the marker allowlist as unbounded substrings, so
workstation/laptop parts "NVIDIA RTX A1000" and "NVIDIA RTX A3000" matched
"a100"/"a30" and were wrongly tuned as data-center GPUs (forcing FP32 cuBLAS
accumulation and the multi-GPU env, which carry a real cost on those cards).
Switch to a word-boundary regex.
- gpu_indices carries physical GPU ids (translated from torch ordinals by
_get_gpu_free_memory via CUDA_VISIBLE_DEVICES), but they were passed straight
into torch.cuda.get_device_properties, which expects mask-relative ordinals.
On a masked host (e.g. CUDA_VISIBLE_DEVICES=4,5,6,7) a selection like [4,5]
fell out of range and silently dropped the tuning, and on a mixed mask it could
probe the wrong GPU class. Build a physical-id to device-name map mirroring
_get_gpu_free_memory, then look up the selection by physical id.
Add regression tests for the A1000/A3000 false positives and for masked-host
physical-id selection (reordered and mixed-class masks included).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: tighten data-center GPU tuning comments
Comment-only pass over the DC tuning block and its tests: shorten verbose
docstrings/comments, drop ones that restate the code, collapse multi-line
blocks. Keep the load-bearing rationale (physical-id vs ordinal mapping, the
word-boundary reason, the B200 benchmark numbers). No code change: verified
with comment_tools.py check --strip-docstrings (code unchanged, comments only).
---------
Co-authored-by: danielhanchen
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
---
studio/backend/core/inference/llama_cpp.py | 107 +++++++
.../tests/test_datacenter_gpu_tuning.py | 278 ++++++++++++++++++
2 files changed, 385 insertions(+)
create mode 100644 studio/backend/tests/test_datacenter_gpu_tuning.py
diff --git a/studio/backend/core/inference/llama_cpp.py b/studio/backend/core/inference/llama_cpp.py
index 8cf37ed9ec..3393f2b4be 100644
--- a/studio/backend/core/inference/llama_cpp.py
+++ b/studio/backend/core/inference/llama_cpp.py
@@ -1329,6 +1329,105 @@ class LlamaCppBackend:
return False
return False
+ # Datacenter / professional NVIDIA parts that benefit from the llama.cpp
+ # FP32-accum / P2P tunings. Whole-word (\b) so short markers don't match
+ # workstation parts as substrings: "a100" must not fire on "RTX A1000".
+ _DATACENTER_GPU_RE = re.compile(
+ r"\b(?:a100|a30|h100|h200|h800|gh200|b200|b100|b300|gb200|gb300|"
+ r"l40s?|l4|rtx pro 6000|rtx 6000 ada)\b"
+ )
+
+ @staticmethod
+ def _is_datacenter_gpu(gpu_indices = None) -> bool:
+ """True iff every selected NVIDIA GPU is a datacenter/professional part.
+ NVIDIA-only, fails open to False (consumer GeForce, ROCm, CPU and errors
+ are left untouched); a mixed DC+consumer selection counts as non-DC.
+
+ gpu_indices are PHYSICAL ids (see _get_gpu_free_memory), but
+ get_device_properties wants mask-relative ordinals, so we rebuild the
+ ordinal->physical map from CUDA_VISIBLE_DEVICES and key names by physical
+ id. Otherwise a masked host (CUDA_VISIBLE_DEVICES=4,5,6,7, selection [4,5])
+ would drop the tuning or probe the wrong GPU."""
+ try:
+ import torch
+
+ if getattr(torch.version, "hip", None) is not None:
+ return False # ROCm reuses torch.cuda.*; not a CUDA part
+ if not (hasattr(torch, "cuda") and torch.cuda.is_available()):
+ return False
+ count = torch.cuda.device_count()
+
+ # Mirror _get_gpu_free_memory: map visible ordinal -> physical id via
+ # CUDA_VISIBLE_DEVICES; unset/unparsable leaves physical id == ordinal.
+ physical_ids: Optional[list[int]] = None
+ cvd = os.environ.get("CUDA_VISIBLE_DEVICES")
+ if cvd is not None:
+ try:
+ physical_ids = [int(x.strip()) for x in cvd.split(",") if x.strip()]
+ except ValueError:
+ physical_ids = None
+
+ pattern = LlamaCppBackend._DATACENTER_GPU_RE
+ names_by_id: dict[int, str] = {}
+ for ordinal in range(count):
+ try:
+ name = (torch.cuda.get_device_properties(ordinal).name or "").lower()
+ except Exception:
+ continue
+ pid = (
+ physical_ids[ordinal]
+ if physical_ids is not None and ordinal < len(physical_ids)
+ else ordinal
+ )
+ names_by_id[pid] = name
+
+ indices = list(gpu_indices) if gpu_indices else list(names_by_id)
+ saw = False
+ for _i in indices:
+ name = names_by_id.get(_i)
+ if name is None:
+ continue # not visible -> skip (fail conservative)
+ saw = True
+ if not pattern.search(name):
+ return False
+ return saw
+ except Exception:
+ return False
+
+ @staticmethod
+ def _effective_gpu_count(gpu_indices = None) -> int:
+ """GPUs llama-server will use: len(selection), else the visible CUDA
+ device count (None = every visible GPU). 0 on error so multi-GPU tuning
+ stays off when the count is unknown."""
+ if gpu_indices is not None:
+ return len(gpu_indices)
+ try:
+ import torch
+ if hasattr(torch, "cuda") and torch.cuda.is_available():
+ return torch.cuda.device_count()
+ except Exception:
+ return 0
+ return 0
+
+ @staticmethod
+ def _apply_datacenter_env(env: dict, gpu_indices = None) -> bool:
+ """Inject DC llama.cpp tuning into env in place via setdefault (user
+ values win); return whether the box qualified. Opt out with
+ UNSLOTH_DISABLE_DC_TUNING=1; only datacenter NVIDIA parts qualify
+ (consumer/ROCm/CPU/error are a no-op). Sets GGML_CUDA_FORCE_CUBLAS_COMPUTE_32F
+ for any qualifying GPU (FP32 accum: ~0% cost on B200, real cost on GeForce),
+ plus GGML_CUDA_P2P + CUDA_SCALE_LAUNCH_QUEUES=4x for multi-GPU (+33-51% pp
+ tensor-split, +8-16% pipeline split on B200)."""
+ if os.environ.get("UNSLOTH_DISABLE_DC_TUNING") == "1":
+ return False
+ if not LlamaCppBackend._is_datacenter_gpu(gpu_indices):
+ return False
+ env.setdefault("GGML_CUDA_FORCE_CUBLAS_COMPUTE_32F", "1")
+ if LlamaCppBackend._effective_gpu_count(gpu_indices) > 1:
+ env.setdefault("GGML_CUDA_P2P", "1")
+ env.setdefault("CUDA_SCALE_LAUNCH_QUEUES", "4x")
+ return True
+
@staticmethod
def _get_gpu_free_memory() -> list[tuple[int, int]]:
"""Query free memory per GPU.
@@ -3406,6 +3505,14 @@ class LlamaCppBackend:
env.setdefault("GGML_CUDA_ENABLE_UNIFIED_MEMORY", "1")
logger.info("AMD unified-memory APU: set GGML_CUDA_ENABLE_UNIFIED_MEMORY=1")
+ # DC NVIDIA GPUs: FP32 accum (+ P2P / launch queues for multi-GPU).
+ # See _apply_datacenter_env; opt out with UNSLOTH_DISABLE_DC_TUNING=1.
+ if self._apply_datacenter_env(env, gpu_indices):
+ multi_gpu = self._effective_gpu_count(gpu_indices) > 1
+ logger.info(
+ f"Data-center GPU detected: applied DC llama.cpp env tuning (multi_gpu={multi_gpu})"
+ )
+
if sys.platform == "win32":
# Ordering: see _build_windows_path_dirs. #5106.
path_dirs = self._build_windows_path_dirs(
diff --git a/studio/backend/tests/test_datacenter_gpu_tuning.py b/studio/backend/tests/test_datacenter_gpu_tuning.py
new file mode 100644
index 0000000000..fd9b291e8a
--- /dev/null
+++ b/studio/backend/tests/test_datacenter_gpu_tuning.py
@@ -0,0 +1,278 @@
+# SPDX-License-Identifier: AGPL-3.0-only
+# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
+
+"""Data-center llama.cpp env tuning: FP32 accum (+ P2P / launch queues for
+multi-GPU) must apply only to datacenter NVIDIA parts, never consumer GeForce,
+AMD/ROCm, CPU or macOS. User values win; UNSLOTH_DISABLE_DC_TUNING=1 disables.
+"""
+
+from __future__ import annotations
+
+import sys
+import types
+
+import pytest
+
+from core.inference.llama_cpp import LlamaCppBackend
+
+
+def _fake_torch(
+ names,
+ *,
+ hip = None,
+ cuda_ok = True,
+):
+ """torch stub: version.hip, cuda.is_available/device_count, get_device_properties(i).name."""
+ t = types.ModuleType("torch")
+ t.version = types.SimpleNamespace(hip = hip)
+ t.cuda = types.SimpleNamespace(
+ is_available = lambda: cuda_ok,
+ device_count = lambda: len(names),
+ get_device_properties = lambda i: types.SimpleNamespace(name = names[i]),
+ )
+ return t
+
+
+@pytest.fixture(autouse = True)
+def _clear_cuda_visible_devices(monkeypatch):
+ """Detection reads CUDA_VISIBLE_DEVICES, so clear it by default (run unmasked,
+ physical id == ordinal) regardless of host; masked tests set it explicitly."""
+ monkeypatch.delenv("CUDA_VISIBLE_DEVICES", raising = False)
+
+
+# ---------------------------------------------------------------------------
+# _is_datacenter_gpu
+# ---------------------------------------------------------------------------
+
+
+@pytest.mark.parametrize(
+ "names,expected",
+ [
+ # Datacenter / professional parts.
+ (["NVIDIA A100-SXM4-80GB"], True),
+ (["NVIDIA A30"], True),
+ (["NVIDIA H100 80GB HBM3"], True),
+ (["NVIDIA H200"], True),
+ (["NVIDIA H800"], True),
+ (["NVIDIA GH200 480GB"], True),
+ (["NVIDIA B200"], True),
+ (["NVIDIA GB200"], True),
+ (["NVIDIA L40S"], True),
+ (["NVIDIA L4"], True),
+ (["NVIDIA RTX PRO 6000 Blackwell Server Edition"], True),
+ (["NVIDIA RTX 6000 Ada Generation"], True),
+ # Consumer GeForce: never.
+ (["NVIDIA GeForce RTX 4090"], False),
+ (["NVIDIA GeForce RTX 5090"], False),
+ (["NVIDIA GeForce RTX 3090"], False),
+ (["NVIDIA GeForce RTX 2080 Ti"], False),
+ (["NVIDIA GeForce GTX 1080"], False),
+ # Workstation/laptop: short markers must not match as substrings
+ # ("a100" in "A1000", "a30" in "A3000").
+ (["NVIDIA RTX A1000 Laptop GPU"], False),
+ (["NVIDIA RTX A1000 6GB Laptop GPU"], False),
+ (["NVIDIA RTX A3000 Laptop GPU"], False),
+ # Homogeneous multi-DC: all must match.
+ (["NVIDIA B200", "NVIDIA B200"], True),
+ (["NVIDIA H100 80GB HBM3", "NVIDIA H100 80GB HBM3"], True),
+ # Mixed DC + consumer: non-DC, so tuning never lands on the GeForce.
+ (["NVIDIA B200", "NVIDIA GeForce RTX 4090"], False),
+ (["NVIDIA GeForce RTX 4090", "NVIDIA B200"], False),
+ ],
+)
+def test_is_datacenter_gpu(monkeypatch, names, expected):
+ monkeypatch.setitem(sys.modules, "torch", _fake_torch(names))
+ assert LlamaCppBackend._is_datacenter_gpu() is expected
+
+
+def test_is_datacenter_gpu_respects_selection(monkeypatch):
+ # A mixed box where only the DC GPU is selected -> True; only consumer -> False.
+ monkeypatch.setitem(
+ sys.modules,
+ "torch",
+ _fake_torch(["NVIDIA B200", "NVIDIA GeForce RTX 4090"]),
+ )
+ assert LlamaCppBackend._is_datacenter_gpu([0]) is True
+ assert LlamaCppBackend._is_datacenter_gpu([1]) is False
+ assert LlamaCppBackend._is_datacenter_gpu([0, 1]) is False
+
+
+def test_is_datacenter_gpu_out_of_range_indices_skipped(monkeypatch):
+ monkeypatch.setitem(sys.modules, "torch", _fake_torch(["NVIDIA B200"]))
+ # Out-of-range / negative indices are skipped; the one valid DC GPU still wins.
+ assert LlamaCppBackend._is_datacenter_gpu([0, 5, -1]) is True
+ # Only invalid indices -> nothing seen -> False (fail closed for the flag).
+ assert LlamaCppBackend._is_datacenter_gpu([5, 9]) is False
+
+
+def test_is_datacenter_gpu_masked_host_physical_ids(monkeypatch):
+ # Mask 4,5,6,7 -> ordinals 0..3 == physical 4..7. PHYSICAL selection [4,5]
+ # must resolve, not index out of range (the pre-fix bug: 4 >= device_count).
+ monkeypatch.setenv("CUDA_VISIBLE_DEVICES", "4,5,6,7")
+ monkeypatch.setitem(sys.modules, "torch", _fake_torch(["NVIDIA B200"] * 4))
+ assert LlamaCppBackend._is_datacenter_gpu([4, 5]) is True
+ assert LlamaCppBackend._is_datacenter_gpu([4, 5, 6, 7]) is True
+ assert LlamaCppBackend._is_datacenter_gpu(None) is True
+ assert LlamaCppBackend._is_datacenter_gpu([0, 1]) is False # not visible -> skip
+
+
+def test_is_datacenter_gpu_masked_host_reordered(monkeypatch):
+ # Reordered mask preserves order: ordinal 0 -> physical 7, 1 -> 4, ...
+ monkeypatch.setenv("CUDA_VISIBLE_DEVICES", "7,4,5,6")
+ monkeypatch.setitem(sys.modules, "torch", _fake_torch(["NVIDIA H100 80GB HBM3"] * 4))
+ assert LlamaCppBackend._is_datacenter_gpu([7, 4]) is True
+
+
+def test_is_datacenter_gpu_masked_host_mixed_class(monkeypatch):
+ # Mask 4,5: physical 4 = GeForce, physical 5 = B200. Detection must follow the
+ # selected physical GPU, not a same-numbered ordinal.
+ monkeypatch.setenv("CUDA_VISIBLE_DEVICES", "4,5")
+ monkeypatch.setitem(
+ sys.modules,
+ "torch",
+ _fake_torch(["NVIDIA GeForce RTX 4090", "NVIDIA B200"]),
+ )
+ assert LlamaCppBackend._is_datacenter_gpu([4]) is False
+ assert LlamaCppBackend._is_datacenter_gpu([5]) is True
+ assert LlamaCppBackend._is_datacenter_gpu([4, 5]) is False
+
+
+def test_is_datacenter_gpu_unparsable_mask_falls_back(monkeypatch):
+ # Unparsable (UUID) mask falls back to physical id == ordinal (mirrors
+ # _get_gpu_free_memory), so ordinal lookup still classifies the device.
+ monkeypatch.setenv("CUDA_VISIBLE_DEVICES", "GPU-abcdef12")
+ monkeypatch.setitem(sys.modules, "torch", _fake_torch(["NVIDIA B200"]))
+ assert LlamaCppBackend._is_datacenter_gpu([0]) is True
+
+
+def test_is_datacenter_gpu_rocm_is_false(monkeypatch):
+ # ROCm reuses torch.cuda.*; an MI300X must not qualify.
+ monkeypatch.setitem(
+ sys.modules,
+ "torch",
+ _fake_torch(["AMD Instinct MI300X"], hip = "6.2.0"),
+ )
+ assert LlamaCppBackend._is_datacenter_gpu() is False
+
+
+def test_is_datacenter_gpu_no_cuda_is_false(monkeypatch):
+ monkeypatch.setitem(sys.modules, "torch", _fake_torch([], cuda_ok = False))
+ assert LlamaCppBackend._is_datacenter_gpu() is False
+
+
+def test_is_datacenter_gpu_missing_torch_is_false(monkeypatch):
+ monkeypatch.setitem(sys.modules, "torch", None)
+ assert LlamaCppBackend._is_datacenter_gpu() is False
+
+
+# ---------------------------------------------------------------------------
+# _effective_gpu_count
+# ---------------------------------------------------------------------------
+
+
+def test_effective_gpu_count_explicit_selection(monkeypatch):
+ monkeypatch.setitem(sys.modules, "torch", _fake_torch(["NVIDIA B200"] * 4))
+ assert LlamaCppBackend._effective_gpu_count([0]) == 1
+ assert LlamaCppBackend._effective_gpu_count([0, 1, 2]) == 3
+
+
+def test_effective_gpu_count_none_uses_visible(monkeypatch):
+ # None -> visible device count.
+ monkeypatch.setitem(sys.modules, "torch", _fake_torch(["NVIDIA B200"] * 4))
+ assert LlamaCppBackend._effective_gpu_count(None) == 4
+
+
+def test_effective_gpu_count_no_cuda_is_zero(monkeypatch):
+ monkeypatch.setitem(sys.modules, "torch", _fake_torch([], cuda_ok = False))
+ assert LlamaCppBackend._effective_gpu_count(None) == 0
+
+
+def test_effective_gpu_count_missing_torch_is_zero(monkeypatch):
+ monkeypatch.setitem(sys.modules, "torch", None)
+ assert LlamaCppBackend._effective_gpu_count(None) == 0
+
+
+# ---------------------------------------------------------------------------
+# _apply_datacenter_env (the env-injection decision)
+# ---------------------------------------------------------------------------
+
+
+def test_apply_env_single_dc_gpu_sets_only_fp32(monkeypatch):
+ monkeypatch.delenv("UNSLOTH_DISABLE_DC_TUNING", raising = False)
+ monkeypatch.setitem(sys.modules, "torch", _fake_torch(["NVIDIA B200"]))
+ env: dict = {}
+ assert LlamaCppBackend._apply_datacenter_env(env, [0]) is True
+ assert env == {"GGML_CUDA_FORCE_CUBLAS_COMPUTE_32F": "1"}
+ assert "GGML_CUDA_P2P" not in env # no multi-GPU flags on one GPU
+ assert "CUDA_SCALE_LAUNCH_QUEUES" not in env
+
+
+def test_apply_env_multi_dc_gpu_sets_all(monkeypatch):
+ monkeypatch.delenv("UNSLOTH_DISABLE_DC_TUNING", raising = False)
+ monkeypatch.setitem(sys.modules, "torch", _fake_torch(["NVIDIA B200"] * 4))
+ env: dict = {}
+ assert LlamaCppBackend._apply_datacenter_env(env, [0, 1]) is True
+ assert env["GGML_CUDA_FORCE_CUBLAS_COMPUTE_32F"] == "1"
+ assert env["GGML_CUDA_P2P"] == "1"
+ assert env["CUDA_SCALE_LAUNCH_QUEUES"] == "4x"
+
+
+def test_apply_env_none_indices_uses_visible_count(monkeypatch):
+ # None on a 2x DC box -> multi-GPU flags applied.
+ monkeypatch.delenv("UNSLOTH_DISABLE_DC_TUNING", raising = False)
+ monkeypatch.setitem(sys.modules, "torch", _fake_torch(["NVIDIA H100", "NVIDIA H100"]))
+ env: dict = {}
+ assert LlamaCppBackend._apply_datacenter_env(env, None) is True
+ assert env["GGML_CUDA_P2P"] == "1"
+ assert env["CUDA_SCALE_LAUNCH_QUEUES"] == "4x"
+
+
+def test_apply_env_consumer_gpu_is_noop(monkeypatch):
+ monkeypatch.delenv("UNSLOTH_DISABLE_DC_TUNING", raising = False)
+ monkeypatch.setitem(sys.modules, "torch", _fake_torch(["NVIDIA GeForce RTX 4090"] * 2))
+ env: dict = {}
+ assert LlamaCppBackend._apply_datacenter_env(env, [0, 1]) is False
+ assert env == {}
+
+
+def test_apply_env_user_value_wins(monkeypatch):
+ monkeypatch.delenv("UNSLOTH_DISABLE_DC_TUNING", raising = False)
+ monkeypatch.setitem(sys.modules, "torch", _fake_torch(["NVIDIA B200"] * 2))
+ env = {
+ "GGML_CUDA_FORCE_CUBLAS_COMPUTE_32F": "0", # user explicitly disabled
+ "CUDA_SCALE_LAUNCH_QUEUES": "8x", # user override
+ }
+ assert LlamaCppBackend._apply_datacenter_env(env, [0, 1]) is True
+ # setdefault must not clobber user values; the unset one still defaults.
+ assert env["GGML_CUDA_FORCE_CUBLAS_COMPUTE_32F"] == "0"
+ assert env["CUDA_SCALE_LAUNCH_QUEUES"] == "8x"
+ assert env["GGML_CUDA_P2P"] == "1"
+
+
+def test_apply_env_disable_flag_respected(monkeypatch):
+ monkeypatch.setenv("UNSLOTH_DISABLE_DC_TUNING", "1")
+ monkeypatch.setitem(sys.modules, "torch", _fake_torch(["NVIDIA B200"] * 2))
+ env: dict = {}
+ assert LlamaCppBackend._apply_datacenter_env(env, [0, 1]) is False
+ assert env == {}
+
+
+def test_apply_env_fail_open_on_detection_error(monkeypatch):
+ monkeypatch.delenv("UNSLOTH_DISABLE_DC_TUNING", raising = False)
+ monkeypatch.setitem(sys.modules, "torch", None) # detection raises -> False
+ env: dict = {}
+ assert LlamaCppBackend._apply_datacenter_env(env, [0]) is False
+ assert env == {}
+
+
+def test_apply_env_masked_host_multi_dc(monkeypatch):
+ # End-to-end masked host (mask 4,5,6,7, physical selection [4,5]): pre-fix
+ # applied no tuning; now all three multi-GPU flags must be set.
+ monkeypatch.delenv("UNSLOTH_DISABLE_DC_TUNING", raising = False)
+ monkeypatch.setenv("CUDA_VISIBLE_DEVICES", "4,5,6,7")
+ monkeypatch.setitem(sys.modules, "torch", _fake_torch(["NVIDIA B200"] * 4))
+ env: dict = {}
+ assert LlamaCppBackend._apply_datacenter_env(env, [4, 5]) is True
+ assert env["GGML_CUDA_FORCE_CUBLAS_COMPUTE_32F"] == "1"
+ assert env["GGML_CUDA_P2P"] == "1"
+ assert env["CUDA_SCALE_LAUNCH_QUEUES"] == "4x"
From 6a0a62ef65f56510d6475f35690ae2a937c3d4b5 Mon Sep 17 00:00:00 2001
From: Daniel Han
Date: Fri, 12 Jun 2026 02:43:55 -0700
Subject: [PATCH 23/50] Studio: drop the on-disk freshness cache after a
llama.cpp update (#6234)
The post-install path cleared only the in-memory freshness caches and then
re-primed the 24h disk cache with a forced GitHub refresh. When that refresh
cannot reach GitHub, latest_published_release falls back to the last-good disk
value, so a still-fresh same-base mix tag cached before the swap (b9596-mix-aaa
vs the just-installed b9596-mix-bbb) is replayed and the prebuilt reads as
behind, surfacing a false update banner that points back at the build that was
just replaced.
Give reset_caches a drop_disk option and use it on the update path: with the
disk cache gone, an offline post-install refresh leaves latest as None and the
banner fails open (off) instead of lingering on the stale same-base value. The
no-arg form stays in-memory only. Adds regression coverage for the drop, the
default no-op, and the fail-open vs stale-replay contrast.
---
.../backend/tests/test_llama_cpp_freshness.py | 87 +++++++++++++++++++
studio/backend/utils/llama_cpp_freshness.py | 19 +++-
studio/backend/utils/llama_cpp_update.py | 11 ++-
3 files changed, 111 insertions(+), 6 deletions(-)
diff --git a/studio/backend/tests/test_llama_cpp_freshness.py b/studio/backend/tests/test_llama_cpp_freshness.py
index f8e4619ded..f90c4ba0e7 100644
--- a/studio/backend/tests/test_llama_cpp_freshness.py
+++ b/studio/backend/tests/test_llama_cpp_freshness.py
@@ -433,3 +433,90 @@ def test_fetch_latest_release_tag_uses_publish_time(monkeypatch):
]
monkeypatch.setattr(urllib.request, "urlopen", lambda req, timeout = 5.0: _Resp(payload))
assert fr._fetch_latest_release_tag("unslothai/llama.cpp") == "b9596-mix-e6f2453"
+
+
+# reset_caches(drop_disk=...) -- post-update stale same-base mix disk cache.
+
+
+def _seed_disk_cache(tmp_path: Path, latest_tag: str) -> Path:
+ # Matches _cache_path_for under the fixture's stubbed _cache_dir.
+ cache_dir = tmp_path / ".freshness"
+ cache_dir.mkdir(exist_ok = True)
+ cache_file = cache_dir / "unslothai__llama.cpp.json"
+ cache_file.write_text(json.dumps({"fetched_at": time.time(), "latest_tag": latest_tag}))
+ return cache_file
+
+
+def test_reset_caches_drop_disk_removes_disk_cache(tmp_path):
+ cache_file = _seed_disk_cache(tmp_path, "b9596-mix-aaa")
+ assert cache_file.exists()
+ fr.reset_caches(drop_disk = True)
+ assert not cache_file.exists()
+
+
+def test_reset_caches_default_keeps_disk_cache(tmp_path):
+ # The no-arg form is in-memory only (its existing test-only contract); it
+ # must not delete the on-disk cache.
+ cache_file = _seed_disk_cache(tmp_path, "b9596-mix-aaa")
+ fr.reset_caches()
+ assert cache_file.exists()
+
+
+def test_reset_caches_drop_disk_on_missing_dir_is_noop(tmp_path):
+ # Fresh machine, no cache dir yet: drop_disk must be a quiet no-op.
+ assert not (tmp_path / ".freshness").exists()
+ fr.reset_caches(drop_disk = True) # must not raise
+
+
+def test_drop_disk_lets_banner_fail_open_after_same_base_mix_swap(monkeypatch, tmp_path):
+ # P2 #2: the disk cache holds a still-fresh same-base mix (b9596-mix-aaa)
+ # from before an update to a *different* same-base mix (b9596-mix-bbb).
+ # The post-install path drops the disk cache; if the forced refresh is then
+ # offline, latest reads as None and the banner fails open -- instead of
+ # replaying the stale b9596-mix-aaa and falsely reading "behind".
+ _seed_disk_cache(tmp_path, "b9596-mix-aaa")
+ install_dir = tmp_path / "llama.cpp"
+ _write_marker(
+ install_dir,
+ tag = "b9596",
+ release_tag = "b9596-mix-bbb",
+ installed_at_utc = (datetime.now(tz = timezone.utc) - timedelta(days = 5))
+ .isoformat()
+ .replace("+00:00", "Z"),
+ )
+ bin_path = _fake_binary(install_dir, layout = "root")
+ # GitHub unreachable for the rest of the test (the offline post-install
+ # refresh, and the later status check).
+ monkeypatch.setattr(fr, "_fetch_latest_release_tag", lambda repo, timeout = 5.0: None)
+
+ fr.reset_caches(drop_disk = True) # exactly what the apply path now does
+ info = fr.check_prebuilt_freshness(str(bin_path))
+ assert info["latest_tag"] is None
+ assert info["behind"] is False
+ assert info["stale"] is False
+
+
+def test_in_memory_only_reset_replays_stale_same_base_mix(monkeypatch, tmp_path):
+ # Contrast/guard for the case above: an in-memory-only reset leaves the
+ # stale same-base mix on disk, so an offline check replays it and falsely
+ # reads behind/stale. This is exactly the failure drop_disk removes; if a
+ # future change makes the no-arg reset also clear disk, the apply-path call
+ # and this guard should be revisited together.
+ _seed_disk_cache(tmp_path, "b9596-mix-aaa")
+ install_dir = tmp_path / "llama.cpp"
+ _write_marker(
+ install_dir,
+ tag = "b9596",
+ release_tag = "b9596-mix-bbb",
+ installed_at_utc = (datetime.now(tz = timezone.utc) - timedelta(days = 5))
+ .isoformat()
+ .replace("+00:00", "Z"),
+ )
+ bin_path = _fake_binary(install_dir, layout = "root")
+ monkeypatch.setattr(fr, "_fetch_latest_release_tag", lambda repo, timeout = 5.0: None)
+
+ fr.reset_caches() # in-memory only -> stale disk value survives
+ info = fr.check_prebuilt_freshness(str(bin_path))
+ assert info["latest_tag"] == "b9596-mix-aaa"
+ assert info["behind"] is True
+ assert info["stale"] is True
diff --git a/studio/backend/utils/llama_cpp_freshness.py b/studio/backend/utils/llama_cpp_freshness.py
index f5fd745334..87d0d2ec01 100644
--- a/studio/backend/utils/llama_cpp_freshness.py
+++ b/studio/backend/utils/llama_cpp_freshness.py
@@ -301,7 +301,22 @@ def format_stale_warning(info: dict) -> str:
)
-def reset_caches() -> None:
- """Test-only: drop all in-memory caches."""
+def reset_caches(*, drop_disk: bool = False) -> None:
+ """Drop the in-memory freshness caches. The no-arg form is test-only.
+
+ With ``drop_disk = True`` also delete the on-disk 24h release cache. Used by
+ the post-install/update path: in-memory clearing alone leaves the stale
+ same-base value on disk, so if the post-install GitHub refresh can't reach
+ the network, ``latest_published_release`` would replay that stale disk value
+ (see its last-good fallback) and the banner could linger. Dropping the disk
+ cache makes latest read as None in that offline case, so the banner fails
+ open (off) instead of pointing at the just-replaced build."""
_marker_cache.clear()
_release_memo.clear()
+ if drop_disk:
+ import shutil
+
+ # _cache_dir() is a dedicated freshness-only subdir; it is re-created on
+ # the next _save_disk_cache. ignore_errors so a missing/locked dir is a
+ # no-op rather than breaking an otherwise successful install.
+ shutil.rmtree(_cache_dir(), ignore_errors = True)
diff --git a/studio/backend/utils/llama_cpp_update.py b/studio/backend/utils/llama_cpp_update.py
index 654ade6cd4..6eb34ffe34 100644
--- a/studio/backend/utils/llama_cpp_update.py
+++ b/studio/backend/utils/llama_cpp_update.py
@@ -406,10 +406,13 @@ def _run_update(install_dir: Path, repo: str, asset: Optional[str], script: Path
tail = "".join(tail_lines).strip()[-1500:]
raise RuntimeError(f"installer exited {returncode}: {tail or 'no output'}")
- # New UNSLOTH_PREBUILT_INFO.json is on disk; drop in-memory caches and
- # re-prime the 24h disk freshness cache with the true newest, so the
- # banner can't linger on a stale same-base value after the swap.
- reset_caches()
+ # New UNSLOTH_PREBUILT_INFO.json is on disk; drop the in-memory AND the
+ # on-disk freshness caches, then re-prime the 24h disk cache with the
+ # true newest, so the banner can't linger on a stale same-base value
+ # after the swap. drop_disk matters when the refresh below can't reach
+ # GitHub: without it, latest_published_release would replay the stale
+ # disk value; with it, latest reads as None and the banner fails open.
+ reset_caches(drop_disk = True)
try:
latest_published_release(repo, force_refresh = True)
except Exception as exc: # pragma: no cover - network defensive
From e0d6674ff6f19498f6825f2ff2c38e8091674fd2 Mon Sep 17 00:00:00 2001
From: Daniel Han
Date: Fri, 12 Jun 2026 02:54:26 -0700
Subject: [PATCH 24/50] Add RAG runtime deps to no-torch-runtime.txt (#6236)
The --local / GGUF-only install resolves its Python deps from
no-torch-runtime.txt, installed with --no-deps. That file was missing the
RAG group that studio.txt declares (sqlite-vec, pymupdf, python-docx), so a
fresh `unsloth studio` came up with RAG disabled: rag_db.py cannot import
sqlite_vec and logs "RAG unavailable: sqlite-vec extension could not be
loaded", and the knowledge-base routes return 503. python-docx was also
absent, so DOCX ingestion failed.
Add the three RAG store and document-parsing deps with the same pins as
studio.txt so knowledge bases work out of the box on the no-torch path.
sentence-transformers (dense embeddings) was already present.
---
studio/backend/requirements/no-torch-runtime.txt | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/studio/backend/requirements/no-torch-runtime.txt b/studio/backend/requirements/no-torch-runtime.txt
index 85294114b1..6efe91d448 100644
--- a/studio/backend/requirements/no-torch-runtime.txt
+++ b/studio/backend/requirements/no-torch-runtime.txt
@@ -68,3 +68,9 @@ trl>=0.18.2,!=0.19.0,<=0.24.0
sentence-transformers
cut_cross_entropy
pillow
+
+# RAG store + document parsing, mirroring studio.txt. Pinned here because
+# this file installs --no-deps; without them Studio runs with RAG disabled.
+sqlite-vec==0.1.9
+pymupdf==1.27.2.3
+python-docx==1.2.0
From 068b2c120fa389f93e3027def6f44bd4cc98f39e Mon Sep 17 00:00:00 2001
From: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
Date: Fri, 12 Jun 2026 03:03:24 -0700
Subject: [PATCH 25/50] Studio: rounded rectangle hover states for menu items
instead of pills (#6210)
* Studio: use rounded rectangles for menu item hover states instead of pills
Dropdown, select, and model picker items previously used fully rounded
pill highlights. Switch them to an 11px rounded rectangle so hover and
selected states match across the plus menu, profile menu, run settings,
selects, and the model picker. Also add a small side gutter to the plus
menu so item highlights sit slightly inset from the menu edge.
* Studio: concentric menu corners, wider gutters, single-item pill menus
Container radius now equals the item hover radius plus the side gutter
(12px + 10px = 22px) so the curves run parallel. Menus with a single
item render as fully rounded pills. The profile menu gets the same
gutter and hover radius. Model picker rows go back to their original
fully rounded hover.
---
.../frontend/src/components/app-sidebar.tsx | 2 +-
.../src/components/ui/dropdown-menu.tsx | 8 ++--
studio/frontend/src/components/ui/select.tsx | 2 +-
studio/frontend/src/index.css | 40 +++++++++++++------
4 files changed, 34 insertions(+), 18 deletions(-)
diff --git a/studio/frontend/src/components/app-sidebar.tsx b/studio/frontend/src/components/app-sidebar.tsx
index f1a200b09b..faa98abeea 100644
--- a/studio/frontend/src/components/app-sidebar.tsx
+++ b/studio/frontend/src/components/app-sidebar.tsx
@@ -1099,7 +1099,7 @@ export function AppSidebar() {
side="top"
align="center"
sideOffset={8}
- className="app-user-menu menu-soft-surface-up ring-0 w-[16rem] px-1.5 py-2.5 font-heading rounded-[20px] border-0"
+ className="app-user-menu menu-soft-surface-up ring-0 w-[16rem] px-2.5 py-2.5 font-heading rounded-[20px] border-0"
>
:nth-child(2))) {
+ border-radius: 9999px !important;
+ }
+ .unsloth-plus-menu[data-slot]:not(:has(> :nth-child(2)))
+ :is([data-slot="dropdown-menu-item"], [data-slot="dropdown-menu-sub-trigger"]) {
+ border-radius: 9999px;
}
.dark .unsloth-plus-menu[data-slot] {
@@ -1319,14 +1334,15 @@
[data-slot="dropdown-menu-item"],
[data-slot="dropdown-menu-sub-trigger"]
) {
- @apply gap-3 pl-4 pr-3 py-2 text-[14px];
+ @apply gap-3 pl-3 pr-3 py-2 text-[14px];
cursor: pointer;
- /* Pin hover-box radius so dark matches light (same as the container). */
- border-radius: 1.1rem;
+ /* Pin hover-box radius so dark matches light (container radius minus the
+ side gutter keeps the curves concentric). */
+ border-radius: 12px;
}
.unsloth-plus-menu [data-slot="dropdown-menu-label"] {
- @apply pl-4 pr-3 py-1.5 text-[12px];
+ @apply pl-3 pr-3 py-1.5 text-[12px];
}
/* Active (green) items keep their primary text and icon color on hover. */
@@ -1370,8 +1386,8 @@
[data-slot="dropdown-menu-sub-trigger"]
)
svg {
- width: 1.05rem;
- height: 1.05rem;
+ width: 1.15rem;
+ height: 1.15rem;
}
/* Destructive items keep red text and a red-tinted hover, not the grey one. */
From c773d45a2ead76167c5354154fbe78f02462ad27 Mon Sep 17 00:00:00 2001
From: Agnibha Mukherjee
Date: Fri, 12 Jun 2026 15:37:04 +0530
Subject: [PATCH 26/50] docs: repository cleanup (#5617)
* docs: small repository cleanup
* docs: improve contribution guidelines
---------
Co-authored-by: Agnibha007
Co-authored-by: Roland Tannous <115670425+rolandtannous@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
---
CONTRIBUTING.md | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index eb60a5a201..6eb8d1bc6e 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -27,3 +27,9 @@ Your support extends beyond code:
Finally, please be mindful of our [Code of Conduct](https://github.com/unslothai/unsloth/blob/main/CODE_OF_CONDUCT.md) to ensure a welcoming and inclusive environment for everyone.
Thank you so much for reading and we hope you have lots of fun using Unsloth! 🦥
+
+
+## Pull Request Guidelines
+- Keep PRs focused on a single change
+- Include a concise description and motivation
+- Link related issues when applicable
From 36ea9a9196938fcc18a5690ab0687295a1715669 Mon Sep 17 00:00:00 2001
From: Daniel Han
Date: Fri, 12 Jun 2026 03:40:50 -0700
Subject: [PATCH 27/50] Run cross-platform parity test on Windows and macOS in
CI (#6241)
---
.../workflows/cross-platform-parity-ci.yml | 61 +++++++++++++++++++
1 file changed, 61 insertions(+)
create mode 100644 .github/workflows/cross-platform-parity-ci.yml
diff --git a/.github/workflows/cross-platform-parity-ci.yml b/.github/workflows/cross-platform-parity-ci.yml
new file mode 100644
index 0000000000..4632794587
--- /dev/null
+++ b/.github/workflows/cross-platform-parity-ci.yml
@@ -0,0 +1,61 @@
+# SPDX-License-Identifier: AGPL-3.0-only
+# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.
+
+# Runs tests/python/test_cross_platform_parity.py on Windows and macOS.
+#
+# Why: that test is the guard that install.sh and install.ps1 stay in
+# sync, but today it only runs on ubuntu-latest (auto-discovered by
+# studio-backend-ci.yml's "Repo tests (CPU)" job). The test reads both
+# installer scripts, and on Windows Path.read_text() defaults to the
+# cp1252 locale encoding, so a non-cp1252 byte in install.sh (it already
+# contains a U+274C) raises UnicodeDecodeError there even though Linux and
+# macOS default to UTF-8. The reads were pinned to encoding="utf-8" in
+# #6166; this job keeps that from silently regressing by exercising the
+# test on the platforms it claims parity for. Pure pytest, no GPU,
+# sub-second, so the matrix is cheap.
+
+name: Cross-platform parity
+
+on:
+ pull_request:
+ paths:
+ - 'install.sh'
+ - 'install.ps1'
+ - 'tests/python/test_cross_platform_parity.py'
+ - '.github/workflows/cross-platform-parity-ci.yml'
+ push:
+ branches: [main]
+ paths:
+ - 'install.sh'
+ - 'install.ps1'
+ - 'tests/python/test_cross_platform_parity.py'
+ - '.github/workflows/cross-platform-parity-ci.yml'
+ workflow_dispatch:
+
+concurrency:
+ group: ${{ github.workflow }}-${{ github.ref }}
+ cancel-in-progress: true
+
+permissions:
+ contents: read
+
+jobs:
+ parity:
+ name: parity (${{ matrix.os }})
+ strategy:
+ fail-fast: false
+ matrix:
+ os: [windows-latest, macos-latest]
+ runs-on: ${{ matrix.os }}
+ timeout-minutes: 10
+ steps:
+ - uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
+ with:
+ persist-credentials: false
+ - uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
+ with:
+ python-version: '3.12'
+ cache: 'pip'
+ - run: python -m pip install -U pip pytest
+ - name: Cross-platform parity test
+ run: python -m pytest tests/python/test_cross_platform_parity.py -q
From 6d206b488c46d8407336c7f762cd72ed3b9b687b Mon Sep 17 00:00:00 2001
From: Daniel Han
Date: Fri, 12 Jun 2026 03:51:59 -0700
Subject: [PATCH 28/50] chore(studio/frontend): normalize line endings to LF
(#6012)
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
* chore(studio/frontend): normalize line endings to LF
45 source files under studio/frontend/ were committed with CRLF or mixed
line endings while the rest of the repo and the JS/TS tooling assume LF.
Add a scoped `studio/frontend/** text=auto eol=lf` rule to .gitattributes
and run `git add --renormalize studio/frontend` so these files are stored
with LF in the index. The rule is scoped to the frontend tree (not a
repo-wide *.ts/*.tsx/... policy) so it cannot force LF on files elsewhere;
text=auto leaves binary assets (logos, fonts) untouched.
This commit is whitespace-only (CRLF -> LF) — no source content changed
(verified with `git diff --ignore-cr-at-eol`). It is intentionally
isolated so it can be listed in .git-blame-ignore-revs and skipped by
reviewers and `git blame`.
Co-Authored-By: Claude Opus 4.8
* chore: ignore the frontend LF-normalization commit in git blame
Add .git-blame-ignore-revs listing the whitespace-only line-ending
normalization commit so it doesn't pollute `git blame` output. GitHub
applies this file automatically; locally run
`git config blame.ignoreRevsFile .git-blame-ignore-revs`.
Co-Authored-By: Claude Opus 4.8
---------
Co-authored-by: Claude Opus 4.8
---
.git-blame-ignore-revs | 8 +
.gitattributes | 6 +
studio/frontend/index.html | 26 +-
.../frontend/public/hub/profile/logo/meta.svg | 36 +-
.../public/provider-logos/misc/meta.svg | 36 +-
.../frontend/src/components/ui/accordion.tsx | 190 ++---
.../src/components/ui/alert-dialog.tsx | 368 ++++-----
studio/frontend/src/components/ui/alert.tsx | 152 ++--
.../src/components/ui/animated-shiny-text.tsx | 76 +-
.../src/components/ui/aspect-ratio.tsx | 18 +-
studio/frontend/src/components/ui/avatar.tsx | 220 ++---
studio/frontend/src/components/ui/badge.tsx | 102 +--
.../frontend/src/components/ui/breadcrumb.tsx | 246 +++---
.../frontend/src/components/ui/calendar.tsx | 468 +++++------
studio/frontend/src/components/ui/card.tsx | 200 ++---
studio/frontend/src/components/ui/chart.tsx | 718 ++++++++--------
.../frontend/src/components/ui/checkbox.tsx | 62 +-
.../frontend/src/components/ui/combobox.tsx | 764 +++++++++---------
studio/frontend/src/components/ui/command.tsx | 414 +++++-----
.../src/components/ui/context-menu.tsx | 528 ++++++------
.../src/components/ui/dropdown-menu.tsx | 558 ++++++-------
studio/frontend/src/components/ui/field.tsx | 472 +++++------
.../frontend/src/components/ui/hover-card.tsx | 90 +--
.../src/components/ui/input-group.tsx | 306 +++----
studio/frontend/src/components/ui/input.tsx | 38 +-
studio/frontend/src/components/ui/label.tsx | 48 +-
.../frontend/src/components/ui/light-rays.tsx | 286 +++----
studio/frontend/src/components/ui/menubar.tsx | 562 ++++++-------
.../src/components/ui/navigation-menu.tsx | 348 ++++----
.../frontend/src/components/ui/pagination.tsx | 276 +++----
studio/frontend/src/components/ui/popover.tsx | 184 ++---
.../frontend/src/components/ui/progress.tsx | 74 +-
.../src/components/ui/radio-group.tsx | 96 +--
.../src/components/ui/scroll-area.tsx | 110 +--
.../frontend/src/components/ui/separator.tsx | 52 +-
.../frontend/src/components/ui/skeleton.tsx | 26 +-
studio/frontend/src/components/ui/sonner.tsx | 168 ++--
.../src/components/ui/sparkles-text.tsx | 308 +++----
studio/frontend/src/components/ui/switch.tsx | 62 +-
studio/frontend/src/components/ui/table.tsx | 228 +++---
studio/frontend/src/components/ui/tabs.tsx | 268 +++---
.../frontend/src/components/ui/textarea.tsx | 10 +-
.../src/components/ui/toggle-group.tsx | 178 ++--
studio/frontend/src/components/ui/toggle.tsx | 92 +--
.../inline/inline-category-badges.tsx | 150 ++--
studio/frontend/tsconfig.app.json | 62 +-
studio/frontend/tsconfig.json | 26 +-
studio/frontend/tsconfig.node.json | 52 +-
48 files changed, 4891 insertions(+), 4877 deletions(-)
create mode 100644 .git-blame-ignore-revs
diff --git a/.git-blame-ignore-revs b/.git-blame-ignore-revs
new file mode 100644
index 0000000000..17d96cd0f5
--- /dev/null
+++ b/.git-blame-ignore-revs
@@ -0,0 +1,8 @@
+# Commits listed here are skipped by `git blame` so that bulk, whitespace-only
+# changes don't obscure the real authorship of a line.
+#
+# GitHub honors this file automatically. To use it locally, run once:
+# git config blame.ignoreRevsFile .git-blame-ignore-revs
+
+# chore(studio/frontend): normalize line endings to LF
+c50b8ab910f5aa56dd7ae0022d2c7b96bfe3384a
diff --git a/.gitattributes b/.gitattributes
index 75fba5d6ab..5f04b5e9d1 100644
--- a/.gitattributes
+++ b/.gitattributes
@@ -5,3 +5,9 @@
# clone (core.autocrlf=true) rewrites them to CRLF, and the trailing \r breaks
# them when run in WSL/Linux (e.g. `set -e` -> "set: Illegal option -").
*.sh text eol=lf
+
+# Normalize Studio frontend sources to LF. Scoped to the frontend tree (rather
+# than repo-wide *.ts/*.tsx/... rules) so the policy can't force LF on files
+# elsewhere. text=auto lets Git detect and leave binary assets (logos, fonts)
+# untouched while text files (.ts/.tsx/.json/.html/.svg/...) are stored as LF.
+studio/frontend/** text=auto eol=lf
diff --git a/studio/frontend/index.html b/studio/frontend/index.html
index 4f81ffd4ff..0fbb4eaeeb 100644
--- a/studio/frontend/index.html
+++ b/studio/frontend/index.html
@@ -1,16 +1,16 @@
-
+
-
-
-
-
-
- Unsloth Studio
-
-
-
-
-
-
+
+
+
+
+
+ Unsloth Studio
+
+
+
+
+
+
diff --git a/studio/frontend/public/hub/profile/logo/meta.svg b/studio/frontend/public/hub/profile/logo/meta.svg
index 9fa656bd6b..fe3709aeea 100644
--- a/studio/frontend/public/hub/profile/logo/meta.svg
+++ b/studio/frontend/public/hub/profile/logo/meta.svg
@@ -1,19 +1,19 @@
-
-
-Logo of Meta Platforms -- Graphic created by Detmar Owen
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+
+
+Logo of Meta Platforms -- Graphic created by Detmar Owen
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/studio/frontend/public/provider-logos/misc/meta.svg b/studio/frontend/public/provider-logos/misc/meta.svg
index 9fa656bd6b..fe3709aeea 100644
--- a/studio/frontend/public/provider-logos/misc/meta.svg
+++ b/studio/frontend/public/provider-logos/misc/meta.svg
@@ -1,19 +1,19 @@
-
-
-Logo of Meta Platforms -- Graphic created by Detmar Owen
-
-
-
-
-
-
-
-
-
-
-
-
-
-
-
+
+
+Logo of Meta Platforms -- Graphic created by Detmar Owen
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/studio/frontend/src/components/ui/accordion.tsx b/studio/frontend/src/components/ui/accordion.tsx
index 7754c78a11..35de233858 100644
--- a/studio/frontend/src/components/ui/accordion.tsx
+++ b/studio/frontend/src/components/ui/accordion.tsx
@@ -1,98 +1,98 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
-"use client";
-
-import { Accordion as AccordionPrimitive } from "radix-ui";
-import type * as React from "react";
-
-import { cn } from "@/lib/utils";
-import { ArrowDown01Icon, ArrowUp01Icon } from "@hugeicons/core-free-icons";
-import { HugeiconsIcon } from "@hugeicons/react";
-
-function Accordion({
- className,
- ...props
-}: React.ComponentProps) {
- return (
-
- );
-}
-
-function AccordionItem({
- className,
- ...props
-}: React.ComponentProps) {
- return (
-
- );
-}
-
-function AccordionTrigger({
- className,
- children,
- ...props
-}: React.ComponentProps) {
- return (
-
-
- {children}
-
-
-
-
- );
-}
-
-function AccordionContent({
- className,
- children,
- ...props
-}: React.ComponentProps) {
- return (
-
-
- {children}
-
-
- );
-}
-
-export { Accordion, AccordionItem, AccordionTrigger, AccordionContent };
+"use client";
+
+import { Accordion as AccordionPrimitive } from "radix-ui";
+import type * as React from "react";
+
+import { cn } from "@/lib/utils";
+import { ArrowDown01Icon, ArrowUp01Icon } from "@hugeicons/core-free-icons";
+import { HugeiconsIcon } from "@hugeicons/react";
+
+function Accordion({
+ className,
+ ...props
+}: React.ComponentProps) {
+ return (
+
+ );
+}
+
+function AccordionItem({
+ className,
+ ...props
+}: React.ComponentProps) {
+ return (
+
+ );
+}
+
+function AccordionTrigger({
+ className,
+ children,
+ ...props
+}: React.ComponentProps) {
+ return (
+
+
+ {children}
+
+
+
+
+ );
+}
+
+function AccordionContent({
+ className,
+ children,
+ ...props
+}: React.ComponentProps) {
+ return (
+
+
+ {children}
+
+
+ );
+}
+
+export { Accordion, AccordionItem, AccordionTrigger, AccordionContent };
diff --git a/studio/frontend/src/components/ui/alert-dialog.tsx b/studio/frontend/src/components/ui/alert-dialog.tsx
index f5c1dbacca..97f4be7f44 100644
--- a/studio/frontend/src/components/ui/alert-dialog.tsx
+++ b/studio/frontend/src/components/ui/alert-dialog.tsx
@@ -1,50 +1,50 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
-import { AlertDialog as AlertDialogPrimitive } from "radix-ui";
-import type * as React from "react";
-
-import { Button } from "@/components/ui/button";
-import { cn } from "@/lib/utils";
-
-function AlertDialog({
- ...props
-}: React.ComponentProps) {
- return ;
-}
-
-function AlertDialogTrigger({
- ...props
-}: React.ComponentProps) {
- return (
-
- );
-}
-
-function AlertDialogPortal({
- ...props
-}: React.ComponentProps) {
- return (
-
- );
-}
-
-function AlertDialogOverlay({
- className,
- ...props
-}: React.ComponentProps) {
- return (
-
- );
-}
-
+import { AlertDialog as AlertDialogPrimitive } from "radix-ui";
+import type * as React from "react";
+
+import { Button } from "@/components/ui/button";
+import { cn } from "@/lib/utils";
+
+function AlertDialog({
+ ...props
+}: React.ComponentProps) {
+ return ;
+}
+
+function AlertDialogTrigger({
+ ...props
+}: React.ComponentProps) {
+ return (
+
+ );
+}
+
+function AlertDialogPortal({
+ ...props
+}: React.ComponentProps) {
+ return (
+
+ );
+}
+
+function AlertDialogOverlay({
+ className,
+ ...props
+}: React.ComponentProps) {
+ return (
+
+ );
+}
+
function AlertDialogContent({
className,
size = "default",
@@ -60,143 +60,143 @@ function AlertDialogContent({
-
- );
-}
-
-function AlertDialogHeader({
- className,
- ...props
-}: React.ComponentProps<"div">) {
- return (
-
- );
-}
-
-function AlertDialogFooter({
- className,
- ...props
-}: React.ComponentProps<"div">) {
- return (
-
- );
-}
-
-function AlertDialogMedia({
- className,
- ...props
-}: React.ComponentProps<"div">) {
- return (
-
- );
-}
-
-function AlertDialogTitle({
- className,
- ...props
-}: React.ComponentProps) {
- return (
-
- );
-}
-
-function AlertDialogDescription({
- className,
- ...props
-}: React.ComponentProps) {
- return (
-
- );
-}
-
-function AlertDialogAction({
- className,
- variant = "default",
- size = "default",
- ...props
-}: React.ComponentProps &
- Pick, "variant" | "size">) {
- return (
-
-
-
- );
-}
-
-function AlertDialogCancel({
- className,
- variant = "outline",
- size = "default",
- ...props
-}: React.ComponentProps &
- Pick, "variant" | "size">) {
- return (
-
-
-
- );
-}
-
-export {
- AlertDialog,
- AlertDialogAction,
- AlertDialogCancel,
- AlertDialogContent,
- AlertDialogDescription,
- AlertDialogFooter,
- AlertDialogHeader,
- AlertDialogMedia,
- AlertDialogOverlay,
- AlertDialogPortal,
- AlertDialogTitle,
- AlertDialogTrigger,
-};
+ className={cn(
+ "data-open:animate-in data-closed:animate-out data-closed:fade-out-0 data-open:fade-in-0 data-closed:zoom-out-95 data-open:zoom-in-95 bg-background ring-foreground/5 gap-6 rounded-4xl p-6 ring-1 duration-100 data-[size=default]:max-w-xs data-[size=sm]:max-w-xs data-[size=default]:sm:max-w-md group/alert-dialog-content fixed top-1/2 left-1/2 z-50 grid w-full -translate-x-1/2 -translate-y-1/2 outline-none",
+ className,
+ )}
+ {...props}
+ />
+
+ );
+}
+
+function AlertDialogHeader({
+ className,
+ ...props
+}: React.ComponentProps<"div">) {
+ return (
+
+ );
+}
+
+function AlertDialogFooter({
+ className,
+ ...props
+}: React.ComponentProps<"div">) {
+ return (
+
+ );
+}
+
+function AlertDialogMedia({
+ className,
+ ...props
+}: React.ComponentProps<"div">) {
+ return (
+
+ );
+}
+
+function AlertDialogTitle({
+ className,
+ ...props
+}: React.ComponentProps) {
+ return (
+
+ );
+}
+
+function AlertDialogDescription({
+ className,
+ ...props
+}: React.ComponentProps) {
+ return (
+
+ );
+}
+
+function AlertDialogAction({
+ className,
+ variant = "default",
+ size = "default",
+ ...props
+}: React.ComponentProps &
+ Pick, "variant" | "size">) {
+ return (
+
+
+
+ );
+}
+
+function AlertDialogCancel({
+ className,
+ variant = "outline",
+ size = "default",
+ ...props
+}: React.ComponentProps &
+ Pick, "variant" | "size">) {
+ return (
+
+
+
+ );
+}
+
+export {
+ AlertDialog,
+ AlertDialogAction,
+ AlertDialogCancel,
+ AlertDialogContent,
+ AlertDialogDescription,
+ AlertDialogFooter,
+ AlertDialogHeader,
+ AlertDialogMedia,
+ AlertDialogOverlay,
+ AlertDialogPortal,
+ AlertDialogTitle,
+ AlertDialogTrigger,
+};
diff --git a/studio/frontend/src/components/ui/alert.tsx b/studio/frontend/src/components/ui/alert.tsx
index a4a5f4c4b7..094b607d9a 100644
--- a/studio/frontend/src/components/ui/alert.tsx
+++ b/studio/frontend/src/components/ui/alert.tsx
@@ -1,79 +1,79 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
-import { type VariantProps, cva } from "class-variance-authority";
-import type * as React from "react";
-
-import { cn } from "@/lib/utils";
-
-const alertVariants = cva(
- "grid gap-0.5 rounded-lg border px-4 py-3 text-left text-sm has-data-[slot=alert-action]:relative has-data-[slot=alert-action]:pr-18 has-[>svg]:grid-cols-[auto_1fr] has-[>svg]:gap-x-2.5 *:[svg]:row-span-2 *:[svg]:translate-y-0.5 *:[svg]:text-current *:[svg:not([class*='size-'])]:size-4 w-full relative group/alert",
- {
- variants: {
- variant: {
- default: "bg-card text-card-foreground",
- destructive:
- "text-destructive bg-card *:data-[slot=alert-description]:text-destructive/90 *:[svg]:text-current",
- },
- },
- defaultVariants: {
- variant: "default",
- },
- },
-);
-
-function Alert({
- className,
- variant,
- ...props
-}: React.ComponentProps<"div"> & VariantProps) {
- return (
-
- );
-}
-
-function AlertTitle({ className, ...props }: React.ComponentProps<"div">) {
- return (
- svg]/alert:col-start-2 [&_a]:hover:text-foreground [&_a]:underline [&_a]:underline-offset-3",
- className,
- )}
- {...props}
- />
- );
-}
-
-function AlertDescription({
- className,
- ...props
-}: React.ComponentProps<"div">) {
- return (
-
- );
-}
-
-function AlertAction({ className, ...props }: React.ComponentProps<"div">) {
- return (
-
- );
-}
-
-export { Alert, AlertTitle, AlertDescription, AlertAction };
+import { type VariantProps, cva } from "class-variance-authority";
+import type * as React from "react";
+
+import { cn } from "@/lib/utils";
+
+const alertVariants = cva(
+ "grid gap-0.5 rounded-lg border px-4 py-3 text-left text-sm has-data-[slot=alert-action]:relative has-data-[slot=alert-action]:pr-18 has-[>svg]:grid-cols-[auto_1fr] has-[>svg]:gap-x-2.5 *:[svg]:row-span-2 *:[svg]:translate-y-0.5 *:[svg]:text-current *:[svg:not([class*='size-'])]:size-4 w-full relative group/alert",
+ {
+ variants: {
+ variant: {
+ default: "bg-card text-card-foreground",
+ destructive:
+ "text-destructive bg-card *:data-[slot=alert-description]:text-destructive/90 *:[svg]:text-current",
+ },
+ },
+ defaultVariants: {
+ variant: "default",
+ },
+ },
+);
+
+function Alert({
+ className,
+ variant,
+ ...props
+}: React.ComponentProps<"div"> & VariantProps
) {
+ return (
+
+ );
+}
+
+function AlertTitle({ className, ...props }: React.ComponentProps<"div">) {
+ return (
+ svg]/alert:col-start-2 [&_a]:hover:text-foreground [&_a]:underline [&_a]:underline-offset-3",
+ className,
+ )}
+ {...props}
+ />
+ );
+}
+
+function AlertDescription({
+ className,
+ ...props
+}: React.ComponentProps<"div">) {
+ return (
+
+ );
+}
+
+function AlertAction({ className, ...props }: React.ComponentProps<"div">) {
+ return (
+
+ );
+}
+
+export { Alert, AlertTitle, AlertDescription, AlertAction };
diff --git a/studio/frontend/src/components/ui/animated-shiny-text.tsx b/studio/frontend/src/components/ui/animated-shiny-text.tsx
index 4c650f1003..8d366ca3d6 100644
--- a/studio/frontend/src/components/ui/animated-shiny-text.tsx
+++ b/studio/frontend/src/components/ui/animated-shiny-text.tsx
@@ -1,41 +1,41 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
-import type { ComponentPropsWithoutRef, CSSProperties, FC } from "react"
-
-import { cn } from "@/lib/utils"
-
-export interface AnimatedShinyTextProps extends ComponentPropsWithoutRef<"span"> {
- shimmerWidth?: number
-}
-
-export const AnimatedShinyText: FC
= ({
- children,
- className,
- shimmerWidth = 100,
- ...props
-}) => {
- return (
-
- {children}
-
- )
-}
+import type { ComponentPropsWithoutRef, CSSProperties, FC } from "react"
+
+import { cn } from "@/lib/utils"
+
+export interface AnimatedShinyTextProps extends ComponentPropsWithoutRef<"span"> {
+ shimmerWidth?: number
+}
+
+export const AnimatedShinyText: FC = ({
+ children,
+ className,
+ shimmerWidth = 100,
+ ...props
+}) => {
+ return (
+
+ {children}
+
+ )
+}
diff --git a/studio/frontend/src/components/ui/aspect-ratio.tsx b/studio/frontend/src/components/ui/aspect-ratio.tsx
index cb605f01eb..2471f4333d 100644
--- a/studio/frontend/src/components/ui/aspect-ratio.tsx
+++ b/studio/frontend/src/components/ui/aspect-ratio.tsx
@@ -1,12 +1,12 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
-import { AspectRatio as AspectRatioPrimitive } from "radix-ui";
-
-function AspectRatio({
- ...props
-}: React.ComponentProps) {
- return ;
-}
-
-export { AspectRatio };
+import { AspectRatio as AspectRatioPrimitive } from "radix-ui";
+
+function AspectRatio({
+ ...props
+}: React.ComponentProps) {
+ return ;
+}
+
+export { AspectRatio };
diff --git a/studio/frontend/src/components/ui/avatar.tsx b/studio/frontend/src/components/ui/avatar.tsx
index 2250bb849a..31262b32f7 100644
--- a/studio/frontend/src/components/ui/avatar.tsx
+++ b/studio/frontend/src/components/ui/avatar.tsx
@@ -1,113 +1,113 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
-import { Avatar as AvatarPrimitive } from "radix-ui";
-import type * as React from "react";
-
-import { cn } from "@/lib/utils";
-
-function Avatar({
- className,
- size = "default",
- ...props
-}: React.ComponentProps & {
- size?: "default" | "sm" | "lg";
-}) {
- return (
-
- );
-}
-
-function AvatarImage({
- className,
- ...props
-}: React.ComponentProps) {
- return (
-
- );
-}
-
-function AvatarFallback({
- className,
- ...props
-}: React.ComponentProps) {
- return (
-
- );
-}
-
-function AvatarBadge({ className, ...props }: React.ComponentProps<"span">) {
- return (
- svg]:hidden",
- "group-data-[size=default]/avatar:size-2.5 group-data-[size=default]/avatar:[&>svg]:size-2",
- "group-data-[size=lg]/avatar:size-3 group-data-[size=lg]/avatar:[&>svg]:size-2",
- className,
- )}
- {...props}
- />
- );
-}
-
-function AvatarGroup({ className, ...props }: React.ComponentProps<"div">) {
- return (
-
- );
-}
-
-function AvatarGroupCount({
- className,
- ...props
-}: React.ComponentProps<"div">) {
- return (
- svg]:size-4 group-has-data-[size=lg]/avatar-group:[&>svg]:size-5 group-has-data-[size=sm]/avatar-group:[&>svg]:size-3 ring-background relative flex shrink-0 items-center justify-center ring-2",
- className,
- )}
- {...props}
- />
- );
-}
-
-export {
- Avatar,
- AvatarImage,
- AvatarFallback,
- AvatarGroup,
- AvatarGroupCount,
- AvatarBadge,
-};
+import { Avatar as AvatarPrimitive } from "radix-ui";
+import type * as React from "react";
+
+import { cn } from "@/lib/utils";
+
+function Avatar({
+ className,
+ size = "default",
+ ...props
+}: React.ComponentProps
& {
+ size?: "default" | "sm" | "lg";
+}) {
+ return (
+
+ );
+}
+
+function AvatarImage({
+ className,
+ ...props
+}: React.ComponentProps) {
+ return (
+
+ );
+}
+
+function AvatarFallback({
+ className,
+ ...props
+}: React.ComponentProps) {
+ return (
+
+ );
+}
+
+function AvatarBadge({ className, ...props }: React.ComponentProps<"span">) {
+ return (
+ svg]:hidden",
+ "group-data-[size=default]/avatar:size-2.5 group-data-[size=default]/avatar:[&>svg]:size-2",
+ "group-data-[size=lg]/avatar:size-3 group-data-[size=lg]/avatar:[&>svg]:size-2",
+ className,
+ )}
+ {...props}
+ />
+ );
+}
+
+function AvatarGroup({ className, ...props }: React.ComponentProps<"div">) {
+ return (
+
+ );
+}
+
+function AvatarGroupCount({
+ className,
+ ...props
+}: React.ComponentProps<"div">) {
+ return (
+ svg]:size-4 group-has-data-[size=lg]/avatar-group:[&>svg]:size-5 group-has-data-[size=sm]/avatar-group:[&>svg]:size-3 ring-background relative flex shrink-0 items-center justify-center ring-2",
+ className,
+ )}
+ {...props}
+ />
+ );
+}
+
+export {
+ Avatar,
+ AvatarImage,
+ AvatarFallback,
+ AvatarGroup,
+ AvatarGroupCount,
+ AvatarBadge,
+};
diff --git a/studio/frontend/src/components/ui/badge.tsx b/studio/frontend/src/components/ui/badge.tsx
index 3951ae9de0..0f2f334986 100644
--- a/studio/frontend/src/components/ui/badge.tsx
+++ b/studio/frontend/src/components/ui/badge.tsx
@@ -1,54 +1,54 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
-/* eslint-disable react-refresh/only-export-components */
-
-import { type VariantProps, cva } from "class-variance-authority";
-import { Slot } from "radix-ui";
-import type * as React from "react";
-
-import { cn } from "@/lib/utils";
-
-export const badgeVariants = cva(
- "h-5 gap-1 rounded-4xl border border-transparent px-2 py-0.5 text-xs font-medium transition-all has-data-[icon=inline-end]:pr-1.5 has-data-[icon=inline-start]:pl-1.5 [&>svg]:size-3! inline-flex items-center justify-center w-fit whitespace-nowrap shrink-0 [&>svg]:pointer-events-none focus-visible:border-ring focus-visible:ring-ring/50 focus-visible:ring-[3px] aria-invalid:ring-destructive/20 dark:aria-invalid:ring-destructive/40 aria-invalid:border-destructive overflow-hidden group/badge",
- {
- variants: {
- variant: {
- default: "bg-primary text-primary-foreground [a]:hover:bg-primary/80",
- secondary:
- "bg-secondary text-secondary-foreground [a]:hover:bg-secondary/80",
- destructive:
- "bg-destructive/10 [a]:hover:bg-destructive/20 focus-visible:ring-destructive/20 dark:focus-visible:ring-destructive/40 text-destructive dark:bg-destructive/20",
- outline:
- "border-border text-foreground [a]:hover:bg-muted [a]:hover:text-muted-foreground bg-input/30",
- ghost:
- "hover:bg-muted hover:text-muted-foreground dark:hover:bg-muted/50",
- link: "text-primary underline-offset-4 hover:underline",
- },
- },
- defaultVariants: {
- variant: "default",
- },
- },
-);
-
-export function Badge({
- className,
- variant = "default",
- asChild = false,
- ...props
-}: React.ComponentProps<"span"> &
- VariantProps
& {
- asChild?: boolean;
- }): React.ReactElement {
- const Comp = asChild ? Slot.Root : "span";
-
- return (
-
- );
-}
+/* eslint-disable react-refresh/only-export-components */
+
+import { type VariantProps, cva } from "class-variance-authority";
+import { Slot } from "radix-ui";
+import type * as React from "react";
+
+import { cn } from "@/lib/utils";
+
+export const badgeVariants = cva(
+ "h-5 gap-1 rounded-4xl border border-transparent px-2 py-0.5 text-xs font-medium transition-all has-data-[icon=inline-end]:pr-1.5 has-data-[icon=inline-start]:pl-1.5 [&>svg]:size-3! inline-flex items-center justify-center w-fit whitespace-nowrap shrink-0 [&>svg]:pointer-events-none focus-visible:border-ring focus-visible:ring-ring/50 focus-visible:ring-[3px] aria-invalid:ring-destructive/20 dark:aria-invalid:ring-destructive/40 aria-invalid:border-destructive overflow-hidden group/badge",
+ {
+ variants: {
+ variant: {
+ default: "bg-primary text-primary-foreground [a]:hover:bg-primary/80",
+ secondary:
+ "bg-secondary text-secondary-foreground [a]:hover:bg-secondary/80",
+ destructive:
+ "bg-destructive/10 [a]:hover:bg-destructive/20 focus-visible:ring-destructive/20 dark:focus-visible:ring-destructive/40 text-destructive dark:bg-destructive/20",
+ outline:
+ "border-border text-foreground [a]:hover:bg-muted [a]:hover:text-muted-foreground bg-input/30",
+ ghost:
+ "hover:bg-muted hover:text-muted-foreground dark:hover:bg-muted/50",
+ link: "text-primary underline-offset-4 hover:underline",
+ },
+ },
+ defaultVariants: {
+ variant: "default",
+ },
+ },
+);
+
+export function Badge({
+ className,
+ variant = "default",
+ asChild = false,
+ ...props
+}: React.ComponentProps<"span"> &
+ VariantProps & {
+ asChild?: boolean;
+ }): React.ReactElement {
+ const Comp = asChild ? Slot.Root : "span";
+
+ return (
+
+ );
+}
diff --git a/studio/frontend/src/components/ui/breadcrumb.tsx b/studio/frontend/src/components/ui/breadcrumb.tsx
index dc026994ce..a2dad8783f 100644
--- a/studio/frontend/src/components/ui/breadcrumb.tsx
+++ b/studio/frontend/src/components/ui/breadcrumb.tsx
@@ -1,126 +1,126 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
-import { Slot } from "radix-ui";
-import type * as React from "react";
-
-import { cn } from "@/lib/utils";
-import {
- ArrowRight01Icon,
- MoreHorizontalCircle01Icon,
-} from "@hugeicons/core-free-icons";
-import { HugeiconsIcon } from "@hugeicons/react";
-
-function Breadcrumb({ className, ...props }: React.ComponentProps<"nav">) {
- return (
-
- );
-}
-
-function BreadcrumbList({ className, ...props }: React.ComponentProps<"ol">) {
- return (
-
- );
-}
-
-function BreadcrumbItem({ className, ...props }: React.ComponentProps<"li">) {
- return (
-
- );
-}
-
-function BreadcrumbLink({
- asChild,
- className,
- ...props
-}: React.ComponentProps<"a"> & {
- asChild?: boolean;
-}) {
- const Comp = asChild ? Slot.Root : "a";
-
- return (
-
- );
-}
-
-function BreadcrumbPage({ className, ...props }: React.ComponentProps<"span">) {
- return (
-
- );
-}
-
-function BreadcrumbSeparator({
- children,
- className,
- ...props
-}: React.ComponentProps<"li">) {
- return (
- svg]:size-3.5", className)}
- {...props}
- >
- {children ?? }
-
- );
-}
-
-function BreadcrumbEllipsis({
- className,
- ...props
-}: React.ComponentProps<"span">) {
- return (
- svg]:size-4 flex items-center justify-center",
- className,
- )}
- {...props}
- >
-
- More
-
- );
-}
-
-export {
- Breadcrumb,
- BreadcrumbList,
- BreadcrumbItem,
- BreadcrumbLink,
- BreadcrumbPage,
- BreadcrumbSeparator,
- BreadcrumbEllipsis,
-};
+import { Slot } from "radix-ui";
+import type * as React from "react";
+
+import { cn } from "@/lib/utils";
+import {
+ ArrowRight01Icon,
+ MoreHorizontalCircle01Icon,
+} from "@hugeicons/core-free-icons";
+import { HugeiconsIcon } from "@hugeicons/react";
+
+function Breadcrumb({ className, ...props }: React.ComponentProps<"nav">) {
+ return (
+
+ );
+}
+
+function BreadcrumbList({ className, ...props }: React.ComponentProps<"ol">) {
+ return (
+
+ );
+}
+
+function BreadcrumbItem({ className, ...props }: React.ComponentProps<"li">) {
+ return (
+
+ );
+}
+
+function BreadcrumbLink({
+ asChild,
+ className,
+ ...props
+}: React.ComponentProps<"a"> & {
+ asChild?: boolean;
+}) {
+ const Comp = asChild ? Slot.Root : "a";
+
+ return (
+
+ );
+}
+
+function BreadcrumbPage({ className, ...props }: React.ComponentProps<"span">) {
+ return (
+
+ );
+}
+
+function BreadcrumbSeparator({
+ children,
+ className,
+ ...props
+}: React.ComponentProps<"li">) {
+ return (
+ svg]:size-3.5", className)}
+ {...props}
+ >
+ {children ?? }
+
+ );
+}
+
+function BreadcrumbEllipsis({
+ className,
+ ...props
+}: React.ComponentProps<"span">) {
+ return (
+ svg]:size-4 flex items-center justify-center",
+ className,
+ )}
+ {...props}
+ >
+
+ More
+
+ );
+}
+
+export {
+ Breadcrumb,
+ BreadcrumbList,
+ BreadcrumbItem,
+ BreadcrumbLink,
+ BreadcrumbPage,
+ BreadcrumbSeparator,
+ BreadcrumbEllipsis,
+};
diff --git a/studio/frontend/src/components/ui/calendar.tsx b/studio/frontend/src/components/ui/calendar.tsx
index 554c5c0f7e..be52f9c354 100644
--- a/studio/frontend/src/components/ui/calendar.tsx
+++ b/studio/frontend/src/components/ui/calendar.tsx
@@ -1,237 +1,237 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
-import * as React from "react";
-import {
- type DayButton,
- DayPicker,
- getDefaultClassNames,
-} from "react-day-picker";
-
-import { Button, buttonVariants } from "@/components/ui/button";
-import { cn } from "@/lib/utils";
-import {
- ArrowDownIcon,
- ArrowLeftIcon,
- ArrowRightIcon,
-} from "@hugeicons/core-free-icons";
-import { HugeiconsIcon } from "@hugeicons/react";
-
-function Calendar({
- className,
- classNames,
- showOutsideDays = true,
- captionLayout = "label",
- buttonVariant = "ghost",
- formatters,
- components,
- ...props
-}: React.ComponentProps & {
- buttonVariant?: React.ComponentProps["variant"];
-}) {
- const defaultClassNames = getDefaultClassNames();
-
- return (
- svg]:rotate-180`,
- String.raw`rtl:**:[.rdp-button\_previous>svg]:rotate-180`,
- className,
- )}
- captionLayout={captionLayout}
- formatters={{
- formatMonthDropdown: (date) =>
- date.toLocaleString("default", { month: "short" }),
- ...formatters,
- }}
- classNames={{
- root: cn("w-fit", defaultClassNames.root),
- months: cn(
- "flex gap-4 flex-col md:flex-row relative",
- defaultClassNames.months,
- ),
- month: cn("flex flex-col w-full gap-4", defaultClassNames.month),
- nav: cn(
- "flex items-center gap-1 w-full absolute top-0 inset-x-0 justify-between",
- defaultClassNames.nav,
- ),
- button_previous: cn(
- buttonVariants({ variant: buttonVariant }),
- "size-(--cell-size) aria-disabled:opacity-50 p-0 select-none",
- defaultClassNames.button_previous,
- ),
- button_next: cn(
- buttonVariants({ variant: buttonVariant }),
- "size-(--cell-size) aria-disabled:opacity-50 p-0 select-none",
- defaultClassNames.button_next,
- ),
- month_caption: cn(
- "flex items-center justify-center h-(--cell-size) w-full px-(--cell-size)",
- defaultClassNames.month_caption,
- ),
- dropdowns: cn(
- "w-full flex items-center text-sm font-medium justify-center h-(--cell-size) gap-1.5",
- defaultClassNames.dropdowns,
- ),
- dropdown_root: cn(
- "relative cn-calendar-dropdown-root rounded-(--cell-radius)",
- defaultClassNames.dropdown_root,
- ),
- dropdown: cn(
- "absolute bg-popover inset-0 opacity-0",
- defaultClassNames.dropdown,
- ),
- caption_label: cn(
- "select-none font-medium",
- captionLayout === "label"
- ? "text-sm"
- : "cn-calendar-caption-label rounded-(--cell-radius) flex items-center gap-1 text-sm [&>svg]:text-muted-foreground [&>svg]:size-3.5",
- defaultClassNames.caption_label,
- ),
- table: "w-full border-collapse",
- weekdays: cn("flex", defaultClassNames.weekdays),
- weekday: cn(
- "text-muted-foreground rounded-(--cell-radius) flex-1 font-normal text-[0.8rem] select-none",
- defaultClassNames.weekday,
- ),
- week: cn("flex w-full mt-2", defaultClassNames.week),
- week_number_header: cn(
- "select-none w-(--cell-size)",
- defaultClassNames.week_number_header,
- ),
- week_number: cn(
- "text-[0.8rem] select-none text-muted-foreground",
- defaultClassNames.week_number,
- ),
- day: cn(
- "relative w-full rounded-(--cell-radius) h-full p-0 text-center [&:last-child[data-selected=true]_button]:rounded-r-(--cell-radius) group/day aspect-square select-none",
- props.showWeekNumber
- ? "[&:nth-child(2)[data-selected=true]_button]:rounded-l-(--cell-radius)"
- : "[&:first-child[data-selected=true]_button]:rounded-l-(--cell-radius)",
- defaultClassNames.day,
- ),
- range_start: cn(
- "rounded-l-(--cell-radius) bg-muted relative after:bg-muted after:absolute after:inset-y-0 after:w-4 after:right-0 -z-0 isolate",
- defaultClassNames.range_start,
- ),
- range_middle: cn("rounded-none", defaultClassNames.range_middle),
- range_end: cn(
- "rounded-r-(--cell-radius) bg-muted relative after:bg-muted-200 after:absolute after:inset-y-0 after:w-4 after:left-0 -z-0 isolate",
- defaultClassNames.range_end,
- ),
- today: cn(
- "bg-muted text-foreground rounded-(--cell-radius) data-[selected=true]:rounded-none",
- defaultClassNames.today,
- ),
- outside: cn(
- "text-muted-foreground aria-selected:text-muted-foreground",
- defaultClassNames.outside,
- ),
- disabled: cn(
- "text-muted-foreground opacity-50",
- defaultClassNames.disabled,
- ),
- hidden: cn("invisible", defaultClassNames.hidden),
- ...classNames,
- }}
- components={{
- Root: ({ className, rootRef, ...props }) => {
- return (
-
- );
- },
- Chevron: ({ className, orientation, ...props }) => {
- if (orientation === "left") {
- return (
-
- );
- }
-
- if (orientation === "right") {
- return (
-
- );
- }
-
- return (
-
- );
- },
- DayButton: CalendarDayButton,
- WeekNumber: ({ children, ...props }) => {
- return (
-
-
- {children}
-
-
- );
- },
- ...components,
- }}
- {...props}
- />
- );
-}
-
-function CalendarDayButton({
- className,
- day,
- modifiers,
- ...props
-}: React.ComponentProps) {
- const defaultClassNames = getDefaultClassNames();
-
- const ref = React.useRef(null);
- React.useEffect(() => {
- if (modifiers.focused) ref.current?.focus();
- }, [modifiers.focused]);
-
- return (
- span]:text-xs [&>span]:opacity-70",
- defaultClassNames.day,
- className,
- )}
- {...props}
- />
- );
-}
-
-export { Calendar, CalendarDayButton };
+import * as React from "react";
+import {
+ type DayButton,
+ DayPicker,
+ getDefaultClassNames,
+} from "react-day-picker";
+
+import { Button, buttonVariants } from "@/components/ui/button";
+import { cn } from "@/lib/utils";
+import {
+ ArrowDownIcon,
+ ArrowLeftIcon,
+ ArrowRightIcon,
+} from "@hugeicons/core-free-icons";
+import { HugeiconsIcon } from "@hugeicons/react";
+
+function Calendar({
+ className,
+ classNames,
+ showOutsideDays = true,
+ captionLayout = "label",
+ buttonVariant = "ghost",
+ formatters,
+ components,
+ ...props
+}: React.ComponentProps & {
+ buttonVariant?: React.ComponentProps["variant"];
+}) {
+ const defaultClassNames = getDefaultClassNames();
+
+ return (
+ svg]:rotate-180`,
+ String.raw`rtl:**:[.rdp-button\_previous>svg]:rotate-180`,
+ className,
+ )}
+ captionLayout={captionLayout}
+ formatters={{
+ formatMonthDropdown: (date) =>
+ date.toLocaleString("default", { month: "short" }),
+ ...formatters,
+ }}
+ classNames={{
+ root: cn("w-fit", defaultClassNames.root),
+ months: cn(
+ "flex gap-4 flex-col md:flex-row relative",
+ defaultClassNames.months,
+ ),
+ month: cn("flex flex-col w-full gap-4", defaultClassNames.month),
+ nav: cn(
+ "flex items-center gap-1 w-full absolute top-0 inset-x-0 justify-between",
+ defaultClassNames.nav,
+ ),
+ button_previous: cn(
+ buttonVariants({ variant: buttonVariant }),
+ "size-(--cell-size) aria-disabled:opacity-50 p-0 select-none",
+ defaultClassNames.button_previous,
+ ),
+ button_next: cn(
+ buttonVariants({ variant: buttonVariant }),
+ "size-(--cell-size) aria-disabled:opacity-50 p-0 select-none",
+ defaultClassNames.button_next,
+ ),
+ month_caption: cn(
+ "flex items-center justify-center h-(--cell-size) w-full px-(--cell-size)",
+ defaultClassNames.month_caption,
+ ),
+ dropdowns: cn(
+ "w-full flex items-center text-sm font-medium justify-center h-(--cell-size) gap-1.5",
+ defaultClassNames.dropdowns,
+ ),
+ dropdown_root: cn(
+ "relative cn-calendar-dropdown-root rounded-(--cell-radius)",
+ defaultClassNames.dropdown_root,
+ ),
+ dropdown: cn(
+ "absolute bg-popover inset-0 opacity-0",
+ defaultClassNames.dropdown,
+ ),
+ caption_label: cn(
+ "select-none font-medium",
+ captionLayout === "label"
+ ? "text-sm"
+ : "cn-calendar-caption-label rounded-(--cell-radius) flex items-center gap-1 text-sm [&>svg]:text-muted-foreground [&>svg]:size-3.5",
+ defaultClassNames.caption_label,
+ ),
+ table: "w-full border-collapse",
+ weekdays: cn("flex", defaultClassNames.weekdays),
+ weekday: cn(
+ "text-muted-foreground rounded-(--cell-radius) flex-1 font-normal text-[0.8rem] select-none",
+ defaultClassNames.weekday,
+ ),
+ week: cn("flex w-full mt-2", defaultClassNames.week),
+ week_number_header: cn(
+ "select-none w-(--cell-size)",
+ defaultClassNames.week_number_header,
+ ),
+ week_number: cn(
+ "text-[0.8rem] select-none text-muted-foreground",
+ defaultClassNames.week_number,
+ ),
+ day: cn(
+ "relative w-full rounded-(--cell-radius) h-full p-0 text-center [&:last-child[data-selected=true]_button]:rounded-r-(--cell-radius) group/day aspect-square select-none",
+ props.showWeekNumber
+ ? "[&:nth-child(2)[data-selected=true]_button]:rounded-l-(--cell-radius)"
+ : "[&:first-child[data-selected=true]_button]:rounded-l-(--cell-radius)",
+ defaultClassNames.day,
+ ),
+ range_start: cn(
+ "rounded-l-(--cell-radius) bg-muted relative after:bg-muted after:absolute after:inset-y-0 after:w-4 after:right-0 -z-0 isolate",
+ defaultClassNames.range_start,
+ ),
+ range_middle: cn("rounded-none", defaultClassNames.range_middle),
+ range_end: cn(
+ "rounded-r-(--cell-radius) bg-muted relative after:bg-muted-200 after:absolute after:inset-y-0 after:w-4 after:left-0 -z-0 isolate",
+ defaultClassNames.range_end,
+ ),
+ today: cn(
+ "bg-muted text-foreground rounded-(--cell-radius) data-[selected=true]:rounded-none",
+ defaultClassNames.today,
+ ),
+ outside: cn(
+ "text-muted-foreground aria-selected:text-muted-foreground",
+ defaultClassNames.outside,
+ ),
+ disabled: cn(
+ "text-muted-foreground opacity-50",
+ defaultClassNames.disabled,
+ ),
+ hidden: cn("invisible", defaultClassNames.hidden),
+ ...classNames,
+ }}
+ components={{
+ Root: ({ className, rootRef, ...props }) => {
+ return (
+
+ );
+ },
+ Chevron: ({ className, orientation, ...props }) => {
+ if (orientation === "left") {
+ return (
+
+ );
+ }
+
+ if (orientation === "right") {
+ return (
+
+ );
+ }
+
+ return (
+
+ );
+ },
+ DayButton: CalendarDayButton,
+ WeekNumber: ({ children, ...props }) => {
+ return (
+
+
+ {children}
+
+
+ );
+ },
+ ...components,
+ }}
+ {...props}
+ />
+ );
+}
+
+function CalendarDayButton({
+ className,
+ day,
+ modifiers,
+ ...props
+}: React.ComponentProps) {
+ const defaultClassNames = getDefaultClassNames();
+
+ const ref = React.useRef(null);
+ React.useEffect(() => {
+ if (modifiers.focused) ref.current?.focus();
+ }, [modifiers.focused]);
+
+ return (
+ span]:text-xs [&>span]:opacity-70",
+ defaultClassNames.day,
+ className,
+ )}
+ {...props}
+ />
+ );
+}
+
+export { Calendar, CalendarDayButton };
diff --git a/studio/frontend/src/components/ui/card.tsx b/studio/frontend/src/components/ui/card.tsx
index f64a1cbdac..21558618c2 100644
--- a/studio/frontend/src/components/ui/card.tsx
+++ b/studio/frontend/src/components/ui/card.tsx
@@ -1,103 +1,103 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
-import type * as React from "react";
-
-import { cn } from "@/lib/utils";
-
-function Card({
- className,
- size = "default",
- ...props
-}: React.ComponentProps<"div"> & { size?: "default" | "sm" }) {
- return (
- img:first-child]:pt-0 data-[size=sm]:gap-4 data-[size=sm]:py-4 *:[img:first-child]:rounded-t-xl *:[img:last-child]:rounded-b-xl group/card flex flex-col",
- className,
- )}
- {...props}
- />
- );
-}
-
-function CardHeader({ className, ...props }: React.ComponentProps<"div">) {
- return (
-
- );
-}
-
-function CardTitle({ className, ...props }: React.ComponentProps<"div">) {
- return (
-
- );
-}
-
-function CardDescription({ className, ...props }: React.ComponentProps<"div">) {
- return (
-
- );
-}
-
-function CardAction({ className, ...props }: React.ComponentProps<"div">) {
- return (
-
- );
-}
-
-function CardContent({ className, ...props }: React.ComponentProps<"div">) {
- return (
-
- );
-}
-
-function CardFooter({ className, ...props }: React.ComponentProps<"div">) {
- return (
-
- );
-}
-
-export {
- Card,
- CardHeader,
- CardFooter,
- CardTitle,
- CardAction,
- CardDescription,
- CardContent,
-};
+import type * as React from "react";
+
+import { cn } from "@/lib/utils";
+
+function Card({
+ className,
+ size = "default",
+ ...props
+}: React.ComponentProps<"div"> & { size?: "default" | "sm" }) {
+ return (
+
img:first-child]:pt-0 data-[size=sm]:gap-4 data-[size=sm]:py-4 *:[img:first-child]:rounded-t-xl *:[img:last-child]:rounded-b-xl group/card flex flex-col",
+ className,
+ )}
+ {...props}
+ />
+ );
+}
+
+function CardHeader({ className, ...props }: React.ComponentProps<"div">) {
+ return (
+
+ );
+}
+
+function CardTitle({ className, ...props }: React.ComponentProps<"div">) {
+ return (
+
+ );
+}
+
+function CardDescription({ className, ...props }: React.ComponentProps<"div">) {
+ return (
+
+ );
+}
+
+function CardAction({ className, ...props }: React.ComponentProps<"div">) {
+ return (
+
+ );
+}
+
+function CardContent({ className, ...props }: React.ComponentProps<"div">) {
+ return (
+
+ );
+}
+
+function CardFooter({ className, ...props }: React.ComponentProps<"div">) {
+ return (
+
+ );
+}
+
+export {
+ Card,
+ CardHeader,
+ CardFooter,
+ CardTitle,
+ CardAction,
+ CardDescription,
+ CardContent,
+};
diff --git a/studio/frontend/src/components/ui/chart.tsx b/studio/frontend/src/components/ui/chart.tsx
index 127d0c52d7..32da410bb5 100644
--- a/studio/frontend/src/components/ui/chart.tsx
+++ b/studio/frontend/src/components/ui/chart.tsx
@@ -1,257 +1,257 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
-import * as React from "react";
-import * as RechartsPrimitive from "recharts";
-
-import { cn } from "@/lib/utils";
-
-// Format: { THEME_NAME: CSS_SELECTOR }
-const THEMES = { light: "", dark: ".dark" } as const;
-
-export type ChartConfig = {
- [k in string]: {
- label?: React.ReactNode;
- icon?: React.ComponentType;
- } & (
- | { color?: string; theme?: never }
- | { color?: never; theme: Record
}
- );
-};
-
-type ChartContextProps = {
- config: ChartConfig;
-};
-
-const ChartContext = React.createContext(null);
-
-function useChart() {
- const context = React.useContext(ChartContext);
-
- if (!context) {
- throw new Error("useChart must be used within a ");
- }
-
- return context;
-}
-
-function ChartContainer({
- id,
- className,
- children,
- config,
- ...props
-}: React.ComponentProps<"div"> & {
- config: ChartConfig;
- children: React.ComponentProps<
- typeof RechartsPrimitive.ResponsiveContainer
- >["children"];
-}) {
- const uniqueId = React.useId();
- const chartId = `chart-${id || uniqueId.replace(/:/g, "")}`;
- const containerRef = React.useRef(null);
- const [containerSize, setContainerSize] = React.useState<{
- width: number;
- height: number;
- } | null>(null);
-
- React.useEffect(() => {
- const element = containerRef.current;
- if (!element) return;
-
- const updateSizeState = () => {
- const { width, height } = element.getBoundingClientRect();
- const nextSize =
- width > 0 && height > 0
- ? {
- width: Math.round(width),
- height: Math.round(height),
- }
- : null;
-
- setContainerSize((currentSize) => {
- if (!nextSize) {
- // Keep the last valid size once mounted to avoid unmount/remount thrash.
- return currentSize;
- }
- if (
- currentSize &&
- currentSize.width === nextSize.width &&
- currentSize.height === nextSize.height
- ) {
- return currentSize;
- }
- return nextSize;
- });
- };
-
- updateSizeState();
-
- if (typeof ResizeObserver === "undefined") {
- const recheckSize = () => {
- if (document.visibilityState === "visible") {
- updateSizeState();
- }
- };
-
- window.addEventListener("resize", recheckSize);
- window.addEventListener("orientationchange", recheckSize);
- document.addEventListener("visibilitychange", recheckSize);
-
- return () => {
- window.removeEventListener("resize", recheckSize);
- window.removeEventListener("orientationchange", recheckSize);
- document.removeEventListener("visibilitychange", recheckSize);
- };
- }
-
- const observer = new ResizeObserver(() => {
- updateSizeState();
- });
- observer.observe(element);
-
- return () => observer.disconnect();
- }, []);
-
- return (
-
-
-
- {containerSize ? (
-
- {children}
-
- ) : null}
-
-
- );
-}
-
-const ChartStyle = ({ id, config }: { id: string; config: ChartConfig }) => {
- const colorConfig = Object.entries(config).filter(
- ([, config]) => config.theme || config.color,
- );
-
- if (!colorConfig.length) {
- return null;
- }
-
- return (
-