ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Speed up Studio startup path (#6899)

Browse source 
* Speed up Studio startup path

* Studio: recheck managed binary executability on preflight cache hit and ignore stale unauthenticated platform fetches

Preflight: a matching capability cache fingerprint no longer skips the
runnability check when the managed binary's executable bit was cleared
(size and mtime unchanged, since chmod bumps ctime not mtime). The cache
fast path now confirms the binary is still executable, otherwise it falls
back to the CLI help probe so preflight reports Stale and can repair,
instead of returning Ready and failing later at backend start. Adds a
regression test.

Frontend: now that first render is no longer gated on fetchDeviceType,
the initial unauthenticated health call can resolve after an
authenticated platform fetch. Guard the store so a late unauthenticated
or failed non-forced response cannot overwrite an already authoritative
device type, tunnel URL, or secure flag. Forced refreshes and the first
unauthenticated load are unaffected.

* Studio: use access(X_OK) for the preflight cache executability guard

A mode bitmask treats any execute bit as launchable, but the executable
bits can be set only for another owner or group, or be denied by an ACL,
so the current user could still hit PermissionDenied at launch and the
cached fast path would wrongly return Ready. access(X_OK) checks real
executability for the calling user, so an ownership or permission change
correctly falls back to the CLI help probe and the Stale repair path.

* Studio: ignore any stale non-forced platform fetch once authoritative

Extend the platform store guard so a non-forced health response never
overwrites an already authoritative result, not only unauthenticated
ones. With a saved token the post-render non-forced request can be
authenticated but older than a later forced refresh that already picked
up the tunnel URL and secure flag; if that earlier request resolves last
it would null those fields. Now any non-forced response is dropped once
the store holds a server-reported platform. Forced refreshes and the
first authoritative write are unaffected.

* Studio: run the managed CLI help probe before trusting the preflight cache

Restore running the managed CLI help probe before returning Ready from
the desktop capability cache, so a managed install whose venv interpreter
or a runtime dependency is broken (while path, size, mtime, and markers
are unchanged) is reported Stale for repair rather than proceeding to a
backend start that cannot spawn. The capability cache still skips the
heavier desktop-capabilities probe on a hit, so a warm cache runs one
probe instead of two. Removes the executable-access shortcut, which the
help probe now subsumes.

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>
This commit is contained in:
Wasim Yousef Said 2026-07-08 03:08:07 +02:00 • committed by GitHub
commit 49d1fb3863
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
10 changed files with 308 additions and 44 deletions
Download patch file Download diff file Expand all files Collapse all files

8
studio/backend/core/inference/orchestrator.py
View file

@ -108,13 +108,11 @@ class InferenceOrchestrator:
@property
def default_models(self) -> list[str]:
# Wait up to 5s for background HF fetch
self._top_models_ready.wait(timeout = 5)
top_gguf = self._top_gguf_cache or []
top_hub = self._top_hub_cache or []
# Curated static defaults first, then HF download-ranked to backfill.
# Send extras so the frontend keeps 4 per category after removing
# downloaded ones.
# Never wait for the remote Hugging Face ranking during startup. Chat's
# first /api/models/list needs curated defaults immediately; the
# background fetch backfills extra choices on later calls.
result: list[str] = []
seen: set[str] = set()
for m in self._static_models + top_gguf + top_hub:

42
studio/backend/tests/test_inference_default_models_non_blocking.py Normal file
View file

@ -0,0 +1,42 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
"""Default Chat model metadata must not block on remote Hugging Face discovery."""
from __future__ import annotations
import sys
import time
from pathlib import Path
_BACKEND = Path(__file__).resolve().parent.parent
if str(_BACKEND) not in sys.path:
sys.path.insert(0, str(_BACKEND))
from core.inference.orchestrator import InferenceOrchestrator # noqa: E402
def test_default_models_returns_static_defaults_before_top_fetch(monkeypatch):
sleep_seconds = 2.0
def _slow_fetch(self: InferenceOrchestrator) -> None:
time.sleep(sleep_seconds)
self._top_gguf_cache = ["unsloth/slow-GGUF"]
self._top_models_ready.set()
monkeypatch.setattr(InferenceOrchestrator, "_fetch_top_models", _slow_fetch)
orchestrator = InferenceOrchestrator()
started = time.monotonic()
defaults = orchestrator.default_models
elapsed = time.monotonic() - started
assert elapsed < 0.5, f"default_models blocked for {elapsed:.2f}s"
assert defaults == orchestrator._static_models
assert "unsloth/slow-GGUF" not in defaults
deadline = time.monotonic() + sleep_seconds + 5
while not orchestrator._top_models_ready.is_set() and time.monotonic() < deadline:
time.sleep(0.05)
assert "unsloth/slow-GGUF" in orchestrator.default_models

45
studio/frontend/src/components/app-sidebar.tsx
View file

@ -81,11 +81,6 @@ import {
TestTube01Icon,
ZapIcon,
} from "@hugeicons/core-free-icons";
import {
exportConversationRawJsonl,
exportConversationCsv,
exportConversationShareGPT,
} from "@/features/chat/prompt-storage/prompt-storage-dialog";
import { listStoredChatThreads } from "@/features/chat/utils/chat-history-storage";
import {
Tooltip,
@ -174,6 +169,36 @@ const TestTubeOutlineIcon = TestTube01Icon.slice(
3,
) as typeof TestTube01Icon;
type ConversationExportFormat = "raw-jsonl" | "csv" | "sharegpt-jsonl";
const CHAT_EXPORT_OPTIONS: Array<{
label: string;
format: ConversationExportFormat;
}> = [
{ label: "Raw JSONL", format: "raw-jsonl" },
{ label: "CSV", format: "csv" },
{ label: "ShareGPT JSONL", format: "sharegpt-jsonl" },
];
async function exportConversationByFormat(
threadId: string,
format: ConversationExportFormat,
): Promise<void> {
const exports = await import(
"@/features/chat/prompt-storage/prompt-storage-dialog"
);
switch (format) {
case "raw-jsonl":
return exports.exportConversationRawJsonl(threadId);
case "csv":
return exports.exportConversationCsv(threadId);
case "sharegpt-jsonl":
return exports.exportConversationShareGPT(threadId);
}
}
function runStatusDotClass(status: TrainingRunSummary["status"]): string {
switch (status) {
case "running":
@ -899,11 +924,7 @@ export function AppSidebar() {
<span>Export</span>
</DropdownMenuSubTrigger>
<DropdownMenuSubContent sideOffset={8} alignOffset={-4} className="unsloth-plus-menu w-52">
{[
{ label: "Raw JSONL", fn: exportConversationRawJsonl },
{ label: "CSV", fn: exportConversationCsv },
{ label: "ShareGPT JSONL", fn: exportConversationShareGPT },
].map(({ label, fn }) => (
{CHAT_EXPORT_OPTIONS.map(({ label, format }) => (
<DropdownMenuItem
key={label}
onSelect={async () => {
@ -911,7 +932,9 @@ export function AppSidebar() {
const ids = item.type === "single"
? [item.id]
: (await listStoredChatThreads({ pairId: item.id })).map((t) => t.id);
await Promise.all(ids.map((id) => fn(id)));
await Promise.all(
ids.map((id) => exportConversationByFormat(id, format)),
);
} catch {
toast.error("Export failed.");
}

26
studio/frontend/src/config/env.ts
View file

@ -54,6 +54,17 @@ export const usePlatformStore = create<PlatformState>()((_, get) => ({
isChatOnly: () => get().chatOnly,
}));
// Once an authoritative (server-reported) platform has been fetched, a
// non-forced response must not overwrite it. The post-render fetchDeviceType()
// in main.tsx runs before auth is ready and can resolve after the authed
// root-route/provider fetches; such a late write would reset deviceType,
// cloudflareUrl/serverUrl/secure, and fetched, whether it is a browser fallback
// (unauthenticated) or an earlier authenticated request that landed after a
// later forced refresh. Forced refreshes are explicit re-reads, so they still write.
function shouldKeepAuthoritativePlatform(force?: boolean): boolean {
return !force && usePlatformStore.getState().fetched;
}
// `force` re-reads /api/health even if cached, to pick up a late-arriving tunnel URL.
export async function fetchDeviceType(options?: {
force?: boolean;
@ -81,6 +92,15 @@ export async function fetchDeviceType(options?: {
server_url?: string | null;
secure?: boolean;
};
// Once the store holds an authoritative (server-reported) platform, a
// non-forced response must not overwrite it. It may be an unauthenticated
// fallback, or an earlier authenticated request that resolved after a
// later forced refresh already picked up device_type and the tunnel
// fields; writing either would reset device type or null the tunnel
// fields. Forced refreshes are explicit re-reads, so they still write.
if (shouldKeepAuthoritativePlatform(options?.force)) {
return usePlatformStore.getState().deviceType;
}
const deviceType = data.device_type ?? detectLocalPlatform();
const chatOnly = data.chat_only ?? false;
const chatOnlyReason = data.chat_only_reason ?? null;
@ -101,7 +121,11 @@ export async function fetchDeviceType(options?: {
} catch {
// Backend not ready: use client-side detection so chat-only guard works
// on initial load (important for macOS). Keep fetched=false so a later
// call retries against the backend.
// call retries against the backend. But a late non-forced failure must not
// wipe an authoritative platform that already resolved.
if (shouldKeepAuthoritativePlatform(options?.force)) {
return usePlatformStore.getState().deviceType;
}
const deviceType = detectLocalPlatform();
const chatOnly = deviceType === "mac";
usePlatformStore.setState({ deviceType, chatOnly, fetched: false });

43
studio/frontend/src/features/chat/chat-page.tsx
View file

@ -35,7 +35,6 @@ import {
useNativeModelDrop,
useNativePathLeasesSupported,
} from "@/features/native-intents";
import { ProjectSourcesPanel } from "@/features/rag/components/project-sources-panel";
import { GuidedTour, useGuidedTourController } from "@/features/tour";
import { isTauri } from "@/lib/api-base";
import { toast } from "@/lib/toast";
@ -51,7 +50,9 @@ import { Tooltip as TooltipPrimitive } from "radix-ui";
import {
type CSSProperties,
type ReactElement,
lazy,
memo,
Suspense,
useCallback,
useEffect,
useMemo,
@ -134,6 +135,13 @@ import {
} from "./utils/chat-history-storage";
import { isAssistantLocalThreadId } from "./utils/thread-ids";
const ProjectSourcesPanel = lazy(() =>
import("@/features/rag/components/project-sources-panel").then((module) => ({
default: module.ProjectSourcesPanel,
})),
);
type LoraCandidate = {
id: string;
baseModel: string;
@ -1018,7 +1026,15 @@ function ProjectLanding({
</div>
{projectTab === "sources" ? (
<ProjectSourcesPanel projectId={projectId} />
<Suspense
fallback={
<div className="mt-8 rounded-[26px] bg-muted/30 px-6 py-10 text-center text-sm text-muted-foreground">
Loading sources…
</div>
}
>
<ProjectSourcesPanel projectId={projectId} />
</Suspense>
) : (
<div className="mt-8 flex flex-col gap-1">
{items.map((item) => {
@ -2246,12 +2262,29 @@ export function ChatPage({
return [...fromLoras, ...localModels];
}, [lorasFromStore, localModels]);
useEffect(() => {
if (getTrainingCompareHandoff()) return;
void refresh();
const inventoryRefreshStartedRef = useRef(false);
const refreshDeferredModelInventories = useCallback(() => {
inventoryRefreshStartedRef.current = true;
void refresh({ includeLoras: true });
refreshLocalModels();
}, [refresh, refreshLocalModels]);
useEffect(() => {
if (getTrainingCompareHandoff()) return;
void refresh({ includeLoras: false });
const timeoutId = window.setTimeout(() => {
if (!inventoryRefreshStartedRef.current) {
refreshDeferredModelInventories();
}
}, 1200);
return () => window.clearTimeout(timeoutId);
}, [refresh, refreshDeferredModelInventories]);
useEffect(() => {
if (!active || !modelSelectorOpen) return;
refreshDeferredModelInventories();
}, [active, modelSelectorOpen, refreshDeferredModelInventories]);
useEffect(() => {
// ChatPage no longer remounts on navigation, so re-check the handoff whenever
// we return to /chat (e.g. from the training progress "compare in chat" action).

12
studio/frontend/src/features/chat/hooks/use-chat-model-runtime.ts
View file

@ -312,14 +312,18 @@ export function useChatModelRuntime() {
[],
);
const refresh = useCallback(async (options?: { signal?: AbortSignal }) => {
const refresh = useCallback(async (options?: {
signal?: AbortSignal;
includeLoras?: boolean;
}) => {
const signal = options?.signal;
const includeLoras = options?.includeLoras ?? true;
setModelsError(null);
try {
const [listRes, statusRes, lorasRes] = await Promise.all([
listModels(),
getInferenceStatus(),
listLoras(),
includeLoras ? listLoras() : Promise.resolve(null),
]);
// Cancellation can land while the requests above are in flight. Bail
@ -327,7 +331,9 @@ export function useChatModelRuntime() {
if (signal?.aborted) return;
setModels(listRes.models.map(toChatModelSummary));
setLoras(lorasRes.loras.map(toLoraSummary));
if (lorasRes) {
setLoras(lorasRes.loras.map(toLoraSummary));
}
const selectedCheckpoint = useChatRuntimeStore.getState().params.checkpoint;
const isExternalSelectionActive = isExternalModelId(selectedCheckpoint);

12
studio/frontend/src/features/chat/runtime-provider.tsx
View file

@ -22,7 +22,6 @@ import {
unstable_useRemoteThreadListRuntime as useRemoteThreadListRuntime,
} from "@assistant-ui/react";
import { createAssistantStream } from "assistant-stream";
import mammoth from "mammoth";
import {
type ReactElement,
type ReactNode,
@ -33,7 +32,6 @@ import {
useMemo,
useRef,
} from "react";
import { extractText, getDocumentProxy } from "unpdf";
import { toast } from "sonner";
import { StudioWebSpeechDictationAdapter } from "./adapters/studio-web-speech-dictation-adapter";
import {
@ -181,7 +179,10 @@ class PDFAttachmentAdapter implements AttachmentAdapter {
}
async send(attachment: PendingAttachment): Promise<CompleteAttachment> {
const buffer = new Uint8Array(await attachment.file.arrayBuffer());
const [{ extractText, getDocumentProxy }, buffer] = await Promise.all([
import("unpdf"),
attachment.file.arrayBuffer().then((bytes) => new Uint8Array(bytes)),
]);
const pdf = await getDocumentProxy(buffer);
const { text } = await extractText(pdf, { mergePages: true });
return {
@ -298,7 +299,10 @@ class DocxAttachmentAdapter implements AttachmentAdapter {
}
async send(attachment: PendingAttachment): Promise<CompleteAttachment> {
const arrayBuffer = await attachment.file.arrayBuffer();
const [{ default: mammoth }, arrayBuffer] = await Promise.all([
import("mammoth"),
attachment.file.arrayBuffer(),
]);
const { value } = await mammoth.extractRawText({ arrayBuffer });
return {
id: attachment.id,

16
studio/frontend/src/main.tsx
View file

@ -5,8 +5,8 @@ import { StrictMode } from "react";
import { createRoot } from "react-dom/client";
import "./index.css";
import { fetchDeviceType } from "./config/env";
import { App } from "./app/app";
import { fetchDeviceType } from "./config/env";
import { initializeLocale } from "./i18n";
const globalCrypto = globalThis.crypto as Crypto | undefined;
@ -36,10 +36,10 @@ if (!rootElement) {
initializeLocale();
fetchDeviceType().then(() => {
createRoot(rootElement).render(
<StrictMode>
<App />
</StrictMode>,
);
});
createRoot(rootElement).render(
<StrictMode>
<App />
</StrictMode>,
);
fetchDeviceType().catch(() => undefined);

132
studio/src-tauri/src/preflight.rs
View file

@ -498,19 +498,68 @@ mod tests {
}
#[cfg(unix)]
fn remove_managed_capability_cache() {
let _ = std::fs::remove_file(
dirs::home_dir()
static MANAGED_CAPABILITY_CACHE_TEST_LOCK: std::sync::LazyLock<tokio::sync::Mutex<()>> =
std::sync::LazyLock::new(|| tokio::sync::Mutex::new(()));
#[cfg(unix)]
struct ManagedCapabilityCacheHome {
path: PathBuf,
previous: Option<std::ffi::OsString>,
}
#[cfg(unix)]
impl ManagedCapabilityCacheHome {
fn new(test_name: &str) -> Self {
use std::time::{SystemTime, UNIX_EPOCH};
let nanos = SystemTime::now()
.duration_since(UNIX_EPOCH)
.unwrap()
.join(".unsloth")
.join("studio")
.join("desktop_capability_cache.json"),
);
.as_nanos();
let path = std::env::temp_dir().join(format!(
"unsloth-preflight-cache-{test_name}-{}-{nanos}",
std::process::id()
));
std::fs::create_dir_all(&path).unwrap();
let previous = std::env::var_os("UNSLOTH_TEST_DESKTOP_CAPABILITY_CACHE_HOME");
std::env::set_var("UNSLOTH_TEST_DESKTOP_CAPABILITY_CACHE_HOME", &path);
Self { path, previous }
}
}
#[cfg(unix)]
impl Drop for ManagedCapabilityCacheHome {
fn drop(&mut self) {
if let Some(previous) = &self.previous {
std::env::set_var("UNSLOTH_TEST_DESKTOP_CAPABILITY_CACHE_HOME", previous);
} else {
std::env::remove_var("UNSLOTH_TEST_DESKTOP_CAPABILITY_CACHE_HOME");
}
let _ = std::fs::remove_dir_all(&self.path);
}
}
#[cfg(unix)]
fn managed_capability_cache_path_for_test() -> PathBuf {
std::env::var_os("UNSLOTH_TEST_DESKTOP_CAPABILITY_CACHE_HOME")
.map(PathBuf::from)
.or_else(dirs::home_dir)
.unwrap()
.join(".unsloth")
.join("studio")
.join("desktop_capability_cache.json")
}
#[cfg(unix)]
fn remove_managed_capability_cache() {
let _ = std::fs::remove_file(managed_capability_cache_path_for_test());
}
#[cfg(unix)]
#[tokio::test]
async fn managed_cli_capability_probe_classifies_core_cases() {
let _cache_guard = MANAGED_CAPABILITY_CACHE_TEST_LOCK.lock().await;
let _cache_home = ManagedCapabilityCacheHome::new("core-cases");
remove_managed_capability_cache();
for (name, script, stale_reason) in [
@ -567,6 +616,75 @@ exit 1
}
}
#[cfg(unix)]
#[tokio::test]
async fn managed_cli_capability_help_probe_runs_before_cache() {
use std::fs;
let _cache_guard = MANAGED_CAPABILITY_CACHE_TEST_LOCK.lock().await;
let _cache_home = ManagedCapabilityCacheHome::new("cache-hit");
remove_managed_capability_cache();
// `-h` always succeeds unless `modeh` exists; the desktop-capabilities
// probe always succeeds unless `modecap` exists. Toggling those lets us
// prove the ordering: -h runs on every probe (even a cache hit), while
// the heavier capability probe is skipped once the cache is warm.
let fake = fake_cli(
"cap-cache-hit",
r#"#!/bin/sh
log="$0.calls"
modeh="$0.modeh"
modecap="$0.modecap"
printf '%s\n' "$*" >> "$log"
if [ "$1" = "-h" ]; then
if [ -f "$modeh" ]; then exit 42; fi
exit 0
fi
if [ "$1" = "studio" ] && [ "$2" = "desktop-capabilities" ] && [ "$3" = "--json" ]; then
if [ -f "$modecap" ]; then exit 42; fi
printf '{"desktop_protocol_version":1,"desktop_manageability_version":1,"supports_api_only":true,"supports_provision_desktop_auth":true,"supports_desktop_backend_ownership":true,"version":"2026.5.3"}'
exit 0
fi
exit 1
"#,
);
let bin = fake.bin.clone();
let calls = bin.with_extension("calls");
let modeh = bin.with_extension("modeh");
let modecap = bin.with_extension("modecap");
// Cold probe: runs -h and the capability probe, then caches the result.
assert!(matches!(
probe_managed_bin(bin.clone()).await,
ManagedProbe::Ready { .. }
));
let first_calls = fs::read_to_string(&calls).unwrap();
assert!(first_calls.contains("-h"));
assert!(first_calls.contains("studio desktop-capabilities --json"));
// Cache hit: -h still runs, but the capability probe is skipped (breaking
// it via `modecap` proves it is not invoked).
fs::write(&modecap, "broken").unwrap();
fs::write(&calls, "").unwrap();
assert!(matches!(
probe_managed_bin(bin.clone()).await,
ManagedProbe::Ready { .. }
));
assert_eq!(fs::read_to_string(&calls).unwrap(), "-h\n");
// A non-launchable CLI is caught by the -h probe even with a warm cache:
// preflight reports Stale (for repair) and never trusts the cache.
fs::write(&modeh, "broken").unwrap();
fs::write(&calls, "").unwrap();
assert!(matches!(
probe_managed_bin(bin).await,
ManagedProbe::Stale { .. }
));
assert_eq!(fs::read_to_string(&calls).unwrap(), "-h\n");
remove_managed_capability_cache();
}
const EXPECTED_ROOT_ID: &str =
"aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa";
const OTHER_ROOT_ID: &str = "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb";

16
studio/src-tauri/src/preflight/managed.rs
View file

@ -188,6 +188,16 @@ fn managed_bin_fingerprint(bin: &Path) -> Option<ManagedBinFingerprint> {
}
fn capability_cache_path() -> Option<PathBuf> {
#[cfg(test)]
if let Some(home) = std::env::var_os("UNSLOTH_TEST_DESKTOP_CAPABILITY_CACHE_HOME") {
return Some(
PathBuf::from(home)
.join(".unsloth")
.join("studio")
.join("desktop_capability_cache.json"),
);
}
dirs::home_dir().map(|home| {
home.join(".unsloth")
.join("studio")
@ -400,6 +410,12 @@ fn desktop_capability_ready(capability: &DesktopCapability) -> bool {
pub(super) async fn probe_managed_bin(bin: PathBuf) -> ManagedProbe {
let started = Instant::now();
// Always verify the managed CLI actually launches before trusting the cache.
// A matching capability fingerprint does not prove the binary can still run:
// its venv interpreter or a runtime dependency can be broken while the
// path/size/mtime/markers are unchanged, so the -h probe runs first and a
// non-launchable install is reported Stale for repair. The capability cache
// below still skips the heavier desktop-capabilities probe on a hit.
if !run_cli_probe(&bin, &["-h"]).await {
info!(
"Managed preflight: cli unusable for {:?} in {}ms",