ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

feat: custom scan folders for GGUF model discovery (#4723)

Browse source 
* feat: add scan_folders table and CRUD functions to studio_db

* feat: add scan folders API endpoints and integrate into model scan

* feat: add scan folders API client and update source types

* feat: add custom source to model filters and selector

* feat: add Model Folders section to chat settings sidebar

* style: fix biome formatting in ModelFoldersSection

* fix: address review findings for custom scan folders

empty string bypass, concurrent delete crash guard,
Windows case normalization, response_model on endpoints,
logging, deduplicated filter/map, module level cache for
custom folder models, consistent source labels, handleRemove
error surfacing, per folder scan cap

* fix: show custom folders section regardless of chatOnly mode

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* refactor: extract shared refreshLocalModelsList in pickers

* Harden custom scan folder validation and scanning

- Validate path exists, is a directory, and is readable before persisting
- Apply per-folder model cap during traversal instead of after (avoids
  scanning millions of inodes in large directories)
- Wrap per-folder scan in try/except so one unreadable folder does not
  break the entire /api/models/local endpoint for all callers
- Normalize case on Windows before storing so C:\Models and c:\models
  dedup correctly
- Extend macOS denylist to cover /private/etc and /private/tmp (realpath
  resolves /etc -> /private/etc, bypassing the original denylist)
- Add /boot and /run to Linux denylist

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Improve scan robustness and preserve Windows path casing

- Preserve original Windows path casing in DB instead of lowercasing
  (normcase used only for dedup comparison, not storage)
- Catch PermissionError per child directory so one unreadable subdirectory
  does not skip the entire custom folder scan
- Wrap list_scan_folders() DB call in try/except so a DB issue does not
  break the entire /api/models/local endpoint

* fix: scan custom folders for both flat and HF cache layouts

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix Windows case-insensitive path dedup with COLLATE NOCASE

Use COLLATE NOCASE on the scan_folders.path column so that the UNIQUE
constraint correctly deduplicates C:\Models and c:\models on Windows
without lowercasing the stored path. Also use COLLATE NOCASE in the
pre-insert lookup query on Windows to catch existing rows with
different casing.

* Restore early-exit limit in _scan_models_dir for custom folders

Keep the limit parameter so _scan_models_dir stops iterating once
enough models are found, avoiding unbounded traversal of large
directories. The post-traversal slice is still applied after combining
with _scan_hf_cache results.

* feat: scan custom folders with LM Studio layout too

* Fix custom folder models being hidden by dedup

Custom folder entries were appended after HF cache and models_dir
entries.  The dedup loop kept the first occurrence of each model id,
so custom models with the same id as an existing HF cache entry were
silently dropped -- they never appeared in the "Custom Folders" UI
section.

Use a separate dedup key for custom-source entries so they always
survive deduplication.  This way a model can appear under both
"Downloaded" (from HF cache) and "Custom Folders" (from the
user-registered directory) at the same time.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Harden LM Studio scan and fix COLLATE NOCASE on Linux

- Add per-child and per-publisher OSError handling in _scan_lmstudio_dir
  so one unreadable subdirectory does not discard the entire custom
  folder's results
- Only apply COLLATE NOCASE on the scan_folders schema on Windows where
  paths are case-insensitive; keep default BINARY collation on Linux
  and macOS where /Models and /models are distinct directories

* Use COLLATE NOCASE in post-IntegrityError fallback SELECT on Windows

The fallback SELECT after an IntegrityError race now uses the same
case-insensitive collation as the pre-insert check, so a concurrent
writer that stored the path with different casing does not cause a
false "Folder was concurrently removed" error.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
This commit is contained in:
Wasim Yousef Said 2026-03-31 15:40:31 +02:00 • committed by GitHub
commit 1e8875584d
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
10 changed files with 1709 additions and 920 deletions
Download patch file Download diff file Expand all files Collapse all files

18
studio/backend/models/models.py
View file

@ -165,7 +165,7 @@ class LocalModelInfo(BaseModel):
id: str = Field(..., description = "Identifier to use for loading/training")
display_name: str = Field(..., description = "Display label")
path: str = Field(..., description = "Local path where model data was discovered")
source: Literal["models_dir", "hf_cache", "lmstudio"] = Field(
source: Literal["models_dir", "hf_cache", "lmstudio", "custom"] = Field(
...,
description = "Discovery source",
)
@ -197,3 +197,19 @@ class LocalModelListResponse(BaseModel):
default_factory = list,
description = "Discovered local/cached models",
)
class AddScanFolderRequest(BaseModel):
"""Request body for adding a custom scan folder."""
path: str = Field(
..., description = "Absolute or relative directory path to scan for models"
)
class ScanFolderInfo(BaseModel):
"""A registered custom model scan folder."""
id: int = Field(..., description = "Database row ID")
path: str = Field(..., description = "Normalized absolute path")
created_at: str = Field(..., description = "ISO 8601 creation timestamp")

259
studio/backend/routes/models.py
View file

@ -94,7 +94,13 @@ from models import (
LoRAInfo,
ModelListResponse,
)
from models.models import GgufVariantDetail, GgufVariantsResponse, ModelType
from models.models import (
GgufVariantDetail,
GgufVariantsResponse,
ModelType,
ScanFolderInfo,
AddScanFolderRequest,
)
from models.responses import (
LoRABaseModelResponse,
VisionCheckResponse,
@ -128,21 +134,32 @@ def _resolve_hf_cache_dir() -> Path:
return Path.home() / ".cache" / "huggingface" / "hub"
def _scan_models_dir(models_dir: Path) -> List[LocalModelInfo]:
def _scan_models_dir(
models_dir: Path,
*,
limit: int | None = None,
) -> List[LocalModelInfo]:
if not models_dir.exists() or not models_dir.is_dir():
return []
found: List[LocalModelInfo] = []
for child in models_dir.iterdir():
if not child.is_dir():
if limit is not None and len(found) >= limit:
break
try:
if not child.is_dir():
continue
has_model_files = (
(child / "config.json").exists()
or (child / "adapter_config.json").exists()
or any(child.glob("*.safetensors"))
or any(child.glob("*.bin"))
or any(child.glob("*.gguf"))
)
except OSError:
# Skip individual children that are unreadable (permissions, broken
# symlinks, etc.) rather than failing the entire scan.
continue
has_model_files = (
(child / "config.json").exists()
or (child / "adapter_config.json").exists()
or any(child.glob("*.safetensors"))
or any(child.glob("*.bin"))
or any(child.glob("*.gguf"))
)
if not has_model_files:
continue
try:
@ -159,21 +176,24 @@ def _scan_models_dir(models_dir: Path) -> List[LocalModelInfo]:
),
)
# Also scan for standalone .gguf files directly in the models directory
for gguf_file in models_dir.glob("*.gguf"):
if gguf_file.is_file():
try:
updated_at = gguf_file.stat().st_mtime
except OSError:
updated_at = None
found.append(
LocalModelInfo(
id = str(gguf_file),
display_name = gguf_file.stem,
path = str(gguf_file),
source = "models_dir",
updated_at = updated_at,
),
)
if limit is None or len(found) < limit:
for gguf_file in models_dir.glob("*.gguf"):
if limit is not None and len(found) >= limit:
break
if gguf_file.is_file():
try:
updated_at = gguf_file.stat().st_mtime
except OSError:
updated_at = None
found.append(
LocalModelInfo(
id = str(gguf_file),
display_name = gguf_file.stem,
path = str(gguf_file),
source = "models_dir",
updated_at = updated_at,
),
)
return found
@ -221,63 +241,69 @@ def _scan_lmstudio_dir(lm_dir: Path) -> List[LocalModelInfo]:
found: List[LocalModelInfo] = []
for child in lm_dir.iterdir():
if not child.is_dir():
if child.suffix == ".gguf" and child.is_file():
try:
updated_at = child.stat().st_mtime
except OSError:
updated_at = None
found.append(
LocalModelInfo(
id = str(child),
display_name = child.stem,
path = str(child),
source = "lmstudio",
updated_at = updated_at,
),
)
continue
try:
if not child.is_dir():
if child.suffix == ".gguf" and child.is_file():
try:
updated_at = child.stat().st_mtime
except OSError:
updated_at = None
found.append(
LocalModelInfo(
id = str(child),
display_name = child.stem,
path = str(child),
source = "lmstudio",
updated_at = updated_at,
),
)
continue
# child is a publisher directory — scan its sub-directories
for model_dir in child.iterdir():
if model_dir.is_dir():
has_model = (
any(model_dir.glob("*.gguf"))
or (model_dir / "config.json").exists()
or any(model_dir.glob("*.safetensors"))
)
if not has_model:
# child is a publisher directory -- scan its sub-directories
for model_dir in child.iterdir():
try:
if model_dir.is_dir():
has_model = (
any(model_dir.glob("*.gguf"))
or (model_dir / "config.json").exists()
or any(model_dir.glob("*.safetensors"))
)
if not has_model:
continue
model_id = f"{child.name}/{model_dir.name}"
try:
updated_at = model_dir.stat().st_mtime
except OSError:
updated_at = None
found.append(
LocalModelInfo(
id = str(model_dir),
model_id = model_id,
display_name = model_dir.name,
path = str(model_dir),
source = "lmstudio",
updated_at = updated_at,
),
)
elif model_dir.suffix == ".gguf" and model_dir.is_file():
try:
updated_at = model_dir.stat().st_mtime
except OSError:
updated_at = None
found.append(
LocalModelInfo(
id = str(model_dir),
model_id = f"{child.name}/{model_dir.stem}",
display_name = model_dir.stem,
path = str(model_dir),
source = "lmstudio",
updated_at = updated_at,
),
)
except OSError:
continue
model_id = f"{child.name}/{model_dir.name}"
try:
updated_at = model_dir.stat().st_mtime
except OSError:
updated_at = None
found.append(
LocalModelInfo(
id = str(model_dir),
model_id = model_id,
display_name = model_dir.name,
path = str(model_dir),
source = "lmstudio",
updated_at = updated_at,
),
)
elif model_dir.suffix == ".gguf" and model_dir.is_file():
try:
updated_at = model_dir.stat().st_mtime
except OSError:
updated_at = None
found.append(
LocalModelInfo(
id = str(model_dir),
model_id = f"{child.name}/{model_dir.stem}",
display_name = model_dir.stem,
path = str(model_dir),
source = "lmstudio",
updated_at = updated_at,
),
)
except OSError:
continue
return found
@ -351,10 +377,39 @@ async def list_local_models(
for lm_dir in lm_dirs:
local_models += _scan_lmstudio_dir(lm_dir)
# Scan user-added custom folders (cap per-folder to avoid unbounded scans)
from storage.studio_db import list_scan_folders
_MAX_MODELS_PER_FOLDER = 200
try:
custom_folders = list_scan_folders()
except Exception as e:
logger.warning("Could not load custom scan folders: %s", e)
custom_folders = []
for folder in custom_folders:
folder_path = Path(folder["path"])
try:
custom_models = (
_scan_models_dir(folder_path, limit = _MAX_MODELS_PER_FOLDER)
+ _scan_hf_cache(folder_path)
+ _scan_lmstudio_dir(folder_path)
)[:_MAX_MODELS_PER_FOLDER]
except OSError as e:
logger.warning("Skipping unreadable scan folder %s: %s", folder_path, e)
continue
local_models += [
m.model_copy(update = {"source": "custom"}) for m in custom_models
]
# Deduplicate models, but always keep custom folder entries so they
# appear in the "Custom Folders" UI section even when the same model
# also exists in the HF cache or default models directory. Use a
# (id, source) key for custom entries to avoid collisions.
deduped: dict[str, LocalModelInfo] = {}
for model in local_models:
if model.id not in deduped:
deduped[model.id] = model
key = f"{model.id}\x00custom" if model.source == "custom" else model.id
if key not in deduped:
deduped[key] = model
models = sorted(
deduped.values(),
@ -376,6 +431,46 @@ async def list_local_models(
)
@router.get("/scan-folders")
async def get_scan_folders(
current_subject: str = Depends(get_current_subject),
):
"""List all registered custom model scan folders."""
from storage.studio_db import list_scan_folders
return {"folders": list_scan_folders()}
@router.post("/scan-folders", response_model = ScanFolderInfo, status_code = 201)
async def add_scan_folder_endpoint(
body: AddScanFolderRequest,
current_subject: str = Depends(get_current_subject),
):
"""Register a new directory to scan for local models."""
from storage.studio_db import add_scan_folder
try:
folder = add_scan_folder(body.path)
except ValueError as e:
logger.warning("Scan folder rejected: %s (path=%s)", e, body.path)
raise HTTPException(status_code = 400, detail = str(e))
logger.info("Scan folder added: %s", folder.get("path"))
return folder
@router.delete("/scan-folders/{folder_id}")
async def remove_scan_folder_endpoint(
folder_id: int,
current_subject: str = Depends(get_current_subject),
):
"""Remove a registered custom scan folder."""
from storage.studio_db import remove_scan_folder
remove_scan_folder(folder_id)
logger.info("Scan folder removed: id=%s", folder_id)
return {"ok": True}
@router.get("/list")
async def list_models(
current_subject: str = Depends(get_current_subject),

130
studio/backend/storage/studio_db.py
View file

@ -12,14 +12,46 @@ raw sqlite3, per-function connections. Enhancements over auth:
import json
import logging
import os
import platform
import sqlite3
import threading
from datetime import datetime, timezone
logger = logging.getLogger(__name__)
from typing import Optional
from utils.paths import studio_db_path, ensure_dir
def _denied_path_prefixes() -> list[str]:
"""Platform-aware denylist of system directories."""
system = platform.system()
if system == "Linux":
return ["/proc", "/sys", "/dev", "/etc", "/boot", "/run"]
if system == "Darwin":
# realpath() resolves /etc -> /private/etc, /tmp -> /private/tmp on macOS,
# so include the /private variants to avoid bypasses.
return [
"/System",
"/Library",
"/dev",
"/etc",
"/private/etc",
"/tmp",
"/private/tmp",
"/var",
"/private/var",
]
if system == "Windows":
win = os.environ.get("SystemRoot", r"C:\Windows")
pf = os.environ.get("ProgramFiles", r"C:\Program Files")
pf86 = os.environ.get("ProgramFiles(x86)", r"C:\Program Files (x86)")
return [os.path.normcase(p) for p in [win, pf, pf86]]
return []
_schema_lock = threading.Lock()
_schema_ready = False
@ -67,6 +99,19 @@ def _ensure_schema(conn: sqlite3.Connection) -> None:
conn.execute(
"CREATE INDEX IF NOT EXISTS idx_metrics_run_id ON training_metrics(run_id)"
)
# Use COLLATE NOCASE on Windows so C:\Models and c:\models dedup via the
# UNIQUE constraint. On Linux/macOS (case-sensitive FS) keep the default
# BINARY collation so /Models and /models remain distinct.
collation = "COLLATE NOCASE" if platform.system() == "Windows" else ""
conn.execute(
f"""
CREATE TABLE IF NOT EXISTS scan_folders (
id INTEGER PRIMARY KEY AUTOINCREMENT,
path TEXT NOT NULL UNIQUE {collation},
created_at TEXT NOT NULL
)
"""
)
def get_connection() -> sqlite3.Connection:
@ -343,8 +388,6 @@ def delete_run(id: str) -> None:
def cleanup_orphaned_runs() -> None:
"""Mark any 'running' rows as errored on startup (server restarted mid-training)."""
from datetime import datetime, timezone
conn = get_connection()
try:
conn.execute(
@ -360,3 +403,86 @@ def cleanup_orphaned_runs() -> None:
conn.commit()
finally:
conn.close()
def list_scan_folders() -> list[dict]:
conn = get_connection()
try:
rows = conn.execute(
"SELECT id, path, created_at FROM scan_folders ORDER BY created_at"
).fetchall()
return [dict(row) for row in rows]
finally:
conn.close()
def add_scan_folder(path: str) -> dict:
"""Add a directory to the custom scan folder list. Returns the row."""
if not path or not path.strip():
raise ValueError("Path cannot be empty")
normalized = os.path.realpath(os.path.expanduser(path.strip()))
# Validate the path is an existing, readable directory before persisting.
if not os.path.exists(normalized):
raise ValueError("Path does not exist")
if not os.path.isdir(normalized):
raise ValueError("Path must be a directory, not a file")
if not os.access(normalized, os.R_OK | os.X_OK):
raise ValueError("Path is not readable")
# On Windows, use normcase for denylist comparison but store the
# original-cased path so downstream consumers see the native
# drive-letter casing the user expects (e.g. C:\Models, not c:\models).
is_win = platform.system() == "Windows"
check = os.path.normcase(normalized) if is_win else normalized
for prefix in _denied_path_prefixes():
if check == prefix or check.startswith(prefix + os.sep):
raise ValueError(f"Path under {prefix} is not allowed")
conn = get_connection()
try:
now = datetime.now(timezone.utc).isoformat()
# On Windows, use case-insensitive lookup so C:\Models and c:\models
# dedup correctly while preserving the originally-stored casing.
if is_win:
existing = conn.execute(
"SELECT id, path, created_at FROM scan_folders WHERE path = ? COLLATE NOCASE",
(normalized,),
).fetchone()
else:
existing = conn.execute(
"SELECT id, path, created_at FROM scan_folders WHERE path = ?",
(normalized,),
).fetchone()
if existing is not None:
return dict(existing)
try:
conn.execute(
"INSERT INTO scan_folders (path, created_at) VALUES (?, ?)",
(normalized, now),
)
conn.commit()
except sqlite3.IntegrityError:
pass # duplicate -- fall through to SELECT
# Use the same collation as the pre-check so we find the row even
# when a concurrent writer stored it with different casing (Windows).
fallback_sql = (
"SELECT id, path, created_at FROM scan_folders WHERE path = ? COLLATE NOCASE"
if is_win
else "SELECT id, path, created_at FROM scan_folders WHERE path = ?"
)
row = conn.execute(fallback_sql, (normalized,)).fetchone()
if row is None:
raise ValueError("Folder was concurrently removed")
return dict(row)
finally:
conn.close()
def remove_scan_folder(id: int) -> None:
conn = get_connection()
try:
conn.execute("DELETE FROM scan_folders WHERE id = ?", (id,))
conn.commit()
finally:
conn.close()

443
studio/frontend/src/components/assistant-ui/model-selector/pickers.tsx
View file

@ -18,10 +18,20 @@ import {
TooltipContent,
TooltipTrigger,
} from "@/components/ui/tooltip";
import { deleteCachedModel, listCachedGguf, listCachedModels, listGgufVariants, listLocalModels } from "@/features/chat/api/chat-api";
import type { CachedGgufRepo, CachedModelRepo, LocalModelInfo } from "@/features/chat/api/chat-api";
import type { GgufVariantDetail } from "@/features/chat/types/api";
import { usePlatformStore } from "@/config/env";
import {
deleteCachedModel,
listCachedGguf,
listCachedModels,
listGgufVariants,
listLocalModels,
} from "@/features/chat/api/chat-api";
import type {
CachedGgufRepo,
CachedModelRepo,
LocalModelInfo,
} from "@/features/chat/api/chat-api";
import type { GgufVariantDetail } from "@/features/chat/types/api";
import {
useDebouncedValue,
useGpuInfo,
@ -35,7 +45,13 @@ import { checkVramFit, estimateLoadingVram } from "@/lib/vram";
import { Search01Icon } from "@hugeicons/core-free-icons";
import { HugeiconsIcon } from "@hugeicons/react";
import { Trash2Icon } from "lucide-react";
import { useCallback, useEffect, useMemo, useRef, useState, type ReactNode } from "react";
import {
type ReactNode,
useCallback,
useEffect,
useMemo,
useState,
} from "react";
import { toast } from "sonner";
import type {
LoraModelOption,
@ -135,7 +151,7 @@ function ModelRow({
if (vramTooltipText) {
return (
<Tooltip>
<TooltipTrigger asChild>{content}</TooltipTrigger>
<TooltipTrigger asChild={true}>{content}</TooltipTrigger>
<TooltipContent side="left" className="max-w-xs break-all">
{label}
<span className="block text-[10px] mt-1">{vramTooltipText}</span>
@ -147,7 +163,7 @@ function ModelRow({
if (tooltipText) {
return (
<Tooltip>
<TooltipTrigger asChild>{content}</TooltipTrigger>
<TooltipTrigger asChild={true}>{content}</TooltipTrigger>
<TooltipContent side="left" className="max-w-xs break-all">
{tooltipText}
</TooltipContent>
@ -192,7 +208,9 @@ function GgufVariantExpander({
})
.catch((err) => {
if (canceled) return;
setError(err instanceof Error ? err.message : "Failed to load variants");
setError(
err instanceof Error ? err.message : "Failed to load variants",
);
})
.finally(() => {
if (!canceled) setLoading(false);
@ -204,7 +222,9 @@ function GgufVariantExpander({
}, [repoId]);
// Covers Unix absolute (/), Windows drive (C:\, D:/), UNC (\\server), relative (./, ../), tilde (~/)
const isLocalPath = /^(\/|\.{1,2}[\\\/]|~[\\\/]|[A-Za-z]:[\\\/]|\\\\)/.test(repoId);
const isLocalPath = /^(\/|\.{1,2}[\\\/]|~[\\\/]|[A-Za-z]:[\\\/]|\\\\)/.test(
repoId,
);
const handleVariantClick = useCallback(
(quant: string, downloaded?: boolean, sizeBytes?: number) => {
@ -223,13 +243,13 @@ function GgufVariantExpander({
// fits = model <= 0.7 * total GPU memory
// tight = model > 0.7 * GPU but <= 0.7 * GPU + 0.7 * system RAM (--fit uses CPU offload)
// oom = model > 0.7 * GPU + 0.7 * system RAM
const gpuBudgetGb = (gpuGb ?? 0) * 0.70;
const totalBudgetGb = gpuBudgetGb + (systemRamGb ?? 0) * 0.70;
const gpuBudgetGb = (gpuGb ?? 0) * 0.7;
const totalBudgetGb = gpuBudgetGb + (systemRamGb ?? 0) * 0.7;
const getGgufFit = useCallback(
(sizeBytes: number): "fits" | "tight" | "oom" => {
if (!gpuGb || gpuGb <= 0) return "fits";
const gb = sizeBytes / (1024 ** 3);
const gb = sizeBytes / 1024 ** 3;
if (gb <= 0 || gb <= gpuBudgetGb) return "fits";
if (gb <= totalBudgetGb) return "tight";
return "oom";
@ -242,7 +262,8 @@ function GgufVariantExpander({
const effectiveRecommended = useMemo(() => {
if (!variants || !gpuGb || gpuGb <= 0) return defaultVariant;
const defaultV = variants.find((v) => v.quant === defaultVariant);
if (defaultV && getGgufFit(defaultV.size_bytes) !== "oom") return defaultVariant;
if (defaultV && getGgufFit(defaultV.size_bytes) !== "oom")
return defaultVariant;
// Default is OOM -- pick largest non-OOM variant (best quality that fits)
const fitting = variants.filter((v) => getGgufFit(v.size_bytes) !== "oom");
if (fitting.length > 0) {
@ -276,7 +297,9 @@ function GgufVariantExpander({
// fits: largest first (best quality that fits in GPU)
// tight/OOM: smallest first (closest to fitting, fastest to run)
const fitsInGpu = aTier === 0 || aTier === 2;
return fitsInGpu ? b.size_bytes - a.size_bytes : a.size_bytes - b.size_bytes;
return fitsInGpu
? b.size_bytes - a.size_bytes
: a.size_bytes - b.size_bytes;
});
}, [variants, effectiveRecommended, getGgufFit]);
@ -290,9 +313,7 @@ function GgufVariantExpander({
}
if (error) {
return (
<div className="px-5 py-2 text-xs text-destructive">{error}</div>
);
return <div className="px-5 py-2 text-xs text-destructive">{error}</div>;
}
if (!sortedVariants || sortedVariants.length === 0) {
@ -321,7 +342,9 @@ function GgufVariantExpander({
<div key={v.filename} className="flex items-center gap-0.5">
<button
type="button"
onClick={() => handleVariantClick(v.quant, v.downloaded, v.size_bytes)}
onClick={() =>
handleVariantClick(v.quant, v.downloaded, v.size_bytes)
}
className={cn(
"flex min-w-0 flex-1 items-center justify-between gap-2 rounded-md px-2.5 py-1 text-left text-sm transition-colors hover:bg-accent",
)}
@ -340,10 +363,14 @@ function GgufVariantExpander({
</span>
<span className="flex items-center gap-1.5 shrink-0">
{oom && (
<span className="text-[9px] font-medium text-red-400">OOM</span>
<span className="text-[9px] font-medium text-red-400">
OOM
</span>
)}
{tight && (
<span className="text-[9px] font-medium text-amber-400">TIGHT</span>
<span className="text-[9px] font-medium text-amber-400">
TIGHT
</span>
)}
<span className="text-[10px] text-muted-foreground">
{formatBytes(v.size_bytes)}
@ -353,7 +380,10 @@ function GgufVariantExpander({
{v.downloaded && onDeleteVariant && (
<button
type="button"
onClick={(e) => { e.stopPropagation(); onDeleteVariant(v.quant); }}
onClick={(e) => {
e.stopPropagation();
onDeleteVariant(v.quant);
}}
className="shrink-0 rounded-md p-1 text-muted-foreground/60 transition-colors hover:bg-destructive/10 hover:text-destructive"
>
<Trash2Icon className="size-3" />
@ -384,6 +414,7 @@ function extractParamLabel(id: string): string | undefined {
let _cachedGgufCache: CachedGgufRepo[] = [];
let _cachedModelsCache: CachedModelRepo[] = [];
let _lmStudioCache: LocalModelInfo[] = [];
let _customFolderCache: LocalModelInfo[] = [];
/** Sort LM Studio models with unsloth publisher first. */
function sortLmStudio(models: LocalModelInfo[]): LocalModelInfo[] {
@ -391,7 +422,9 @@ function sortLmStudio(models: LocalModelInfo[]): LocalModelInfo[] {
const aUnsloth = (a.model_id ?? "").startsWith("unsloth/") ? 0 : 1;
const bUnsloth = (b.model_id ?? "").startsWith("unsloth/") ? 0 : 1;
if (aUnsloth !== bUnsloth) return aUnsloth - bUnsloth;
return (a.model_id ?? a.display_name).localeCompare(b.model_id ?? b.display_name);
return (a.model_id ?? a.display_name).localeCompare(
b.model_id ?? b.display_name,
);
});
}
@ -409,9 +442,8 @@ export function HubModelPicker({
const gpu = useGpuInfo();
const [query, setQuery] = useState("");
const debouncedQuery = useDebouncedValue(query);
const { results, isLoading, isLoadingMore, fetchMore } = useHfModelSearch(
debouncedQuery,
);
const { results, isLoading, isLoadingMore, fetchMore } =
useHfModelSearch(debouncedQuery);
// Track which GGUF repo is expanded for variant selection
const [expandedGguf, setExpandedGguf] = useState<string | null>(null);
@ -422,38 +454,75 @@ export function HubModelPicker({
// Cached (already downloaded) repos -- use module-level cache so
// re-mounting the popover does not flash an empty "Downloaded" section.
const [cachedGguf, setCachedGguf] = useState<CachedGgufRepo[]>(_cachedGgufCache);
const [cachedModels, setCachedModels] = useState<CachedModelRepo[]>(_cachedModelsCache);
const alreadyCached = _cachedGgufCache.length > 0 || _cachedModelsCache.length > 0;
const [cachedGguf, setCachedGguf] =
useState<CachedGgufRepo[]>(_cachedGgufCache);
const [cachedModels, setCachedModels] =
useState<CachedModelRepo[]>(_cachedModelsCache);
const alreadyCached =
_cachedGgufCache.length > 0 || _cachedModelsCache.length > 0;
const [cachedReady, setCachedReady] = useState(alreadyCached);
// LM Studio local models -- module-level cache so re-mounting the
// popover does not flash an empty section (same pattern as GGUF/models).
const [lmStudioModels, setLmStudioModels] = useState<LocalModelInfo[]>(_lmStudioCache);
const [lmStudioModels, setLmStudioModels] =
useState<LocalModelInfo[]>(_lmStudioCache);
const [customFolderModels, setCustomFolderModels] =
useState<LocalModelInfo[]>(_customFolderCache);
const refreshCachedLists = useCallback(() => {
listCachedGguf().then((v) => { _cachedGgufCache = v; setCachedGguf(v); }).catch(() => {});
listCachedModels().then((v) => { _cachedModelsCache = v; setCachedModels(v); }).catch(() => {});
listLocalModels().then((res) => {
const next = sortLmStudio(res.models.filter((m) => m.source === "lmstudio"));
_lmStudioCache = next;
setLmStudioModels(next);
}).catch(() => {});
const refreshLocalModelsList = useCallback(() => {
listLocalModels()
.then((res) => {
const lm = sortLmStudio(
res.models.filter((m) => m.source === "lmstudio"),
);
_lmStudioCache = lm;
setLmStudioModels(lm);
const cf = res.models.filter((m) => m.source === "custom");
_customFolderCache = cf;
setCustomFolderModels(cf);
})
.catch(() => {});
}, []);
const refreshCachedLists = useCallback(() => {
listCachedGguf()
.then((v) => {
_cachedGgufCache = v;
setCachedGguf(v);
})
.catch(() => {});
listCachedModels()
.then((v) => {
_cachedModelsCache = v;
setCachedModels(v);
})
.catch(() => {});
refreshLocalModelsList();
}, [refreshLocalModelsList]);
useEffect(() => {
// Always refresh LM Studio models (not gated by alreadyCached)
listLocalModels().then((res) => {
const next = sortLmStudio(res.models.filter((m) => m.source === "lmstudio"));
_lmStudioCache = next;
setLmStudioModels(next);
}).catch(() => {});
// Always refresh LM Studio + custom folder models (not gated by alreadyCached)
refreshLocalModelsList();
if (alreadyCached) return;
let done = 0;
const check = () => { if (++done >= 2) setCachedReady(true); };
listCachedGguf().then((v) => { _cachedGgufCache = v; setCachedGguf(v); }).catch(() => {}).finally(check);
listCachedModels().then((v) => { _cachedModelsCache = v; setCachedModels(v); }).catch(() => {}).finally(check);
const check = () => {
if (++done >= 2) setCachedReady(true);
};
listCachedGguf()
.then((v) => {
_cachedGgufCache = v;
setCachedGguf(v);
})
.catch(() => {})
.finally(check);
listCachedModels()
.then((v) => {
_cachedModelsCache = v;
setCachedModels(v);
})
.catch(() => {})
.finally(check);
}, [alreadyCached]);
const handleDeleteConfirm = useCallback(async () => {
@ -468,7 +537,9 @@ export function HubModelPicker({
toast.success(`Deleted ${variant ? `${repoId} ${variant}` : repoId}`);
refreshCachedLists();
} catch (err) {
toast.error(err instanceof Error ? err.message : "Failed to delete model");
toast.error(
err instanceof Error ? err.message : "Failed to delete model",
);
} finally {
setDeleting(false);
setDeleteTarget(null);
@ -504,12 +575,18 @@ export function HubModelPicker({
// Infinite scroll paging for the recommended section
const [recommendedPage, setRecommendedPage] = useState(1);
// Reset page when the underlying list changes
useEffect(() => { setRecommendedPage(1); }, [models, chatOnly]);
useEffect(() => {
setRecommendedPage(1);
}, [models, chatOnly]);
const visibleRecommendedIds = useMemo(() => {
const hubStartIndex = recommendedIds.findIndex((id) => !isGgufRepo(id));
const allGguf = hubStartIndex === -1 ? recommendedIds : recommendedIds.slice(0, hubStartIndex);
const allHub = hubStartIndex === -1 ? [] : recommendedIds.slice(hubStartIndex);
const allGguf =
hubStartIndex === -1
? recommendedIds
: recommendedIds.slice(0, hubStartIndex);
const allHub =
hubStartIndex === -1 ? [] : recommendedIds.slice(hubStartIndex);
// Interleave in chunks of 4: [4 gguf, 4 hub, 4 gguf, 4 hub, ...]
const result: string[] = [];
for (let p = 0; p < recommendedPage; p++) {
@ -519,7 +596,8 @@ export function HubModelPicker({
return result;
}, [recommendedIds, recommendedPage]);
const hasMoreRecommended = visibleRecommendedIds.length < recommendedIds.length;
const hasMoreRecommended =
visibleRecommendedIds.length < recommendedIds.length;
const showHfSection = debouncedQuery.trim().length > 0;
@ -544,7 +622,8 @@ export function HubModelPicker({
useRecommendedModelVram(idsForVram);
const recommendedSet = useMemo(
() => new Set(showHfSection ? filteredRecommendedIds : visibleRecommendedIds),
() =>
new Set(showHfSection ? filteredRecommendedIds : visibleRecommendedIds),
[showHfSection, filteredRecommendedIds, visibleRecommendedIds],
);
@ -610,15 +689,25 @@ export function HubModelPicker({
}
}
return map;
}, [showHfSection, filteredRecommendedIds, visibleRecommendedIds, recommendedParamCountById, gpu]);
}, [
showHfSection,
filteredRecommendedIds,
visibleRecommendedIds,
recommendedParamCountById,
gpu,
]);
const { scrollRef, sentinelRef } = useInfiniteScroll(fetchMore, results.length);
const { scrollRef, sentinelRef } = useInfiniteScroll(
fetchMore,
results.length,
);
// Sentinel + IntersectionObserver for recommended infinite scroll.
// We disconnect after each fire so the observer doesn't loop while
// React re-renders; the effect re-creates it on the next page.
// Uses a callback ref for the sentinel so we detect mount/unmount reliably.
const [recommendedSentinel, setRecommendedSentinel] = useState<HTMLDivElement | null>(null);
const [recommendedSentinel, setRecommendedSentinel] =
useState<HTMLDivElement | null>(null);
const recommendedSentinelRef = useCallback((node: HTMLDivElement | null) => {
setRecommendedSentinel(node);
}, []);
@ -637,7 +726,10 @@ export function HubModelPicker({
);
// Small delay so the browser finishes layout after the previous page render
const timer = setTimeout(() => obs.observe(recommendedSentinel), 100);
return () => { clearTimeout(timer); obs.disconnect(); };
return () => {
clearTimeout(timer);
obs.disconnect();
};
}, [recommendedSentinel, hasMoreRecommended, recommendedPage, scrollRef]);
/** Handle clicking a model row — GGUF repos expand, others load directly. */
@ -676,9 +768,13 @@ export function HubModelPicker({
{!cachedReady && !showHfSection ? (
<div className="flex items-center gap-2 px-5 py-3">
<Spinner className="size-3 text-muted-foreground" />
<span className="text-xs text-muted-foreground">Loading models…</span>
<span className="text-xs text-muted-foreground">
Loading models…
</span>
</div>
) : !showHfSection && (cachedGguf.length > 0 || (!chatOnly && cachedModels.length > 0)) ? (
) : !showHfSection &&
(cachedGguf.length > 0 ||
(!chatOnly && cachedModels.length > 0)) ? (
<>
<ListLabel>{"\uD83E\uDDA5"} Downloaded</ListLabel>
{cachedGguf.map((c) => (
@ -695,32 +791,46 @@ export function HubModelPicker({
repoId={c.repo_id}
onSelect={onSelect}
gpuGb={gpu.available ? gpu.memoryTotalGb : undefined}
systemRamGb={gpu.available ? gpu.systemRamAvailableGb : undefined}
onDeleteVariant={(quant) => setDeleteTarget(`${c.repo_id}::${quant}`)}
systemRamGb={
gpu.available ? gpu.systemRamAvailableGb : undefined
}
onDeleteVariant={(quant) =>
setDeleteTarget(`${c.repo_id}::${quant}`)
}
/>
)}
</div>
))}
{!chatOnly && cachedModels.map((c) => (
<div key={c.repo_id} className="flex items-center gap-0.5">
<div className="min-w-0 flex-1">
<ModelRow
label={c.repo_id}
meta={formatBytes(c.size_bytes)}
selected={value === c.repo_id}
onClick={() => onSelect(c.repo_id, { source: "hub", isLora: false, isDownloaded: true })}
vramStatus={null}
/>
{!chatOnly &&
cachedModels.map((c) => (
<div key={c.repo_id} className="flex items-center gap-0.5">
<div className="min-w-0 flex-1">
<ModelRow
label={c.repo_id}
meta={formatBytes(c.size_bytes)}
selected={value === c.repo_id}
onClick={() =>
onSelect(c.repo_id, {
source: "hub",
isLora: false,
isDownloaded: true,
})
}
vramStatus={null}
/>
</div>
<button
type="button"
onClick={(e) => {
e.stopPropagation();
setDeleteTarget(c.repo_id);
}}
className="shrink-0 rounded-md p-1.5 text-muted-foreground/60 transition-colors hover:bg-destructive/10 hover:text-destructive"
>
<Trash2Icon className="size-3.5" />
</button>
</div>
<button
type="button"
onClick={(e) => { e.stopPropagation(); setDeleteTarget(c.repo_id); }}
className="shrink-0 rounded-md p-1.5 text-muted-foreground/60 transition-colors hover:bg-destructive/10 hover:text-destructive"
>
<Trash2Icon className="size-3.5" />
</button>
</div>
))}
))}
</>
) : null}
@ -733,13 +843,21 @@ export function HubModelPicker({
<div key={m.id}>
<ModelRow
label={m.model_id ?? m.display_name}
meta={isGguf || m.path.endsWith(".gguf") ? "GGUF" : "Local"}
meta={
isGguf || m.path.endsWith(".gguf") ? "GGUF" : "Local"
}
selected={value === m.id}
onClick={() => {
if (isGguf) {
setExpandedGguf((prev) => (prev === m.id ? null : m.id));
setExpandedGguf((prev) =>
prev === m.id ? null : m.id,
);
} else {
onSelect(m.id, { source: "local", isLora: false, isDownloaded: true });
onSelect(m.id, {
source: "local",
isLora: false,
isDownloaded: true,
});
}
}}
vramStatus={null}
@ -749,7 +867,54 @@ export function HubModelPicker({
repoId={m.id}
onSelect={onSelect}
gpuGb={gpu.available ? gpu.memoryTotalGb : undefined}
systemRamGb={gpu.available ? gpu.systemRamAvailableGb : undefined}
systemRamGb={
gpu.available ? gpu.systemRamAvailableGb : undefined
}
/>
)}
</div>
);
})}
</>
) : null}
{!showHfSection && customFolderModels.length > 0 ? (
<>
<ListLabel>Custom Folders</ListLabel>
{customFolderModels.map((m) => {
const isGguf =
isGgufRepo(m.id) ||
isGgufRepo(m.display_name) ||
m.path.endsWith(".gguf");
return (
<div key={m.id}>
<ModelRow
label={m.model_id ?? m.display_name}
meta={isGguf ? "GGUF" : "Local"}
selected={value === m.id}
onClick={() => {
if (isGguf) {
setExpandedGguf((prev) =>
prev === m.id ? null : m.id,
);
} else {
onSelect(m.id, {
source: "local",
isLora: false,
isDownloaded: true,
});
}
}}
vramStatus={null}
/>
{expandedGguf === m.id && (
<GgufVariantExpander
repoId={m.id}
onSelect={onSelect}
gpuGb={gpu.available ? gpu.memoryTotalGb : undefined}
systemRamGb={
gpu.available ? gpu.systemRamAvailableGb : undefined
}
/>
)}
</div>
@ -775,16 +940,25 @@ export function HubModelPicker({
meta={
isGgufRepo(id)
? "GGUF"
: vram?.detail ?? extractParamLabel(id)
: (vram?.detail ?? extractParamLabel(id))
}
selected={value === id}
onClick={() => handleModelClick(id)}
vramStatus={isGgufRepo(id) ? null : vram?.status ?? null}
vramStatus={
isGgufRepo(id) ? null : (vram?.status ?? null)
}
vramEst={isGgufRepo(id) ? undefined : vram?.est}
gpuGb={gpu.available ? gpu.memoryTotalGb : undefined}
/>
{expandedGguf === id && (
<GgufVariantExpander repoId={id} onSelect={onSelect} gpuGb={gpu.available ? gpu.memoryTotalGb : undefined} systemRamGb={gpu.available ? gpu.systemRamAvailableGb : undefined} />
<GgufVariantExpander
repoId={id}
onSelect={onSelect}
gpuGb={gpu.available ? gpu.memoryTotalGb : undefined}
systemRamGb={
gpu.available ? gpu.systemRamAvailableGb : undefined
}
/>
)}
</div>
);
@ -813,16 +987,25 @@ export function HubModelPicker({
meta={
isGgufRepo(id)
? "GGUF"
: vram?.detail ?? extractParamLabel(id)
: (vram?.detail ?? extractParamLabel(id))
}
selected={value === id}
onClick={() => handleModelClick(id)}
vramStatus={isGgufRepo(id) ? null : vram?.status ?? null}
vramStatus={
isGgufRepo(id) ? null : (vram?.status ?? null)
}
vramEst={isGgufRepo(id) ? undefined : vram?.est}
gpuGb={gpu.available ? gpu.memoryTotalGb : undefined}
/>
{expandedGguf === id && (
<GgufVariantExpander repoId={id} onSelect={onSelect} gpuGb={gpu.available ? gpu.memoryTotalGb : undefined} systemRamGb={gpu.available ? gpu.systemRamAvailableGb : undefined} />
<GgufVariantExpander
repoId={id}
onSelect={onSelect}
gpuGb={gpu.available ? gpu.memoryTotalGb : undefined}
systemRamGb={
gpu.available ? gpu.systemRamAvailableGb : undefined
}
/>
)}
</div>
);
@ -832,7 +1015,9 @@ export function HubModelPicker({
{showHfSection ? (
<>
{(hfIds.length > 0 || isLoading) && <ListLabel>Hugging Face</ListLabel>}
{(hfIds.length > 0 || isLoading) && (
<ListLabel>Hugging Face</ListLabel>
)}
{hfIds.length === 0 && !isLoading ? (
filteredRecommendedIds.length === 0 ? (
<div className="px-2.5 py-2 text-xs text-muted-foreground">
@ -849,16 +1034,25 @@ export function HubModelPicker({
meta={
isGgufRepo(id)
? "GGUF"
: metricsById.get(id) ?? extractParamLabel(id)
: (metricsById.get(id) ?? extractParamLabel(id))
}
selected={value === id}
onClick={() => handleModelClick(id)}
vramStatus={isGgufRepo(id) ? null : vram?.status ?? null}
vramStatus={
isGgufRepo(id) ? null : (vram?.status ?? null)
}
vramEst={isGgufRepo(id) ? undefined : vram?.est}
gpuGb={gpu.available ? gpu.memoryTotalGb : undefined}
/>
{expandedGguf === id && (
<GgufVariantExpander repoId={id} onSelect={onSelect} gpuGb={gpu.available ? gpu.memoryTotalGb : undefined} systemRamGb={gpu.available ? gpu.systemRamAvailableGb : undefined} />
<GgufVariantExpander
repoId={id}
onSelect={onSelect}
gpuGb={gpu.available ? gpu.memoryTotalGb : undefined}
systemRamGb={
gpu.available ? gpu.systemRamAvailableGb : undefined
}
/>
)}
</div>
);
@ -875,12 +1069,23 @@ export function HubModelPicker({
</div>
</div>
<AlertDialog open={deleteTarget !== null} onOpenChange={(open) => { if (!open && !deleting) setDeleteTarget(null); }}>
<AlertDialog
open={deleteTarget !== null}
onOpenChange={(open) => {
if (!open && !deleting) setDeleteTarget(null);
}}
>
<AlertDialogContent size="sm">
<AlertDialogHeader>
<AlertDialogTitle>Delete cached model?</AlertDialogTitle>
<AlertDialogDescription>
This will remove <span className="font-medium text-foreground">{deleteTarget?.includes("::") ? `${deleteTarget.split("::")[0]} (${deleteTarget.split("::")[1]})` : deleteTarget}</span> from disk. You can re-download it later.
This will remove{" "}
<span className="font-medium text-foreground">
{deleteTarget?.includes("::")
? `${deleteTarget.split("::")[0]} (${deleteTarget.split("::")[1]})`
: deleteTarget}
</span>{" "}
from disk. You can re-download it later.
</AlertDialogDescription>
</AlertDialogHeader>
<AlertDialogFooter>
@ -888,7 +1093,10 @@ export function HubModelPicker({
<AlertDialogAction
variant="destructive"
disabled={deleting}
onClick={(e) => { e.preventDefault(); handleDeleteConfirm(); }}
onClick={(e) => {
e.preventDefault();
handleDeleteConfirm();
}}
>
{deleting ? "Deleting..." : "Yes"}
</AlertDialogAction>
@ -917,7 +1125,8 @@ export function LoraModelPicker({
loraModels
.map((model) => ({
...model,
baseModel: model.baseModel || model.description || "Unknown base model",
baseModel:
model.baseModel || model.description || "Unknown base model",
}))
.sort((a, b) => {
const baseCmp = a.baseModel.localeCompare(b.baseModel);
@ -941,7 +1150,9 @@ export function LoraModelPicker({
const out = new Map<string, LoraModelOption[]>();
for (const model of normalized) {
const searchText = normalizeForSearch(`${model.name} ${model.baseModel} ${model.id}`);
const searchText = normalizeForSearch(
`${model.name} ${model.baseModel} ${model.id}`,
);
if (needle && !searchText.includes(needle)) continue;
const key = model.baseModel || "Unknown base model";
@ -989,15 +1200,27 @@ export function LoraModelPicker({
const isExported = adapter.source === "exported";
const isMerged = adapter.exportType === "merged";
const isGguf = adapter.exportType === "gguf";
const isLocalGgufDir = isLocal && (isGgufRepo(adapter.id) || isGgufRepo(adapter.name));
const isLocalGgufDir =
isLocal &&
(isGgufRepo(adapter.id) || isGgufRepo(adapter.name));
const tag = isLocal
? isLocalGgufDir ? "GGUF" : "Local"
? isLocalGgufDir
? "GGUF"
: "Local"
: isGguf
? "GGUF"
: isExported
? isMerged ? "Merged" : "LoRA"
? isMerged
? "Merged"
: "LoRA"
: "LoRA";
const meta = isLocal ? (isLocalGgufDir ? "GGUF" : "Local") : isExported ? `${tag} · Exported` : tag;
const meta = isLocal
? isLocalGgufDir
? "GGUF"
: "Local"
: isExported
? `${tag} · Exported`
: tag;
return (
<div key={adapter.id}>
<ModelRow
@ -1006,10 +1229,16 @@ export function LoraModelPicker({
selected={value === adapter.id}
onClick={() => {
if (isLocalGgufDir) {
setExpandedGguf((prev) => (prev === adapter.id ? null : adapter.id));
setExpandedGguf((prev) =>
prev === adapter.id ? null : adapter.id,
);
} else {
onSelect(adapter.id, {
source: isLocal ? "local" : isExported ? "exported" : "lora",
source: isLocal
? "local"
: isExported
? "exported"
: "lora",
isLora: !isLocal && !isMerged && !isGguf,
isDownloaded: true,
});
@ -1017,7 +1246,9 @@ export function LoraModelPicker({
}}
tooltipText={
<>
<span className="block break-words">{adapter.name}</span>
<span className="block break-words">
{adapter.name}
</span>
<span className="block mt-1 text-[10px] text-muted-foreground break-all">
{adapter.id}
</span>
@ -1029,7 +1260,9 @@ export function LoraModelPicker({
repoId={adapter.id}
onSelect={onSelect}
gpuGb={gpu.available ? gpu.memoryTotalGb : undefined}
systemRamGb={gpu.available ? gpu.systemRamAvailableGb : undefined}
systemRamGb={
gpu.available ? gpu.systemRamAvailableGb : undefined
}
/>
)}
</div>

30
studio/frontend/src/features/chat/api/chat-api.ts
View file

@ -129,7 +129,7 @@ export interface LocalModelInfo {
id: string;
display_name: string;
path: string;
source: "models_dir" | "hf_cache" | "lmstudio";
source: "models_dir" | "hf_cache" | "lmstudio" | "custom";
model_id?: string | null;
updated_at?: number | null;
}
@ -174,6 +174,34 @@ export async function deleteCachedModel(repoId: string, variant?: string): Promi
await parseJsonOrThrow<unknown>(response);
}
export interface ScanFolderInfo {
id: number;
path: string;
created_at: string;
}
export async function listScanFolders(): Promise<ScanFolderInfo[]> {
const response = await authFetch("/api/models/scan-folders");
const data = await parseJsonOrThrow<{ folders: ScanFolderInfo[] }>(response);
return data.folders;
}
export async function addScanFolder(path: string): Promise<ScanFolderInfo> {
const response = await authFetch("/api/models/scan-folders", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ path }),
});
return parseJsonOrThrow<ScanFolderInfo>(response);
}
export async function removeScanFolder(id: number): Promise<void> {
const response = await authFetch(`/api/models/scan-folders/${id}`, {
method: "DELETE",
});
await parseJsonOrThrow<unknown>(response);
}
export async function listGgufVariants(
repoId: string,
hfToken?: string,

245
studio/frontend/src/features/chat/chat-page.tsx
View file

@ -8,7 +8,6 @@ import {
} from "@/components/assistant-ui/model-selector";
import { Thread } from "@/components/assistant-ui/thread";
import { Button } from "@/components/ui/button";
import { SidebarProvider, SidebarTrigger, useSidebar } from "@/components/ui/sidebar";
import {
Sheet,
SheetContent,
@ -16,7 +15,17 @@ import {
SheetHeader,
SheetTitle,
} from "@/components/ui/sheet";
import { Tooltip, TooltipContent, TooltipTrigger } from "@/components/ui/tooltip";
import {
SidebarProvider,
SidebarTrigger,
useSidebar,
} from "@/components/ui/sidebar";
import {
Tooltip,
TooltipContent,
TooltipTrigger,
} from "@/components/ui/tooltip";
import { GuidedTour, useGuidedTourController } from "@/features/tour";
import { cn } from "@/lib/utils";
import {
ColumnInsertIcon,
@ -36,7 +45,6 @@ import {
useState,
} from "react";
import { toast } from "sonner";
import { GuidedTour, useGuidedTourController } from "@/features/tour";
import { listLocalModels } from "./api/chat-api";
import { ChatSettingsPanel } from "./chat-settings-sheet";
import { ContextUsageBar } from "./components/context-usage-bar";
@ -48,16 +56,16 @@ import {
getTrainingCompareHandoff,
} from "./lib/training-compare-handoff";
import { ChatRuntimeProvider } from "./runtime-provider";
import { useChatRuntimeStore } from "./stores/chat-runtime-store";
import {
type CompareHandle,
CompareHandlesProvider,
RegisterCompareHandle,
SharedComposer,
} from "./shared-composer";
import { useChatRuntimeStore } from "./stores/chat-runtime-store";
import { ThreadSidebar } from "./thread-sidebar";
import type { ChatView, MessageRecord } from "./types";
import { buildChatTourSteps } from "./tour";
import type { ChatView, MessageRecord } from "./types";
type LoraCandidate = {
id: string;
@ -101,7 +109,9 @@ function messageHasImage(message: MessageRecord): boolean {
if (contentParts.some((part) => part.type === "image")) {
return true;
}
const attachments = Array.isArray(message.attachments) ? message.attachments : [];
const attachments = Array.isArray(message.attachments)
? message.attachments
: [];
for (const attachment of attachments) {
const parts = Array.isArray(attachment.content) ? attachment.content : [];
for (const part of parts as Array<{ type?: string }>) {
@ -152,12 +162,22 @@ const CompareContent = memo(function CompareContent({
pairId,
models,
loraModels,
}: { pairId: string; models: ModelOption[]; loraModels: LoraModelOption[] }): ReactElement {
}: {
pairId: string;
models: ModelOption[];
loraModels: LoraModelOption[];
}): ReactElement {
const isLoraCompare = useIsLoraCompare();
return isLoraCompare
? <LoraCompareContent pairId={pairId} />
: <GeneralCompareContent pairId={pairId} models={models} loraModels={loraModels} />;
return isLoraCompare ? (
<LoraCompareContent pairId={pairId} />
) : (
<GeneralCompareContent
pairId={pairId}
models={models}
loraModels={loraModels}
/>
);
});
/** Fast path: same model, adapter on/off, simultaneous generation. */
@ -179,7 +199,9 @@ const LoraCompareContent = memo(function LoraCompareContent({
setBaseThreadId(threads.find((t) => t.modelType === "base")?.id);
setLoraThreadId(threads.find((t) => t.modelType === "lora")?.id);
});
return () => { isActive = false; };
return () => {
isActive = false;
};
}, [pairId]);
return (
@ -196,7 +218,11 @@ const LoraCompareContent = memo(function LoraCompareContent({
</span>
</div>
<div className="min-h-0 flex-1">
<ChatRuntimeProvider modelType="base" pairId={pairId} initialThreadId={baseThreadId}>
<ChatRuntimeProvider
modelType="base"
pairId={pairId}
initialThreadId={baseThreadId}
>
<RegisterCompareHandle name="base" />
<Thread hideComposer={true} hideWelcome={true} />
</ChatRuntimeProvider>
@ -209,7 +235,11 @@ const LoraCompareContent = memo(function LoraCompareContent({
</span>
</div>
<div className="min-h-0 flex-1">
<ChatRuntimeProvider modelType="lora" pairId={pairId} initialThreadId={loraThreadId}>
<ChatRuntimeProvider
modelType="lora"
pairId={pairId}
initialThreadId={loraThreadId}
>
<RegisterCompareHandle name="lora" />
<Thread hideComposer={true} hideWelcome={true} />
</ChatRuntimeProvider>
@ -229,7 +259,11 @@ const GeneralCompareContent = memo(function GeneralCompareContent({
pairId,
models,
loraModels,
}: { pairId: string; models: ModelOption[]; loraModels: LoraModelOption[] }): ReactElement {
}: {
pairId: string;
models: ModelOption[];
loraModels: LoraModelOption[];
}): ReactElement {
const handlesRef = useRef<Record<string, CompareHandle>>({});
const [model1ThreadId, setModel1ThreadId] = useState<string>();
const [model2ThreadId, setModel2ThreadId] = useState<string>();
@ -241,7 +275,10 @@ const GeneralCompareContent = memo(function GeneralCompareContent({
isLora: false,
ggufVariant: globalGgufVariant ?? undefined,
});
const [model2, setModel2] = useState<CompareModelSelection>({ id: "", isLora: false });
const [model2, setModel2] = useState<CompareModelSelection>({
id: "",
isLora: false,
});
useEffect(() => {
let isActive = true;
@ -252,13 +289,19 @@ const GeneralCompareContent = memo(function GeneralCompareContent({
.then((threads) => {
if (!isActive) return;
setModel1ThreadId(
threads.find((t) => t.modelType === "model1" || t.modelType === "base")?.id,
threads.find(
(t) => t.modelType === "model1" || t.modelType === "base",
)?.id,
);
setModel2ThreadId(
threads.find((t) => t.modelType === "model2" || t.modelType === "lora")?.id,
threads.find(
(t) => t.modelType === "model2" || t.modelType === "lora",
)?.id,
);
});
return () => { isActive = false; };
return () => {
isActive = false;
};
}, [pairId]);
return (
@ -277,7 +320,13 @@ const GeneralCompareContent = memo(function GeneralCompareContent({
models={models}
loraModels={loraModels}
value={model1.id}
onValueChange={(id, meta) => setModel1({ id, isLora: meta.isLora, ggufVariant: meta.ggufVariant })}
onValueChange={(id, meta) =>
setModel1({
id,
isLora: meta.isLora,
ggufVariant: meta.ggufVariant,
})
}
variant="ghost"
size="sm"
className="max-w-[50%]"
@ -303,7 +352,13 @@ const GeneralCompareContent = memo(function GeneralCompareContent({
models={models}
loraModels={loraModels}
value={model2.id}
onValueChange={(id, meta) => setModel2({ id, isLora: meta.isLora, ggufVariant: meta.ggufVariant })}
onValueChange={(id, meta) =>
setModel2({
id,
isLora: meta.isLora,
ggufVariant: meta.ggufVariant,
})
}
variant="ghost"
size="sm"
className="max-w-[50%]"
@ -322,7 +377,11 @@ const GeneralCompareContent = memo(function GeneralCompareContent({
</div>
</div>
<div className="mx-auto w-full max-w-4xl px-4 py-4">
<SharedComposer handlesRef={handlesRef} model1={model1} model2={model2} />
<SharedComposer
handlesRef={handlesRef}
model1={model1}
model2={model2}
/>
</div>
</div>
</CompareHandlesProvider>
@ -364,8 +423,7 @@ function InlineSidebar({
data-sidebar="sidebar"
className={cn(
"bg-muted/70 text-sidebar-foreground h-full overflow-hidden rounded-2xl corner-squircle transition-[width] duration-200 ease-linear",
!collapsed &&
side === "right" && "border-l border-sidebar-border/70",
!collapsed && side === "right" && "border-l border-sidebar-border/70",
collapsed ? "w-0" : "w-(--sidebar-width)",
)}
>
@ -381,7 +439,11 @@ function TopBarActions({
onNewThread,
onNewCompare,
showCompare,
}: { onNewThread: () => void; onNewCompare: () => void; showCompare: boolean }) {
}: {
onNewThread: () => void;
onNewCompare: () => void;
showCompare: boolean;
}) {
const { state } = useSidebar();
if (state !== "collapsed") {
return null;
@ -424,8 +486,12 @@ export function ChatPage(): ReactElement {
);
const inferenceParams = useChatRuntimeStore((state) => state.params);
const setInferenceParams = useChatRuntimeStore((state) => state.setParams);
const activeGgufVariant = useChatRuntimeStore((state) => state.activeGgufVariant);
const ggufContextLength = useChatRuntimeStore((state) => state.ggufContextLength);
const activeGgufVariant = useChatRuntimeStore(
(state) => state.activeGgufVariant,
);
const ggufContextLength = useChatRuntimeStore(
(state) => state.ggufContextLength,
);
const contextUsage = useChatRuntimeStore((state) => state.contextUsage);
const autoTitle = useChatRuntimeStore((state) => state.autoTitle);
const setAutoTitle = useChatRuntimeStore((state) => state.setAutoTitle);
@ -441,8 +507,7 @@ export function ChatPage(): ReactElement {
loadingModel,
loadProgress,
loadToastDismissed,
} =
useChatModelRuntime();
} = useChatModelRuntime();
const refreshRef = useRef(refresh);
const selectModelRef = useRef(selectModel);
@ -455,11 +520,24 @@ export function ChatPage(): ReactElement {
}, [inferenceParams.checkpoint]);
const handleCheckpointChange = useCallback(
(value: string, meta?: { isLora: boolean; ggufVariant?: string; isDownloaded?: boolean; expectedBytes?: number }) => {
(
value: string,
meta?: {
isLora: boolean;
ggufVariant?: string;
isDownloaded?: boolean;
expectedBytes?: number;
},
) => {
const store = useChatRuntimeStore.getState();
const currentCheckpoint = store.params.checkpoint;
const currentVariant = store.activeGgufVariant;
if (!value || (value === currentCheckpoint && (meta?.ggufVariant ?? null) === (currentVariant ?? null))) return;
if (
!value ||
(value === currentCheckpoint &&
(meta?.ggufVariant ?? null) === (currentVariant ?? null))
)
return;
void (async () => {
let showImageCompatibilityWarning = false;
if (view.mode === "single" && activeThreadId) {
@ -471,7 +549,9 @@ export function ChatPage(): ReactElement {
.toArray();
if (messages.length > 0) {
const hasImage = messages.some(messageHasImage);
const targetModel = modelsFromStore.find((model) => model.id === value);
const targetModel = modelsFromStore.find(
(model) => model.id === value,
);
showImageCompatibilityWarning =
hasImage && targetModel?.isVision === false;
}
@ -499,20 +579,14 @@ export function ChatPage(): ReactElement {
const handleEject = useCallback(() => {
void ejectModel();
}, [ejectModel]);
const handleNewThread = useCallback(
() => {
useChatRuntimeStore.getState().setActiveThreadId(null);
setView({ mode: "single", newThreadNonce: crypto.randomUUID() });
},
[],
);
const handleNewCompare = useCallback(
() => {
setView({ mode: "compare", pairId: crypto.randomUUID() });
useChatRuntimeStore.getState().setContextUsage(null);
},
[],
);
const handleNewThread = useCallback(() => {
useChatRuntimeStore.getState().setActiveThreadId(null);
setView({ mode: "single", newThreadNonce: crypto.randomUUID() });
}, []);
const handleNewCompare = useCallback(() => {
setView({ mode: "compare", pairId: crypto.randomUUID() });
useChatRuntimeStore.getState().setContextUsage(null);
}, []);
const openModelSelector = useCallback(() => {
setModelSelectorLocked(true);
@ -556,18 +630,17 @@ export function ChatPage(): ReactElement {
.first()
.then((msg) => {
const saved = msg?.metadata as Record<string, unknown> | undefined;
const usage = saved?.contextUsage as typeof store.contextUsage | undefined;
const usage = saved?.contextUsage as
| typeof store.contextUsage
| undefined;
if (usage) store.setContextUsage(usage);
});
}
}, [viewBeforeCompare]);
const handleThreadSelect = useCallback(
(nextView: ChatView) => {
setView(nextView);
},
[],
);
const handleThreadSelect = useCallback((nextView: ChatView) => {
setView(nextView);
}, []);
const models = useMemo<ModelOption[]>(
() =>
@ -581,6 +654,37 @@ export function ChatPage(): ReactElement {
const [localModels, setLocalModels] = useState<LoraModelOption[]>([]);
const refreshLocalModels = useCallback(() => {
void listLocalModels()
.then((res) => {
setLocalModels(
res.models
.filter(
(m) =>
m.source === "lmstudio" ||
m.source === "models_dir" ||
m.source === "custom",
)
.map((m) => ({
id: m.id,
name:
m.source === "lmstudio" && m.model_id
? m.model_id
: m.display_name,
baseModel:
m.source === "lmstudio"
? "LM Studio"
: m.source === "custom"
? "Custom Folders"
: "Local models",
updatedAt: m.updated_at ?? undefined,
source: "local" as const,
})),
);
})
.catch(() => {});
}, []);
const loraModels = useMemo<LoraModelOption[]>(() => {
const fromLoras = lorasFromStore.map((lora) => ({
id: lora.id,
@ -596,20 +700,8 @@ export function ChatPage(): ReactElement {
useEffect(() => {
if (getTrainingCompareHandoff()) return;
void refresh();
void listLocalModels().then((res) => {
setLocalModels(
res.models
.filter((m) => m.source === "lmstudio" || m.source === "models_dir")
.map((m) => ({
id: m.id,
name: m.source === "lmstudio" && m.model_id ? m.model_id : m.display_name,
baseModel: m.source === "lmstudio" ? "LM Studio" : "Local models",
updatedAt: m.updated_at ?? undefined,
source: "local" as const,
})),
);
}).catch(() => {});
}, [refresh]);
refreshLocalModels();
}, [refresh, refreshLocalModels]);
useEffect(() => {
const handoff = getTrainingCompareHandoff();
@ -649,7 +741,10 @@ export function ChatPage(): ReactElement {
console.info("[chat-handoff] no lora match, loading base", {
id: handoff.baseModel,
});
await selectModelRef.current({ id: handoff.baseModel, isLora: false });
await selectModelRef.current({
id: handoff.baseModel,
isLora: false,
});
if (canceled) return;
} else {
console.warn("[chat-handoff] no lora/base match found", {
@ -767,9 +862,11 @@ export function ChatPage(): ReactElement {
? "Loading model…"
: "Downloading model…"
}
title={loadingModel.isDownloaded
? `Loading ${loadingModel.displayName} from cache.`
: `Loading ${loadingModel.displayName}. This may include downloading.`}
title={
loadingModel.isDownloaded
? `Loading ${loadingModel.displayName} from cache.`
: `Loading ${loadingModel.displayName}. This may include downloading.`
}
progressPercent={loadProgress?.percent}
progressLabel={loadProgress?.label}
onStop={cancelLoading}
@ -809,7 +906,12 @@ export function ChatPage(): ReactElement {
newThreadNonce={view.newThreadNonce}
/>
) : (
<CompareContent key={view.pairId} pairId={view.pairId} models={models} loraModels={loraModels} />
<CompareContent
key={view.pairId}
pairId={view.pairId}
models={models}
loraModels={loraModels}
/>
)}
</div>
@ -832,6 +934,7 @@ export function ChatPage(): ReactElement {
});
}
}}
onFoldersChange={refreshLocalModels}
/>
</SidebarProvider>
</div>

812
studio/frontend/src/features/chat/chat-settings-sheet.tsx
View file

@ -1,16 +1,6 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
import {
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue,
} from "@/components/ui/select";
import { Slider } from "@/components/ui/slider";
import { Textarea } from "@/components/ui/textarea";
import { Input } from "@/components/ui/input";
import { Button } from "@/components/ui/button";
import {
Dialog,
@ -20,11 +10,31 @@ import {
DialogHeader,
DialogTitle,
} from "@/components/ui/dialog";
import { Input } from "@/components/ui/input";
import {
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue,
} from "@/components/ui/select";
import {
Sheet,
SheetContent,
SheetDescription,
SheetHeader,
SheetTitle,
} from "@/components/ui/sheet";
import { Slider } from "@/components/ui/slider";
import { Switch } from "@/components/ui/switch";
import { Textarea } from "@/components/ui/textarea";
import { useIsMobile } from "@/hooks/use-mobile";
import {
ArrowDown01Icon,
CodeIcon,
Delete02Icon,
FloppyDiskIcon,
FolderSearchIcon,
PencilEdit01Icon,
Settings02Icon,
SlidersHorizontalIcon,
@ -33,22 +43,19 @@ import {
} from "@hugeicons/core-free-icons";
import { HugeiconsIcon } from "@hugeicons/react";
import { AnimatePresence, motion } from "motion/react";
import {
Sheet,
SheetContent,
SheetDescription,
SheetHeader,
SheetTitle,
} from "@/components/ui/sheet";
import { useIsMobile } from "@/hooks/use-mobile";
import type { ReactNode } from "react";
import { useEffect, useMemo, useState } from "react";
import { useCallback, useEffect, useMemo, useState } from "react";
import {
type ScanFolderInfo,
addScanFolder,
listScanFolders,
removeScanFolder,
} from "./api/chat-api";
import { useChatRuntimeStore } from "./stores/chat-runtime-store";
import {
DEFAULT_INFERENCE_PARAMS,
type InferenceParams,
} from "./types/runtime";
import { useChatRuntimeStore } from "./stores/chat-runtime-store";
import { Switch } from "@/components/ui/switch";
export const defaultInferenceParams = DEFAULT_INFERENCE_PARAMS;
export type { InferenceParams } from "./types/runtime";
@ -174,7 +181,11 @@ function loadCollapsibleState(): Record<string, boolean> {
const raw = localStorage.getItem(COLLAPSIBLE_STATE_KEY);
if (!raw) return {};
const parsed = JSON.parse(raw);
if (typeof parsed !== "object" || parsed === null || Array.isArray(parsed)) {
if (
typeof parsed !== "object" ||
parsed === null ||
Array.isArray(parsed)
) {
return {};
}
return Object.fromEntries(
@ -255,6 +266,108 @@ function CollapsibleSection({
);
}
function ModelFoldersSection({
onFoldersChange,
}: { onFoldersChange?: () => void }) {
const [folders, setFolders] = useState<ScanFolderInfo[]>([]);
const [input, setInput] = useState("");
const [error, setError] = useState<string | null>(null);
const [loading, setLoading] = useState(false);
const refresh = useCallback(() => {
listScanFolders()
.then(setFolders)
.catch(() => {});
}, []);
useEffect(() => {
refresh();
}, [refresh]);
const handleAdd = async () => {
const trimmed = input.trim();
if (!trimmed) return;
setError(null);
setLoading(true);
try {
await addScanFolder(trimmed);
setInput("");
refresh();
onFoldersChange?.();
} catch (e) {
setError(e instanceof Error ? e.message : "Failed to add folder");
} finally {
setLoading(false);
}
};
const handleRemove = async (id: number) => {
try {
await removeScanFolder(id);
onFoldersChange?.();
} catch (e) {
setError(e instanceof Error ? e.message : "Failed to remove folder");
} finally {
refresh();
}
};
return (
<CollapsibleSection icon={FolderSearchIcon} label="Model Folders">
<div className="flex flex-col gap-2 py-1">
{folders.length > 0 && (
<div className="flex flex-col gap-1">
{folders.map((f) => (
<div
key={f.id}
className="group flex items-center gap-1.5 rounded-md px-1.5 py-1 text-xs transition-colors hover:bg-accent"
>
<span
className="min-w-0 flex-1 truncate text-muted-foreground"
title={f.path}
>
{f.path}
</span>
<button
type="button"
onClick={() => handleRemove(f.id)}
className="shrink-0 rounded p-0.5 text-muted-foreground/50 opacity-0 transition-opacity group-hover:opacity-100 hover:text-destructive"
>
<HugeiconsIcon icon={Delete02Icon} className="size-3" />
</button>
</div>
))}
</div>
)}
<div className="flex gap-1.5">
<Input
value={input}
onChange={(e) => {
setInput(e.target.value);
setError(null);
}}
onKeyDown={(e) => {
if (e.key === "Enter") handleAdd();
}}
placeholder="/path/to/models"
className="h-7 flex-1 text-xs font-mono"
disabled={loading}
/>
<button
type="button"
onClick={handleAdd}
disabled={loading || !input.trim()}
className="h-7 rounded-md border px-2 text-[11px] font-medium text-muted-foreground transition-colors hover:bg-accent disabled:cursor-not-allowed disabled:opacity-50"
>
Add
</button>
</div>
{error && <p className="text-[11px] text-destructive">{error}</p>}
</div>
</CollapsibleSection>
);
}
interface ChatSettingsPanelProps {
open: boolean;
onOpenChange?: (open: boolean) => void;
@ -263,6 +376,7 @@ interface ChatSettingsPanelProps {
autoTitle: boolean;
onAutoTitleChange: (enabled: boolean) => void;
onReloadModel?: () => void;
onFoldersChange?: () => void;
}
export function ChatSettingsPanel({
@ -273,16 +387,21 @@ export function ChatSettingsPanel({
autoTitle,
onAutoTitleChange,
onReloadModel,
onFoldersChange,
}: ChatSettingsPanelProps) {
const isMobile = useIsMobile();
const isGguf = useChatRuntimeStore((s) => s.activeGgufVariant) != null;
const ggufContextLength = useChatRuntimeStore((s) => s.ggufContextLength);
const ggufMaxContextLength = useChatRuntimeStore((s) => s.ggufMaxContextLength);
const ggufMaxContextLength = useChatRuntimeStore(
(s) => s.ggufMaxContextLength,
);
const kvCacheDtype = useChatRuntimeStore((s) => s.kvCacheDtype);
const setKvCacheDtype = useChatRuntimeStore((s) => s.setKvCacheDtype);
const loadedKvCacheDtype = useChatRuntimeStore((s) => s.loadedKvCacheDtype);
const customContextLength = useChatRuntimeStore((s) => s.customContextLength);
const setCustomContextLength = useChatRuntimeStore((s) => s.setCustomContextLength);
const setCustomContextLength = useChatRuntimeStore(
(s) => s.setCustomContextLength,
);
const ctxDisplayValue = customContextLength ?? ggufContextLength ?? "";
const ctxMaxValue = ggufMaxContextLength ?? ggufContextLength ?? null;
@ -292,7 +411,9 @@ export function ChatSettingsPanel({
const [customPresets, setCustomPresets] = useState<Preset[]>(() =>
loadSavedCustomPresets(),
);
const [activePreset, setActivePreset] = useState(() => loadSavedActivePreset());
const [activePreset, setActivePreset] = useState(() =>
loadSavedActivePreset(),
);
const [savePresetOpen, setSavePresetOpen] = useState(false);
const [presetNameDraft, setPresetNameDraft] = useState("");
const presets = useMemo(
@ -417,325 +538,356 @@ export function ChatSettingsPanel({
<div className="flex-1 overflow-y-auto px-1.5">
{/* mt-4 matches the Playground sidebar gap (SidebarHeader py-3 + SidebarGroup pt-1) */}
<div className="mt-4 px-2 pb-3">
<div className="flex items-center gap-2">
<Select value={activePreset} onValueChange={applyPreset}>
<SelectTrigger className="h-8 flex-1 corner-squircle text-xs">
<SelectValue />
</SelectTrigger>
<SelectContent>
{presets.map((p) => (
<SelectItem key={p.name} value={p.name}>
{p.name}
</SelectItem>
))}
</SelectContent>
</Select>
<button
type="button"
onClick={openSavePresetDialog}
className="flex h-8 items-center gap-1.5 rounded-md border px-2.5 text-xs text-muted-foreground transition-colors hover:bg-accent"
title="Save preset"
>
<HugeiconsIcon icon={FloppyDiskIcon} className="size-3.5" />
Save
</button>
<button
type="button"
onClick={() => deletePreset(activePreset)}
disabled={isBuiltinPreset}
className="flex h-8 items-center gap-1.5 rounded-md border px-2.5 text-xs text-muted-foreground transition-colors hover:bg-accent disabled:cursor-not-allowed disabled:opacity-50"
title={
isBuiltinPreset
? "Built-in presets cannot be deleted"
: "Delete selected preset"
}
>
<HugeiconsIcon icon={Delete02Icon} className="size-3.5" />
Delete
</button>
</div>
</div>
<div className="px-2 pb-4">
<label
htmlFor="system-prompt"
className="mb-1.5 block text-xs font-medium"
<div className="flex items-center gap-2">
<Select value={activePreset} onValueChange={applyPreset}>
<SelectTrigger className="h-8 flex-1 corner-squircle text-xs">
<SelectValue />
</SelectTrigger>
<SelectContent>
{presets.map((p) => (
<SelectItem key={p.name} value={p.name}>
{p.name}
</SelectItem>
))}
</SelectContent>
</Select>
<button
type="button"
onClick={openSavePresetDialog}
className="flex h-8 items-center gap-1.5 rounded-md border px-2.5 text-xs text-muted-foreground transition-colors hover:bg-accent"
title="Save preset"
>
System Prompt
</label>
<Textarea
id="system-prompt"
value={params.systemPrompt}
onChange={(e) => set("systemPrompt")(e.target.value)}
placeholder="You are a helpful assistant..."
className="min-h-20 text-xs corner-squircle"
rows={3}
/>
<HugeiconsIcon icon={FloppyDiskIcon} className="size-3.5" />
Save
</button>
<button
type="button"
onClick={() => deletePreset(activePreset)}
disabled={isBuiltinPreset}
className="flex h-8 items-center gap-1.5 rounded-md border px-2.5 text-xs text-muted-foreground transition-colors hover:bg-accent disabled:cursor-not-allowed disabled:opacity-50"
title={
isBuiltinPreset
? "Built-in presets cannot be deleted"
: "Delete selected preset"
}
>
<HugeiconsIcon icon={Delete02Icon} className="size-3.5" />
Delete
</button>
</div>
</div>
<CollapsibleSection icon={Settings02Icon} label="Model" defaultOpen={true}>
<div className="flex flex-col gap-3 py-1">
{isGguf && (
<>
<div className="space-y-2">
<div className="flex items-center justify-between">
<span className="text-xs font-medium">Context Length</span>
<Input
type="number"
value={typeof ctxDisplayValue === "number" ? ctxDisplayValue : (ggufContextLength ?? "")}
placeholder="..."
min={128}
max={ctxMaxValue ?? undefined}
step={1024}
className="h-6 w-[100px] text-right text-xs tabular-nums"
onChange={(e) => {
const raw = e.target.value;
if (raw === "") {
setCustomContextLength(null);
return;
}
const v = parseInt(raw, 10);
if (!Number.isNaN(v) && v >= 0) {
const maxCtx = ctxMaxValue ?? Infinity;
const clamped = Math.min(v, maxCtx);
setCustomContextLength(clamped === (ggufContextLength ?? 0) ? null : clamped);
}
}}
/>
</div>
<Slider
min={1024}
max={ctxMaxValue ?? 4096}
<div className="px-2 pb-4">
<label
htmlFor="system-prompt"
className="mb-1.5 block text-xs font-medium"
>
System Prompt
</label>
<Textarea
id="system-prompt"
value={params.systemPrompt}
onChange={(e) => set("systemPrompt")(e.target.value)}
placeholder="You are a helpful assistant..."
className="min-h-20 text-xs corner-squircle"
rows={3}
/>
</div>
<CollapsibleSection
icon={Settings02Icon}
label="Model"
defaultOpen={true}
>
<div className="flex flex-col gap-3 py-1">
{isGguf && (
<>
<div className="space-y-2">
<div className="flex items-center justify-between">
<span className="text-xs font-medium">Context Length</span>
<Input
type="number"
value={
typeof ctxDisplayValue === "number"
? ctxDisplayValue
: (ggufContextLength ?? "")
}
placeholder="..."
min={128}
max={ctxMaxValue ?? undefined}
step={1024}
value={[Math.min(typeof ctxDisplayValue === "number" ? ctxDisplayValue : (ggufContextLength ?? 4096), ctxMaxValue ?? 4096)]}
onValueChange={([v]) => {
setCustomContextLength(v === (ggufContextLength ?? 0) ? null : v);
className="h-6 w-[100px] text-right text-xs tabular-nums"
onChange={(e) => {
const raw = e.target.value;
if (raw === "") {
setCustomContextLength(null);
return;
}
const v = Number.parseInt(raw, 10);
if (!Number.isNaN(v) && v >= 0) {
const maxCtx =
ctxMaxValue ?? Number.POSITIVE_INFINITY;
const clamped = Math.min(v, maxCtx);
setCustomContextLength(
clamped === (ggufContextLength ?? 0)
? null
: clamped,
);
}
}}
/>
</div>
<div className="flex items-center justify-between gap-3">
<div className="min-w-0">
<div className="text-xs font-medium">KV Cache Dtype</div>
<div className="text-[11px] text-muted-foreground">
Quantize KV cache to reduce VRAM.
</div>
</div>
<Select
value={kvCacheDtype ?? "f16"}
onValueChange={(v) => {
setKvCacheDtype(v === "f16" ? null : v);
}}
>
<SelectTrigger className="h-7 w-[90px] text-xs">
<SelectValue />
</SelectTrigger>
<SelectContent>
<SelectItem value="f16">f16</SelectItem>
<SelectItem value="bf16">bf16</SelectItem>
<SelectItem value="q8_0">q8_0</SelectItem>
<SelectItem value="q5_1">q5_1</SelectItem>
<SelectItem value="q4_1">q4_1</SelectItem>
</SelectContent>
</Select>
</div>
{modelSettingsDirty && (
<div className="flex flex-wrap gap-1.5 pt-1">
<button
type="button"
onClick={() => onReloadModel?.()}
className="rounded-md bg-primary px-2.5 py-1 text-[11px] font-medium text-primary-foreground transition-colors hover:bg-primary/90"
>
Apply
</button>
<button
type="button"
onClick={() => {
setCustomContextLength(null);
setKvCacheDtype(loadedKvCacheDtype);
}}
className="rounded-md border px-2.5 py-1 text-[11px] font-medium text-muted-foreground transition-colors hover:bg-accent"
>
Reset
</button>
</div>
)}
</>
)}
{!isGguf && params.checkpoint && (
<div className="flex items-center justify-between gap-3">
<div className="min-w-0">
<div className="text-xs font-medium">Enable custom code</div>
<div className="text-[11px] text-muted-foreground">
Allow models with custom code (e.g. Nemotron). Only enable if sure.
</div>
</div>
<Switch
checked={params.trustRemoteCode ?? false}
onCheckedChange={set("trustRemoteCode")}
<Slider
min={1024}
max={ctxMaxValue ?? 4096}
step={1024}
value={[
Math.min(
typeof ctxDisplayValue === "number"
? ctxDisplayValue
: (ggufContextLength ?? 4096),
ctxMaxValue ?? 4096,
),
]}
onValueChange={([v]) => {
setCustomContextLength(
v === (ggufContextLength ?? 0) ? null : v,
);
}}
/>
</div>
)}
</div>
</CollapsibleSection>
<CollapsibleSection
icon={SlidersHorizontalIcon}
label="Sampling"
defaultOpen={true}
>
<div className="flex flex-col gap-5">
<ParamSlider
label="Temperature"
value={params.temperature}
min={0}
max={2}
step={0.1}
onChange={set("temperature")}
/>
<ParamSlider
label="Top P"
value={params.topP}
min={0}
max={1}
step={0.05}
onChange={set("topP")}
displayValue={params.topP === 1 ? "Off" : undefined}
/>
<ParamSlider
label="Top K"
value={params.topK}
min={0}
max={100}
step={1}
onChange={set("topK")}
displayValue={params.topK === 0 ? "Off" : undefined}
/>
<ParamSlider
label="Min P"
value={params.minP}
min={0}
max={1}
step={0.01}
onChange={set("minP")}
/>
<ParamSlider
label="Repetition Penalty"
value={params.repetitionPenalty}
min={1}
max={2}
step={0.05}
onChange={set("repetitionPenalty")}
displayValue={params.repetitionPenalty === 1 ? "Off" : undefined}
/>
<ParamSlider
label="Presence Penalty"
value={params.presencePenalty}
min={0}
max={2}
step={0.1}
onChange={set("presencePenalty")}
displayValue={params.presencePenalty === 0 ? "Off" : undefined}
/>
{!isGguf && (
<ParamSlider
label="Max Seq Length"
value={params.maxSeqLength}
min={128}
max={32768}
step={128}
onChange={set("maxSeqLength")}
/>
)}
<ParamSlider
label="Max Tokens"
value={params.maxTokens}
min={64}
max={isGguf && ggufContextLength ? ggufContextLength : 32768}
step={64}
onChange={set("maxTokens")}
displayValue={
isGguf && ggufContextLength && params.maxTokens >= ggufContextLength
? "Max"
: undefined
}
/>
</div>
</CollapsibleSection>
<CollapsibleSection icon={Wrench01Icon} label="Tools">
<div className="flex flex-col gap-3 py-1">
<AutoHealToolCallsToggle />
<MaxToolCallsSlider />
<ToolCallTimeoutSlider />
</div>
</CollapsibleSection>
<CollapsibleSection icon={UserSettings01Icon} label="Preferences" defaultOpen={true}>
<div className="flex flex-col gap-3 py-1">
<div className="flex items-center justify-between gap-3">
<div className="min-w-0">
<div className="text-xs font-medium">KV Cache Dtype</div>
<div className="text-[11px] text-muted-foreground">
Quantize KV cache to reduce VRAM.
</div>
</div>
<Select
value={kvCacheDtype ?? "f16"}
onValueChange={(v) => {
setKvCacheDtype(v === "f16" ? null : v);
}}
>
<SelectTrigger className="h-7 w-[90px] text-xs">
<SelectValue />
</SelectTrigger>
<SelectContent>
<SelectItem value="f16">f16</SelectItem>
<SelectItem value="bf16">bf16</SelectItem>
<SelectItem value="q8_0">q8_0</SelectItem>
<SelectItem value="q5_1">q5_1</SelectItem>
<SelectItem value="q4_1">q4_1</SelectItem>
</SelectContent>
</Select>
</div>
{modelSettingsDirty && (
<div className="flex flex-wrap gap-1.5 pt-1">
<button
type="button"
onClick={() => onReloadModel?.()}
className="rounded-md bg-primary px-2.5 py-1 text-[11px] font-medium text-primary-foreground transition-colors hover:bg-primary/90"
>
Apply
</button>
<button
type="button"
onClick={() => {
setCustomContextLength(null);
setKvCacheDtype(loadedKvCacheDtype);
}}
className="rounded-md border px-2.5 py-1 text-[11px] font-medium text-muted-foreground transition-colors hover:bg-accent"
>
Reset
</button>
</div>
)}
</>
)}
{!isGguf && params.checkpoint && (
<div className="flex items-center justify-between gap-3">
<div className="min-w-0">
<div className="text-xs font-medium">Auto title</div>
<div className="text-xs font-medium">Enable custom code</div>
<div className="text-[11px] text-muted-foreground">
Generate short title after reply.
Allow models with custom code (e.g. Nemotron). Only enable
if sure.
</div>
</div>
<Switch
checked={autoTitle}
onCheckedChange={onAutoTitleChange}
checked={params.trustRemoteCode ?? false}
onCheckedChange={set("trustRemoteCode")}
/>
</div>
<HfTokenField />
</div>
</CollapsibleSection>
)}
</div>
</CollapsibleSection>
<ChatTemplateSection onReloadModel={onReloadModel} />
</div>
<Dialog
open={savePresetOpen}
onOpenChange={(nextOpen) => {
setSavePresetOpen(nextOpen);
if (!nextOpen) {
setPresetNameDraft("");
}
}}
<CollapsibleSection
icon={SlidersHorizontalIcon}
label="Sampling"
defaultOpen={true}
>
<DialogContent className="corner-squircle sm:max-w-sm">
<DialogHeader>
<DialogTitle>Save Preset</DialogTitle>
<DialogDescription>
Enter a name for this inference preset.
</DialogDescription>
</DialogHeader>
<form
onSubmit={(event) => {
event.preventDefault();
savePresetWithName(presetNameDraft);
}}
className="space-y-4"
>
<Input
autoFocus={true}
value={presetNameDraft}
onChange={(event) => setPresetNameDraft(event.target.value)}
placeholder="Preset name"
maxLength={80}
<div className="flex flex-col gap-5">
<ParamSlider
label="Temperature"
value={params.temperature}
min={0}
max={2}
step={0.1}
onChange={set("temperature")}
/>
<ParamSlider
label="Top P"
value={params.topP}
min={0}
max={1}
step={0.05}
onChange={set("topP")}
displayValue={params.topP === 1 ? "Off" : undefined}
/>
<ParamSlider
label="Top K"
value={params.topK}
min={0}
max={100}
step={1}
onChange={set("topK")}
displayValue={params.topK === 0 ? "Off" : undefined}
/>
<ParamSlider
label="Min P"
value={params.minP}
min={0}
max={1}
step={0.01}
onChange={set("minP")}
/>
<ParamSlider
label="Repetition Penalty"
value={params.repetitionPenalty}
min={1}
max={2}
step={0.05}
onChange={set("repetitionPenalty")}
displayValue={params.repetitionPenalty === 1 ? "Off" : undefined}
/>
<ParamSlider
label="Presence Penalty"
value={params.presencePenalty}
min={0}
max={2}
step={0.1}
onChange={set("presencePenalty")}
displayValue={params.presencePenalty === 0 ? "Off" : undefined}
/>
{!isGguf && (
<ParamSlider
label="Max Seq Length"
value={params.maxSeqLength}
min={128}
max={32768}
step={128}
onChange={set("maxSeqLength")}
/>
<DialogFooter>
<Button
type="button"
variant="outline"
onClick={() => setSavePresetOpen(false)}
>
Cancel
</Button>
<Button type="submit" disabled={presetNameDraft.trim().length === 0}>
Save
</Button>
</DialogFooter>
</form>
</DialogContent>
</Dialog>
</>
)}
<ParamSlider
label="Max Tokens"
value={params.maxTokens}
min={64}
max={isGguf && ggufContextLength ? ggufContextLength : 32768}
step={64}
onChange={set("maxTokens")}
displayValue={
isGguf &&
ggufContextLength &&
params.maxTokens >= ggufContextLength
? "Max"
: undefined
}
/>
</div>
</CollapsibleSection>
<CollapsibleSection icon={Wrench01Icon} label="Tools">
<div className="flex flex-col gap-3 py-1">
<AutoHealToolCallsToggle />
<MaxToolCallsSlider />
<ToolCallTimeoutSlider />
</div>
</CollapsibleSection>
<CollapsibleSection
icon={UserSettings01Icon}
label="Preferences"
defaultOpen={true}
>
<div className="flex flex-col gap-3 py-1">
<div className="flex items-center justify-between gap-3">
<div className="min-w-0">
<div className="text-xs font-medium">Auto title</div>
<div className="text-[11px] text-muted-foreground">
Generate short title after reply.
</div>
</div>
<Switch checked={autoTitle} onCheckedChange={onAutoTitleChange} />
</div>
<HfTokenField />
</div>
</CollapsibleSection>
<ModelFoldersSection onFoldersChange={onFoldersChange} />
<ChatTemplateSection onReloadModel={onReloadModel} />
</div>
<Dialog
open={savePresetOpen}
onOpenChange={(nextOpen) => {
setSavePresetOpen(nextOpen);
if (!nextOpen) {
setPresetNameDraft("");
}
}}
>
<DialogContent className="corner-squircle sm:max-w-sm">
<DialogHeader>
<DialogTitle>Save Preset</DialogTitle>
<DialogDescription>
Enter a name for this inference preset.
</DialogDescription>
</DialogHeader>
<form
onSubmit={(event) => {
event.preventDefault();
savePresetWithName(presetNameDraft);
}}
className="space-y-4"
>
<Input
autoFocus={true}
value={presetNameDraft}
onChange={(event) => setPresetNameDraft(event.target.value)}
placeholder="Preset name"
maxLength={80}
/>
<DialogFooter>
<Button
type="button"
variant="outline"
onClick={() => setSavePresetOpen(false)}
>
Cancel
</Button>
<Button
type="submit"
disabled={presetNameDraft.trim().length === 0}
>
Save
</Button>
</DialogFooter>
</form>
</DialogContent>
</Dialog>
</>
);
if (isMobile) {
@ -763,7 +915,9 @@ export function ChatSettingsPanel({
function MaxToolCallsSlider() {
const maxToolCalls = useChatRuntimeStore((s) => s.maxToolCallsPerMessage);
const setMaxToolCalls = useChatRuntimeStore((s) => s.setMaxToolCallsPerMessage);
const setMaxToolCalls = useChatRuntimeStore(
(s) => s.setMaxToolCallsPerMessage,
);
// Slider range 0-41; 41 maps to 9999 ("Max")
const sliderValue = maxToolCalls >= 9999 ? 41 : Math.min(maxToolCalls, 40);
@ -776,7 +930,9 @@ function MaxToolCallsSlider() {
max={41}
step={1}
onChange={(v) => setMaxToolCalls(v >= 41 ? 9999 : v)}
displayValue={sliderValue >= 41 ? "Max" : sliderValue === 0 ? "Off" : undefined}
displayValue={
sliderValue >= 41 ? "Max" : sliderValue === 0 ? "Off" : undefined
}
/>
);
}
@ -810,7 +966,9 @@ function ToolCallTimeoutSlider() {
function AutoHealToolCallsToggle() {
const autoHealToolCalls = useChatRuntimeStore((s) => s.autoHealToolCalls);
const setAutoHealToolCalls = useChatRuntimeStore((s) => s.setAutoHealToolCalls);
const setAutoHealToolCalls = useChatRuntimeStore(
(s) => s.setAutoHealToolCalls,
);
return (
<div className="flex items-center justify-between gap-3">

4
studio/frontend/src/features/export/export-page.tsx
View file

@ -909,7 +909,9 @@ export function ExportPage() {
const source =
model?.source === "hf_cache"
? "HF cache"
: "Local dir";
: model?.source === "custom"
? "Custom Folders"
: "Local dir";
return (
<ComboboxItem key={id} value={id} className="gap-2">
<span className="block min-w-0 flex-1 truncate">

696
studio/frontend/src/features/studio/sections/model-section.tsx
View file

@ -28,7 +28,16 @@ import {
TooltipContent,
TooltipTrigger,
} from "@/components/ui/tooltip";
import { MODEL_TYPE_TO_HF_TASK, PRIORITY_TRAINING_MODELS, applyPriorityOrdering } from "@/config/training";
import {
MODEL_TYPE_TO_HF_TASK,
PRIORITY_TRAINING_MODELS,
applyPriorityOrdering,
} from "@/config/training";
import {
type LocalModelInfo,
listLocalModels,
useTrainingConfigStore,
} from "@/features/training";
import {
useDebouncedValue,
useGpuInfo,
@ -38,15 +47,10 @@ import {
} from "@/hooks";
import { formatCompact } from "@/lib/utils";
import {
type TrainingMethod as VramTrainingMethod,
type VramFitStatus,
type TrainingMethod as VramTrainingMethod,
buildModelVramMap,
} from "@/lib/vram";
import {
listLocalModels,
type LocalModelInfo,
useTrainingConfigStore,
} from "@/features/training";
import type { TrainingMethod } from "@/types/training";
import {
ChipIcon,
@ -150,7 +154,9 @@ export function ModelSection() {
.catch((error) => {
if (controller.signal.aborted) return;
setLocalModelsError(
error instanceof Error ? error.message : "Failed to load local models",
error instanceof Error
? error.message
: "Failed to load local models",
);
})
.finally(() => {
@ -241,7 +247,9 @@ export function ModelSection() {
{ est: number; status: VramFitStatus | null; detail: string | null }
>();
for (const r of hfResults) {
const detail = r.totalParams ? formatCompact(r.totalParams) : extractParamLabel(r.id);
const detail = r.totalParams
? formatCompact(r.totalParams)
: extractParamLabel(r.id);
const fit = fitMap.get(r.id);
map.set(r.id, {
est: fit?.est ?? 0,
@ -271,363 +279,383 @@ export function ModelSection() {
className="shadow-border ring-border"
>
<div className="grid min-w-0 gap-4 md:grid-cols-2 xl:grid-cols-4">
<div data-tour="studio-local-model" className="flex min-w-0 flex-col gap-2">
<div
data-tour="studio-local-model"
className="flex min-w-0 flex-col gap-2"
>
<span className="flex items-center gap-1.5 text-xs font-medium text-muted-foreground">
Local Model
<Tooltip>
<TooltipTrigger asChild={true}>
<button
type="button"
className="text-foreground/70 hover:text-foreground"
>
<HugeiconsIcon
icon={InformationCircleIcon}
className="size-3"
/>
</button>
</TooltipTrigger>
<TooltipContent>
Path to a locally downloaded model or a custom HF repo.
</TooltipContent>
</Tooltip>
</span>
<div ref={localComboboxAnchorRef} className="min-w-0">
<Combobox
items={localResultIds}
filteredItems={localFilteredIds}
filter={null}
value={localModelInput || null}
onValueChange={(id) => {
const next = id ?? "";
setLocalModelInput(next);
if (next) setSelectedModel(next);
}}
onInputValueChange={setLocalModelInput}
itemToStringValue={(id) => id}
autoHighlight={true}
>
<ComboboxInput
placeholder={
isLoadingLocalModels
? "Scanning local and cached models..."
: "./models/my-model"
}
className="w-full bg-foreground text-background [&_input]:text-background [&_input]:placeholder:text-background/40 [&_svg]:text-background/50 hover:bg-foreground/90"
onBlur={() => applyLocalModel(localModelInput)}
onKeyDown={(event) => {
if (event.key !== "Enter") return;
event.preventDefault();
applyLocalModel(localModelInput);
<Tooltip>
<TooltipTrigger asChild={true}>
<button
type="button"
className="text-foreground/70 hover:text-foreground"
>
<HugeiconsIcon
icon={InformationCircleIcon}
className="size-3"
/>
</button>
</TooltipTrigger>
<TooltipContent>
Path to a locally downloaded model or a custom HF repo.
</TooltipContent>
</Tooltip>
</span>
<div ref={localComboboxAnchorRef} className="min-w-0">
<Combobox
items={localResultIds}
filteredItems={localFilteredIds}
filter={null}
value={localModelInput || null}
onValueChange={(id) => {
const next = id ?? "";
setLocalModelInput(next);
if (next) setSelectedModel(next);
}}
onInputValueChange={setLocalModelInput}
itemToStringValue={(id) => id}
autoHighlight={true}
>
<InputGroupAddon>
<HugeiconsIcon icon={FolderSearchIcon} className="size-4" />
</InputGroupAddon>
</ComboboxInput>
<ComboboxContent
anchor={localComboboxAnchorRef}
className={DARK_COMBOBOX_CONTENT}
>
{isLoadingLocalModels ? (
<div className="flex items-center justify-center gap-2 py-4 text-xs text-muted-foreground">
<Spinner className="size-4" /> Scanning...
</div>
) : localModelsError ? (
<div className="px-3 py-2 text-xs text-red-500">
{localModelsError}
</div>
) : (
<ComboboxEmpty>No local models found</ComboboxEmpty>
)}
<ComboboxList className="p-1">
{(id: string) => {
const model = localMetaById.get(id);
const source =
model?.source === "hf_cache"
? "HF cache"
: model?.source === "lmstudio"
? "LM Studio"
: "Local dir";
return (
<ComboboxItem key={id} value={id} className="gap-2">
<Tooltip>
<TooltipTrigger asChild={true}>
<span className="block min-w-0 flex-1 truncate">
{model?.display_name ?? id}
</span>
</TooltipTrigger>
<TooltipContent side="left" className="max-w-xs break-all">
{model?.path ?? id}
</TooltipContent>
</Tooltip>
<span className="ml-auto shrink-0 text-[10px] text-muted-foreground">
{source}
</span>
</ComboboxItem>
);
<ComboboxInput
placeholder={
isLoadingLocalModels
? "Scanning local and cached models..."
: "./models/my-model"
}
className="w-full bg-foreground text-background [&_input]:text-background [&_input]:placeholder:text-background/40 [&_svg]:text-background/50 hover:bg-foreground/90"
onBlur={() => applyLocalModel(localModelInput)}
onKeyDown={(event) => {
if (event.key !== "Enter") return;
event.preventDefault();
applyLocalModel(localModelInput);
}}
</ComboboxList>
</ComboboxContent>
</Combobox>
</div>
{isLoadingLocalModels ? (
<p className="text-[10px] text-muted-foreground">Scanning local models...</p>
) : localModelsError ? (
<p className="text-[10px] text-red-500">{localModelsError}</p>
) : (
<p className="text-[10px] text-muted-foreground">
{trainableLocalModels.length > 0
? `${trainableLocalModels.length} local/cached models found`
: "No local models found. Enter path manually."}
</p>
)}
</div>
<div data-tour="studio-base-model" className="flex min-w-0 flex-col gap-2">
<span className="flex items-center gap-1.5 text-xs font-medium text-muted-foreground">
Hugging Face Model
<Tooltip>
<TooltipTrigger asChild={true}>
<button
type="button"
className="text-foreground/70 hover:text-foreground"
>
<HugeiconsIcon
icon={InformationCircleIcon}
className="size-3"
/>
</button>
</TooltipTrigger>
<TooltipContent>
Search Hugging Face models or pick from our recommended list.{" "}
<a
href="https://unsloth.ai/docs/get-started/fine-tuning-llms-guide/what-model-should-i-use"
target="_blank"
rel="noopener noreferrer"
className="text-primary underline"
<InputGroupAddon>
<HugeiconsIcon icon={FolderSearchIcon} className="size-4" />
</InputGroupAddon>
</ComboboxInput>
<ComboboxContent
anchor={localComboboxAnchorRef}
className={DARK_COMBOBOX_CONTENT}
>
Read more
</a>
</TooltipContent>
</Tooltip>
</span>
<div
ref={comboboxAnchorRef}
className="min-w-0"
onKeyDown={(event) => {
if (event.key !== "Enter") return;
if (!(event.target instanceof HTMLInputElement)) return;
event.preventDefault();
if (hfResults.length > 0) {
handleModelSelect(hfResults[0].id);
} else {
const text = event.target.value.trim();
if (text) handleModelSelect(text);
}
}}
>
<Combobox
items={resultIds}
filteredItems={resultIds}
filter={null}
value={selectedModel}
onValueChange={handleModelSelect}
onInputValueChange={handleInputChange}
itemToStringValue={(id) => id}
autoHighlight={true}
>
<ComboboxInput
placeholder="Search models..."
className="w-full leading-5"
>
<InputGroupAddon>
<HugeiconsIcon icon={Search01Icon} className="size-4" />
</InputGroupAddon>
</ComboboxInput>
<ComboboxContent anchor={comboboxAnchorRef}>
{isLoading ? (
<div className="flex items-center justify-center py-4 gap-2 text-xs text-muted-foreground">
<Spinner className="size-4" /> Searching…
</div>
) : (
<ComboboxEmpty>No models found</ComboboxEmpty>
)}
<div
ref={scrollRef}
className="max-h-64 overflow-y-auto overscroll-contain [scrollbar-width:thin]"
>
<ComboboxList className="p-1 !max-h-none !overflow-visible">
{isLoadingLocalModels ? (
<div className="flex items-center justify-center gap-2 py-4 text-xs text-muted-foreground">
<Spinner className="size-4" /> Scanning...
</div>
) : localModelsError ? (
<div className="px-3 py-2 text-xs text-red-500">
{localModelsError}
</div>
) : (
<ComboboxEmpty>No local models found</ComboboxEmpty>
)}
<ComboboxList className="p-1">
{(id: string) => {
const entry = vramMap.get(id);
const detail = entry?.detail ?? null;
const fitStatus = entry?.status ?? null;
const vramEst = entry?.est ?? null;
const exceeds = fitStatus === "exceeds";
const model = localMetaById.get(id);
const source =
model?.source === "hf_cache"
? "HF cache"
: model?.source === "lmstudio"
? "LM Studio"
: model?.source === "custom"
? "Custom Folders"
: "Local dir";
return (
<ComboboxItem
key={id}
value={id}
className={`gap-2 ${exceeds ? "opacity-50" : ""}`}
>
<ComboboxItem key={id} value={id} className="gap-2">
<Tooltip>
<TooltipTrigger asChild={true}>
<span className={`block min-w-0 flex-1 truncate ${exceeds ? "line-through decoration-muted-foreground/50" : ""}`}>
{id}
<span className="block min-w-0 flex-1 truncate">
{model?.display_name ?? id}
</span>
</TooltipTrigger>
<TooltipContent
side="left"
className="max-w-xs break-all"
>
{id}
{vramEst != null && vramEst > 0 && gpu.available && (
<span className="block text-[10px] mt-1">
{exceeds
? `Needs ~${vramEst}GB VRAM (GPU: ${gpu.memoryTotalGb}GB)`
: fitStatus === "tight"
? `~${vramEst}GB VRAM (tight fit on ${gpu.memoryTotalGb}GB)`
: `~${vramEst}GB VRAM`}
</span>
)}
{model?.path ?? id}
</TooltipContent>
</Tooltip>
<span className="ml-auto flex items-center gap-1.5 shrink-0">
{fitStatus === "exceeds" && (
<span className="text-[9px] font-medium text-red-400">
OOM
</span>
)}
{fitStatus === "tight" && (
<span className="text-[9px] font-medium text-amber-400">
TIGHT
</span>
)}
{detail && (
<span className="text-[10px] text-muted-foreground">
{detail}
</span>
)}
<span className="ml-auto shrink-0 text-[10px] text-muted-foreground">
{source}
</span>
</ComboboxItem>
);
}}
</ComboboxList>
<div ref={sentinelRef} className="h-px" />
{isLoadingMore && (
<div className="flex items-center justify-center py-2">
<Spinner className="size-3.5 text-muted-foreground" />
</div>
)}
</div>
</ComboboxContent>
</Combobox>
</ComboboxContent>
</Combobox>
</div>
{isLoadingLocalModels ? (
<p className="text-[10px] text-muted-foreground">
Scanning local models...
</p>
) : localModelsError ? (
<p className="text-[10px] text-red-500">{localModelsError}</p>
) : (
<p className="text-[10px] text-muted-foreground">
{trainableLocalModels.length > 0
? `${trainableLocalModels.length} local/cached models found`
: "No local models found. Enter path manually."}
</p>
)}
</div>
</div>
<div data-tour="studio-method" className="flex min-w-0 flex-col gap-2">
<span className="flex items-center gap-1.5 text-xs font-medium text-muted-foreground">
Method
<Tooltip>
<TooltipTrigger asChild={true}>
<button
type="button"
className="text-foreground/70 hover:text-foreground"
<div
data-tour="studio-base-model"
className="flex min-w-0 flex-col gap-2"
>
<span className="flex items-center gap-1.5 text-xs font-medium text-muted-foreground">
Hugging Face Model
<Tooltip>
<TooltipTrigger asChild={true}>
<button
type="button"
className="text-foreground/70 hover:text-foreground"
>
<HugeiconsIcon
icon={InformationCircleIcon}
className="size-3"
/>
</button>
</TooltipTrigger>
<TooltipContent>
Search Hugging Face models or pick from our recommended list.{" "}
<a
href="https://unsloth.ai/docs/get-started/fine-tuning-llms-guide/what-model-should-i-use"
target="_blank"
rel="noopener noreferrer"
className="text-primary underline"
>
Read more
</a>
</TooltipContent>
</Tooltip>
</span>
<div
ref={comboboxAnchorRef}
className="min-w-0"
onKeyDown={(event) => {
if (event.key !== "Enter") return;
if (!(event.target instanceof HTMLInputElement)) return;
event.preventDefault();
if (hfResults.length > 0) {
handleModelSelect(hfResults[0].id);
} else {
const text = event.target.value.trim();
if (text) handleModelSelect(text);
}
}}
>
<Combobox
items={resultIds}
filteredItems={resultIds}
filter={null}
value={selectedModel}
onValueChange={handleModelSelect}
onInputValueChange={handleInputChange}
itemToStringValue={(id) => id}
autoHighlight={true}
>
<ComboboxInput
placeholder="Search models..."
className="w-full leading-5"
>
<HugeiconsIcon
icon={InformationCircleIcon}
className="size-3"
/>
</button>
</TooltipTrigger>
<TooltipContent className="max-w-xs">
QLoRA uses 4-bit quantization for lowest VRAM. LoRA uses 16-bit.
Full updates all weights.{" "}
<InputGroupAddon>
<HugeiconsIcon icon={Search01Icon} className="size-4" />
</InputGroupAddon>
</ComboboxInput>
<ComboboxContent anchor={comboboxAnchorRef}>
{isLoading ? (
<div className="flex items-center justify-center py-4 gap-2 text-xs text-muted-foreground">
<Spinner className="size-4" /> Searching…
</div>
) : (
<ComboboxEmpty>No models found</ComboboxEmpty>
)}
<div
ref={scrollRef}
className="max-h-64 overflow-y-auto overscroll-contain [scrollbar-width:thin]"
>
<ComboboxList className="p-1 !max-h-none !overflow-visible">
{(id: string) => {
const entry = vramMap.get(id);
const detail = entry?.detail ?? null;
const fitStatus = entry?.status ?? null;
const vramEst = entry?.est ?? null;
const exceeds = fitStatus === "exceeds";
return (
<ComboboxItem
key={id}
value={id}
className={`gap-2 ${exceeds ? "opacity-50" : ""}`}
>
<Tooltip>
<TooltipTrigger asChild={true}>
<span
className={`block min-w-0 flex-1 truncate ${exceeds ? "line-through decoration-muted-foreground/50" : ""}`}
>
{id}
</span>
</TooltipTrigger>
<TooltipContent
side="left"
className="max-w-xs break-all"
>
{id}
{vramEst != null &&
vramEst > 0 &&
gpu.available && (
<span className="block text-[10px] mt-1">
{exceeds
? `Needs ~${vramEst}GB VRAM (GPU: ${gpu.memoryTotalGb}GB)`
: fitStatus === "tight"
? `~${vramEst}GB VRAM (tight fit on ${gpu.memoryTotalGb}GB)`
: `~${vramEst}GB VRAM`}
</span>
)}
</TooltipContent>
</Tooltip>
<span className="ml-auto flex items-center gap-1.5 shrink-0">
{fitStatus === "exceeds" && (
<span className="text-[9px] font-medium text-red-400">
OOM
</span>
)}
{fitStatus === "tight" && (
<span className="text-[9px] font-medium text-amber-400">
TIGHT
</span>
)}
{detail && (
<span className="text-[10px] text-muted-foreground">
{detail}
</span>
)}
</span>
</ComboboxItem>
);
}}
</ComboboxList>
<div ref={sentinelRef} className="h-px" />
{isLoadingMore && (
<div className="flex items-center justify-center py-2">
<Spinner className="size-3.5 text-muted-foreground" />
</div>
)}
</div>
</ComboboxContent>
</Combobox>
</div>
</div>
<div
data-tour="studio-method"
className="flex min-w-0 flex-col gap-2"
>
<span className="flex items-center gap-1.5 text-xs font-medium text-muted-foreground">
Method
<Tooltip>
<TooltipTrigger asChild={true}>
<button
type="button"
className="text-foreground/70 hover:text-foreground"
>
<HugeiconsIcon
icon={InformationCircleIcon}
className="size-3"
/>
</button>
</TooltipTrigger>
<TooltipContent className="max-w-xs">
QLoRA uses 4-bit quantization for lowest VRAM. LoRA uses
16-bit. Full updates all weights.{" "}
<a
href="https://unsloth.ai/docs/get-started/fine-tuning-llms-guide/lora-hyperparameters-guide"
target="_blank"
rel="noopener noreferrer"
className="text-primary underline"
>
Read more
</a>
</TooltipContent>
</Tooltip>
</span>
<Select
value={trainingMethod}
onValueChange={(v) => setTrainingMethod(v as TrainingMethod)}
>
<SelectTrigger className={DARK_TRIGGER}>
<SelectValue />
</SelectTrigger>
<SelectContent
position="popper"
className={`${DARK_CONTENT} w-[var(--radix-select-trigger-width)]`}
>
<SelectItem value="qlora">
<span className="flex items-center gap-2">
<span
className={`size-2 shrink-0 rounded-full ${METHOD_DOTS.qlora}`}
/>
QLoRA (4-bit)
</span>
</SelectItem>
<SelectItem value="lora">
<span className="flex items-center gap-2">
<span
className={`size-2 shrink-0 rounded-full ${METHOD_DOTS.lora}`}
/>
LoRA (16-bit)
</span>
</SelectItem>
<SelectItem value="full">
<span className="flex items-center gap-2">
<span
className={`size-2 shrink-0 rounded-full ${METHOD_DOTS.full}`}
/>
Full Fine-tune
</span>
</SelectItem>
</SelectContent>
</Select>
</div>
<div className="flex min-w-0 flex-col gap-2">
<span className="text-xs font-medium text-muted-foreground">
Hugging Face Token (Optional)
</span>
<InputGroup>
<InputGroupAddon>
<HugeiconsIcon icon={Key01Icon} className="size-4" />
</InputGroupAddon>
<InputGroupInput
type="password"
autoComplete="new-password"
name="hf-token"
placeholder="hf_..."
value={hfToken}
onChange={(e) => setHfToken(e.target.value)}
/>
</InputGroup>
{(tokenValidationError ?? hfSearchError) && (
<p className="text-xs text-destructive">
{tokenValidationError ?? hfSearchError}
{" — "}
<a
href="https://unsloth.ai/docs/get-started/fine-tuning-llms-guide/lora-hyperparameters-guide"
href="https://huggingface.co/settings/tokens"
target="_blank"
rel="noopener noreferrer"
className="text-primary underline"
className="underline"
>
Read more
Get or update token
</a>
</TooltipContent>
</Tooltip>
</span>
<Select
value={trainingMethod}
onValueChange={(v) => setTrainingMethod(v as TrainingMethod)}
>
<SelectTrigger className={DARK_TRIGGER}>
<SelectValue />
</SelectTrigger>
<SelectContent
position="popper"
className={`${DARK_CONTENT} w-[var(--radix-select-trigger-width)]`}
>
<SelectItem value="qlora">
<span className="flex items-center gap-2">
<span
className={`size-2 shrink-0 rounded-full ${METHOD_DOTS.qlora}`}
/>
QLoRA (4-bit)
</span>
</SelectItem>
<SelectItem value="lora">
<span className="flex items-center gap-2">
<span
className={`size-2 shrink-0 rounded-full ${METHOD_DOTS.lora}`}
/>
LoRA (16-bit)
</span>
</SelectItem>
<SelectItem value="full">
<span className="flex items-center gap-2">
<span
className={`size-2 shrink-0 rounded-full ${METHOD_DOTS.full}`}
/>
Full Fine-tune
</span>
</SelectItem>
</SelectContent>
</Select>
</div>
<div className="flex min-w-0 flex-col gap-2">
<span className="text-xs font-medium text-muted-foreground">
Hugging Face Token (Optional)
</span>
<InputGroup>
<InputGroupAddon>
<HugeiconsIcon icon={Key01Icon} className="size-4" />
</InputGroupAddon>
<InputGroupInput
type="password"
autoComplete="new-password"
name="hf-token"
placeholder="hf_..."
value={hfToken}
onChange={(e) => setHfToken(e.target.value)}
/>
</InputGroup>
{(tokenValidationError ?? hfSearchError) && (
<p className="text-xs text-destructive">
{tokenValidationError ?? hfSearchError}
{" — "}
<a
href="https://huggingface.co/settings/tokens"
target="_blank"
rel="noopener noreferrer"
className="underline"
>
Get or update token
</a>
</p>
)}
{isCheckingToken && (
<p className="text-xs text-muted-foreground">Checking token…</p>
)}
</div>
</p>
)}
{isCheckingToken && (
<p className="text-xs text-muted-foreground">Checking token…</p>
)}
</div>
</div>
</SectionCard>
</div>

2
studio/frontend/src/features/training/api/models-api.ts
View file

@ -79,7 +79,7 @@ export interface LocalModelInfo {
id: string;
display_name: string;
path: string;
source: "models_dir" | "hf_cache" | "lmstudio";
source: "models_dir" | "hf_cache" | "lmstudio" | "custom";
model_id?: string | null;
updated_at?: number | null;
}