ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Feat/model picker per model config (#6647)

Browse source 
* refactor(studio): move chat model picker into features/model-picker

Relocate model-selector + its support files from components/assistant-ui
into a self-contained features/model-picker feature (own barrel), mirroring
the modular Hub layout. Pure move + import repoint; no behaviour change.

* feat(model-picker): add per-model config persistence layer

Superset PerModelConfig (customContextLength, kvCacheDtype, speculativeType,
specDraftNMax, tensorParallel, chatTemplateOverride, trustRemoteCode) persisted
to localStorage (unsloth_model_configs) with schema versioning + LRU budget.
KV-dtype and speculative value sets match main's sidebar (no q4_0/ngram-simple).
Reuses features/hub/lib/model-identity for normalization; adds storage-key layer
and applyPerModelConfigToRuntime (sets tensorParallel, which the old PR omitted).

* feat(picker): modular backend for chat-template validate + default fetch

New studio/backend/picker package (schemas/service/routes) mounted at /api/picker:
- POST /api/picker/validate-chat-template (Jinja syntax validation, no false positives)
- GET  /api/picker/chat-template/{model_name} (default template from tokenizer_config.json,
  reusing get_cache_path/resolve_cached_repo_id_case; graceful null, no model-code exec)
Frontend api/templates.ts client + hooks/use-model-defaults lazy cache. No backend
changes to the existing inference load route (per-model load fields already supported).

* feat(model-picker): bind picker on-device list to shared hub inventory

Picker now sources cached + local models from useHubInventory (the Hub's shared
store) via a thin adapter, replacing its own /api/models/* fetchers + module
caches. Hub, download manager, and picker now share one source of truth, so
completed downloads reflect in the picker automatically. Partial/live-download
rows are filtered from the cached lists (unchanged rendering). Local naming/search
preserved via additive LocalInventoryRow modelId/displayName. Variant expander,
scan-folder management, recommended-fit, search, external providers untouched.

Known minor: cached 'Downloaded date' sort tiebreak degrades to alphabetical
(hub cached rows carry no mtime); default 'recent' (load-time) sort preserved.

* feat(model-picker): per-model config step inside the picker

Picking a (non-external) model now opens an in-picker config view built from
main's current load controls (context length, KV cache dtype, speculative
decoding, draft tokens, tensor parallel) plus a chat-template editor backed by
the picker validate/default endpoints. 'Remember for this model' persists the
config per model+variant; Run forwards the config to the existing load flow via
meta.config. External models bypass the step. Two-view orchestration lives in
model-selector (single interception point); pickers.tsx call sites untouched.
trustRemoteCode dropped from PerModelConfig to preserve main's per-load consent.

* feat(chat): apply/persist per-model config through the load flow

handleCheckpointChange threads meta.config into the selection; stageOrLoad and
the autoload/Hub-run paths now apply the picker config (explicit pick or saved
remembered config) via applyPerModelConfigToRuntime before staging/loading, with
keepSpeculative set so a remembered speculative mode survives the model switch.
Replaces the old remembered-load-settings seeding (resolveInitialConfig now the
single source). SelectedModelInput carries config.

* refactor(chat): remove per-model load config from the right sidebar

The load knobs (context, KV cache, speculative, draft tokens, tensor parallel)
and the chat-template editor now live only in the picker config step. The sheet's
Model section keeps the staged Load/Cancel flow (config is applied at pick time);
sampling params, system prompt, and RAG are unchanged. Deletes the superseded
remembered-load-settings module + the store's applyRememberedLoadSettings action,
removes the now-dead sheet state/imports, and points the settings reset at
unsloth_model_configs. Delete-cleanup deferred (stale config is LRU-capped).

* fix(model-picker): remove leftover sidebar-staging cogwheel + empty Model section

The downloaded-variant gear (ModelLoadSettingsAction) staged a model straight
into the right-sidebar Run-settings flow -- the old 'configure before load' path
now fully replaced by the in-picker config step. Removed the gear + its component.
Also gate the sheet's 'Model' section to staged picks only (pendingSelection):
after the load-knob strip its content is staged-only, so it was rendering an
empty section header whenever a model was merely loaded.

* chore(chat): remove dead per-model-config setters + modelControlsDisabled

After the load-config UI moved into the picker, the store's per-model setters
(setKvCacheDtype/setSpeculativeType/setSpecDraftNMax/setTensorParallel/
setCustomContextLength/setChatTemplateOverride) had zero callers
(applyPerModelConfigToRuntime writes via setState), and the sheet's
modelControlsDisabled was unreferenced. Verified dead across the whole tree.

* fix(chat): config-step Load actually loads (ignore Load-on-selection)

Root cause: with Settings > Chat > 'Load on selection' turned OFF, the config
step's load went down the deferred-staging path -- opening the right sidebar with
'<model> is staged, not loaded yet / Choose Load model'. The in-picker config step
IS the deliberate load action, so its Load now loads immediately (or downloads +
auto-loads when not cached) regardless of the toggle. Renamed the button
'Run model' -> 'Load model' to match. Native/dropped picks still honor the toggle.

* refactor(chat,hub): retire 'Load on selection' — config step is the only load flow

The in-picker config step (and the Hub Run button) now fully supersede the old
stage-to-sidebar flow, so the Load-on-selection toggle is removed everywhere:
- chat stageOrLoad: every pick loads immediately, or downloads + auto-loads when
  not cached (the previous default behaviour, now universal).
- hub Run: drops the stage branch; downloaded GGUFs load directly with their saved
  per-model config (no collision with the chat config step — both end at selectModel).
- store: removed loadOnSelection field/setter/key/default; Settings>Chat toggle and
  its settings-reset entry removed.
- staged sidebar section is now a download-progress view (auto-loads on completion).
No manual staging remains; stageModel is used only for background auto-load downloads.

* feat(model-picker): default chat template from GGUF + thread variant through config flow

Read the embedded tokenizer.chat_template from GGUF files (read_gguf_chat_template
in gguf_metadata) and use it as the per-model default. Plumb gguf_variant through
the picker service, /api/picker/chat-template route, frontend templates API, and
use-model-defaults so the right variant's template is fetched.

Also refine the picker config-page/model-selector wiring, drop the dead
ggufNativeContextLength runtime path, and add the per-model-config storage keys to
the settings prefs export.

* feat(model-picker): read safetensors chat template + hide editor where it has no effect

Resolve the default chat template for safetensors models: prefer the modern
chat_template.jinja, fall back to the tokenizer_config.json chat_template field,
then chat_template.json (multimodal processor), then the GGUF embedded template.
Applied to local dirs, the HF cache snapshot scan, and the HF remote fetch.

Hide the chat-template editor in the picker for safetensors models — the override
is only applied at load by the GGUF/llama.cpp backend, so editing it on safetensors
currently has no effect. GGUF keeps the editor. Nothing removed; the dialog stays
for when the safetensors apply path is wired up in a later branch.

* fix(model-picker): set legacy-migration flag only after the write succeeds

Set unsloth_model_configs_migrated only once writeMap confirms the migrated
map persisted, so a quota/storage failure no longer marks migration done and
silently drops the user's pre-existing remembered settings — the next load retries.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* MVP model picker fixes

* MVP picker config fix

* MVP safetensors config

* MVP max seq config

* MVP max seq fix

* Fix static max tokens cap ignoring model context

* Fix picker GGUF scan parity

* fix(studio): harden model picker config loading

Apply remembered per-model configs consistently from picker and Hub loads, keep default configs from overriding standing speculative settings, add config access for direct local GGUF files, and support saving or forgetting active model settings without a reload.

* Fix model picker config flow

* Fix model picker config loads

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Avoid recursive per-model config migration reads

* Apply the displayed context length when loading a GGUF

* Fix template validation, cached template lookup, and failed load rollback

- Validate chat templates with the loopcontrols extension so templates
  that use break or continue tags pass the picker validator, matching the
  inference renderer that already accepts them.
- Read the default chat template from the newest cache snapshot rather than
  an arbitrary iterdir order, so an older cached revision no longer prefills
  a stale template.
- Capture the runtime per-model config before a load and reapply it when the
  load fails, so a failed switch leaves the active model context, KV cache,
  template, and speculative settings as they were.

* Make chat template view only for safetensors models

Custom chat template overrides are applied at inference only for GGUF
models, which pass the template to llama-server. The safetensors backend
renders with the model built-in template and ignores the override, so
editing it would save a value that never loads. For safetensors the
config page now opens the template as a read-only preview with a note
that editing is not available yet. This can become editable once
inference support for custom safetensors templates lands in main.

* Fix model picker config edge cases

- Restore prior runtime config when a load no-ops for the active model
- Cap the picker validator request body via the protected prefixes
- Keep the GGUF context slider max above the loaded context
- Fetch subfolder chat templates for uncached Hub repos
- Show the compare side config when reopening the picker

* Keep saved GGUF context above the fallback ceiling

* Show the model config in the run settings sidebar

* Fix model config sidebar reset and context slider

- Stack the remember toggle and action buttons in the sidebar
- Reset the config to defaults instead of the loaded values
- Fetch the native context so the slider max is not the loaded value

* Fix model picker config and download regressions

- Run picker chat template routes off the event loop
- Depth and root guard local template directory scans
- Restore download manager flow for uncached hub picks
- Apply per model context length on reload
- Import model picker symbols from the feature barrel

* Fix model picker config and cached download sorting

- Restore load settings when a Hub run is rejected mid load
- Reuse one NumericValueInput instead of a duplicate copy
- Fix double decode of the model name in the template route
- Remove the unused reset-to-loaded settings action
- Fix cached model download sorting

* Fix model picker per-model config edge cases

Honor a saved or typed max seq length above the model's native context so
RoPE extended values are no longer clamped and silently overwritten. Allow
typing past native while the slider keeps native as a soft ceiling.

Guard the fetch success paths in use-model-defaults against an aborted
signal, and refetch when the HF token changes.

Hash the chat template content in the sidebar remount key instead of its
length. Enable reset for a GGUF whose native context is unknown, and floor
the context slider max so it can never fall below the min.

* Fix GGUF context auto-fit and gated model config token

Stop forcing a 32768 context when a GGUF native context is unknown so the backend auto-fits to VRAM again, while still honoring an explicit context edit.

Send the HF token as a query param so gated safetensors models resolve their max position embeddings.

Derive model default state during render to drop the set-state-in-effect calls.

* Fix native GGUF context ceiling and guard picker template reads

Restore the native context store field so the sidebar slider keeps the
full ceiling for drag and drop GGUFs. Limit local chat template reads to
the browse allowlist, skip malformed repo ids, and drop unused model
picker exports.

* Fix model picker lint boundaries

* Fix model picker review findings

Chat template editor never seeded its draft. Radix only calls onOpenChange
from internal events, so the seed in the nextOpen branch was dead and a model
with a saved override opened empty. Saving then cleared the override. Drop the
dead branch, treat draft as an untouched sentinel, and reset it on every close.

Uncached Hub picks could auto load a model after the user left the chat. Main
detached the staged pick on route exit and on chat context change. Carry the
context key on the pending pick and skip the load when it no longer matches.

Also clear configTarget when the picker closes, restore the onUpdated ref so
variant rows stop resubscribing on every parent render, skip the LRU write when
the entry is already most recent, import NumericValueInput relatively, and drop
the unused ModelUpdateAction barrel export.

* Preserve GGUF context on active reload

* Fix model picker per-model config regressions

- Stop reloading the already loaded model on re-pick
- Hide infra models from the chat picker
- Detect vision support on cached GGUF repos
- Honor saved maxSeqLength on auto load
- Restore default chat template for local GGUFs
- Warn on save failure and revert config on cancel
- Refetch picker inventory on open
- Persist read only per model config safely

* Fix stale model auto load

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Fix model picker numeric input sizing and constraints

Size value inputs to their content so long context lengths are not clipped,
restrict them to numeric characters, and stop the speculative decoding label
from truncating in the sidebar.

* Fix picker CI tests and harden chat template resolution for PR #6647

- tests: point the descender guard at the moved model-selector.tsx path
- tests: exclude the disabled Reload model button from the regenerate locator so .first targets the real Regenerate
- picker/service.py: reject symlinked template/gguf leaves that resolve outside the browse allowlist (HF cache reads unchanged)
- compare mode: resolve each pane's own remembered chat template instead of inheriting the other pane's from the store

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Protect future-schema per-model configs from deletion for PR #6647

savePerModelConfig already refuses to overwrite a stored config whose schema version is newer than this client understands, but deletePerModelConfig did not. Unchecking Remember on an older client therefore silently destroyed a newer client's saved config. Apply the same guard on delete and surface the blocked case through the existing saveFailed toast.

* Protect future-schema per-model configs from quota eviction for PR #6647

The save and delete guards already refuse to touch a stored config whose schema version is newer than this client understands, but the quota-eviction path did not, so a full store on an older client could still evict a newer client's config. Skip future-schema entries when evicting and fail the save if the budget cannot be met without them.

* Fix GGUF context persistence, compare context, and rollback settings for PR #6647

Persist a GGUF context override from the user's intent instead of collapsing it against the loaded context, which reintroduced the context-reset (f4838782cb reverted the native-baseline fix). model-config-page now collapses the saved value against native, and use-chat-model-runtime and chat-adapter retain the requested context on load so re-saving another setting keeps the override; a null request stays null so a VRAM auto-fit never becomes a stored override.

shared-composer: a compare pane with no explicit GGUF context now loads at native (0) like single-view, not the session maxSeqLength that silently shrank the shown context.

use-chat-model-runtime: restore the previous model's KV cache dtype and chat template on a failed-load rollback so it runs as it was, not with backend defaults.

* Preserve native path token when reloading the active model for PR #6647

handleReloadActiveModel rebuilt the selection without the store's activeNativePathToken, so reloading a file-picked GGUF after a settings change validated the display label as a repo/path and failed. Thread the active native token through the reload selection so native-loaded models reopen correctly.

* Make picker template validation resilient and accept HF generation tags for PR #6647

Import Jinja lazily inside validate_chat_template so a backend without the optional jinja2 package (GGUF-only installs) still starts instead of raising ModuleNotFoundError at import time. Register a no-op extension for the Transformers {% generation %} assistant-mask tag so pasting a valid HF chat template validates, matching the renderer, rather than being rejected as an unknown tag.

* Honor remembered compare config and parse processor chat_template.json for PR #6647

* Fix failed-load rollback context and processor template map fallback for PR #6647

* Restore speculative decoding config on failed-switch rollback

When a model switch fails after the previous model was unloaded, the
rollback reload restored tensor_parallel, KV cache dtype and the chat
template override, but omitted speculative_type and spec_draft_n_max and
cleared their loaded shadows to null. The previous model therefore came
back running at backend defaults (speculation off) while the UI still
showed it enabled, and the status resync confirmed the off state. Resend
the previous model's speculative settings in the rollback load and keep
the store's active and loaded speculative fields in sync with them.

* Reset max sequence length when a model has no saved config

applyPerModelConfigToRuntime reset every per-model field except
maxSeqLength, which it only wrote when the incoming config had one.
maxSeqLength is the sole field carried on store.params, so selecting a
model with no remembered config left the previous model's value in place
and later loaded the new model at that leaked length. Fall back to the
standing default so an unremembered model loads at its own default.

* Surface a message when a variant update cannot start

startManagedUpdate handled the conflict and error start outcomes but let
busy fall through as if the update began, so the confirm dialog closed
with no job created and the cached variant stayed stale. Show an info
message when the repo is busy with a sibling transfer so the click is
not silently dropped.

* Keep per-model speculative choices out of the global default

A staged load with a per-model or one-off config sets keepSpeculative,
which already skips reading the global speculative preference. The
matching save still ran unconditionally, so the model-specific choice was
written to the global unsloth_chat_speculative_type and a later model with
no saved config started from it instead of Auto. Skip saveSpeculativeType
when keepSpeculative so the per-model choice stays isolated.

* Seed non-active model settings from the app default max length

The Run settings page captured initialMaxSeqLength from the loaded
model's runtime params and fell back to it for a model with no saved
config. Opening settings for a different, unloaded model and clicking
Load then sent the active model's context (for example 64k) instead of
the 4096 default, risking validation failures or OOMs. Seed the default
for non-active models and keep the runtime value only for the active one.

* Prefer sidecar tokenizer chat template over the GGUF copy for variants

_chat_template_from_dir returned the embedded GGUF template first when a
variant was selected, reversing the tokenizer-first precedence of the
no-variant path. A model whose chat_template.jinja or tokenizer_config.json
supersedes a stale embedded template then got the wrong template on
variant selection. Keep tokenizer files first regardless of variant; the
variant only picks which GGUF is the fallback. Adds regression tests for
both the tokenizer-wins and gguf-fallback cases.

* Keep per-model speculative choices load-local in autoload and compare

The interactive load path treats a per-model speculative choice as
load-local and skips writing it to the global default. Autoload and
generalized compare still called saveSpeculativeType unconditionally, so a
remembered off or ngram setting leaked into unsloth_chat_speculative_type
and later models with no saved config inherited it. Persist the global
preference only when the value came from the global settings.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: record the compare pane's loaded context in runtime state so the active model's settings and any reload or save use it, not the previous context

* Studio: notify the user when a Hub autoload can't start because another download for the model is already running, instead of silently dropping it

* Studio: drop the merge's orphaned staged-model store helpers and unused alert imports

The main merge left isPendingGguf and pendingSelectionMatches referencing the
removed PendingModelSelection type, and the alert-dialog/alert imports unused
after the permission-mode dropdown replaced the bypass dialog, so tsc -b failed.

* Studio: cache a null default chat template so the viewer stops re-fetching it

A model with no sidecar or embedded template resolves to a terminal null, but
that result was never cached, so reopening the template viewer re-ran the
backend and Hugging Face lookup every time.

* Studio: detect direct-file GGUFs in run settings so Max Tokens uses their context

A GGUF loaded from a local file or custom folder has no variant label, so the
run-settings panel treated it as non-GGUF and clamped Max Tokens to the session
max_seq_length instead of the loaded GGUF context. Detect it via the reported
GGUF context and the .gguf checkpoint suffix, matching the chat page.

* Studio: prompt to re-select a local model file when its lease expired before reload

A file-picked GGUF is reachable only through a native path token that the
desktop host prunes after a TTL. Reloading reused that token blindly, so a
reload long after the initial load failed with an opaque error. Track the
token's expiry and, when it has passed, ask the user to re-select the file
instead of attempting a doomed reload.

* Fix descender-clipping test to tolerate sidebar layout utilities

The sidebar account-block div carries layout utilities (min-w-0, flex-1)
between 'flex' and 'flex-col', so the descender-clipping guard's regex,
which required 'flex' immediately followed by 'flex-col', no longer matched
and the test failed to locate the account-block div. Generalize the prefix
to allow intervening flex utilities while still capturing the leading-*
class before the collapsible visibility utility and asserting leading-tight,
so the guard against clipped glyph descenders is fully preserved.

* Harden picker chat-template resolution

Enforce the 64 KiB chat-template contract at the validate endpoint's request
model so a direct caller cannot submit a template far larger than the frontend
allows (MaxBodyMiddleware only bounds the whole request body, not this field);
oversized templates now return a clean 422.

Apply sidecar-over-GGUF template precedence globally across cached snapshots
instead of per snapshot. A repo with multiple cached revisions previously
returned the first snapshot's template, so a newer GGUF-only revision could
win over an older revision's maintained chat_template.jinja sidecar, which
contradicted the documented intent that sidecars supersede the embedded copy.

* Guard per-model config against future-schema and lossy migration

Two forward-compatibility gaps in the versioned per-model config store:

- The load/apply path returned and normalized a stored record without checking
  its schema version, so a record written by a newer client was reinterpreted
  under the current schema and applied to a live model load, even though save,
  delete and eviction all refuse to touch future-schema records. Reject
  future-schema records on load too.
- The one-time legacy migration enforced the storage budget without protecting
  the entries it had just migrated and set the completion flag unconditionally.
  When storage was already full of future-schema records (which are unevictable
  by an older client), the migrated entries were the only evictable ones and
  could be dropped while migration was still marked complete. Protect the
  migrated keys during eviction and only mark migration complete when they
  survive, so it retries once space frees up.

* Discard chat-template validation results after the dialog closes

Server-side template validation is async, but closing or cancelling the editor
did not abort it, so a late-arriving valid response still called onSave and
applied a template the user had already dismissed. Track a validation token
that is bumped on close and ignore any validation result whose token is stale.

* Record native lease expiry when loading a picked GGUF from the chip

The pending-native-model chip loaded via stageOrLoad directly, bypassing
loadNativeModelIntent, so activeNativePathExpiresAtMs was never recorded for a
chip-loaded file. A later reload then either skipped the lease-expiry guard
entirely (expiry left null) or compared against a previously loaded file's
stale expiry, so reload could reuse an already-pruned token or wrongly block a
still-valid one. Route the chip through loadNativeModelIntent, which builds the
same selection and records the expiry.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Prefer sidecar template for a directly selected local GGUF file

A direct .gguf file path read its embedded chat template without checking the
parent directory for a maintained sidecar (chat_template.jinja /
tokenizer_config.json), while directory and variant selections already prefer
the sidecar. That let the config editor preview or save a stale embedded
template for the same model depending on how it was selected. Check the parent
directory sidecars first, then fall back to the embedded copy, and cover both
paths with tests.

* Resolve cached chat template per revision, newest first

The earlier change searched every cached snapshot for a sidecar before
considering any snapshot's embedded GGUF template, which let an obsolete sidecar
from an older revision override the newest revision's template. Restore
per-snapshot resolution (newest first): a revision's sidecar still supersedes
its own embedded GGUF copy, but a newer revision is no longer overridden by an
older revision's sidecar.

* Preserve autoload transport conflicts and surface background busy downloads

- When a Hub autoload hits a transport conflict, keep pendingHubAutoLoad bound
  instead of clearing it. Clearing it re-keyed the download surface and its
  cleanup cancelled the conflict the toast tells the user to resolve, so the
  Hub resume affordance was gone the moment it appeared. Return early on
  conflict, mirroring the started branch, so resolving it from the Hub still
  auto-loads on completion.
- The background-download branch handled started and conflict but silently
  dropped a busy outcome, leaving the user with no feedback when a peer variant
  of the same repo was already downloading. Surface the same busy toast the
  autoload path uses.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
This commit is contained in:
Eyera 2026-07-17 15:08:01 +02:00 • committed by GitHub
commit 8cbdfbe355
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
62 changed files with 4476 additions and 2126 deletions
Download patch file Download diff file Expand all files Collapse all files

1
studio/backend/hub/schemas/inventory.py
View file

@ -160,6 +160,7 @@ class CachedRepoBase(BaseModel):
repo_id: str
size_bytes: int = 0
cache_path: Optional[str] = None
last_modified: Optional[float] = None
partial: bool = False
partial_transport: Optional[str] = None
inventory_id: Optional[str] = None

74
studio/backend/hub/services/models/cache_inventory.py
View file

@ -31,6 +31,7 @@ from hub.services.models.common import (
_is_checkpoint_weight_name,
_is_gguf_filename,
_is_main_gguf_filename,
_is_mmproj_filename,
_is_transformers_safetensors_weight_name,
_local_inventory_id,
_prefer_complete_larger,
@ -125,6 +126,34 @@ def _repo_has_gguf_files(repo_info) -> bool:
return _repo_gguf_size_bytes(repo_info) > 0
def _blob_mtime(file_obj) -> float:
ts = getattr(file_obj, "blob_last_modified", None)
if isinstance(ts, (int, float)) and ts > 0:
return float(ts)
blob_path = getattr(file_obj, "blob_path", None)
if blob_path:
try:
return float(Path(blob_path).stat().st_mtime)
except OSError:
pass
return 0.0
def _repo_gguf_last_modified(repo_info) -> float:
latest = 0.0
for revision in repo_info.revisions:
for f in revision.files:
if _is_main_gguf_filename(f.file_name):
latest = max(latest, _blob_mtime(f))
return latest
def _repo_has_mmproj(repo_info) -> bool:
return any(
_is_mmproj_filename(f.file_name) for revision in repo_info.revisions for f in revision.files
)
def _cached_repo_file_name(file_obj) -> str:
file_path = getattr(file_obj, "file_path", None)
if file_path:
@ -266,6 +295,7 @@ def _scan_cached_gguf() -> list[dict]:
continue
key = repo_id.lower()
existing = seen_lower.get(key)
last_modified = _repo_gguf_last_modified(repo_info)
row = {
"repo_id": repo_id,
"size_bytes": max(total_size, variant_state_size),
@ -275,6 +305,9 @@ def _scan_cached_gguf() -> list[dict]:
# per-variant detail lives on GgufVariantDetail.
"partial_transport": None,
}
last_modified = max(last_modified, (existing or {}).get("last_modified", 0.0))
if last_modified > 0:
row["last_modified"] = last_modified
row.update(
_cache_inventory_fields(
repo_id,
@ -283,8 +316,12 @@ def _scan_cached_gguf() -> list[dict]:
requires_variant = True,
)
)
if _repo_has_mmproj(repo_info):
row["capabilities"]["supports_vision"] = True
if _prefer_cache_row(row, existing):
seen_lower[key] = row
elif last_modified > existing.get("last_modified", 0.0):
existing["last_modified"] = last_modified
except Exception as e:
repo_label = getattr(repo_info, "repo_id", "<unknown>")
logger.warning(f"Skipping cached GGUF repo {repo_label}: {e}")
@ -312,13 +349,14 @@ class _CachedNonGgufPayload(NamedTuple):
size_bytes: int
has_runnable_weights: bool
model_format: ModelFormat
last_modified: float
def _repo_non_gguf_model_payload(repo_info) -> _CachedNonGgufPayload:
all_weight_blobs: dict[str, int] = {}
adapter_blobs: dict[str, int] = {}
safetensors_blobs: dict[str, int] = {}
checkpoint_blobs: dict[str, int] = {}
all_weight_blobs: dict[str, tuple[int, float]] = {}
adapter_blobs: dict[str, tuple[int, float]] = {}
safetensors_blobs: dict[str, tuple[int, float]] = {}
checkpoint_blobs: dict[str, tuple[int, float]] = {}
has_config = False
has_adapter_config = False
has_adapter_weights = False
@ -326,12 +364,15 @@ def _repo_non_gguf_model_payload(repo_info) -> _CachedNonGgufPayload:
has_transformers_safetensors = False
has_checkpoint = False
def _record_blob(target: dict[str, int], file_obj, rev_id: str, file_name: str) -> None:
def _record_blob(
target: dict[str, tuple[int, float]], file_obj, rev_id: str, file_name: str
) -> None:
blob_path = getattr(file_obj, "blob_path", None)
size = int(file_obj.size_on_disk or 0)
key = str(blob_path) if blob_path else f"{rev_id}:{file_name}"
target[key] = size
all_weight_blobs[key] = size
value = (size, _blob_mtime(file_obj))
target[key] = value
all_weight_blobs[key] = value
for revision in repo_info.revisions:
rev_id = getattr(revision, "commit_hash", None) or str(id(revision))
@ -375,18 +416,19 @@ def _repo_non_gguf_model_payload(repo_info) -> _CachedNonGgufPayload:
or "unknown"
)
if model_format == "adapter":
size_bytes = sum(adapter_blobs.values())
selected_blobs = adapter_blobs
elif model_format == "safetensors":
size_bytes = sum(safetensors_blobs.values())
selected_blobs = safetensors_blobs
elif model_format == "checkpoint":
size_bytes = sum(checkpoint_blobs.values())
selected_blobs = checkpoint_blobs
else:
size_bytes = sum(all_weight_blobs.values())
selected_blobs = all_weight_blobs
return _CachedNonGgufPayload(
size_bytes = size_bytes,
size_bytes = sum(size for size, _mtime in selected_blobs.values()),
has_runnable_weights = model_format != "unknown",
model_format = model_format,
last_modified = max((mtime for _size, mtime in selected_blobs.values()), default = 0.0),
)
@ -508,6 +550,12 @@ def _scan_cached_models() -> list[dict]:
),
**_cached_model_local_metadata(repo_path),
}
last_modified = max(
payload.last_modified,
(existing or {}).get("last_modified", 0.0),
)
if last_modified > 0:
row["last_modified"] = last_modified
row.update(
_cache_inventory_fields(
repo_id,
@ -517,6 +565,8 @@ def _scan_cached_models() -> list[dict]:
)
if _prefer_cache_row(row, existing):
seen_lower[key] = row
elif last_modified > existing.get("last_modified", 0.0):
existing["last_modified"] = last_modified
except Exception as e:
repo_label = getattr(repo_info, "repo_id", "<unknown>")
logger.warning(f"Skipping cached model repo {repo_label}: {e}")

3
studio/backend/main.py
View file

@ -313,6 +313,7 @@ from hub.routes import (
inventory_router as hub_inventory_router,
datasets_router as hub_datasets_router,
)
from picker.routes import templates_router as picker_templates_router
from hub.schemas.downloads import TransportCapabilities
from hub.utils.download_registry import (
get_download_transport_capabilities,
@ -745,6 +746,7 @@ _BODY_PROTECTED_PREFIXES = (
"/v1/completions",
"/p/",
"/api/inference",
"/api/picker",
"/api/data-recipe",
"/api/datasets",
"/api/hub",
@ -975,6 +977,7 @@ app.include_router(rag_router, prefix = "/api/rag", tags = ["rag"])
app.include_router(training_history_router, prefix = "/api/train", tags = ["training-history"])
app.include_router(hub_inventory_router, prefix = "/api/hub", tags = ["hub"])
app.include_router(hub_datasets_router, prefix = "/api/hub/datasets", tags = ["hub"])
app.include_router(picker_templates_router, prefix = "/api/picker", tags = ["picker"])
# Re-wrap client-error responses on the /v1/* surface into OpenAI/Anthropic
# error envelopes; non-/v1 paths keep FastAPI's default {"detail": ...} shape.

2
studio/backend/picker/__init__.py Normal file
View file

@ -0,0 +1,2 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0

6
studio/backend/picker/routes/__init__.py Normal file
View file

@ -0,0 +1,6 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
from .templates import router as templates_router
__all__ = ["templates_router"]

42
studio/backend/picker/routes/templates.py Normal file
View file

@ -0,0 +1,42 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
from __future__ import annotations
import asyncio
from typing import Optional
from fastapi import APIRouter, Body, Depends, Query
from auth.authentication import get_current_subject
from hub.dependencies import get_hf_token
from ..schemas import (
ModelTemplateResponse,
ValidateChatTemplateRequest,
ValidateChatTemplateResponse,
)
from ..service import read_default_chat_template, validate_chat_template
router = APIRouter()
@router.post("/validate-chat-template", response_model = ValidateChatTemplateResponse)
async def validate_chat_template_route(
body: ValidateChatTemplateRequest = Body(...),
current_subject: str = Depends(get_current_subject),
) -> ValidateChatTemplateResponse:
return await asyncio.to_thread(validate_chat_template, body.template)
@router.get("/chat-template/{model_name:path}", response_model = ModelTemplateResponse)
async def get_default_chat_template_route(
model_name: str,
gguf_variant: Optional[str] = Query(None),
hf_token: Optional[str] = Depends(get_hf_token),
current_subject: str = Depends(get_current_subject),
) -> ModelTemplateResponse:
template = await asyncio.to_thread(
read_default_chat_template, model_name, hf_token, gguf_variant
)
return ModelTemplateResponse(model_name = model_name, chat_template = template)

32
studio/backend/picker/schemas.py Normal file
View file

@ -0,0 +1,32 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
from typing import Optional
from pydantic import BaseModel, Field, field_validator
# Mirror the frontend's 64 KiB chat-template contract (per-model-config.ts) at
# the API boundary so a direct caller cannot make Jinja parse an oversized
# template. MaxBodyMiddleware only caps the whole request body, not this field.
MAX_CHAT_TEMPLATE_BYTES = 65_536
class ValidateChatTemplateRequest(BaseModel):
template: str = Field(default = "")
@field_validator("template")
@classmethod
def _enforce_template_size(cls, value: str) -> str:
if len(value.encode("utf-8")) > MAX_CHAT_TEMPLATE_BYTES:
raise ValueError(f"Chat template exceeds the {MAX_CHAT_TEMPLATE_BYTES}-byte limit.")
return value
class ValidateChatTemplateResponse(BaseModel):
valid: bool
error: Optional[str] = None
class ModelTemplateResponse(BaseModel):
model_name: str
chat_template: Optional[str] = None

361
studio/backend/picker/service.py Normal file
View file

@ -0,0 +1,361 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
from __future__ import annotations
import json
import logging
import os
import re
from pathlib import Path
from typing import Optional
from hub.services.models.folder_browser import (
_build_browse_allowlist,
_is_path_inside_allowlist,
)
from hub.utils.gguf import iter_hf_cache_snapshots
from utils.models.gguf_metadata import read_gguf_chat_template
from utils.models.model_config import (
_extract_quant_label,
_is_big_endian_gguf_path,
_is_mmproj,
_is_mtp_drafter,
)
from utils.paths.path_utils import (
is_local_path,
normalize_path,
resolve_cached_repo_id_case,
)
from .schemas import ValidateChatTemplateResponse
logger = logging.getLogger(__name__)
_VALID_REPO_ID = re.compile(r"^[A-Za-z0-9._-]+/[A-Za-z0-9._-]+$")
def _is_valid_repo_id(repo_id: str) -> bool:
return bool(_VALID_REPO_ID.fullmatch(repo_id))
_TOKENIZER_CONFIG_PATHS = ("tokenizer_config.json", "LLM/tokenizer_config.json")
_JINJA_TEMPLATE_PATHS = ("chat_template.jinja", "LLM/chat_template.jinja")
_PROCESSOR_TEMPLATE_PATHS = ("chat_template.json", "LLM/chat_template.json")
def _leaf_inside_allowlist(path: Path, allow_roots: Optional[list[Path]]) -> bool:
# Block symlinked children from escaping the validated directory
# (realpath-checked). None = trusted caller (HF cache / remote download).
return allow_roots is None or _is_path_inside_allowlist(path, allow_roots)
def validate_chat_template(template: str) -> ValidateChatTemplateResponse:
text = (template or "").strip()
if not text:
return ValidateChatTemplateResponse(valid = True, error = None)
# Import Jinja lazily: it is optional at runtime (e.g. GGUF-only installs),
# so a missing dependency must not crash API startup through this module.
try:
from jinja2 import TemplateError
from jinja2.ext import Extension
from jinja2.sandbox import ImmutableSandboxedEnvironment
except ImportError:
return ValidateChatTemplateResponse(valid = True, error = None)
class _GenerationTag(Extension):
# Accept Transformers' {% generation %}...{% endgeneration %} assistant
# mask tag so a pasted HF chat template validates (we only parse it).
tags = {"generation"}
def parse(self, parser):
next(parser.stream)
return parser.parse_statements(["name:endgeneration"], drop_needle = True)
try:
env = ImmutableSandboxedEnvironment(
trim_blocks = True,
lstrip_blocks = True,
extensions = ["jinja2.ext.loopcontrols", _GenerationTag],
)
env.parse(text)
return ValidateChatTemplateResponse(valid = True, error = None)
except TemplateError as exc:
message = getattr(exc, "message", None) or str(exc)
lineno = getattr(exc, "lineno", None)
if lineno:
message = f"Line {lineno}: {message}"
return ValidateChatTemplateResponse(valid = False, error = message)
except Exception as exc:
return ValidateChatTemplateResponse(valid = False, error = str(exc))
def _chat_template_from_tokenizer_config(config: dict) -> Optional[str]:
if not isinstance(config, dict):
return None
raw = config.get("chat_template")
if isinstance(raw, str) and raw.strip():
return raw
if isinstance(raw, list):
fallback: Optional[str] = None
for entry in raw:
if not isinstance(entry, dict):
continue
template = entry.get("template")
if not isinstance(template, str):
continue
if entry.get("name") == "default":
return template
if fallback is None:
fallback = template
return fallback
return None
def _chat_template_from_jinja_file(
dir_path: Path, allow_roots: Optional[list[Path]] = None
) -> Optional[str]:
for rel in _JINJA_TEMPLATE_PATHS:
template_file = dir_path / rel
if not template_file.exists() or not _leaf_inside_allowlist(template_file, allow_roots):
continue
try:
template = template_file.read_text(encoding = "utf-8")
except Exception:
continue
if template.strip():
return template
return None
def _chat_template_from_processor_payload(payload: object) -> Optional[str]:
# processor chat_template.json may be the template string itself or a
# {name: template} map, not only a tokenizer_config-shaped object.
if isinstance(payload, str):
return payload if payload.strip() else None
template = _chat_template_from_tokenizer_config(payload) # type: ignore[arg-type]
if template:
return template
if isinstance(payload, dict):
# Named-template map: prefer "default", else the first non-empty entry
# (mirrors the tokenizer-config list fallback).
default = payload.get("default")
if isinstance(default, str) and default.strip():
return default
for value in payload.values():
if isinstance(value, str) and value.strip():
return value
return None
def _chat_template_from_processor_json(
dir_path: Path, allow_roots: Optional[list[Path]] = None
) -> Optional[str]:
for rel in _PROCESSOR_TEMPLATE_PATHS:
config_file = dir_path / rel
if not config_file.exists() or not _leaf_inside_allowlist(config_file, allow_roots):
continue
try:
payload = json.loads(config_file.read_text(encoding = "utf-8"))
except Exception:
continue
template = _chat_template_from_processor_payload(payload)
if template:
return template
return None
def _chat_template_from_tokenizer_dir(
dir_path: Path, allow_roots: Optional[list[Path]] = None
) -> Optional[str]:
jinja = _chat_template_from_jinja_file(dir_path, allow_roots)
if jinja:
return jinja
for rel in _TOKENIZER_CONFIG_PATHS:
config_file = dir_path / rel
if not config_file.exists() or not _leaf_inside_allowlist(config_file, allow_roots):
continue
try:
config = json.loads(config_file.read_text(encoding = "utf-8"))
except Exception:
continue
template = _chat_template_from_tokenizer_config(config)
if template:
return template
return _chat_template_from_processor_json(dir_path, allow_roots)
_GGUF_SCAN_MAX_DEPTH = 2
def _iter_ggufs(dir_path: Path) -> list[Path]:
if dir_path == dir_path.parent:
return []
root = str(dir_path)
found: list[Path] = []
for current, dirs, files in os.walk(root, followlinks = False):
rel = os.path.relpath(current, root)
depth = 0 if rel == os.curdir else rel.count(os.sep) + 1
if depth >= _GGUF_SCAN_MAX_DEPTH:
dirs[:] = []
for name in files:
if not name.lower().endswith(".gguf") or _is_mmproj(name):
continue
path = Path(current) / name
try:
rel = path.relative_to(dir_path).as_posix()
except ValueError:
rel = name
quant = _extract_quant_label(rel)
if _is_mtp_drafter(rel) or _is_big_endian_gguf_path(rel, quant):
continue
found.append(path)
return found
def _variant_matches(relative_path: str, needle: str) -> bool:
quant = _extract_quant_label(relative_path).lower()
if quant == needle:
return True
prefix = f"{needle}-"
if not quant.startswith(prefix):
return False
suffix = quant[len(prefix) :]
if not suffix.endswith("bpw"):
return False
value = suffix[:-3]
return bool(value) and value.replace(".", "", 1).isdigit()
def _find_gguf_in_dir(dir_path: Path, gguf_variant: Optional[str]) -> Optional[Path]:
try:
ggufs = sorted(_iter_ggufs(dir_path))
except OSError:
return None
if not ggufs:
return None
needle = (gguf_variant or "").strip().lower()
if needle:
for path in ggufs:
try:
relative = path.relative_to(dir_path).as_posix()
except ValueError:
relative = path.name
if _variant_matches(relative, needle):
return path
return None
try:
return max(ggufs, key = lambda path: path.stat().st_size)
except OSError:
return ggufs[0]
def _chat_template_from_dir(
dir_path: Path,
gguf_variant: Optional[str] = None,
allow_roots: Optional[list[Path]] = None,
) -> Optional[str]:
def from_gguf() -> Optional[str]:
gguf = _find_gguf_in_dir(dir_path, gguf_variant)
if gguf is None or not _leaf_inside_allowlist(gguf, allow_roots):
return None
return read_gguf_chat_template(str(gguf))
# Sidecar tokenizer files (chat_template.jinja / tokenizer_config.json) are
# the model author's maintained template and supersede the GGUF's embedded
# copy, which can be stale. The variant only selects which GGUF to fall back
# to, so keep tokenizer-first precedence whether or not a variant is given.
return _chat_template_from_tokenizer_dir(dir_path, allow_roots) or from_gguf()
def read_default_chat_template(
model_name: str,
hf_token: Optional[str] = None,
gguf_variant: Optional[str] = None,
) -> Optional[str]:
if not isinstance(model_name, str) or not model_name.strip():
return None
name = model_name.strip()
if is_local_path(name):
try:
target = Path(normalize_path(name)).expanduser()
allow_roots = _build_browse_allowlist()
if not _is_path_inside_allowlist(target, allow_roots):
logger.debug("Refused chat template read outside allowed folders: %s", name)
return None
if name.lower().endswith(".gguf"):
# Prefer a maintained sidecar template (chat_template.jinja /
# tokenizer_config.json) next to the file over the GGUF's embedded
# copy, matching the tokenizer-first precedence used for directory
# and variant selections.
sidecar = _chat_template_from_tokenizer_dir(target.parent, allow_roots)
if sidecar:
return sidecar
return read_gguf_chat_template(str(target))
return _chat_template_from_dir(target, gguf_variant, allow_roots)
except Exception as exc:
logger.debug("Could not read local chat template for %s: %s", name, exc)
return None
if not _is_valid_repo_id(name):
return None
resolved = resolve_cached_repo_id_case(name)
try:
# Resolve within each cached revision, newest first. A revision's
# maintained sidecar (chat_template.jinja / tokenizer_config.json)
# supersedes its own embedded GGUF copy, but a newer revision must not be
# overridden by an older revision's sidecar, so precedence stays
# per-snapshot rather than searching all sidecars globally first.
for snapshot in iter_hf_cache_snapshots(resolved):
template = _chat_template_from_dir(snapshot, gguf_variant)
if template:
return template
except Exception as exc:
logger.debug("Could not read cached chat template for %s: %s", resolved, exc)
try:
from huggingface_hub import hf_hub_download
def _download_text(rel: str) -> Optional[str]:
try:
path = hf_hub_download(resolved, rel, token = hf_token)
return Path(path).read_text(encoding = "utf-8")
except Exception:
return None
for rel in _JINJA_TEMPLATE_PATHS:
template = _download_text(rel)
if template and template.strip():
return template
for rel in _TOKENIZER_CONFIG_PATHS:
raw = _download_text(rel)
if not raw:
continue
try:
config = json.loads(raw)
except Exception:
continue
template = _chat_template_from_tokenizer_config(config)
if template:
return template
for rel in _PROCESSOR_TEMPLATE_PATHS:
raw = _download_text(rel)
if not raw:
continue
try:
payload = json.loads(raw)
except Exception:
continue
template = _chat_template_from_processor_payload(payload)
if template:
return template
return None
except Exception as exc:
logger.debug("Could not fetch chat template for %s: %s", resolved, exc)
return None

46
studio/backend/tests/test_model_update_robustness.py
View file

@ -314,6 +314,7 @@ def test_cached_model_scan_keeps_local_safetensors_repo(monkeypatch, tmp_path):
file_name = "model.safetensors",
size_on_disk = 100,
blob_path = str(repo_path / "blobs" / "modelsha"),
blob_last_modified = 3_000.0,
),
]
)
@ -336,6 +337,51 @@ def test_cached_model_scan_keeps_local_safetensors_repo(monkeypatch, tmp_path):
assert rows[0]["repo_id"] == "Org/SafeTensorRepo"
assert rows[0]["model_format"] == "safetensors"
assert rows[0]["size_bytes"] == 100
assert rows[0]["last_modified"] == 3_000.0
def test_cached_gguf_scan_keeps_download_timestamp(monkeypatch, tmp_path):
repo_path = tmp_path / "models--Org--GgufRepo"
repo = SimpleNamespace(
repo_id = "Org/GgufRepo",
repo_type = "model",
repo_path = repo_path,
revisions = [
SimpleNamespace(
files = [
SimpleNamespace(
file_name = "model-Q4_K_M.gguf",
size_on_disk = 100,
blob_path = None,
blob_last_modified = 5_000.0,
),
]
)
],
)
monkeypatch.setattr(
CI,
"all_hf_cache_scans",
lambda: [SimpleNamespace(repos = [repo])],
)
monkeypatch.setattr(
CI.hf_cache_scan,
"is_gguf_repo_partial",
lambda *args, **kwargs: False,
)
monkeypatch.setattr(
CI,
"_gguf_variant_state_summary",
lambda _repo_id: (False, 0),
)
rows = CI._scan_cached_gguf()
assert len(rows) == 1
assert rows[0]["repo_id"] == "Org/GgufRepo"
assert rows[0]["model_format"] == "gguf"
assert rows[0]["size_bytes"] == 100
assert rows[0]["last_modified"] == 5_000.0
# ── hf_hub_download_with_xet_fallback force_download bypass (X2/F2) ───

162
studio/backend/tests/test_picker_service.py Normal file
View file

@ -0,0 +1,162 @@
# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
import json
from picker.service import (
_chat_template_from_dir,
_chat_template_from_tokenizer_config,
_chat_template_from_tokenizer_dir,
_find_gguf_in_dir,
_iter_ggufs,
read_default_chat_template,
validate_chat_template,
)
def test_iter_ggufs_skips_gguf_companions(tmp_path):
mtp_dir = tmp_path / "MTP"
mtp_dir.mkdir()
main = tmp_path / "model-Q8_0.gguf"
main.write_bytes(b"")
(tmp_path / "mmproj-F16.gguf").write_bytes(b"")
(tmp_path / "mtp-model-Q8_0.gguf").write_bytes(b"")
(mtp_dir / "model-Q8_0-MTP.gguf").write_bytes(b"")
(tmp_path / "model-Q8_0-be.gguf").write_bytes(b"")
assert _iter_ggufs(tmp_path) == [main]
def test_find_gguf_in_dir_matches_quant_label(tmp_path):
mtp_dir = tmp_path / "MTP"
mtp_dir.mkdir()
main = tmp_path / "model-Q8_0.gguf"
main.write_bytes(b"")
(mtp_dir / "model-Q8_0-MTP.gguf").write_bytes(b"")
(tmp_path / "model-Q4_K_M.gguf").write_bytes(b"")
assert _find_gguf_in_dir(tmp_path, "Q8_0") == main
assert _find_gguf_in_dir(tmp_path, "Q4_K") is None
def test_find_gguf_in_dir_without_variant_prefers_largest_model(tmp_path):
smaller = tmp_path / "a-model-Q4_K_M.gguf"
larger = tmp_path / "z-model-Q8_0.gguf"
smaller.write_bytes(b"0")
larger.write_bytes(b"00")
assert _find_gguf_in_dir(tmp_path, None) == larger
def test_find_gguf_in_dir_matches_bpw_variant_base_label(tmp_path):
target = tmp_path / "model-IQ4_XS-3.53bpw.gguf"
target.write_bytes(b"")
(tmp_path / "model-Q4_K_M.gguf").write_bytes(b"")
assert _find_gguf_in_dir(tmp_path, "IQ4_XS") == target
assert _find_gguf_in_dir(tmp_path, "IQ4_XS-3.53bpw") == target
assert _find_gguf_in_dir(tmp_path, "Q4_K") is None
def test_validate_chat_template_accepts_valid_and_empty():
assert validate_chat_template("{{ messages[0].content }}").valid is True
assert validate_chat_template("").valid is True
assert validate_chat_template(" ").valid is True
def test_validate_chat_template_reports_syntax_error_with_line():
result = validate_chat_template("{% if %}{% endif %}")
assert result.valid is False
assert result.error is not None
assert result.error.startswith("Line ")
def test_chat_template_from_tokenizer_config_reads_string():
assert _chat_template_from_tokenizer_config({"chat_template": "HELLO"}) == "HELLO"
assert _chat_template_from_tokenizer_config({"chat_template": " "}) is None
assert _chat_template_from_tokenizer_config({}) is None
def test_chat_template_from_tokenizer_config_prefers_named_default():
config = {
"chat_template": [
{"name": "tool_use", "template": "TOOL"},
{"name": "default", "template": "DEFAULT"},
]
}
assert _chat_template_from_tokenizer_config(config) == "DEFAULT"
def test_chat_template_from_tokenizer_config_falls_back_to_first_entry():
config = {
"chat_template": [
{"name": "tool_use", "template": "TOOL"},
{"name": "other", "template": "OTHER"},
]
}
assert _chat_template_from_tokenizer_config(config) == "TOOL"
def test_chat_template_from_tokenizer_dir_prefers_jinja_file(tmp_path):
(tmp_path / "chat_template.jinja").write_text("FROM_JINJA", encoding = "utf-8")
(tmp_path / "tokenizer_config.json").write_text(
json.dumps({"chat_template": "FROM_CONFIG"}), encoding = "utf-8"
)
assert _chat_template_from_tokenizer_dir(tmp_path) == "FROM_JINJA"
def test_chat_template_from_tokenizer_dir_reads_tokenizer_config(tmp_path):
(tmp_path / "tokenizer_config.json").write_text(
json.dumps({"chat_template": "FROM_CONFIG"}), encoding = "utf-8"
)
assert _chat_template_from_tokenizer_dir(tmp_path) == "FROM_CONFIG"
def test_chat_template_from_dir_without_variant_prefers_tokenizer(tmp_path):
(tmp_path / "tokenizer_config.json").write_text(
json.dumps({"chat_template": "FROM_CONFIG"}), encoding = "utf-8"
)
assert _chat_template_from_dir(tmp_path) == "FROM_CONFIG"
def test_chat_template_from_dir_with_variant_still_prefers_tokenizer(tmp_path, monkeypatch):
(tmp_path / "tokenizer_config.json").write_text(
json.dumps({"chat_template": "FROM_CONFIG"}), encoding = "utf-8"
)
(tmp_path / "model-Q4_K_M.gguf").write_bytes(b"")
monkeypatch.setattr("picker.service.read_gguf_chat_template", lambda _path: "FROM_GGUF")
# Selecting a variant must not flip precedence to the embedded GGUF template.
assert _chat_template_from_dir(tmp_path, "Q4_K_M") == "FROM_CONFIG"
def test_chat_template_from_dir_with_variant_falls_back_to_gguf(tmp_path, monkeypatch):
(tmp_path / "model-Q4_K_M.gguf").write_bytes(b"")
monkeypatch.setattr("picker.service.read_gguf_chat_template", lambda _path: "FROM_GGUF")
# With no tokenizer sidecar, the embedded GGUF template is still the fallback.
assert _chat_template_from_dir(tmp_path, "Q4_K_M") == "FROM_GGUF"
def test_chat_template_from_dir_returns_none_when_absent(tmp_path):
assert _chat_template_from_dir(tmp_path) is None
def test_read_default_chat_template_direct_gguf_prefers_sidecar(tmp_path, monkeypatch):
gguf = tmp_path / "model-Q4_K_M.gguf"
gguf.write_bytes(b"")
(tmp_path / "tokenizer_config.json").write_text(
json.dumps({"chat_template": "FROM_CONFIG"}), encoding = "utf-8"
)
monkeypatch.setattr("picker.service._build_browse_allowlist", lambda: [tmp_path])
monkeypatch.setattr("picker.service.read_gguf_chat_template", lambda _path: "FROM_GGUF")
# A directly selected .gguf file must prefer a maintained sidecar template
# over its embedded copy, matching directory/variant precedence.
assert read_default_chat_template(str(gguf)) == "FROM_CONFIG"
def test_read_default_chat_template_direct_gguf_falls_back_to_embedded(tmp_path, monkeypatch):
gguf = tmp_path / "model-Q4_K_M.gguf"
gguf.write_bytes(b"")
monkeypatch.setattr("picker.service._build_browse_allowlist", lambda: [tmp_path])
monkeypatch.setattr("picker.service.read_gguf_chat_template", lambda _path: "FROM_GGUF")
# With no sidecar next to the file, the embedded GGUF template is the fallback.
assert read_default_chat_template(str(gguf)) == "FROM_GGUF"

79
studio/backend/utils/models/gguf_metadata.py
View file

@ -50,6 +50,8 @@ _CACHE_MAX_ENTRIES = 4096
# keyed by (file cache key, wanted key). None = key absent / file unreadable.
_BOOL_CACHE: Dict[Tuple[_CacheKey, str], Optional[bool]] = {}
_STRING_CACHE: Dict[Tuple[_CacheKey, str], Optional[str]] = {}
# Native training context length (``{arch}.context_length``). None = absent /
# unreadable. Lets the UI show the real context ceiling before a model loads.
_CONTEXT_CACHE: Dict[_CacheKey, Optional[int]] = {}
@ -353,6 +355,83 @@ def _read_gguf_bool(path: str, wanted_key: str) -> Optional[bool]:
return result
def _parse_gguf_string(path: str, wanted_key: str) -> Optional[str]:
try:
with open(path, "rb") as f:
head = f.read(24)
if len(head) < 24:
return None
magic, _version, _tcount, kv_count = struct.unpack("<IIQQ", head)
if magic != _GGUF_MAGIC:
return None
for _ in range(kv_count):
try:
klen_bytes = f.read(8)
if len(klen_bytes) < 8:
break
klen = struct.unpack("<Q", klen_bytes)[0]
if klen > 1 << 20:
break
kbytes = f.read(klen)
if len(kbytes) < klen:
break
key = kbytes.decode("utf-8", "replace")
vt_bytes = f.read(4)
if len(vt_bytes) < 4:
break
vtype = struct.unpack("<I", vt_bytes)[0]
if key == wanted_key and vtype == 8:
slen_bytes = f.read(8)
if len(slen_bytes) < 8:
break
slen = struct.unpack("<Q", slen_bytes)[0]
if slen > 1 << 22:
break
sbytes = f.read(slen)
if len(sbytes) < slen:
break
return sbytes.decode("utf-8", "replace")
if not _skip_gguf_value(f, vtype):
break
except (struct.error, UnicodeDecodeError):
break
except OSError as e:
logger.debug(f"_parse_gguf_string: cannot open {path}: {e}")
return None
except Exception as e:
logger.debug(f"_parse_gguf_string: parse failure on {path}: {e}")
return None
return None
def _read_gguf_string(path: str, wanted_key: str) -> Optional[str]:
fkey = _cache_key(path)
if fkey is None:
return None
ckey = (fkey, wanted_key)
with _CACHE_LOCK:
if ckey in _STRING_CACHE:
return _STRING_CACHE[ckey]
result = _parse_gguf_string(path, wanted_key)
with _CACHE_LOCK:
while len(_STRING_CACHE) >= _CACHE_MAX_ENTRIES:
try:
_STRING_CACHE.pop(next(iter(_STRING_CACHE)))
except StopIteration:
break
_STRING_CACHE[ckey] = result
return result
def read_gguf_chat_template(path: str) -> Optional[str]:
template = _read_gguf_string(path, "tokenizer.chat_template")
if isinstance(template, str) and template.strip():
return template
return None
def read_mmproj_audio_capability(path: str) -> Optional[bool]:
"""``clip.has_audio_encoder`` from an mmproj GGUF (e.g. Gemma 4's
gemma4ua): ``True``/``False`` if present, ``None`` if absent/unreadable.

7
studio/frontend/src/app/routes/__root.tsx
View file

@ -195,9 +195,6 @@ function RootLayout() {
chatRuntime.setActiveThreadId(null);
chatRuntime.setActiveProjectId(null);
chatRuntime.setIncognito(false);
// Detach the staging UI but keep any in-flight download running, like Hub.
if (chatRuntime.pendingSelection)
chatRuntime.abandonStagedModel({ keepDownload: true });
void navigate({
to: "/chat",
search: { new: crypto.randomUUID() },
@ -220,10 +217,6 @@ function RootLayout() {
chatRuntime.setActiveProjectId(null);
chatRuntime.setActiveThreadId(null);
chatRuntime.setIncognito(false);
// Leaving chat must not kill an in-flight download: detach the staging UI
// but keep the transfer running in the manager, like a Hub download.
if (chatRuntime.pendingSelection)
chatRuntime.abandonStagedModel({ keepDownload: true });
}, [isChatRoute]);
return (

69
studio/frontend/src/components/assistant-ui/model-selector/remembered-load-settings.ts
View file

@ -1,69 +0,0 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
// Per-model pre-load inference settings, persisted in localStorage so the load
// dialog can offer "Remember settings for <model>".
const KEY = "unsloth_load_settings";
export interface RememberedLoadSettings {
contextLength: number | null;
kvCacheDtype: string | null;
speculativeType: string | null;
specDraftNMax: number | null;
tensorParallel: boolean;
}
// Storage key for a pick's remembered settings. The remembered knobs are
// VRAM-budget driven (context override, KV-cache dtype, tensor-parallel), so the
// right values differ per quant. An HF repo collapses all its GGUF variants into
// one `id`, so fold the variant in to scope settings per quant. Local .gguf
// paths key by their file path (already file-specific); native drag-drop files
// key by display label, so same-named files in different folders share an entry.
export function rememberedLoadSettingsKey(selection: {
id: string;
ggufVariant?: string | null;
}): string {
return selection.ggufVariant
? `${selection.id}::${selection.ggufVariant}`
: selection.id;
}
function readAll(): Record<string, RememberedLoadSettings> {
try {
return JSON.parse(localStorage.getItem(KEY) ?? "{}");
} catch {
return {};
}
}
function writeAll(all: Record<string, RememberedLoadSettings>) {
try {
localStorage.setItem(KEY, JSON.stringify(all));
} catch {
// Ignore quota / unavailable storage.
}
}
export function loadRememberedLoadSettings(
key: string,
): RememberedLoadSettings | null {
return readAll()[key] ?? null;
}
export function saveRememberedLoadSettings(
key: string,
settings: RememberedLoadSettings,
) {
const all = readAll();
all[key] = settings;
writeAll(all);
}
export function clearRememberedLoadSettings(key: string) {
const all = readAll();
if (key in all) {
delete all[key];
writeAll(all);
}
}

50
studio/frontend/src/features/chat/api/chat-adapter.ts
View file

@ -2,10 +2,7 @@
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
import { getAuthToken } from "@/features/auth";
import {
loadRememberedLoadSettings,
rememberedLoadSettingsKey,
} from "@/components/assistant-ui/model-selector/remembered-load-settings";
import { resolveInitialConfig } from "@/features/model-picker";
import { projectHasSources } from "@/features/rag/api/rag-api";
import { apiUrl } from "@/lib/api-base";
import { parseParamCountB } from "@/lib/model-size";
@ -1520,27 +1517,25 @@ async function autoLoadSmallestModel(): Promise<{
return false;
}
const currentStore = useChatRuntimeStore.getState();
const remembered = loadRememberedLoadSettings(
rememberedLoadSettingsKey({
id: candidate.id,
ggufVariant: candidate.ggufVariant,
}),
);
const { config } = resolveInitialConfig(candidate.id, candidate.ggufVariant);
const effectiveMaxSeqLength = resolveLoadMaxSeqLength({
modelId: candidate.id,
ggufVariant: candidate.ggufVariant,
isGguf: candidate.kind === "gguf",
customContextLength: remembered?.contextLength ?? null,
customContextLength: config.customContextLength,
ggufContextLength: null,
currentCheckpoint: currentStore.params.checkpoint,
activeGgufVariant: currentStore.activeGgufVariant,
maxSeqLength: candidate.maxSeqLength,
maxSeqLength: config.maxSeqLength ?? candidate.maxSeqLength,
presetSource: currentStore.activePresetSource,
});
const effectiveSpeculativeType =
remembered?.speculativeType ?? specSettings.speculativeType;
config.speculativeType ?? specSettings.speculativeType;
const effectiveSpecDraftNMax =
remembered?.specDraftNMax ?? specSettings.specDraftNMax;
config.specDraftNMax ?? specSettings.specDraftNMax;
const effectiveChatTemplateOverride = config.chatTemplateOverride?.trim()
? config.chatTemplateOverride
: null;
if (
!(await canAutoLoad({
model_path: candidate.id,
@ -1563,12 +1558,18 @@ async function autoLoadSmallestModel(): Promise<{
is_lora: false,
gguf_variant: candidate.ggufVariant,
trust_remote_code: trustRemoteCode,
cache_type_kv: remembered?.kvCacheDtype ?? null,
chat_template_override: effectiveChatTemplateOverride,
cache_type_kv: config.kvCacheDtype,
speculative_type: effectiveSpeculativeType,
spec_draft_n_max: effectiveSpecDraftNMax,
tensor_parallel: remembered?.tensorParallel ?? false,
tensor_parallel: config.tensorParallel,
});
saveSpeculativeType(effectiveSpeculativeType);
// Only persist the global preference when the value came from the global
// settings. A per-model config's choice must stay load-local, or autoloading
// a remembered model on startup would rewrite the global default.
if (config.speculativeType == null) {
saveSpeculativeType(effectiveSpeculativeType);
}
useChatRuntimeStore
.getState()
.setCheckpoint(candidate.id, candidate.ggufVariant ?? undefined);
@ -1578,6 +1579,9 @@ async function autoLoadSmallestModel(): Promise<{
);
store.setParams({
...store.params,
...(candidate.kind === "gguf"
? {}
: { maxSeqLength: effectiveMaxSeqLength }),
maxTokens:
candidate.kind === "gguf"
? loadResp.context_length ?? 131072
@ -1614,8 +1618,11 @@ async function autoLoadSmallestModel(): Promise<{
tensorParallel: loadResp.tensor_parallel ?? false,
loadedTensorParallel: loadResp.tensor_parallel ?? false,
defaultChatTemplate: loadResp.chat_template ?? null,
chatTemplateOverride: null,
loadedChatTemplateOverride: null,
chatTemplateOverride: effectiveChatTemplateOverride,
loadedChatTemplateOverride: effectiveChatTemplateOverride,
// Retain the saved requested context so re-saving the config keeps the
// override; null stays null (auto/VRAM-fit).
customContextLength: config.customContextLength,
loadedIsMultimodal: isMultimodalResponse(loadResp),
loadedIsDiffusion: loadResp.is_diffusion ?? false,
...resolveLoadedSpeculativeSettings(loadResp),
@ -1634,8 +1641,9 @@ async function autoLoadSmallestModel(): Promise<{
tensorParallel: loadResp.tensor_parallel ?? false,
loadedTensorParallel: loadResp.tensor_parallel ?? false,
defaultChatTemplate: loadResp.chat_template ?? null,
chatTemplateOverride: null,
loadedChatTemplateOverride: null,
chatTemplateOverride: effectiveChatTemplateOverride,
loadedChatTemplateOverride: effectiveChatTemplateOverride,
customContextLength: null,
...resolveLoadedSpeculativeSettings(loadResp),
loadedIsMultimodal: isMultimodalResponse(loadResp),
loadedIsDiffusion: loadResp.is_diffusion ?? false,

517
studio/frontend/src/features/chat/chat-page.tsx
View file

@ -2,16 +2,18 @@
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
import {
applyModelLoadConfigToRuntime,
currentRuntimePerModelConfig,
type DeletedModelRef,
type ExternalModelOption,
type LoraModelOption,
type ModelOption,
ModelSelector,
} from "@/components/assistant-ui/model-selector";
import {
loadRememberedLoadSettings,
rememberedLoadSettingsKey,
} from "@/components/assistant-ui/model-selector/remembered-load-settings";
type ModelSelectorChangeMeta,
type PerModelConfig,
resolveInitialConfig,
SidebarModelConfig,
} from "@/features/model-picker";
import { ProjectComposer, Thread } from "@/components/assistant-ui/thread";
import { CopyableErrorChip } from "@/components/ui/copyable-error-chip";
import {
@ -27,10 +29,10 @@ import {
} from "@/components/ui/resizable";
import { useSidebar } from "@/components/ui/sidebar";
import { Tooltip, TooltipContent } from "@/components/ui/tooltip";
import { useLatestRef } from "@/features/hub/hooks/use-latest-ref";
import {
DOWNLOAD_KIND,
downloadManager,
useRepoDownload,
} from "@/features/hub/download-manager";
import {
type NativeIntent,
@ -93,7 +95,6 @@ import {
renameChatItem,
useChatSidebarItems,
} from "./hooks/use-chat-sidebar-items";
import { useStagedModelPreparation } from "./hooks/use-staged-model-preparation";
import {
clearTrainingCompareHandoff,
getTrainingCompareHandoff,
@ -128,10 +129,8 @@ import {
hasGgufSource,
isDownloadableHubRepo,
loadOptionalBool,
pendingSelectionMatches,
useChatRuntimeStore,
} from "./stores/chat-runtime-store";
import type { PendingModelSelection } from "./stores/chat-runtime-store";
import { useChatPreferencesStore } from "./stores/chat-preferences-store";
import { useExternalProvidersStore } from "./stores/external-providers-store";
import { buildChatTourSteps } from "./tour";
@ -385,6 +384,7 @@ type CompareModelSelection = {
id: string;
isLora: boolean;
ggufVariant?: string;
config?: PerModelConfig;
};
function modelMatchesDeleted(
@ -645,6 +645,8 @@ function GeneralCompareHeader({
loraModels,
externalModels,
value,
selectedConfig,
selectedGgufVariant,
onValueChange,
onFoldersChange,
onModelsChange,
@ -655,9 +657,11 @@ function GeneralCompareHeader({
loraModels: LoraModelOption[];
externalModels: ExternalModelOption[];
value: string;
selectedConfig?: PerModelConfig | null;
selectedGgufVariant?: string | null;
onValueChange: (
id: string,
meta: { isLora: boolean; ggufVariant?: string },
meta: ModelSelectorChangeMeta,
) => void;
onFoldersChange?: () => void;
onModelsChange?: (deletedModel?: DeletedModelRef) => void;
@ -684,6 +688,8 @@ function GeneralCompareHeader({
loraModels={loraModels}
externalModels={externalModels}
value={value}
selectedConfig={selectedConfig}
selectedGgufVariant={selectedGgufVariant}
onValueChange={onValueChange}
onFoldersChange={onFoldersChange}
onModelsChange={onModelsChange}
@ -811,11 +817,14 @@ const GeneralCompareContent = memo(function GeneralCompareContent({
loraModels={loraModels}
externalModels={externalModels}
value={model1.id}
selectedConfig={model1.config}
selectedGgufVariant={model1.ggufVariant}
onValueChange={(id, meta) =>
setModel1({
id,
isLora: meta.isLora,
ggufVariant: meta.ggufVariant,
config: meta.config,
})
}
onFoldersChange={onFoldersChange}
@ -838,11 +847,14 @@ const GeneralCompareContent = memo(function GeneralCompareContent({
loraModels={loraModels}
externalModels={externalModels}
value={model2.id}
selectedConfig={model2.config}
selectedGgufVariant={model2.ggufVariant}
onValueChange={(id, meta) =>
setModel2({
id,
isLora: meta.isLora,
ggufVariant: meta.ggufVariant,
config: meta.config,
})
}
onFoldersChange={onFoldersChange}
@ -1236,6 +1248,13 @@ export function validateChatSearch(search: Record<string, unknown>): ChatSearch
};
}
type PendingHubAutoLoad = {
selection: SelectedModelInput;
contextKey: string;
originCheckpoint: string;
originGgufVariant: string | null;
};
// `search` comes from RootLayout (not useSearch) so ChatPage stays mounted off-route
// (keeping an in-flight generation alive), frozen to the last /chat search. `active`
// is false off-route: close body-portaled surfaces and stop route-specific listeners
@ -1248,30 +1267,6 @@ export function ChatPage({
const settingsOpen = useChatRuntimeStore((s) => s.settingsPanelOpen);
const setSettingsOpen = useChatRuntimeStore((s) => s.setSettingsPanelOpen);
// Deferred-load staging: downloads a staged GGUF (if needed) and reads its
// header context so the sheet can show the context slider before the load.
// autoLoad picks instead load the cached file as soon as the download ends;
// selectModel is defined below, so the load runs through a ref.
const autoLoadStagedRef = useRef<
((pending: PendingModelSelection) => void) | null
>(null);
const stagedDownload = useStagedModelPreparation({
onAutoLoad: (pending) => autoLoadStagedRef.current?.(pending),
});
// Abandon a staged pick: the store action cancels its in-flight download and
// reverts the edited knobs, so nothing lingers after the user walks away.
const abandonStaged = useCallback(() => {
useChatRuntimeStore.getState().abandonStagedModel();
}, []);
// Detach a staged pick on navigation without cancelling its download: the
// transfer keeps running in the manager and lands in cache, like Hub.
const detachStaged = useCallback(() => {
useChatRuntimeStore.getState().abandonStagedModel({ keepDownload: true });
}, []);
// Tracks whether the chat page is still mounted, so a staged-load failure that
// resolves after the user left chat doesn't resurrect the abandoned pick.
const mountedRef = useRef(true);
useEffect(() => () => void (mountedRef.current = false), []);
const incognito = useChatRuntimeStore((s) => s.incognito);
const setIncognito = useChatRuntimeStore((s) => s.setIncognito);
const incognitoLabel = incognito
@ -1363,6 +1358,9 @@ export function ChatPage({
const ggufContextLength = useChatRuntimeStore(
(state) => state.ggufContextLength,
);
const ggufNativeContextLength = useChatRuntimeStore(
(state) => state.ggufNativeContextLength,
);
const contextUsage = useChatRuntimeStore((state) => state.contextUsage);
const modelsFromStore = useChatRuntimeStore((state) => state.models);
const lorasFromStore = useChatRuntimeStore((state) => state.loras);
@ -1440,37 +1438,82 @@ export function ChatPage({
refreshRef.current = refresh;
selectModelRef.current = selectModel;
}, [refresh, selectModel]);
// Load a cached autoLoad pick once its download finishes. The sheet was never
// opened, so on a load failure just drop the orphaned staged knobs. The knobs
// were already seeded on stage, so keepSpeculative only when a config was
// saved -- otherwise the standing speculative preference should win.
autoLoadStagedRef.current = (pending) => {
const remembered = loadRememberedLoadSettings(
rememberedLoadSettingsKey(pending),
);
void selectModel({
...pending,
isDownloaded: true,
forceReload: true,
keepSpeculative: remembered != null,
throwOnError: true,
}).catch(() => {
const store = useChatRuntimeStore.getState();
// selectModel only clears pendingSelection on success, so a failed
// auto-load leaves our staged pick (and its edited load knobs) behind.
// Abandon it when it is still the active stage; otherwise just revert the
// settings if the stage was already cleared by something else.
if (pendingSelectionMatches(store.pendingSelection, pending)) {
store.abandonStagedModel();
} else if (!store.pendingSelection) {
store.resetModelSettingsToLoaded();
}
});
};
const rememberedConfigFor = useCallback(
(selection: {
id: string;
ggufVariant?: string | null;
source?: string;
}) => {
if (selection.source === "external") return null;
const resolved = resolveInitialConfig(selection.id, selection.ggufVariant);
return resolved.remembered ? resolved.config : null;
},
[],
);
const isExternalModel = useMemo(
() => isExternalModelId(inferenceParams.checkpoint),
[inferenceParams.checkpoint],
);
const runtimeCustomContextLength = useChatRuntimeStore(
(s) => s.customContextLength,
);
const runtimeKvCacheDtype = useChatRuntimeStore((s) => s.kvCacheDtype);
const runtimeSpeculativeType = useChatRuntimeStore((s) => s.speculativeType);
const runtimeSpecDraftNMax = useChatRuntimeStore((s) => s.specDraftNMax);
const runtimeTensorParallel = useChatRuntimeStore((s) => s.tensorParallel);
const runtimeChatTemplateOverride = useChatRuntimeStore(
(s) => s.chatTemplateOverride,
);
const activeModelConfig = useMemo<PerModelConfig | null>(() => {
if (!inferenceParams.checkpoint || isExternalModel) return null;
const activeModelIsGguf =
activeGgufVariant != null ||
ggufContextLength != null ||
inferenceParams.checkpoint.toLowerCase().endsWith(".gguf");
return {
customContextLength: runtimeCustomContextLength ?? null,
maxSeqLength: activeModelIsGguf ? null : inferenceParams.maxSeqLength,
kvCacheDtype: runtimeKvCacheDtype ?? null,
speculativeType: runtimeSpeculativeType ?? "auto",
specDraftNMax: runtimeSpecDraftNMax ?? null,
tensorParallel: runtimeTensorParallel ?? false,
chatTemplateOverride: runtimeChatTemplateOverride ?? null,
};
}, [
inferenceParams.checkpoint,
inferenceParams.maxSeqLength,
isExternalModel,
activeGgufVariant,
ggufContextLength,
runtimeCustomContextLength,
runtimeKvCacheDtype,
runtimeSpeculativeType,
runtimeSpecDraftNMax,
runtimeTensorParallel,
runtimeChatTemplateOverride,
]);
const activeModelIsGguf = useMemo(() => {
const checkpoint = inferenceParams.checkpoint;
if (!checkpoint || isExternalModel) return false;
return (
activeGgufVariant != null ||
ggufContextLength != null ||
checkpoint.toLowerCase().endsWith(".gguf")
);
}, [
inferenceParams.checkpoint,
isExternalModel,
activeGgufVariant,
ggufContextLength,
]);
const activeModelIsLora = useMemo(() => {
const checkpoint = inferenceParams.checkpoint;
if (!checkpoint || isExternalModel) return false;
const model = modelsFromStore.find((entry) => entry.id === checkpoint);
if (model) return model.isLora;
const lora = lorasFromStore.find((entry) => entry.id === checkpoint);
return lora?.exportType === "lora";
}, [inferenceParams.checkpoint, isExternalModel, modelsFromStore, lorasFromStore]);
const reasoningEnabled = useChatRuntimeStore((s) => s.reasoningEnabled);
const reasoningStyle = useChatRuntimeStore((s) => s.reasoningStyle);
const reasoningEffort = useChatRuntimeStore((s) => s.reasoningEffort);
@ -1783,75 +1826,21 @@ export function ChatPage({
closeArtifactSurface();
}, [activeThreadId, closeArtifactSurface, selectedArtifact, view]);
// Abandon a staged (not-yet-loaded) pick when the chat context actually
// changes — switching threads, leaving single view, or starting a new chat /
// project — so a stale Load button can't resurface in a different context.
// New Chat keeps activeThreadId null and only bumps the `new` search nonce, so
// the key includes the route identity, not just the thread. Mirrors the
// incognito reset pattern. (Route exit is handled in __root.tsx, which runs
// after this unmounts.) Clear only on a real change, never on mount: staging
// from the Hub sets pendingSelection then navigates here, and clearing on
// mount would wipe it. Comparing the previous context (rather than a first-run
// flag) is also safe under StrictMode's double-invoke and component remounts.
const chatContextKey = `${view.mode}|${activeThreadId ?? ""}|${search.new ?? ""}|${search.project ?? ""}`;
const chatContextKeyRef = useLatestRef(chatContextKey);
const prevChatContextRef = useRef<string | null>(null);
useEffect(() => {
const prev = prevChatContextRef.current;
prevChatContextRef.current = chatContextKey;
if (prev === null || prev === chatContextKey) return;
detachStaged();
}, [chatContextKey, detachStaged]);
const hasActiveModel = Boolean(inferenceParams.checkpoint);
// Load immediately, or — when "Load on selection" is off — stage the pick so
// its load options can be set first. Shared by the main selector, native
// drag-drop/picker, and the dropped-file chip (the Hub stages via the store).
const chatContextKey = `${view.mode}|${activeThreadId ?? ""}|${search.new ?? ""}|${search.project ?? ""}`;
const [pendingHubAutoLoad, setPendingHubAutoLoad] =
useState<PendingHubAutoLoad | null>(null);
const stageOrLoad = useCallback(
async (selection: SelectedModelInput) => {
const store = useChatRuntimeStore.getState();
// An un-cached HF repo (GGUF variant or a full non-GGUF snapshot) downloads
// through the manager first (global indicator), then auto-loads. Everything
// else -- cached picks, local/native files, LoRA, external -- loads now.
const wantManagerDownload =
isDownloadableHubRepo(selection) && !selection.isDownloaded;
if (
(!hasGgufSource(selection) && !wantManagerDownload) ||
(store.loadOnSelection && selection.isDownloaded)
) {
// Detach any staged pick first so its edited knobs (e.g. a custom
// context length) don't leak into this immediate load -- resolveLoad
// reads customContextLength before checking the target is GGUF. Detach
// (not abandon) keeps its download running.
detachStaged();
// Load-on-selection skips the sheet, so seed the saved knobs here the
// way the sheet's restore effect would; the switch would otherwise reset
// the remembered speculative choice (keepSpeculative below prevents it).
const remembered = hasGgufSource(selection)
? loadRememberedLoadSettings(rememberedLoadSettingsKey(selection))
: null;
if (remembered) store.applyRememberedLoadSettings(remembered);
await selectModel(
remembered ? { ...selection, keepSpeculative: true } : selection,
);
return;
}
// Loads can't queue behind each other, but a download is independent: if
// the pick needs downloading, start it in the manager so it runs alongside
// the load. Nothing to download (already on device) just waits.
if (store.modelLoading) {
// Both an uncached non-GGUF snapshot (wantManagerDownload) and an
// uncached remote GGUF quant download through the manager, so either can
// run in the background while another model loads. wantManagerDownload
// excludes GGUF by design, so the GGUF case is checked separately.
const wantBackgroundDownload =
wantManagerDownload ||
(selection.source === "hub" &&
hasGgufSource(selection) &&
!selection.isDownloaded);
// The model currently loading already downloads as part of its own load
// (the /load flow fetches before setting the checkpoint), so re-picking
// it must not kick off a second transfer against the same cache.
const isLoadingThisPick =
!!loadingModel &&
normalizeModelRef(loadingModel.id) ===
@ -1862,11 +1851,6 @@ export function ChatPage({
description: "It's downloading as part of the load in progress.",
});
} else if (wantBackgroundDownload) {
// Only claim the download started once a job is actually created. A
// transport conflict records state that is only resolvable from the
// Hub download card, so point the user there instead of showing a
// success toast for a transfer that never began; "busy" and "error"
// already surface their own toasts.
const outcome = await downloadManager.requestStart({
kind: DOWNLOAD_KIND.MODEL,
repoId: selection.id,
@ -1883,6 +1867,11 @@ export function ChatPage({
description:
"An earlier partial download used a different transport. Open the Hub tab to resume or restart it.",
});
} else if (outcome === "busy") {
toast.info("Download already in progress", {
description:
"Another download for this model is still running. Reselect it once that finishes to load it.",
});
}
} else {
toast.info("Another model is already loading", {
@ -1891,23 +1880,118 @@ export function ChatPage({
}
return;
}
// Detach the prior staged pick (keeping its download) before rebinding, so
// a second pick downloads alongside the first instead of cancelling it.
detachStaged();
store.stageModel({
id: selection.id,
isLora: selection.isLora,
ggufVariant: selection.ggufVariant,
isDownloaded: selection.isDownloaded,
expectedBytes: selection.expectedBytes,
nativePathToken: selection.nativePathToken,
isGguf: selection.isGguf,
isHubRepo: wantManagerDownload || undefined,
autoLoad: store.loadOnSelection,
const wantManagerStage =
wantManagerDownload ||
(selection.source === "hub" &&
hasGgufSource(selection) &&
!selection.isDownloaded);
if (wantManagerStage) {
setPendingHubAutoLoad({
selection,
contextKey: chatContextKey,
originCheckpoint: store.params.checkpoint,
originGgufVariant: store.activeGgufVariant,
});
return;
}
setPendingHubAutoLoad(null);
const previousConfig = currentRuntimePerModelConfig({
includeMaxSeqLength: true,
});
const hasAppliedConfig = applyModelLoadConfigToRuntime(
selection.config ?? rememberedConfigFor(selection),
);
await selectModel({
...selection,
...(hasAppliedConfig ? { keepSpeculative: true } : {}),
previousConfig,
});
},
[detachStaged, selectModel, loadingModel],
[selectModel, loadingModel, rememberedConfigFor, chatContextKey],
);
useRepoDownload({
kind: DOWNLOAD_KIND.MODEL,
repoId: pendingHubAutoLoad?.selection.id ?? "__hub_autoload_idle__",
activeVariant: pendingHubAutoLoad?.selection.ggufVariant ?? null,
onComplete: (variant) => {
const pending = pendingHubAutoLoad;
if (
!pending ||
(pending.selection.ggufVariant ?? null) !== (variant ?? null)
) {
return;
}
setPendingHubAutoLoad(null);
const store = useChatRuntimeStore.getState();
if (
!active ||
pending.contextKey !== chatContextKey ||
normalizeModelRef(pending.originCheckpoint) !==
normalizeModelRef(store.params.checkpoint) ||
pending.originGgufVariant !== store.activeGgufVariant
) {
return;
}
void stageOrLoad({ ...pending.selection, isDownloaded: true });
},
onError: (variant) => {
if (
pendingHubAutoLoad &&
(pendingHubAutoLoad.selection.ggufVariant ?? null) === (variant ?? null)
) {
setPendingHubAutoLoad(null);
}
},
onCancelled: (variant) => {
if (
pendingHubAutoLoad &&
(pendingHubAutoLoad.selection.ggufVariant ?? null) === (variant ?? null)
) {
setPendingHubAutoLoad(null);
}
},
});
useEffect(() => {
const pending = pendingHubAutoLoad;
if (!pending) return;
let active = true;
void (async () => {
const outcome = await downloadManager.requestStart({
kind: DOWNLOAD_KIND.MODEL,
repoId: pending.selection.id,
variant: pending.selection.ggufVariant ?? null,
expectedBytes: pending.selection.expectedBytes ?? 0,
});
if (!active) return;
if (outcome === "started") {
toast.info("Downloading model", {
description: "It'll load automatically once the download finishes.",
});
return;
}
if (outcome === "conflict") {
// Keep pendingHubAutoLoad bound so this surface's cleanup does not wipe
// the conflict just recorded by requestStart (which the toast points the
// user to); resolving it from the Hub completes the download and this
// surface's onComplete auto-loads, mirroring the "started" branch.
toast.info("Resume this download from the Hub", {
description:
"An earlier partial download used a different transport. Open the Hub tab to resume or restart it.",
});
return;
}
if (outcome === "busy") {
toast.info("Download already in progress", {
description:
"Another download for this model is still running. Reselect it once that finishes to load it.",
});
}
setPendingHubAutoLoad((current) => (current === pending ? null : current));
})();
return () => {
active = false;
};
}, [pendingHubAutoLoad]);
const loadNativeModelIntent = useCallback(
async (intent: NativeIntent, loadingDescription: string) => {
const label =
@ -1920,6 +2004,11 @@ export function ChatPage({
forceReload: true,
throwOnError: true,
});
// Record when this file lease expires so a later reload can prompt
// re-selection instead of reusing a token the host has already pruned.
useChatRuntimeStore.setState({
activeNativePathExpiresAtMs: intent.path.expiresAtMs ?? null,
});
useNativeIntentStore.getState().clearModelIntent(intent.id);
},
[stageOrLoad],
@ -1965,28 +2054,20 @@ export function ChatPage({
const handleCheckpointChange = useCallback(
(
value: string,
meta?: {
source?: string;
isLora: boolean;
ggufVariant?: string;
isDownloaded?: boolean;
expectedBytes?: number;
isGguf?: boolean;
},
meta?: ModelSelectorChangeMeta,
) => {
const store = useChatRuntimeStore.getState();
const currentCheckpoint = store.params.checkpoint;
const currentVariant = store.activeGgufVariant;
if (
!value ||
(value === currentCheckpoint &&
(meta?.ggufVariant ?? null) === (currentVariant ?? null))
)
if (!value) return;
setPendingHubAutoLoad(null);
const isSameLoadedModel =
value === currentCheckpoint &&
(meta?.ggufVariant ?? null) === (currentVariant ?? null);
if (isSameLoadedModel && !meta?.forceReload) {
return;
}
if (meta?.source === "external" || isExternalModelId(value)) {
// Switching to an external model abandons any staged local pick: cancel
// its download too (setCheckpoint below only clears the pending + knobs).
abandonStaged();
const selectedExternal = parseExternalModelId(value);
const selectedProvider = selectedExternal
? externalProvidersForChat.find(
@ -2158,19 +2239,17 @@ export function ChatPage({
source: meta?.source,
isLora: meta?.isLora,
ggufVariant: meta?.ggufVariant,
isDownloaded: meta?.isDownloaded,
isDownloaded: meta?.isDownloaded || isSameLoadedModel,
expectedBytes: meta?.expectedBytes,
isGguf: meta?.isGguf,
config: meta?.config,
nativePathToken: meta?.nativePathToken,
forceReload: isSameLoadedModel || undefined,
};
// "Load on selection" off: stage the model and open settings so its
// load knobs (tensor parallel, context length…) can be set, then it
// loads once via the sheet's Load button. The currently loaded model
// stays put until the user commits.
await stageOrLoad(selection);
})();
},
[
abandonStaged,
activeThreadId,
externalProvidersForChat,
modelsFromStore,
@ -2178,6 +2257,44 @@ export function ChatPage({
view,
],
);
const handleReloadActiveModel = useCallback(
(config: PerModelConfig) => {
const checkpoint = inferenceParams.checkpoint;
if (!checkpoint) return;
const runtime = useChatRuntimeStore.getState();
const nativeToken = runtime.activeNativePathToken;
const nativeExpiry = runtime.activeNativePathExpiresAtMs;
// A file-picked GGUF is reachable only via its native path token, which
// the desktop host prunes after a TTL. Reusing an expired token makes the
// reload fail with an opaque error, so prompt the user to re-select the
// file instead.
if (nativeToken && nativeExpiry != null && Date.now() >= nativeExpiry) {
toast.error("This local model file's access has expired.", {
description: "Re-select the model file to reload it.",
});
return;
}
handleCheckpointChange(checkpoint, {
source: "local",
isLora: activeModelIsLora,
ggufVariant: activeGgufVariant ?? undefined,
// Without the native token the reload validates the display label as a
// repo and fails.
nativePathToken: nativeToken ?? undefined,
isGguf: activeModelIsGguf,
isDownloaded: true,
config,
forceReload: true,
});
},
[
inferenceParams.checkpoint,
activeGgufVariant,
activeModelIsLora,
activeModelIsGguf,
handleCheckpointChange,
],
);
const handleEject = useCallback(() => {
void (async () => {
if (await ejectModel()) {
@ -2580,6 +2697,8 @@ export function ChatPage({
externalModels={externalModels}
value={inferenceParams.checkpoint}
activeGgufVariant={activeGgufVariant}
activeModelConfig={activeModelConfig}
activeGgufContextLength={ggufContextLength}
onValueChange={handleCheckpointChange}
onEject={handleEject}
onFoldersChange={refreshLocalModels}
@ -2633,7 +2752,12 @@ export function ChatPage({
<NativeModelChip
intent={pendingNativeModelIntent}
nativeReadsDisabled={!nativePathLeasesSupported}
onLoad={(selection) => stageOrLoad(selection)}
onLoad={() =>
loadNativeModelIntent(
pendingNativeModelIntent,
"Loading selected local GGUF model.",
)
}
/>
) : null}
{loadingModel && loadToastDismissed ? (
@ -2790,13 +2914,22 @@ export function ChatPage({
open={active && settingsOpen}
onOpenChange={(open) => {
setSettingsOpen(open);
// Closing the sheet abandons a staged (not-yet-loaded) pick: cancel its
// download and revert the staged knobs so nothing lingers as a dirty
// edit (or a background download) on the loaded model.
if (!open) abandonStaged();
}}
params={inferenceParams}
onParamsChange={setInferenceParams}
modelConfig={
view.mode !== "compare" && activeModelConfig && !modelLoading ? (
<SidebarModelConfig
modelId={inferenceParams.checkpoint}
ggufVariant={activeGgufVariant ?? null}
isGguf={activeModelIsGguf}
nativeContextLength={ggufNativeContextLength}
loadedContextLength={ggufContextLength}
loadedConfig={activeModelConfig}
onReload={handleReloadActiveModel}
/>
) : null
}
isExternalModel={isExternalModel}
providerCapabilities={activeProviderCapabilities}
activeExternalProvider={activeExternalProvider}
@ -2808,62 +2941,6 @@ export function ChatPage({
);
}}
externalProviderType={activeExternalProviderType}
loadingModel={loadingModel}
onReloadModel={() => {
const state = useChatRuntimeStore.getState();
if (state.params.checkpoint) {
selectModel({
id: state.params.checkpoint,
ggufVariant: state.activeGgufVariant ?? undefined,
forceReload: true,
isDownloaded: true,
loadingDescription: "Reloading with updated chat template.",
});
}
}}
onLoadPendingModel={() => {
const pending = useChatRuntimeStore.getState().pendingSelection;
if (!pending) return;
const keyAtLoad = chatContextKey;
// forceReload: the staged model isn't loaded yet, so bypass the
// same-checkpoint dedupe. keepSpeculative: honor the speculative mode
// set on the sidebar.
void selectModel({
...pending,
forceReload: true,
keepSpeculative: true,
throwOnError: true,
}).catch(() => {
// Recoverable failure (expired token, gated repo, OOM…): the pick is
// cleared only on success, so it normally stays staged with edited
// knobs intact — nothing to restore.
const store = useChatRuntimeStore.getState();
// Still staged (this pick, or a newer one queued meanwhile): leave it.
if (store.pendingSelection) return;
// Cleared mid-load (sheet closed / switched chats). Re-stage only if
// the staged-load is still wanted: same chat context, sheet still
// open, page still mounted.
const stillWanted =
mountedRef.current &&
store.settingsPanelOpen &&
chatContextKeyRef.current === keyAtLoad;
if (stillWanted) {
store.setPendingSelection(pending);
} else {
// Abandoned (closed the sheet / switched chats / left chat): drop
// the orphaned staged knob edits so they don't linger as dirty
// settings over the loaded model.
store.resetModelSettingsToLoaded();
}
});
}}
stagedDownloadFraction={stagedDownload.progress?.fraction ?? null}
onCancelStagedDownload={() =>
stagedDownload.cancelDownload(
useChatRuntimeStore.getState().pendingSelection?.ggufVariant ??
null,
)
}
/>
</div>
</ChatActiveContext.Provider>

947
studio/frontend/src/features/chat/chat-settings-sheet.tsx
View file

File diff suppressed because it is too large Load diff

105
studio/frontend/src/features/chat/hooks/use-chat-model-runtime.ts
View file

@ -30,7 +30,6 @@ import {
import { formatEta, formatRate } from "../utils/format-transfer";
import {
isLocalModelPath,
pendingSelectionMatches,
readPersistedSpeculativeType,
resolveToolsEnabledOnLoad,
saveSpeculativeType,
@ -53,7 +52,10 @@ import {
isMultimodalResponse,
} from "../types/api";
import { isExternalModelId } from "../external-providers";
import { cancelStagedModelDownload } from "@/features/hub";
import {
applyPerModelConfigToRuntime,
type PerModelConfig,
} from "@/features/model-picker";
import type {
ChatLoraSummary,
ChatModelSummary,
@ -78,9 +80,10 @@ export type SelectedModelInput = {
isGguf?: boolean;
throwOnError?: boolean;
/** Keep the current speculative-decoding choice across the model switch
* instead of resetting it to the standing preference. Set by the deferred
* ("Load on selection") Load, where the user picked it for this model. */
* instead of resetting it to the standing preference. */
keepSpeculative?: boolean;
config?: PerModelConfig;
previousConfig?: PerModelConfig;
};
// Approved fingerprints by checkpoint, so a rollback after a failed switch can resend
@ -455,27 +458,11 @@ export function useChatModelRuntime() {
typeof selection === "string" ? false : selection.throwOnError ?? false;
const keepSpeculative =
typeof selection === "string" ? false : selection.keepSpeculative ?? false;
// Picking/loading any model abandons a staged (deferred) selection.
// Before the early-returns below so even a no-op re-select clears the
// stage.
const staged = useChatRuntimeStore.getState().pendingSelection;
if (staged) {
// Loading a DIFFERENT model abandons this stage. Loading the staged pick
// ITSELF keeps it so the sidebar can show its load settings (context, KV
// cache, …) during the load. Cleared on success below; on failure it's
// left staged so the user can retry (see onLoadPendingModel's catch).
const loadingStagedPick = pendingSelectionMatches(staged, {
id: modelId,
ggufVariant,
nativePathToken,
});
if (!loadingStagedPick) {
cancelStagedModelDownload(staged);
useChatRuntimeStore.getState().setPendingSelection(null);
}
}
const currentVariant = useChatRuntimeStore.getState().activeGgufVariant;
if (!forceReload && (!modelId || (params.checkpoint === modelId && (ggufVariant ?? null) === (currentVariant ?? null)))) {
if (typeof selection !== "string" && selection.previousConfig) {
applyPerModelConfigToRuntime(selection.previousConfig);
}
return;
}
// A load is already in flight. If it's this exact pick (id + GGUF variant +
@ -488,6 +475,9 @@ export function useChatModelRuntime() {
// every entry point is covered, not just the staged Load button.
const inFlightLoad = loadingModelRef.current;
if (inFlightLoad) {
if (typeof selection !== "string" && selection.previousConfig) {
applyPerModelConfigToRuntime(selection.previousConfig);
}
const loadingSamePick =
inFlightLoad.id === modelId &&
(inFlightLoad.ggufVariant ?? null) === (ggufVariant ?? null) &&
@ -582,20 +572,24 @@ export function useChatModelRuntime() {
previousModel?.isGguf === true
|| previousVariant != null
|| (previousCheckpoint?.toLowerCase().endsWith(".gguf") ?? false);
// Roll back to the previous model's own context. previousConfig was
// snapshotted before this load pre-applied the next model's config to
// the shared store, so params.maxSeqLength may already be the next
// model's value; fall back to it only when no snapshot exists.
const previousMaxSeqLength =
(typeof selection !== "string"
? selection.previousConfig?.maxSeqLength
: null) ?? maxSeqLength;
const rollbackMaxSeqLength = previousIsGguf
? (stateBeforeUnload.ggufContextLength ?? 0)
: maxSeqLength;
: previousMaxSeqLength;
const hfToken = stateBeforeUnload.hfToken || null;
const previousModelRequiresTrustRemoteCode =
stateBeforeUnload.modelRequiresTrustRemoteCode;
const previousActiveNativePathToken =
stateBeforeUnload.activeNativePathToken;
// Snapshot the load settings at click time, before the awaits below
// (validation, the trust dialog, unload). For a staged Load these knobs
// stay editable and a sheet-close revert (abandonStagedModel) can fire
// mid-load; reading them live just before loadModel would let the load
// use post-click values. The model-switch speculative reset below
// updates this snapshot in lock-step so non-staged loads are unchanged.
// (validation, the trust dialog, unload).
const loadChatTemplateOverride = stateBeforeUnload.chatTemplateOverride;
const loadKvCacheDtype = stateBeforeUnload.kvCacheDtype;
const loadCustomContextLength = stateBeforeUnload.customContextLength;
@ -746,7 +740,12 @@ export function useChatModelRuntime() {
// The load applied this spec mode, so persist the user's standing
// preference now (the requested intent, not the resolved echo;
// saveSpeculativeType keeps only the universal auto/ngram/off).
saveSpeculativeType(loadSpeculativeType);
// Skip for a per-model/one-off config (keepSpeculative): that choice
// is model-specific and must not overwrite the global default, or a
// later model with no saved config would start from it instead of Auto.
if (!keepSpeculative) {
saveSpeculativeType(loadSpeculativeType);
}
const currentParams = useChatRuntimeStore.getState().params;
setParams(
@ -782,9 +781,11 @@ export function useChatModelRuntime() {
const reportedNativeCtx = loadResponse.is_gguf
? (loadResponse.native_context_length ?? null)
: null;
// A successful reload has applied settings, so clear pending custom
// context state and display the backend-reported effective context.
const keepCustomCtx = null;
// Retain the user's requested context so re-opening or re-saving
// the config keeps the intended override, not the backend's
// effective (auto-fit) context; null stays null so an auto-fit
// never becomes a stored override.
const keepCustomCtx = loadCustomContextLength;
const reasoningAlwaysOn = loadResponse.reasoning_always_on ?? false;
const reasoningStyle = loadResponse.reasoning_style ?? "enable_thinking";
const supportsReasoning = loadResponse.supports_reasoning ?? false;
@ -901,25 +902,6 @@ export function useChatModelRuntime() {
recordLastLocalModelLoad({ id: modelId, kind: "model" });
}
}
// A successful load owns the shared (pick-unscoped) settings fields,
// so any surviving stage is stale: the just-loaded pick itself, or a
// pick queued for a different model mid-load whose knobs this load
// overwrote. Drop it. Only a DIFFERENT pick's download needs
// cancelling; the loaded pick's is already consumed, and cancelling
// it inside its post-complete linger window would flicker its card.
const staleStage = useChatRuntimeStore.getState().pendingSelection;
if (staleStage) {
if (
!pendingSelectionMatches(staleStage, {
id: modelId,
ggufVariant,
nativePathToken,
})
) {
cancelStagedModelDownload(staleStage);
}
useChatRuntimeStore.getState().setPendingSelection(null);
}
} catch (error) {
// Skip rollback if user cancelled -- model is already being unloaded.
if (abortCtrl.signal.aborted) throw error;
@ -954,11 +936,23 @@ export function useChatModelRuntime() {
// Restore the previous model in the split mode it was running,
// not the default layer split.
tensor_parallel: stateBeforeUnload.loadedTensorParallel ?? false,
// Restore its KV cache dtype and chat template too, so the
// rollback runs the model as it was, not with backend defaults.
cache_type_kv: stateBeforeUnload.loadedKvCacheDtype ?? null,
chat_template_override:
stateBeforeUnload.loadedChatTemplateOverride ?? null,
// Restore its speculative-decoding config too. Omitting these
// reloaded the model at backend defaults (speculation off)
// while the UI still showed it enabled.
speculative_type: stateBeforeUnload.loadedSpeculativeType ?? null,
spec_draft_n_max: stateBeforeUnload.loadedSpecDraftNMax ?? null,
});
useChatRuntimeStore.setState({
activeNativePathToken: previousActiveNativePathToken ?? null,
loadedSpeculativeType: null,
loadedSpecDraftNMax: null,
speculativeType: stateBeforeUnload.loadedSpeculativeType ?? null,
loadedSpeculativeType: stateBeforeUnload.loadedSpeculativeType ?? null,
specDraftNMax: stateBeforeUnload.loadedSpecDraftNMax ?? null,
loadedSpecDraftNMax: stateBeforeUnload.loadedSpecDraftNMax ?? null,
});
await refresh();
} catch {
@ -1277,6 +1271,9 @@ export function useChatModelRuntime() {
resetLoadingUi();
}
} catch (error) {
if (typeof selection !== "string" && selection.previousConfig) {
applyPerModelConfigToRuntime(selection.previousConfig);
}
if (abortCtrl.signal.aborted) return; // User cancelled, nothing to report
resetLoadingUi();
const message =

155
studio/frontend/src/features/chat/hooks/use-staged-model-preparation.ts
View file

@ -1,155 +0,0 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
import { useCallback, useEffect } from "react";
import { useRepoDownload } from "@/features/hub/download-manager/use-repo-download";
import type { DownloadJob } from "@/features/hub/download-manager/use-repo-download";
import { useLatestRef } from "@/features/hub/hooks/use-latest-ref";
import { fetchGgufContextLength } from "../api/chat-api";
import {
isPendingGguf,
pendingSelectionMatches,
useChatRuntimeStore,
} from "../stores/chat-runtime-store";
import type { PendingModelSelection } from "../stores/chat-runtime-store";
/**
* Drives the deferred ("Load on selection" off) staging flow for a GGUF:
* download the file if needed (HF repo) or read it in place (native drag-drop /
* picked file), then read its header context length so the settings sheet can
* show the real context slider before the single GPU load. The staged context
* lands on `pendingSelection.contextLength` (scoped to the staged model, never
* the loaded model's `ggufContextLength`). Returns the live download job so the
* sheet can render progress / cancel. Mount once on the chat page.
*/
export function useStagedModelPreparation(opts?: {
/** Load the cached file once an autoLoad pick's download completes. */
onAutoLoad?: (pending: PendingModelSelection) => void;
}): DownloadJob {
const pendingId = useChatRuntimeStore((s) => s.pendingSelection?.id ?? null);
const pendingVariant = useChatRuntimeStore(
(s) => s.pendingSelection?.ggufVariant ?? null,
);
const pendingNativeToken = useChatRuntimeStore(
(s) => s.pendingSelection?.nativePathToken ?? null,
);
// Only GGUF picks (HF variant or native file) have a header worth reading.
const pendingIsGguf = useChatRuntimeStore((s) =>
isPendingGguf(s.pendingSelection),
);
// Non-GGUF HF repos download a full snapshot (variant null) but have no header.
const pendingIsHubRepo = useChatRuntimeStore(
(s) => s.pendingSelection?.isHubRepo ?? false,
);
const pendingDownloaded = useChatRuntimeStore(
(s) => s.pendingSelection?.isDownloaded ?? false,
);
const pendingHasContext = useChatRuntimeStore(
(s) => s.pendingSelection?.contextLength != null,
);
const setPendingSelection = useChatRuntimeStore((s) => s.setPendingSelection);
const onAutoLoadRef = useLatestRef(opts?.onAutoLoad);
// A failed or cancelled autoLoad download has no sheet to retry from, so drop
// the staged pick rather than leave it waiting on a load that won't come.
const handleAutoLoadAbort = useCallback((variant: string | null) => {
const latest = useChatRuntimeStore.getState().pendingSelection;
if (
latest?.autoLoad &&
(latest.ggufVariant ?? null) === (variant ?? null)
) {
useChatRuntimeStore.getState().abandonStagedModel();
}
}, []);
const fetchContextMetadata = useCallback(async () => {
const current = useChatRuntimeStore.getState().pendingSelection;
if (!current?.id || !isPendingGguf(current)) return;
const { id, ggufVariant, nativePathToken } = current;
try {
const contextLength = await fetchGgufContextLength({
model_path: id,
gguf_variant: ggufVariant,
hf_token: useChatRuntimeStore.getState().hfToken || null,
nativePathToken,
});
// Apply only if the same model is still staged (the user may have switched
// picks or loaded/cancelled while the request was in flight).
const latest = useChatRuntimeStore.getState().pendingSelection;
if (
latest &&
contextLength != null &&
pendingSelectionMatches(latest, { id, ggufVariant, nativePathToken })
) {
setPendingSelection({ ...latest, contextLength });
}
} catch {
// Leave contextLength null: the context slider stays hidden and the user
// can still load (context fills in from the load response afterwards).
}
}, [setPendingSelection]);
const job = useRepoDownload({
kind: "model",
// useRepoDownload must be called unconditionally; an idle repo id keeps it
// inert until something is staged.
repoId: pendingId ?? "__staged_idle__",
activeVariant: pendingVariant,
onComplete: (variant) => {
// autoLoad picks load the cached file now; staged picks read the header so
// the sheet's context slider can show before a manual load.
const latest = useChatRuntimeStore.getState().pendingSelection;
if (
latest?.autoLoad &&
(latest.ggufVariant ?? null) === (variant ?? null)
) {
onAutoLoadRef.current?.(latest);
return;
}
void fetchContextMetadata();
},
onError: handleAutoLoadAbort,
onCancelled: handleAutoLoadAbort,
});
// job.requestStartDownload's identity changes per render; hold it in a ref so
// the staging effect re-runs only when the staged model itself changes.
const startDownloadRef = useLatestRef(job.requestStartDownload);
const fetchMetadataRef = useLatestRef(fetchContextMetadata);
useEffect(() => {
// GGUF picks (header worth reading) and uncached non-GGUF hub repos (full
// snapshot, no header) both run here; everything else is loaded directly.
if (
!pendingId ||
(!pendingIsGguf && !pendingIsHubRepo) ||
pendingHasContext
) {
return;
}
// Native files and already-downloaded HF files are local: read the header
// now. Otherwise download first (a GGUF variant, or a null-variant snapshot
// for a hub repo); onComplete then reads the header or auto-loads.
if (pendingNativeToken || pendingDownloaded) {
void fetchMetadataRef.current();
} else {
const expectedBytes =
useChatRuntimeStore.getState().pendingSelection?.expectedBytes ?? 0;
void startDownloadRef.current(pendingVariant, expectedBytes);
}
}, [
pendingId,
pendingVariant,
pendingNativeToken,
pendingIsGguf,
pendingIsHubRepo,
pendingDownloaded,
pendingHasContext,
startDownloadRef,
fetchMetadataRef,
]);
return job;
}

18
studio/frontend/src/features/chat/index.ts
View file

@ -3,11 +3,23 @@
export { ChatPage, validateChatSearch, type ChatSearch } from "./chat-page";
export {
addScanFolder,
browseFolders,
deleteCachedModel,
deleteFineTunedModel,
fetchGgufContextLength,
getInferenceStatus,
listGgufVariants,
listLocalModels,
listRecommendedFolders,
listScanFolders,
loadModel,
removeScanFolder,
type BrowseFoldersResponse,
type CachedGgufRepo,
type CachedModelRepo,
type LocalModelInfo,
type ScanFolderInfo,
} from "./api/chat-api";
export type { GgufVariantDetail } from "./types/api";
export {
@ -17,6 +29,10 @@ export {
type Preset,
} from "./chat-settings-sheet";
export { useChatRuntimeStore } from "./stores/chat-runtime-store";
export {
normalizeSpeculativeType,
readPersistedSpeculativeType,
} from "./stores/chat-runtime-store";
export {
preferFullToolOutput,
toolOutputKey,
@ -37,9 +53,11 @@ export {
} from "./hooks/use-chat-model-runtime";
export {
customProviderDisplayName,
isCustomProviderType,
isExternalModelId,
parseExternalModelId,
} from "./external-providers";
export { ApiProviderLogo } from "./api-provider-logo";
export { useExternalProvidersStore } from "./stores/external-providers-store";
export { ChatSearchDialog } from "./components/chat-search-dialog";
export { setTrainingCompareHandoff } from "./lib/training-compare-handoff";

5
studio/frontend/src/features/chat/lib/apply-inference-status-to-store.ts
View file

@ -204,11 +204,6 @@ export function applyActiveModelStatusToStore(
ggufContextLength: currentGgufContextLength,
ggufMaxContextLength,
ggufNativeContextLength,
// A non-GGUF status must also drop a stale native-path token: without this the
// isGguf OR (activeGgufVariant || activeNativePathToken || ggufContextLength)
// stays true after switching from a native GGUF to a transformers model, so a
// Codex-only detection would auto-select for a model its preflight rejects. A real
// GGUF load reports is_gguf: true, so its token is preserved (the load path owns it).
...(status.is_gguf ? {} : { activeNativePathToken: null }),
modelRequiresTrustRemoteCode: status.requires_trust_remote_code ?? false,
defaultChatTemplate: nextDefaultChatTemplate,

111
studio/frontend/src/features/chat/shared-composer.tsx
View file

@ -79,6 +79,11 @@ import { KnowledgeBaseComposerButton } from "@/features/rag/components/knowledge
import { NewProjectDialog } from "./components/new-project-dialog";
import { useChatProjects } from "./hooks/use-chat-projects";
import { confirmRemoteCodeIfNeeded } from "@/features/security";
import {
normalizeMaxSeqLength,
resolveInitialConfig,
type PerModelConfig,
} from "@/features/model-picker";
import {
confirmTransformersUpgradeIfNeeded,
useTransformersUpgradeDialogStore,
@ -490,8 +495,24 @@ type CompareModelSelection = {
id: string;
isLora: boolean;
ggufVariant?: string;
config?: PerModelConfig;
};
function cleanCompareChatTemplate(
value: string | null | undefined,
): string | null {
return value?.trim() ? value : null;
}
function resolveCompareSpecDraftNMax(
speculativeType: string | null,
value: number | null,
): number | null {
return speculativeType === "mtp" || speculativeType === "mtp+ngram"
? value
: null;
}
// Tool icon plus an X overlay CSS reveals on hover when the pill is active.
function PillGlyph({ children }: { children: ReactNode }) {
return (
@ -1024,13 +1045,11 @@ export function SharedComposer({
const store = useChatRuntimeStore.getState();
const maxSeqLength = store.params.maxSeqLength;
const trustRemoteCode = store.params.trustRemoteCode ?? false;
const chatTemplateOverride = store.chatTemplateOverride;
const effectiveChatTemplateOverride = chatTemplateOverride?.trim()
? chatTemplateOverride
: null;
const fallbackTensorParallel = store.tensorParallel;
const specSettings = resolveSpeculativeSettingsForLoad({
usePersistedPreference: true,
});
let loadedFromConfig = false;
function modelDisplayName(id: string): string {
const parts = id.split("/");
@ -1046,21 +1065,53 @@ export function SharedComposer({
sel: CompareModelSelection,
): Promise<string> {
const currentStore = useChatRuntimeStore.getState();
const config = sel.config ?? null;
// This pane's effective config: an explicit selection config, else the
// remembered store config for this model/quant (never the other pane's).
// A model with no saved config resolves to all-null defaults, so every
// setting below still falls through to its session default.
const resolved = config
? { config, remembered: true }
: resolveInitialConfig(sel.id, sel.ggufVariant ?? null);
const ownConfig = resolved.config;
const ownRemembered = resolved.remembered;
// Mirror single-view resolveLoadMaxSeqLength: a GGUF pane with no
// explicit context loads at native (0 -> n_ctx_train), not the session
// maxSeqLength, which would silently shrink the picker's shown context.
const isGgufLoad =
(sel.ggufVariant ?? null) != null ||
sel.id.toLowerCase().endsWith(".gguf");
const effectiveMaxSeqLength =
ownConfig.customContextLength ??
normalizeMaxSeqLength(ownConfig.maxSeqLength) ??
(isGgufLoad ? 0 : maxSeqLength);
const effectiveChatTemplateOverride = cleanCompareChatTemplate(
ownConfig.chatTemplateOverride,
);
const effectiveSpeculativeType =
ownConfig.speculativeType ?? specSettings.speculativeType;
const effectiveSpecDraftNMax = ownRemembered
? resolveCompareSpecDraftNMax(
effectiveSpeculativeType,
ownConfig.specDraftNMax,
)
: specSettings.specDraftNMax;
const effectiveTensorParallel = ownRemembered
? ownConfig.tensorParallel
: fallbackTensorParallel;
let loadTrustRemoteCode = trustRemoteCode;
let approvedRemoteCodeFingerprint: string | null = null;
const isAlreadyActive =
currentStore.params.checkpoint === sel.id &&
(currentStore.activeGgufVariant ?? null) ===
(sel.ggufVariant ?? null);
// Already loaded (gate passed at first load): skip a redundant reload that would
// re-trigger the gate without the approval fingerprint and fail for HIGH custom code.
if (isAlreadyActive) {
if (isAlreadyActive && !config && !loadedFromConfig) {
return "ready";
}
const validation = await validateModel({
model_path: sel.id,
hf_token: currentStore.hfToken || null,
max_seq_length: maxSeqLength,
max_seq_length: effectiveMaxSeqLength,
load_in_4bit: true,
is_lora: sel.isLora,
gguf_variant: sel.ggufVariant ?? null,
@ -1114,19 +1165,24 @@ export function SharedComposer({
const resp = await loadModel({
model_path: sel.id,
hf_token: useChatRuntimeStore.getState().hfToken || null,
max_seq_length: maxSeqLength,
max_seq_length: effectiveMaxSeqLength,
load_in_4bit: true,
is_lora: sel.isLora,
gguf_variant: sel.ggufVariant ?? null,
trust_remote_code: loadTrustRemoteCode,
approved_remote_code_fingerprint: approvedRemoteCodeFingerprint,
chat_template_override: effectiveChatTemplateOverride,
speculative_type: specSettings.speculativeType,
spec_draft_n_max: specSettings.specDraftNMax,
// Honor the Tensor Parallelism toggle on compare loads too.
tensor_parallel: currentStore.tensorParallel,
cache_type_kv: ownConfig.kvCacheDtype ?? null,
speculative_type: effectiveSpeculativeType,
spec_draft_n_max: effectiveSpecDraftNMax,
tensor_parallel: effectiveTensorParallel,
});
saveSpeculativeType(specSettings.speculativeType);
// Keep a compare pane's per-model speculative choice load-local: only
// persist the global preference when it came from the global settings,
// matching the single-model load path.
if (ownConfig.speculativeType == null) {
saveSpeculativeType(effectiveSpeculativeType);
}
upgradeUnloadedActive = false;
const store = useChatRuntimeStore.getState();
store.setCheckpoint(
@ -1142,11 +1198,38 @@ export function SharedComposer({
...reasoningCapsFromLoad(resp),
supportsPreserveThinking: resp.supports_preserve_thinking ?? false,
supportsTools: resp.supports_tools ?? false,
kvCacheDtype: resp.cache_type_kv ?? null,
loadedKvCacheDtype: resp.cache_type_kv ?? null,
tensorParallel: resp.tensor_parallel ?? false,
loadedTensorParallel: resp.tensor_parallel ?? false,
defaultChatTemplate: resp.chat_template ?? null,
chatTemplateOverride: effectiveChatTemplateOverride,
loadedChatTemplateOverride: effectiveChatTemplateOverride,
loadedIsMultimodal: isMultimodalResponse(resp),
// Record the context this pane actually loaded with, mirroring the
// single-model load path, so that when the last-loaded pane becomes
// the active model the settings UI and any subsequent reload or save
// use its context instead of the previous/default one.
customContextLength: isGgufLoad
? (ownConfig.customContextLength ?? null)
: null,
ggufContextLength: resp.is_gguf ? (resp.context_length ?? null) : null,
ggufNativeContextLength: resp.is_gguf
? (resp.native_context_length ?? null)
: null,
ggufMaxContextLength: resp.is_gguf
? (resp.max_context_length ?? null)
: null,
...resolveLoadedSpeculativeSettings(resp),
});
if (!isGgufLoad) {
// Non-GGUF panes carry their context in params.maxSeqLength.
store.setParams({
...store.params,
maxSeqLength: effectiveMaxSeqLength,
});
}
loadedFromConfig = config != null;
// Sync the models[] entry with the load response so attach/send gates
// read fresh capabilities. /api/models/list can lag a model's actual
// state (e.g. a GGUF whose mmproj arrived after the snapshot).

186
studio/frontend/src/features/chat/stores/chat-runtime-store.ts
View file

@ -1,12 +1,7 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
import type { RememberedLoadSettings } from "@/components/assistant-ui/model-selector/remembered-load-settings";
import {
cancelStagedModelDownload,
mirrorHfTokenInto,
useHfTokenStore,
} from "@/features/hub";
import { mirrorHfTokenInto, useHfTokenStore } from "@/features/hub";
import { toast } from "@/lib/toast";
import { create } from "zustand";
import { isExternalModelId, parseExternalModelId } from "../external-providers";
@ -42,7 +37,6 @@ export const CHAT_ALLOW_ARTIFACT_NETWORK_ACCESS_KEY =
"unsloth_chat_allow_artifact_network_access";
export const CHAT_MCP_ENABLED_KEY = "unsloth_chat_mcp_enabled";
export const CHAT_CONFIRM_TOOL_CALLS_KEY = "unsloth_chat_confirm_tool_calls";
export const CHAT_LOAD_ON_SELECTION_KEY = "unsloth_chat_load_on_selection";
export const CHAT_EXPAND_QUANTIZATIONS_KEY =
"unsloth_chat_expand_quantizations";
export const CHAT_SHOW_ALL_QUANTIZATIONS_KEY =
@ -497,34 +491,7 @@ export function saveSpeculativeType(value: string | null): void {
}
}
/** A local model staged for a deferred load (see `pendingSelection`). Shape is
* a subset of the load hook's `SelectedModelInput`, structurally assignable. */
export type PendingModelSelection = {
id: string;
isLora?: boolean;
ggufVariant?: string;
isDownloaded?: boolean;
expectedBytes?: number;
/** Native (drag-drop / picked-from-disk) GGUF: the path token used to read
* the header and to load. Absent for HF-repo models. */
nativePathToken?: string;
/** Direct local .gguf file (custom folder / LM Studio): a GGUF source even
* though it carries neither an HF variant nor a native path token. */
isGguf?: boolean;
/** Native context length read from the GGUF header once the file is local.
* Scoped here (not the shared `ggufContextLength`) so a staged model's
* metadata never pollutes the currently-loaded model's context display. */
contextLength?: number | null;
/** "Load on selection" on + un-cached GGUF: download via the manager (global
* indicator) without opening the sheet, then load once the download finishes. */
autoLoad?: boolean;
/** Uncached non-GGUF HF repo: download the full snapshot via the manager
* (variant null) the same way GGUF picks download a variant. */
isHubRepo?: boolean;
};
/** A pick is a GGUF (HF variant, native file, or a direct local .gguf) and so
* has pre-load options worth staging. Works on a selection or a staged pick. */
/** A pick is a GGUF: HF variant, native file, or a direct local .gguf. */
export function hasGgufSource(x: {
ggufVariant?: string;
nativePathToken?: string;
@ -562,30 +529,6 @@ export function isDownloadableHubRepo(x: {
);
}
export function isPendingGguf(pending: PendingModelSelection | null): boolean {
return pending != null && hasGgufSource(pending);
}
/** Whether `pending` refers to the same model as `pick` (id + GGUF variant +
* native path token, optionals null-normalized). Native ids are display labels
* that can collide, so the token must match too — id alone can land on the
* wrong file. */
export function pendingSelectionMatches(
pending: PendingModelSelection | null,
pick: {
id: string;
ggufVariant?: string | null;
nativePathToken?: string | null;
},
): boolean {
return (
pending != null &&
pending.id === pick.id &&
(pending.ggufVariant ?? null) === (pick.ggufVariant ?? null) &&
(pending.nativePathToken ?? null) === (pick.nativePathToken ?? null)
);
}
type ChatRuntimeStore = {
settingsHydrated: boolean;
params: InferenceParams;
@ -743,10 +686,6 @@ type ChatRuntimeStore = {
tensorParallel: boolean;
/** Backend-reported tensor-parallel state; null until first hydrated. */
loadedTensorParallel: boolean | null;
/** Persisted: when false, picking a local model stages it as
* `pendingSelection` (and opens settings) instead of loading immediately,
* so load settings can be set before the single load. */
loadOnSelection: boolean;
/** Persisted: expand every On Device GGUF repo's quantizations by default
* instead of waiting for a click. */
expandQuantizations: boolean;
@ -755,9 +694,6 @@ type ChatRuntimeStore = {
/** Persisted, shared by the chat model selector and the Hub page: list only
* models whose size fits this device's memory budget. */
fitOnDeviceOnly: boolean;
/** A local model picked while `loadOnSelection` is off: staged, not loaded.
* The settings sheet shows its load knobs and a Load button. */
pendingSelection: PendingModelSelection | null;
loadedIsMultimodal: boolean;
/** Active model is a block-diffusion model (DiffusionGemma): drives the
* denoising-canvas artifact auto-render. */
@ -794,6 +730,10 @@ type ChatRuntimeStore = {
} | null;
modelLoading: boolean;
activeNativePathToken: string | null;
// Wall-clock expiry (ms) of the active native path token. The desktop host
// prunes file leases after a TTL, so a reload checks this to prompt
// re-selection instead of reusing a dead token.
activeNativePathExpiresAtMs: number | null;
hydratePersistedSettings: () => Promise<void>;
setModelLoading: (loading: boolean) => void;
setModelRequiresTrustRemoteCode: (required: boolean) => void;
@ -871,33 +811,9 @@ type ChatRuntimeStore = {
setNudgeToolCalls: (enabled: boolean) => void;
setMaxToolCallsPerMessage: (value: number) => void;
setToolCallTimeout: (value: number) => void;
setKvCacheDtype: (dtype: string | null) => void;
setSpeculativeType: (type: string | null) => void;
setSpecDraftNMax: (value: number | null) => void;
/** Revert the editable load knobs to the loaded model's baseline (or defaults
* when nothing is loaded). Used by the settings-sheet Reset button and to
* start each deferred-staging session clean so one staged pick's settings
* don't leak onto the next. */
resetModelSettingsToLoaded: () => void;
/** Seed the editable load knobs from a model's remembered settings. Shared by
* the settings sheet's restore effect and the "Load on selection" paths,
* which skip the sheet but must still honor a saved config. */
applyRememberedLoadSettings: (settings: RememberedLoadSettings) => void;
setTensorParallel: (value: boolean) => void;
setLoadOnSelection: (value: boolean) => void;
setExpandQuantizations: (value: boolean) => void;
setShowAllQuantizations: (value: boolean) => void;
setFitOnDeviceOnly: (value: boolean) => void;
setPendingSelection: (selection: PendingModelSelection | null) => void;
/** Stage a pick for a deferred load: revert knobs to the loaded baseline,
* record the selection, and open the settings sheet. */
stageModel: (selection: PendingModelSelection) => void;
/** Abandon a staged pick without loading: revert knobs to the loaded baseline
* and clear the pending selection. Cancels its in-flight download too, unless
* `keepDownload` is set (navigation keeps the transfer running, like Hub). */
abandonStagedModel: (opts?: { keepDownload?: boolean }) => void;
setCustomContextLength: (v: number | null) => void;
setChatTemplateOverride: (template: string | null) => void;
setPendingAudio: (base64: string, name: string) => void;
clearPendingAudio: () => void;
setPendingImageEditReference: (
@ -1099,23 +1015,6 @@ function setScalarSettingVersion<K extends ScalarSettingKey>(
saveSettingsPatch({ [key]: value });
}
/** The "revert to the loaded model" baseline for the editable load knobs.
* Shared by resetModelSettingsToLoaded (full revert) and stageModel (which
* overrides speculative to start a fresh pick from the standing default). */
function loadedBaselineSettings(s: ChatRuntimeStore) {
const hasLoadedModel = Boolean(s.params.checkpoint);
return {
customContextLength: null,
kvCacheDtype: s.loadedKvCacheDtype,
tensorParallel: s.loadedTensorParallel ?? false,
speculativeType: hasLoadedModel
? s.loadedSpeculativeType
: readPersistedSpeculativeType(),
specDraftNMax: hasLoadedModel ? s.loadedSpecDraftNMax : null,
chatTemplateOverride: s.loadedChatTemplateOverride,
};
}
export const useChatRuntimeStore = create<ChatRuntimeStore>((set, get) => ({
settingsHydrated: false,
// Hydrate the last external checkpoint so the external picker survives a
@ -1213,11 +1112,9 @@ export const useChatRuntimeStore = create<ChatRuntimeStore>((set, get) => ({
loadedSpecDraftNMax: null,
tensorParallel: false,
loadedTensorParallel: null,
loadOnSelection: loadBool(CHAT_LOAD_ON_SELECTION_KEY, true),
expandQuantizations: loadBool(CHAT_EXPAND_QUANTIZATIONS_KEY, false),
showAllQuantizations: loadBool(CHAT_SHOW_ALL_QUANTIZATIONS_KEY, true),
fitOnDeviceOnly: loadBool(MODELS_FIT_ON_DEVICE_ONLY_KEY, false),
pendingSelection: null,
loadedIsMultimodal: false,
loadedIsDiffusion: false,
customContextLength: null,
@ -1235,6 +1132,7 @@ export const useChatRuntimeStore = create<ChatRuntimeStore>((set, get) => ({
contextUsage: null,
modelLoading: false,
activeNativePathToken: null,
activeNativePathExpiresAtMs: null,
hydratePersistedSettings: async () => {
if (get().settingsHydrated) {
return;
@ -1353,13 +1251,6 @@ export const useChatRuntimeStore = create<ChatRuntimeStore>((set, get) => ({
// Clear stale per-turn usage on model change; the relaxed external-provider
// render gate would otherwise show old counters until the next completion.
const checkpointChanged = state.params.checkpoint !== modelId;
const pendingToClear =
checkpointChanged && state.params.checkpoint
? state.pendingSelection
: null;
if (pendingToClear) {
cancelStagedModelDownload(pendingToClear);
}
// Clamp maxTokens to the new model's cap when switching into an external
// model so a value carried over from a local session doesn't exceed the
// slider's max.
@ -1387,14 +1278,6 @@ export const useChatRuntimeStore = create<ChatRuntimeStore>((set, get) => ({
},
activeGgufVariant: ggufVariant ?? null,
...(checkpointChanged ? { contextUsage: null } : {}),
// Switching away from a loaded model (e.g. picking an external provider)
// abandons any staged pick, so its Load button and edited knobs don't
// linger over the newly active model. Same revert as abandonStagedModel.
// Guarded on a non-empty current checkpoint: an establishing set from a
// background status sync (empty -> active) must not wipe a fresh stage.
...(pendingToClear
? { ...loadedBaselineSettings(state), pendingSelection: null }
: {}),
};
}),
setActiveThreadId: (activeThreadId) =>
@ -1408,7 +1291,6 @@ export const useChatRuntimeStore = create<ChatRuntimeStore>((set, get) => ({
// clear any stored external selection so the next refresh doesn't snap
// back to a model the user intentionally cleared.
saveLastExternalCheckpoint(null);
cancelStagedModelDownload(get().pendingSelection);
return set((state) => ({
params: {
...state.params,
@ -1416,7 +1298,7 @@ export const useChatRuntimeStore = create<ChatRuntimeStore>((set, get) => ({
},
activeGgufVariant: null,
activeNativePathToken: null,
pendingSelection: null,
activeNativePathExpiresAtMs: null,
ggufContextLength: null,
ggufMaxContextLength: null,
ggufNativeContextLength: null,
@ -1749,25 +1631,6 @@ export const useChatRuntimeStore = create<ChatRuntimeStore>((set, get) => ({
);
return { toolCallTimeout };
}),
setKvCacheDtype: (kvCacheDtype) => set({ kvCacheDtype }),
setSpeculativeType: (speculativeType) => set({ speculativeType }),
setSpecDraftNMax: (specDraftNMax) => set({ specDraftNMax }),
setTensorParallel: (tensorParallel) => set({ tensorParallel }),
resetModelSettingsToLoaded: () => set((s) => loadedBaselineSettings(s)),
applyRememberedLoadSettings: (settings) =>
// Coalesce every field: a blob persisted by an older/newer build can omit
// keys, and a raw spread would push `undefined` into fields typed non-null.
set({
customContextLength: settings.contextLength ?? null,
kvCacheDtype: settings.kvCacheDtype ?? null,
speculativeType: settings.speculativeType ?? "auto",
specDraftNMax: settings.specDraftNMax ?? null,
tensorParallel: settings.tensorParallel ?? false,
}),
setLoadOnSelection: (loadOnSelection) => {
saveBool(CHAT_LOAD_ON_SELECTION_KEY, loadOnSelection);
set({ loadOnSelection });
},
setExpandQuantizations: (expandQuantizations) => {
saveBool(CHAT_EXPAND_QUANTIZATIONS_KEY, expandQuantizations);
set({ expandQuantizations });
@ -1780,39 +1643,6 @@ export const useChatRuntimeStore = create<ChatRuntimeStore>((set, get) => ({
saveBool(MODELS_FIT_ON_DEVICE_ONLY_KEY, fitOnDeviceOnly);
set({ fitOnDeviceOnly });
},
setPendingSelection: (pendingSelection) => set({ pendingSelection }),
stageModel: (selection) => {
// Refuse staging mid-load: post-load cleanup would silently drop the queued
// pick. stageOrLoad toasts first for callers that can.
if (get().modelLoading) return;
// Rebinding to a new pick keeps the prior pick's download running so the
// user can queue multiple downloads at once (Hub-style).
set((s) => {
return {
...loadedBaselineSettings(s),
pendingSelection: selection,
// autoLoad downloads silently and loads on completion, so keep the sheet shut.
settingsPanelOpen: !selection.autoLoad,
// Speculative starts from the standing default, not the loaded model's
// mode, so a fresh pick doesn't inherit (and then carry, via the staged
// Load's keepSpeculative) a forced MTP mode onto a model that may lack it.
speculativeType: readPersistedSpeculativeType(),
specDraftNMax: null,
};
});
},
abandonStagedModel: (opts) => {
const { pendingSelection } = get();
if (!pendingSelection) return;
// Cancel the staged pick's in-flight download (centralized for every abandon
// path: sheet close, thread switch, route exit, new chat). `keepDownload`
// opts out so navigation leaves the transfer running, like a Hub download.
if (!opts?.keepDownload) cancelStagedModelDownload(pendingSelection);
set((s) => ({ ...loadedBaselineSettings(s), pendingSelection: null }));
},
setCustomContextLength: (customContextLength) => set({ customContextLength }),
setChatTemplateOverride: (chatTemplateOverride) =>
set({ chatTemplateOverride }),
setPendingAudio: (base64, name) =>
set({ pendingAudioBase64: base64, pendingAudioName: name }),
clearPendingAudio: () =>

71
studio/frontend/src/features/export/components/export-run-panel.tsx
View file

@ -2,7 +2,6 @@
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
import { Button } from "@/components/ui/button";
import { FolderBrowser } from "@/components/assistant-ui/model-selector/folder-browser";
import { Input } from "@/components/ui/input";
import {
InputGroup,
@ -17,6 +16,7 @@ import {
TooltipContent,
TooltipTrigger,
} from "@/components/ui/tooltip";
import { FolderBrowser } from "@/features/model-picker";
import {
AlertCircleIcon,
ArrowRight01Icon,
@ -28,17 +28,17 @@ import {
import { HugeiconsIcon } from "@hugeicons/react";
import { useEffect, useRef, useState } from "react";
import { useShallow } from "zustand/react/shallow";
import type { ExportLogEntry } from "../api/export-api";
import {
EXPORT_METHODS,
type ExportMethod,
findMergedFormat,
} from "../constants";
import type { ExportLogEntry } from "../api/export-api";
import { getExportLogLineClass } from "../lib/log-style";
import {
type ExportDestination,
selectExportProgressPercent,
useExportRuntimeStore,
type ExportDestination,
} from "../stores/export-runtime-store";
function useElapsedSeconds(startedAt: number | null, running: boolean): number {
@ -165,7 +165,9 @@ export function ExportRunPanel(props: ExportRunPanelProps) {
const isExporting = run.isExporting;
const isTerminal =
run.phase === "success" || run.phase === "error" || run.phase === "canceled";
run.phase === "success" ||
run.phase === "error" ||
run.phase === "canceled";
const showConfig = run.phase === "idle";
// Gate the log area on the active run's method (from the store) as well as the
// local form selection, so it stays visible after navigating away and back
@ -197,7 +199,9 @@ export function ExportRunPanel(props: ExportRunPanelProps) {
setFollowTail(nearBottom);
};
const methodTitle = EXPORT_METHODS.find((m) => m.value === exportMethod)?.title;
const methodTitle = EXPORT_METHODS.find(
(m) => m.value === exportMethod,
)?.title;
const summary = run.summary;
const summaryBaseModel = summary?.baseModelName ?? baseModelName;
const summaryCheckpoint = summary?.checkpointLabel ?? checkpoint;
@ -290,7 +294,10 @@ export function ExportRunPanel(props: ExportRunPanelProps) {
onClick={() => setFolderBrowserOpen(true)}
aria-label="Browse save folder"
>
<HugeiconsIcon icon={FolderSearchIcon} className="size-4" />
<HugeiconsIcon
icon={FolderSearchIcon}
className="size-4"
/>
</Button>
</TooltipTrigger>
<TooltipContent>Browse</TooltipContent>
@ -301,8 +308,8 @@ export function ExportRunPanel(props: ExportRunPanelProps) {
<>Default: {defaultSaveDirectory}</>
) : (
<>
Paste an absolute path if the folder browser cannot reach the
drive.
Paste an absolute path if the folder browser cannot reach
the drive.
</>
)}
</p>
@ -410,7 +417,10 @@ export function ExportRunPanel(props: ExportRunPanelProps) {
: [];
const showLabels = items.length > 1;
return items.map((o, i) => (
<div key={`${o.path}-${i}`} className="flex min-w-0 flex-col gap-0.5">
<div
key={`${o.path}-${i}`}
className="flex min-w-0 flex-col gap-0.5"
>
{showLabels && o.label ? (
<span className="text-xs text-emerald-700/80 dark:text-emerald-300/80">
{o.label}
@ -431,14 +441,22 @@ export function ExportRunPanel(props: ExportRunPanelProps) {
{run.phase === "canceled" && (
<div className="flex items-start gap-2 rounded-lg bg-amber-500/10 p-3 text-sm text-amber-700 dark:text-amber-300">
<HugeiconsIcon icon={CancelCircleIcon} className="mt-0.5 size-4 shrink-0" />
<span>Export canceled. Training and inference were not affected.</span>
<HugeiconsIcon
icon={CancelCircleIcon}
className="mt-0.5 size-4 shrink-0"
/>
<span>
Export canceled. Training and inference were not affected.
</span>
</div>
)}
{run.phase === "error" && run.error && (
<div className="flex items-start gap-2 rounded-lg bg-destructive/10 p-3 text-sm text-destructive">
<HugeiconsIcon icon={AlertCircleIcon} className="size-4 mt-0.5 shrink-0" />
<HugeiconsIcon
icon={AlertCircleIcon}
className="size-4 mt-0.5 shrink-0"
/>
<span>{run.error}</span>
</div>
)}
@ -447,15 +465,21 @@ export function ExportRunPanel(props: ExportRunPanelProps) {
<div className="rounded-xl bg-muted/50 p-3 text-xs text-muted-foreground flex flex-col gap-1">
<div className="flex justify-between">
<span>Base Model</span>
<span className="font-medium text-foreground">{summaryBaseModel}</span>
<span className="font-medium text-foreground">
{summaryBaseModel}
</span>
</div>
<div className="flex justify-between">
<span>{isAdapter ? "Checkpoint" : "Model"}</span>
<span className="font-medium text-foreground">{summaryCheckpoint}</span>
<span className="font-medium text-foreground">
{summaryCheckpoint}
</span>
</div>
<div className="flex justify-between">
<span>Export Method</span>
<span className="font-medium text-foreground">{summaryMethodLabel}</span>
<span className="font-medium text-foreground">
{summaryMethodLabel}
</span>
</div>
{summaryMethod === "merged" && summaryFormats.length > 0 && (
<div className="flex justify-between gap-3">
@ -484,7 +508,12 @@ export function ExportRunPanel(props: ExportRunPanelProps) {
</span>
{summaryMethod === "gguf" && run.quantTotal > 1 && (
<span className="text-[10px] tabular-nums text-muted-foreground">
Quant {Math.min(run.quantIndex + (isExporting ? 1 : 0), run.quantTotal)} of {run.quantTotal}
Quant{" "}
{Math.min(
run.quantIndex + (isExporting ? 1 : 0),
run.quantTotal,
)}{" "}
of {run.quantTotal}
</span>
)}
<span className="rounded-full border border-border/60 px-2.5 py-1 text-[10px] font-medium tabular-nums text-muted-foreground">
@ -506,7 +535,10 @@ export function ExportRunPanel(props: ExportRunPanelProps) {
}
/>
{run.stage && (
<p className="truncate text-[11px] text-muted-foreground/80" title={run.stage}>
<p
className="truncate text-[11px] text-muted-foreground/80"
title={run.stage}
>
{run.stage}
</p>
)}
@ -556,10 +588,7 @@ export function ExportRunPanel(props: ExportRunPanelProps) {
) : (
<div className="whitespace-pre-wrap break-words">
{run.logLines.map((entry, idx) => (
<div
key={idx}
className={getExportLogLineClass(entry)}
>
<div key={idx} className={getExportLogLineClass(entry)}>
{formatLogLine(entry)}
</div>
))}

41
studio/frontend/src/features/hub/catalog/models-catalog-rows.tsx
View file

@ -1,7 +1,6 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
import { ModelDeleteAction } from "@/components/assistant-ui/model-selector/model-delete-action";
import {
Tooltip,
TooltipContent,
@ -9,18 +8,22 @@ import {
} from "@/components/ui/tooltip";
import {
type GgufVariantDetail,
deleteCachedModel,
deleteCachedDataset,
deleteCachedModel,
formatLocalUpdated,
listGgufVariants,
useGgufVariantsCacheVersion,
} from "@/features/hub/inventory";
import { classifyUnslothSupport } from "@/features/hub/hooks/use-hub-model-search";
import { formatBytes, formatRelativeShort } from "@/features/hub/lib/format";
import { ggufVariantDisplayLabel } from "@/features/hub/lib/gguf-variant-sort";
import { modelIdsMatch } from "@/features/hub/lib/model-identity";
} from "@/features/hub";
import {
classifyUnslothSupport,
formatBytes,
formatRelativeShort,
ggufVariantDisplayLabel,
modelIdsMatch,
useHfTokenStore,
} from "@/features/hub";
import { ModelDeleteAction } from "@/features/model-picker";
import { cn, formatCompact } from "@/lib/utils";
import { useHfTokenStore } from "@/features/hub/stores/hf-token-store";
import {
Download01Icon,
FavouriteIcon,
@ -39,6 +42,7 @@ import {
useRef,
useState,
} from "react";
import { paramLabelFromId } from "../lib/view-models";
import type {
CachedInventoryRow,
DiscoverRow,
@ -46,7 +50,6 @@ import type {
} from "../types";
import { OwnerAvatar } from "./owner-avatar";
import { AccessGlyphs } from "./shared";
import { paramLabelFromId } from "../lib/view-models";
const COARSE_POINTER =
typeof window !== "undefined" &&
@ -142,15 +145,15 @@ function CachedSizeChipLive({
);
const rows: Array<{ label: string; size_bytes: number }> | null =
!needsVariantFetch
? [{ label: repoId, size_bytes: totalBytes }]
: currentVariantState.status === "loaded" &&
needsVariantFetch
? currentVariantState.status === "loaded" &&
currentVariantState.variants.length > 0
? currentVariantState.variants.map((variant) => ({
label: ggufVariantDisplayLabel(variant),
size_bytes: variant.size_bytes,
}))
: null;
: null
: [{ label: repoId, size_bytes: totalBytes }];
const variantMessage =
currentVariantState.status === "loading"
? "Loading downloaded variants..."
@ -275,7 +278,9 @@ function CatalogRow({
)}
/>
<CatalogRowInteractiveContext.Provider value={interactive}>
<div className={cn("pointer-events-none relative", card && "z-[1] w-full")}>
<div
className={cn("pointer-events-none relative", card && "z-[1] w-full")}
>
{children}
</div>
</CatalogRowInteractiveContext.Provider>
@ -653,7 +658,9 @@ export const InventoryRow = memo(function InventoryRow({
<div className="hidden shrink-0 items-center gap-1.5 sm:flex">
{/* Format already shows as the status dot, so the pill stays neutral. */}
{formatLabel && <span className="hub-chip">{formatLabel}</span>}
{paramLabel && <span className="hub-chip tabular-nums">{paramLabel}</span>}
{paramLabel && (
<span className="hub-chip tabular-nums">{paramLabel}</span>
)}
{quantLabel && (
<span className="hub-chip font-mono text-[10.5px] uppercase">
{quantLabel}
@ -697,9 +704,7 @@ export const InventoryRow = memo(function InventoryRow({
const compactMarkers =
partialRepoId || unsupported ? (
<span className="flex shrink-0 items-center gap-1">
{partialRepoId && (
<StatusDot tone="warning" label="Partial download" />
)}
{partialRepoId && <StatusDot tone="warning" label="Partial download" />}
{unsupported && (
<StatusDot tone="danger" label="May not be supported yet" />
)}

33
studio/frontend/src/features/hub/catalog/on-device-folders-dialog.tsx
View file

@ -1,7 +1,6 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
import { FolderBrowser } from "@/components/assistant-ui/model-selector/folder-browser";
import { Button } from "@/components/ui/button";
import {
Dialog,
@ -22,9 +21,11 @@ import {
addScanFolder,
listScanFolders,
removeScanFolder,
} from "@/features/hub/inventory";
import { openModelsDir } from "@/features/native-intents/api";
} from "@/features/hub";
import { FolderBrowser } from "@/features/model-picker";
import { openModelsDir } from "@/features/native-intents";
import { isTauri } from "@/lib/api-base";
import { toast } from "@/lib/toast";
import { cn } from "@/lib/utils";
import {
Delete02Icon,
@ -38,7 +39,6 @@ import {
} from "@hugeicons/core-free-icons";
import { HugeiconsIcon } from "@hugeicons/react";
import { useCallback, useEffect, useMemo, useRef, useState } from "react";
import { toast } from "@/lib/toast";
function pathTail(path: string): string {
const parts = path.split(/[\\/]/).filter(Boolean);
@ -123,7 +123,9 @@ export function OnDeviceFoldersDialog({
setPath("");
mutationVersionRef.current += 1;
setFolders((current) => {
const withoutDuplicate = current.filter((row) => row.id !== folder.id);
const withoutDuplicate = current.filter(
(row) => row.id !== folder.id,
);
return [...withoutDuplicate, folder];
});
toast.success("Location added", {
@ -184,9 +186,12 @@ export function OnDeviceFoldersDialog({
overlayClassName="bg-black/20 backdrop-blur-none"
>
<DialogHeader className="border-b border-border/60 px-5 py-4">
<DialogTitle className="text-[15px]">On-device locations</DialogTitle>
<DialogTitle className="text-[15px]">
On-device locations
</DialogTitle>
<DialogDescription className="sr-only">
Hugging Face model folders, GGUF files, and adapters are indexed here.
Hugging Face model folders, GGUF files, and adapters are indexed
here.
</DialogDescription>
</DialogHeader>
@ -342,9 +347,7 @@ export function OnDeviceFoldersDialog({
</p>
<Tooltip>
<TooltipTrigger asChild={true}>
<p
className="block w-full truncate font-mono text-[10.5px] text-muted-foreground"
>
<p className="block w-full truncate font-mono text-[10.5px] text-muted-foreground">
{folder.path}
</p>
</TooltipTrigger>
@ -372,7 +375,10 @@ export function OnDeviceFoldersDialog({
/>
</button>
</TooltipTrigger>
<TooltipContent side="left" className="tooltip-compact">
<TooltipContent
side="left"
className="tooltip-compact"
>
Open in file manager
</TooltipContent>
</Tooltip>
@ -397,7 +403,10 @@ export function OnDeviceFoldersDialog({
)}
</button>
</TooltipTrigger>
<TooltipContent side="left" className="tooltip-compact">
<TooltipContent
side="left"
className="tooltip-compact"
>
Remove from list
</TooltipContent>
</Tooltip>

23
studio/frontend/src/features/hub/download-manager/download-manager-controller.ts
View file

@ -1,14 +1,10 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
import { DOWNLOAD_KIND } from "./constants";
import {
createDownloadManagerInitialState,
jobKeyOf,
removeJob,
selectActiveJob,
setState,
useDownloadManagerStore,
} from "./download-manager-state";
import { resetDownloadApiAdapterState } from "./download-api-adapter";
import {
@ -69,25 +65,6 @@ export const downloadManager: DownloadManagerController = {
dismiss: removeJob,
};
/** Cancel the in-flight download for a staged model pick. No-op when nothing is
* downloading (e.g. a native/local file that was never fetched). Lets non-React
* callers (the chat store's abandon paths) stop a staged transfer without the
* useRepoDownload hook. */
export function cancelStagedModelDownload(
pending: { id: string; ggufVariant?: string | null } | null,
): void {
if (!pending) return;
const variant = pending.ggufVariant ?? null;
const activeJob = selectActiveJob(
useDownloadManagerStore.getState(),
DOWNLOAD_KIND.MODEL,
pending.id,
variant,
);
void downloadManager.cancel(
activeJob?.key ?? jobKeyOf(DOWNLOAD_KIND.MODEL, pending.id, variant),
);
}
if (import.meta.hot) {
import.meta.hot.dispose(() => {

1
studio/frontend/src/features/hub/download-manager/index.ts
View file

@ -20,7 +20,6 @@ export {
} from "./constants";
export {
__resetDownloadManagerForTests,
cancelStagedModelDownload,
clearCompletedInventoryHint,
downloadManager,
hydrateDownloadManager,

129
studio/frontend/src/features/hub/hub-page.tsx
View file

@ -1,34 +1,32 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
import {
loadRememberedLoadSettings,
rememberedLoadSettingsKey,
} from "@/components/assistant-ui/model-selector/remembered-load-settings";
import { hfModelFitsDevice } from "@/components/assistant-ui/model-selector/recommended-fit";
import { useHubInventory } from "@/features/hub/inventory";
import { useDebouncedValue } from "@/hooks/use-debounced-value";
import { useGpuInfo } from "@/hooks/use-gpu-info";
import {
type HfModelSearchChannel,
type HfSortDirection,
type HfSortKey,
} from "@/features/hub/hooks/use-hub-model-search";
import { useOnlineStatus } from "@/features/hub/hooks/use-online-status";
import { useHubInfiniteScroll } from "@/features/hub/hooks/use-hub-infinite-scroll";
import { ggufVariantsMatch, modelIdsMatch } from "@/features/hub/lib/model-identity";
import { cn } from "@/lib/utils";
import { usePlatformStore } from "@/config/env";
import {
hfApiToken,
useHfTokenStore,
} from "@/features/hub/stores/hf-token-store";
import {
getInferenceStatus,
isExternalModelId,
useChatModelRuntime,
useChatRuntimeStore,
} from "@/features/chat";
import { useHubInventory } from "@/features/hub";
import type {
HfModelSearchChannel,
HfSortDirection,
HfSortKey,
} from "@/features/hub";
import { useOnlineStatus } from "@/features/hub";
import { useHubInfiniteScroll } from "@/features/hub";
import { ggufVariantsMatch, modelIdsMatch } from "@/features/hub";
import { hfApiToken, useHfTokenStore } from "@/features/hub";
import {
applyModelLoadConfigToRuntime,
currentRuntimePerModelConfig,
hfModelFitsDevice,
resolveInitialConfig,
} from "@/features/model-picker";
import { useDebouncedValue } from "@/hooks/use-debounced-value";
import { useGpuInfo } from "@/hooks/use-gpu-info";
import { cn } from "@/lib/utils";
import { useNavigate, useSearch } from "@tanstack/react-router";
import {
useCallback,
@ -38,17 +36,10 @@ import {
useRef,
useState,
} from "react";
import { ExternalLinkConfirmDialog } from "./catalog/external-link-confirm-dialog";
import { HubDetailView } from "./catalog/hub-detail-view";
import { HubTopBar } from "./catalog/hub-top-bar";
import { HubFeed } from "./catalog/hub-feed";
import { OwnerScopeToggle } from "./catalog/owner-scope-toggle";
import {
type AllModelsView,
HubListHeader,
type InventorySort,
InventorySortControl,
ResultListHeader,
} from "./catalog/models-table";
import { HubTopBar } from "./catalog/hub-top-bar";
import {
ModelsCatalog,
type ModelsCatalogHandlers,
@ -56,9 +47,16 @@ import {
type ModelsCatalogState,
} from "./catalog/models-catalog";
import { ModelsHeader } from "./catalog/models-header";
import {
type AllModelsView,
HubListHeader,
type InventorySort,
InventorySortControl,
ResultListHeader,
} from "./catalog/models-table";
import { ModelsToolbar } from "./catalog/models-toolbar";
import { ExternalLinkConfirmDialog } from "./catalog/external-link-confirm-dialog";
import { OnDeviceFoldersDialog } from "./catalog/on-device-folders-dialog";
import { OwnerScopeToggle } from "./catalog/owner-scope-toggle";
import { useDiscoverSearch } from "./hooks/use-discover-search";
import { useFeedWriteBack } from "./hooks/use-feed-write-back";
import { useHubFeed } from "./hooks/use-hub-feed";
@ -568,15 +566,15 @@ export function ModelsPage() {
const deferredCapabilityFilter = useDeferredValue(capabilityFilter);
const hasQuery = deferredDebouncedQuery.trim() !== "";
const mode: DiscoverMode = !isModelDiscover
? "search"
: hasQuery
const mode: DiscoverMode = isModelDiscover
? hasQuery
? "search"
: urlSection != null
? "channel-list"
: sortBrowseActive
? "search"
: "feed";
: "feed"
: "search";
const isFeedMode = mode === "feed";
const isChannelListMode = mode === "channel-list";
const isSortBrowseMode =
@ -767,7 +765,10 @@ export function ModelsPage() {
}
return merged;
}, [isFeedMode, feedTrendingRows, filteredDiscoverRows]);
const feedResults = useMemo(() => feedRows.map((row) => row.result), [feedRows]);
const feedResults = useMemo(
() => feedRows.map((row) => row.result),
[feedRows],
);
const selectionDiscoverRows = isFeedMode ? feedRows : discoverRows;
const selectionFilteredDiscoverRows = isFeedMode
? feedRows
@ -1108,50 +1109,22 @@ export function ModelsPage() {
(opts: ModelLoadOptions, isDownloaded: boolean) => {
if (!selectedModel) return;
const runId = selectedModel.resource.runId;
// "Load on selection" off: stage GGUF picks instead of loading, so the
// chat page's staging flow can read the header and show the load options.
// Non-GGUF models have nothing to configure pre-load, so they load now.
if (
!useChatRuntimeStore.getState().loadOnSelection &&
(opts.ggufVariant != null || selectedModel.isGguf)
) {
useChatRuntimeStore.getState().stageModel({
id: runId,
ggufVariant: opts.ggufVariant,
isGguf: selectedModel.isGguf,
isDownloaded,
expectedBytes: opts.expectedBytes,
});
openNewChat();
return;
}
// Detach any leftover staged pick first so its edited knobs (e.g. a custom
// context length) don't leak into this load -- mirrors the chat page's
// detachStaged(); keepDownload keeps any staged download running.
useChatRuntimeStore.getState().abandonStagedModel({ keepDownload: true });
// Load-on-selection skips the chat sheet, so seed this GGUF pick's saved
// load knobs here the way the sheet's restore effect would; otherwise the
// remembered config is silently ignored on the Hub run path. keepSpeculative
// then honors the restored speculative choice across the switch.
const remembered =
opts.ggufVariant != null || selectedModel.isGguf
? loadRememberedLoadSettings(
rememberedLoadSettingsKey({
id: runId,
ggufVariant: opts.ggufVariant,
}),
)
: null;
if (remembered) {
useChatRuntimeStore.getState().applyRememberedLoadSettings(remembered);
}
const resolvedConfig = resolveInitialConfig(runId, opts.ggufVariant);
const rememberedConfig = resolvedConfig.remembered
? resolvedConfig.config
: null;
const previousConfig = currentRuntimePerModelConfig({
includeMaxSeqLength: true,
});
const hasAppliedConfig = applyModelLoadConfigToRuntime(rememberedConfig);
void selectModel({
id: runId,
ggufVariant: opts.ggufVariant,
isDownloaded,
expectedBytes: opts.expectedBytes,
keepSpeculative: remembered != null,
keepSpeculative: hasAppliedConfig,
throwOnError: true,
previousConfig,
})
.then(() => {
// Read fresh: the load is async, so the checkpoint may have changed.
@ -1375,16 +1348,18 @@ export function ModelsPage() {
</div>
);
}
const ownerToggle = !isDatasetMode ? (
const ownerToggle = isDatasetMode ? undefined : (
<OwnerScopeToggle value={ownerScope} onChange={setOwnerScope} />
) : undefined;
);
// Compact pill so it stays beside the view-mode tabs even in the narrow
// split pane instead of dropping to its own row.
return (
<div className="flex flex-col gap-3 pt-6">
{isChannelListMode ? (
<HubListHeader
title={channelSection ? HUB_SECTION_TITLE[channelSection] : "Models"}
title={
channelSection ? HUB_SECTION_TITLE[channelSection] : "Models"
}
count={listCount}
view={allModelsView}
onViewChange={setAllModelsView}

58
studio/frontend/src/features/hub/index.ts
View file

@ -1,9 +1,65 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
export { cancelStagedModelDownload } from "./download-manager";
export {
cancelModelDownload,
downloadManager,
jobKeyOf,
subscribeJobListeners,
useDownloadManagerStore,
type TransportConflictInfo,
} from "./download-manager";
export {
useHubInventory,
type CachedInventoryRow,
type GgufVariantDetail,
type HubInventory,
type HubInventoryKind,
type InventoryRow,
type LocalInventoryRow,
type LocalSource,
type ScanFolderInfo,
addScanFolder,
deleteCachedDataset,
deleteCachedModel,
formatLocalUpdated,
listGgufVariants,
listScanFolders,
removeScanFolder,
useGgufVariantsCacheVersion,
} from "./inventory";
export {
type HfModelResult,
type HfModelSearchChannel,
type HfSortDirection,
type HfSortKey,
useHubModelSearch,
} from "./hooks/use-hub-model-search";
export { useOnlineStatus } from "./hooks/use-online-status";
export { useHubInfiniteScroll } from "./hooks/use-hub-infinite-scroll";
export {
getHfToken,
hfApiToken,
mirrorHfTokenInto,
useHfTokenStore,
} from "./stores/hf-token-store";
export { looksLikeLocalPath } from "./lib/local-path";
export { hubTokenHeader } from "./lib/hub-token-header";
export {
ggufVariantsMatch,
modelIdsMatch,
normalizeGgufVariantIdentity,
normalizeModelIdentity,
} from "./lib/model-identity";
export { formatBytes, formatRelativeShort } from "./lib/format";
export { ggufVariantDisplayLabel } from "./lib/gguf-variant-sort";
export {
DeleteConfirmDialog,
UpdateConfirmDialog,
} from "./catalog/download-card";
export { HubOptionMenu, type HubOption } from "./catalog/hub-option-menu";
export { DotTag } from "./catalog/dot-tag";
export { TransportConflictDialog } from "./catalog/transport-conflict-dialog";
export { TrainIcon } from "./components/train-icon";
export { isHiddenModelId } from "./lib/hidden-models";
export { classifyUnslothSupport } from "./lib/unsloth-support";

2
studio/frontend/src/features/hub/inventory/api.ts
View file

@ -48,6 +48,7 @@ export interface CachedGgufRepo {
capabilities?: BackendModelCapabilities | null;
size_bytes: number;
cache_path?: string;
last_modified?: number | null;
partial?: boolean;
partial_transport?: string | null;
pipeline_tag?: string | null;
@ -65,6 +66,7 @@ export interface CachedModelRepo {
capabilities?: BackendModelCapabilities | null;
size_bytes: number;
cache_path?: string;
last_modified?: number | null;
partial?: boolean;
partial_transport?: string | null;
pipeline_tag?: string | null;

3
studio/frontend/src/features/hub/inventory/types.ts
View file

@ -47,6 +47,7 @@ export interface CachedInventoryRow {
capabilities: ModelInventoryCapabilities;
bytes: number;
cachePath?: string | null;
lastModified?: number | null;
partial?: boolean;
partialTransport?: string | null;
pipelineTag?: string | null;
@ -65,6 +66,8 @@ export interface LocalInventoryRow {
title: string;
source: LocalSource;
sourceLabel: string;
modelId?: string | null;
displayName?: string;
path: string;
isGguf: boolean;
modelFormat: ModelInventoryFormat;

9
studio/frontend/src/features/hub/inventory/view-models.ts
View file

@ -176,6 +176,7 @@ export function buildCachedInventoryRow(
runtime?: string | null;
format_variant?: string | null;
capabilities?: BackendModelCapabilities | null;
last_modified?: number | null;
},
fallbackFormat: ModelInventoryFormat,
): CachedInventoryRow {
@ -210,6 +211,12 @@ export function buildCachedInventoryRow(
),
bytes: row.size_bytes,
cachePath: row.cache_path ?? null,
lastModified:
typeof row.last_modified === "number" &&
Number.isFinite(row.last_modified) &&
row.last_modified > 0
? row.last_modified
: null,
partial: row.partial ?? false,
partialTransport: row.partial_transport ?? null,
pipelineTag: row.pipeline_tag ?? null,
@ -272,6 +279,8 @@ export function buildLocalInventoryRows(
title,
source: model.source,
sourceLabel: localSourceLabel(model.source),
modelId: model.model_id ?? null,
displayName: model.display_name,
path: model.path,
isGguf: modelFormat === "gguf",
modelFormat,

20
studio/frontend/src/features/model-picker/api/model-metadata.ts Normal file
View file

@ -0,0 +1,20 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
import { getModelConfig } from "@/features/training";
export async function fetchModelMaxPositionEmbeddings(
modelName: string,
hfToken?: string | null,
signal?: AbortSignal,
): Promise<number | null> {
const config = await getModelConfig(
modelName,
signal,
hfToken?.trim() || undefined,
);
const value = config.max_position_embeddings;
return typeof value === "number" && Number.isFinite(value) && value > 0
? Math.floor(value)
: null;
}

51
studio/frontend/src/features/model-picker/api/templates.ts Normal file
View file

@ -0,0 +1,51 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
import { authFetch } from "@/features/auth";
import { hubTokenHeader } from "@/features/hub";
import { readFastApiError } from "@/lib/format-fastapi-error";
export interface ValidateChatTemplateResult {
valid: boolean;
error: string | null;
}
async function parseJsonOrThrow<T>(response: Response): Promise<T> {
if (!response.ok) {
throw new Error(await readFastApiError(response));
}
return response.json();
}
export async function validateChatTemplate(
template: string,
signal?: AbortSignal,
): Promise<ValidateChatTemplateResult> {
const response = await authFetch("/api/picker/validate-chat-template", {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ template }),
signal,
});
return parseJsonOrThrow<ValidateChatTemplateResult>(response);
}
export async function fetchDefaultChatTemplate(
modelName: string,
ggufVariant?: string | null,
hfToken?: string | null,
signal?: AbortSignal,
): Promise<string | null> {
const query = ggufVariant
? `?gguf_variant=${encodeURIComponent(ggufVariant)}`
: "";
const response = await authFetch(
`/api/picker/chat-template/${encodeURIComponent(modelName)}${query}`,
{ headers: hubTokenHeader(hfToken), signal },
);
const data = await parseJsonOrThrow<{
model_name: string;
chat_template: string | null;
}>(response);
return data.chat_template ?? null;
}

191
studio/frontend/src/features/model-picker/components/chat-template-editor-dialog.tsx Normal file
View file

@ -0,0 +1,191 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
import { Button } from "@/components/ui/button";
import {
Dialog,
DialogContent,
DialogDescription,
DialogFooter,
DialogHeader,
DialogTitle,
} from "@/components/ui/dialog";
import { Spinner } from "@/components/ui/spinner";
import { Textarea } from "@/components/ui/textarea";
import { useRef, useState } from "react";
import { validateChatTemplate } from "../api/templates";
import {
MAX_CHAT_TEMPLATE_BYTES,
chatTemplateByteLength,
isChatTemplateWithinLimit,
} from "../model-config/per-model-config";
interface ChatTemplateEditorDialogProps {
open: boolean;
onOpenChange: (open: boolean) => void;
value: string | null;
defaultTemplate: string | null;
defaultLoading: boolean;
onSave: (override: string | null) => void;
readOnly?: boolean;
}
export function ChatTemplateEditorDialog({
open,
onOpenChange,
value,
defaultTemplate,
defaultLoading,
onSave,
readOnly = false,
}: ChatTemplateEditorDialogProps) {
const [draft, setDraft] = useState<string | null>(null);
const [error, setError] = useState<string | null>(null);
const [validating, setValidating] = useState(false);
// Bumped whenever the dialog closes so a validation still in flight cannot
// apply a template the user has already dismissed.
const validationToken = useRef(0);
const renderedDraft = draft ?? value ?? defaultTemplate ?? "";
const byteLength = chatTemplateByteLength(renderedDraft);
const overLimit = !isChatTemplateWithinLimit(renderedDraft);
const matchesDefault =
defaultTemplate != null && renderedDraft === defaultTemplate;
const handleClose = () => {
validationToken.current += 1;
setDraft(null);
setError(null);
setValidating(false);
onOpenChange(false);
};
const handleSave = async () => {
if (renderedDraft.trim().length === 0 || matchesDefault) {
onSave(null);
handleClose();
return;
}
if (overLimit) {
setError("Template exceeds the size limit.");
return;
}
setValidating(true);
const token = validationToken.current;
try {
const result = await validateChatTemplate(renderedDraft);
// Dialog was closed (or reopened) while validating; drop the result so a
// discarded template is never applied.
if (token !== validationToken.current) {
return;
}
if (!result.valid) {
setError(result.error ?? "Invalid Jinja template.");
return;
}
onSave(renderedDraft);
handleClose();
} catch {
if (token === validationToken.current) {
setError("Could not validate the template.");
}
} finally {
if (token === validationToken.current) {
setValidating(false);
}
}
};
return (
<Dialog
open={open}
onOpenChange={(nextOpen) => {
if (nextOpen) {
onOpenChange(true);
return;
}
handleClose();
}}
>
<DialogContent className="corner-squircle dialog-soft-surface sm:max-w-3xl">
<DialogHeader>
<DialogTitle>
{readOnly ? "Chat Template" : "Edit Chat Template"}
</DialogTitle>
<DialogDescription>
{readOnly
? "This is the model's chat template. Custom templates apply to GGUF models for now, so it is view only for safetensors models."
: "Override the model's chat template with custom Jinja. The change applies when the model loads. Saving an empty template or one that matches the default clears the override."}
</DialogDescription>
</DialogHeader>
<Textarea
value={renderedDraft}
onChange={(event) => {
if (readOnly) return;
setDraft(event.target.value);
setError(null);
}}
readOnly={readOnly}
className="min-h-[20rem] max-h-[50vh] overflow-y-auto border-0 font-mono text-xs leading-5 corner-squircle focus-visible:ring-0"
rows={14}
spellCheck={false}
placeholder={defaultLoading ? "Loading model default..." : ""}
/>
{readOnly ? null : (
<div className="flex items-center justify-between gap-3 px-0.5 text-[11px]">
<span
className={overLimit ? "text-amber-500" : "text-muted-foreground"}
>
{byteLength.toLocaleString()} /{" "}
{MAX_CHAT_TEMPLATE_BYTES.toLocaleString()} bytes
</span>
{error ? (
<span className="truncate text-red-500" title={error}>
{error}
</span>
) : null}
</div>
)}
<DialogFooter className="flex-wrap gap-2 sm:justify-between">
{readOnly ? (
<div className="flex w-full justify-end">
<Button type="button" onClick={handleClose}>
Close
</Button>
</div>
) : (
<>
<Button
type="button"
variant="ghost"
onClick={() => setDraft(defaultTemplate ?? "")}
disabled={
defaultLoading || renderedDraft === (defaultTemplate ?? "")
}
className="text-muted-foreground"
>
{defaultLoading ? (
<Spinner className="size-3.5" />
) : (
"Reset to default"
)}
</Button>
<div className="flex gap-2">
<Button type="button" variant="ghost" onClick={handleClose}>
Cancel
</Button>
<Button
type="button"
onClick={handleSave}
disabled={validating || overLimit}
>
{validating ? "Validating..." : "Save"}
</Button>
</div>
</>
)}
</DialogFooter>
</DialogContent>
</Dialog>
);
}

742
studio/frontend/src/features/model-picker/components/model-config-page.tsx Normal file
View file

@ -0,0 +1,742 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
import { Button } from "@/components/ui/button";
import { Checkbox } from "@/components/ui/checkbox";
import { InfoHint } from "@/components/ui/info-hint";
import {
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue,
} from "@/components/ui/select";
import { Slider } from "@/components/ui/slider";
import { Switch } from "@/components/ui/switch";
import {
fetchGgufContextLength,
readPersistedSpeculativeType,
useChatRuntimeStore,
} from "@/features/chat";
import { ChevronDownStandardIcon } from "@/lib/chevron-icons";
import { toast } from "@/lib/toast";
import { ArrowLeft01Icon } from "@hugeicons/core-free-icons";
import { HugeiconsIcon } from "@hugeicons/react";
import { useEffect, useId, useState } from "react";
import {
useDefaultChatTemplate,
useModelMaxPositionEmbeddings,
} from "../hooks/use-model-defaults";
import { perModelConfigsEqual } from "../model-config/apply-per-model-config";
import {
DEFAULT_PER_MODEL_CONFIG,
KV_CACHE_DTYPES,
MAX_SEQ_LENGTH_MAX,
MAX_SEQ_LENGTH_MIN,
MAX_SEQ_LENGTH_STEP,
MTP_SPECULATIVE_TYPES,
type PerModelConfig,
SPECULATIVE_TYPES,
deletePerModelConfig,
isDefaultConfig,
normalizeMaxSeqLength,
resolveInitialConfig,
savePerModelConfig,
} from "../model-config/per-model-config";
import { ChatTemplateEditorDialog } from "./chat-template-editor-dialog";
import type { ModelPickTarget } from "./model-selector/types";
import { NumericValueInput } from "./numeric-value-input";
const ROW_CLASS = "flex min-h-8 items-center justify-between gap-3";
const LABEL_CLASS =
"min-w-0 truncate text-[13px] font-medium leading-[1.25] tracking-nav text-nav-fg";
const LABEL_CLASS_WRAP =
"min-w-0 text-[13px] font-medium leading-[1.25] tracking-nav text-nav-fg";
const CONTROL_SURFACE =
"rounded-full border-transparent bg-black/[0.04] dark:bg-white/[0.05] hover:bg-black/[0.06] dark:hover:bg-white/[0.1]";
const SELECT_TRIGGER_CLASS = `grid h-8 min-w-0 grid-cols-[minmax(0,1fr)_auto] items-center gap-1 ${CONTROL_SURFACE} pl-3 pr-2 py-0 text-[13px]! font-medium text-nav-fg focus-visible:ring-0 focus-visible:border-transparent [&_[data-slot=select-value]]:min-w-0 [&_[data-slot=select-value]]:truncate [&>svg]:shrink-0`;
const NUMBER_INPUT_CLASS = `h-8 w-[92px] ${CONTROL_SURFACE} pl-3 pr-2 py-0 text-right text-[13px] font-medium text-nav-fg outline-none focus-visible:ring-0`;
const KV_CACHE_DTYPE_DEFAULT = "f16";
const SPECULATIVE_TYPE_LABELS: Record<(typeof SPECULATIVE_TYPES)[number], string> =
{
auto: "Auto",
mtp: "MTP",
ngram: "Ngram",
"mtp+ngram": "MTP+Ngram",
off: "Off",
};
function hasNonDefaultAdvanced(config: PerModelConfig): boolean {
return (
config.kvCacheDtype != null ||
(config.speculativeType ?? "auto") !== "auto" ||
config.specDraftNMax != null ||
config.tensorParallel ||
config.chatTemplateOverride != null
);
}
function resolveCustomContextLength(
value: number,
native: number | null,
): number | null {
return native != null && value === native ? null : value;
}
function ChatTemplateSetting({
config,
onEditTemplate,
readOnly = false,
}: {
config: PerModelConfig;
onEditTemplate: () => void;
readOnly?: boolean;
}) {
return (
<div className={ROW_CLASS}>
<div className="flex min-w-0 items-center gap-1.5">
<span className={LABEL_CLASS}>Chat Template</span>
<InfoHint>
{readOnly
? "Preview the model's chat template. Custom overrides apply to GGUF models for now."
: "Override the model's chat template with custom Jinja. Applies when the model loads."}
</InfoHint>
</div>
<div className="flex shrink-0 items-center gap-2">
{readOnly ? null : (
<span className="text-[12px] text-muted-foreground">
{config.chatTemplateOverride ? "Custom" : "Default"}
</span>
)}
<Button
type="button"
size="sm"
variant="ghost"
className={`h-8 px-3 text-[13px] ${CONTROL_SURFACE}`}
onClick={onEditTemplate}
>
{readOnly ? "View" : "Edit"}
</Button>
</div>
</div>
);
}
function MaxSeqLengthSetting({
value,
max,
inputMax,
onChange,
}: {
value: number;
max: number;
inputMax: number;
onChange: (value: number) => void;
}) {
return (
<div className="space-y-3">
<div className={ROW_CLASS}>
<div className="flex min-w-0 items-center gap-1.5">
<span className={LABEL_CLASS}>Max Seq Length</span>
<InfoHint>
Maximum context window size in tokens. Applies when the model loads.
</InfoHint>
</div>
<NumericValueInput
value={value}
min={MAX_SEQ_LENGTH_MIN}
max={inputMax}
step={MAX_SEQ_LENGTH_STEP}
onChange={onChange}
ariaLabel="Max Seq Length"
className={NUMBER_INPUT_CLASS}
size={8}
/>
</div>
<Slider
min={MAX_SEQ_LENGTH_MIN}
max={max}
step={MAX_SEQ_LENGTH_STEP}
value={[value]}
onValueChange={([next]) => onChange(next)}
className="panel-slider"
aria-label="Max Seq Length"
/>
</div>
);
}
function clampMaxSeqLength(value: number, max: number): number {
const normalized = normalizeMaxSeqLength(value) ?? MAX_SEQ_LENGTH_MIN;
return Math.max(MAX_SEQ_LENGTH_MIN, Math.min(max, normalized));
}
function GgufAdvancedSettings({
config,
update,
isMtp,
speculativeFallback,
onEditTemplate,
}: {
config: PerModelConfig;
update: (patch: Partial<PerModelConfig>) => void;
isMtp: boolean;
speculativeFallback: string;
onEditTemplate: () => void;
}) {
return (
<>
<div className={ROW_CLASS}>
<div className="flex min-w-0 items-center gap-1.5">
<span className={LABEL_CLASS}>KV Cache Dtype</span>
<InfoHint>
Lower KV cache precision to save VRAM at the cost of some quality.
f16/bf16 are full precision; q8_0/q5_1/q4_1 are quantized.
</InfoHint>
</div>
<Select
value={config.kvCacheDtype ?? KV_CACHE_DTYPE_DEFAULT}
onValueChange={(v) =>
update({ kvCacheDtype: v === KV_CACHE_DTYPE_DEFAULT ? null : v })
}
>
<SelectTrigger
animateRadius={false}
icon={ChevronDownStandardIcon}
iconClassName="size-3.5"
className={`w-[92px] ${SELECT_TRIGGER_CLASS}`}
>
<SelectValue />
</SelectTrigger>
<SelectContent className="menu-soft-surface ring-0 border-0 rounded-lg">
<SelectItem value={KV_CACHE_DTYPE_DEFAULT}>
{KV_CACHE_DTYPE_DEFAULT}
</SelectItem>
{KV_CACHE_DTYPES.map((dtype) => (
<SelectItem key={dtype} value={dtype}>
{dtype}
</SelectItem>
))}
</SelectContent>
</Select>
</div>
<div className={ROW_CLASS}>
<div className="flex min-w-0 items-center gap-1.5">
<span className={LABEL_CLASS_WRAP}>Speculative Decoding</span>
<InfoHint>
Faster generation with no accuracy hit. Auto picks MTP / ngram based
on the model and platform. Pick a strategy to force it.
</InfoHint>
</div>
<Select
value={config.speculativeType ?? speculativeFallback}
onValueChange={(v) =>
update({
speculativeType: v,
specDraftNMax:
v === "mtp" || v === "mtp+ngram" ? config.specDraftNMax : null,
})
}
>
<SelectTrigger
animateRadius={false}
icon={ChevronDownStandardIcon}
iconClassName="size-3.5"
className={`w-[124px] shrink-0 ${SELECT_TRIGGER_CLASS}`}
>
<SelectValue />
</SelectTrigger>
<SelectContent className="menu-soft-surface ring-0 border-0 rounded-lg">
{SPECULATIVE_TYPES.map((type) => (
<SelectItem key={type} value={type}>
{SPECULATIVE_TYPE_LABELS[type]}
</SelectItem>
))}
</SelectContent>
</Select>
</div>
{isMtp && (
<div className={ROW_CLASS}>
<div className="flex min-w-0 items-center gap-1.5">
<span className={LABEL_CLASS}>Draft Tokens</span>
<InfoHint>
Max MTP draft tokens per step. Leave blank for the platform
default (2 on GPU, 3 on CPU/Mac).
</InfoHint>
</div>
<input
type="number"
min={1}
max={16}
step={1}
value={config.specDraftNMax ?? ""}
placeholder="auto"
onChange={(event) => {
const raw = event.target.value;
if (raw === "") {
update({ specDraftNMax: null });
return;
}
const parsed = Number.parseInt(raw, 10);
if (Number.isFinite(parsed)) {
update({ specDraftNMax: Math.max(1, Math.min(16, parsed)) });
}
}}
aria-label="Speculative decoding draft tokens"
className={NUMBER_INPUT_CLASS}
/>
</div>
)}
<div className={ROW_CLASS}>
<div className="flex min-w-0 items-center gap-1.5">
<span className={LABEL_CLASS}>Tensor Parallelism</span>
<InfoHint>
No effect on a single GPU. On multi-GPU setups, improves tokens/sec
for dense models. MoE models don't benefit.
</InfoHint>
</div>
<Switch
className="panel-switch shrink-0"
checked={config.tensorParallel}
onCheckedChange={(checked) => update({ tensorParallel: checked })}
/>
</div>
<ChatTemplateSetting config={config} onEditTemplate={onEditTemplate} />
</>
);
}
interface ModelConfigPageProps {
target: ModelPickTarget;
onBack?: () => void;
onRun: (config: PerModelConfig) => void;
loadedConfig?: PerModelConfig | null;
loadedContextLength?: number | null;
initialConfig?: PerModelConfig | null;
variant?: "page" | "sidebar";
}
export function ModelConfigPage({
target,
onBack,
onRun,
loadedConfig = null,
loadedContextLength = null,
initialConfig = null,
variant = "page",
}: ModelConfigPageProps) {
const rememberId = useId();
const isActiveModel = loadedConfig != null;
const runtimeMaxSeqLength = useChatRuntimeStore((s) => s.params.maxSeqLength);
const hfToken = useChatRuntimeStore((s) => s.hfToken);
const loadedDefaultChatTemplate = useChatRuntimeStore(
(s) => s.defaultChatTemplate,
);
const loadedMaxContextLength = useChatRuntimeStore(
(s) => s.ggufMaxContextLength,
);
// Only the active model seeds its max sequence length from the loaded
// runtime params. A different, unloaded model with no saved config seeds the
// app default instead, so opening its settings and loading does not inherit
// the currently loaded model's context.
const [initialMaxSeqLength] = useState(() =>
isActiveModel ? (normalizeMaxSeqLength(runtimeMaxSeqLength) ?? 4096) : 4096,
);
const resolveInitial = () => {
const resolved = resolveInitialConfig(target.id, target.ggufVariant);
if (loadedConfig) {
return { config: loadedConfig, remembered: resolved.remembered };
}
if (initialConfig) {
return {
config: initialConfig,
remembered:
resolved.remembered &&
perModelConfigsEqual(initialConfig, resolved.config),
};
}
return resolved;
};
const [initial] = useState(resolveInitial);
const [config, setConfig] = useState<PerModelConfig>(() => initial.config);
const [remember, setRemember] = useState(() => initial.remembered);
const [savedRemember, setSavedRemember] = useState(() => initial.remembered);
const [speculativeFallback] = useState(readPersistedSpeculativeType);
const [templateOpen, setTemplateOpen] = useState(false);
const [showAdvanced, setShowAdvanced] = useState(() =>
hasNonDefaultAdvanced(config),
);
const templateDefaults = useDefaultChatTemplate(
target.id,
target.ggufVariant,
templateOpen,
);
const modelMaxPosition = useModelMaxPositionEmbeddings(
target.id,
!target.isGguf,
);
const hasLoadedDefaultTemplate =
isActiveModel && loadedDefaultChatTemplate != null;
const resolvedDefaultTemplate = hasLoadedDefaultTemplate
? loadedDefaultChatTemplate
: templateDefaults.template;
const resolvedDefaultLoading = hasLoadedDefaultTemplate
? false
: templateDefaults.loading;
const update = (patch: Partial<PerModelConfig>) =>
setConfig((current) => ({ ...current, ...patch }));
const contextFetchKey =
target.isGguf && target.meta.contextLength == null
? `${target.id}\n${target.ggufVariant ?? ""}\n${hfToken || ""}`
: null;
const [fetchedContextLength, setFetchedContextLength] = useState<{
key: string;
value: number | null;
} | null>(null);
useEffect(() => {
if (contextFetchKey == null) {
return;
}
let cancelled = false;
void fetchGgufContextLength({
model_path: target.id,
gguf_variant: target.ggufVariant ?? null,
hf_token: hfToken || null,
})
.then((contextLength) => {
if (!cancelled) {
setFetchedContextLength({
key: contextFetchKey,
value: contextLength,
});
}
})
.catch(() => {
if (!cancelled) {
setFetchedContextLength({ key: contextFetchKey, value: null });
}
});
return () => {
cancelled = true;
};
}, [contextFetchKey, target.id, target.ggufVariant, hfToken]);
const isMtp =
config.speculativeType != null &&
MTP_SPECULATIVE_TYPES.has(config.speculativeType);
const nativeContextLength =
target.meta.contextLength ??
(fetchedContextLength?.key === contextFetchKey
? fetchedContextLength.value
: null);
const activeLoadedContext =
isActiveModel && target.isGguf ? loadedContextLength : null;
const minContext = 128;
const maxContext = Math.max(
minContext,
Math.max(
nativeContextLength ?? 0,
activeLoadedContext ?? 0,
config.customContextLength ?? 0,
) || 32768,
);
const contextValue = Math.min(
Math.max(
config.customContextLength ??
activeLoadedContext ??
nativeContextLength ??
maxContext,
minContext,
),
maxContext,
);
const setContextLength = (v: number) =>
update({
customContextLength:
nativeContextLength != null && v === nativeContextLength ? null : v,
});
const baseline = loadedConfig ?? DEFAULT_PER_MODEL_CONFIG;
const atBaseline = perModelConfigsEqual(config, baseline);
const contextAtDefault =
!target.isGguf ||
(nativeContextLength == null
? config.customContextLength == null
: contextValue === nativeContextLength);
const atDefault =
contextAtDefault &&
perModelConfigsEqual(
{ ...config, customContextLength: null },
DEFAULT_PER_MODEL_CONFIG,
);
const nativeMaxSeqLength =
normalizeMaxSeqLength(modelMaxPosition.maxPositionEmbeddings) ??
MAX_SEQ_LENGTH_MAX;
const maxSeqLengthValue =
normalizeMaxSeqLength(config.maxSeqLength) ??
clampMaxSeqLength(initialMaxSeqLength, nativeMaxSeqLength);
const maxSeqLengthMax = Math.max(nativeMaxSeqLength, maxSeqLengthValue);
const runtimeConfig = target.isGguf
? {
...config,
// Persist the user's intent collapsed against native (not the loaded
// context, which equals the override): an explicit sub-native value is
// kept, "at native" becomes null. null stays null (auto/VRAM-fit).
customContextLength:
config.customContextLength == null
? null
: resolveCustomContextLength(
config.customContextLength,
nativeContextLength,
),
}
: {
...config,
maxSeqLength: maxSeqLengthValue,
};
const rememberChanged = remember !== savedRemember;
const persistenceOnly = isActiveModel && atBaseline && rememberChanged;
const primaryActionLabel = persistenceOnly
? remember
? "Save settings"
: "Forget settings"
: isActiveModel
? "Reload model"
: "Load model";
const handleRun = () => {
const defaultConfig = isDefaultConfig(runtimeConfig);
let saveFailed = false;
if (remember) {
saveFailed = !savePerModelConfig(
target.id,
target.ggufVariant,
runtimeConfig,
);
} else {
saveFailed = !deletePerModelConfig(target.id, target.ggufVariant);
}
if (persistenceOnly) {
if (saveFailed) {
toast.error("Couldn't save settings for this model.");
return;
}
const nextRemember = remember && !defaultConfig;
setSavedRemember(nextRemember);
setRemember(nextRemember);
toast.success(
nextRemember
? "Settings saved."
: remember
? "Default settings kept."
: "Settings forgotten.",
);
return;
}
if (saveFailed) {
toast.error("Couldn't save these settings, loading with them anyway.");
}
onRun(runtimeConfig);
};
return (
<div className="flex flex-col">
{variant === "page" && (
<div className="flex items-center gap-2.5 pb-4">
{onBack && (
<button
type="button"
onClick={onBack}
className="nav-icon-btn shrink-0 text-nav-icon-idle hover:bg-panel-surface-hover hover:text-black dark:hover:text-white"
aria-label="Back to model list"
>
<HugeiconsIcon
icon={ArrowLeft01Icon}
className="size-4"
strokeWidth={1.75}
/>
</button>
)}
<div className="min-w-0 flex-1">
<div className="text-[10px] font-semibold uppercase leading-none tracking-wider text-muted-foreground">
Run settings
</div>
<div className="mt-1.5 truncate text-[14px] font-semibold leading-tight text-nav-fg">
{target.displayName}
</div>
</div>
</div>
)}
<div className="space-y-3.5">
{target.isGguf && (
<>
<div className="space-y-3">
<div className={ROW_CLASS}>
<div className="flex min-w-0 items-center gap-1.5">
<span className={LABEL_CLASS}>Context Length</span>
<InfoHint>
Tokens of context to allocate. Higher uses more VRAM.
{nativeContextLength != null
? ` This model's native context is ${nativeContextLength.toLocaleString()} tokens.`
: ""}
</InfoHint>
</div>
<NumericValueInput
value={contextValue}
min={minContext}
max={maxContext}
step={1}
onChange={setContextLength}
ariaLabel="Context Length"
className={NUMBER_INPUT_CLASS}
size={8}
/>
</div>
{nativeContextLength != null ? (
<Slider
min={minContext}
max={maxContext}
step={128}
value={[contextValue]}
onValueChange={([v]) => setContextLength(v)}
className="panel-slider"
aria-label="Context Length"
/>
) : null}
{isActiveModel &&
loadedMaxContextLength != null &&
contextValue > loadedMaxContextLength && (
<p className="text-[11px] text-amber-500">
Exceeds estimated VRAM capacity (
{loadedMaxContextLength.toLocaleString()} tokens). The model
may use system RAM.
</p>
)}
</div>
{showAdvanced && (
<GgufAdvancedSettings
config={config}
update={update}
isMtp={isMtp}
speculativeFallback={speculativeFallback}
onEditTemplate={() => setTemplateOpen(true)}
/>
)}
<div className={ROW_CLASS}>
<div className="flex min-w-0 items-center gap-1.5">
<span className="min-w-0 text-[13px] font-medium leading-[1.25] tracking-nav text-muted-foreground">
Advanced settings
</span>
<InfoHint>
Extra options for how the model loads. Most setups don't need
these.
</InfoHint>
</div>
<Switch
className="panel-switch shrink-0"
checked={showAdvanced}
onCheckedChange={setShowAdvanced}
aria-label="Show advanced settings"
/>
</div>
</>
)}
{!target.isGguf && (
<>
<MaxSeqLengthSetting
value={maxSeqLengthValue}
max={maxSeqLengthMax}
inputMax={MAX_SEQ_LENGTH_MAX}
onChange={(value) =>
update({
maxSeqLength: clampMaxSeqLength(value, MAX_SEQ_LENGTH_MAX),
})
}
/>
<ChatTemplateSetting
config={config}
onEditTemplate={() => setTemplateOpen(true)}
readOnly={true}
/>
</>
)}
</div>
<div
className={
variant === "sidebar"
? "mt-4 flex flex-col gap-3 border-t border-border/60 pt-4"
: "mt-4 flex items-center justify-between gap-3 border-t border-border/60 pt-4"
}
>
<div className="flex min-w-0 items-center gap-2">
<Checkbox
id={rememberId}
checked={remember}
onCheckedChange={(checked) => setRemember(checked === true)}
/>
<label
htmlFor={rememberId}
className="cursor-pointer select-none truncate text-[13px] text-nav-fg"
>
Remember for this model
</label>
</div>
<div
className={
variant === "sidebar"
? "flex items-center justify-end gap-2"
: "flex shrink-0 items-center gap-2"
}
>
<Button
type="button"
variant="ghost"
size="sm"
className="h-8"
disabled={atDefault}
onClick={() =>
setConfig({
...DEFAULT_PER_MODEL_CONFIG,
customContextLength:
target.isGguf && nativeContextLength != null
? nativeContextLength
: null,
})
}
>
Reset
</Button>
<Button
type="button"
size="sm"
className="h-8"
disabled={isActiveModel && atBaseline && !rememberChanged}
onClick={handleRun}
>
{primaryActionLabel}
</Button>
</div>
</div>
<ChatTemplateEditorDialog
open={templateOpen}
onOpenChange={setTemplateOpen}
value={config.chatTemplateOverride}
defaultTemplate={resolvedDefaultTemplate}
defaultLoading={resolvedDefaultLoading}
readOnly={!target.isGguf}
onSave={(override) => update({ chatTemplateOverride: override })}
/>
</div>
);
}

130
studio/frontend/src/components/assistant-ui/model-selector.tsx → studio/frontend/src/features/model-picker/components/model-selector.tsx
View file

@ -3,6 +3,7 @@
"use client";
import { Input } from "@/components/ui/input";
import {
Popover,
PopoverContent,
@ -10,7 +11,7 @@ import {
} from "@/components/ui/popover";
import { TooltipProvider } from "@/components/ui/tooltip";
import { usePlatformStore } from "@/config/env";
import { isCustomProviderType } from "@/features/chat/external-providers";
import { isCustomProviderType } from "@/features/chat";
import { ChevronDownStandardIcon } from "@/lib/chevron-icons";
import { cn } from "@/lib/utils";
import {
@ -33,7 +34,11 @@ import {
useRef,
useState,
} from "react";
import { Input } from "../ui/input";
import {
type PerModelConfig,
resolveInitialConfig,
} from "../model-config/per-model-config";
import { ModelConfigPage } from "./model-config-page";
import { HubModelPicker, hasDownloadedModels } from "./model-selector/pickers";
import { PillTabs } from "./model-selector/pill-tabs";
import {
@ -45,6 +50,7 @@ import type {
ExternalModelOption,
LoraModelOption,
ModelOption,
ModelPickTarget,
ModelSelectorChangeMeta,
} from "./model-selector/types";
@ -122,6 +128,10 @@ interface ModelSelectorProps {
value?: string;
defaultValue?: string;
activeGgufVariant?: string | null;
activeModelConfig?: PerModelConfig | null;
activeGgufContextLength?: number | null;
selectedConfig?: PerModelConfig | null;
selectedGgufVariant?: string | null;
onValueChange?: (value: string, meta: ModelSelectorChangeMeta) => void;
onEject?: () => void;
onFoldersChange?: () => void;
@ -285,7 +295,8 @@ function saveLastHubSection(section: HubSection): void {
// when they have downloads, else Recommended.
function defaultHubSection(): HubSection {
return (
loadLastHubSection() ?? (hasDownloadedModels() ? "downloaded" : "recommended")
loadLastHubSection() ??
(hasDownloadedModels() ? "downloaded" : "recommended")
);
}
@ -308,6 +319,11 @@ function ModelSelectorContent({
loraModels,
externalModels,
value,
activeGgufVariant,
activeModelConfig,
activeGgufContextLength,
selectedConfig,
selectedGgufVariant,
onSelect,
onEject,
onFoldersChange,
@ -323,6 +339,11 @@ function ModelSelectorContent({
loraModels: LoraModelOption[];
externalModels: ExternalModelOption[];
value?: string;
activeGgufVariant?: string | null;
activeModelConfig?: PerModelConfig | null;
activeGgufContextLength?: number | null;
selectedConfig?: PerModelConfig | null;
selectedGgufVariant?: string | null;
onSelect: (id: string, meta: ModelSelectorChangeMeta) => void;
onEject?: () => void;
onFoldersChange?: () => void;
@ -391,6 +412,10 @@ function ModelSelectorContent({
const effectiveHubSection: HubSection =
hubSection === "connected" && !hasExternal ? "recommended" : hubSection;
const [configTarget, setConfigTarget] = useState<ModelPickTarget | null>(
null,
);
// The picker below remounts on each open, but this tab state does not, so a
// persisted selection that lands in lora/external after async load would
// reopen on Hub. Re-derive the default tab on the open edge.
@ -402,6 +427,9 @@ function ModelSelectorContent({
// user has downloads, else their last section.
setHubSection(wantsConnectedDefault ? "connected" : defaultHubSection());
}
if (!open && wasOpen.current) {
setConfigTarget(null);
}
wasOpen.current = open;
}, [
open,
@ -452,6 +480,29 @@ function ModelSelectorContent({
}
}
const visibleConfigTarget = open ? configTarget : null;
const openConfigPage = (id: string, meta: ModelSelectorChangeMeta) => {
const leaf = id.includes("/") ? id.slice(id.lastIndexOf("/") + 1) : id;
setConfigTarget({
id,
displayName: meta.ggufVariant ? `${leaf} · ${meta.ggufVariant}` : leaf,
ggufVariant: meta.ggufVariant ?? null,
isGguf: meta.isGguf ?? Boolean(meta.ggufVariant),
meta,
});
};
const handlePick = (id: string, meta: ModelSelectorChangeMeta) => {
if (meta.source === "external") {
onSelect(id, meta);
return;
}
const resolved = resolveInitialConfig(id, meta.ggufVariant);
onSelect(id, {
...meta,
...(resolved.remembered ? { config: resolved.config } : {}),
});
};
return (
<PopoverContent
align="start"
@ -459,12 +510,17 @@ function ModelSelectorContent({
data-tour={dataTour}
onKeyDown={handlePickerEntryKeyDown}
className={cn(
"unsloth-model-selector-menu menu-soft-surface ring-0 max-w-[calc(100vw-1rem)] min-w-0 gap-0 pt-4 pb-0 pl-4",
// Sized so the left-packed row keeps uniform gaps and the last dropdown's
// right gap matches the pill's left gap (pl-4 vs pr-4).
hasExternal
? "w-[min(614px,calc(100vw-1rem))] pr-4"
: "w-[min(506px,calc(100vw-1rem))] pr-2",
"unsloth-model-selector-menu menu-soft-surface ring-0 max-w-[calc(100vw-1rem)] min-w-0 gap-0",
visibleConfigTarget
? "w-[min(468px,calc(100vw-1rem))] px-4 pt-4 pb-4"
: cn(
"pt-4 pb-0 pl-4",
// Sized so the left-packed row keeps uniform gaps and the last
// dropdown's right gap matches the pill's left gap (pl-4 vs pr-4).
hasExternal
? "w-[min(614px,calc(100vw-1rem))] pr-4"
: "w-[min(506px,calc(100vw-1rem))] pr-2",
),
className,
)}
>
@ -477,6 +533,42 @@ function ModelSelectorContent({
skipDelayDuration={0}
disableHoverableContent={true}
>
{visibleConfigTarget ? (
<ModelConfigPage
key={`${visibleConfigTarget.id}::${visibleConfigTarget.ggufVariant ?? ""}`}
target={visibleConfigTarget}
onBack={() => setConfigTarget(null)}
onRun={(config) =>
onSelect(visibleConfigTarget.id, {
...visibleConfigTarget.meta,
config,
forceReload: true,
})
}
loadedConfig={
value === visibleConfigTarget.id &&
(activeGgufVariant ?? null) ===
(visibleConfigTarget.ggufVariant ?? null)
? (activeModelConfig ?? null)
: null
}
loadedContextLength={
value === visibleConfigTarget.id &&
(activeGgufVariant ?? null) ===
(visibleConfigTarget.ggufVariant ?? null)
? (activeGgufContextLength ?? null)
: null
}
initialConfig={
value === visibleConfigTarget.id &&
(selectedGgufVariant ?? null) ===
(visibleConfigTarget.ggufVariant ?? null)
? (selectedConfig ?? null)
: null
}
/>
) : (
<>
{tabs.length > 1 ? (
<PillTabs
ariaLabel="Model source"
@ -494,13 +586,14 @@ function ModelSelectorContent({
loraModels={fineTunedModels}
externalModels={externalModels}
value={value}
onSelect={onSelect}
onSelect={handlePick}
onFoldersChange={onFoldersChange}
onBrowseHub={onBrowseHub}
onModelsChange={onModelsChange}
onConfigure={openConfigPage}
deleteDisabled={deleteDisabled}
section={effectiveHubSection}
onEject={hasSelection && onEject ? onEject : undefined}
section={effectiveHubSection}
sectionToggle={
<PillTabs
ariaLabel="Hub section"
@ -538,10 +631,8 @@ function ModelSelectorContent({
</button>
</div>
) : null}
{/* Hub renders Eject inline as the last list row; other tabs keep the
footer button. */}
{effectiveTab !== "hub" && hasSelection && onEject ? (
<div className="mt-1.5 pt-1.5">
<div className="mt-1.5 border-t border-border/70 pt-1.5 pb-2">
<button
type="button"
onClick={onEject}
@ -553,6 +644,8 @@ function ModelSelectorContent({
</button>
</div>
) : null}
</>
)}
</TooltipProvider>
</PopoverContent>
);
@ -565,6 +658,10 @@ export function ModelSelector({
value,
defaultValue,
activeGgufVariant,
activeModelConfig,
activeGgufContextLength,
selectedConfig,
selectedGgufVariant,
onValueChange,
onEject,
onFoldersChange,
@ -693,6 +790,11 @@ export function ModelSelector({
loraModels={loraModels}
externalModels={externalModels}
value={selected}
activeGgufVariant={activeGgufVariant}
activeModelConfig={activeModelConfig}
activeGgufContextLength={activeGgufContextLength}
selectedConfig={selectedConfig}
selectedGgufVariant={selectedGgufVariant}
onSelect={handleSelect}
onEject={onEject ? handleEject : undefined}
onFoldersChange={onFoldersChange}

79
studio/frontend/src/components/assistant-ui/model-selector/folder-browser.tsx → studio/frontend/src/features/model-picker/components/model-selector/folder-browser.tsx
View file

@ -14,10 +14,7 @@ import {
DialogTitle,
} from "@/components/ui/dialog";
import { Spinner } from "@/components/ui/spinner";
import {
type BrowseFoldersResponse,
browseFolders,
} from "@/features/chat/api/chat-api";
import { type BrowseFoldersResponse, browseFolders } from "@/features/chat";
import { ChevronUpStandardIcon } from "@/lib/chevron-icons";
import { cn } from "@/lib/utils";
import { Folder02Icon } from "@hugeicons/core-free-icons";
@ -90,47 +87,43 @@ export function FolderBrowser({
const [error, setError] = useState<string | null>(null);
const abortRef = useRef<AbortController | null>(null);
const navigate = useCallback(
(
target: string | undefined,
hidden: boolean,
opts?: { fallbackOnError?: boolean },
) => {
abortRef.current?.abort();
const ctrl = new AbortController();
abortRef.current = ctrl;
setLoading(true);
setError(null);
// Forward the signal so cancelled navigation aborts the backend
// enumeration, not just the response.
browseFolders(target, hidden, ctrl.signal)
.then((res) => {
if (ctrl.signal.aborted) return;
setData(res);
setPath(res.current);
})
.catch((err) => {
if (ctrl.signal.aborted) return;
// Surface the error; if the first request (e.g. a bad initialPath)
// fails, fall back to HOME so the modal stays navigable.
const message = err instanceof Error ? err.message : String(err);
setError(message);
if (opts?.fallbackOnError && target !== undefined) {
// Re-issue without a target -> backend defaults to HOME.
// Don't recurse if HOME itself fails (allowlist always has HOME).
queueMicrotask(() => navigate(undefined, hidden));
}
})
.finally(() => {
if (!ctrl.signal.aborted) setLoading(false);
});
},
[],
);
function navigate(
target: string | undefined,
hidden: boolean,
opts?: { fallbackOnError?: boolean },
) {
abortRef.current?.abort();
const ctrl = new AbortController();
abortRef.current = ctrl;
setLoading(true);
setError(null);
// Forward the signal so cancelled navigation aborts the backend
// enumeration, not just the response.
browseFolders(target, hidden, ctrl.signal)
.then((res) => {
if (ctrl.signal.aborted) return;
setData(res);
setPath(res.current);
})
.catch((err) => {
if (ctrl.signal.aborted) return;
// Surface the error; if the first request (e.g. a bad initialPath)
// fails, fall back to HOME so the modal stays navigable.
const message = err instanceof Error ? err.message : String(err);
setError(message);
if (opts?.fallbackOnError && target !== undefined) {
// Re-issue without a target -> backend defaults to HOME.
// Don't recurse if HOME itself fails (allowlist always has HOME).
queueMicrotask(() => navigate(undefined, hidden));
}
})
.finally(() => {
if (!ctrl.signal.aborted) setLoading(false);
});
}
// Fetch only on closed -> open; later navigation is driven by `navigate()`,
// so `path` is deliberately kept out of the dependency list.
// eslint-disable-next-line react-hooks/exhaustive-deps
useEffect(() => {
if (!open) return;
// fallbackOnError: recover into HOME if initialPath is bad, rather than
@ -147,7 +140,7 @@ export function FolderBrowser({
const crumbs = useMemo(
() => (data?.current ? splitBreadcrumb(data.current) : []),
[data?.current],
[data],
);
return (

0
studio/frontend/src/components/assistant-ui/model-selector/model-capabilities.ts → studio/frontend/src/features/model-picker/components/model-selector/model-capabilities.ts
View file

9
studio/frontend/src/components/assistant-ui/model-selector/model-delete-action.tsx → studio/frontend/src/features/model-picker/components/model-selector/model-delete-action.tsx
View file

@ -1,12 +1,12 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
import { DeleteConfirmDialog } from "@/features/hub/catalog/download-card";
import { DeleteConfirmDialog } from "@/features/hub";
import { toast } from "@/lib/toast";
import { cn } from "@/lib/utils";
import { Delete02Icon } from "@hugeicons/core-free-icons";
import { HugeiconsIcon } from "@hugeicons/react";
import { useCallback, useState, type ReactNode } from "react";
import { toast } from "@/lib/toast";
import { type ReactNode, useCallback, useState } from "react";
interface ModelDeleteActionProps {
ariaLabel: string;
@ -63,7 +63,8 @@ export function ModelDeleteAction({
disabled={disabled}
className={cn(
"shrink-0 rounded-md p-1.5 text-muted-foreground/60 transition-colors hover:bg-destructive/10 hover:text-destructive",
disabled && "cursor-not-allowed opacity-40 hover:bg-transparent hover:text-muted-foreground/60",
disabled &&
"cursor-not-allowed opacity-40 hover:bg-transparent hover:text-muted-foreground/60",
buttonClassName,
)}
>

19
studio/frontend/src/components/assistant-ui/model-selector/model-load-settings-action.tsx → studio/frontend/src/features/model-picker/components/model-selector/model-load-settings-action.tsx
View file

@ -6,24 +6,16 @@ import {
TooltipContent,
TooltipTrigger,
} from "@/components/ui/tooltip";
import { useChatRuntimeStore } from "@/features/chat/stores/chat-runtime-store";
import { cn } from "@/lib/utils";
import { Settings02Icon } from "@hugeicons/core-free-icons";
import { HugeiconsIcon } from "@hugeicons/react";
/** Gear button on a downloaded quant row. Stages the model into the Run
* settings sidebar (always, regardless of the Load-on-selection toggle) so the
* user can set load options, then click Load model. */
export function ModelLoadSettingsAction({
ariaLabel,
repoId,
quant,
maxContext,
onConfigure,
}: {
ariaLabel: string;
repoId: string;
quant: string;
maxContext?: number | null;
onConfigure: () => void;
}) {
return (
<Tooltip delayDuration={0}>
@ -32,12 +24,7 @@ export function ModelLoadSettingsAction({
type="button"
onClick={(e) => {
e.stopPropagation();
useChatRuntimeStore.getState().stageModel({
id: repoId,
ggufVariant: quant,
isDownloaded: true,
contextLength: maxContext ?? null,
});
onConfigure();
}}
aria-label={ariaLabel}
className={cn(

25
studio/frontend/src/components/assistant-ui/model-selector/model-update-action.tsx → studio/frontend/src/features/model-picker/components/model-selector/model-update-action.tsx
View file

@ -1,12 +1,20 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
import { subscribeJobListeners } from "@/features/hub/download-manager";
import { UpdateConfirmDialog } from "@/features/hub/catalog/download-card";
import { ggufVariantsMatch } from "@/features/hub/lib/model-identity";
import {
UpdateConfirmDialog,
ggufVariantsMatch,
subscribeJobListeners,
} from "@/features/hub";
import { cn } from "@/lib/utils";
import { RefreshCw } from "lucide-react";
import { useCallback, useEffect, useRef, useState, type ReactNode } from "react";
import {
type ReactNode,
useCallback,
useEffect,
useRef,
useState,
} from "react";
import { toast } from "sonner";
interface ModelUpdateActionProps {
@ -42,10 +50,10 @@ export function ModelUpdateAction({
}: ModelUpdateActionProps) {
const [open, setOpen] = useState(false);
// Refresh the caller when this repo+variant's download finishes so the "update available" cue
// clears. A ref keeps the subscription stable across renders.
const onUpdatedRef = useRef(onUpdated);
onUpdatedRef.current = onUpdated;
useEffect(() => {
onUpdatedRef.current = onUpdated;
}, [onUpdated]);
useEffect(() => {
return subscribeJobListeners("model", repoId, {
onComplete: (completedVariant) => {
@ -83,7 +91,8 @@ export function ModelUpdateAction({
disabled={disabled}
className={cn(
"shrink-0 rounded-md p-1.5 text-muted-foreground/60 transition-colors hover:bg-amber-500/10 hover:text-amber-700 dark:hover:bg-amber-500/15 dark:hover:text-amber-300",
disabled && "cursor-not-allowed opacity-40 hover:bg-transparent hover:text-muted-foreground/60",
disabled &&
"cursor-not-allowed opacity-40 hover:bg-transparent hover:text-muted-foreground/60",
buttonClassName,
)}
>

3
studio/frontend/src/components/assistant-ui/model-selector/model-usage.ts → studio/frontend/src/features/model-picker/components/model-selector/model-usage.ts
View file

@ -40,7 +40,8 @@ export function loadedAt(times: ModelLoadTimes, id: string): number {
export function useModelLoadTimes(currentValue?: string): ModelLoadTimes {
const [times, setTimes] = useState<ModelLoadTimes>(() => readLoadTimes());
useEffect(() => {
if (currentValue) setTimes(recordModelLoaded(currentValue));
if (!currentValue) return;
queueMicrotask(() => setTimes(recordModelLoaded(currentValue)));
}, [currentValue]);
return times;
}

645
studio/frontend/src/components/assistant-ui/model-selector/pickers.tsx → studio/frontend/src/features/model-picker/components/model-selector/pickers.tsx
View file

File diff suppressed because it is too large Load diff

3
studio/frontend/src/components/assistant-ui/model-selector/pill-tabs.tsx → studio/frontend/src/features/model-picker/components/model-selector/pill-tabs.tsx
View file

@ -78,7 +78,8 @@ export function PillTabs({
onValueChange(tabs[next].value);
e.currentTarget.parentElement
?.querySelectorAll<HTMLElement>('button[role="tab"]')
[next]?.focus();
.item(next)
?.focus();
}}
onClick={() => onValueChange(tab.value)}
className={cn(

0
studio/frontend/src/components/assistant-ui/model-selector/recommended-fit.ts → studio/frontend/src/features/model-picker/components/model-selector/recommended-fit.ts
View file

0
studio/frontend/src/components/assistant-ui/model-selector/row-meta.ts → studio/frontend/src/features/model-picker/components/model-selector/row-meta.ts
View file

0
studio/frontend/src/components/assistant-ui/model-selector/source-tabs.ts → studio/frontend/src/features/model-picker/components/model-selector/source-tabs.ts
View file

13
studio/frontend/src/components/assistant-ui/model-selector/types.ts → studio/frontend/src/features/model-picker/components/model-selector/types.ts
View file

@ -2,6 +2,7 @@
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
import type { ReactNode } from "react";
import type { PerModelConfig } from "../../model-config/per-model-config";
export interface ModelOption {
id: string;
@ -36,6 +37,18 @@ export interface ModelSelectorChangeMeta {
/** Direct local .gguf file picked without a variant (custom folder / LM
* Studio). Marks it as a GGUF source for the deferred-load staging flow. */
isGguf?: boolean;
config?: PerModelConfig;
forceReload?: boolean;
/** Native path token so an active-model reload can reopen a file-picked GGUF. */
nativePathToken?: string;
}
export interface ModelPickTarget {
id: string;
displayName: string;
ggufVariant?: string | null;
isGguf: boolean;
meta: ModelSelectorChangeMeta;
}
export interface DeletedModelRef {

113
studio/frontend/src/features/model-picker/components/numeric-value-input.tsx Normal file
View file

@ -0,0 +1,113 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
import { cn } from "@/lib/utils";
import { useRef, useState } from "react";
export function snapToStep(
value: number,
step: number,
min?: number,
max?: number,
): number {
const lo = min ?? Number.NEGATIVE_INFINITY;
const hi = max ?? Number.POSITIVE_INFINITY;
const clamped = Math.min(Math.max(value, lo), hi);
const stepStr = String(step);
const decimals = stepStr.includes(".") ? stepStr.split(".")[1].length : 0;
const base = Number.isFinite(lo) ? lo : 0;
const snapped = base + Math.round((clamped - base) / step) * step;
const reclamped = Math.min(Math.max(snapped, lo), hi);
return Number(reclamped.toFixed(decimals));
}
function sanitizeNumeric(raw: string, allowNegative: boolean): string {
const sign = allowNegative && raw.startsWith("-") ? "-" : "";
const [head, ...rest] = raw.replace(/[^\d.]/g, "").split(".");
const tail = rest.length > 0 ? `.${rest.join("")}` : "";
return `${sign}${head}${tail}`;
}
export function NumericValueInput({
value,
min,
max,
step,
onChange,
displayValue,
className,
ariaLabel,
size: sizeAttr,
disabled = false,
}: {
value: number;
min?: number;
max?: number;
step: number;
onChange: (v: number) => void;
displayValue?: string;
className?: string;
ariaLabel?: string;
size?: number;
disabled?: boolean;
}) {
const [focused, setFocused] = useState(false);
const [draft, setDraft] = useState("");
const cancelBlurCommitRef = useRef(false);
const commit = (raw: string) => {
const parsed = Number.parseFloat(raw);
if (!Number.isFinite(parsed)) {
return;
}
const final = snapToStep(parsed, step, min, max);
if (final !== value) {
onChange(final);
}
};
const displayed = focused ? draft : (displayValue ?? String(value));
return (
<input
type="text"
inputMode="decimal"
disabled={disabled}
size={sizeAttr}
style={{
boxSizing: "content-box",
width: `calc(${Math.max(displayed.length, 4)}ch + 2px)`,
}}
value={displayed}
aria-label={ariaLabel}
onFocus={(e) => {
cancelBlurCommitRef.current = false;
setDraft(String(value));
setFocused(true);
const target = e.currentTarget;
requestAnimationFrame(() => target.select());
}}
onBlur={() => {
if (cancelBlurCommitRef.current) {
cancelBlurCommitRef.current = false;
} else {
commit(draft);
}
setFocused(false);
}}
onChange={(e) =>
setDraft(sanitizeNumeric(e.target.value, (min ?? 0) < 0))
}
onKeyDown={(e) => {
if (e.key === "Enter") {
e.currentTarget.blur();
} else if (e.key === "Escape") {
cancelBlurCommitRef.current = true;
setDraft(String(value));
e.currentTarget.blur();
}
}}
className={cn(className)}
/>
);
}

89
studio/frontend/src/features/model-picker/components/sidebar-model-config.tsx Normal file
View file

@ -0,0 +1,89 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
import { useMemo } from "react";
import type { PerModelConfig } from "../model-config/per-model-config";
import { ModelConfigPage } from "./model-config-page";
import type { ModelPickTarget } from "./model-selector/types";
interface SidebarModelConfigProps {
modelId: string;
ggufVariant: string | null;
isGguf: boolean;
nativeContextLength: number | null;
loadedContextLength: number | null;
loadedConfig: PerModelConfig;
onReload: (config: PerModelConfig) => void;
}
const TRAILING_SEPARATORS = /[\\/]+$/;
function leafName(id: string): string {
const trimmed = id.replace(TRAILING_SEPARATORS, "");
const separator = Math.max(
trimmed.lastIndexOf("/"),
trimmed.lastIndexOf("\\"),
);
return separator >= 0 ? trimmed.slice(separator + 1) : trimmed;
}
function hashString(value: string): number {
let hash = 5381;
for (let i = 0; i < value.length; i += 1) {
hash = (Math.imul(hash, 33) ^ value.charCodeAt(i)) >>> 0;
}
return hash;
}
function configSignature(config: PerModelConfig): string {
return [
config.customContextLength ?? "",
config.maxSeqLength ?? "",
config.kvCacheDtype ?? "",
config.speculativeType ?? "",
config.specDraftNMax ?? "",
config.tensorParallel ? "1" : "0",
config.chatTemplateOverride == null
? ""
: `${config.chatTemplateOverride.length}:${hashString(config.chatTemplateOverride)}`,
].join("|");
}
export function SidebarModelConfig({
modelId,
ggufVariant,
isGguf,
nativeContextLength,
loadedContextLength,
loadedConfig,
onReload,
}: SidebarModelConfigProps) {
const target = useMemo<ModelPickTarget>(() => {
const leaf = leafName(modelId);
return {
id: modelId,
displayName: ggufVariant ? `${leaf} · ${ggufVariant}` : leaf,
ggufVariant,
isGguf,
meta: {
source: "local",
isLora: false,
ggufVariant: ggufVariant ?? undefined,
isGguf,
isDownloaded: true,
contextLength: nativeContextLength,
},
};
}, [modelId, ggufVariant, isGguf, nativeContextLength]);
return (
<ModelConfigPage
key={`${modelId}::${ggufVariant ?? ""}::${configSignature(loadedConfig)}`}
target={target}
onRun={onReload}
loadedConfig={loadedConfig}
loadedContextLength={loadedContextLength}
variant="sidebar"
/>
);
}

176
studio/frontend/src/features/model-picker/hooks/use-model-defaults.ts Normal file
View file

@ -0,0 +1,176 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
import { useHfTokenStore } from "@/features/hub";
import { useEffect, useState } from "react";
import { fetchModelMaxPositionEmbeddings } from "../api/model-metadata";
import { fetchDefaultChatTemplate } from "../api/templates";
export interface DefaultChatTemplateState {
template: string | null;
loading: boolean;
error: string | null;
}
export interface ModelMaxPositionState {
maxPositionEmbeddings: number | null;
loading: boolean;
error: string | null;
}
const TEMPLATE_CACHE_MAX_ENTRIES = 50;
const templateCache = new Map<string, string | null>();
const maxPositionCache = new Map<string, number | null>();
function cacheTemplate(key: string, template: string | null): void {
templateCache.delete(key);
templateCache.set(key, template);
while (templateCache.size > TEMPLATE_CACHE_MAX_ENTRIES) {
const oldest = templateCache.keys().next().value;
if (oldest === undefined) {
break;
}
templateCache.delete(oldest);
}
}
function cacheMaxPosition(key: string, value: number | null): void {
maxPositionCache.delete(key);
maxPositionCache.set(key, value);
while (maxPositionCache.size > TEMPLATE_CACHE_MAX_ENTRIES) {
const oldest = maxPositionCache.keys().next().value;
if (oldest === undefined) {
break;
}
maxPositionCache.delete(oldest);
}
}
export function useDefaultChatTemplate(
modelId: string | null,
ggufVariant: string | null | undefined,
enabled: boolean,
): DefaultChatTemplateState {
const token = useHfTokenStore((s) => s.token);
const cacheKey =
enabled && modelId ? `${modelId}::${ggufVariant ?? ""}::${token}` : null;
const [fetched, setFetched] = useState<{
key: string;
state: DefaultChatTemplateState;
} | null>(null);
useEffect(() => {
if (cacheKey == null || !modelId || templateCache.has(cacheKey)) {
return;
}
const controller = new AbortController();
fetchDefaultChatTemplate(modelId, ggufVariant, token, controller.signal)
.then((template) => {
if (controller.signal.aborted) {
return;
}
// Cache the terminal result, including a null "no default template",
// so reopening the viewer for such a model reuses it instead of
// re-running the backend/Hugging Face lookup every time.
cacheTemplate(cacheKey, template);
setFetched({
key: cacheKey,
state: { template, loading: false, error: null },
});
})
.catch((err: unknown) => {
if (controller.signal.aborted) {
return;
}
setFetched({
key: cacheKey,
state: {
template: null,
loading: false,
error:
err instanceof Error ? err.message : "Failed to load template",
},
});
});
return () => controller.abort();
}, [cacheKey, modelId, ggufVariant, token]);
if (cacheKey == null) {
return { template: null, loading: false, error: null };
}
if (templateCache.has(cacheKey)) {
return {
template: templateCache.get(cacheKey) ?? null,
loading: false,
error: null,
};
}
if (fetched?.key === cacheKey) {
return fetched.state;
}
return { template: null, loading: true, error: null };
}
export function useModelMaxPositionEmbeddings(
modelId: string | null,
enabled: boolean,
): ModelMaxPositionState {
const token = useHfTokenStore((s) => s.token);
const cacheKey = enabled && modelId ? `${modelId}::${token}` : null;
const [fetched, setFetched] = useState<{
key: string;
state: ModelMaxPositionState;
} | null>(null);
useEffect(() => {
if (cacheKey == null || !modelId || maxPositionCache.has(cacheKey)) {
return;
}
const controller = new AbortController();
fetchModelMaxPositionEmbeddings(modelId, token, controller.signal)
.then((maxPositionEmbeddings) => {
if (controller.signal.aborted) {
return;
}
cacheMaxPosition(cacheKey, maxPositionEmbeddings);
setFetched({
key: cacheKey,
state: { maxPositionEmbeddings, loading: false, error: null },
});
})
.catch((err: unknown) => {
if (controller.signal.aborted) {
return;
}
setFetched({
key: cacheKey,
state: {
maxPositionEmbeddings: null,
loading: false,
error:
err instanceof Error
? err.message
: "Failed to load model metadata",
},
});
});
return () => controller.abort();
}, [cacheKey, modelId, token]);
if (cacheKey == null) {
return { maxPositionEmbeddings: null, loading: false, error: null };
}
if (maxPositionCache.has(cacheKey)) {
return {
maxPositionEmbeddings: maxPositionCache.get(cacheKey) ?? null,
loading: false,
error: null,
};
}
if (fetched?.key === cacheKey) {
return fetched.state;
}
return { maxPositionEmbeddings: null, loading: true, error: null };
}

30
studio/frontend/src/features/model-picker/index.ts Normal file
View file

@ -0,0 +1,30 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
export { ModelSelector } from "./components/model-selector";
export { FolderBrowser } from "./components/model-selector/folder-browser";
export { ModelDeleteAction } from "./components/model-selector/model-delete-action";
export { hfModelFitsDevice } from "./components/model-selector/recommended-fit";
export {
NumericValueInput,
snapToStep,
} from "./components/numeric-value-input";
export { SidebarModelConfig } from "./components/sidebar-model-config";
export type {
DeletedModelRef,
ExternalModelOption,
LoraModelOption,
ModelOption,
ModelSelectorChangeMeta,
} from "./components/model-selector";
export {
applyModelLoadConfigToRuntime,
applyPerModelConfigToRuntime,
currentRuntimePerModelConfig,
perModelConfigsEqual,
} from "./model-config/apply-per-model-config";
export {
normalizeMaxSeqLength,
type PerModelConfig,
resolveInitialConfig,
} from "./model-config/per-model-config";

118
studio/frontend/src/features/model-picker/inventory/use-chat-picker-inventory.ts Normal file
View file

@ -0,0 +1,118 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
import type {
CachedGgufRepo,
CachedModelRepo,
LocalModelInfo,
} from "@/features/chat";
import {
type CachedInventoryRow,
type LocalInventoryRow,
type LocalSource,
isHiddenModelId,
useHubInventory,
} from "@/features/hub";
import { useMemo } from "react";
const PICKER_LOCAL_SOURCES: ReadonlySet<LocalSource> = new Set([
"lmstudio",
"models_dir",
"custom",
]);
function isCompleteCachedRow(row: CachedInventoryRow): boolean {
return !row.partial && !row.liveDownload;
}
function toCachedGgufRepo(row: CachedInventoryRow): CachedGgufRepo {
return {
repo_id: row.repoId,
size_bytes: row.bytes,
cache_path: row.cachePath ?? "",
last_modified: row.lastModified ?? undefined,
has_vision: row.capabilities.supportsVision,
};
}
function toCachedModelRepo(row: CachedInventoryRow): CachedModelRepo {
return {
repo_id: row.repoId,
size_bytes: row.bytes,
last_modified: row.lastModified ?? undefined,
};
}
function toLocalModelInfo(row: LocalInventoryRow): LocalModelInfo {
return {
id: row.loadId,
display_name: row.displayName ?? row.title,
path: row.path,
source: row.source as LocalModelInfo["source"],
model_id: row.modelId ?? row.repoId,
model_format: row.modelFormat,
updated_at: row.updatedAt,
};
}
export interface ChatPickerInventory {
cachedGguf: CachedGgufRepo[];
cachedModels: CachedModelRepo[];
cachedReady: boolean;
localModels: LocalModelInfo[];
refreshInventory: () => Promise<void>;
}
export function useChatPickerInventory(
options: { enabled?: boolean } = {},
): ChatPickerInventory {
const inventory = useHubInventory({
kind: "models",
enabled: options.enabled,
includeLocal: true,
});
const cachedGguf = useMemo(
() =>
inventory.cachedRows
.filter(
(row) =>
row.modelFormat === "gguf" &&
isCompleteCachedRow(row) &&
!isHiddenModelId(row.repoId),
)
.map(toCachedGgufRepo),
[inventory.cachedRows],
);
const cachedModels = useMemo(
() =>
inventory.cachedRows
.filter(
(row) =>
row.modelFormat !== "gguf" &&
isCompleteCachedRow(row) &&
!isHiddenModelId(row.repoId),
)
.map(toCachedModelRepo),
[inventory.cachedRows],
);
const localModels = useMemo(
() =>
inventory.localRows
.filter(
(row) =>
PICKER_LOCAL_SOURCES.has(row.source) &&
!isHiddenModelId(row.modelId, row.repoId, row.path),
)
.map(toLocalModelInfo),
[inventory.localRows],
);
return {
cachedGguf,
cachedModels,
cachedReady: inventory.downloadedReady,
localModels,
refreshInventory: inventory.refreshInventory,
};
}

85
studio/frontend/src/features/model-picker/model-config/apply-per-model-config.ts Normal file
View file

@ -0,0 +1,85 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
import {
defaultInferenceParams,
normalizeSpeculativeType,
readPersistedSpeculativeType,
useChatRuntimeStore,
} from "@/features/chat";
import {
DEFAULT_PER_MODEL_CONFIG,
type PerModelConfig,
normalizeMaxSeqLength,
} from "./per-model-config";
function cleanTemplate(value: string | null | undefined): string | null {
return value?.trim() ? value : null;
}
export function applyPerModelConfigToRuntime(config: PerModelConfig): void {
// Fall back to the standing default when the model has no saved
// maxSeqLength. maxSeqLength is the only per-model field carried on
// params (the rest are reset below), so without this a model with no
// remembered config would inherit the previously loaded model's value.
const maxSeqLength =
normalizeMaxSeqLength(config.maxSeqLength) ??
defaultInferenceParams.maxSeqLength;
const store = useChatRuntimeStore.getState();
if (maxSeqLength !== store.params.maxSeqLength) {
store.setParams({ ...store.params, maxSeqLength });
}
useChatRuntimeStore.setState({
customContextLength: config.customContextLength ?? null,
kvCacheDtype: config.kvCacheDtype ?? null,
speculativeType:
normalizeSpeculativeType(config.speculativeType) ??
readPersistedSpeculativeType(),
specDraftNMax: config.specDraftNMax ?? null,
tensorParallel: config.tensorParallel ?? false,
chatTemplateOverride: cleanTemplate(config.chatTemplateOverride),
});
}
export function applyModelLoadConfigToRuntime(
config: PerModelConfig | null | undefined,
): boolean {
const hasConfig = config != null;
applyPerModelConfigToRuntime(config ?? DEFAULT_PER_MODEL_CONFIG);
return hasConfig;
}
export function currentRuntimePerModelConfig(
options: { includeMaxSeqLength?: boolean } = {},
): PerModelConfig {
const s = useChatRuntimeStore.getState();
return {
customContextLength: s.customContextLength ?? null,
maxSeqLength: options.includeMaxSeqLength
? normalizeMaxSeqLength(s.params.maxSeqLength)
: null,
kvCacheDtype: s.kvCacheDtype ?? null,
speculativeType: normalizeSpeculativeType(s.speculativeType),
specDraftNMax: s.specDraftNMax ?? null,
tensorParallel: s.tensorParallel ?? false,
chatTemplateOverride: cleanTemplate(s.chatTemplateOverride),
};
}
export function perModelConfigsEqual(
a: PerModelConfig,
b: PerModelConfig,
): boolean {
return (
(a.customContextLength ?? null) === (b.customContextLength ?? null) &&
normalizeMaxSeqLength(a.maxSeqLength) ===
normalizeMaxSeqLength(b.maxSeqLength) &&
(a.kvCacheDtype ?? null) === (b.kvCacheDtype ?? null) &&
normalizeSpeculativeType(a.speculativeType) ===
normalizeSpeculativeType(b.speculativeType) &&
(a.specDraftNMax ?? null) === (b.specDraftNMax ?? null) &&
Boolean(a.tensorParallel) === Boolean(b.tensorParallel) &&
cleanTemplate(a.chatTemplateOverride) ===
cleanTemplate(b.chatTemplateOverride)
);
}

69
studio/frontend/src/features/model-picker/model-config/model-identity.ts Normal file
View file

@ -0,0 +1,69 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
import {
normalizeGgufVariantIdentity,
normalizeModelIdentity,
} from "@/features/hub";
export {
normalizeGgufVariantIdentity,
normalizeModelIdentity,
} from "@/features/hub";
const MODEL_STORAGE_KEY_PREFIX = "v2:";
type ParsedModelStorageKey = {
modelId: string;
ggufVariant: string;
};
function parseVersionedModelStorageKey(
key: string,
): ParsedModelStorageKey | null {
if (!key.startsWith(MODEL_STORAGE_KEY_PREFIX)) {
return null;
}
try {
const parsed = JSON.parse(key.slice(MODEL_STORAGE_KEY_PREFIX.length));
if (
!Array.isArray(parsed) ||
parsed.length !== 2 ||
typeof parsed[0] !== "string" ||
typeof parsed[1] !== "string"
) {
return null;
}
return { modelId: parsed[0], ggufVariant: parsed[1] };
} catch {
return null;
}
}
export function modelStorageKey(
modelId: string,
ggufVariant?: string | null,
): string {
return `${MODEL_STORAGE_KEY_PREFIX}${JSON.stringify([
normalizeModelIdentity(modelId),
normalizeGgufVariantIdentity(ggufVariant),
])}`;
}
export function modelIdFromStorageKey(key: string): string | null {
const parsed = parseVersionedModelStorageKey(key);
if (parsed) {
return parsed.modelId;
}
const separator = key.lastIndexOf("::");
return separator >= 0 ? key.slice(0, separator) : null;
}
export function ggufVariantFromStorageKey(key: string): string | null {
const parsed = parseVersionedModelStorageKey(key);
if (parsed) {
return parsed.ggufVariant;
}
const separator = key.lastIndexOf("::");
return separator >= 0 ? key.slice(separator + 2) : null;
}

571
studio/frontend/src/features/model-picker/model-config/per-model-config.ts Normal file
View file

@ -0,0 +1,571 @@
// SPDX-License-Identifier: AGPL-3.0-only
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
import {
ggufVariantFromStorageKey,
modelIdFromStorageKey,
modelStorageKey,
normalizeGgufVariantIdentity,
normalizeModelIdentity,
} from "./model-identity";
export interface PerModelConfig {
customContextLength: number | null;
maxSeqLength: number | null;
kvCacheDtype: string | null;
speculativeType: string | null;
specDraftNMax: number | null;
tensorParallel: boolean;
chatTemplateOverride: string | null;
}
export const DEFAULT_PER_MODEL_CONFIG: PerModelConfig = {
customContextLength: null,
maxSeqLength: null,
kvCacheDtype: null,
speculativeType: null,
specDraftNMax: null,
tensorParallel: false,
chatTemplateOverride: null,
};
export const MAX_SEQ_LENGTH_MIN = 128;
export const MAX_SEQ_LENGTH_MAX = 1048576;
export const MAX_SEQ_LENGTH_STEP = 128;
export const KV_CACHE_DTYPES = ["bf16", "q8_0", "q5_1", "q4_1"] as const;
const VALID_KV_CACHE_DTYPES = new Set<string>(KV_CACHE_DTYPES);
export const SPECULATIVE_TYPES = [
"auto",
"mtp",
"ngram",
"mtp+ngram",
"off",
] as const;
export const MTP_SPECULATIVE_TYPES: ReadonlySet<string> = new Set([
"mtp",
"mtp+ngram",
]);
const STORAGE_KEY = "unsloth_model_configs";
const LEGACY_STORAGE_KEY = "unsloth_load_settings";
const LEGACY_MIGRATION_FLAG = "unsloth_model_configs_migrated";
const STORAGE_SCHEMA_VERSION = 1;
const MAX_ENTRIES = 500;
const MAX_PER_MODEL_CONFIG_STORAGE_BYTES = 1024 * 1024;
export const MAX_CHAT_TEMPLATE_BYTES = 65_536;
type StoredPerModelConfig = PerModelConfig & {
version: typeof STORAGE_SCHEMA_VERSION;
};
type StoredMap = Record<string, PerModelConfig | StoredPerModelConfig>;
type RawConfig = Partial<PerModelConfig> & { version?: unknown };
const STORED_CONFIG_FIELDS = new Set([
"version",
"customContextLength",
"maxSeqLength",
"kvCacheDtype",
"speculativeType",
"specDraftNMax",
"tensorParallel",
"chatTemplateOverride",
]);
function canonicalizeSpeculativeType(value: string): string | null {
const s = value.trim().toLowerCase();
if (!s) {
return null;
}
if (s === "auto" || s === "default") {
return "auto";
}
if (s === "off") {
return "off";
}
if (s === "mtp" || s === "draft-mtp") {
return "mtp";
}
if (s === "ngram" || s === "ngram-mod" || s === "ngram-simple") {
return "ngram";
}
if (s === "mtp+ngram") {
return "mtp+ngram";
}
return null;
}
export function normalizeMaxSeqLength(value: unknown): number | null {
if (typeof value !== "number" || !Number.isFinite(value) || value <= 0) {
return null;
}
const snapped = Math.round(value / MAX_SEQ_LENGTH_STEP) * MAX_SEQ_LENGTH_STEP;
return Math.max(MAX_SEQ_LENGTH_MIN, Math.min(MAX_SEQ_LENGTH_MAX, snapped));
}
function canUseStorage(): boolean {
return typeof window !== "undefined";
}
function serializedByteLength(value: string): number {
return typeof TextEncoder !== "undefined"
? new TextEncoder().encode(value).byteLength
: value.length;
}
export function chatTemplateByteLength(value: string): number {
return serializedByteLength(value);
}
export function isChatTemplateWithinLimit(value: string): boolean {
return chatTemplateByteLength(value) <= MAX_CHAT_TEMPLATE_BYTES;
}
function serializedMapSize(map: StoredMap): number {
return serializedByteLength(JSON.stringify(map));
}
function serializedMapEntrySize(key: string, value: StoredMap[string]): number {
return (
serializedByteLength(JSON.stringify(key)) +
1 +
serializedByteLength(JSON.stringify(value))
);
}
function deleteOldestEvictableEntry(
map: StoredMap,
protectedKeys?: ReadonlySet<string>,
): { key: string; value: StoredMap[string] } | null {
for (const key of Object.keys(map)) {
// Never evict a future-schema entry an older client cannot interpret,
// matching the save/delete guards.
if (
protectedKeys?.has(key) ||
storedConfigVersion(map[key]) > STORAGE_SCHEMA_VERSION
) {
continue;
}
const value = map[key];
delete map[key];
return { key, value };
}
return null;
}
function enforceStorageBudget(
map: StoredMap,
protectedKeys?: ReadonlySet<string>,
): boolean {
let entryCount = Object.keys(map).length;
while (entryCount > MAX_ENTRIES) {
if (!deleteOldestEvictableEntry(map, protectedKeys)) {
return false;
}
entryCount -= 1;
}
let bytes = serializedMapSize(map);
while (bytes > MAX_PER_MODEL_CONFIG_STORAGE_BYTES) {
const removed = deleteOldestEvictableEntry(map, protectedKeys);
if (!removed) {
return false;
}
bytes -=
serializedMapEntrySize(removed.key, removed.value) +
(entryCount > 1 ? 1 : 0);
entryCount -= 1;
}
return true;
}
function storedConfigVersion(raw: unknown): number {
if (!raw || typeof raw !== "object" || Array.isArray(raw)) {
return 0;
}
const version = (raw as RawConfig).version;
return typeof version === "number" && Number.isFinite(version) ? version : 0;
}
let legacyMigrationChecked = false;
function parseLegacyModelKey(
key: string,
): { modelId: string; ggufVariant: string | null } | null {
const separator = key.lastIndexOf("::");
if (separator >= 0) {
const modelId = key.slice(0, separator);
return modelId
? { modelId, ggufVariant: key.slice(separator + 2) || null }
: null;
}
return key ? { modelId: key, ggufVariant: null } : null;
}
function legacyEntryToConfig(raw: Record<string, unknown>): PerModelConfig {
return normalizeV1({
customContextLength:
typeof raw.contextLength === "number" ? raw.contextLength : null,
maxSeqLength: null,
kvCacheDtype:
typeof raw.kvCacheDtype === "string" ? raw.kvCacheDtype : null,
speculativeType:
typeof raw.speculativeType === "string" ? raw.speculativeType : null,
specDraftNMax:
typeof raw.specDraftNMax === "number" ? raw.specDraftNMax : null,
tensorParallel:
typeof raw.tensorParallel === "boolean" ? raw.tensorParallel : false,
chatTemplateOverride: null,
});
}
function mergeLegacyEntries(
map: StoredMap,
legacy: Record<string, unknown>,
): string[] {
const addedKeys: string[] = [];
for (const [legacyKey, value] of Object.entries(legacy)) {
if (!value || typeof value !== "object") {
continue;
}
const parsedKey = parseLegacyModelKey(legacyKey);
if (!parsedKey) {
continue;
}
const migrated = legacyEntryToConfig(value as Record<string, unknown>);
const key = modelStorageKey(parsedKey.modelId, parsedKey.ggufVariant);
if (isDefaultConfig(migrated) || Object.hasOwn(map, key)) {
continue;
}
map[key] = toStoredConfig(migrated);
addedKeys.push(key);
}
return addedKeys;
}
function migrateLegacyLoadSettingsOnce(): void {
if (legacyMigrationChecked || !canUseStorage()) {
return;
}
legacyMigrationChecked = true;
try {
if (localStorage.getItem(LEGACY_MIGRATION_FLAG)) {
return;
}
let legacy: unknown = null;
try {
legacy = JSON.parse(localStorage.getItem(LEGACY_STORAGE_KEY) ?? "null");
} catch {
legacy = null;
}
if (!legacy || typeof legacy !== "object" || Array.isArray(legacy)) {
localStorage.setItem(LEGACY_MIGRATION_FLAG, "1");
return;
}
const map = readMapRaw();
const migratedKeys = mergeLegacyEntries(
map,
legacy as Record<string, unknown>,
);
if (migratedKeys.length === 0) {
localStorage.setItem(LEGACY_MIGRATION_FLAG, "1");
return;
}
// Protect the just-migrated entries during eviction. If the budget cannot
// fit them (e.g. storage is full of future-schema records an older client
// cannot evict), leave the flag unset so migration retries once space frees
// up rather than marking it complete and dropping the migrated config.
if (!enforceStorageBudget(map, new Set(migratedKeys))) {
return;
}
if (writeMap(map)) {
localStorage.setItem(LEGACY_MIGRATION_FLAG, "1");
}
} catch (err) {
console.warn("Failed to migrate legacy load settings:", err);
}
}
function readMapRaw(): StoredMap {
if (!canUseStorage()) {
return {};
}
try {
const raw = localStorage.getItem(STORAGE_KEY);
if (!raw) {
return {};
}
const parsed = JSON.parse(raw);
if (!parsed || typeof parsed !== "object" || Array.isArray(parsed)) {
return {};
}
return parsed as StoredMap;
} catch {
return {};
}
}
function readMap(): StoredMap {
migrateLegacyLoadSettingsOnce();
return readMapRaw();
}
function writeMap(map: StoredMap): boolean {
if (!canUseStorage()) {
return false;
}
try {
localStorage.setItem(STORAGE_KEY, JSON.stringify(map));
return true;
} catch (err) {
console.warn("Failed to persist per-model config:", err);
return false;
}
}
function warnDroppedFields(raw: Record<string, unknown>, version: number): void {
if (!import.meta.env?.DEV) {
return;
}
const dropped = Object.keys(raw).filter(
(key) => !STORED_CONFIG_FIELDS.has(key),
);
if (dropped.length > 0) {
console.warn("Dropped unknown per-model config fields:", dropped);
}
if (version > STORAGE_SCHEMA_VERSION) {
console.warn("Per-model config schema is newer than this app:", version);
}
}
function normalizeV1(partial: RawConfig): PerModelConfig {
const rawSpecType =
typeof partial.speculativeType === "string"
? canonicalizeSpeculativeType(partial.speculativeType)
: null;
const speculativeType = rawSpecType ?? DEFAULT_PER_MODEL_CONFIG.speculativeType;
const specDraftNMax =
speculativeType != null &&
MTP_SPECULATIVE_TYPES.has(speculativeType) &&
typeof partial.specDraftNMax === "number" &&
Number.isFinite(partial.specDraftNMax)
? Math.max(1, Math.min(16, Math.round(partial.specDraftNMax)))
: null;
return {
customContextLength:
typeof partial.customContextLength === "number" &&
Number.isFinite(partial.customContextLength) &&
partial.customContextLength > 0
? Math.floor(partial.customContextLength)
: null,
maxSeqLength: normalizeMaxSeqLength(partial.maxSeqLength),
kvCacheDtype:
typeof partial.kvCacheDtype === "string" &&
VALID_KV_CACHE_DTYPES.has(partial.kvCacheDtype)
? partial.kvCacheDtype
: null,
speculativeType,
specDraftNMax,
tensorParallel:
typeof partial.tensorParallel === "boolean"
? partial.tensorParallel
: DEFAULT_PER_MODEL_CONFIG.tensorParallel,
chatTemplateOverride:
typeof partial.chatTemplateOverride === "string" &&
isChatTemplateWithinLimit(partial.chatTemplateOverride)
? partial.chatTemplateOverride
: null,
};
}
function normalize(raw: unknown): PerModelConfig {
if (!raw || typeof raw !== "object" || Array.isArray(raw)) {
return normalizeV1({});
}
const partial = raw as RawConfig;
const version =
typeof partial.version === "number" && Number.isFinite(partial.version)
? partial.version
: 0;
warnDroppedFields(raw as Record<string, unknown>, version);
return normalizeV1(partial);
}
function toStoredConfig(config: PerModelConfig): StoredPerModelConfig {
return {
version: STORAGE_SCHEMA_VERSION,
...normalize(config),
};
}
function legacyModelStorageKey(
modelId: string,
ggufVariant?: string | null,
): string {
return `${modelId}::${ggufVariant ?? ""}`;
}
function storageKeysForModelVariant(
modelId: string,
ggufVariant?: string | null,
): string[] {
const key = modelStorageKey(modelId, ggufVariant);
const legacyKey = legacyModelStorageKey(modelId, ggufVariant);
return key === legacyKey ? [key] : [key, legacyKey];
}
function configKeyMatchesModelVariant(
key: string,
modelId: string,
ggufVariant?: string | null,
): boolean {
const storedModelId = modelIdFromStorageKey(key);
if (!storedModelId) {
return false;
}
return (
normalizeModelIdentity(storedModelId) === normalizeModelIdentity(modelId) &&
normalizeGgufVariantIdentity(ggufVariantFromStorageKey(key)) ===
normalizeGgufVariantIdentity(ggufVariant)
);
}
function findConfigKeyForModelVariant(
map: StoredMap,
modelId: string,
ggufVariant?: string | null,
): string | null {
for (const key of storageKeysForModelVariant(modelId, ggufVariant)) {
if (Object.hasOwn(map, key)) {
return key;
}
}
for (const key of Object.keys(map)) {
if (configKeyMatchesModelVariant(key, modelId, ggufVariant)) {
return key;
}
}
return null;
}
function hasFutureConfigForModelVariant(
map: StoredMap,
modelId: string,
ggufVariant?: string | null,
): boolean {
for (const key of Object.keys(map)) {
if (
configKeyMatchesModelVariant(key, modelId, ggufVariant) &&
storedConfigVersion(map[key]) > STORAGE_SCHEMA_VERSION
) {
return true;
}
}
return false;
}
function deleteConfigEntriesForModelVariant(
map: StoredMap,
modelId: string,
ggufVariant?: string | null,
): boolean {
let changed = false;
for (const key of Object.keys(map)) {
if (!configKeyMatchesModelVariant(key, modelId, ggufVariant)) {
continue;
}
delete map[key];
changed = true;
}
return changed;
}
function loadPerModelConfig(
modelId: string,
ggufVariant?: string | null,
): PerModelConfig | null {
const map = readMap();
const key = findConfigKeyForModelVariant(map, modelId, ggufVariant);
if (!key) {
return null;
}
// Never apply a future-schema record an older client cannot interpret,
// matching the save/delete/evict guards.
if (storedConfigVersion(map[key]) > STORAGE_SCHEMA_VERSION) {
return null;
}
return normalize(map[key]);
}
export function isDefaultConfig(config: PerModelConfig): boolean {
return (
config.customContextLength == null &&
config.maxSeqLength == null &&
(config.kvCacheDtype ?? null) === DEFAULT_PER_MODEL_CONFIG.kvCacheDtype &&
config.speculativeType === DEFAULT_PER_MODEL_CONFIG.speculativeType &&
config.specDraftNMax == null &&
Boolean(config.tensorParallel) ===
Boolean(DEFAULT_PER_MODEL_CONFIG.tensorParallel) &&
(config.chatTemplateOverride ?? null) === null
);
}
export function savePerModelConfig(
modelId: string,
ggufVariant: string | null | undefined,
config: PerModelConfig,
): boolean {
if (
typeof config.chatTemplateOverride === "string" &&
!isChatTemplateWithinLimit(config.chatTemplateOverride)
) {
return false;
}
const normalized = normalize(config);
const map = readMap();
if (hasFutureConfigForModelVariant(map, modelId, ggufVariant)) {
return false;
}
if (isDefaultConfig(normalized)) {
const changed = deleteConfigEntriesForModelVariant(
map,
modelId,
ggufVariant,
);
return changed ? writeMap(map) : true;
}
const [key] = storageKeysForModelVariant(modelId, ggufVariant);
deleteConfigEntriesForModelVariant(map, modelId, ggufVariant);
map[key] = toStoredConfig(normalized);
if (!enforceStorageBudget(map, new Set([key]))) {
return false;
}
return writeMap(map);
}
export function deletePerModelConfig(
modelId: string,
ggufVariant?: string | null,
): boolean {
const map = readMap();
// Mirror savePerModelConfig: never let an older client destroy a
// future-schema entry it cannot interpret.
if (hasFutureConfigForModelVariant(map, modelId, ggufVariant)) {
return false;
}
if (!deleteConfigEntriesForModelVariant(map, modelId, ggufVariant)) {
return true;
}
return writeMap(map);
}
export function resolveInitialConfig(
modelId: string,
ggufVariant?: string | null,
): { config: PerModelConfig; remembered: boolean } {
const saved = loadPerModelConfig(modelId, ggufVariant);
if (saved) {
return { config: saved, remembered: true };
}
return { config: { ...DEFAULT_PER_MODEL_CONFIG }, remembered: false };
}

39
studio/frontend/src/features/settings/tabs/chat-tab.tsx
View file

@ -43,7 +43,6 @@ import {
Folder01Icon,
McpServerIcon,
PencilRulerIcon,
Settings02Icon,
ShieldBanIcon,
Upload01Icon,
} from "@hugeicons/core-free-icons";
@ -196,10 +195,6 @@ export function ChatTab() {
const hydratePersistedSettings = useChatRuntimeStore(
(state) => state.hydratePersistedSettings,
);
const loadOnSelection = useChatRuntimeStore((state) => state.loadOnSelection);
const setLoadOnSelection = useChatRuntimeStore(
(state) => state.setLoadOnSelection,
);
const expandQuantizations = useChatRuntimeStore(
(state) => state.expandQuantizations,
);
@ -333,40 +328,6 @@ export function ChatTab() {
</header>
<SettingsSection title="Select model settings">
<SettingsRow
label="Load on selection"
alignTop={true}
description={
<span>
On: Unsloth auto-picks the best settings and loads it.
<br />
Off: opens Run settings to customize, then load.
<br />
The gear always opens Run settings:{" "}
<span className="ml-2 inline-flex items-center gap-3 align-middle">
<span className="font-mono text-xs text-foreground">
Q4_K_M
</span>
<span className="text-[9px] font-medium text-green-400">
downloaded
</span>
<span className="text-[10px] text-muted-foreground">16 GB</span>
<span className="inline-flex size-4 items-center justify-center rounded bg-black/[0.06] dark:bg-white/[0.08]">
<HugeiconsIcon
icon={Settings02Icon}
strokeWidth={1.75}
className="size-2.5 text-muted-foreground/80"
/>
</span>
</span>
</span>
}
>
<Switch
checked={loadOnSelection}
onCheckedChange={setLoadOnSelection}
/>
</SettingsRow>
<SettingsRow
label="Expand quantizations"
description={

3
studio/frontend/src/features/settings/tabs/general-tab.tsx
View file

@ -92,9 +92,10 @@ const PREFS_KEYS: string[] = [
"unsloth_chat_inference_params",
"unsloth_chat_collapsible_state",
"unsloth_chat_preferences",
"unsloth_model_configs",
"unsloth_model_configs_migrated",
"unsloth_load_settings",
// Model selector settings ("Select model settings" group)
"unsloth_chat_load_on_selection",
"unsloth_chat_expand_quantizations",
"unsloth_chat_show_all_quantizations",
"unsloth_models_fit_on_device_only",

4
studio/frontend/src/features/training/index.ts
View file

@ -24,8 +24,8 @@ export { HfDatasetSubsetSplitSelectors } from "./components/hf-dataset-subset-sp
export { useDatasetPreviewDialogStore } from "./stores/dataset-preview-dialog-store";
export { listLocalDatasets, uploadTrainingDataset } from "./api/datasets-api";
export type { LocalDatasetInfo } from "./types/datasets";
export { listLocalModels } from "./api/models-api";
export type { LocalModelInfo } from "./api/models-api";
export { getModelConfig, listLocalModels } from "./api/models-api";
export type { LocalModelInfo, ModelConfigResponse } from "./api/models-api";
export type {
TrainingPhase,
TrainingViewData,