Feat/model picker per model config v2 (#7207)
* refactor(studio): move chat model picker into features/model-picker
Relocate model-selector + its support files from components/assistant-ui
into a self-contained features/model-picker feature (own barrel), mirroring
the modular Hub layout. Pure move + import repoint; no behaviour change.
* feat(model-picker): add per-model config persistence layer
Superset PerModelConfig (customContextLength, kvCacheDtype, speculativeType,
specDraftNMax, tensorParallel, chatTemplateOverride, trustRemoteCode) persisted
to localStorage (unsloth_model_configs) with schema versioning + LRU budget.
KV-dtype and speculative value sets match main's sidebar (no q4_0/ngram-simple).
Reuses features/hub/lib/model-identity for normalization; adds storage-key layer
and applyPerModelConfigToRuntime (sets tensorParallel, which the old PR omitted).
* feat(picker): modular backend for chat-template validate + default fetch
New studio/backend/picker package (schemas/service/routes) mounted at /api/picker:
- POST /api/picker/validate-chat-template (Jinja syntax validation, no false positives)
- GET /api/picker/chat-template/{model_name} (default template from tokenizer_config.json,
reusing get_cache_path/resolve_cached_repo_id_case; graceful null, no model-code exec)
Frontend api/templates.ts client + hooks/use-model-defaults lazy cache. No backend
changes to the existing inference load route (per-model load fields already supported).
* feat(model-picker): bind picker on-device list to shared hub inventory
Picker now sources cached + local models from useHubInventory (the Hub's shared
store) via a thin adapter, replacing its own /api/models/* fetchers + module
caches. Hub, download manager, and picker now share one source of truth, so
completed downloads reflect in the picker automatically. Partial/live-download
rows are filtered from the cached lists (unchanged rendering). Local naming/search
preserved via additive LocalInventoryRow modelId/displayName. Variant expander,
scan-folder management, recommended-fit, search, external providers untouched.
Known minor: cached 'Downloaded date' sort tiebreak degrades to alphabetical
(hub cached rows carry no mtime); default 'recent' (load-time) sort preserved.
* feat(model-picker): per-model config step inside the picker
Picking a (non-external) model now opens an in-picker config view built from
main's current load controls (context length, KV cache dtype, speculative
decoding, draft tokens, tensor parallel) plus a chat-template editor backed by
the picker validate/default endpoints. 'Remember for this model' persists the
config per model+variant; Run forwards the config to the existing load flow via
meta.config. External models bypass the step. Two-view orchestration lives in
model-selector (single interception point); pickers.tsx call sites untouched.
trustRemoteCode dropped from PerModelConfig to preserve main's per-load consent.
* feat(chat): apply/persist per-model config through the load flow
handleCheckpointChange threads meta.config into the selection; stageOrLoad and
the autoload/Hub-run paths now apply the picker config (explicit pick or saved
remembered config) via applyPerModelConfigToRuntime before staging/loading, with
keepSpeculative set so a remembered speculative mode survives the model switch.
Replaces the old remembered-load-settings seeding (resolveInitialConfig now the
single source). SelectedModelInput carries config.
* refactor(chat): remove per-model load config from the right sidebar
The load knobs (context, KV cache, speculative, draft tokens, tensor parallel)
and the chat-template editor now live only in the picker config step. The sheet's
Model section keeps the staged Load/Cancel flow (config is applied at pick time);
sampling params, system prompt, and RAG are unchanged. Deletes the superseded
remembered-load-settings module + the store's applyRememberedLoadSettings action,
removes the now-dead sheet state/imports, and points the settings reset at
unsloth_model_configs. Delete-cleanup deferred (stale config is LRU-capped).
* fix(model-picker): remove leftover sidebar-staging cogwheel + empty Model section
The downloaded-variant gear (ModelLoadSettingsAction) staged a model straight
into the right-sidebar Run-settings flow -- the old 'configure before load' path
now fully replaced by the in-picker config step. Removed the gear + its component.
Also gate the sheet's 'Model' section to staged picks only (pendingSelection):
after the load-knob strip its content is staged-only, so it was rendering an
empty section header whenever a model was merely loaded.
* chore(chat): remove dead per-model-config setters + modelControlsDisabled
After the load-config UI moved into the picker, the store's per-model setters
(setKvCacheDtype/setSpeculativeType/setSpecDraftNMax/setTensorParallel/
setCustomContextLength/setChatTemplateOverride) had zero callers
(applyPerModelConfigToRuntime writes via setState), and the sheet's
modelControlsDisabled was unreferenced. Verified dead across the whole tree.
* fix(chat): config-step Load actually loads (ignore Load-on-selection)
Root cause: with Settings > Chat > 'Load on selection' turned OFF, the config
step's load went down the deferred-staging path -- opening the right sidebar with
'<model> is staged, not loaded yet / Choose Load model'. The in-picker config step
IS the deliberate load action, so its Load now loads immediately (or downloads +
auto-loads when not cached) regardless of the toggle. Renamed the button
'Run model' -> 'Load model' to match. Native/dropped picks still honor the toggle.
* refactor(chat,hub): retire 'Load on selection' — config step is the only load flow
The in-picker config step (and the Hub Run button) now fully supersede the old
stage-to-sidebar flow, so the Load-on-selection toggle is removed everywhere:
- chat stageOrLoad: every pick loads immediately, or downloads + auto-loads when
not cached (the previous default behaviour, now universal).
- hub Run: drops the stage branch; downloaded GGUFs load directly with their saved
per-model config (no collision with the chat config step — both end at selectModel).
- store: removed loadOnSelection field/setter/key/default; Settings>Chat toggle and
its settings-reset entry removed.
- staged sidebar section is now a download-progress view (auto-loads on completion).
No manual staging remains; stageModel is used only for background auto-load downloads.
* feat(model-picker): default chat template from GGUF + thread variant through config flow
Read the embedded tokenizer.chat_template from GGUF files (read_gguf_chat_template
in gguf_metadata) and use it as the per-model default. Plumb gguf_variant through
the picker service, /api/picker/chat-template route, frontend templates API, and
use-model-defaults so the right variant's template is fetched.
Also refine the picker config-page/model-selector wiring, drop the dead
ggufNativeContextLength runtime path, and add the per-model-config storage keys to
the settings prefs export.
* feat(model-picker): read safetensors chat template + hide editor where it has no effect
Resolve the default chat template for safetensors models: prefer the modern
chat_template.jinja, fall back to the tokenizer_config.json chat_template field,
then chat_template.json (multimodal processor), then the GGUF embedded template.
Applied to local dirs, the HF cache snapshot scan, and the HF remote fetch.
Hide the chat-template editor in the picker for safetensors models — the override
is only applied at load by the GGUF/llama.cpp backend, so editing it on safetensors
currently has no effect. GGUF keeps the editor. Nothing removed; the dialog stays
for when the safetensors apply path is wired up in a later branch.
* fix(model-picker): set legacy-migration flag only after the write succeeds
Set unsloth_model_configs_migrated only once writeMap confirms the migrated
map persisted, so a quota/storage failure no longer marks migration done and
silently drops the user's pre-existing remembered settings — the next load retries.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* MVP model picker fixes
* MVP picker config fix
* MVP safetensors config
* MVP max seq config
* MVP max seq fix
* Fix static max tokens cap ignoring model context
* Fix picker GGUF scan parity
* fix(studio): harden model picker config loading
Apply remembered per-model configs consistently from picker and Hub loads, keep default configs from overriding standing speculative settings, add config access for direct local GGUF files, and support saving or forgetting active model settings without a reload.
* Fix model picker config flow
* Fix model picker config loads
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Avoid recursive per-model config migration reads
* Apply the displayed context length when loading a GGUF
* Fix template validation, cached template lookup, and failed load rollback
- Validate chat templates with the loopcontrols extension so templates
that use break or continue tags pass the picker validator, matching the
inference renderer that already accepts them.
- Read the default chat template from the newest cache snapshot rather than
an arbitrary iterdir order, so an older cached revision no longer prefills
a stale template.
- Capture the runtime per-model config before a load and reapply it when the
load fails, so a failed switch leaves the active model context, KV cache,
template, and speculative settings as they were.
* Make chat template view only for safetensors models
Custom chat template overrides are applied at inference only for GGUF
models, which pass the template to llama-server. The safetensors backend
renders with the model built-in template and ignores the override, so
editing it would save a value that never loads. For safetensors the
config page now opens the template as a read-only preview with a note
that editing is not available yet. This can become editable once
inference support for custom safetensors templates lands in main.
* Fix model picker config edge cases
- Restore prior runtime config when a load no-ops for the active model
- Cap the picker validator request body via the protected prefixes
- Keep the GGUF context slider max above the loaded context
- Fetch subfolder chat templates for uncached Hub repos
- Show the compare side config when reopening the picker
* Keep saved GGUF context above the fallback ceiling
* Show the model config in the run settings sidebar
* Fix model config sidebar reset and context slider
- Stack the remember toggle and action buttons in the sidebar
- Reset the config to defaults instead of the loaded values
- Fetch the native context so the slider max is not the loaded value
* Fix model picker config and download regressions
- Run picker chat template routes off the event loop
- Depth and root guard local template directory scans
- Restore download manager flow for uncached hub picks
- Apply per model context length on reload
- Import model picker symbols from the feature barrel
* Fix model picker config and cached download sorting
- Restore load settings when a Hub run is rejected mid load
- Reuse one NumericValueInput instead of a duplicate copy
- Fix double decode of the model name in the template route
- Remove the unused reset-to-loaded settings action
- Fix cached model download sorting
* Fix model picker per-model config edge cases
Honor a saved or typed max seq length above the model's native context so
RoPE extended values are no longer clamped and silently overwritten. Allow
typing past native while the slider keeps native as a soft ceiling.
Guard the fetch success paths in use-model-defaults against an aborted
signal, and refetch when the HF token changes.
Hash the chat template content in the sidebar remount key instead of its
length. Enable reset for a GGUF whose native context is unknown, and floor
the context slider max so it can never fall below the min.
* Fix GGUF context auto-fit and gated model config token
Stop forcing a 32768 context when a GGUF native context is unknown so the backend auto-fits to VRAM again, while still honoring an explicit context edit.
Send the HF token as a query param so gated safetensors models resolve their max position embeddings.
Derive model default state during render to drop the set-state-in-effect calls.
* Fix native GGUF context ceiling and guard picker template reads
Restore the native context store field so the sidebar slider keeps the
full ceiling for drag and drop GGUFs. Limit local chat template reads to
the browse allowlist, skip malformed repo ids, and drop unused model
picker exports.
* Fix model picker lint boundaries
* Fix model picker review findings
Chat template editor never seeded its draft. Radix only calls onOpenChange
from internal events, so the seed in the nextOpen branch was dead and a model
with a saved override opened empty. Saving then cleared the override. Drop the
dead branch, treat draft as an untouched sentinel, and reset it on every close.
Uncached Hub picks could auto load a model after the user left the chat. Main
detached the staged pick on route exit and on chat context change. Carry the
context key on the pending pick and skip the load when it no longer matches.
Also clear configTarget when the picker closes, restore the onUpdated ref so
variant rows stop resubscribing on every parent render, skip the LRU write when
the entry is already most recent, import NumericValueInput relatively, and drop
the unused ModelUpdateAction barrel export.
* Preserve GGUF context on active reload
* Fix model picker per-model config regressions
- Stop reloading the already loaded model on re-pick
- Hide infra models from the chat picker
- Detect vision support on cached GGUF repos
- Honor saved maxSeqLength on auto load
- Restore default chat template for local GGUFs
- Warn on save failure and revert config on cancel
- Refetch picker inventory on open
- Persist read only per model config safely
* Fix stale model auto load
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix model picker numeric input sizing and constraints
Size value inputs to their content so long context lengths are not clipped,
restrict them to numeric characters, and stop the speculative decoding label
from truncating in the sidebar.
* Fix picker CI tests and harden chat template resolution for PR #6647
- tests: point the descender guard at the moved model-selector.tsx path
- tests: exclude the disabled Reload model button from the regenerate locator so .first targets the real Regenerate
- picker/service.py: reject symlinked template/gguf leaves that resolve outside the browse allowlist (HF cache reads unchanged)
- compare mode: resolve each pane's own remembered chat template instead of inheriting the other pane's from the store
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Protect future-schema per-model configs from deletion for PR #6647
savePerModelConfig already refuses to overwrite a stored config whose schema version is newer than this client understands, but deletePerModelConfig did not. Unchecking Remember on an older client therefore silently destroyed a newer client's saved config. Apply the same guard on delete and surface the blocked case through the existing saveFailed toast.
* Protect future-schema per-model configs from quota eviction for PR #6647
The save and delete guards already refuse to touch a stored config whose schema version is newer than this client understands, but the quota-eviction path did not, so a full store on an older client could still evict a newer client's config. Skip future-schema entries when evicting and fail the save if the budget cannot be met without them.
* Fix GGUF context persistence, compare context, and rollback settings for PR #6647
Persist a GGUF context override from the user's intent instead of collapsing it against the loaded context, which reintroduced the context-reset (f4838782cb reverted the native-baseline fix). model-config-page now collapses the saved value against native, and use-chat-model-runtime and chat-adapter retain the requested context on load so re-saving another setting keeps the override; a null request stays null so a VRAM auto-fit never becomes a stored override.
shared-composer: a compare pane with no explicit GGUF context now loads at native (0) like single-view, not the session maxSeqLength that silently shrank the shown context.
use-chat-model-runtime: restore the previous model's KV cache dtype and chat template on a failed-load rollback so it runs as it was, not with backend defaults.
* Preserve native path token when reloading the active model for PR #6647
handleReloadActiveModel rebuilt the selection without the store's activeNativePathToken, so reloading a file-picked GGUF after a settings change validated the display label as a repo/path and failed. Thread the active native token through the reload selection so native-loaded models reopen correctly.
* Make picker template validation resilient and accept HF generation tags for PR #6647
Import Jinja lazily inside validate_chat_template so a backend without the optional jinja2 package (GGUF-only installs) still starts instead of raising ModuleNotFoundError at import time. Register a no-op extension for the Transformers {% generation %} assistant-mask tag so pasting a valid HF chat template validates, matching the renderer, rather than being rejected as an unknown tag.
* Honor remembered compare config and parse processor chat_template.json for PR #6647
* Fix failed-load rollback context and processor template map fallback for PR #6647
* Restore speculative decoding config on failed-switch rollback
When a model switch fails after the previous model was unloaded, the
rollback reload restored tensor_parallel, KV cache dtype and the chat
template override, but omitted speculative_type and spec_draft_n_max and
cleared their loaded shadows to null. The previous model therefore came
back running at backend defaults (speculation off) while the UI still
showed it enabled, and the status resync confirmed the off state. Resend
the previous model's speculative settings in the rollback load and keep
the store's active and loaded speculative fields in sync with them.
* Reset max sequence length when a model has no saved config
applyPerModelConfigToRuntime reset every per-model field except
maxSeqLength, which it only wrote when the incoming config had one.
maxSeqLength is the sole field carried on store.params, so selecting a
model with no remembered config left the previous model's value in place
and later loaded the new model at that leaked length. Fall back to the
standing default so an unremembered model loads at its own default.
* Surface a message when a variant update cannot start
startManagedUpdate handled the conflict and error start outcomes but let
busy fall through as if the update began, so the confirm dialog closed
with no job created and the cached variant stayed stale. Show an info
message when the repo is busy with a sibling transfer so the click is
not silently dropped.
* Keep per-model speculative choices out of the global default
A staged load with a per-model or one-off config sets keepSpeculative,
which already skips reading the global speculative preference. The
matching save still ran unconditionally, so the model-specific choice was
written to the global unsloth_chat_speculative_type and a later model with
no saved config started from it instead of Auto. Skip saveSpeculativeType
when keepSpeculative so the per-model choice stays isolated.
* Seed non-active model settings from the app default max length
The Run settings page captured initialMaxSeqLength from the loaded
model's runtime params and fell back to it for a model with no saved
config. Opening settings for a different, unloaded model and clicking
Load then sent the active model's context (for example 64k) instead of
the 4096 default, risking validation failures or OOMs. Seed the default
for non-active models and keep the runtime value only for the active one.
* Prefer sidecar tokenizer chat template over the GGUF copy for variants
_chat_template_from_dir returned the embedded GGUF template first when a
variant was selected, reversing the tokenizer-first precedence of the
no-variant path. A model whose chat_template.jinja or tokenizer_config.json
supersedes a stale embedded template then got the wrong template on
variant selection. Keep tokenizer files first regardless of variant; the
variant only picks which GGUF is the fallback. Adds regression tests for
both the tokenizer-wins and gguf-fallback cases.
* Keep per-model speculative choices load-local in autoload and compare
The interactive load path treats a per-model speculative choice as
load-local and skips writing it to the global default. Autoload and
generalized compare still called saveSpeculativeType unconditionally, so a
remembered off or ngram setting leaked into unsloth_chat_speculative_type
and later models with no saved config inherited it. Persist the global
preference only when the value came from the global settings.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: record the compare pane's loaded context in runtime state so the active model's settings and any reload or save use it, not the previous context
* Studio: notify the user when a Hub autoload can't start because another download for the model is already running, instead of silently dropping it
* Studio: drop the merge's orphaned staged-model store helpers and unused alert imports
The main merge left isPendingGguf and pendingSelectionMatches referencing the
removed PendingModelSelection type, and the alert-dialog/alert imports unused
after the permission-mode dropdown replaced the bypass dialog, so tsc -b failed.
* Studio: cache a null default chat template so the viewer stops re-fetching it
A model with no sidecar or embedded template resolves to a terminal null, but
that result was never cached, so reopening the template viewer re-ran the
backend and Hugging Face lookup every time.
* Studio: detect direct-file GGUFs in run settings so Max Tokens uses their context
A GGUF loaded from a local file or custom folder has no variant label, so the
run-settings panel treated it as non-GGUF and clamped Max Tokens to the session
max_seq_length instead of the loaded GGUF context. Detect it via the reported
GGUF context and the .gguf checkpoint suffix, matching the chat page.
* Studio: prompt to re-select a local model file when its lease expired before reload
A file-picked GGUF is reachable only through a native path token that the
desktop host prunes after a TTL. Reloading reused that token blindly, so a
reload long after the initial load failed with an opaque error. Track the
token's expiry and, when it has passed, ask the user to re-select the file
instead of attempting a doomed reload.
* Fix descender-clipping test to tolerate sidebar layout utilities
The sidebar account-block div carries layout utilities (min-w-0, flex-1)
between 'flex' and 'flex-col', so the descender-clipping guard's regex,
which required 'flex' immediately followed by 'flex-col', no longer matched
and the test failed to locate the account-block div. Generalize the prefix
to allow intervening flex utilities while still capturing the leading-*
class before the collapsible visibility utility and asserting leading-tight,
so the guard against clipped glyph descenders is fully preserved.
* Harden picker chat-template resolution
Enforce the 64 KiB chat-template contract at the validate endpoint's request
model so a direct caller cannot submit a template far larger than the frontend
allows (MaxBodyMiddleware only bounds the whole request body, not this field);
oversized templates now return a clean 422.
Apply sidecar-over-GGUF template precedence globally across cached snapshots
instead of per snapshot. A repo with multiple cached revisions previously
returned the first snapshot's template, so a newer GGUF-only revision could
win over an older revision's maintained chat_template.jinja sidecar, which
contradicted the documented intent that sidecars supersede the embedded copy.
* Guard per-model config against future-schema and lossy migration
Two forward-compatibility gaps in the versioned per-model config store:
- The load/apply path returned and normalized a stored record without checking
its schema version, so a record written by a newer client was reinterpreted
under the current schema and applied to a live model load, even though save,
delete and eviction all refuse to touch future-schema records. Reject
future-schema records on load too.
- The one-time legacy migration enforced the storage budget without protecting
the entries it had just migrated and set the completion flag unconditionally.
When storage was already full of future-schema records (which are unevictable
by an older client), the migrated entries were the only evictable ones and
could be dropped while migration was still marked complete. Protect the
migrated keys during eviction and only mark migration complete when they
survive, so it retries once space frees up.
* Discard chat-template validation results after the dialog closes
Server-side template validation is async, but closing or cancelling the editor
did not abort it, so a late-arriving valid response still called onSave and
applied a template the user had already dismissed. Track a validation token
that is bumped on close and ignore any validation result whose token is stale.
* Record native lease expiry when loading a picked GGUF from the chip
The pending-native-model chip loaded via stageOrLoad directly, bypassing
loadNativeModelIntent, so activeNativePathExpiresAtMs was never recorded for a
chip-loaded file. A later reload then either skipped the lease-expiry guard
entirely (expiry left null) or compared against a previously loaded file's
stale expiry, so reload could reuse an already-pruned token or wrongly block a
still-valid one. Route the chip through loadNativeModelIntent, which builds the
same selection and records the expiry.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Prefer sidecar template for a directly selected local GGUF file
A direct .gguf file path read its embedded chat template without checking the
parent directory for a maintained sidecar (chat_template.jinja /
tokenizer_config.json), while directory and variant selections already prefer
the sidecar. That let the config editor preview or save a stale embedded
template for the same model depending on how it was selected. Check the parent
directory sidecars first, then fall back to the embedded copy, and cover both
paths with tests.
* Resolve cached chat template per revision, newest first
The earlier change searched every cached snapshot for a sidecar before
considering any snapshot's embedded GGUF template, which let an obsolete sidecar
from an older revision override the newest revision's template. Restore
per-snapshot resolution (newest first): a revision's sidecar still supersedes
its own embedded GGUF copy, but a newer revision is no longer overridden by an
older revision's sidecar.
* Preserve autoload transport conflicts and surface background busy downloads
- When a Hub autoload hits a transport conflict, keep pendingHubAutoLoad bound
instead of clearing it. Clearing it re-keyed the download surface and its
cleanup cancelled the conflict the toast tells the user to resolve, so the
Hub resume affordance was gone the moment it appeared. Return early on
conflict, mirroring the started branch, so resolving it from the Hub still
auto-loads on completion.
- The background-download branch handled started and conflict but silently
dropped a busy outcome, leaving the user with no feedback when a peer variant
of the same repo was already downloading. Surface the same busy toast the
autoload path uses.
* Fix context length, GGUF template, fetch state and lease expiry bugs
Keep explicit context length values instead of collapsing to null at
native. The collapse made the slider jump back at the native maximum
and made Reload load the previous context instead of the chosen one.
Prefer the first split when resolving a GGUF without a variant. Later
splits carry no chat template metadata, so picking the largest file
could return no template for a sharded model.
Clear stale fetch state when template and metadata lookups retry, so
a previous terminal error is not shown while a new fetch is running.
Record native path lease expiry together with the token when a load
commits. The expiry was written by only one load path and even when
the load did not start, so a reload could be blocked with an expired
file message for a still valid token.
* fix(model-picker): resolve review findings across config, inventory, and templates
- Apply remembered per-model config in the training-compare chat handoff so a
prior model's customContextLength no longer leaks into the next load
- Match GGUF variant labels with the inventory extractor too, so cached
no-quant-token files resolve their default chat template
- Show "Auto" instead of a fabricated 32768 when native context is unknown
- Reuse the identical staged auto-load object on same-pick so a re-pick during
download pre-flight no longer disarms auto-load via "busy"
- Union supports_vision when deduping cross-cache inventory rows
- Serve hidden-model needles from a new GET /api/hub/hidden-models endpoint and
merge them client-side, covering runtime-configured RAG embedders
- Clamp GET chat templates to MAX_CHAT_TEMPLATE_BYTES (route + jinja sidecar),
matching the validate endpoint's contract
- Lower-clamp stored customContextLength to shared CONTEXT_LENGTH_MIN
- Wipe unsloth_chat_load_on_selection in Settings "Reset all"
- Drop stale pendingHasContext comment describing deleted staging machinery
* Fix stale defaults cache, token in query string and rounded up context ceiling
Refresh cached chat template and max position data when a model update
completes. Send the HF token for model config requests in the dedicated
header instead of the URL. Snap the native sequence length ceiling down
to the nearest step so the slider cannot exceed the declared maximum.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix compare pane reverting active checkpoint on non-GGUF load
Re-read runtime params after setCheckpoint so the fresh checkpoint is
kept instead of being overwritten by the pre-setCheckpoint snapshot.
* Send the HF token via header for the vision and embedding checks
checkVisionModel and checkEmbeddingModel still passed the HuggingFace
token as a ?hf_token= query parameter, so it landed in server access
logs, proxy logs, and browser history. Move them to the
X-Unsloth-HF-Token header like getModelConfig already does, and accept
the header on the check-vision and check-embedding routes with the
existing query parameter kept as a fallback for older clients.
* Cap the chat template on the model load path
The load endpoint accepted an unbounded chat_template_override, so a
direct caller could hand llama.cpp an arbitrarily large Jinja template
even though the frontend, the validate endpoint, and the read paths all
enforce the 64 KiB limit. Reuse MAX_CHAT_TEMPLATE_BYTES in the
LoadRequest validator, rejecting oversized templates with a fast
character-count check before the exact UTF-8 byte check.
* Protect existing per-model configs during legacy migration
When the one-time legacy import pushes the store over budget, eviction
now protects the entries the user already has and drops only the
just-migrated legacy entries, so importing old load settings can never
discard a newer per-model config.
* Reset clears the context override instead of pinning the native value
Reset wrote the discovered native context into customContextLength for
GGUF models, but isDefaultConfig treats any non-null customContextLength
as an explicit pin, so Reset with Remember enabled persisted a fixed
context and future loads stopped using the native auto context. Reset
now restores the full default (customContextLength null); the native
value is still shown through the existing display fallback.
* Bound chat-template sidecar reads to a size limit
The chat_template.json, tokenizer_config.json, and Hub-downloaded sidecar
readers decoded and json-parsed the whole file before the extracted
template hit the 64 KiB response cap, so an oversized metadata file could
exhaust memory. Read them through a bounded reader (4 MiB envelope) that
returns None when the file is larger, matching the existing chat_template.jinja
size guard. Adds tests for oversized tokenizer_config.json and chat_template.json.
* Keep the native-path token and lease expiry in sync
Rollback after a failed reload restored the previous token but left the
failed load's expiry in the store, so a later reload could be falsely
blocked as expired (token A paired with load B's lease). Restore the
previous lease alongside the token, and clear the expiry wherever the
token is cleared on a non-GGUF transition, so the two never diverge.
* Clear the native file lease on compare-pane loads
* Studio: add regression tests for the model-picker per-model-config
Guard the specific regressions that reverted the predecessor change:
- backend pytest (studio/backend/tests/test_model_picker_regression.py):
infra-model hiding, HF token via header with query fallback, and the
chat-template byte caps.
- source contracts (tests/studio/test_model_picker_contracts.py): the token
stays out of the URL, the context ceiling is floored, the native lease is
cleared on compare-load and restored on rollback, the default caches key on
the inventory version, and the hidden needles stay present.
- Playwright E2E (tests/studio/playwright_model_config.py) wired into
studio-ui-smoke.yml on port 18898: Context Length persists across a reload,
Reset clears the stored override, and infra models are absent from the picker.
- optional GPU-gated inference smoke (tests/studio/test_gpu_inference_smoke.py)
that auto-skips on GPU-less CI and stays short on a GPU.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: model pinning, row menus, hub inference settings, and inventory filters
Pinning
- Add a pinned models store (localStorage) with repo and per-quant pins
- Pinned section in the model selector's On Device list and the hub inventory,
with newest pins first so Pin to top lands on top
- Deleting a repo drops its pins
Row menus
- Replace loose row icons with a shared 3-dots menu (pin, reveal in file
manager, copy identifier, copy path, delete) on picker rows, hub quant rows,
the hub run bar, and on-device inventory rows
- Menus only render for models actually on disk; platform-aware reveal labels
- Backend: cached-model-path and reveal-cached-model endpoints resolving
managed HF-cache repos only
Hub inference settings
- Gear in the GGUF run bar opens an Inference settings dialog reusing the chat
page's controls: model config (context length, KV cache, speculative
decoding, chat template), system prompt, reasoning, sampling, tools and
retrieval
Inventory
- Model-type filter (text, vision, embedding, STT, TTS, diffusion) beside the
sort pill, both with a sort icon, capped widths and truncation so the
On device heading never wraps
- Unsloth-owned repos without an upstream provider logo fall back to the
Unsloth mascot avatar
- Discover / On Device tabs widened; hub search bar narrowed to match
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: revert the Unsloth mascot avatar fallback
Unsloth-owned repos without an upstream provider match go back to the
colored-initial tile, and unslothai is no longer a relabeled owner.
* Studio: run-bar options on single models, and aligned type/capability filters
- Give single-model (non-GGUF) run bars the same 3-dots options menu and
settings gear as GGUF, at repo level
- Drop Pin to top from the run-bar menus; pinning stays in the On Device list
- Add an Image to text (diffusion) capability with detection, and surface it
in both the hub Discover capability filter and the On Device type filter
- Align the On Device type filter with the Discover capability options and
share the same detection so both dropdowns match
* Studio: apply hub inference config on reload, eject action, and run-bar polish
- Fix inference settings not applying: the hub dialog now writes the config to
the runtime before reload, matching the chat page (selectModel reads runtime
state, not the selection)
- Order the settings gear before the 3-dots menu in the run bars
- Replace the loaded-model run-bar action (New Chat) with Eject, wired through
the inspector to the hub's ejectModel
- Truncate the results heading so a long search query clips instead of
overlapping the header pills in split view
- Use a plain magnifying-glass icon for the no-results empty state
* Studio: fix GPU settings loss, load guards, pins, filters, and cached paths
Reloading a model from the chat sidebar or the hub gear dialog rebuilt the
per-model config without the GPU memory fields, so manual GPU layers, MoE
placement, and the GPU pick were reset on every reload and could be saved
over a remembered config. The active config now comes from a shared
useActiveModelConfig hook that carries the GPU fields for GGUF models, and
the sidebar remount signature tracks them through a shared gpuFieldsSignature
helper.
The in-flight load guard lived in a ref inside each useChatModelRuntime
instance, so the chat page, hub page, and gear dialog could not see each
other's loads. A load started from the gear dialog left the hub page free to
eject the model mid-reload or start a second concurrent load. The runtime
store now records the loading pick, selectModel checks it across instances,
and ejectModel refuses with a toast while any load is in flight.
The cached-model-path endpoint matched GGUF files by basename and excluded
only mmproj, so Copy path and Reveal could return an MTP drafter for a quant
and returned 404 for directory layouts like BF16/model-00001.gguf. Variant
files are now resolved from snapshot-relative paths with the same drafter,
mmproj, and big-endian exclusions as the load path, shared through a new
_main_variant_gguf_label helper.
Hub and picker fixes:
- rename the diffusion capability label from "Image to text" to
"Image generation", since it detects image generators
- validate pinned quants through the cached variant listing, keep the last
verified set while revalidating, and drop deleted quants immediately
- pass a measured scroll margin to the on-device virtual list so rows past
the overscan stay visible below the pinned block
- keep the delete menu for stopped partial safetensors downloads
- give the inventory type filter a reset in Clear filters, a truthful empty
state with a Show all types action, and hide it on the datasets view
- order picker pinned rows by pin recency, include pinned matches in the
empty-state check, and sync pins across browser tabs
- count only the visible rows in the On device list header
Tests: contract checks for each fix in test_model_picker_contracts.py and a
backend test for the variant label selection.
* Studio: reveal cached models in Windows Explorer under WSL
The reveal endpoint only branched on macOS, Windows, and generic Linux.
Under WSL the Linux branch spawned xdg-open, which is missing on a stock
distro without a Linux desktop, so the request failed with a 500 and the
UI showed a failed to open file manager error.
WSL is now detected with the existing helper and the path is converted
with wslpath before opening explorer.exe, selecting the file the same
way native Windows does. Directories open directly. When interop is
unavailable the old xdg-open fallback still runs. The macOS, native
Windows, and native Linux branches are unchanged, and the Tauri app is
covered since its hub reveal calls this same local endpoint.
Tests: platform guards for the WSL reveal, the interop fallback, and
the unchanged native Linux behavior in tests/studio/test_reveal_file_manager.py.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Adjust model picker row spacing and cogwheel hover consistency
* Studio: exact hidden model ids and newest revision cached paths
A custom RAG embedder repo was published to the frontend as a basename
substring needle, so a generic name like org/model could hide unrelated
models in the pickers. The hidden-models endpoint now sends full repo ids
that are matched exactly.
Copy path and Reveal picked a GGUF variant from an arbitrary cache
revision when the same file existed in more than one. The newest revision
now wins, matching the whole repo lookup.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix model picker GPU config, metadata, and cache selection
Load each compare model with its saved GPU memory mode, GPU layers, CPU MoE layers, and selected GPU IDs. Reconcile saved GPU IDs with the current hardware. Include the active native GGUF path token in metadata checks. Search all Hugging Face cache roots when resolving cached models and select the largest visible cache entry. Remove obsolete barrel exports and the staging-only GPU memory helper.
* Studio: hide hub inference settings gear for now
The cogwheel in the hub download cards is out of scope for this PR. The
dialog component stays in place and a TODO marks where the button
returns in a future PR.
* Refresh hidden model matchers
* Fix GGUF detection, compare context pin, and picker delete staleness
Treat any pick with a GGUF variant as GGUF in selectModel so the first
load after downloading an uncached quant validates and sizes with the
right GPU settings instead of unloading the current model on a wrong
preflight. Variant picks now also set isGguf on their selection meta.
Stop compare panes from inheriting the active model's context pin when
their own saved config says Auto. Null context in a remembered config
now means no pin, matching how the pane settings are shown.
Route picker deletes through the hub inventory client, which
invalidates the HF cache scan and the variants cache. The legacy
delete route left the scan cache warm, so deleted models reappeared
in the picker until the TTL expired. Removed the now unused legacy
delete client and updated the contract test to match.
* Studio: fix stale GGUF load-marker ordering test
The load-in-flight marker still precedes the hub-download guard and the
unload, but the llama_extra_args inheritance that used to sit between the
marker and the guard now runs ahead of the GGUF branch, so it is no
longer a landmark inside the sliced source. Drop it from the ordering
assertion and keep the marker -> guard -> unload invariant.
* Studio: fix per-model config edge cases in compare loads and saved defaults
- chat-settings-sheet: gate the MTP fallback note and context/VRAM warning on
the broader isGguf (variant, loaded gguf context, or .gguf suffix) instead of
isLoadedGguf, so direct-file and custom-folder GGUF loads still surface
those diagnostics.
- shared-composer: a compare pane's context now comes from its own config only
(a saved pin, else null for Auto/native). It no longer inherits the active
model's shared snapshot, which resolveFitMaxSeqLength treated as an explicit
pin and could load a pane at another model's context (VRAM/OOM), matching the
single-model load path.
- model-config-page: when an auto-fit GGUF is saved with fixed GPU layers
(Manual) and Remember, pin the displayed fitted context so a later fresh load
keeps the placement instead of sending native/0 and recreating the OOM.
- per-model-config: treat Auto GPU memory mode and Auto/default speculative type
as follow-global defaults; do not persist them as per-model overrides so later
global preference changes keep applying.
* Studio: gate vision capability on GGUF projectors and bound remote template downloads
- cache_inventory: only mark a cached repo vision-capable when it holds an actual
GGUF mmproj projector, not any file whose name merely contains "mmproj" (e.g.
mmproj_config.json), matching the runtime's GGUF-only projector detection.
- picker/service: pre-check the remote file size before downloading an uncached
repo's chat template / tokenizer config, so a maliciously large sidecar is
skipped instead of fetched and retained in full, mirroring the size gate the
local-file path already applies.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: add source-contract guards for the per-model-config edge-case fixes
Guard the four per-model-config fixes against silent regression in CI:
- local GGUF diagnostics gate on the broad isGguf, not the variant-only isLoadedGguf
- fixed-layer GGUF saves pin the displayed context
- Auto GPU mode and Auto/default speculative are not persisted as per-model overrides
- a compare pane's context comes from its own config, not the active model's snapshot
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: clear manual GPU knobs on Default and resolve local embedders before repo-id
- model-config-page: switching GPU Memory back to Default now clears the Manual-only
knobs (gpuLayers/nCpuMoe/selectedGpuIds); otherwise a remembered config kept stale
pins that a later load re-applied when the global GPU preference was Manual, despite
the page showing Default.
- routes/models hidden_model_matchers: resolve an existing local path before the repo-id
regex, mirroring is_hidden_model, so a local embedder shaped like "models/embedder" is
hidden by exact path instead of leaking as a chat model.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: add _is_mtp_drafter to the model_config stub in the export-paths test
routes/models.py imports _is_mtp_drafter from utils.models.model_config at module
load, but the lightweight stub in test_export_absolute_paths.py did not provide it,
so loading the module under the stub raised ImportError on Backend CI. Add the stub.
* Studio: read a picked GGUF's chat template through the native path lease
The picker chat-template GET has no native-path-lease plumbing, so a
desktop-picked (drag-drop) GGUF could not show its default chat template
in Run Settings until the model was loaded: the endpoint only receives
the display label, not the leased file path.
Read the embedded template through the existing lease-aware
/api/inference/validate probe instead. A new include_chat_template flag
resolves the granted canonical path and returns the GGUF's own embedded
template, never a sibling sidecar (the grant authorizes just that one
file); it skips the training guard like include_context_length and is
bounded by MAX_CHAT_TEMPLATE_BYTES. The frontend fetch mints a one-shot
validate-model lease when a native token is present and keeps the plain
GET path for HF and allowlisted local models.
Adds backend and source-contract regression tests.
* Studio: call worker.direct_wheel_url in the ROCm wheel-url test
The ROCm Mamba/SSM test referenced worker.py's private _direct_wheel_url,
but the worker imports the wheel helper under its public name
direct_wheel_url (utils.wheel_utils). When the worker module loads (its
imports resolve in CI), worker_mod._direct_wheel_url raised AttributeError;
the test only masked it by skipping when the worker could not be imported.
Call the name that actually exists so the assertion runs; it still returns
None for an empty cuda_major (ROCm).
* Studio: reset max sequence length to the app default, not the loaded value
For a non-GGUF active model, the per-model config seeds maxSeqLength from
the loaded runtime value so the panel opens showing the running context.
Reset set config.maxSeqLength to null, but the null fallback resolved back
to that captured runtime value, so the field kept showing the old custom
length and the config saved/reloaded it again. A remembered or active
max-length override therefore could not be cleared from Run settings.
Fall the null/default case back to the app default (clamped to the model's
native ceiling) instead of the active runtime snapshot, so Reset actually
clears the override. The initial view is unaffected: an active model's
config.maxSeqLength is already non-null, so it still shows the loaded value.
Adds a source-contract regression guard.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: persist default max length, refresh deleted quants, hide non-chat locals
Three follow-up fixes from review of the per-model-config picker:
- Max sequence length: the persisted per-model record now keeps config's
maxSeqLength (null after Reset) so isDefaultConfig can clear a remembered
override; the resolved app-default is substituted only into the load
request, never the saved record. Previously Reset saved the concrete
default and left the model pinned/remembered.
- GGUF variant expander: deleting a downloaded quant from a repo that still
has other cached quants now bumps the expander refresh key, so the removed
quant stops showing as downloaded and clickable (which would try to reload
the deleted file) until the repo is collapsed and reopened.
- Local picker rows: require capabilities.canChat before listing a local
models-folder / LM Studio row. A weightless folder (only config.json) is
classified non-chat, and toLocalModelInfo drops capabilities, so selecting
such a row would try to load a path the inventory already marked non-chat.
Adds source-contract regression guards for all three.
* Fix compare-pane and Reset context defaults in model picker
Two related per-model-config default regressions:
- A non-GGUF compare pane with no saved maxSeqLength fell back to the
active model's shared runtime snapshot, so comparing a saved 128K model
against an unconfigured pane loaded the latter at 128K and could OOM. It
now falls back to the shared app default (DEFAULT_MAX_SEQ_LENGTH), the
same fallback the single-model config path uses.
- contextAtDefault treated an explicit customContextLength equal to the
native ceiling as a default, which wedged the Reset button disabled for
a deliberate pin-to-native. It now counts as default only when there is
no override at all.
DEFAULT_MAX_SEQ_LENGTH becomes a single exported constant in
per-model-config.ts so the single-model config and the compare path share
one source of truth. Adds source-contract guards for both fixes.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Skip over-cap remote Jinja templates so the tokenizer template wins
The remote chat-template resolver bounded raw chat_template.jinja downloads
only by MAX_TEMPLATE_METADATA_BYTES (4 MiB), then returned the first
non-empty Jinja unconditionally. The picker route drops any template larger
than MAX_CHAT_TEMPLATE_BYTES (64 KiB), so an uncached repo whose
chat_template.jinja sits between 64 KiB and 4 MiB returned no template at
all, even when a valid smaller tokenizer_config.json template existed. The
local path already skips oversized .jinja files and falls through.
Gate the extracted Jinja on MAX_CHAT_TEMPLATE_BYTES and continue searching
when it exceeds the cap, matching _chat_template_from_jinja_file. The 4 MiB
download bound stays for JSON files that merely embed a small template. Adds
a regression test that a big Jinja plus a valid tokenizer config resolves to
the tokenizer template.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Guard legacy per-model-config migration idempotency
The v1->v2 localStorage migration (unsloth_load_settings ->
unsloth_model_configs) runs on every store read, so it must migrate exactly
once and never re-run, duplicate, or clobber a newer per-model config on a
reload or restart. That was covered only by a manual proof, so add durable
guards:
- Source-contract test pinning the three idempotency layers (the in-memory
legacyMigrationChecked guard, the persistent unsloth_model_configs_migrated
flag set in every terminal branch, and the non-overwriting Object.hasOwn
merge-skip) plus the readMap invocation. Reddens if any layer is dropped.
- Playwright model-config E2E: promote the legacy-migration step to a gating
check (soft_fail, which gates under the CI STUDIO_UI_STRICT=1) that the
migrated value is preserved and the flag is set, then reload again with a
fresh legacy seed present and assert the stored key set is unchanged, so a
second reload cannot re-migrate, duplicate, or clobber.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Note the migration E2E now gates idempotency under STUDIO_UI_STRICT
* Tighten model-picker per-model-config code comments
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
Co-authored-by: shimmyshimmer <shimmyshimmer@users.noreply.github.com>
Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local>
Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
This commit is contained in:
Eyera
GitHub
parent
27f3473c7e
commit
27b6d553fe
104 changed files with 10736 additions and 4770 deletions
50
.github/workflows/studio-ui-smoke.yml
vendored
50
.github/workflows/studio-ui-smoke.yml
vendored
|
|
@ -237,6 +237,54 @@ jobs:
|
|||
kill "${STUDIO_EXTRA_PID}" 2>/dev/null || true
|
||||
sleep 2
|
||||
|
||||
# Model-picker per-model-config regression (PR #7207 re-land of #6647).
|
||||
# Fourth Unsloth on its own port; loads the tiny GGUF and drives the
|
||||
# picker's run-settings surface: Context Length persists across a reload,
|
||||
# Reset clears the stored override (never pins it), and the infra models
|
||||
# (RAG embedder + llama.cpp probe) stay hidden from the picker.
|
||||
- name: Reset auth + boot Unsloth for model-config tests (port 18898)
|
||||
run: |
|
||||
unsloth studio reset-password
|
||||
mkdir -p logs
|
||||
UNSLOTH_API_ONLY=1 unsloth studio -H 127.0.0.1 -p 18898 \
|
||||
> logs/studio_modelcfg.log 2>&1 &
|
||||
echo "STUDIO_MODELCFG_PID=$!" >> "$GITHUB_ENV"
|
||||
|
||||
- name: Wait for /api/health on 18898
|
||||
run: |
|
||||
for i in $(seq 1 180); do
|
||||
if curl -fs "http://127.0.0.1:18898/api/health" > /tmp/health4.json; then
|
||||
jq -e '.status == "healthy"' /tmp/health4.json && break
|
||||
fi
|
||||
sleep 1
|
||||
done
|
||||
jq -e '.status == "healthy"' /tmp/health4.json
|
||||
|
||||
- name: Pass bootstrap pw for model-config test
|
||||
run: |
|
||||
NEW="CIModelCfg-$(python -c 'import secrets; print(secrets.token_urlsafe(16))')"
|
||||
echo "::add-mask::$NEW"
|
||||
echo "STUDIO_MODELCFG_NEW_PW=$NEW" >> "$GITHUB_ENV"
|
||||
|
||||
- name: Drive model-picker per-model-config with Playwright
|
||||
env:
|
||||
BASE_URL: http://127.0.0.1:18898
|
||||
STUDIO_NEW_PW: ${{ env.STUDIO_MODELCFG_NEW_PW }}
|
||||
PW_ART_DIR: logs/playwright_modelcfg
|
||||
STUDIO_UI_STRICT: '1'
|
||||
GGUF_REPO: ${{ env.GGUF_REPO }}
|
||||
GGUF_VARIANT: ${{ env.GGUF_VARIANT }}
|
||||
STUDIO_MODEL_HINT: gemma-3-270m
|
||||
run: |
|
||||
mkdir -p logs/playwright_modelcfg
|
||||
python tests/studio/playwright_model_config.py
|
||||
|
||||
- name: Stop fourth Unsloth
|
||||
if: always()
|
||||
run: |
|
||||
kill "${STUDIO_MODELCFG_PID}" 2>/dev/null || true
|
||||
sleep 2
|
||||
|
||||
# IME + multilingual paste regression (issue #5318 / PR #5327).
|
||||
# Third Unsloth on its own port so a hang here cannot poison the
|
||||
# earlier UI tests. No GGUF -- the bug surface is the composer.
|
||||
|
|
@ -297,12 +345,14 @@ jobs:
|
|||
path: |
|
||||
logs/studio.log
|
||||
logs/studio_extra.log
|
||||
logs/studio_modelcfg.log
|
||||
logs/studio_ime.log
|
||||
logs/install.log
|
||||
logs/server-logs/
|
||||
logs/playwright
|
||||
logs/playwright-permissions-*
|
||||
logs/playwright_extra
|
||||
logs/playwright_modelcfg
|
||||
logs/playwright_ime
|
||||
logs/studio-permissions-*.log
|
||||
retention-days: 7
|
||||
|
|
|
|||
|
|
@ -28,6 +28,7 @@ from hub.schemas.inventory import (
|
|||
CachedModelsResponse,
|
||||
DeleteCachedModelResponse,
|
||||
GgufVariantsResponse,
|
||||
HiddenModelsResponse,
|
||||
LocalModelListResponse,
|
||||
ModelsFolderResponse,
|
||||
RecommendedFoldersResponse,
|
||||
|
|
@ -214,6 +215,16 @@ async def list_cached_models(
|
|||
return await cache_inventory.list_cached_models_response(hf_token)
|
||||
|
||||
|
||||
@router.get("/hidden-models", response_model = HiddenModelsResponse)
|
||||
async def list_hidden_models(current_subject: str = Depends(get_current_subject)):
|
||||
import asyncio
|
||||
|
||||
from routes.models import hidden_model_matchers
|
||||
|
||||
needles, exact_ids, exact_paths = await asyncio.to_thread(hidden_model_matchers)
|
||||
return HiddenModelsResponse(needles = needles, exact_ids = exact_ids, exact_paths = exact_paths)
|
||||
|
||||
|
||||
@router.delete(
|
||||
"/delete-cached",
|
||||
response_model = DeleteCachedModelResponse,
|
||||
|
|
|
|||
|
|
@ -160,6 +160,7 @@ class CachedRepoBase(BaseModel):
|
|||
repo_id: str
|
||||
size_bytes: int = 0
|
||||
cache_path: Optional[str] = None
|
||||
last_modified: Optional[float] = None
|
||||
partial: bool = False
|
||||
partial_transport: Optional[str] = None
|
||||
inventory_id: Optional[str] = None
|
||||
|
|
@ -189,6 +190,12 @@ class CachedModelsResponse(BaseModel):
|
|||
cached: List[CachedModelRepo] = Field(default_factory = list)
|
||||
|
||||
|
||||
class HiddenModelsResponse(BaseModel):
|
||||
needles: List[str] = Field(default_factory = list)
|
||||
exact_ids: List[str] = Field(default_factory = list)
|
||||
exact_paths: List[str] = Field(default_factory = list)
|
||||
|
||||
|
||||
class AddScanFolderRequest(BaseModel):
|
||||
"""Request body for adding a custom scan folder."""
|
||||
|
||||
|
|
|
|||
|
|
@ -31,6 +31,7 @@ from hub.services.models.common import (
|
|||
_is_checkpoint_weight_name,
|
||||
_is_gguf_filename,
|
||||
_is_main_gguf_filename,
|
||||
_is_mmproj_filename,
|
||||
_is_transformers_safetensors_weight_name,
|
||||
_local_inventory_id,
|
||||
_prefer_complete_larger,
|
||||
|
|
@ -132,6 +133,39 @@ def _repo_has_gguf_files(repo_info) -> bool:
|
|||
return _repo_gguf_size_bytes(repo_info) > 0
|
||||
|
||||
|
||||
def _blob_mtime(file_obj) -> float:
|
||||
ts = getattr(file_obj, "blob_last_modified", None)
|
||||
if isinstance(ts, (int, float)) and ts > 0:
|
||||
return float(ts)
|
||||
blob_path = getattr(file_obj, "blob_path", None)
|
||||
if blob_path:
|
||||
try:
|
||||
return float(Path(blob_path).stat().st_mtime)
|
||||
except OSError:
|
||||
pass
|
||||
return 0.0
|
||||
|
||||
|
||||
def _repo_gguf_last_modified(repo_info) -> float:
|
||||
latest = 0.0
|
||||
for revision in repo_info.revisions:
|
||||
for f in revision.files:
|
||||
if _is_main_gguf_filename(f.file_name):
|
||||
latest = max(latest, _blob_mtime(f))
|
||||
return latest
|
||||
|
||||
|
||||
def _repo_has_mmproj(repo_info) -> bool:
|
||||
# An mmproj file only makes a repo vision-capable when it is an actual GGUF
|
||||
# projector; a non-GGUF sidecar (e.g. mmproj_config.json) does not, and the
|
||||
# runtime's projector detection is GGUF-only.
|
||||
return any(
|
||||
_is_gguf_filename(f.file_name) and _is_mmproj_filename(f.file_name)
|
||||
for revision in repo_info.revisions
|
||||
for f in revision.files
|
||||
)
|
||||
|
||||
|
||||
def _cached_repo_file_name(file_obj) -> str:
|
||||
file_path = getattr(file_obj, "file_path", None)
|
||||
if file_path:
|
||||
|
|
@ -291,6 +325,7 @@ def _scan_cached_gguf() -> list[dict]:
|
|||
continue
|
||||
key = repo_id.lower()
|
||||
existing = seen_lower.get(key)
|
||||
last_modified = _repo_gguf_last_modified(repo_info)
|
||||
row = {
|
||||
"repo_id": repo_id,
|
||||
"size_bytes": max(total_size, variant_state_size),
|
||||
|
|
@ -300,6 +335,9 @@ def _scan_cached_gguf() -> list[dict]:
|
|||
# per-variant detail lives on GgufVariantDetail.
|
||||
"partial_transport": None,
|
||||
}
|
||||
last_modified = max(last_modified, (existing or {}).get("last_modified", 0.0))
|
||||
if last_modified > 0:
|
||||
row["last_modified"] = last_modified
|
||||
row.update(
|
||||
_cache_inventory_fields(
|
||||
repo_id,
|
||||
|
|
@ -308,11 +346,20 @@ def _scan_cached_gguf() -> list[dict]:
|
|||
requires_variant = True,
|
||||
)
|
||||
)
|
||||
if _repo_has_mmproj(repo_info):
|
||||
row["capabilities"]["supports_vision"] = True
|
||||
# Visible infra variants remain management-only.
|
||||
if is_hidden_infra:
|
||||
row["capabilities"]["can_chat"] = False
|
||||
if _prefer_cache_row(row, existing):
|
||||
if existing and existing["capabilities"].get("supports_vision"):
|
||||
row["capabilities"]["supports_vision"] = True
|
||||
seen_lower[key] = row
|
||||
else:
|
||||
if last_modified > existing.get("last_modified", 0.0):
|
||||
existing["last_modified"] = last_modified
|
||||
if row["capabilities"].get("supports_vision"):
|
||||
existing["capabilities"]["supports_vision"] = True
|
||||
except Exception as e:
|
||||
repo_label = getattr(repo_info, "repo_id", "<unknown>")
|
||||
logger.warning(f"Skipping cached GGUF repo {repo_label}: {e}")
|
||||
|
|
@ -340,13 +387,14 @@ class _CachedNonGgufPayload(NamedTuple):
|
|||
size_bytes: int
|
||||
has_runnable_weights: bool
|
||||
model_format: ModelFormat
|
||||
last_modified: float
|
||||
|
||||
|
||||
def _repo_non_gguf_model_payload(repo_info) -> _CachedNonGgufPayload:
|
||||
all_weight_blobs: dict[str, int] = {}
|
||||
adapter_blobs: dict[str, int] = {}
|
||||
safetensors_blobs: dict[str, int] = {}
|
||||
checkpoint_blobs: dict[str, int] = {}
|
||||
all_weight_blobs: dict[str, tuple[int, float]] = {}
|
||||
adapter_blobs: dict[str, tuple[int, float]] = {}
|
||||
safetensors_blobs: dict[str, tuple[int, float]] = {}
|
||||
checkpoint_blobs: dict[str, tuple[int, float]] = {}
|
||||
has_config = False
|
||||
has_adapter_config = False
|
||||
has_adapter_weights = False
|
||||
|
|
@ -354,12 +402,15 @@ def _repo_non_gguf_model_payload(repo_info) -> _CachedNonGgufPayload:
|
|||
has_transformers_safetensors = False
|
||||
has_checkpoint = False
|
||||
|
||||
def _record_blob(target: dict[str, int], file_obj, rev_id: str, file_name: str) -> None:
|
||||
def _record_blob(
|
||||
target: dict[str, tuple[int, float]], file_obj, rev_id: str, file_name: str
|
||||
) -> None:
|
||||
blob_path = getattr(file_obj, "blob_path", None)
|
||||
size = int(file_obj.size_on_disk or 0)
|
||||
key = str(blob_path) if blob_path else f"{rev_id}:{file_name}"
|
||||
target[key] = size
|
||||
all_weight_blobs[key] = size
|
||||
value = (size, _blob_mtime(file_obj))
|
||||
target[key] = value
|
||||
all_weight_blobs[key] = value
|
||||
|
||||
for revision in repo_info.revisions:
|
||||
rev_id = getattr(revision, "commit_hash", None) or str(id(revision))
|
||||
|
|
@ -403,18 +454,19 @@ def _repo_non_gguf_model_payload(repo_info) -> _CachedNonGgufPayload:
|
|||
or "unknown"
|
||||
)
|
||||
if model_format == "adapter":
|
||||
size_bytes = sum(adapter_blobs.values())
|
||||
selected_blobs = adapter_blobs
|
||||
elif model_format == "safetensors":
|
||||
size_bytes = sum(safetensors_blobs.values())
|
||||
selected_blobs = safetensors_blobs
|
||||
elif model_format == "checkpoint":
|
||||
size_bytes = sum(checkpoint_blobs.values())
|
||||
selected_blobs = checkpoint_blobs
|
||||
else:
|
||||
size_bytes = sum(all_weight_blobs.values())
|
||||
selected_blobs = all_weight_blobs
|
||||
|
||||
return _CachedNonGgufPayload(
|
||||
size_bytes = size_bytes,
|
||||
size_bytes = sum(size for size, _mtime in selected_blobs.values()),
|
||||
has_runnable_weights = model_format != "unknown",
|
||||
model_format = model_format,
|
||||
last_modified = max((mtime for _size, mtime in selected_blobs.values()), default = 0.0),
|
||||
)
|
||||
|
||||
|
||||
|
|
@ -544,6 +596,12 @@ def _scan_cached_models() -> list[dict]:
|
|||
),
|
||||
**_cached_model_local_metadata(repo_path),
|
||||
}
|
||||
last_modified = max(
|
||||
payload.last_modified,
|
||||
(existing or {}).get("last_modified", 0.0),
|
||||
)
|
||||
if last_modified > 0:
|
||||
row["last_modified"] = last_modified
|
||||
row.update(
|
||||
_cache_inventory_fields(
|
||||
repo_id,
|
||||
|
|
@ -553,6 +611,8 @@ def _scan_cached_models() -> list[dict]:
|
|||
)
|
||||
if _prefer_cache_row(row, existing):
|
||||
seen_lower[key] = row
|
||||
elif last_modified > existing.get("last_modified", 0.0):
|
||||
existing["last_modified"] = last_modified
|
||||
except Exception as e:
|
||||
repo_label = getattr(repo_info, "repo_id", "<unknown>")
|
||||
logger.warning(f"Skipping cached model repo {repo_label}: {e}")
|
||||
|
|
|
|||
|
|
@ -315,6 +315,7 @@ from hub.routes import (
|
|||
datasets_router as hub_datasets_router,
|
||||
token_router as hub_token_router,
|
||||
)
|
||||
from picker.routes import templates_router as picker_templates_router
|
||||
from hub.schemas.downloads import TransportCapabilities
|
||||
from hub.utils.download_registry import (
|
||||
get_download_transport_capabilities,
|
||||
|
|
@ -764,6 +765,7 @@ _BODY_PROTECTED_PREFIXES = (
|
|||
"/v1/completions",
|
||||
"/p/",
|
||||
"/api/inference",
|
||||
"/api/picker",
|
||||
"/api/data-recipe",
|
||||
"/api/datasets",
|
||||
"/api/hub",
|
||||
|
|
@ -995,6 +997,7 @@ app.include_router(rag_router, prefix = "/api/rag", tags = ["rag"])
|
|||
app.include_router(training_history_router, prefix = "/api/train", tags = ["training-history"])
|
||||
app.include_router(hub_inventory_router, prefix = "/api/hub", tags = ["hub"])
|
||||
app.include_router(hub_datasets_router, prefix = "/api/hub/datasets", tags = ["hub"])
|
||||
app.include_router(picker_templates_router, prefix = "/api/picker", tags = ["picker"])
|
||||
app.include_router(hub_token_router, prefix = "/api/hub", tags = ["hub"])
|
||||
|
||||
# Re-wrap client-error responses on the /v1/* surface into OpenAI/Anthropic
|
||||
|
|
|
|||
|
|
@ -18,6 +18,8 @@ from pydantic import (
|
|||
model_validator,
|
||||
)
|
||||
|
||||
from picker.schemas import MAX_CHAT_TEMPLATE_BYTES
|
||||
|
||||
|
||||
class LoadRequest(BaseModel):
|
||||
"""Request to load a model for inference"""
|
||||
|
|
@ -54,8 +56,16 @@ class LoadRequest(BaseModel):
|
|||
@field_validator("chat_template_override")
|
||||
@classmethod
|
||||
def normalize_blank_chat_template_override(cls, value: Optional[str]) -> Optional[str]:
|
||||
if value is not None and value.strip() == "":
|
||||
if value is None:
|
||||
return None
|
||||
# Char count is a lower bound on UTF-8 byte length: reject an oversized
|
||||
# template before spending work encoding it.
|
||||
if len(value) > MAX_CHAT_TEMPLATE_BYTES:
|
||||
raise ValueError(f"Chat template exceeds the {MAX_CHAT_TEMPLATE_BYTES}-byte limit.")
|
||||
if value.strip() == "":
|
||||
return None
|
||||
if len(value.encode("utf-8")) > MAX_CHAT_TEMPLATE_BYTES:
|
||||
raise ValueError(f"Chat template exceeds the {MAX_CHAT_TEMPLATE_BYTES}-byte limit.")
|
||||
return value
|
||||
|
||||
cache_type_kv: Optional[str] = Field(
|
||||
|
|
@ -206,6 +216,13 @@ class ValidateModelRequest(BaseModel):
|
|||
description = "Also read the native context length from the local GGUF header. "
|
||||
"Opt-in so the normal load preflight doesn't pay for a cache scan it doesn't need.",
|
||||
)
|
||||
include_chat_template: bool = Field(
|
||||
False,
|
||||
description = "Also read the embedded chat template from the local GGUF header, so a "
|
||||
"native (picked / drag-drop) file's default template can be shown before it is loaded. "
|
||||
"Opt-in and, like include_context_length, a metadata-only probe that skips the training "
|
||||
"guard. Only the leased file's own embedded template is read, never sibling sidecars.",
|
||||
)
|
||||
|
||||
|
||||
class TransformersUpgradeInfo(BaseModel):
|
||||
|
|
@ -266,6 +283,11 @@ class ValidateModelResponse(BaseModel):
|
|||
description = "MoE expert-layer count (the manual --n-cpu-moe ceiling), read from the GGUF "
|
||||
"header alongside context_length; 0 for dense models, None when not read.",
|
||||
)
|
||||
chat_template: Optional[str] = Field(
|
||||
None,
|
||||
description = "Embedded GGUF chat template, read from the header when include_chat_template "
|
||||
"is set (native lease-backed picks); None for non-GGUF, over-cap, or not-read templates.",
|
||||
)
|
||||
# Additive fields; the consuming consent dialog ships in a follow-up frontend PR.
|
||||
requires_transformers_upgrade: bool = Field(
|
||||
False,
|
||||
|
|
|
|||
2
studio/backend/picker/__init__.py
Normal file
2
studio/backend/picker/__init__.py
Normal file
|
|
@ -0,0 +1,2 @@
|
|||
# SPDX-License-Identifier: AGPL-3.0-only
|
||||
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
6
studio/backend/picker/routes/__init__.py
Normal file
6
studio/backend/picker/routes/__init__.py
Normal file
|
|
@ -0,0 +1,6 @@
|
|||
# SPDX-License-Identifier: AGPL-3.0-only
|
||||
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
from .templates import router as templates_router
|
||||
|
||||
__all__ = ["templates_router"]
|
||||
45
studio/backend/picker/routes/templates.py
Normal file
45
studio/backend/picker/routes/templates.py
Normal file
|
|
@ -0,0 +1,45 @@
|
|||
# SPDX-License-Identifier: AGPL-3.0-only
|
||||
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
from typing import Optional
|
||||
|
||||
from fastapi import APIRouter, Body, Depends, Query
|
||||
|
||||
from auth.authentication import get_current_subject
|
||||
from hub.dependencies import get_hf_token
|
||||
|
||||
from ..schemas import (
|
||||
MAX_CHAT_TEMPLATE_BYTES,
|
||||
ModelTemplateResponse,
|
||||
ValidateChatTemplateRequest,
|
||||
ValidateChatTemplateResponse,
|
||||
)
|
||||
from ..service import read_default_chat_template, validate_chat_template
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
@router.post("/validate-chat-template", response_model = ValidateChatTemplateResponse)
|
||||
async def validate_chat_template_route(
|
||||
body: ValidateChatTemplateRequest = Body(...),
|
||||
current_subject: str = Depends(get_current_subject),
|
||||
) -> ValidateChatTemplateResponse:
|
||||
return await asyncio.to_thread(validate_chat_template, body.template)
|
||||
|
||||
|
||||
@router.get("/chat-template/{model_name:path}", response_model = ModelTemplateResponse)
|
||||
async def get_default_chat_template_route(
|
||||
model_name: str,
|
||||
gguf_variant: Optional[str] = Query(None),
|
||||
hf_token: Optional[str] = Depends(get_hf_token),
|
||||
current_subject: str = Depends(get_current_subject),
|
||||
) -> ModelTemplateResponse:
|
||||
template = await asyncio.to_thread(
|
||||
read_default_chat_template, model_name, hf_token, gguf_variant
|
||||
)
|
||||
if template is not None and len(template.encode("utf-8")) > MAX_CHAT_TEMPLATE_BYTES:
|
||||
template = None
|
||||
return ModelTemplateResponse(model_name = model_name, chat_template = template)
|
||||
32
studio/backend/picker/schemas.py
Normal file
32
studio/backend/picker/schemas.py
Normal file
|
|
@ -0,0 +1,32 @@
|
|||
# SPDX-License-Identifier: AGPL-3.0-only
|
||||
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
from typing import Optional
|
||||
|
||||
from pydantic import BaseModel, Field, field_validator
|
||||
|
||||
# Mirror the frontend's 64 KiB chat-template contract (per-model-config.ts) at
|
||||
# the API boundary so a direct caller cannot make Jinja parse an oversized
|
||||
# template. MaxBodyMiddleware only caps the whole request body, not this field.
|
||||
MAX_CHAT_TEMPLATE_BYTES = 65_536
|
||||
|
||||
|
||||
class ValidateChatTemplateRequest(BaseModel):
|
||||
template: str = Field(default = "")
|
||||
|
||||
@field_validator("template")
|
||||
@classmethod
|
||||
def _enforce_template_size(cls, value: str) -> str:
|
||||
if len(value.encode("utf-8")) > MAX_CHAT_TEMPLATE_BYTES:
|
||||
raise ValueError(f"Chat template exceeds the {MAX_CHAT_TEMPLATE_BYTES}-byte limit.")
|
||||
return value
|
||||
|
||||
|
||||
class ValidateChatTemplateResponse(BaseModel):
|
||||
valid: bool
|
||||
error: Optional[str] = None
|
||||
|
||||
|
||||
class ModelTemplateResponse(BaseModel):
|
||||
model_name: str
|
||||
chat_template: Optional[str] = None
|
||||
426
studio/backend/picker/service.py
Normal file
426
studio/backend/picker/service.py
Normal file
|
|
@ -0,0 +1,426 @@
|
|||
# SPDX-License-Identifier: AGPL-3.0-only
|
||||
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
from hub.services.models.folder_browser import (
|
||||
_build_browse_allowlist,
|
||||
_is_path_inside_allowlist,
|
||||
)
|
||||
from hub.utils.gguf import extract_quant_label, iter_hf_cache_snapshots
|
||||
from utils.models.gguf_metadata import read_gguf_chat_template
|
||||
from utils.models.model_config import (
|
||||
_extract_quant_label,
|
||||
_is_big_endian_gguf_path,
|
||||
_is_mmproj,
|
||||
_is_mtp_drafter,
|
||||
)
|
||||
from utils.paths.path_utils import (
|
||||
is_local_path,
|
||||
normalize_path,
|
||||
resolve_cached_repo_id_case,
|
||||
)
|
||||
|
||||
from .schemas import MAX_CHAT_TEMPLATE_BYTES, ValidateChatTemplateResponse
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_VALID_REPO_ID = re.compile(r"^[A-Za-z0-9._-]+/[A-Za-z0-9._-]+$")
|
||||
|
||||
|
||||
def _is_valid_repo_id(repo_id: str) -> bool:
|
||||
return bool(_VALID_REPO_ID.fullmatch(repo_id))
|
||||
|
||||
|
||||
_TOKENIZER_CONFIG_PATHS = ("tokenizer_config.json", "LLM/tokenizer_config.json")
|
||||
_JINJA_TEMPLATE_PATHS = ("chat_template.jinja", "LLM/chat_template.jinja")
|
||||
_PROCESSOR_TEMPLATE_PATHS = ("chat_template.json", "LLM/chat_template.json")
|
||||
|
||||
# Cap sidecar reads so a malformed or hostile metadata file cannot exhaust memory
|
||||
# before its template is size-checked. The JSON envelope may exceed a bare template
|
||||
# (it carries other tokenizer metadata); the extracted template is still bounded by
|
||||
# MAX_CHAT_TEMPLATE_BYTES downstream.
|
||||
MAX_TEMPLATE_METADATA_BYTES = 4 * 1024 * 1024
|
||||
|
||||
|
||||
def _read_bounded_text(path: Path, limit: int) -> Optional[str]:
|
||||
"""Read at most `limit` bytes of UTF-8 text; None if larger or unreadable."""
|
||||
try:
|
||||
with path.open("rb") as f:
|
||||
data = f.read(limit + 1)
|
||||
except OSError:
|
||||
return None
|
||||
if len(data) > limit:
|
||||
return None
|
||||
try:
|
||||
return data.decode("utf-8")
|
||||
except UnicodeError:
|
||||
return None
|
||||
|
||||
|
||||
def _leaf_inside_allowlist(path: Path, allow_roots: Optional[list[Path]]) -> bool:
|
||||
# Block symlinked children from escaping the validated directory (realpath-checked).
|
||||
# None = trusted caller (HF cache / remote download).
|
||||
return allow_roots is None or _is_path_inside_allowlist(path, allow_roots)
|
||||
|
||||
|
||||
def validate_chat_template(template: str) -> ValidateChatTemplateResponse:
|
||||
text = (template or "").strip()
|
||||
if not text:
|
||||
return ValidateChatTemplateResponse(valid = True, error = None)
|
||||
# Import Jinja lazily: optional at runtime (e.g. GGUF-only installs), so a
|
||||
# missing dependency must not crash API startup.
|
||||
try:
|
||||
from jinja2 import TemplateError
|
||||
from jinja2.ext import Extension
|
||||
from jinja2.sandbox import ImmutableSandboxedEnvironment
|
||||
except ImportError:
|
||||
return ValidateChatTemplateResponse(valid = True, error = None)
|
||||
|
||||
class _GenerationTag(Extension):
|
||||
# Accept Transformers' {% generation %} assistant-mask tag so a pasted HF
|
||||
# chat template validates (we only parse it).
|
||||
tags = {"generation"}
|
||||
|
||||
def parse(self, parser):
|
||||
next(parser.stream)
|
||||
return parser.parse_statements(["name:endgeneration"], drop_needle = True)
|
||||
|
||||
try:
|
||||
env = ImmutableSandboxedEnvironment(
|
||||
trim_blocks = True,
|
||||
lstrip_blocks = True,
|
||||
extensions = ["jinja2.ext.loopcontrols", _GenerationTag],
|
||||
)
|
||||
env.parse(text)
|
||||
return ValidateChatTemplateResponse(valid = True, error = None)
|
||||
except TemplateError as exc:
|
||||
message = getattr(exc, "message", None) or str(exc)
|
||||
lineno = getattr(exc, "lineno", None)
|
||||
if lineno:
|
||||
message = f"Line {lineno}: {message}"
|
||||
return ValidateChatTemplateResponse(valid = False, error = message)
|
||||
except Exception as exc:
|
||||
return ValidateChatTemplateResponse(valid = False, error = str(exc))
|
||||
|
||||
|
||||
def _chat_template_from_tokenizer_config(config: dict) -> Optional[str]:
|
||||
if not isinstance(config, dict):
|
||||
return None
|
||||
raw = config.get("chat_template")
|
||||
if isinstance(raw, str) and raw.strip():
|
||||
return raw
|
||||
if isinstance(raw, list):
|
||||
fallback: Optional[str] = None
|
||||
for entry in raw:
|
||||
if not isinstance(entry, dict):
|
||||
continue
|
||||
template = entry.get("template")
|
||||
if not isinstance(template, str):
|
||||
continue
|
||||
if entry.get("name") == "default":
|
||||
return template
|
||||
if fallback is None:
|
||||
fallback = template
|
||||
return fallback
|
||||
return None
|
||||
|
||||
|
||||
def _chat_template_from_jinja_file(
|
||||
dir_path: Path, allow_roots: Optional[list[Path]] = None
|
||||
) -> Optional[str]:
|
||||
for rel in _JINJA_TEMPLATE_PATHS:
|
||||
template_file = dir_path / rel
|
||||
if not template_file.exists() or not _leaf_inside_allowlist(template_file, allow_roots):
|
||||
continue
|
||||
try:
|
||||
if template_file.stat().st_size > MAX_CHAT_TEMPLATE_BYTES:
|
||||
continue
|
||||
template = template_file.read_text(encoding = "utf-8")
|
||||
except Exception:
|
||||
continue
|
||||
if template.strip():
|
||||
return template
|
||||
return None
|
||||
|
||||
|
||||
def _chat_template_from_processor_payload(payload: object) -> Optional[str]:
|
||||
# processor chat_template.json may be the template string itself or a
|
||||
# {name: template} map, not only a tokenizer_config-shaped object.
|
||||
if isinstance(payload, str):
|
||||
return payload if payload.strip() else None
|
||||
template = _chat_template_from_tokenizer_config(payload) # type: ignore[arg-type]
|
||||
if template:
|
||||
return template
|
||||
if isinstance(payload, dict):
|
||||
# Named-template map: prefer "default", else the first non-empty entry
|
||||
# (mirrors the tokenizer-config list fallback).
|
||||
default = payload.get("default")
|
||||
if isinstance(default, str) and default.strip():
|
||||
return default
|
||||
for value in payload.values():
|
||||
if isinstance(value, str) and value.strip():
|
||||
return value
|
||||
return None
|
||||
|
||||
|
||||
def _chat_template_from_processor_json(
|
||||
dir_path: Path, allow_roots: Optional[list[Path]] = None
|
||||
) -> Optional[str]:
|
||||
for rel in _PROCESSOR_TEMPLATE_PATHS:
|
||||
config_file = dir_path / rel
|
||||
if not config_file.exists() or not _leaf_inside_allowlist(config_file, allow_roots):
|
||||
continue
|
||||
raw = _read_bounded_text(config_file, MAX_TEMPLATE_METADATA_BYTES)
|
||||
if raw is None:
|
||||
continue
|
||||
try:
|
||||
payload = json.loads(raw)
|
||||
except Exception:
|
||||
continue
|
||||
template = _chat_template_from_processor_payload(payload)
|
||||
if template:
|
||||
return template
|
||||
return None
|
||||
|
||||
|
||||
def _chat_template_from_tokenizer_dir(
|
||||
dir_path: Path, allow_roots: Optional[list[Path]] = None
|
||||
) -> Optional[str]:
|
||||
jinja = _chat_template_from_jinja_file(dir_path, allow_roots)
|
||||
if jinja:
|
||||
return jinja
|
||||
for rel in _TOKENIZER_CONFIG_PATHS:
|
||||
config_file = dir_path / rel
|
||||
if not config_file.exists() or not _leaf_inside_allowlist(config_file, allow_roots):
|
||||
continue
|
||||
raw = _read_bounded_text(config_file, MAX_TEMPLATE_METADATA_BYTES)
|
||||
if raw is None:
|
||||
continue
|
||||
try:
|
||||
config = json.loads(raw)
|
||||
except Exception:
|
||||
continue
|
||||
template = _chat_template_from_tokenizer_config(config)
|
||||
if template:
|
||||
return template
|
||||
return _chat_template_from_processor_json(dir_path, allow_roots)
|
||||
|
||||
|
||||
_GGUF_SCAN_MAX_DEPTH = 2
|
||||
|
||||
|
||||
def _iter_ggufs(dir_path: Path) -> list[Path]:
|
||||
if dir_path == dir_path.parent:
|
||||
return []
|
||||
root = str(dir_path)
|
||||
found: list[Path] = []
|
||||
for current, dirs, files in os.walk(root, followlinks = False):
|
||||
rel = os.path.relpath(current, root)
|
||||
depth = 0 if rel == os.curdir else rel.count(os.sep) + 1
|
||||
if depth >= _GGUF_SCAN_MAX_DEPTH:
|
||||
dirs[:] = []
|
||||
for name in files:
|
||||
if not name.lower().endswith(".gguf") or _is_mmproj(name):
|
||||
continue
|
||||
path = Path(current) / name
|
||||
try:
|
||||
rel = path.relative_to(dir_path).as_posix()
|
||||
except ValueError:
|
||||
rel = name
|
||||
quant = _extract_quant_label(rel)
|
||||
if _is_mtp_drafter(rel) or _is_big_endian_gguf_path(rel, quant):
|
||||
continue
|
||||
found.append(path)
|
||||
return found
|
||||
|
||||
|
||||
def _variant_matches(relative_path: str, needle: str) -> bool:
|
||||
quant = _extract_quant_label(relative_path).lower()
|
||||
if quant == needle:
|
||||
return True
|
||||
if extract_quant_label(relative_path).lower() == needle:
|
||||
return True
|
||||
prefix = f"{needle}-"
|
||||
if not quant.startswith(prefix):
|
||||
return False
|
||||
suffix = quant[len(prefix) :]
|
||||
if not suffix.endswith("bpw"):
|
||||
return False
|
||||
value = suffix[:-3]
|
||||
return bool(value) and value.replace(".", "", 1).isdigit()
|
||||
|
||||
|
||||
_GGUF_SPLIT_INDEX_RE = re.compile(r"-(\d{3,})-of-\d{3,}$", re.IGNORECASE)
|
||||
|
||||
|
||||
def _is_nonfirst_gguf_split(path: Path) -> bool:
|
||||
match = _GGUF_SPLIT_INDEX_RE.search(path.stem)
|
||||
return match is not None and int(match.group(1)) != 1
|
||||
|
||||
|
||||
def _find_gguf_in_dir(dir_path: Path, gguf_variant: Optional[str]) -> Optional[Path]:
|
||||
try:
|
||||
ggufs = sorted(_iter_ggufs(dir_path))
|
||||
except OSError:
|
||||
return None
|
||||
if not ggufs:
|
||||
return None
|
||||
needle = (gguf_variant or "").strip().lower()
|
||||
if needle:
|
||||
for path in ggufs:
|
||||
try:
|
||||
relative = path.relative_to(dir_path).as_posix()
|
||||
except ValueError:
|
||||
relative = path.name
|
||||
if _variant_matches(relative, needle):
|
||||
return path
|
||||
return None
|
||||
candidates = [path for path in ggufs if not _is_nonfirst_gguf_split(path)] or ggufs
|
||||
try:
|
||||
return max(candidates, key = lambda path: path.stat().st_size)
|
||||
except OSError:
|
||||
return candidates[0]
|
||||
|
||||
|
||||
def _chat_template_from_dir(
|
||||
dir_path: Path,
|
||||
gguf_variant: Optional[str] = None,
|
||||
allow_roots: Optional[list[Path]] = None,
|
||||
) -> Optional[str]:
|
||||
def from_gguf() -> Optional[str]:
|
||||
gguf = _find_gguf_in_dir(dir_path, gguf_variant)
|
||||
if gguf is None or not _leaf_inside_allowlist(gguf, allow_roots):
|
||||
return None
|
||||
return read_gguf_chat_template(str(gguf))
|
||||
|
||||
# Sidecar tokenizer files (chat_template.jinja / tokenizer_config.json) are the
|
||||
# author's maintained template and supersede the GGUF's possibly-stale embedded
|
||||
# copy. The variant only picks the GGUF fallback, so tokenizer-first precedence
|
||||
# holds whether or not a variant is given.
|
||||
return _chat_template_from_tokenizer_dir(dir_path, allow_roots) or from_gguf()
|
||||
|
||||
|
||||
def read_default_chat_template(
|
||||
model_name: str,
|
||||
hf_token: Optional[str] = None,
|
||||
gguf_variant: Optional[str] = None,
|
||||
) -> Optional[str]:
|
||||
if not isinstance(model_name, str) or not model_name.strip():
|
||||
return None
|
||||
name = model_name.strip()
|
||||
|
||||
if is_local_path(name):
|
||||
try:
|
||||
target = Path(normalize_path(name)).expanduser()
|
||||
allow_roots = _build_browse_allowlist()
|
||||
if not _is_path_inside_allowlist(target, allow_roots):
|
||||
logger.debug("Refused chat template read outside allowed folders: %s", name)
|
||||
return None
|
||||
if name.lower().endswith(".gguf"):
|
||||
# Prefer a maintained sidecar next to the file over the GGUF's
|
||||
# embedded copy (tokenizer-first precedence, as elsewhere).
|
||||
sidecar = _chat_template_from_tokenizer_dir(target.parent, allow_roots)
|
||||
if sidecar:
|
||||
return sidecar
|
||||
return read_gguf_chat_template(str(target))
|
||||
return _chat_template_from_dir(target, gguf_variant, allow_roots)
|
||||
except Exception as exc:
|
||||
logger.debug("Could not read local chat template for %s: %s", name, exc)
|
||||
return None
|
||||
|
||||
if not _is_valid_repo_id(name):
|
||||
return None
|
||||
|
||||
resolved = resolve_cached_repo_id_case(name)
|
||||
|
||||
try:
|
||||
# Resolve within each cached revision, newest first. A revision's sidecar
|
||||
# supersedes its own embedded GGUF copy, but must not override a newer
|
||||
# revision, so precedence stays per-snapshot rather than global.
|
||||
for snapshot in iter_hf_cache_snapshots(resolved):
|
||||
template = _chat_template_from_dir(snapshot, gguf_variant)
|
||||
if template:
|
||||
return template
|
||||
except Exception as exc:
|
||||
logger.debug("Could not read cached chat template for %s: %s", resolved, exc)
|
||||
|
||||
try:
|
||||
from huggingface_hub import HfApi, hf_hub_download
|
||||
|
||||
_api = HfApi()
|
||||
|
||||
def _remote_exceeds_cap(rel: str) -> bool:
|
||||
# Best-effort: skip the download when the remote's advertised size
|
||||
# exceeds the cap, so a maliciously large sidecar is never fetched.
|
||||
try:
|
||||
infos = _api.get_paths_info(resolved, [rel], repo_type = "model", token = hf_token)
|
||||
except Exception:
|
||||
return False
|
||||
for info in infos:
|
||||
size = getattr(info, "size", None)
|
||||
if (
|
||||
getattr(info, "path", None) == rel
|
||||
and isinstance(size, int)
|
||||
and size > MAX_TEMPLATE_METADATA_BYTES
|
||||
):
|
||||
return True
|
||||
return False
|
||||
|
||||
def _download_text(rel: str) -> Optional[str]:
|
||||
if _remote_exceeds_cap(rel):
|
||||
return None
|
||||
try:
|
||||
path = hf_hub_download(resolved, rel, token = hf_token)
|
||||
return _read_bounded_text(Path(path), MAX_TEMPLATE_METADATA_BYTES)
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
for rel in _JINJA_TEMPLATE_PATHS:
|
||||
template = _download_text(rel)
|
||||
if not template or not template.strip():
|
||||
continue
|
||||
# A raw Jinja sidecar is the whole template, so it must fit the route's
|
||||
# response cap (the local path skips oversized .jinja too). Download stays
|
||||
# bounded at MAX_TEMPLATE_METADATA_BYTES so a large JSON embedding a small
|
||||
# template still extracts below, but an over-cap Jinja is dropped so the
|
||||
# search falls through to the tokenizer/processor template.
|
||||
if len(template.encode("utf-8")) > MAX_CHAT_TEMPLATE_BYTES:
|
||||
continue
|
||||
return template
|
||||
|
||||
for rel in _TOKENIZER_CONFIG_PATHS:
|
||||
raw = _download_text(rel)
|
||||
if not raw:
|
||||
continue
|
||||
try:
|
||||
config = json.loads(raw)
|
||||
except Exception:
|
||||
continue
|
||||
template = _chat_template_from_tokenizer_config(config)
|
||||
if template:
|
||||
return template
|
||||
|
||||
for rel in _PROCESSOR_TEMPLATE_PATHS:
|
||||
raw = _download_text(rel)
|
||||
if not raw:
|
||||
continue
|
||||
try:
|
||||
payload = json.loads(raw)
|
||||
except Exception:
|
||||
continue
|
||||
template = _chat_template_from_processor_payload(payload)
|
||||
if template:
|
||||
return template
|
||||
|
||||
return None
|
||||
except Exception as exc:
|
||||
logger.debug("Could not fetch chat template for %s: %s", resolved, exc)
|
||||
return None
|
||||
|
|
@ -5144,10 +5144,10 @@ async def validate_model(
|
|||
latest_tier_active_for, config.identifier, request.hf_token
|
||||
):
|
||||
effective_load_in_4bit = False
|
||||
# A metadata-only probe just reads the GGUF header and allocates no VRAM,
|
||||
# so it must not be refused by the training guard. Real loads validate
|
||||
# without include_context_length and /load applies the guard again.
|
||||
if not request.include_context_length:
|
||||
# A metadata-only probe reads the GGUF header and allocates no VRAM, so the
|
||||
# training guard must not refuse it. Real loads omit include_context_length /
|
||||
# include_chat_template, and /load applies the guard again.
|
||||
if not (request.include_context_length or request.include_chat_template):
|
||||
# Match /load's inherited llama.cpp extras and parallel slot count so
|
||||
# validation cannot pass a smaller estimate than the subsequent load.
|
||||
effective_extra_args = _resolve_inherited_extra_args(
|
||||
|
|
@ -5189,9 +5189,15 @@ async def validate_model(
|
|||
context_length: Optional[int] = None
|
||||
layer_count: Optional[int] = None
|
||||
moe_layer_count: Optional[int] = None
|
||||
if request.include_context_length and is_gguf:
|
||||
chat_template: Optional[str] = None
|
||||
# Both header probes read the same local GGUF, so resolve it once.
|
||||
if (request.include_context_length or request.include_chat_template) and is_gguf:
|
||||
from hub.utils.gguf import resolve_local_gguf_path
|
||||
from utils.models.gguf_metadata import read_gguf_staged_dims
|
||||
from picker.schemas import MAX_CHAT_TEMPLATE_BYTES
|
||||
from utils.models.gguf_metadata import (
|
||||
read_gguf_chat_template,
|
||||
read_gguf_staged_dims,
|
||||
)
|
||||
|
||||
# Best-effort: a header-read failure must never fail validation of an
|
||||
# otherwise-valid model (the outer except turns it into a 400).
|
||||
|
|
@ -5207,13 +5213,24 @@ async def validate_model(
|
|||
model_identifier, request.gguf_variant
|
||||
)
|
||||
if local_gguf:
|
||||
# Header walk reads tokenizer arrays for dense models (tens of
|
||||
# ms); keep it off the event loop.
|
||||
dims = await asyncio.to_thread(read_gguf_staged_dims, local_gguf)
|
||||
if dims:
|
||||
context_length = dims["context_length"]
|
||||
layer_count = dims["layer_count"]
|
||||
moe_layer_count = dims["moe_layer_count"]
|
||||
if request.include_context_length:
|
||||
# Header walk reads tokenizer arrays (tens of ms); keep it
|
||||
# off the event loop.
|
||||
dims = await asyncio.to_thread(read_gguf_staged_dims, local_gguf)
|
||||
if dims:
|
||||
context_length = dims["context_length"]
|
||||
layer_count = dims["layer_count"]
|
||||
moe_layer_count = dims["moe_layer_count"]
|
||||
if request.include_chat_template:
|
||||
# Read only the leased GGUF's own embedded template (the copy
|
||||
# llama.cpp loads), never a sibling sidecar: the native grant
|
||||
# authorizes just this path, so neighbours would be scope escalation.
|
||||
raw_template = await asyncio.to_thread(read_gguf_chat_template, local_gguf)
|
||||
if (
|
||||
raw_template is not None
|
||||
and len(raw_template.encode("utf-8")) <= MAX_CHAT_TEMPLATE_BYTES
|
||||
):
|
||||
chat_template = raw_template
|
||||
except Exception as e:
|
||||
logger.debug("Header probe failed for %s: %s", model_log_label, e)
|
||||
|
||||
|
|
@ -5232,6 +5249,7 @@ async def validate_model(
|
|||
context_length = context_length,
|
||||
layer_count = layer_count,
|
||||
moe_layer_count = moe_layer_count,
|
||||
chat_template = chat_template,
|
||||
requires_transformers_upgrade = transformers_upgrade is not None,
|
||||
transformers_upgrade = transformers_upgrade,
|
||||
)
|
||||
|
|
|
|||
|
|
@ -60,13 +60,52 @@ def _safe_is_dir(path) -> bool:
|
|||
|
||||
|
||||
# Shared with the hub inventory scans; keep the private aliases so existing
|
||||
# importers (core.inference.local_model_resolver, tests) stay valid.
|
||||
# importers stay valid. ``_HF_REPO_ID_RE`` is the Hub repo id shape ("owner/name");
|
||||
# anything else is treated as a local filesystem path.
|
||||
from utils.hidden_models import (
|
||||
_HF_REPO_ID_RE,
|
||||
_existing_resolved_path,
|
||||
_safe_resolve,
|
||||
is_hidden_model as _is_hidden_model,
|
||||
)
|
||||
|
||||
|
||||
def hidden_model_matchers() -> tuple[list[str], list[str], list[str]]:
|
||||
"""Substring needles, exact repo ids, and exact resolved paths identifying
|
||||
infra models (the RAG embedder and the llama.cpp install validation probe)
|
||||
that pickers hide. Served by the ``/api/hub/hidden-models`` endpoint. A
|
||||
configured HF-repo embedder is published as its exact lowercased repo id
|
||||
(mirroring ``utils.hidden_models.is_hidden_model``) and a local-path
|
||||
embedder as its exact resolved path only: a generic basename like "model"
|
||||
must not substring-hide unrelated chat models."""
|
||||
from core.rag import config as rag_config
|
||||
|
||||
needles = [
|
||||
# The validation probe's repo and its exact filename. The filename carries
|
||||
# .gguf so it won't hide unrelated repos like ``user/stories260K-finetune-GGUF``.
|
||||
"ggml-org/models",
|
||||
"stories260k.gguf",
|
||||
]
|
||||
exact_ids: list[str] = []
|
||||
exact_paths: list[str] = []
|
||||
for model in (
|
||||
rag_config.effective_embedding_model(),
|
||||
rag_config.effective_gguf_repo(),
|
||||
):
|
||||
# Resolve an existing local path before the repo-id regex: a local embedder
|
||||
# shaped like "models/embedder" is an exact path, not a Hub repo id.
|
||||
existing_path = _existing_resolved_path(model)
|
||||
if existing_path:
|
||||
exact_paths.append(existing_path.lower())
|
||||
elif _HF_REPO_ID_RE.match(model):
|
||||
exact_ids.append(model.lower())
|
||||
else:
|
||||
resolved = _safe_resolve(Path(model).expanduser())
|
||||
if resolved:
|
||||
exact_paths.append(resolved.lower())
|
||||
return needles, exact_ids, exact_paths
|
||||
|
||||
|
||||
backend_path = Path(__file__).parent.parent.parent
|
||||
if str(backend_path) not in sys.path:
|
||||
sys.path.insert(0, str(backend_path))
|
||||
|
|
@ -91,6 +130,7 @@ try:
|
|||
_pick_best_gguf,
|
||||
_extract_quant_label,
|
||||
_is_big_endian_gguf_path,
|
||||
_is_mtp_drafter,
|
||||
is_audio_input_type,
|
||||
)
|
||||
from core.inference import get_inference_backend
|
||||
|
|
@ -123,6 +163,7 @@ except ImportError:
|
|||
_pick_best_gguf,
|
||||
_extract_quant_label,
|
||||
_is_big_endian_gguf_path,
|
||||
_is_mtp_drafter,
|
||||
is_audio_input_type,
|
||||
)
|
||||
from core.inference import get_inference_backend
|
||||
|
|
@ -803,7 +844,7 @@ def collect_local_models(models_root: Path) -> List[LocalModelInfo]:
|
|||
|
||||
models = sorted(
|
||||
deduped.values(),
|
||||
key = lambda item: (item.updated_at or 0),
|
||||
key = lambda item: item.updated_at or 0,
|
||||
reverse = True,
|
||||
)
|
||||
return [m for m in models if not _is_hidden_model(m.id, m.model_id, m.path)]
|
||||
|
|
@ -1750,9 +1791,11 @@ def _get_model_size_bytes(model_name: str, hf_token: Optional[str] = None) -> Op
|
|||
async def get_model_config(
|
||||
model_name: str,
|
||||
hf_token: Optional[str] = Query(None),
|
||||
header_hf_token: Optional[str] = Depends(get_hf_token),
|
||||
current_subject: str = Depends(get_current_subject),
|
||||
):
|
||||
"""Get configuration for a specific model (wraps load_model_defaults)."""
|
||||
hf_token = _normalize_hf_token(header_hf_token) or _normalize_hf_token(hf_token)
|
||||
try:
|
||||
if not is_local_path(model_name):
|
||||
resolved = resolve_cached_repo_id_case(model_name)
|
||||
|
|
@ -2471,6 +2514,7 @@ async def get_lora_base_model(lora_path: str, current_subject: str = Depends(get
|
|||
async def check_vision_model(
|
||||
model_name: str,
|
||||
hf_token: Optional[str] = Query(None),
|
||||
header_hf_token: Optional[str] = Depends(get_hf_token),
|
||||
current_subject: str = Depends(get_current_subject),
|
||||
):
|
||||
"""
|
||||
|
|
@ -2478,6 +2522,7 @@ async def check_vision_model(
|
|||
|
||||
This endpoint wraps the backend is_vision_model function.
|
||||
"""
|
||||
hf_token = _normalize_hf_token(header_hf_token) or _normalize_hf_token(hf_token)
|
||||
try:
|
||||
logger.info(f"Checking if vision model: {model_name}")
|
||||
# Authenticate so a gated/private VLM classifies correctly (else 404 -> non-vision).
|
||||
|
|
@ -2503,6 +2548,7 @@ async def check_vision_model(
|
|||
async def check_embedding_model(
|
||||
model_name: str,
|
||||
hf_token: Optional[str] = Query(None),
|
||||
header_hf_token: Optional[str] = Depends(get_hf_token),
|
||||
current_subject: str = Depends(get_current_subject),
|
||||
):
|
||||
"""
|
||||
|
|
@ -2510,6 +2556,7 @@ async def check_embedding_model(
|
|||
|
||||
This endpoint wraps the backend is_embedding_model function.
|
||||
"""
|
||||
hf_token = _normalize_hf_token(header_hf_token) or _normalize_hf_token(hf_token)
|
||||
try:
|
||||
logger.info(f"Checking if embedding model: {model_name}")
|
||||
is_embedding = is_embedding_model(model_name, hf_token = hf_token)
|
||||
|
|
@ -2573,12 +2620,6 @@ def _resolve_quant_gguf(repo_id: str, quant: str, is_local: bool) -> tuple[Optio
|
|||
Q8_0 weights). Never raises.
|
||||
"""
|
||||
try:
|
||||
from utils.models.model_config import (
|
||||
_extract_quant_label,
|
||||
_is_big_endian_gguf_path,
|
||||
_is_mtp_drafter,
|
||||
)
|
||||
|
||||
if is_local:
|
||||
roots = [Path(repo_id)]
|
||||
else:
|
||||
|
|
@ -2595,25 +2636,19 @@ def _resolve_quant_gguf(repo_id: str, quant: str, is_local: bool) -> tuple[Optio
|
|||
if snaps.is_dir():
|
||||
roots.extend(s for s in snaps.iterdir() if s.is_dir())
|
||||
|
||||
want = quant.lower().replace("-", "").replace("_", "")
|
||||
want = _normalized_quant_label(quant)
|
||||
best_total = 0
|
||||
best_first: Optional[str] = None
|
||||
for root in roots:
|
||||
matches: list[tuple[str, Path]] = []
|
||||
total = 0
|
||||
for f in _iter_gguf_paths(root):
|
||||
if _is_mmproj_filename(f.name):
|
||||
continue
|
||||
try:
|
||||
rel = f.relative_to(root).as_posix()
|
||||
except ValueError:
|
||||
rel = f.name
|
||||
if _is_mtp_drafter(rel):
|
||||
continue
|
||||
q = _extract_quant_label(rel)
|
||||
if _is_big_endian_gguf_path(rel, q):
|
||||
continue
|
||||
if q.lower().replace("-", "").replace("_", "") != want:
|
||||
q = _main_variant_gguf_label(rel)
|
||||
if q is None or _normalized_quant_label(q) != want:
|
||||
continue
|
||||
try:
|
||||
total += f.stat().st_size
|
||||
|
|
@ -3035,6 +3070,22 @@ def _is_main_gguf_filename(name: str) -> bool:
|
|||
return _is_gguf_filename(name) and not _is_mmproj_filename(name)
|
||||
|
||||
|
||||
def _main_variant_gguf_label(rel_path: str) -> Optional[str]:
|
||||
name = rel_path.rsplit("/", 1)[-1]
|
||||
if not _is_main_gguf_filename(name):
|
||||
return None
|
||||
if _is_mtp_drafter(rel_path):
|
||||
return None
|
||||
label = _extract_quant_label(rel_path)
|
||||
if _is_big_endian_gguf_path(rel_path, label):
|
||||
return None
|
||||
return label
|
||||
|
||||
|
||||
def _normalized_quant_label(label: str) -> str:
|
||||
return label.lower().replace("-", "").replace("_", "")
|
||||
|
||||
|
||||
def _repo_has_mmproj(repo_info) -> bool:
|
||||
"""True if the repo ships a GGUF vision adapter (mmproj), so it can
|
||||
take image inputs. Cheap: scans already-listed file names only."""
|
||||
|
|
@ -3362,6 +3413,170 @@ async def delete_cached_model(
|
|||
)
|
||||
|
||||
|
||||
def _resolve_cached_model_path(repo_id: str, variant: Optional[str]) -> Path:
|
||||
"""Absolute path of a cached repo (newest snapshot dir) or, with *variant*,
|
||||
that quant's main GGUF file (first split of a sharded quant). Paths come
|
||||
from the HF cache scan only, so callers can't probe arbitrary paths."""
|
||||
cache_scans = _all_hf_cache_scans()
|
||||
|
||||
matching_repos = []
|
||||
for hf_cache in cache_scans:
|
||||
for repo_info in hf_cache.repos:
|
||||
if repo_info.repo_type != "model":
|
||||
continue
|
||||
if repo_info.repo_id.lower() == repo_id.lower():
|
||||
matching_repos.append(repo_info)
|
||||
if not matching_repos:
|
||||
raise HTTPException(status_code = 404, detail = "Model not found in cache")
|
||||
|
||||
if variant:
|
||||
want = _normalized_quant_label(variant)
|
||||
candidate_revisions = sorted(
|
||||
(rev for repo_info in matching_repos for rev in repo_info.revisions),
|
||||
key = lambda rev: getattr(rev, "last_modified", 0) or 0,
|
||||
reverse = True,
|
||||
)
|
||||
for rev in candidate_revisions:
|
||||
snapshot = getattr(rev, "snapshot_path", None)
|
||||
matches = []
|
||||
for f in rev.files:
|
||||
p = Path(f.file_path)
|
||||
rel = f.file_name
|
||||
if snapshot:
|
||||
try:
|
||||
rel = p.relative_to(snapshot).as_posix()
|
||||
except ValueError:
|
||||
pass
|
||||
label = _main_variant_gguf_label(rel)
|
||||
if label is None or _normalized_quant_label(label) != want:
|
||||
continue
|
||||
if p.exists() or p.is_symlink():
|
||||
matches.append((rel, p))
|
||||
if matches:
|
||||
# Path-sorted so a sharded quant deterministically yields its first split.
|
||||
return sorted(matches, key = lambda m: m[0].lower())[0][1]
|
||||
raise HTTPException(
|
||||
status_code = 404,
|
||||
detail = f"Variant {variant} not found in cache for {repo_id}",
|
||||
)
|
||||
|
||||
def repo_size(repo_info) -> int:
|
||||
gguf_size = _repo_gguf_size_bytes(repo_info)
|
||||
if gguf_size > 0:
|
||||
return gguf_size
|
||||
return sum(
|
||||
(getattr(f, "size_on_disk", None) or 0)
|
||||
for rev in repo_info.revisions
|
||||
for f in rev.files
|
||||
)
|
||||
|
||||
def repo_last_modified(repo_info) -> float:
|
||||
return max(
|
||||
(getattr(rev, "last_modified", 0) or 0 for rev in repo_info.revisions),
|
||||
default = 0,
|
||||
)
|
||||
|
||||
target_repo = max(
|
||||
matching_repos,
|
||||
key = lambda repo_info: (repo_size(repo_info), repo_last_modified(repo_info)),
|
||||
)
|
||||
|
||||
# Whole repo: the newest revision's snapshot dir holds the visible files.
|
||||
revisions = sorted(
|
||||
(rev for rev in target_repo.revisions if getattr(rev, "snapshot_path", None)),
|
||||
key = lambda rev: getattr(rev, "last_modified", 0) or 0,
|
||||
reverse = True,
|
||||
)
|
||||
for rev in revisions:
|
||||
p = Path(rev.snapshot_path)
|
||||
if p.exists():
|
||||
return p
|
||||
p = Path(target_repo.repo_path)
|
||||
if p.exists():
|
||||
return p
|
||||
raise HTTPException(status_code = 404, detail = "Cached model path not found")
|
||||
|
||||
|
||||
def _wsl_reveal_in_explorer(path: Path) -> bool:
|
||||
import subprocess
|
||||
|
||||
from utils.paths.path_utils import _IS_WSL
|
||||
|
||||
if not _IS_WSL:
|
||||
return False
|
||||
try:
|
||||
windows_path = subprocess.run(
|
||||
["wslpath", "-w", str(path)],
|
||||
capture_output = True,
|
||||
text = True,
|
||||
check = True,
|
||||
timeout = 10,
|
||||
).stdout.strip()
|
||||
if not windows_path:
|
||||
return False
|
||||
argument = f"/select,{windows_path}" if path.is_file() else windows_path
|
||||
subprocess.Popen(["explorer.exe", argument])
|
||||
return True
|
||||
except (OSError, subprocess.SubprocessError):
|
||||
return False
|
||||
|
||||
|
||||
def _reveal_in_file_manager(path: Path) -> None:
|
||||
"""Open the OS file manager with *path* selected (best effort per platform)."""
|
||||
import subprocess
|
||||
|
||||
target = str(path)
|
||||
if sys.platform == "darwin":
|
||||
cmd = ["open", "-R", target] if path.is_file() else ["open", target]
|
||||
subprocess.Popen(cmd)
|
||||
elif os.name == "nt":
|
||||
if path.is_file():
|
||||
subprocess.Popen(["explorer", f"/select,{target}"])
|
||||
else:
|
||||
os.startfile(target) # noqa: S606 - local user's own file manager
|
||||
elif not _wsl_reveal_in_explorer(path):
|
||||
# No cross-desktop "select file" standard on Linux; open the directory.
|
||||
directory = target if path.is_dir() else str(path.parent)
|
||||
subprocess.Popen(["xdg-open", directory])
|
||||
|
||||
|
||||
class CachedModelPathResponse(BaseModel):
|
||||
path: str
|
||||
is_dir: bool
|
||||
|
||||
|
||||
@router.get("/cached-model-path", response_model = CachedModelPathResponse)
|
||||
async def get_cached_model_path(
|
||||
repo_id: str = Query(..., description = "HuggingFace repo ID"),
|
||||
variant: str = Query("", description = "Quantization variant (empty for whole repo)"),
|
||||
current_subject: str = Depends(get_current_subject),
|
||||
):
|
||||
"""Absolute on-disk path of a cached repo or one of its GGUF variants."""
|
||||
if not _is_valid_repo_id(repo_id):
|
||||
raise HTTPException(status_code = 400, detail = "Invalid repo_id format")
|
||||
path = await asyncio.to_thread(_resolve_cached_model_path, repo_id, variant.strip() or None)
|
||||
return {"path": str(path), "is_dir": path.is_dir()}
|
||||
|
||||
|
||||
@router.post("/reveal-cached-model")
|
||||
async def reveal_cached_model(
|
||||
repo_id: str = Body(...),
|
||||
variant: Optional[str] = Body(None),
|
||||
current_subject: str = Depends(get_current_subject),
|
||||
):
|
||||
"""Reveal a cached repo (or one GGUF variant's file) in the OS file manager."""
|
||||
if not _is_valid_repo_id(repo_id):
|
||||
raise HTTPException(status_code = 400, detail = "Invalid repo_id format")
|
||||
variant = (variant or "").strip() or None
|
||||
path = await asyncio.to_thread(_resolve_cached_model_path, repo_id, variant)
|
||||
try:
|
||||
await asyncio.to_thread(_reveal_in_file_manager, path)
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to reveal {path}: {e}")
|
||||
raise HTTPException(status_code = 500, detail = "Failed to open file manager")
|
||||
return {"status": "ok", "path": str(path)}
|
||||
|
||||
|
||||
@router.get("/checkpoints", response_model = CheckpointListResponse)
|
||||
async def list_checkpoints(
|
||||
outputs_dir: str = Query(
|
||||
|
|
|
|||
|
|
@ -801,6 +801,80 @@ class TestValidateRefusesDuringTraining(unittest.TestCase):
|
|||
asyncio.run(self.route.validate_model(request, current_subject = "u"))
|
||||
self.assertEqual(guard_called, [])
|
||||
|
||||
def _validate_gguf_template(
|
||||
self,
|
||||
*,
|
||||
template,
|
||||
canonical_path = "/picked/model.gguf",
|
||||
):
|
||||
# Drive validate_model for a native lease-backed GGUF template probe and
|
||||
# capture what the embedded-template reader was called with.
|
||||
from models.inference import ValidateModelRequest
|
||||
|
||||
request = ValidateModelRequest(
|
||||
model_path = "model.gguf",
|
||||
gguf_variant = "Q4_K_M",
|
||||
native_path_lease = "signed-lease",
|
||||
include_chat_template = True,
|
||||
)
|
||||
cfg = SimpleNamespace(
|
||||
identifier = canonical_path,
|
||||
display_name = "model.gguf",
|
||||
is_gguf = True,
|
||||
is_lora = False,
|
||||
is_vision = False,
|
||||
gguf_file = canonical_path,
|
||||
path = None,
|
||||
base_model = None,
|
||||
)
|
||||
import utils.models.gguf_metadata as gguf_meta
|
||||
|
||||
seen = {}
|
||||
|
||||
def _fake_read(path):
|
||||
seen["path"] = path
|
||||
return template
|
||||
|
||||
guard_called = []
|
||||
with (
|
||||
patch.object(
|
||||
self.route,
|
||||
"_resolve_model_identifier_for_request",
|
||||
return_value = (canonical_path, "model.gguf", True),
|
||||
),
|
||||
patch.object(self.route.ModelConfig, "from_identifier", return_value = cfg),
|
||||
patch.object(self.route, "load_inference_config", return_value = {}),
|
||||
patch.object(gguf_meta, "read_gguf_chat_template", _fake_read),
|
||||
patch.object(
|
||||
self.route,
|
||||
"_guard_chat_load_against_training",
|
||||
lambda *a, **kw: guard_called.append(True),
|
||||
),
|
||||
):
|
||||
resp = asyncio.run(self.route.validate_model(request, current_subject = "u"))
|
||||
return resp, seen, guard_called
|
||||
|
||||
def test_include_chat_template_reads_leased_gguf_embedded_template(self):
|
||||
# The picker chat-template GET has no lease plumbing, so a native picked
|
||||
# GGUF surfaces its default template through this lease-aware probe: the
|
||||
# embedded template is read from the granted canonical path and returned.
|
||||
resp, seen, _ = self._validate_gguf_template(template = "{{ messages }}")
|
||||
self.assertEqual(resp.chat_template, "{{ messages }}")
|
||||
# Read strictly the leased file's own embedded template, never a sibling
|
||||
# sidecar: the grant authorizes just this one path.
|
||||
self.assertEqual(seen["path"], "/picked/model.gguf")
|
||||
|
||||
def test_include_chat_template_skips_training_guard(self):
|
||||
# A template-only probe allocates no VRAM, so like include_context_length
|
||||
# it must not be refused by the training guard.
|
||||
_, _, guard_called = self._validate_gguf_template(template = "{{ messages }}")
|
||||
self.assertEqual(guard_called, [])
|
||||
|
||||
def test_include_chat_template_over_cap_is_dropped(self):
|
||||
from picker.schemas import MAX_CHAT_TEMPLATE_BYTES
|
||||
resp, _, _ = self._validate_gguf_template(template = "a" * (MAX_CHAT_TEMPLATE_BYTES + 1))
|
||||
self.assertIsNone(resp.chat_template)
|
||||
|
||||
|
||||
# ── _estimate_gguf_required_gb (sizes the same weights the loader loads) ──────
|
||||
|
||||
|
|
|
|||
|
|
@ -158,6 +158,7 @@ def _install_lightweight_backend_stubs(monkeypatch):
|
|||
utils_model_config._pick_best_gguf = lambda variants: variants[0] if variants else None
|
||||
utils_model_config._extract_quant_label = lambda value: value
|
||||
utils_model_config._is_big_endian_gguf_path = lambda *args, **kwargs: False
|
||||
utils_model_config._is_mtp_drafter = lambda *args, **kwargs: False
|
||||
utils_model_config.is_audio_input_type = lambda *args, **kwargs: None
|
||||
monkeypatch.setitem(
|
||||
sys.modules,
|
||||
|
|
|
|||
232
studio/backend/tests/test_model_picker_regression.py
Normal file
232
studio/backend/tests/test_model_picker_regression.py
Normal file
|
|
@ -0,0 +1,232 @@
|
|||
# SPDX-License-Identifier: AGPL-3.0-only
|
||||
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
"""Regression guards for the model-picker per-model-config feature (the set of
|
||||
bugs that got the predecessor PR reverted). Pure-function / validation checks
|
||||
only, so they run on CPU in the backend pytest job with no model download.
|
||||
|
||||
Covers, at the backend layer:
|
||||
- infra-model hiding: the RAG embedder (bge-small-en-v1.5) and the llama.cpp
|
||||
install-validation probe (ggml-org/models / stories260K) stay hidden, while
|
||||
normal chat repos are not hidden;
|
||||
- the HF token is honored from the dedicated header with the query string as a
|
||||
fallback, never the other way around;
|
||||
- the chat-template byte caps reject oversized overrides (both the char-count
|
||||
fast path and the UTF-8 byte path) and the sidecar reader is size-bounded.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
import types
|
||||
|
||||
import pytest
|
||||
|
||||
# Keep this test runnable without the optional structlog dependency (mirrors
|
||||
# tests/test_cached_gguf_routes.py), since importing routes.models pulls it in.
|
||||
if "structlog" not in sys.modules:
|
||||
|
||||
class _DummyLogger:
|
||||
def __getattr__(self, _name):
|
||||
return lambda *args, **kwargs: None
|
||||
|
||||
sys.modules["structlog"] = types.SimpleNamespace(
|
||||
BoundLogger = _DummyLogger,
|
||||
get_logger = lambda *args, **kwargs: _DummyLogger(),
|
||||
)
|
||||
|
||||
import routes.models as models_route
|
||||
from core.rag import config as rag_config
|
||||
from hub.dependencies import get_hf_token
|
||||
from models.inference import LoadRequest
|
||||
from picker.schemas import MAX_CHAT_TEMPLATE_BYTES
|
||||
from picker.service import _read_bounded_text
|
||||
from utils.hidden_models import is_hidden_model
|
||||
|
||||
|
||||
@pytest.fixture(autouse = True)
|
||||
def _pin_default_embedder(monkeypatch):
|
||||
"""Pin the effective embedder to Studio's static default so hiding is
|
||||
deterministic and cannot depend on ambient RAG config / env."""
|
||||
default = "unsloth/bge-small-en-v1.5"
|
||||
monkeypatch.setattr(rag_config, "EMBEDDING_MODEL", default, raising = False)
|
||||
monkeypatch.setattr(rag_config, "effective_embedding_model", lambda: default)
|
||||
monkeypatch.setattr(rag_config, "effective_gguf_repo", lambda: default)
|
||||
monkeypatch.setattr(rag_config, "default_gguf_repo", lambda: default)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Infra-model hiding (the "infra models resurfaced in the picker" regression) #
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"value",
|
||||
[
|
||||
"ggml-org/models", # the probe repo id
|
||||
"unsloth/bge-small-en-v1.5", # the RAG embedder repo
|
||||
"unsloth/bge-small-en-v1.5-GGUF", # its GGUF companion
|
||||
"/root/.cache/huggingface/hub/x/stories260K.gguf", # probe on disk
|
||||
"/root/.cache/x/Stories260K.GGUF", # case-insensitive
|
||||
r"C:\\models\\stories260K.gguf", # windows-style path
|
||||
"/opt/models/bge-small-en-v1.5", # embedder basename folder
|
||||
"/opt/models/bge-small-en-v1.5-Q8_0.gguf", # suffixed local weight
|
||||
],
|
||||
)
|
||||
def test_infra_models_are_hidden(value):
|
||||
assert is_hidden_model(value) is True
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"value",
|
||||
[
|
||||
"unsloth/gemma-3-270m-it-GGUF", # a normal small chat GGUF
|
||||
"unsloth/Qwen3-0.6B", # a normal non-GGUF chat model
|
||||
"user/stories260K-finetune-GGUF", # repo id merely contains "stories260k"
|
||||
"user/model-chat", # generic repo must not be hidden
|
||||
"meta-llama/Llama-3.1-8B-Instruct",
|
||||
],
|
||||
)
|
||||
def test_normal_models_are_not_hidden(value):
|
||||
assert is_hidden_model(value) is False
|
||||
|
||||
|
||||
def test_is_hidden_model_ignores_empty_values():
|
||||
assert is_hidden_model(None) is False
|
||||
assert is_hidden_model("") is False
|
||||
assert is_hidden_model(None, "", "unsloth/gemma-3-270m-it-GGUF") is False
|
||||
|
||||
|
||||
def test_hidden_model_matchers_expose_probe_needles():
|
||||
needles, exact_ids, _exact_paths = models_route.hidden_model_matchers()
|
||||
lowered = [n.lower() for n in needles]
|
||||
assert "ggml-org/models" in lowered
|
||||
assert "stories260k.gguf" in lowered
|
||||
# The configured embedder is exposed as an exact repo id, never as a
|
||||
# basename needle that would substring-hide unrelated chat models.
|
||||
assert "bge-small-en-v1.5" not in lowered
|
||||
assert "unsloth/bge-small-en-v1.5" in exact_ids
|
||||
|
||||
|
||||
def test_hidden_model_matchers_custom_repo_publishes_exact_ids(monkeypatch):
|
||||
monkeypatch.setattr(rag_config, "effective_embedding_model", lambda: "org/model")
|
||||
monkeypatch.setattr(rag_config, "effective_gguf_repo", lambda: "org/model-GGUF")
|
||||
needles, exact_ids, exact_paths = models_route.hidden_model_matchers()
|
||||
assert needles == ["ggml-org/models", "stories260k.gguf"]
|
||||
assert "org/model" in exact_ids
|
||||
assert "org/model-gguf" in exact_ids
|
||||
assert exact_paths == []
|
||||
|
||||
|
||||
def test_hidden_model_matchers_local_owner_name_path_is_exact_path(monkeypatch, tmp_path):
|
||||
# A local embedder shaped like owner/name that exists on disk must be an
|
||||
# exact resolved path, not a Hub repo id (mirroring is_hidden_model), so the
|
||||
# local row stays hidden instead of showing as a chat model.
|
||||
(tmp_path / "models" / "embedder").mkdir(parents = True)
|
||||
monkeypatch.chdir(tmp_path)
|
||||
monkeypatch.setattr(rag_config, "effective_embedding_model", lambda: "models/embedder")
|
||||
monkeypatch.setattr(rag_config, "effective_gguf_repo", lambda: "ggml-org/models")
|
||||
_needles, exact_ids, exact_paths = models_route.hidden_model_matchers()
|
||||
resolved = str((tmp_path / "models" / "embedder").resolve()).lower()
|
||||
assert resolved in exact_paths
|
||||
assert "models/embedder" not in exact_ids
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# HF token via header, query string only as a fallback (the token-leak fix) #
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
|
||||
def test_get_hf_token_strips_and_returns():
|
||||
assert get_hf_token(" hf_abc ") == "hf_abc"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("value", [None, "", " ", "\n\t"])
|
||||
def test_get_hf_token_blank_is_none(value):
|
||||
assert get_hf_token(value) is None
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"value,expected",
|
||||
[(" hf_x ", "hf_x"), ("", None), (" ", None), (None, None), (1234, None)],
|
||||
)
|
||||
def test_normalize_hf_token(value, expected):
|
||||
assert models_route._normalize_hf_token(value) == expected
|
||||
|
||||
|
||||
def test_header_token_wins_over_query():
|
||||
header, query = "hf_header", "hf_query"
|
||||
resolved = models_route._normalize_hf_token(header) or models_route._normalize_hf_token(query)
|
||||
assert resolved == "hf_header"
|
||||
|
||||
|
||||
def test_query_token_is_fallback_when_header_absent():
|
||||
resolved = models_route._normalize_hf_token(None) or models_route._normalize_hf_token(
|
||||
"hf_query"
|
||||
)
|
||||
assert resolved == "hf_query"
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Chat-template byte caps (the unbounded-template hardening) #
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
|
||||
def _load_request(**overrides):
|
||||
data = {"model_path": "unsloth/test-model-GGUF", "gguf_variant": "Q4_K_M"}
|
||||
data.update(overrides)
|
||||
return LoadRequest.model_validate(data)
|
||||
|
||||
|
||||
def test_blank_chat_template_override_normalizes_to_none():
|
||||
assert _load_request(chat_template_override = " \n\t").chat_template_override is None
|
||||
|
||||
|
||||
def test_nonblank_chat_template_override_preserved_verbatim():
|
||||
template = " {{ messages }} "
|
||||
assert _load_request(chat_template_override = template).chat_template_override == template
|
||||
|
||||
|
||||
def test_chat_template_at_byte_limit_is_accepted():
|
||||
template = "a" * MAX_CHAT_TEMPLATE_BYTES # exactly the limit, 1 byte/char
|
||||
assert (
|
||||
len(_load_request(chat_template_override = template).chat_template_override)
|
||||
== MAX_CHAT_TEMPLATE_BYTES
|
||||
)
|
||||
|
||||
|
||||
def test_chat_template_over_char_limit_is_rejected():
|
||||
with pytest.raises(Exception): # pydantic ValidationError wrapping ValueError
|
||||
_load_request(chat_template_override = "a" * (MAX_CHAT_TEMPLATE_BYTES + 1))
|
||||
|
||||
|
||||
def test_chat_template_over_byte_limit_is_rejected():
|
||||
# Char count stays under the limit but UTF-8 bytes exceed it (3 bytes/char),
|
||||
# so only the byte-count branch can catch this.
|
||||
multibyte = "€" * (MAX_CHAT_TEMPLATE_BYTES // 2) # euro sign, 3 bytes each
|
||||
assert len(multibyte) <= MAX_CHAT_TEMPLATE_BYTES
|
||||
assert len(multibyte.encode("utf-8")) > MAX_CHAT_TEMPLATE_BYTES
|
||||
with pytest.raises(Exception):
|
||||
_load_request(chat_template_override = multibyte)
|
||||
|
||||
|
||||
def test_read_bounded_text_reads_within_limit(tmp_path):
|
||||
p = tmp_path / "t.json"
|
||||
p.write_text("hello", encoding = "utf-8")
|
||||
assert _read_bounded_text(p, 16) == "hello"
|
||||
|
||||
|
||||
def test_read_bounded_text_rejects_over_limit(tmp_path):
|
||||
p = tmp_path / "big.json"
|
||||
p.write_bytes(b"x" * 100)
|
||||
assert _read_bounded_text(p, 50) is None
|
||||
|
||||
|
||||
def test_read_bounded_text_at_limit_is_read(tmp_path):
|
||||
p = tmp_path / "exact.json"
|
||||
p.write_bytes(b"x" * 50)
|
||||
assert _read_bounded_text(p, 50) == "x" * 50
|
||||
|
||||
|
||||
def test_read_bounded_text_missing_file_is_none(tmp_path):
|
||||
assert _read_bounded_text(tmp_path / "nope.json", 50) is None
|
||||
|
|
@ -314,6 +314,7 @@ def test_cached_model_scan_keeps_local_safetensors_repo(monkeypatch, tmp_path):
|
|||
file_name = "model.safetensors",
|
||||
size_on_disk = 100,
|
||||
blob_path = str(repo_path / "blobs" / "modelsha"),
|
||||
blob_last_modified = 3_000.0,
|
||||
),
|
||||
]
|
||||
)
|
||||
|
|
@ -336,6 +337,51 @@ def test_cached_model_scan_keeps_local_safetensors_repo(monkeypatch, tmp_path):
|
|||
assert rows[0]["repo_id"] == "Org/SafeTensorRepo"
|
||||
assert rows[0]["model_format"] == "safetensors"
|
||||
assert rows[0]["size_bytes"] == 100
|
||||
assert rows[0]["last_modified"] == 3_000.0
|
||||
|
||||
|
||||
def test_cached_gguf_scan_keeps_download_timestamp(monkeypatch, tmp_path):
|
||||
repo_path = tmp_path / "models--Org--GgufRepo"
|
||||
repo = SimpleNamespace(
|
||||
repo_id = "Org/GgufRepo",
|
||||
repo_type = "model",
|
||||
repo_path = repo_path,
|
||||
revisions = [
|
||||
SimpleNamespace(
|
||||
files = [
|
||||
SimpleNamespace(
|
||||
file_name = "model-Q4_K_M.gguf",
|
||||
size_on_disk = 100,
|
||||
blob_path = None,
|
||||
blob_last_modified = 5_000.0,
|
||||
),
|
||||
]
|
||||
)
|
||||
],
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
CI,
|
||||
"all_hf_cache_scans",
|
||||
lambda: [SimpleNamespace(repos = [repo])],
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
CI.hf_cache_scan,
|
||||
"is_gguf_repo_partial",
|
||||
lambda *args, **kwargs: False,
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
CI,
|
||||
"_gguf_variant_state_summary",
|
||||
lambda _repo_id: (False, 0),
|
||||
)
|
||||
|
||||
rows = CI._scan_cached_gguf()
|
||||
|
||||
assert len(rows) == 1
|
||||
assert rows[0]["repo_id"] == "Org/GgufRepo"
|
||||
assert rows[0]["model_format"] == "gguf"
|
||||
assert rows[0]["size_bytes"] == 100
|
||||
assert rows[0]["last_modified"] == 5_000.0
|
||||
|
||||
|
||||
# ── hf_hub_download_with_xet_fallback force_download bypass (X2/F2) ───
|
||||
|
|
@ -636,3 +682,18 @@ def test_reclaim_replaced_gguf_variant_keeps_no_symlink_current_file(monkeypatch
|
|||
assert snap.exists() is True # the current file must survive
|
||||
assert result["removed_snapshots"] == 0
|
||||
assert result["deleted_blobs"] == 0
|
||||
|
||||
|
||||
def _mmproj_repo(*file_names: str):
|
||||
return SimpleNamespace(
|
||||
revisions = [SimpleNamespace(files = [SimpleNamespace(file_name = n) for n in file_names])]
|
||||
)
|
||||
|
||||
|
||||
def test_repo_has_mmproj_requires_gguf_projector():
|
||||
# A non-GGUF sidecar whose name merely contains "mmproj" must NOT mark the
|
||||
# repo vision-capable; the runtime's projector detection is GGUF-only.
|
||||
assert CI._repo_has_mmproj(_mmproj_repo("model-Q4_K_M.gguf", "mmproj_config.json")) is False
|
||||
assert CI._repo_has_mmproj(_mmproj_repo("model-Q4_K_M.gguf", "README-mmproj.md")) is False
|
||||
# A real GGUF projector still marks the repo vision-capable.
|
||||
assert CI._repo_has_mmproj(_mmproj_repo("model-Q4_K_M.gguf", "mmproj-F16.gguf")) is True
|
||||
|
|
|
|||
266
studio/backend/tests/test_picker_service.py
Normal file
266
studio/backend/tests/test_picker_service.py
Normal file
|
|
@ -0,0 +1,266 @@
|
|||
# SPDX-License-Identifier: AGPL-3.0-only
|
||||
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
import json
|
||||
from types import SimpleNamespace
|
||||
|
||||
from picker.service import (
|
||||
MAX_TEMPLATE_METADATA_BYTES,
|
||||
_chat_template_from_dir,
|
||||
_chat_template_from_processor_json,
|
||||
_chat_template_from_tokenizer_config,
|
||||
_chat_template_from_tokenizer_dir,
|
||||
_find_gguf_in_dir,
|
||||
_iter_ggufs,
|
||||
read_default_chat_template,
|
||||
validate_chat_template,
|
||||
)
|
||||
|
||||
|
||||
def test_iter_ggufs_skips_gguf_companions(tmp_path):
|
||||
mtp_dir = tmp_path / "MTP"
|
||||
mtp_dir.mkdir()
|
||||
main = tmp_path / "model-Q8_0.gguf"
|
||||
main.write_bytes(b"")
|
||||
(tmp_path / "mmproj-F16.gguf").write_bytes(b"")
|
||||
(tmp_path / "mtp-model-Q8_0.gguf").write_bytes(b"")
|
||||
(mtp_dir / "model-Q8_0-MTP.gguf").write_bytes(b"")
|
||||
(tmp_path / "model-Q8_0-be.gguf").write_bytes(b"")
|
||||
|
||||
assert _iter_ggufs(tmp_path) == [main]
|
||||
|
||||
|
||||
def test_find_gguf_in_dir_matches_quant_label(tmp_path):
|
||||
mtp_dir = tmp_path / "MTP"
|
||||
mtp_dir.mkdir()
|
||||
main = tmp_path / "model-Q8_0.gguf"
|
||||
main.write_bytes(b"")
|
||||
(mtp_dir / "model-Q8_0-MTP.gguf").write_bytes(b"")
|
||||
(tmp_path / "model-Q4_K_M.gguf").write_bytes(b"")
|
||||
|
||||
assert _find_gguf_in_dir(tmp_path, "Q8_0") == main
|
||||
assert _find_gguf_in_dir(tmp_path, "Q4_K") is None
|
||||
|
||||
|
||||
def test_find_gguf_in_dir_without_variant_prefers_largest_model(tmp_path):
|
||||
smaller = tmp_path / "a-model-Q4_K_M.gguf"
|
||||
larger = tmp_path / "z-model-Q8_0.gguf"
|
||||
smaller.write_bytes(b"0")
|
||||
larger.write_bytes(b"00")
|
||||
|
||||
assert _find_gguf_in_dir(tmp_path, None) == larger
|
||||
|
||||
|
||||
def test_find_gguf_in_dir_without_variant_prefers_first_split(tmp_path):
|
||||
first = tmp_path / "model-Q4_K_M-00001-of-00003.gguf"
|
||||
second = tmp_path / "model-Q4_K_M-00002-of-00003.gguf"
|
||||
third = tmp_path / "model-Q4_K_M-00003-of-00003.gguf"
|
||||
first.write_bytes(b"0")
|
||||
second.write_bytes(b"000")
|
||||
third.write_bytes(b"00")
|
||||
|
||||
assert _find_gguf_in_dir(tmp_path, None) == first
|
||||
|
||||
first.unlink()
|
||||
assert _find_gguf_in_dir(tmp_path, None) == second
|
||||
|
||||
|
||||
def test_find_gguf_in_dir_matches_bpw_variant_base_label(tmp_path):
|
||||
target = tmp_path / "model-IQ4_XS-3.53bpw.gguf"
|
||||
target.write_bytes(b"")
|
||||
(tmp_path / "model-Q4_K_M.gguf").write_bytes(b"")
|
||||
|
||||
assert _find_gguf_in_dir(tmp_path, "IQ4_XS") == target
|
||||
assert _find_gguf_in_dir(tmp_path, "IQ4_XS-3.53bpw") == target
|
||||
assert _find_gguf_in_dir(tmp_path, "Q4_K") is None
|
||||
|
||||
|
||||
def test_validate_chat_template_accepts_valid_and_empty():
|
||||
assert validate_chat_template("{{ messages[0].content }}").valid is True
|
||||
assert validate_chat_template("").valid is True
|
||||
assert validate_chat_template(" ").valid is True
|
||||
|
||||
|
||||
def test_validate_chat_template_reports_syntax_error_with_line():
|
||||
result = validate_chat_template("{% if %}{% endif %}")
|
||||
assert result.valid is False
|
||||
assert result.error is not None
|
||||
assert result.error.startswith("Line ")
|
||||
|
||||
|
||||
def test_chat_template_from_tokenizer_config_reads_string():
|
||||
assert _chat_template_from_tokenizer_config({"chat_template": "HELLO"}) == "HELLO"
|
||||
assert _chat_template_from_tokenizer_config({"chat_template": " "}) is None
|
||||
assert _chat_template_from_tokenizer_config({}) is None
|
||||
|
||||
|
||||
def test_chat_template_from_tokenizer_config_prefers_named_default():
|
||||
config = {
|
||||
"chat_template": [
|
||||
{"name": "tool_use", "template": "TOOL"},
|
||||
{"name": "default", "template": "DEFAULT"},
|
||||
]
|
||||
}
|
||||
assert _chat_template_from_tokenizer_config(config) == "DEFAULT"
|
||||
|
||||
|
||||
def test_chat_template_from_tokenizer_config_falls_back_to_first_entry():
|
||||
config = {
|
||||
"chat_template": [
|
||||
{"name": "tool_use", "template": "TOOL"},
|
||||
{"name": "other", "template": "OTHER"},
|
||||
]
|
||||
}
|
||||
assert _chat_template_from_tokenizer_config(config) == "TOOL"
|
||||
|
||||
|
||||
def test_chat_template_from_tokenizer_dir_prefers_jinja_file(tmp_path):
|
||||
(tmp_path / "chat_template.jinja").write_text("FROM_JINJA", encoding = "utf-8")
|
||||
(tmp_path / "tokenizer_config.json").write_text(
|
||||
json.dumps({"chat_template": "FROM_CONFIG"}), encoding = "utf-8"
|
||||
)
|
||||
assert _chat_template_from_tokenizer_dir(tmp_path) == "FROM_JINJA"
|
||||
|
||||
|
||||
def test_chat_template_from_tokenizer_dir_reads_tokenizer_config(tmp_path):
|
||||
(tmp_path / "tokenizer_config.json").write_text(
|
||||
json.dumps({"chat_template": "FROM_CONFIG"}), encoding = "utf-8"
|
||||
)
|
||||
assert _chat_template_from_tokenizer_dir(tmp_path) == "FROM_CONFIG"
|
||||
|
||||
|
||||
def test_chat_template_from_dir_without_variant_prefers_tokenizer(tmp_path):
|
||||
(tmp_path / "tokenizer_config.json").write_text(
|
||||
json.dumps({"chat_template": "FROM_CONFIG"}), encoding = "utf-8"
|
||||
)
|
||||
assert _chat_template_from_dir(tmp_path) == "FROM_CONFIG"
|
||||
|
||||
|
||||
def test_chat_template_from_dir_with_variant_still_prefers_tokenizer(tmp_path, monkeypatch):
|
||||
(tmp_path / "tokenizer_config.json").write_text(
|
||||
json.dumps({"chat_template": "FROM_CONFIG"}), encoding = "utf-8"
|
||||
)
|
||||
(tmp_path / "model-Q4_K_M.gguf").write_bytes(b"")
|
||||
monkeypatch.setattr("picker.service.read_gguf_chat_template", lambda _path: "FROM_GGUF")
|
||||
# Selecting a variant must not flip precedence to the embedded GGUF template.
|
||||
assert _chat_template_from_dir(tmp_path, "Q4_K_M") == "FROM_CONFIG"
|
||||
|
||||
|
||||
def test_chat_template_from_dir_with_variant_falls_back_to_gguf(tmp_path, monkeypatch):
|
||||
(tmp_path / "model-Q4_K_M.gguf").write_bytes(b"")
|
||||
monkeypatch.setattr("picker.service.read_gguf_chat_template", lambda _path: "FROM_GGUF")
|
||||
# With no tokenizer sidecar, the embedded GGUF template is still the fallback.
|
||||
assert _chat_template_from_dir(tmp_path, "Q4_K_M") == "FROM_GGUF"
|
||||
|
||||
|
||||
def test_chat_template_from_dir_returns_none_when_absent(tmp_path):
|
||||
assert _chat_template_from_dir(tmp_path) is None
|
||||
|
||||
|
||||
def test_read_default_chat_template_direct_gguf_prefers_sidecar(tmp_path, monkeypatch):
|
||||
gguf = tmp_path / "model-Q4_K_M.gguf"
|
||||
gguf.write_bytes(b"")
|
||||
(tmp_path / "tokenizer_config.json").write_text(
|
||||
json.dumps({"chat_template": "FROM_CONFIG"}), encoding = "utf-8"
|
||||
)
|
||||
monkeypatch.setattr("picker.service._build_browse_allowlist", lambda: [tmp_path])
|
||||
monkeypatch.setattr("picker.service.read_gguf_chat_template", lambda _path: "FROM_GGUF")
|
||||
# A directly selected .gguf must prefer a maintained sidecar over its embedded copy.
|
||||
assert read_default_chat_template(str(gguf)) == "FROM_CONFIG"
|
||||
|
||||
|
||||
def test_read_default_chat_template_direct_gguf_falls_back_to_embedded(tmp_path, monkeypatch):
|
||||
gguf = tmp_path / "model-Q4_K_M.gguf"
|
||||
gguf.write_bytes(b"")
|
||||
monkeypatch.setattr("picker.service._build_browse_allowlist", lambda: [tmp_path])
|
||||
monkeypatch.setattr("picker.service.read_gguf_chat_template", lambda _path: "FROM_GGUF")
|
||||
# With no sidecar next to the file, the embedded GGUF template is the fallback.
|
||||
assert read_default_chat_template(str(gguf)) == "FROM_GGUF"
|
||||
|
||||
|
||||
def test_tokenizer_config_over_size_limit_is_skipped_not_parsed(tmp_path):
|
||||
# An oversized tokenizer_config.json must be skipped before json.loads so a
|
||||
# hostile sidecar cannot exhaust memory.
|
||||
padding = "x" * (MAX_TEMPLATE_METADATA_BYTES + 1024)
|
||||
(tmp_path / "tokenizer_config.json").write_text(
|
||||
json.dumps({"chat_template": "HELLO", "_pad": padding}), encoding = "utf-8"
|
||||
)
|
||||
assert _chat_template_from_tokenizer_dir(tmp_path) is None
|
||||
|
||||
|
||||
def test_processor_json_over_size_limit_is_skipped_not_parsed(tmp_path):
|
||||
padding = "x" * (MAX_TEMPLATE_METADATA_BYTES + 1024)
|
||||
(tmp_path / "chat_template.json").write_text(
|
||||
json.dumps({"default": "HELLO", "_pad": padding}), encoding = "utf-8"
|
||||
)
|
||||
assert _chat_template_from_processor_json(tmp_path) is None
|
||||
|
||||
|
||||
def test_tokenizer_config_at_size_limit_is_still_read(tmp_path):
|
||||
# A normal-sized config is unaffected by the bound (regression guard).
|
||||
(tmp_path / "tokenizer_config.json").write_text(
|
||||
json.dumps({"chat_template": "FROM_CONFIG"}), encoding = "utf-8"
|
||||
)
|
||||
assert _chat_template_from_tokenizer_dir(tmp_path) == "FROM_CONFIG"
|
||||
|
||||
|
||||
def test_remote_template_over_size_limit_is_skipped_before_download(monkeypatch):
|
||||
# An uncached Hub repo whose template exceeds the cap must be skipped via the
|
||||
# remote size pre-check, never downloaded.
|
||||
import huggingface_hub
|
||||
|
||||
monkeypatch.setattr("picker.service.resolve_cached_repo_id_case", lambda name: name)
|
||||
monkeypatch.setattr("picker.service.iter_hf_cache_snapshots", lambda resolved: [])
|
||||
|
||||
def _fail_download(*args, **kwargs):
|
||||
raise AssertionError("oversized remote template must not be downloaded")
|
||||
|
||||
def _fake_get_paths_info(self, repo_id, paths, **kwargs):
|
||||
return [SimpleNamespace(path = p, size = MAX_TEMPLATE_METADATA_BYTES + 1) for p in paths]
|
||||
|
||||
monkeypatch.setattr(huggingface_hub, "hf_hub_download", _fail_download)
|
||||
monkeypatch.setattr(huggingface_hub.HfApi, "get_paths_info", _fake_get_paths_info)
|
||||
|
||||
assert read_default_chat_template("org/oversized-model") is None
|
||||
|
||||
|
||||
def test_remote_oversized_jinja_falls_through_to_tokenizer_template(tmp_path, monkeypatch):
|
||||
# A raw chat_template.jinja between the response cap (MAX_CHAT_TEMPLATE_BYTES)
|
||||
# and the download bound (MAX_TEMPLATE_METADATA_BYTES) must not be returned: the
|
||||
# route drops it, so the remote path must skip the oversized Jinja and fall
|
||||
# through to the smaller tokenizer_config.json.
|
||||
import huggingface_hub
|
||||
from picker.schemas import MAX_CHAT_TEMPLATE_BYTES
|
||||
|
||||
big_jinja = tmp_path / "chat_template.jinja"
|
||||
big_jinja.write_text("{{ x }}" * (MAX_CHAT_TEMPLATE_BYTES // 4), encoding = "utf-8")
|
||||
assert MAX_CHAT_TEMPLATE_BYTES < big_jinja.stat().st_size < MAX_TEMPLATE_METADATA_BYTES
|
||||
tokenizer_config = tmp_path / "tokenizer_config.json"
|
||||
tokenizer_config.write_text(json.dumps({"chat_template": "SMALL_TEMPLATE"}), encoding = "utf-8")
|
||||
files = {
|
||||
"chat_template.jinja": big_jinja,
|
||||
"tokenizer_config.json": tokenizer_config,
|
||||
}
|
||||
|
||||
monkeypatch.setattr("picker.service.resolve_cached_repo_id_case", lambda name: name)
|
||||
monkeypatch.setattr("picker.service.iter_hf_cache_snapshots", lambda resolved: [])
|
||||
|
||||
def _fake_download(repo_id, rel, **kwargs):
|
||||
target = files.get(rel)
|
||||
if target is None:
|
||||
raise FileNotFoundError(rel)
|
||||
return str(target)
|
||||
|
||||
def _fake_get_paths_info(self, repo_id, paths, **kwargs):
|
||||
return [
|
||||
SimpleNamespace(
|
||||
path = p,
|
||||
size = files[p].stat().st_size if p in files else 0,
|
||||
)
|
||||
for p in paths
|
||||
]
|
||||
|
||||
monkeypatch.setattr(huggingface_hub, "hf_hub_download", _fake_download)
|
||||
monkeypatch.setattr(huggingface_hub.HfApi, "get_paths_info", _fake_get_paths_info)
|
||||
|
||||
assert read_default_chat_template("org/big-jinja-model") == "SMALL_TEMPLATE"
|
||||
|
|
@ -50,10 +50,14 @@ _CACHE_MAX_ENTRIES = 4096
|
|||
# keyed by (file cache key, wanted key). None = key absent / file unreadable.
|
||||
_BOOL_CACHE: Dict[Tuple[_CacheKey, str], Optional[bool]] = {}
|
||||
|
||||
_STRING_CACHE: Dict[Tuple[_CacheKey, str], Optional[str]] = {}
|
||||
|
||||
# GGUF header dims for the staged/deferred-load UI: context_length, layer_count
|
||||
# (block_count), and moe_layer_count (block_count minus leading dense layers; 0
|
||||
# if not MoE). One cached pass fills all three so the staged sheet can size every
|
||||
# slider before the model loads. None = unreadable / not a GGUF.
|
||||
# slider before the model loads. None = unreadable / not a GGUF. The native
|
||||
# training context length (``{arch}.context_length``) the UI shows before a model
|
||||
# loads is read from here via read_gguf_context_length.
|
||||
_DIMS_CACHE: Dict[_CacheKey, Optional[Dict[str, Optional[int]]]] = {}
|
||||
|
||||
|
||||
|
|
@ -408,6 +412,83 @@ def _read_gguf_bool(path: str, wanted_key: str) -> Optional[bool]:
|
|||
return result
|
||||
|
||||
|
||||
def _parse_gguf_string(path: str, wanted_key: str) -> Optional[str]:
|
||||
try:
|
||||
with open(path, "rb") as f:
|
||||
head = f.read(24)
|
||||
if len(head) < 24:
|
||||
return None
|
||||
magic, _version, _tcount, kv_count = struct.unpack("<IIQQ", head)
|
||||
if magic != _GGUF_MAGIC:
|
||||
return None
|
||||
|
||||
for _ in range(kv_count):
|
||||
try:
|
||||
klen_bytes = f.read(8)
|
||||
if len(klen_bytes) < 8:
|
||||
break
|
||||
klen = struct.unpack("<Q", klen_bytes)[0]
|
||||
if klen > 1 << 20:
|
||||
break
|
||||
kbytes = f.read(klen)
|
||||
if len(kbytes) < klen:
|
||||
break
|
||||
key = kbytes.decode("utf-8", "replace")
|
||||
vt_bytes = f.read(4)
|
||||
if len(vt_bytes) < 4:
|
||||
break
|
||||
vtype = struct.unpack("<I", vt_bytes)[0]
|
||||
|
||||
if key == wanted_key and vtype == 8:
|
||||
slen_bytes = f.read(8)
|
||||
if len(slen_bytes) < 8:
|
||||
break
|
||||
slen = struct.unpack("<Q", slen_bytes)[0]
|
||||
if slen > 1 << 22:
|
||||
break
|
||||
sbytes = f.read(slen)
|
||||
if len(sbytes) < slen:
|
||||
break
|
||||
return sbytes.decode("utf-8", "replace")
|
||||
if not _skip_gguf_value(f, vtype):
|
||||
break
|
||||
except (struct.error, UnicodeDecodeError):
|
||||
break
|
||||
except OSError as e:
|
||||
logger.debug(f"_parse_gguf_string: cannot open {path}: {e}")
|
||||
return None
|
||||
except Exception as e:
|
||||
logger.debug(f"_parse_gguf_string: parse failure on {path}: {e}")
|
||||
return None
|
||||
return None
|
||||
|
||||
|
||||
def _read_gguf_string(path: str, wanted_key: str) -> Optional[str]:
|
||||
fkey = _cache_key(path)
|
||||
if fkey is None:
|
||||
return None
|
||||
ckey = (fkey, wanted_key)
|
||||
with _CACHE_LOCK:
|
||||
if ckey in _STRING_CACHE:
|
||||
return _STRING_CACHE[ckey]
|
||||
result = _parse_gguf_string(path, wanted_key)
|
||||
with _CACHE_LOCK:
|
||||
while len(_STRING_CACHE) >= _CACHE_MAX_ENTRIES:
|
||||
try:
|
||||
_STRING_CACHE.pop(next(iter(_STRING_CACHE)))
|
||||
except StopIteration:
|
||||
break
|
||||
_STRING_CACHE[ckey] = result
|
||||
return result
|
||||
|
||||
|
||||
def read_gguf_chat_template(path: str) -> Optional[str]:
|
||||
template = _read_gguf_string(path, "tokenizer.chat_template")
|
||||
if isinstance(template, str) and template.strip():
|
||||
return template
|
||||
return None
|
||||
|
||||
|
||||
def read_mmproj_audio_capability(path: str) -> Optional[bool]:
|
||||
"""``clip.has_audio_encoder`` from an mmproj GGUF (e.g. Gemma 4's
|
||||
gemma4ua): ``True``/``False`` if present, ``None`` if absent/unreadable.
|
||||
|
|
|
|||
|
|
@ -196,9 +196,6 @@ function RootLayout() {
|
|||
chatRuntime.setActiveThreadId(null);
|
||||
chatRuntime.setActiveProjectId(null);
|
||||
chatRuntime.setIncognito(false);
|
||||
// Detach the staging UI but keep any in-flight download running, like Hub.
|
||||
if (chatRuntime.pendingSelection)
|
||||
chatRuntime.abandonStagedModel({ keepDownload: true });
|
||||
void navigate({
|
||||
to: "/chat",
|
||||
search: { new: crypto.randomUUID() },
|
||||
|
|
@ -221,10 +218,6 @@ function RootLayout() {
|
|||
chatRuntime.setActiveProjectId(null);
|
||||
chatRuntime.setActiveThreadId(null);
|
||||
chatRuntime.setIncognito(false);
|
||||
// Leaving chat must not kill an in-flight download: detach the staging UI
|
||||
// but keep the transfer running in the manager, like a Hub download.
|
||||
if (chatRuntime.pendingSelection)
|
||||
chatRuntime.abandonStagedModel({ keepDownload: true });
|
||||
}, [isChatRoute]);
|
||||
|
||||
return (
|
||||
|
|
|
|||
|
|
@ -1011,28 +1011,6 @@ export function AppSidebar() {
|
|||
</DropdownMenuItem>
|
||||
</DropdownMenuContent>
|
||||
</DropdownMenu>
|
||||
{isPinned ? (
|
||||
<Tooltip>
|
||||
<TooltipPrimitive.Trigger asChild>
|
||||
<button
|
||||
type="button"
|
||||
onClick={(e) => {
|
||||
e.stopPropagation();
|
||||
unpinChat(item.id);
|
||||
}}
|
||||
aria-label="Unpin chat"
|
||||
className={cn(actionClass, "is-unpin-action")}
|
||||
>
|
||||
<span className="sidebar-row-action-glyph">
|
||||
<HugeiconsIcon icon={PinOffIcon} strokeWidth={1.75} className="size-4" />
|
||||
</span>
|
||||
</button>
|
||||
</TooltipPrimitive.Trigger>
|
||||
<TooltipContent side="bottom" sideOffset={6} className="tooltip-compact">
|
||||
Unpin
|
||||
</TooltipContent>
|
||||
</Tooltip>
|
||||
) : null}
|
||||
</SidebarMenuItem>
|
||||
);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,79 +0,0 @@
|
|||
// SPDX-License-Identifier: AGPL-3.0-only
|
||||
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
// Per-model pre-load inference settings, persisted in localStorage so the load
|
||||
// dialog can offer "Remember settings for <model>". GGUF picks only: every
|
||||
// field is a llama.cpp load knob, so all save/restore call sites gate on
|
||||
// GGUF-ness (a non-GGUF blob would only snapshot leftover standing values).
|
||||
|
||||
const KEY = "unsloth_load_settings";
|
||||
|
||||
export interface RememberedLoadSettings {
|
||||
contextLength: number | null;
|
||||
kvCacheDtype: string | null;
|
||||
speculativeType: string | null;
|
||||
specDraftNMax: number | null;
|
||||
tensorParallel: boolean;
|
||||
// GPU Memory controls. Optional so an older blob (which lacked them) still
|
||||
// parses, leaving the live knobs untouched on apply. The mode is kept with the
|
||||
// manual knobs (gpuLayers/nCpuMoe are ignored outside Manual mode). A null
|
||||
// selectedGpuIds is meaningful (all GPUs), so it's distinguished from absent.
|
||||
// The per-GPU split ratio is deliberately NOT remembered: it's positionally
|
||||
// bound to the exact GPU set/order and unvalidated, so it would mismatch.
|
||||
gpuMemoryMode?: "auto" | "manual";
|
||||
gpuLayers?: number;
|
||||
nCpuMoe?: number;
|
||||
selectedGpuIds?: number[] | null;
|
||||
}
|
||||
|
||||
// Storage key for a pick's remembered settings, scoped per quant (the VRAM-budget
|
||||
// knobs differ per quant). An HF repo collapses its GGUF variants into one `id`,
|
||||
// so fold the variant in. Local .gguf paths are already file-specific; native
|
||||
// drag-drop files key by display label, so same-named files share an entry.
|
||||
export function rememberedLoadSettingsKey(selection: {
|
||||
id: string;
|
||||
ggufVariant?: string | null;
|
||||
}): string {
|
||||
return selection.ggufVariant
|
||||
? `${selection.id}::${selection.ggufVariant}`
|
||||
: selection.id;
|
||||
}
|
||||
|
||||
function readAll(): Record<string, RememberedLoadSettings> {
|
||||
try {
|
||||
return JSON.parse(localStorage.getItem(KEY) ?? "{}");
|
||||
} catch {
|
||||
return {};
|
||||
}
|
||||
}
|
||||
|
||||
function writeAll(all: Record<string, RememberedLoadSettings>) {
|
||||
try {
|
||||
localStorage.setItem(KEY, JSON.stringify(all));
|
||||
} catch {
|
||||
// Ignore quota / unavailable storage.
|
||||
}
|
||||
}
|
||||
|
||||
export function loadRememberedLoadSettings(
|
||||
key: string,
|
||||
): RememberedLoadSettings | null {
|
||||
return readAll()[key] ?? null;
|
||||
}
|
||||
|
||||
export function saveRememberedLoadSettings(
|
||||
key: string,
|
||||
settings: RememberedLoadSettings,
|
||||
) {
|
||||
const all = readAll();
|
||||
all[key] = settings;
|
||||
writeAll(all);
|
||||
}
|
||||
|
||||
export function clearRememberedLoadSettings(key: string) {
|
||||
const all = readAll();
|
||||
if (key in all) {
|
||||
delete all[key];
|
||||
writeAll(all);
|
||||
}
|
||||
}
|
||||
|
|
@ -2,10 +2,7 @@
|
|||
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
import { getAuthToken } from "@/features/auth";
|
||||
import {
|
||||
loadRememberedLoadSettings,
|
||||
rememberedLoadSettingsKey,
|
||||
} from "@/components/assistant-ui/model-selector/remembered-load-settings";
|
||||
import { resolveInitialConfig } from "@/features/model-picker";
|
||||
import { projectHasSources } from "@/features/rag/api/rag-api";
|
||||
import { apiUrl } from "@/lib/api-base";
|
||||
import { parseParamCountB } from "@/lib/model-size";
|
||||
|
|
@ -46,7 +43,7 @@ import {
|
|||
type PendingImageEditReference,
|
||||
type RagAutoInject,
|
||||
GPU_LAYERS_AUTO,
|
||||
loadedGpuMemoryFieldsUnlessStaged,
|
||||
loadedGpuMemoryFields,
|
||||
reconcilePersistedGpuIds,
|
||||
resolveLoadedSpeculativeSettings,
|
||||
resolveSpeculativeSettingsForLoad,
|
||||
|
|
@ -1533,65 +1530,56 @@ async function autoLoadSmallestModel(): Promise<{
|
|||
return false;
|
||||
}
|
||||
const currentStore = useChatRuntimeStore.getState();
|
||||
// Blobs are saved for GGUF picks only (the sheet gates on it), so don't
|
||||
// let a legacy non-GGUF blob feed a stale context/spec choice into a
|
||||
// safetensors auto-load.
|
||||
const remembered =
|
||||
candidate.kind === "gguf"
|
||||
? loadRememberedLoadSettings(
|
||||
rememberedLoadSettingsKey({
|
||||
id: candidate.id,
|
||||
ggufVariant: candidate.ggufVariant,
|
||||
}),
|
||||
)
|
||||
: null;
|
||||
const { config } = resolveInitialConfig(candidate.id, candidate.ggufVariant);
|
||||
const effectiveMaxSeqLength = resolveLoadMaxSeqLength({
|
||||
modelId: candidate.id,
|
||||
ggufVariant: candidate.ggufVariant,
|
||||
isGguf: candidate.kind === "gguf",
|
||||
customContextLength: remembered?.contextLength ?? null,
|
||||
customContextLength: config.customContextLength,
|
||||
ggufContextLength: null,
|
||||
currentCheckpoint: currentStore.params.checkpoint,
|
||||
activeGgufVariant: currentStore.activeGgufVariant,
|
||||
maxSeqLength: candidate.maxSeqLength,
|
||||
maxSeqLength: config.maxSeqLength ?? candidate.maxSeqLength,
|
||||
presetSource: currentStore.activePresetSource,
|
||||
});
|
||||
// The GPU knobs are per-model, so read them from the same remembered
|
||||
// settings that fed effectiveMaxSeqLength -- on a background auto-load the
|
||||
// live store holds session defaults, not the saved Manual mode / layer pin /
|
||||
// GPU pick. Absent fields fall back like applyRememberedLoadSettings: the
|
||||
// mode to the store (a persisted standing preference), the per-model knobs to
|
||||
// their defaults. The saved GPU pick is reconciled against the GPUs present
|
||||
// now, like the interactive restore.
|
||||
// The GPU knobs are per-model, so read them from the same per-model config
|
||||
// that fed effectiveMaxSeqLength -- on a background auto-load the live store
|
||||
// holds session defaults, not the saved Manual mode / layer pin / GPU pick.
|
||||
// Absent fields fall back like the interactive restore: the mode to the store
|
||||
// (a persisted standing preference), the per-model knobs to their defaults.
|
||||
// The saved GPU pick is reconciled against the GPUs present now.
|
||||
const effectiveGpuMemoryMode =
|
||||
remembered?.gpuMemoryMode ?? currentStore.gpuMemoryMode;
|
||||
const effectiveGpuLayers = remembered?.gpuLayers ?? GPU_LAYERS_AUTO;
|
||||
const effectiveNCpuMoe = remembered?.nCpuMoe ?? 0;
|
||||
if (remembered?.selectedGpuIds != null) {
|
||||
config.gpuMemoryMode ?? currentStore.gpuMemoryMode;
|
||||
const effectiveGpuLayers = config.gpuLayers ?? GPU_LAYERS_AUTO;
|
||||
const effectiveNCpuMoe = config.nCpuMoe ?? 0;
|
||||
if (config.selectedGpuIds != null) {
|
||||
// Warm the device cache first: on a cold cache the reconcile passes the
|
||||
// saved pick through unvalidated, and a stale cross-host pick then fails
|
||||
// the load with the picker hidden.
|
||||
await ensureGpuDeviceCache();
|
||||
}
|
||||
const effectiveGpuIds =
|
||||
remembered?.selectedGpuIds !== undefined
|
||||
? reconcilePersistedGpuIds(remembered.selectedGpuIds)
|
||||
config.selectedGpuIds !== undefined
|
||||
? reconcilePersistedGpuIds(config.selectedGpuIds)
|
||||
: null;
|
||||
// Under Manual GPU memory + Auto layers, llama.cpp's --fit owns context
|
||||
// sizing, so send 0 (or the pinned length). GGUF-only; a no-op otherwise.
|
||||
// The context pin is per-model too, so it comes from remembered settings,
|
||||
// not the live store.
|
||||
// The context pin is per-model too, so it comes from the saved config, not
|
||||
// the live store.
|
||||
const fitMaxSeqLength = resolveFitMaxSeqLength(
|
||||
candidate.kind === "gguf",
|
||||
effectiveGpuMemoryMode,
|
||||
effectiveGpuLayers,
|
||||
remembered?.contextLength ?? null,
|
||||
config.customContextLength ?? null,
|
||||
effectiveMaxSeqLength,
|
||||
);
|
||||
const effectiveSpeculativeType =
|
||||
remembered?.speculativeType ?? specSettings.speculativeType;
|
||||
config.speculativeType ?? specSettings.speculativeType;
|
||||
const effectiveSpecDraftNMax =
|
||||
remembered?.specDraftNMax ?? specSettings.specDraftNMax;
|
||||
config.specDraftNMax ?? specSettings.specDraftNMax;
|
||||
const effectiveChatTemplateOverride = config.chatTemplateOverride?.trim()
|
||||
? config.chatTemplateOverride
|
||||
: null;
|
||||
if (
|
||||
!(await canAutoLoad({
|
||||
model_path: candidate.id,
|
||||
|
|
@ -1621,10 +1609,11 @@ async function autoLoadSmallestModel(): Promise<{
|
|||
is_lora: false,
|
||||
gguf_variant: candidate.ggufVariant,
|
||||
trust_remote_code: trustRemoteCode,
|
||||
cache_type_kv: remembered?.kvCacheDtype ?? null,
|
||||
chat_template_override: effectiveChatTemplateOverride,
|
||||
cache_type_kv: config.kvCacheDtype,
|
||||
speculative_type: effectiveSpeculativeType,
|
||||
spec_draft_n_max: effectiveSpecDraftNMax,
|
||||
tensor_parallel: remembered?.tensorParallel ?? false,
|
||||
tensor_parallel: config.tensorParallel,
|
||||
// GGUF-only: the safetensors fallback loads via HF auto-placement (no
|
||||
// explicit pins). The split ratio is deliberately never remembered
|
||||
// (positionally bound to an exact GPU set), so auto-load leaves llama.cpp's
|
||||
|
|
@ -1638,7 +1627,12 @@ async function autoLoadSmallestModel(): Promise<{
|
|||
}
|
||||
: {}),
|
||||
});
|
||||
saveSpeculativeType(effectiveSpeculativeType);
|
||||
// Only persist the global preference when the value came from the global
|
||||
// settings. A per-model config's choice must stay load-local, or autoloading
|
||||
// a remembered model on startup would rewrite the global default.
|
||||
if (config.speculativeType == null) {
|
||||
saveSpeculativeType(effectiveSpeculativeType);
|
||||
}
|
||||
// Self-gates on is_gguf (skips diffusion), so persists only for a real GGUF load.
|
||||
persistGpuMemoryModeOnLoad(loadResp, effectiveGpuMemoryMode);
|
||||
useChatRuntimeStore
|
||||
|
|
@ -1650,6 +1644,9 @@ async function autoLoadSmallestModel(): Promise<{
|
|||
);
|
||||
store.setParams({
|
||||
...store.params,
|
||||
...(candidate.kind === "gguf"
|
||||
? {}
|
||||
: { maxSeqLength: effectiveMaxSeqLength }),
|
||||
maxTokens:
|
||||
candidate.kind === "gguf"
|
||||
? loadResp.context_length ?? 131072
|
||||
|
|
@ -1676,7 +1673,7 @@ async function autoLoadSmallestModel(): Promise<{
|
|||
const keepCustomCtx = resolveManualAutoCtxPin(
|
||||
effectiveGpuMemoryMode,
|
||||
effectiveGpuLayers,
|
||||
remembered?.contextLength ?? null,
|
||||
config.customContextLength ?? null,
|
||||
);
|
||||
useChatRuntimeStore.setState({
|
||||
ggufContextLength: loadResp.context_length ?? 131072,
|
||||
|
|
@ -1694,13 +1691,14 @@ async function autoLoadSmallestModel(): Promise<{
|
|||
loadedKvCacheDtype: loadResp.cache_type_kv ?? null,
|
||||
tensorParallel: loadResp.tensor_parallel ?? false,
|
||||
loadedTensorParallel: loadResp.tensor_parallel ?? false,
|
||||
...loadedGpuMemoryFieldsUnlessStaged(loadResp, {
|
||||
customContextLength: keepCustomCtx,
|
||||
}),
|
||||
...loadedGpuMemoryFields(loadResp),
|
||||
loadedCustomContextLength: keepCustomCtx,
|
||||
defaultChatTemplate: loadResp.chat_template ?? null,
|
||||
chatTemplateOverride: null,
|
||||
loadedChatTemplateOverride: null,
|
||||
chatTemplateOverride: effectiveChatTemplateOverride,
|
||||
loadedChatTemplateOverride: effectiveChatTemplateOverride,
|
||||
// Retain the saved requested context so re-saving the config keeps the
|
||||
// override; null stays null (auto/VRAM-fit).
|
||||
customContextLength: config.customContextLength,
|
||||
loadedIsMultimodal: isMultimodalResponse(loadResp),
|
||||
loadedIsDiffusion: loadResp.is_diffusion ?? false,
|
||||
...resolveLoadedSpeculativeSettings(loadResp),
|
||||
|
|
@ -1720,10 +1718,11 @@ async function autoLoadSmallestModel(): Promise<{
|
|||
loadedTensorParallel: loadResp.tensor_parallel ?? false,
|
||||
// Non-GGUF response: clears any stale GPU baseline a prior manual-GPU
|
||||
// GGUF load left, matching the interactive/status sibling load paths.
|
||||
...loadedGpuMemoryFieldsUnlessStaged(loadResp),
|
||||
...loadedGpuMemoryFields(loadResp),
|
||||
defaultChatTemplate: loadResp.chat_template ?? null,
|
||||
chatTemplateOverride: null,
|
||||
loadedChatTemplateOverride: null,
|
||||
chatTemplateOverride: effectiveChatTemplateOverride,
|
||||
loadedChatTemplateOverride: effectiveChatTemplateOverride,
|
||||
customContextLength: null,
|
||||
...resolveLoadedSpeculativeSettings(loadResp),
|
||||
loadedIsMultimodal: isMultimodalResponse(loadResp),
|
||||
loadedIsDiffusion: loadResp.is_diffusion ?? false,
|
||||
|
|
@ -1988,7 +1987,7 @@ async function autoLoadSmallestModel(): Promise<{
|
|||
loadedKvCacheDtype: loadResp.cache_type_kv ?? null,
|
||||
tensorParallel: loadResp.tensor_parallel ?? false,
|
||||
loadedTensorParallel: loadResp.tensor_parallel ?? false,
|
||||
...loadedGpuMemoryFieldsUnlessStaged(loadResp),
|
||||
...loadedGpuMemoryFields(loadResp),
|
||||
// Drives the GPU Memory controls' diffusion gate; set alongside the
|
||||
// GPU fields on every load path so the gate can't read stale.
|
||||
loadedIsDiffusion: loadResp.is_diffusion ?? false,
|
||||
|
|
|
|||
|
|
@ -377,14 +377,33 @@ export async function listCachedModels(
|
|||
return data.cached;
|
||||
}
|
||||
|
||||
export async function deleteCachedModel(
|
||||
export interface CachedModelPath {
|
||||
path: string;
|
||||
is_dir: boolean;
|
||||
}
|
||||
|
||||
/** Absolute on-disk path of a cached repo or one of its GGUF variants. */
|
||||
export async function getCachedModelPath(
|
||||
repoId: string,
|
||||
variant?: string,
|
||||
): Promise<CachedModelPath> {
|
||||
const params = new URLSearchParams({ repo_id: repoId });
|
||||
if (variant) params.set("variant", variant);
|
||||
const response = await authFetch(
|
||||
`/api/models/cached-model-path?${params.toString()}`,
|
||||
);
|
||||
return parseJsonOrThrow<CachedModelPath>(response);
|
||||
}
|
||||
|
||||
/** Reveal a cached repo (or one GGUF variant's file) in the OS file manager. */
|
||||
export async function revealCachedModel(
|
||||
repoId: string,
|
||||
variant?: string,
|
||||
): Promise<void> {
|
||||
const payload: Record<string, string> = { repo_id: repoId };
|
||||
if (variant) payload.variant = variant;
|
||||
const response = await authFetch("/api/models/delete-cached", {
|
||||
method: "DELETE",
|
||||
const response = await authFetch("/api/models/reveal-cached-model", {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify(payload),
|
||||
});
|
||||
|
|
|
|||
|
|
@ -2,16 +2,19 @@
|
|||
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
import {
|
||||
applyModelLoadConfigToRuntime,
|
||||
currentRuntimePerModelConfig,
|
||||
type DeletedModelRef,
|
||||
type ExternalModelOption,
|
||||
type LoraModelOption,
|
||||
type ModelOption,
|
||||
ModelSelector,
|
||||
} from "@/components/assistant-ui/model-selector";
|
||||
import {
|
||||
loadRememberedLoadSettings,
|
||||
rememberedLoadSettingsKey,
|
||||
} from "@/components/assistant-ui/model-selector/remembered-load-settings";
|
||||
type ModelSelectorChangeMeta,
|
||||
type PerModelConfig,
|
||||
resolveInitialConfig,
|
||||
SidebarModelConfig,
|
||||
useActiveModelConfig,
|
||||
} from "@/features/model-picker";
|
||||
import { ProjectComposer, Thread } from "@/components/assistant-ui/thread";
|
||||
import { CopyableErrorChip } from "@/components/ui/copyable-error-chip";
|
||||
import {
|
||||
|
|
@ -27,10 +30,10 @@ import {
|
|||
} from "@/components/ui/resizable";
|
||||
import { useSidebar } from "@/components/ui/sidebar";
|
||||
import { Tooltip, TooltipContent } from "@/components/ui/tooltip";
|
||||
import { useLatestRef } from "@/features/hub/hooks/use-latest-ref";
|
||||
import {
|
||||
DOWNLOAD_KIND,
|
||||
downloadManager,
|
||||
useRepoDownload,
|
||||
} from "@/features/hub/download-manager";
|
||||
import {
|
||||
type NativeIntent,
|
||||
|
|
@ -93,7 +96,6 @@ import {
|
|||
renameChatItem,
|
||||
useChatSidebarItems,
|
||||
} from "./hooks/use-chat-sidebar-items";
|
||||
import { useStagedModelPreparation } from "./hooks/use-staged-model-preparation";
|
||||
import {
|
||||
clearTrainingCompareHandoff,
|
||||
getTrainingCompareHandoff,
|
||||
|
|
@ -128,10 +130,8 @@ import {
|
|||
hasGgufSource,
|
||||
isDownloadableHubRepo,
|
||||
loadOptionalBool,
|
||||
pendingSelectionMatches,
|
||||
useChatRuntimeStore,
|
||||
} from "./stores/chat-runtime-store";
|
||||
import type { PendingModelSelection } from "./stores/chat-runtime-store";
|
||||
import { useChatPreferencesStore } from "./stores/chat-preferences-store";
|
||||
import { useExternalProvidersStore } from "./stores/external-providers-store";
|
||||
import { buildChatTourSteps } from "./tour";
|
||||
|
|
@ -385,6 +385,7 @@ type CompareModelSelection = {
|
|||
id: string;
|
||||
isLora: boolean;
|
||||
ggufVariant?: string;
|
||||
config?: PerModelConfig;
|
||||
};
|
||||
|
||||
function modelMatchesDeleted(
|
||||
|
|
@ -645,6 +646,8 @@ function GeneralCompareHeader({
|
|||
loraModels,
|
||||
externalModels,
|
||||
value,
|
||||
selectedConfig,
|
||||
selectedGgufVariant,
|
||||
onValueChange,
|
||||
onFoldersChange,
|
||||
onModelsChange,
|
||||
|
|
@ -655,9 +658,11 @@ function GeneralCompareHeader({
|
|||
loraModels: LoraModelOption[];
|
||||
externalModels: ExternalModelOption[];
|
||||
value: string;
|
||||
selectedConfig?: PerModelConfig | null;
|
||||
selectedGgufVariant?: string | null;
|
||||
onValueChange: (
|
||||
id: string,
|
||||
meta: { isLora: boolean; ggufVariant?: string },
|
||||
meta: ModelSelectorChangeMeta,
|
||||
) => void;
|
||||
onFoldersChange?: () => void;
|
||||
onModelsChange?: (deletedModel?: DeletedModelRef) => void;
|
||||
|
|
@ -684,6 +689,8 @@ function GeneralCompareHeader({
|
|||
loraModels={loraModels}
|
||||
externalModels={externalModels}
|
||||
value={value}
|
||||
selectedConfig={selectedConfig}
|
||||
selectedGgufVariant={selectedGgufVariant}
|
||||
onValueChange={onValueChange}
|
||||
onFoldersChange={onFoldersChange}
|
||||
onModelsChange={onModelsChange}
|
||||
|
|
@ -811,11 +818,14 @@ const GeneralCompareContent = memo(function GeneralCompareContent({
|
|||
loraModels={loraModels}
|
||||
externalModels={externalModels}
|
||||
value={model1.id}
|
||||
selectedConfig={model1.config}
|
||||
selectedGgufVariant={model1.ggufVariant}
|
||||
onValueChange={(id, meta) =>
|
||||
setModel1({
|
||||
id,
|
||||
isLora: meta.isLora,
|
||||
ggufVariant: meta.ggufVariant,
|
||||
config: meta.config,
|
||||
})
|
||||
}
|
||||
onFoldersChange={onFoldersChange}
|
||||
|
|
@ -838,11 +848,14 @@ const GeneralCompareContent = memo(function GeneralCompareContent({
|
|||
loraModels={loraModels}
|
||||
externalModels={externalModels}
|
||||
value={model2.id}
|
||||
selectedConfig={model2.config}
|
||||
selectedGgufVariant={model2.ggufVariant}
|
||||
onValueChange={(id, meta) =>
|
||||
setModel2({
|
||||
id,
|
||||
isLora: meta.isLora,
|
||||
ggufVariant: meta.ggufVariant,
|
||||
config: meta.config,
|
||||
})
|
||||
}
|
||||
onFoldersChange={onFoldersChange}
|
||||
|
|
@ -1236,6 +1249,13 @@ export function validateChatSearch(search: Record<string, unknown>): ChatSearch
|
|||
};
|
||||
}
|
||||
|
||||
type PendingHubAutoLoad = {
|
||||
selection: SelectedModelInput;
|
||||
contextKey: string;
|
||||
originCheckpoint: string;
|
||||
originGgufVariant: string | null;
|
||||
};
|
||||
|
||||
// `search` comes from RootLayout (not useSearch) so ChatPage stays mounted off-route
|
||||
// (keeping an in-flight generation alive), frozen to the last /chat search. `active`
|
||||
// is false off-route: close body-portaled surfaces and stop route-specific listeners
|
||||
|
|
@ -1248,30 +1268,6 @@ export function ChatPage({
|
|||
|
||||
const settingsOpen = useChatRuntimeStore((s) => s.settingsPanelOpen);
|
||||
const setSettingsOpen = useChatRuntimeStore((s) => s.setSettingsPanelOpen);
|
||||
// Deferred-load staging: downloads a staged GGUF (if needed) and reads its
|
||||
// header context so the sheet can show the context slider before the load.
|
||||
// autoLoad picks instead load the cached file as soon as the download ends;
|
||||
// selectModel is defined below, so the load runs through a ref.
|
||||
const autoLoadStagedRef = useRef<
|
||||
((pending: PendingModelSelection) => void) | null
|
||||
>(null);
|
||||
const stagedDownload = useStagedModelPreparation({
|
||||
onAutoLoad: (pending) => autoLoadStagedRef.current?.(pending),
|
||||
});
|
||||
// Abandon a staged pick: the store action cancels its in-flight download and
|
||||
// reverts the edited knobs, so nothing lingers after the user walks away.
|
||||
const abandonStaged = useCallback(() => {
|
||||
useChatRuntimeStore.getState().abandonStagedModel();
|
||||
}, []);
|
||||
// Detach a staged pick on navigation without cancelling its download: the
|
||||
// transfer keeps running in the manager and lands in cache, like Hub.
|
||||
const detachStaged = useCallback(() => {
|
||||
useChatRuntimeStore.getState().abandonStagedModel({ keepDownload: true });
|
||||
}, []);
|
||||
// Tracks whether the chat page is still mounted, so a staged-load failure that
|
||||
// resolves after the user left chat doesn't resurrect the abandoned pick.
|
||||
const mountedRef = useRef(true);
|
||||
useEffect(() => () => void (mountedRef.current = false), []);
|
||||
const incognito = useChatRuntimeStore((s) => s.incognito);
|
||||
const setIncognito = useChatRuntimeStore((s) => s.setIncognito);
|
||||
const incognitoLabel = incognito
|
||||
|
|
@ -1363,6 +1359,9 @@ export function ChatPage({
|
|||
const ggufContextLength = useChatRuntimeStore(
|
||||
(state) => state.ggufContextLength,
|
||||
);
|
||||
const ggufNativeContextLength = useChatRuntimeStore(
|
||||
(state) => state.ggufNativeContextLength,
|
||||
);
|
||||
const contextUsage = useChatRuntimeStore((state) => state.contextUsage);
|
||||
const modelsFromStore = useChatRuntimeStore((state) => state.models);
|
||||
const lorasFromStore = useChatRuntimeStore((state) => state.loras);
|
||||
|
|
@ -1440,39 +1439,37 @@ export function ChatPage({
|
|||
refreshRef.current = refresh;
|
||||
selectModelRef.current = selectModel;
|
||||
}, [refresh, selectModel]);
|
||||
// Load a cached autoLoad pick once its download finishes. The sheet was never
|
||||
// opened, so on a load failure just drop the orphaned staged knobs. The knobs
|
||||
// were already seeded on stage, so keepSpeculative only when a config was
|
||||
// saved -- otherwise the standing speculative preference should win.
|
||||
autoLoadStagedRef.current = (pending) => {
|
||||
// Blobs are saved for GGUF picks only (the sheet gates on it), so don't
|
||||
// let a legacy non-GGUF blob claim a seeded config here.
|
||||
const remembered = hasGgufSource(pending)
|
||||
? loadRememberedLoadSettings(rememberedLoadSettingsKey(pending))
|
||||
: null;
|
||||
void selectModel({
|
||||
...pending,
|
||||
isDownloaded: true,
|
||||
forceReload: true,
|
||||
keepSpeculative: remembered != null,
|
||||
throwOnError: true,
|
||||
}).catch(() => {
|
||||
const store = useChatRuntimeStore.getState();
|
||||
// selectModel only clears pendingSelection on success, so a failed
|
||||
// auto-load leaves our staged pick (and its edited load knobs) behind.
|
||||
// Abandon it when it is still the active stage; otherwise just revert the
|
||||
// settings if the stage was already cleared by something else.
|
||||
if (pendingSelectionMatches(store.pendingSelection, pending)) {
|
||||
store.abandonStagedModel();
|
||||
} else if (!store.pendingSelection) {
|
||||
store.resetModelSettingsToLoaded();
|
||||
}
|
||||
});
|
||||
};
|
||||
const rememberedConfigFor = useCallback(
|
||||
(selection: {
|
||||
id: string;
|
||||
ggufVariant?: string | null;
|
||||
source?: string;
|
||||
}) => {
|
||||
if (selection.source === "external") return null;
|
||||
const resolved = resolveInitialConfig(selection.id, selection.ggufVariant);
|
||||
return resolved.remembered ? resolved.config : null;
|
||||
},
|
||||
[],
|
||||
);
|
||||
const isExternalModel = useMemo(
|
||||
() => isExternalModelId(inferenceParams.checkpoint),
|
||||
[inferenceParams.checkpoint],
|
||||
);
|
||||
const {
|
||||
checkpoint: runtimeCheckpoint,
|
||||
isGguf: runtimeModelIsGguf,
|
||||
config: activeModelConfig,
|
||||
} = useActiveModelConfig();
|
||||
const activeModelIsGguf =
|
||||
runtimeCheckpoint != null && !isExternalModel && runtimeModelIsGguf;
|
||||
const activeModelIsLora = useMemo(() => {
|
||||
const checkpoint = inferenceParams.checkpoint;
|
||||
if (!checkpoint || isExternalModel) return false;
|
||||
const model = modelsFromStore.find((entry) => entry.id === checkpoint);
|
||||
if (model) return model.isLora;
|
||||
const lora = lorasFromStore.find((entry) => entry.id === checkpoint);
|
||||
return lora?.exportType === "lora";
|
||||
}, [inferenceParams.checkpoint, isExternalModel, modelsFromStore, lorasFromStore]);
|
||||
const reasoningEnabled = useChatRuntimeStore((s) => s.reasoningEnabled);
|
||||
const reasoningStyle = useChatRuntimeStore((s) => s.reasoningStyle);
|
||||
const reasoningEffort = useChatRuntimeStore((s) => s.reasoningEffort);
|
||||
|
|
@ -1783,75 +1780,21 @@ export function ChatPage({
|
|||
closeArtifactSurface();
|
||||
}, [activeThreadId, closeArtifactSurface, selectedArtifact, view]);
|
||||
|
||||
// Abandon a staged (not-yet-loaded) pick when the chat context actually
|
||||
// changes — switching threads, leaving single view, or starting a new chat /
|
||||
// project — so a stale Load button can't resurface in a different context.
|
||||
// New Chat keeps activeThreadId null and only bumps the `new` search nonce, so
|
||||
// the key includes the route identity, not just the thread. Mirrors the
|
||||
// incognito reset pattern. (Route exit is handled in __root.tsx, which runs
|
||||
// after this unmounts.) Clear only on a real change, never on mount: staging
|
||||
// from the Hub sets pendingSelection then navigates here, and clearing on
|
||||
// mount would wipe it. Comparing the previous context (rather than a first-run
|
||||
// flag) is also safe under StrictMode's double-invoke and component remounts.
|
||||
const chatContextKey = `${view.mode}|${activeThreadId ?? ""}|${search.new ?? ""}|${search.project ?? ""}`;
|
||||
const chatContextKeyRef = useLatestRef(chatContextKey);
|
||||
const prevChatContextRef = useRef<string | null>(null);
|
||||
useEffect(() => {
|
||||
const prev = prevChatContextRef.current;
|
||||
prevChatContextRef.current = chatContextKey;
|
||||
if (prev === null || prev === chatContextKey) return;
|
||||
detachStaged();
|
||||
}, [chatContextKey, detachStaged]);
|
||||
|
||||
const hasActiveModel = Boolean(inferenceParams.checkpoint);
|
||||
// Load immediately, or — when "Load on selection" is off — stage the pick so
|
||||
// its load options can be set first. Shared by the main selector, native
|
||||
// drag-drop/picker, and the dropped-file chip (the Hub stages via the store).
|
||||
const chatContextKey = `${view.mode}|${activeThreadId ?? ""}|${search.new ?? ""}|${search.project ?? ""}`;
|
||||
const [pendingHubAutoLoad, setPendingHubAutoLoad] =
|
||||
useState<PendingHubAutoLoad | null>(null);
|
||||
const stageOrLoad = useCallback(
|
||||
async (selection: SelectedModelInput) => {
|
||||
const store = useChatRuntimeStore.getState();
|
||||
// An un-cached HF repo (GGUF variant or a full non-GGUF snapshot) downloads
|
||||
// through the manager first (global indicator), then auto-loads. Everything
|
||||
// else -- cached picks, local/native files, LoRA, external -- loads now.
|
||||
const wantManagerDownload =
|
||||
isDownloadableHubRepo(selection) && !selection.isDownloaded;
|
||||
if (
|
||||
(!hasGgufSource(selection) && !wantManagerDownload) ||
|
||||
(store.loadOnSelection && selection.isDownloaded)
|
||||
) {
|
||||
// Detach any staged pick first so its edited knobs (e.g. a custom
|
||||
// context length) don't leak into this immediate load -- resolveLoad
|
||||
// reads customContextLength before checking the target is GGUF. Detach
|
||||
// (not abandon) keeps its download running.
|
||||
detachStaged();
|
||||
// Load-on-selection skips the sheet, so seed the saved knobs here the
|
||||
// way the sheet's restore effect would; the switch would otherwise reset
|
||||
// the remembered speculative choice (keepSpeculative below prevents it).
|
||||
const remembered = hasGgufSource(selection)
|
||||
? loadRememberedLoadSettings(rememberedLoadSettingsKey(selection))
|
||||
: null;
|
||||
if (remembered) store.applyRememberedLoadSettings(remembered);
|
||||
await selectModel(
|
||||
remembered ? { ...selection, keepSpeculative: true } : selection,
|
||||
);
|
||||
return;
|
||||
}
|
||||
// Loads can't queue behind each other, but a download is independent: if
|
||||
// the pick needs downloading, start it in the manager so it runs alongside
|
||||
// the load. Nothing to download (already on device) just waits.
|
||||
if (store.modelLoading) {
|
||||
// Both an uncached non-GGUF snapshot (wantManagerDownload) and an
|
||||
// uncached remote GGUF quant download through the manager, so either can
|
||||
// run in the background while another model loads. wantManagerDownload
|
||||
// excludes GGUF by design, so the GGUF case is checked separately.
|
||||
const wantBackgroundDownload =
|
||||
wantManagerDownload ||
|
||||
(selection.source === "hub" &&
|
||||
hasGgufSource(selection) &&
|
||||
!selection.isDownloaded);
|
||||
// The model currently loading already downloads as part of its own load
|
||||
// (the /load flow fetches before setting the checkpoint), so re-picking
|
||||
// it must not kick off a second transfer against the same cache.
|
||||
const isLoadingThisPick =
|
||||
!!loadingModel &&
|
||||
normalizeModelRef(loadingModel.id) ===
|
||||
|
|
@ -1862,11 +1805,6 @@ export function ChatPage({
|
|||
description: "It's downloading as part of the load in progress.",
|
||||
});
|
||||
} else if (wantBackgroundDownload) {
|
||||
// Only claim the download started once a job is actually created. A
|
||||
// transport conflict records state that is only resolvable from the
|
||||
// Hub download card, so point the user there instead of showing a
|
||||
// success toast for a transfer that never began; "busy" and "error"
|
||||
// already surface their own toasts.
|
||||
const outcome = await downloadManager.requestStart({
|
||||
kind: DOWNLOAD_KIND.MODEL,
|
||||
repoId: selection.id,
|
||||
|
|
@ -1883,6 +1821,11 @@ export function ChatPage({
|
|||
description:
|
||||
"An earlier partial download used a different transport. Open the Hub tab to resume or restart it.",
|
||||
});
|
||||
} else if (outcome === "busy") {
|
||||
toast.info("Download already in progress", {
|
||||
description:
|
||||
"Another download for this model is still running. Reselect it once that finishes to load it.",
|
||||
});
|
||||
}
|
||||
} else {
|
||||
toast.info("Another model is already loading", {
|
||||
|
|
@ -1891,23 +1834,128 @@ export function ChatPage({
|
|||
}
|
||||
return;
|
||||
}
|
||||
// Detach the prior staged pick (keeping its download) before rebinding, so
|
||||
// a second pick downloads alongside the first instead of cancelling it.
|
||||
detachStaged();
|
||||
store.stageModel({
|
||||
id: selection.id,
|
||||
isLora: selection.isLora,
|
||||
ggufVariant: selection.ggufVariant,
|
||||
isDownloaded: selection.isDownloaded,
|
||||
expectedBytes: selection.expectedBytes,
|
||||
nativePathToken: selection.nativePathToken,
|
||||
isGguf: selection.isGguf,
|
||||
isHubRepo: wantManagerDownload || undefined,
|
||||
autoLoad: store.loadOnSelection,
|
||||
const wantManagerStage =
|
||||
wantManagerDownload ||
|
||||
(selection.source === "hub" &&
|
||||
hasGgufSource(selection) &&
|
||||
!selection.isDownloaded);
|
||||
if (wantManagerStage) {
|
||||
setPendingHubAutoLoad((current) =>
|
||||
current &&
|
||||
current.selection.id === selection.id &&
|
||||
(current.selection.ggufVariant ?? null) ===
|
||||
(selection.ggufVariant ?? null) &&
|
||||
current.contextKey === chatContextKey &&
|
||||
current.originCheckpoint === store.params.checkpoint &&
|
||||
current.originGgufVariant === store.activeGgufVariant
|
||||
? current
|
||||
: {
|
||||
selection,
|
||||
contextKey: chatContextKey,
|
||||
originCheckpoint: store.params.checkpoint,
|
||||
originGgufVariant: store.activeGgufVariant,
|
||||
},
|
||||
);
|
||||
return;
|
||||
}
|
||||
setPendingHubAutoLoad(null);
|
||||
const previousConfig = currentRuntimePerModelConfig({
|
||||
includeMaxSeqLength: true,
|
||||
});
|
||||
const hasAppliedConfig = applyModelLoadConfigToRuntime(
|
||||
selection.config ?? rememberedConfigFor(selection),
|
||||
);
|
||||
await selectModel({
|
||||
...selection,
|
||||
...(hasAppliedConfig ? { keepSpeculative: true } : {}),
|
||||
previousConfig,
|
||||
});
|
||||
},
|
||||
[detachStaged, selectModel, loadingModel],
|
||||
[selectModel, loadingModel, rememberedConfigFor, chatContextKey],
|
||||
);
|
||||
useRepoDownload({
|
||||
kind: DOWNLOAD_KIND.MODEL,
|
||||
repoId: pendingHubAutoLoad?.selection.id ?? "__hub_autoload_idle__",
|
||||
activeVariant: pendingHubAutoLoad?.selection.ggufVariant ?? null,
|
||||
onComplete: (variant) => {
|
||||
const pending = pendingHubAutoLoad;
|
||||
if (
|
||||
!pending ||
|
||||
(pending.selection.ggufVariant ?? null) !== (variant ?? null)
|
||||
) {
|
||||
return;
|
||||
}
|
||||
setPendingHubAutoLoad(null);
|
||||
const store = useChatRuntimeStore.getState();
|
||||
if (
|
||||
!active ||
|
||||
pending.contextKey !== chatContextKey ||
|
||||
normalizeModelRef(pending.originCheckpoint) !==
|
||||
normalizeModelRef(store.params.checkpoint) ||
|
||||
pending.originGgufVariant !== store.activeGgufVariant
|
||||
) {
|
||||
return;
|
||||
}
|
||||
void stageOrLoad({ ...pending.selection, isDownloaded: true });
|
||||
},
|
||||
onError: (variant) => {
|
||||
if (
|
||||
pendingHubAutoLoad &&
|
||||
(pendingHubAutoLoad.selection.ggufVariant ?? null) === (variant ?? null)
|
||||
) {
|
||||
setPendingHubAutoLoad(null);
|
||||
}
|
||||
},
|
||||
onCancelled: (variant) => {
|
||||
if (
|
||||
pendingHubAutoLoad &&
|
||||
(pendingHubAutoLoad.selection.ggufVariant ?? null) === (variant ?? null)
|
||||
) {
|
||||
setPendingHubAutoLoad(null);
|
||||
}
|
||||
},
|
||||
});
|
||||
useEffect(() => {
|
||||
const pending = pendingHubAutoLoad;
|
||||
if (!pending) return;
|
||||
let active = true;
|
||||
void (async () => {
|
||||
const outcome = await downloadManager.requestStart({
|
||||
kind: DOWNLOAD_KIND.MODEL,
|
||||
repoId: pending.selection.id,
|
||||
variant: pending.selection.ggufVariant ?? null,
|
||||
expectedBytes: pending.selection.expectedBytes ?? 0,
|
||||
});
|
||||
if (!active) return;
|
||||
if (outcome === "started") {
|
||||
toast.info("Downloading model", {
|
||||
description: "It'll load automatically once the download finishes.",
|
||||
});
|
||||
return;
|
||||
}
|
||||
if (outcome === "conflict") {
|
||||
// Keep pendingHubAutoLoad bound so this surface's cleanup does not wipe
|
||||
// the conflict just recorded by requestStart (which the toast points the
|
||||
// user to); resolving it from the Hub completes the download and this
|
||||
// surface's onComplete auto-loads, mirroring the "started" branch.
|
||||
toast.info("Resume this download from the Hub", {
|
||||
description:
|
||||
"An earlier partial download used a different transport. Open the Hub tab to resume or restart it.",
|
||||
});
|
||||
return;
|
||||
}
|
||||
if (outcome === "busy") {
|
||||
toast.info("Download already in progress", {
|
||||
description:
|
||||
"Another download for this model is still running. Reselect it once that finishes to load it.",
|
||||
});
|
||||
}
|
||||
setPendingHubAutoLoad((current) => (current === pending ? null : current));
|
||||
})();
|
||||
return () => {
|
||||
active = false;
|
||||
};
|
||||
}, [pendingHubAutoLoad]);
|
||||
const loadNativeModelIntent = useCallback(
|
||||
async (intent: NativeIntent, loadingDescription: string) => {
|
||||
const label =
|
||||
|
|
@ -1915,6 +1963,7 @@ export function ChatPage({
|
|||
await stageOrLoad({
|
||||
id: label,
|
||||
nativePathToken: intent.path.token,
|
||||
nativePathExpiresAtMs: intent.path.expiresAtMs ?? null,
|
||||
isDownloaded: true,
|
||||
loadingDescription,
|
||||
forceReload: true,
|
||||
|
|
@ -1965,28 +2014,20 @@ export function ChatPage({
|
|||
const handleCheckpointChange = useCallback(
|
||||
(
|
||||
value: string,
|
||||
meta?: {
|
||||
source?: string;
|
||||
isLora: boolean;
|
||||
ggufVariant?: string;
|
||||
isDownloaded?: boolean;
|
||||
expectedBytes?: number;
|
||||
isGguf?: boolean;
|
||||
},
|
||||
meta?: ModelSelectorChangeMeta,
|
||||
) => {
|
||||
const store = useChatRuntimeStore.getState();
|
||||
const currentCheckpoint = store.params.checkpoint;
|
||||
const currentVariant = store.activeGgufVariant;
|
||||
if (
|
||||
!value ||
|
||||
(value === currentCheckpoint &&
|
||||
(meta?.ggufVariant ?? null) === (currentVariant ?? null))
|
||||
)
|
||||
if (!value) return;
|
||||
setPendingHubAutoLoad(null);
|
||||
const isSameLoadedModel =
|
||||
value === currentCheckpoint &&
|
||||
(meta?.ggufVariant ?? null) === (currentVariant ?? null);
|
||||
if (isSameLoadedModel && !meta?.forceReload) {
|
||||
return;
|
||||
}
|
||||
if (meta?.source === "external" || isExternalModelId(value)) {
|
||||
// Switching to an external model abandons any staged local pick: cancel
|
||||
// its download too (setCheckpoint below only clears the pending + knobs).
|
||||
abandonStaged();
|
||||
const selectedExternal = parseExternalModelId(value);
|
||||
const selectedProvider = selectedExternal
|
||||
? externalProvidersForChat.find(
|
||||
|
|
@ -2087,6 +2128,7 @@ export function ChatPage({
|
|||
ggufMaxContextLength: null,
|
||||
ggufNativeContextLength: null,
|
||||
activeNativePathToken: null,
|
||||
activeNativePathExpiresAtMs: null,
|
||||
// Clear previous-model counters, else the relaxed external-provider
|
||||
// render gate shows stale stats until the next completion.
|
||||
contextUsage: null,
|
||||
|
|
@ -2158,19 +2200,18 @@ export function ChatPage({
|
|||
source: meta?.source,
|
||||
isLora: meta?.isLora,
|
||||
ggufVariant: meta?.ggufVariant,
|
||||
isDownloaded: meta?.isDownloaded,
|
||||
isDownloaded: meta?.isDownloaded || isSameLoadedModel,
|
||||
expectedBytes: meta?.expectedBytes,
|
||||
isGguf: meta?.isGguf,
|
||||
config: meta?.config,
|
||||
nativePathToken: meta?.nativePathToken,
|
||||
nativePathExpiresAtMs: meta?.nativePathExpiresAtMs,
|
||||
forceReload: isSameLoadedModel || undefined,
|
||||
};
|
||||
// "Load on selection" off: stage the model and open settings so its
|
||||
// load knobs (tensor parallel, context length…) can be set, then it
|
||||
// loads once via the sheet's Load button. The currently loaded model
|
||||
// stays put until the user commits.
|
||||
await stageOrLoad(selection);
|
||||
})();
|
||||
},
|
||||
[
|
||||
abandonStaged,
|
||||
activeThreadId,
|
||||
externalProvidersForChat,
|
||||
modelsFromStore,
|
||||
|
|
@ -2178,6 +2219,45 @@ export function ChatPage({
|
|||
view,
|
||||
],
|
||||
);
|
||||
const handleReloadActiveModel = useCallback(
|
||||
(config: PerModelConfig) => {
|
||||
const checkpoint = inferenceParams.checkpoint;
|
||||
if (!checkpoint) return;
|
||||
const runtime = useChatRuntimeStore.getState();
|
||||
const nativeToken = runtime.activeNativePathToken;
|
||||
const nativeExpiry = runtime.activeNativePathExpiresAtMs;
|
||||
// A file-picked GGUF is reachable only via its native path token, which
|
||||
// the desktop host prunes after a TTL. Reusing an expired token makes the
|
||||
// reload fail with an opaque error, so prompt the user to re-select the
|
||||
// file instead.
|
||||
if (nativeToken && nativeExpiry != null && Date.now() >= nativeExpiry) {
|
||||
toast.error("This local model file's access has expired.", {
|
||||
description: "Re-select the model file to reload it.",
|
||||
});
|
||||
return;
|
||||
}
|
||||
handleCheckpointChange(checkpoint, {
|
||||
source: "local",
|
||||
isLora: activeModelIsLora,
|
||||
ggufVariant: activeGgufVariant ?? undefined,
|
||||
// Without the native token the reload validates the display label as a
|
||||
// repo and fails.
|
||||
nativePathToken: nativeToken ?? undefined,
|
||||
nativePathExpiresAtMs: nativeExpiry,
|
||||
isGguf: activeModelIsGguf,
|
||||
isDownloaded: true,
|
||||
config,
|
||||
forceReload: true,
|
||||
});
|
||||
},
|
||||
[
|
||||
inferenceParams.checkpoint,
|
||||
activeGgufVariant,
|
||||
activeModelIsLora,
|
||||
activeModelIsGguf,
|
||||
handleCheckpointChange,
|
||||
],
|
||||
);
|
||||
const handleEject = useCallback(() => {
|
||||
void (async () => {
|
||||
if (await ejectModel()) {
|
||||
|
|
@ -2446,12 +2526,27 @@ export function ChatPage({
|
|||
|
||||
const state = useChatRuntimeStore.getState();
|
||||
const targetLora = pickBestLoraForBase(state.loras, handoff.baseModel);
|
||||
const selectWithConfig = async (
|
||||
selection: Pick<SelectedModelInput, "id" | "isLora">,
|
||||
) => {
|
||||
const previousConfig = currentRuntimePerModelConfig({
|
||||
includeMaxSeqLength: true,
|
||||
});
|
||||
const hasAppliedConfig = applyModelLoadConfigToRuntime(
|
||||
rememberedConfigFor(selection),
|
||||
);
|
||||
await selectModelRef.current({
|
||||
...selection,
|
||||
...(hasAppliedConfig ? { keepSpeculative: true } : {}),
|
||||
previousConfig,
|
||||
});
|
||||
};
|
||||
if (targetLora) {
|
||||
console.info("[chat-handoff] loading lora", {
|
||||
id: targetLora.id,
|
||||
baseModel: targetLora.baseModel,
|
||||
});
|
||||
await selectModelRef.current({ id: targetLora.id, isLora: true });
|
||||
await selectWithConfig({ id: targetLora.id, isLora: true });
|
||||
if (canceled) return;
|
||||
useChatRuntimeStore.getState().setActiveThreadId(null);
|
||||
useChatRuntimeStore.getState().setContextUsage(null);
|
||||
|
|
@ -2468,10 +2563,7 @@ export function ChatPage({
|
|||
console.info("[chat-handoff] no lora match, loading base", {
|
||||
id: handoff.baseModel,
|
||||
});
|
||||
await selectModelRef.current({
|
||||
id: handoff.baseModel,
|
||||
isLora: false,
|
||||
});
|
||||
await selectWithConfig({ id: handoff.baseModel, isLora: false });
|
||||
if (canceled) return;
|
||||
} else {
|
||||
console.warn("[chat-handoff] no lora/base match found", {
|
||||
|
|
@ -2491,7 +2583,7 @@ export function ChatPage({
|
|||
return () => {
|
||||
canceled = true;
|
||||
};
|
||||
}, [active, navigate]);
|
||||
}, [active, navigate, rememberedConfigFor]);
|
||||
|
||||
const tourSteps = useMemo(
|
||||
() =>
|
||||
|
|
@ -2580,6 +2672,8 @@ export function ChatPage({
|
|||
externalModels={externalModels}
|
||||
value={inferenceParams.checkpoint}
|
||||
activeGgufVariant={activeGgufVariant}
|
||||
activeModelConfig={activeModelConfig}
|
||||
activeGgufContextLength={ggufContextLength}
|
||||
onValueChange={handleCheckpointChange}
|
||||
onEject={handleEject}
|
||||
onFoldersChange={refreshLocalModels}
|
||||
|
|
@ -2633,7 +2727,12 @@ export function ChatPage({
|
|||
<NativeModelChip
|
||||
intent={pendingNativeModelIntent}
|
||||
nativeReadsDisabled={!nativePathLeasesSupported}
|
||||
onLoad={(selection) => stageOrLoad(selection)}
|
||||
onLoad={() =>
|
||||
loadNativeModelIntent(
|
||||
pendingNativeModelIntent,
|
||||
"Loading selected local GGUF model.",
|
||||
)
|
||||
}
|
||||
/>
|
||||
) : null}
|
||||
{loadingModel && loadToastDismissed ? (
|
||||
|
|
@ -2790,13 +2889,22 @@ export function ChatPage({
|
|||
open={active && settingsOpen}
|
||||
onOpenChange={(open) => {
|
||||
setSettingsOpen(open);
|
||||
// Closing the sheet abandons a staged (not-yet-loaded) pick: cancel its
|
||||
// download and revert the staged knobs so nothing lingers as a dirty
|
||||
// edit (or a background download) on the loaded model.
|
||||
if (!open) abandonStaged();
|
||||
}}
|
||||
params={inferenceParams}
|
||||
onParamsChange={setInferenceParams}
|
||||
modelConfig={
|
||||
view.mode !== "compare" && activeModelConfig && !modelLoading ? (
|
||||
<SidebarModelConfig
|
||||
modelId={inferenceParams.checkpoint}
|
||||
ggufVariant={activeGgufVariant ?? null}
|
||||
isGguf={activeModelIsGguf}
|
||||
nativeContextLength={ggufNativeContextLength}
|
||||
loadedContextLength={ggufContextLength}
|
||||
loadedConfig={activeModelConfig}
|
||||
onReload={handleReloadActiveModel}
|
||||
/>
|
||||
) : null
|
||||
}
|
||||
isExternalModel={isExternalModel}
|
||||
providerCapabilities={activeProviderCapabilities}
|
||||
activeExternalProvider={activeExternalProvider}
|
||||
|
|
@ -2808,67 +2916,6 @@ export function ChatPage({
|
|||
);
|
||||
}}
|
||||
externalProviderType={activeExternalProviderType}
|
||||
loadingModel={loadingModel}
|
||||
onReloadModel={() => {
|
||||
const state = useChatRuntimeStore.getState();
|
||||
if (state.params.checkpoint) {
|
||||
selectModel({
|
||||
id: state.params.checkpoint,
|
||||
ggufVariant: state.activeGgufVariant ?? undefined,
|
||||
// A native (drag-drop / picked) GGUF's checkpoint is only a display
|
||||
// label, so the reload needs its path token to re-mint a lease --
|
||||
// else applying the now-exposed GPU/context controls can't resolve
|
||||
// the file. Null for non-native loads, which reload by id as before.
|
||||
nativePathToken: state.activeNativePathToken ?? undefined,
|
||||
forceReload: true,
|
||||
isDownloaded: true,
|
||||
loadingDescription: "Reloading with updated chat template.",
|
||||
});
|
||||
}
|
||||
}}
|
||||
onLoadPendingModel={() => {
|
||||
const pending = useChatRuntimeStore.getState().pendingSelection;
|
||||
if (!pending) return;
|
||||
const keyAtLoad = chatContextKey;
|
||||
// forceReload: the staged model isn't loaded yet, so bypass the
|
||||
// same-checkpoint dedupe. keepSpeculative: honor the speculative mode
|
||||
// set on the sidebar.
|
||||
void selectModel({
|
||||
...pending,
|
||||
forceReload: true,
|
||||
keepSpeculative: true,
|
||||
throwOnError: true,
|
||||
}).catch(() => {
|
||||
// Recoverable failure (expired token, gated repo, OOM…): the pick is
|
||||
// cleared only on success, so it normally stays staged with edited
|
||||
// knobs intact — nothing to restore.
|
||||
const store = useChatRuntimeStore.getState();
|
||||
// Still staged (this pick, or a newer one queued meanwhile): leave it.
|
||||
if (store.pendingSelection) return;
|
||||
// Cleared mid-load (sheet closed / switched chats). Re-stage only if
|
||||
// the staged-load is still wanted: same chat context, sheet still
|
||||
// open, page still mounted.
|
||||
const stillWanted =
|
||||
mountedRef.current &&
|
||||
store.settingsPanelOpen &&
|
||||
chatContextKeyRef.current === keyAtLoad;
|
||||
if (stillWanted) {
|
||||
store.setPendingSelection(pending);
|
||||
} else {
|
||||
// Abandoned (closed the sheet / switched chats / left chat): drop
|
||||
// the orphaned staged knob edits so they don't linger as dirty
|
||||
// settings over the loaded model.
|
||||
store.resetModelSettingsToLoaded();
|
||||
}
|
||||
});
|
||||
}}
|
||||
stagedDownloadFraction={stagedDownload.progress?.fraction ?? null}
|
||||
onCancelStagedDownload={() =>
|
||||
stagedDownload.cancelDownload(
|
||||
useChatRuntimeStore.getState().pendingSelection?.ggufVariant ??
|
||||
null,
|
||||
)
|
||||
}
|
||||
/>
|
||||
</div>
|
||||
</ChatActiveContext.Provider>
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load diff
|
|
@ -32,14 +32,13 @@ import {
|
|||
GPU_LAYERS_AUTO,
|
||||
isLocalModelPath,
|
||||
loadedGpuMemoryFields,
|
||||
loadedGpuMemoryFieldsUnlessStaged,
|
||||
pendingSelectionMatches,
|
||||
persistGpuMemoryModeOnLoad,
|
||||
readPersistedSpeculativeType,
|
||||
reconcilePersistedGpuIds,
|
||||
resolveToolsEnabledOnLoad,
|
||||
saveSpeculativeType,
|
||||
useChatRuntimeStore,
|
||||
type LoadingModelPick,
|
||||
type ReasoningEffort,
|
||||
} from "../stores/chat-runtime-store";
|
||||
import { clampReasoningEffortToLevels } from "../provider-capabilities";
|
||||
|
|
@ -61,7 +60,10 @@ import {
|
|||
isMultimodalResponse,
|
||||
} from "../types/api";
|
||||
import { isExternalModelId } from "../external-providers";
|
||||
import { cancelStagedModelDownload } from "@/features/hub";
|
||||
import {
|
||||
applyPerModelConfigToRuntime,
|
||||
type PerModelConfig,
|
||||
} from "@/features/model-picker";
|
||||
import type {
|
||||
ChatLoraSummary,
|
||||
ChatModelSummary,
|
||||
|
|
@ -81,14 +83,16 @@ export type SelectedModelInput = {
|
|||
expectedBytes?: number;
|
||||
forceReload?: boolean;
|
||||
nativePathToken?: string;
|
||||
nativePathExpiresAtMs?: number | null;
|
||||
/** Direct local .gguf file (no HF variant / native token) — still a GGUF
|
||||
* source, so the staging flow treats it as one. */
|
||||
isGguf?: boolean;
|
||||
throwOnError?: boolean;
|
||||
/** Keep the current speculative-decoding choice across the model switch
|
||||
* instead of resetting it to the standing preference. Set by the deferred
|
||||
* ("Load on selection") Load, where the user picked it for this model. */
|
||||
* instead of resetting it to the standing preference. */
|
||||
keepSpeculative?: boolean;
|
||||
config?: PerModelConfig;
|
||||
previousConfig?: PerModelConfig;
|
||||
};
|
||||
|
||||
// Approved fingerprints by checkpoint, so a rollback after a failed switch can resend
|
||||
|
|
@ -347,6 +351,18 @@ export async function resyncInferenceStatusAfterServerModelChange(): Promise<voi
|
|||
await syncInferenceStatusToStore();
|
||||
}
|
||||
|
||||
function pickOf(info: {
|
||||
id: string;
|
||||
ggufVariant?: string | null;
|
||||
nativePathToken?: string | null;
|
||||
}): LoadingModelPick {
|
||||
return {
|
||||
id: info.id,
|
||||
ggufVariant: info.ggufVariant ?? null,
|
||||
nativePathToken: info.nativePathToken ?? null,
|
||||
};
|
||||
}
|
||||
|
||||
export function useChatModelRuntime() {
|
||||
const params = useChatRuntimeStore((state) => state.params);
|
||||
const models = useChatRuntimeStore((state) => state.models);
|
||||
|
|
@ -385,12 +401,16 @@ export function useChatModelRuntime() {
|
|||
}, []);
|
||||
|
||||
const resetLoadingUi = useCallback(() => {
|
||||
const inFlight = loadingModelRef.current;
|
||||
setLoadingModel(null);
|
||||
setLoadProgress(null);
|
||||
loadingModelRef.current = null;
|
||||
loadAbortRef.current = null;
|
||||
loadToastIdRef.current = null;
|
||||
setLoadToastDismissedState(false);
|
||||
if (inFlight) {
|
||||
useChatRuntimeStore.getState().clearLoadingModelPick(pickOf(inFlight));
|
||||
}
|
||||
if (!cancelUnloadPendingRef.current) {
|
||||
useChatRuntimeStore.getState().setModelLoading(false);
|
||||
}
|
||||
|
|
@ -424,6 +444,7 @@ export function useChatModelRuntime() {
|
|||
loadAbortRef.current?.abort();
|
||||
loadAbortRef.current = null;
|
||||
loadingModelRef.current = null;
|
||||
useChatRuntimeStore.getState().clearLoadingModelPick(pickOf(model));
|
||||
const tid = loadToastIdRef.current;
|
||||
loadToastIdRef.current = null;
|
||||
setLoadingModel(null);
|
||||
|
|
@ -460,45 +481,36 @@ export function useChatModelRuntime() {
|
|||
typeof selection === "string" ? false : selection.forceReload ?? false;
|
||||
const nativePathToken =
|
||||
typeof selection === "string" ? undefined : selection.nativePathToken;
|
||||
const nativePathExpiresAtMs =
|
||||
typeof selection === "string"
|
||||
? null
|
||||
: selection.nativePathExpiresAtMs ?? null;
|
||||
const explicitIsGguf =
|
||||
typeof selection === "string" ? undefined : selection.isGguf;
|
||||
const throwOnError =
|
||||
typeof selection === "string" ? false : selection.throwOnError ?? false;
|
||||
const keepSpeculative =
|
||||
typeof selection === "string" ? false : selection.keepSpeculative ?? false;
|
||||
// Picking/loading any model abandons a staged (deferred) selection.
|
||||
// Before the early-returns below so even a no-op re-select clears the
|
||||
// stage.
|
||||
const staged = useChatRuntimeStore.getState().pendingSelection;
|
||||
if (staged) {
|
||||
// Loading a DIFFERENT model abandons this stage. Loading the staged pick
|
||||
// ITSELF keeps it so the sidebar can show its load settings (context, KV
|
||||
// cache, …) during the load. Cleared on success below; on failure it's
|
||||
// left staged so the user can retry (see onLoadPendingModel's catch).
|
||||
const loadingStagedPick = pendingSelectionMatches(staged, {
|
||||
id: modelId,
|
||||
ggufVariant,
|
||||
nativePathToken,
|
||||
});
|
||||
if (!loadingStagedPick) {
|
||||
cancelStagedModelDownload(staged);
|
||||
useChatRuntimeStore.getState().setPendingSelection(null);
|
||||
}
|
||||
}
|
||||
const currentVariant = useChatRuntimeStore.getState().activeGgufVariant;
|
||||
if (!forceReload && (!modelId || (params.checkpoint === modelId && (ggufVariant ?? null) === (currentVariant ?? null)))) {
|
||||
if (typeof selection !== "string" && selection.previousConfig) {
|
||||
applyPerModelConfigToRuntime(selection.previousConfig);
|
||||
}
|
||||
return;
|
||||
}
|
||||
// A load is already in flight. If it's this exact pick (id + GGUF variant +
|
||||
// native path token), ignore the duplicate click. If it's a DIFFERENT model
|
||||
// -- crucially including a different GGUF variant of the same repo, which the
|
||||
// old id+token-only guard wrongly treated as a duplicate and silently
|
||||
// no-op'd -- don't start a second concurrent load (the load path has no clean
|
||||
// supersession) and don't silently swallow the request: surface it so the
|
||||
// user knows to wait for, or cancel, the in-flight load. Centralized here so
|
||||
// every entry point is covered, not just the staged Load button.
|
||||
const inFlightLoad = loadingModelRef.current;
|
||||
// A load is already in flight. If it's this exact pick (id + variant + token),
|
||||
// ignore the duplicate click. If it's a DIFFERENT model (including a different
|
||||
// GGUF variant of the same repo, which the old id+token guard wrongly treated
|
||||
// as a duplicate), don't start a second concurrent load and don't swallow the
|
||||
// request: surface it so the user waits or cancels. Centralized here so every
|
||||
// entry point is covered, not just the staged Load button.
|
||||
const inFlightLoad =
|
||||
loadingModelRef.current ??
|
||||
useChatRuntimeStore.getState().loadingModelPick;
|
||||
if (inFlightLoad) {
|
||||
if (typeof selection !== "string" && selection.previousConfig) {
|
||||
applyPerModelConfigToRuntime(selection.previousConfig);
|
||||
}
|
||||
const loadingSamePick =
|
||||
inFlightLoad.id === modelId &&
|
||||
(inFlightLoad.ggufVariant ?? null) === (ggufVariant ?? null) &&
|
||||
|
|
@ -526,7 +538,11 @@ export function useChatModelRuntime() {
|
|||
// native model intents only grant .gguf files), but its id is a display
|
||||
// label that need not end in ".gguf" -- without this, Manual + Auto
|
||||
// layers would pin the UI context instead of letting --fit size it.
|
||||
const isGguf = explicitIsGguf ?? model?.isGguf ?? nativePathToken != null;
|
||||
const isGguf =
|
||||
explicitIsGguf ??
|
||||
(ggufVariant != null ||
|
||||
nativePathToken != null ||
|
||||
model?.isGguf === true);
|
||||
const loraIsAdapter = lora?.exportType === "lora";
|
||||
const isLora =
|
||||
explicitIsLora ?? model?.isLora ?? loraIsAdapter ?? false;
|
||||
|
|
@ -575,6 +591,7 @@ export function useChatModelRuntime() {
|
|||
};
|
||||
setLoadingModel(loadInfo);
|
||||
useChatRuntimeStore.getState().setModelLoading(true);
|
||||
useChatRuntimeStore.getState().setLoadingModelPick(pickOf(loadInfo));
|
||||
setLoadProgress(
|
||||
isDownloaded || isCachedLora
|
||||
? { percent: null, label: null, phase: "starting" }
|
||||
|
|
@ -600,26 +617,33 @@ export function useChatModelRuntime() {
|
|||
|| previousVariant != null
|
||||
|| previousActiveNativePathToken != null
|
||||
|| (previousCheckpoint?.toLowerCase().endsWith(".gguf") ?? false);
|
||||
// Roll back to the previous model's own context. previousConfig was
|
||||
// snapshotted before this load pre-applied the next model's config, so
|
||||
// params.maxSeqLength may already be the next model's; use it only when
|
||||
// no snapshot exists.
|
||||
const previousMaxSeqLength =
|
||||
(typeof selection !== "string"
|
||||
? selection.previousConfig?.maxSeqLength
|
||||
: null) ?? maxSeqLength;
|
||||
// Respect the rolled-back model's auto-layers mode: a Manual+Auto model
|
||||
// with an unpinned (auto) context must reload with 0 (so --fit
|
||||
// re-auto-sizes), not the positive context it happened to pick (which
|
||||
// the backend would treat as a pin).
|
||||
// with an unpinned context must reload with 0 (so --fit re-auto-sizes),
|
||||
// not the positive context it picked (which the backend treats as a pin).
|
||||
const rollbackMaxSeqLength = resolveFitMaxSeqLength(
|
||||
previousIsGguf,
|
||||
stateBeforeUnload.loadedGpuMemoryMode ?? "auto",
|
||||
stateBeforeUnload.loadedGpuLayers ?? GPU_LAYERS_AUTO,
|
||||
stateBeforeUnload.loadedCustomContextLength,
|
||||
previousIsGguf ? (stateBeforeUnload.ggufContextLength ?? 0) : maxSeqLength,
|
||||
previousIsGguf
|
||||
? (stateBeforeUnload.ggufContextLength ?? 0)
|
||||
: previousMaxSeqLength,
|
||||
);
|
||||
const hfToken = stateBeforeUnload.hfToken || null;
|
||||
const previousModelRequiresTrustRemoteCode =
|
||||
stateBeforeUnload.modelRequiresTrustRemoteCode;
|
||||
const previousActiveNativePathExpiresAtMs =
|
||||
stateBeforeUnload.activeNativePathExpiresAtMs;
|
||||
// Snapshot the load settings at click time, before the awaits below
|
||||
// (validation, the trust dialog, unload). For a staged Load these knobs
|
||||
// stay editable and a sheet-close revert (abandonStagedModel) can fire
|
||||
// mid-load; reading them live just before loadModel would let the load
|
||||
// use post-click values. The model-switch speculative reset below
|
||||
// updates this snapshot in lock-step so non-staged loads are unchanged.
|
||||
// (validation, the trust dialog, unload).
|
||||
const loadChatTemplateOverride = stateBeforeUnload.chatTemplateOverride;
|
||||
const loadKvCacheDtype = stateBeforeUnload.kvCacheDtype;
|
||||
// gpuMemoryMode is a standing preference (kept across a model switch);
|
||||
|
|
@ -657,14 +681,11 @@ export function useChatModelRuntime() {
|
|||
// context can exceed maxSeqLength, so sizing on raw maxSeqLength could
|
||||
// pass, unload, then have /load refuse it. Uses the click-time
|
||||
// snapshot (same values loadModel uses below), so the two agree.
|
||||
// Mirror what /load does on a cross-model switch: the reset below
|
||||
// clears the per-model Auto-layers context pin + GPU pick, and
|
||||
// Manual+Auto sizes context through resolveFitMaxSeqLength.
|
||||
// gpuMemoryMode is a standing preference, kept across the switch.
|
||||
// A same-repo quant switch (same checkpoint, different gguf_variant)
|
||||
// is a different model for per-model knobs: the pinned context,
|
||||
// gpuLayers, GPU pick, and MoE offload are scoped per variant, so
|
||||
// treat a variant change like a model switch and re-baseline them.
|
||||
// Mirror /load on a cross-model switch: the reset below clears the
|
||||
// per-model Auto-layers context pin + GPU pick; gpuMemoryMode is a
|
||||
// standing preference kept across the switch. A same-repo quant switch
|
||||
// (different gguf_variant) is a different model for per-model knobs
|
||||
// (context/gpuLayers/pick/MoE are per variant), so re-baseline them too.
|
||||
const switchingModelOrVariant =
|
||||
currentCheckpoint !== modelId ||
|
||||
(loadActiveGgufVariant ?? null) !== (ggufVariant ?? null);
|
||||
|
|
@ -867,7 +888,11 @@ export function useChatModelRuntime() {
|
|||
// The load applied this spec mode, so persist the user's standing
|
||||
// preference now (the requested intent, not the resolved echo;
|
||||
// saveSpeculativeType keeps only the universal auto/ngram/off).
|
||||
saveSpeculativeType(loadSpeculativeType);
|
||||
// Skip for a per-model config (keepSpeculative): that choice is
|
||||
// model-specific and must not overwrite the global default.
|
||||
if (!keepSpeculative) {
|
||||
saveSpeculativeType(loadSpeculativeType);
|
||||
}
|
||||
// Persist the GPU Memory mode only on a successful load (not on
|
||||
// dropdown change), so an abandoned selection doesn't stick.
|
||||
persistGpuMemoryModeOnLoad(loadResponse, loadGpuMemoryMode);
|
||||
|
|
@ -907,7 +932,9 @@ export function useChatModelRuntime() {
|
|||
? (loadResponse.native_context_length ?? null)
|
||||
: null;
|
||||
// Keep an explicit Manual+Auto context pin (so a later Apply doesn't
|
||||
// revert it to Auto); other cases baseline on ggufContextLength.
|
||||
// revert it to Auto) and retain the user's requested context so
|
||||
// re-open/re-save keeps the intended override, not the backend's
|
||||
// auto-fit context; null stays null.
|
||||
const keepCustomCtx = resolveManualAutoCtxPin(
|
||||
loadGpuMemoryMode,
|
||||
loadGpuLayers,
|
||||
|
|
@ -978,6 +1005,9 @@ export function useChatModelRuntime() {
|
|||
loadedIsMultimodal: isMultimodalResponse(loadResponse),
|
||||
loadedIsDiffusion: loadResponse.is_diffusion ?? false,
|
||||
activeNativePathToken: nativePathToken ?? null,
|
||||
activeNativePathExpiresAtMs: nativePathToken
|
||||
? nativePathExpiresAtMs
|
||||
: null,
|
||||
});
|
||||
// Unlock attach menus for capabilities the catalog entry lacked.
|
||||
syncModelCapabilities(modelId, loadResponse);
|
||||
|
|
@ -1031,25 +1061,6 @@ export function useChatModelRuntime() {
|
|||
recordLastLocalModelLoad({ id: modelId, kind: "model" });
|
||||
}
|
||||
}
|
||||
// A successful load owns the shared (pick-unscoped) settings fields,
|
||||
// so any surviving stage is stale: the just-loaded pick itself, or a
|
||||
// pick queued for a different model mid-load whose knobs this load
|
||||
// overwrote. Drop it. Only a DIFFERENT pick's download needs
|
||||
// cancelling; the loaded pick's is already consumed, and cancelling
|
||||
// it inside its post-complete linger window would flicker its card.
|
||||
const staleStage = useChatRuntimeStore.getState().pendingSelection;
|
||||
if (staleStage) {
|
||||
if (
|
||||
!pendingSelectionMatches(staleStage, {
|
||||
id: modelId,
|
||||
ggufVariant,
|
||||
nativePathToken,
|
||||
})
|
||||
) {
|
||||
cancelStagedModelDownload(staleStage);
|
||||
}
|
||||
useChatRuntimeStore.getState().setPendingSelection(null);
|
||||
}
|
||||
} catch (error) {
|
||||
// Skip rollback if user cancelled -- model is already being unloaded.
|
||||
if (abortCtrl.signal.aborted) throw error;
|
||||
|
|
@ -1091,8 +1102,9 @@ export function useChatModelRuntime() {
|
|||
// Restore the previous model in the split mode it was running,
|
||||
// not the default layer split.
|
||||
tensor_parallel: stateBeforeUnload.loadedTensorParallel ?? false,
|
||||
// Restore the previous model's GPU Memory placement, not backend defaults.
|
||||
gpu_memory_mode: stateBeforeUnload.loadedGpuMemoryMode ?? "auto",
|
||||
gpu_layers: stateBeforeUnload.loadedGpuLayers ?? -1,
|
||||
gpu_layers: stateBeforeUnload.loadedGpuLayers ?? GPU_LAYERS_AUTO,
|
||||
n_cpu_moe: stateBeforeUnload.loadedNCpuMoe ?? 0,
|
||||
tensor_split: stateBeforeUnload.loadedSplitRatio ?? undefined,
|
||||
gpu_ids: stateBeforeUnload.loadedGpuIds ?? undefined,
|
||||
|
|
@ -1102,28 +1114,27 @@ export function useChatModelRuntime() {
|
|||
);
|
||||
useChatRuntimeStore.setState({
|
||||
activeNativePathToken: previousActiveNativePathToken ?? null,
|
||||
// Restore the previous token's lease together with the token so a
|
||||
// rollback never pairs restored token A with failed load B's expiry.
|
||||
activeNativePathExpiresAtMs: previousActiveNativePathToken
|
||||
? (previousActiveNativePathExpiresAtMs ?? null)
|
||||
: null,
|
||||
// Restore the editable speculative knobs to the rolled-back
|
||||
// model's; the loaded baselines below come from its reload echo.
|
||||
speculativeType: stateBeforeUnload.loadedSpeculativeType ?? null,
|
||||
specDraftNMax: stateBeforeUnload.loadedSpecDraftNMax ?? null,
|
||||
loadedSpeculativeType: rollbackSpeculativeType,
|
||||
loadedSpecDraftNMax:
|
||||
rollbackResponse.spec_draft_n_max ?? null,
|
||||
loadedKvCacheDtype: rollbackResponse.cache_type_kv ?? null,
|
||||
loadedChatTemplateOverride:
|
||||
stateBeforeUnload.loadedChatTemplateOverride,
|
||||
// Re-baseline the GPU knobs from the rolled-back load's own
|
||||
// response (the shared seeding every load path uses): the
|
||||
// refresh() below can't do it, since the status reseed is
|
||||
// gated off while modelLoading is still true. A failed staged
|
||||
// Load stays staged for retry, so the staged hold applies.
|
||||
...loadedGpuMemoryFieldsUnlessStaged(rollbackResponse, {
|
||||
tensorParallel: rollbackResponse.tensor_parallel ?? false,
|
||||
loadedTensorParallel:
|
||||
rollbackResponse.tensor_parallel ?? false,
|
||||
// refresh() is held while modelLoading remains true, so
|
||||
// restore the rolled-back model's context pin directly.
|
||||
customContextLength:
|
||||
stateBeforeUnload.loadedCustomContextLength,
|
||||
}),
|
||||
...loadedGpuMemoryFields(rollbackResponse),
|
||||
tensorParallel: rollbackResponse.tensor_parallel ?? false,
|
||||
loadedTensorParallel:
|
||||
rollbackResponse.tensor_parallel ?? false,
|
||||
customContextLength:
|
||||
stateBeforeUnload.loadedCustomContextLength,
|
||||
loadedCustomContextLength:
|
||||
stateBeforeUnload.loadedCustomContextLength,
|
||||
});
|
||||
|
|
@ -1444,6 +1455,9 @@ export function useChatModelRuntime() {
|
|||
resetLoadingUi();
|
||||
}
|
||||
} catch (error) {
|
||||
if (typeof selection !== "string" && selection.previousConfig) {
|
||||
applyPerModelConfigToRuntime(selection.previousConfig);
|
||||
}
|
||||
if (abortCtrl.signal.aborted) return; // User cancelled, nothing to report
|
||||
resetLoadingUi();
|
||||
const message =
|
||||
|
|
@ -1474,6 +1488,13 @@ export function useChatModelRuntime() {
|
|||
if (!params.checkpoint) {
|
||||
return false;
|
||||
}
|
||||
const runtime = useChatRuntimeStore.getState();
|
||||
if (runtime.modelLoading || runtime.loadingModelPick) {
|
||||
toast.info("A model is loading", {
|
||||
description: "Wait for it to finish or cancel it first.",
|
||||
});
|
||||
return false;
|
||||
}
|
||||
setModelsError(null);
|
||||
if (isExternalModelId(params.checkpoint)) {
|
||||
clearCheckpoint();
|
||||
|
|
|
|||
|
|
@ -1,169 +0,0 @@
|
|||
// SPDX-License-Identifier: AGPL-3.0-only
|
||||
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
import { useCallback, useEffect } from "react";
|
||||
|
||||
import { useRepoDownload } from "@/features/hub/download-manager/use-repo-download";
|
||||
import type { DownloadJob } from "@/features/hub/download-manager/use-repo-download";
|
||||
import { useLatestRef } from "@/features/hub/hooks/use-latest-ref";
|
||||
|
||||
import { fetchGgufStagedMetadata } from "../api/chat-api";
|
||||
import {
|
||||
isPendingGguf,
|
||||
pendingSelectionMatches,
|
||||
useChatRuntimeStore,
|
||||
} from "../stores/chat-runtime-store";
|
||||
import type { PendingModelSelection } from "../stores/chat-runtime-store";
|
||||
|
||||
/**
|
||||
* Drives the deferred ("Load on selection" off) staging flow for a GGUF:
|
||||
* download the file if needed (HF repo) or read it in place (native drag-drop /
|
||||
* picked file), then read its header context length so the settings sheet can
|
||||
* show the real context slider before the single GPU load. The staged context
|
||||
* lands on `pendingSelection.contextLength` (scoped to the staged model, never
|
||||
* the loaded model's `ggufContextLength`). Returns the live download job so the
|
||||
* sheet can render progress / cancel. Mount once on the chat page.
|
||||
*/
|
||||
export function useStagedModelPreparation(opts?: {
|
||||
/** Load the cached file once an autoLoad pick's download completes. */
|
||||
onAutoLoad?: (pending: PendingModelSelection) => void;
|
||||
}): DownloadJob {
|
||||
const pendingId = useChatRuntimeStore((s) => s.pendingSelection?.id ?? null);
|
||||
const pendingVariant = useChatRuntimeStore(
|
||||
(s) => s.pendingSelection?.ggufVariant ?? null,
|
||||
);
|
||||
const pendingNativeToken = useChatRuntimeStore(
|
||||
(s) => s.pendingSelection?.nativePathToken ?? null,
|
||||
);
|
||||
// Only GGUF picks (HF variant or native file) have a header worth reading.
|
||||
const pendingIsGguf = useChatRuntimeStore((s) =>
|
||||
isPendingGguf(s.pendingSelection),
|
||||
);
|
||||
// Non-GGUF HF repos download a full snapshot (variant null) but have no header.
|
||||
const pendingIsHubRepo = useChatRuntimeStore(
|
||||
(s) => s.pendingSelection?.isHubRepo ?? false,
|
||||
);
|
||||
const pendingDownloaded = useChatRuntimeStore(
|
||||
(s) => s.pendingSelection?.isDownloaded ?? false,
|
||||
);
|
||||
// "Already probed" must key off layerCount / moeLayerCount, which only the
|
||||
// full header probe fills (it sets all three together, so either is a
|
||||
// reliable marker). contextLength alone can be list-seeded from
|
||||
// /gguf-variants, which returns no layer/MoE counts -- treating it as
|
||||
// complete would skip the probe and leave the GPU Layers slider at its 256
|
||||
// fallback and the MoE slider hidden until the model loads.
|
||||
const pendingHasMetadata = useChatRuntimeStore(
|
||||
(s) =>
|
||||
s.pendingSelection?.layerCount != null ||
|
||||
s.pendingSelection?.moeLayerCount != null,
|
||||
);
|
||||
const setPendingSelection = useChatRuntimeStore((s) => s.setPendingSelection);
|
||||
const onAutoLoadRef = useLatestRef(opts?.onAutoLoad);
|
||||
|
||||
// A failed or cancelled autoLoad download has no sheet to retry from, so drop
|
||||
// the staged pick rather than leave it waiting on a load that won't come.
|
||||
const handleAutoLoadAbort = useCallback((variant: string | null) => {
|
||||
const latest = useChatRuntimeStore.getState().pendingSelection;
|
||||
if (
|
||||
latest?.autoLoad &&
|
||||
(latest.ggufVariant ?? null) === (variant ?? null)
|
||||
) {
|
||||
useChatRuntimeStore.getState().abandonStagedModel();
|
||||
}
|
||||
}, []);
|
||||
|
||||
const fetchContextMetadata = useCallback(async () => {
|
||||
const current = useChatRuntimeStore.getState().pendingSelection;
|
||||
if (!current?.id || !isPendingGguf(current)) return;
|
||||
const { id, ggufVariant, nativePathToken } = current;
|
||||
try {
|
||||
const { contextLength, layerCount, moeLayerCount } =
|
||||
await fetchGgufStagedMetadata({
|
||||
model_path: id,
|
||||
gguf_variant: ggufVariant,
|
||||
hf_token: useChatRuntimeStore.getState().hfToken || null,
|
||||
nativePathToken,
|
||||
});
|
||||
// Apply only if the same model is still staged (the user may have switched
|
||||
// picks or loaded/cancelled while the request was in flight).
|
||||
const latest = useChatRuntimeStore.getState().pendingSelection;
|
||||
if (
|
||||
latest &&
|
||||
pendingSelectionMatches(latest, { id, ggufVariant, nativePathToken }) &&
|
||||
(contextLength != null || layerCount != null || moeLayerCount != null)
|
||||
) {
|
||||
setPendingSelection({
|
||||
...latest,
|
||||
contextLength,
|
||||
layerCount,
|
||||
moeLayerCount,
|
||||
});
|
||||
}
|
||||
} catch {
|
||||
// Leave metadata null: the context/MoE sliders stay hidden and the user
|
||||
// can still load (they fill in from the load response afterwards).
|
||||
}
|
||||
}, [setPendingSelection]);
|
||||
|
||||
const job = useRepoDownload({
|
||||
kind: "model",
|
||||
// useRepoDownload must be called unconditionally; an idle repo id keeps it
|
||||
// inert until something is staged.
|
||||
repoId: pendingId ?? "__staged_idle__",
|
||||
activeVariant: pendingVariant,
|
||||
onComplete: (variant) => {
|
||||
// autoLoad picks load the cached file now; staged picks read the header so
|
||||
// the sheet's context slider can show before a manual load.
|
||||
const latest = useChatRuntimeStore.getState().pendingSelection;
|
||||
if (
|
||||
latest?.autoLoad &&
|
||||
(latest.ggufVariant ?? null) === (variant ?? null)
|
||||
) {
|
||||
onAutoLoadRef.current?.(latest);
|
||||
return;
|
||||
}
|
||||
void fetchContextMetadata();
|
||||
},
|
||||
onError: handleAutoLoadAbort,
|
||||
onCancelled: handleAutoLoadAbort,
|
||||
});
|
||||
|
||||
// job.requestStartDownload's identity changes per render; hold it in a ref so
|
||||
// the staging effect re-runs only when the staged model itself changes.
|
||||
const startDownloadRef = useLatestRef(job.requestStartDownload);
|
||||
const fetchMetadataRef = useLatestRef(fetchContextMetadata);
|
||||
|
||||
useEffect(() => {
|
||||
// GGUF picks (header worth reading) and uncached non-GGUF hub repos (full
|
||||
// snapshot, no header) both run here; everything else is loaded directly.
|
||||
if (
|
||||
!pendingId ||
|
||||
(!pendingIsGguf && !pendingIsHubRepo) ||
|
||||
pendingHasMetadata
|
||||
) {
|
||||
return;
|
||||
}
|
||||
// Native files and already-downloaded HF files are local: read the header
|
||||
// now. Otherwise download first (a GGUF variant, or a null-variant snapshot
|
||||
// for a hub repo); onComplete then reads the header or auto-loads.
|
||||
if (pendingNativeToken || pendingDownloaded) {
|
||||
void fetchMetadataRef.current();
|
||||
} else {
|
||||
const expectedBytes =
|
||||
useChatRuntimeStore.getState().pendingSelection?.expectedBytes ?? 0;
|
||||
void startDownloadRef.current(pendingVariant, expectedBytes);
|
||||
}
|
||||
}, [
|
||||
pendingId,
|
||||
pendingVariant,
|
||||
pendingNativeToken,
|
||||
pendingIsGguf,
|
||||
pendingIsHubRepo,
|
||||
pendingDownloaded,
|
||||
pendingHasMetadata,
|
||||
startDownloadRef,
|
||||
fetchMetadataRef,
|
||||
]);
|
||||
|
||||
return job;
|
||||
}
|
||||
|
|
@ -3,20 +3,34 @@
|
|||
|
||||
export { ChatPage, validateChatSearch, type ChatSearch } from "./chat-page";
|
||||
export {
|
||||
addScanFolder,
|
||||
browseFolders,
|
||||
deleteChatAttachment,
|
||||
deleteFineTunedModel,
|
||||
fetchChatAttachmentBlob,
|
||||
fetchGgufStagedMetadata,
|
||||
getCachedModelPath,
|
||||
getInferenceStatus,
|
||||
listChatAttachments,
|
||||
listGgufVariants,
|
||||
listLocalModels,
|
||||
listRecommendedFolders,
|
||||
listScanFolders,
|
||||
loadModel,
|
||||
removeScanFolder,
|
||||
revealCachedModel,
|
||||
type BrowseFoldersResponse,
|
||||
type CachedGgufRepo,
|
||||
type CachedModelRepo,
|
||||
type ChatAttachmentPage,
|
||||
type ChatAttachmentRecord,
|
||||
type LocalModelInfo,
|
||||
type ScanFolderInfo,
|
||||
} from "./api/chat-api";
|
||||
export type { GgufVariantDetail } from "./types/api";
|
||||
export {
|
||||
ChatSettingsPanel,
|
||||
ParamSlider,
|
||||
defaultInferenceParams,
|
||||
type InferenceParams,
|
||||
type Preset,
|
||||
|
|
@ -25,6 +39,11 @@ export { useChatRuntimeStore } from "./stores/chat-runtime-store";
|
|||
export {
|
||||
CHAT_RAG_CAPTION_KEY,
|
||||
CHAT_RAG_OCR_KEY,
|
||||
normalizeSpeculativeType,
|
||||
readPersistedSpeculativeType,
|
||||
readPersistedGpuMemoryMode,
|
||||
reconcilePersistedGpuIds,
|
||||
GPU_LAYERS_AUTO,
|
||||
} from "./stores/chat-runtime-store";
|
||||
export {
|
||||
preferFullToolOutput,
|
||||
|
|
@ -46,9 +65,11 @@ export {
|
|||
} from "./hooks/use-chat-model-runtime";
|
||||
export {
|
||||
customProviderDisplayName,
|
||||
isCustomProviderType,
|
||||
isExternalModelId,
|
||||
parseExternalModelId,
|
||||
} from "./external-providers";
|
||||
export { ApiProviderLogo } from "./api-provider-logo";
|
||||
export { useExternalProvidersStore } from "./stores/external-providers-store";
|
||||
export { ChatSearchDialog } from "./components/chat-search-dialog";
|
||||
export { setTrainingCompareHandoff } from "./lib/training-compare-handoff";
|
||||
|
|
@ -62,8 +83,8 @@ export {
|
|||
useSelectedChatArtifact,
|
||||
} from "./artifacts/store";
|
||||
export {
|
||||
downloadChatExport,
|
||||
downloadArchivedChatExport,
|
||||
downloadChatExport,
|
||||
} from "./utils/export-chat-history";
|
||||
export {
|
||||
clearNewChatDraft,
|
||||
|
|
|
|||
|
|
@ -280,12 +280,9 @@ export function applyActiveModelStatusToStore(
|
|||
ggufContextLength: currentGgufContextLength,
|
||||
ggufMaxContextLength,
|
||||
ggufNativeContextLength,
|
||||
// A non-GGUF status must also drop a stale native-path token: without this the
|
||||
// isGguf OR (activeGgufVariant || activeNativePathToken || ggufContextLength)
|
||||
// stays true after switching from a native GGUF to a transformers model, so a
|
||||
// Codex-only detection would auto-select for a model its preflight rejects. A real
|
||||
// GGUF load reports is_gguf: true, so its token is preserved (the load path owns it).
|
||||
...(status.is_gguf ? {} : { activeNativePathToken: null }),
|
||||
...(status.is_gguf
|
||||
? {}
|
||||
: { activeNativePathToken: null, activeNativePathExpiresAtMs: null }),
|
||||
modelRequiresTrustRemoteCode: status.requires_trust_remote_code ?? false,
|
||||
defaultChatTemplate: nextDefaultChatTemplate,
|
||||
loadedIsMultimodal: isMultimodalResponse(status),
|
||||
|
|
@ -299,13 +296,11 @@ export function applyActiveModelStatusToStore(
|
|||
// model changed underneath this tab (auto-switch, another client), the
|
||||
// old model's baselines are stale and must adopt the new status.
|
||||
...(seedLoadParams &&
|
||||
prevState.pendingSelection == null &&
|
||||
(prevState.loadedSpeculativeType === null || hydratingExistingModel) && {
|
||||
speculativeType: currentSpecType,
|
||||
loadedSpeculativeType: currentSpecType,
|
||||
}),
|
||||
...(seedLoadParams &&
|
||||
prevState.pendingSelection == null &&
|
||||
status.spec_draft_n_max !== undefined &&
|
||||
(hydratingExistingModel ||
|
||||
(prevState.loadedSpecDraftNMax === null &&
|
||||
|
|
@ -314,14 +309,12 @@ export function applyActiveModelStatusToStore(
|
|||
loadedSpecDraftNMax: status.spec_draft_n_max ?? null,
|
||||
}),
|
||||
...(seedLoadParams &&
|
||||
prevState.pendingSelection == null &&
|
||||
status.cache_type_kv !== undefined &&
|
||||
(prevState.loadedKvCacheDtype === null || hydratingExistingModel) && {
|
||||
kvCacheDtype: status.cache_type_kv,
|
||||
loadedKvCacheDtype: status.cache_type_kv,
|
||||
}),
|
||||
...(seedLoadParams &&
|
||||
prevState.pendingSelection == null &&
|
||||
status.tensor_parallel !== undefined &&
|
||||
(prevState.loadedTensorParallel === null || hydratingExistingModel) && {
|
||||
tensorParallel: status.tensor_parallel,
|
||||
|
|
@ -331,7 +324,6 @@ export function applyActiveModelStatusToStore(
|
|||
// placement change. gpuStatusFields preserves dirty local edits in the last
|
||||
// case while advancing their loaded baselines.
|
||||
...(seedLoadParams &&
|
||||
prevState.pendingSelection == null &&
|
||||
(prevState.loadedGpuMemoryMode === null ||
|
||||
hydratingExistingModel ||
|
||||
gpuStatusChanged) &&
|
||||
|
|
|
|||
|
|
@ -79,6 +79,12 @@ import { KnowledgeBaseComposerButton } from "@/features/rag/components/knowledge
|
|||
import { NewProjectDialog } from "./components/new-project-dialog";
|
||||
import { useChatProjects } from "./hooks/use-chat-projects";
|
||||
import { confirmRemoteCodeIfNeeded } from "@/features/security";
|
||||
import {
|
||||
DEFAULT_MAX_SEQ_LENGTH,
|
||||
normalizeMaxSeqLength,
|
||||
resolveInitialConfig,
|
||||
type PerModelConfig,
|
||||
} from "@/features/model-picker";
|
||||
import {
|
||||
confirmTransformersUpgradeIfNeeded,
|
||||
useTransformersUpgradeDialogStore,
|
||||
|
|
@ -97,7 +103,7 @@ import {
|
|||
usePlusMenuPrefsStore,
|
||||
} from "./stores/plus-menu-prefs-store";
|
||||
import {
|
||||
loadedGpuMemoryFieldsUnlessStaged,
|
||||
loadedGpuMemoryFields,
|
||||
type ReasoningEffort,
|
||||
reconcilePersistedGpuIds,
|
||||
resolveLoadedSpeculativeSettings,
|
||||
|
|
@ -495,8 +501,24 @@ type CompareModelSelection = {
|
|||
id: string;
|
||||
isLora: boolean;
|
||||
ggufVariant?: string;
|
||||
config?: PerModelConfig;
|
||||
};
|
||||
|
||||
function cleanCompareChatTemplate(
|
||||
value: string | null | undefined,
|
||||
): string | null {
|
||||
return value?.trim() ? value : null;
|
||||
}
|
||||
|
||||
function resolveCompareSpecDraftNMax(
|
||||
speculativeType: string | null,
|
||||
value: number | null,
|
||||
): number | null {
|
||||
return speculativeType === "mtp" || speculativeType === "mtp+ngram"
|
||||
? value
|
||||
: null;
|
||||
}
|
||||
|
||||
// Tool icon plus an X overlay CSS reveals on hover when the pill is active.
|
||||
function PillGlyph({ children }: { children: ReactNode }) {
|
||||
return (
|
||||
|
|
@ -1023,15 +1045,12 @@ export function SharedComposer({
|
|||
// Generalized compare: load each model before dispatching to its side
|
||||
if (isGeneralizedCompare) {
|
||||
const store = useChatRuntimeStore.getState();
|
||||
const maxSeqLength = store.params.maxSeqLength;
|
||||
const trustRemoteCode = store.params.trustRemoteCode ?? false;
|
||||
const chatTemplateOverride = store.chatTemplateOverride;
|
||||
const effectiveChatTemplateOverride = chatTemplateOverride?.trim()
|
||||
? chatTemplateOverride
|
||||
: null;
|
||||
const fallbackTensorParallel = store.tensorParallel;
|
||||
const specSettings = resolveSpeculativeSettingsForLoad({
|
||||
usePersistedPreference: true,
|
||||
});
|
||||
let loadedFromConfig = false;
|
||||
|
||||
function modelDisplayName(id: string): string {
|
||||
const parts = id.split("/");
|
||||
|
|
@ -1058,7 +1077,6 @@ export function SharedComposer({
|
|||
// path: an early remember-restore can hold a stale cross-host pick that
|
||||
// /load would reject (the device cache is populated by send time).
|
||||
selectedGpuIds: reconcilePersistedGpuIds(store.selectedGpuIds),
|
||||
tensorParallel: store.tensorParallel,
|
||||
customContextLength: store.customContextLength,
|
||||
};
|
||||
// Set when an accepted transformers install unloaded the active model
|
||||
|
|
@ -1069,15 +1087,68 @@ export function SharedComposer({
|
|||
sel: CompareModelSelection,
|
||||
): Promise<string> {
|
||||
const currentStore = useChatRuntimeStore.getState();
|
||||
const config = sel.config ?? null;
|
||||
// This pane's effective config: an explicit selection config, else the
|
||||
// remembered store config for this model/quant (never the other pane's).
|
||||
// No saved config resolves to all-null defaults, so settings below fall
|
||||
// through to their session default.
|
||||
const resolved = config
|
||||
? { config, remembered: true }
|
||||
: resolveInitialConfig(sel.id, sel.ggufVariant ?? null);
|
||||
const ownConfig = resolved.config;
|
||||
const ownRemembered = resolved.remembered;
|
||||
// Mirror single-view resolveLoadMaxSeqLength: a GGUF pane with no explicit
|
||||
// context loads at native (0 -> n_ctx_train), not the session maxSeqLength,
|
||||
// which would silently shrink the shown context.
|
||||
const isGgufLoad =
|
||||
(sel.ggufVariant ?? null) != null ||
|
||||
sel.id.toLowerCase().endsWith(".gguf");
|
||||
// A non-GGUF pane with no saved maxSeqLength falls back to the app default,
|
||||
// not the active model's shared runtime snapshot: else comparing a saved
|
||||
// 128K model against an unconfigured one loads the latter at 128K and OOMs.
|
||||
const effectiveMaxSeqLength =
|
||||
ownConfig.customContextLength ??
|
||||
normalizeMaxSeqLength(ownConfig.maxSeqLength) ??
|
||||
(isGgufLoad ? 0 : DEFAULT_MAX_SEQ_LENGTH);
|
||||
const effectiveChatTemplateOverride = cleanCompareChatTemplate(
|
||||
ownConfig.chatTemplateOverride,
|
||||
);
|
||||
const effectiveSpeculativeType =
|
||||
ownConfig.speculativeType ?? specSettings.speculativeType;
|
||||
const effectiveSpecDraftNMax = ownRemembered
|
||||
? resolveCompareSpecDraftNMax(
|
||||
effectiveSpeculativeType,
|
||||
ownConfig.specDraftNMax,
|
||||
)
|
||||
: specSettings.specDraftNMax;
|
||||
const effectiveTensorParallel = ownRemembered
|
||||
? ownConfig.tensorParallel
|
||||
: fallbackTensorParallel;
|
||||
if (ownConfig.selectedGpuIds != null) {
|
||||
await ensureGpuDeviceCache();
|
||||
}
|
||||
const effectiveGpuMemoryMode =
|
||||
ownConfig.gpuMemoryMode ?? compareLoadKnobs.gpuMemoryMode;
|
||||
const effectiveGpuLayers =
|
||||
ownConfig.gpuLayers ?? compareLoadKnobs.gpuLayers;
|
||||
const effectiveNCpuMoe =
|
||||
ownConfig.nCpuMoe ?? compareLoadKnobs.nCpuMoe;
|
||||
const effectiveSelectedGpuIds =
|
||||
ownConfig.selectedGpuIds !== undefined
|
||||
? reconcilePersistedGpuIds(ownConfig.selectedGpuIds)
|
||||
: compareLoadKnobs.selectedGpuIds;
|
||||
// A pane's context comes from its own config only: a saved pin, or null
|
||||
// (Auto/native). It must not inherit the active model's shared snapshot --
|
||||
// resolveFitMaxSeqLength would treat that as a pin and load this pane at
|
||||
// the other model's context (changing VRAM/results or OOMing).
|
||||
const effectiveCustomContextLength = ownConfig.customContextLength;
|
||||
let loadTrustRemoteCode = trustRemoteCode;
|
||||
let approvedRemoteCodeFingerprint: string | null = null;
|
||||
const isAlreadyActive =
|
||||
currentStore.params.checkpoint === sel.id &&
|
||||
(currentStore.activeGgufVariant ?? null) ===
|
||||
(sel.ggufVariant ?? null);
|
||||
// Already loaded (gate passed at first load): skip a redundant reload that would
|
||||
// re-trigger the gate without the approval fingerprint and fail for HIGH custom code.
|
||||
if (isAlreadyActive) {
|
||||
if (isAlreadyActive && !config && !loadedFromConfig) {
|
||||
return "ready";
|
||||
}
|
||||
const targetIsGguf =
|
||||
|
|
@ -1087,10 +1158,13 @@ export function SharedComposer({
|
|||
// layers the load sends 0 / the pinned context, not raw maxSeqLength).
|
||||
const compareMaxSeqLength = resolveFitMaxSeqLength(
|
||||
targetIsGguf,
|
||||
compareLoadKnobs.gpuMemoryMode,
|
||||
compareLoadKnobs.gpuLayers,
|
||||
compareLoadKnobs.customContextLength,
|
||||
maxSeqLength,
|
||||
effectiveGpuMemoryMode,
|
||||
effectiveGpuLayers,
|
||||
// Prefer this pane's own saved context pin over the shared snapshot,
|
||||
// falling back to its per-pane effective context (GGUF with no saved
|
||||
// context loads at native, not the session maxSeqLength).
|
||||
effectiveCustomContextLength,
|
||||
effectiveMaxSeqLength,
|
||||
);
|
||||
const validation = await validateModel({
|
||||
model_path: sel.id,
|
||||
|
|
@ -1105,8 +1179,8 @@ export function SharedComposer({
|
|||
// below: a non-GGUF target must not inherit a hidden GGUF GPU pick.
|
||||
...(targetIsGguf
|
||||
? {
|
||||
gpu_ids: compareLoadKnobs.selectedGpuIds ?? undefined,
|
||||
gpu_memory_mode: compareLoadKnobs.gpuMemoryMode,
|
||||
gpu_ids: effectiveSelectedGpuIds ?? undefined,
|
||||
gpu_memory_mode: effectiveGpuMemoryMode,
|
||||
}
|
||||
: {}),
|
||||
});
|
||||
|
|
@ -1164,27 +1238,28 @@ export function SharedComposer({
|
|||
trust_remote_code: loadTrustRemoteCode,
|
||||
approved_remote_code_fingerprint: approvedRemoteCodeFingerprint,
|
||||
chat_template_override: effectiveChatTemplateOverride,
|
||||
speculative_type: specSettings.speculativeType,
|
||||
spec_draft_n_max: specSettings.specDraftNMax,
|
||||
// Honor the Tensor Parallelism + GPU Memory choices on compare loads.
|
||||
// GGUF-only, like the auto-load path: the picker is a GGUF control,
|
||||
// so a non-GGUF target loads via HF auto-placement instead of being
|
||||
// pinned to a leftover GGUF pick it can't even show.
|
||||
tensor_parallel: compareLoadKnobs.tensorParallel,
|
||||
cache_type_kv: ownConfig.kvCacheDtype ?? null,
|
||||
speculative_type: effectiveSpeculativeType,
|
||||
spec_draft_n_max: effectiveSpecDraftNMax,
|
||||
tensor_parallel: effectiveTensorParallel,
|
||||
...(targetIsGguf
|
||||
? {
|
||||
gpu_memory_mode: compareLoadKnobs.gpuMemoryMode,
|
||||
gpu_layers: compareLoadKnobs.gpuLayers,
|
||||
n_cpu_moe: compareLoadKnobs.nCpuMoe,
|
||||
gpu_memory_mode: effectiveGpuMemoryMode,
|
||||
gpu_layers: effectiveGpuLayers,
|
||||
n_cpu_moe: effectiveNCpuMoe,
|
||||
tensor_split: compareLoadKnobs.splitRatio ?? undefined,
|
||||
gpu_ids: compareLoadKnobs.selectedGpuIds ?? undefined,
|
||||
gpu_ids: effectiveSelectedGpuIds ?? undefined,
|
||||
}
|
||||
: {}),
|
||||
});
|
||||
saveSpeculativeType(specSettings.speculativeType);
|
||||
// Keep a compare pane's per-model speculative choice load-local: persist
|
||||
// the global preference only when it came from global settings.
|
||||
if (ownConfig.speculativeType == null) {
|
||||
saveSpeculativeType(effectiveSpeculativeType);
|
||||
}
|
||||
// Persist the GPU Memory mode on a non-diffusion GGUF compare-load too,
|
||||
// so an applied manual choice survives a restart.
|
||||
persistGpuMemoryModeOnLoad(resp, compareLoadKnobs.gpuMemoryMode);
|
||||
persistGpuMemoryModeOnLoad(resp, effectiveGpuMemoryMode);
|
||||
upgradeUnloadedActive = false;
|
||||
const store = useChatRuntimeStore.getState();
|
||||
store.setCheckpoint(
|
||||
|
|
@ -1200,9 +1275,9 @@ export function SharedComposer({
|
|||
// compare loads don't send the pin, so their baseline clears.
|
||||
const keepCustomCtx = targetIsGguf
|
||||
? resolveManualAutoCtxPin(
|
||||
compareLoadKnobs.gpuMemoryMode,
|
||||
compareLoadKnobs.gpuLayers,
|
||||
compareLoadKnobs.customContextLength,
|
||||
effectiveGpuMemoryMode,
|
||||
effectiveGpuLayers,
|
||||
effectiveCustomContextLength,
|
||||
)
|
||||
: null;
|
||||
useChatRuntimeStore.setState({
|
||||
|
|
@ -1211,37 +1286,52 @@ export function SharedComposer({
|
|||
...reasoningCapsFromLoad(resp),
|
||||
supportsPreserveThinking: resp.supports_preserve_thinking ?? false,
|
||||
supportsTools: resp.supports_tools ?? false,
|
||||
kvCacheDtype: resp.cache_type_kv ?? null,
|
||||
loadedKvCacheDtype: resp.cache_type_kv ?? null,
|
||||
tensorParallel: resp.tensor_parallel ?? false,
|
||||
loadedTensorParallel: resp.tensor_parallel ?? false,
|
||||
customContextLength: keepCustomCtx,
|
||||
defaultChatTemplate: resp.chat_template ?? null,
|
||||
chatTemplateOverride: effectiveChatTemplateOverride,
|
||||
loadedChatTemplateOverride: effectiveChatTemplateOverride,
|
||||
// The context baseline this pane loaded with (see keepCustomCtx above),
|
||||
// so a later Apply/Reset can't silently revert a Manual+Auto pin.
|
||||
loadedCustomContextLength: keepCustomCtx,
|
||||
// Seed the loaded GGUF context (interactive/auto-load parity): the
|
||||
// settings sheet keys the GGUF GPU controls off it for a direct .gguf
|
||||
// with no variant, and a later Apply reads it as the resolved context.
|
||||
...(targetIsGguf
|
||||
? {
|
||||
ggufContextLength: resp.context_length ?? 131072,
|
||||
ggufMaxContextLength:
|
||||
resp.max_context_length ?? resp.context_length ?? 131072,
|
||||
ggufNativeContextLength: resp.native_context_length ?? null,
|
||||
}
|
||||
: { ggufContextLength: null }),
|
||||
// Compare loads resolve by id (HF repo / local path), never through a
|
||||
// native-path lease, so a token left by a previously loaded native
|
||||
// GGUF is stale here -- isLoadedGguf keys off it, and a stale token
|
||||
// would dress a non-GGUF compare load in GGUF controls. Mirror the
|
||||
// interactive path, which writes it on every load success.
|
||||
activeNativePathToken: null,
|
||||
// Held under an open staged pick: setCheckpoint preserves a stage on
|
||||
// the empty->active transition, so a compare load can complete with
|
||||
// staged GPU edits still on screen.
|
||||
...loadedGpuMemoryFieldsUnlessStaged(resp),
|
||||
// Adopt the load response's GPU-memory fields (mode/layers/MoE/split/pick
|
||||
// plus loaded baselines) so the GPU controls round-trip. (gguf context,
|
||||
// customContextLength and native-path token/expiry clear in the tail below.)
|
||||
...loadedGpuMemoryFields(resp),
|
||||
// Drives the GPU Memory controls' diffusion gate; set alongside the
|
||||
// GPU fields on every load path so the gate can't read stale.
|
||||
loadedIsDiffusion: resp.is_diffusion ?? false,
|
||||
loadedIsMultimodal: isMultimodalResponse(resp),
|
||||
// Record the context this pane loaded with (like the single-model path)
|
||||
// so when it becomes the active model, the UI and later reload/save use
|
||||
// its context, not the previous/default one.
|
||||
customContextLength: isGgufLoad
|
||||
? (ownConfig.customContextLength ?? keepCustomCtx)
|
||||
: null,
|
||||
ggufContextLength: resp.is_gguf ? (resp.context_length ?? null) : null,
|
||||
ggufNativeContextLength: resp.is_gguf
|
||||
? (resp.native_context_length ?? null)
|
||||
: null,
|
||||
ggufMaxContextLength: resp.is_gguf
|
||||
? (resp.max_context_length ?? null)
|
||||
: null,
|
||||
// Compare selections load by repo/variant, never from the file picker,
|
||||
// so they carry no native lease. Clear any prior picked file's
|
||||
// token/expiry so the reload path never sends a stale lease.
|
||||
activeNativePathToken: null,
|
||||
activeNativePathExpiresAtMs: null,
|
||||
...resolveLoadedSpeculativeSettings(resp),
|
||||
});
|
||||
if (!isGgufLoad) {
|
||||
// Non-GGUF panes carry their context in params.maxSeqLength.
|
||||
store.setParams({
|
||||
...useChatRuntimeStore.getState().params,
|
||||
maxSeqLength: effectiveMaxSeqLength,
|
||||
});
|
||||
}
|
||||
loadedFromConfig = config != null;
|
||||
// Sync the models[] entry with the load response so attach/send gates
|
||||
// read fresh capabilities. /api/models/list can lag a model's actual
|
||||
// state (e.g. a GGUF whose mmproj arrived after the snapshot).
|
||||
|
|
|
|||
|
|
@ -1,16 +1,8 @@
|
|||
// SPDX-License-Identifier: AGPL-3.0-only
|
||||
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
import type { RememberedLoadSettings } from "@/components/assistant-ui/model-selector/remembered-load-settings";
|
||||
import {
|
||||
cancelStagedModelDownload,
|
||||
mirrorHfTokenInto,
|
||||
useHfTokenStore,
|
||||
} from "@/features/hub";
|
||||
import {
|
||||
cachedPinnableGpuIndices,
|
||||
ensureGpuDeviceCache,
|
||||
} from "@/hooks/use-gpu-info";
|
||||
import { mirrorHfTokenInto, useHfTokenStore } from "@/features/hub";
|
||||
import { cachedPinnableGpuIndices } from "@/hooks/use-gpu-info";
|
||||
import { toast } from "@/lib/toast";
|
||||
import { create } from "zustand";
|
||||
import { isExternalModelId, parseExternalModelId } from "../external-providers";
|
||||
|
|
@ -46,7 +38,6 @@ export const CHAT_ALLOW_ARTIFACT_NETWORK_ACCESS_KEY =
|
|||
"unsloth_chat_allow_artifact_network_access";
|
||||
export const CHAT_MCP_ENABLED_KEY = "unsloth_chat_mcp_enabled";
|
||||
export const CHAT_CONFIRM_TOOL_CALLS_KEY = "unsloth_chat_confirm_tool_calls";
|
||||
export const CHAT_LOAD_ON_SELECTION_KEY = "unsloth_chat_load_on_selection";
|
||||
export const CHAT_EXPAND_QUANTIZATIONS_KEY =
|
||||
"unsloth_chat_expand_quantizations";
|
||||
export const CHAT_SHOW_ALL_QUANTIZATIONS_KEY =
|
||||
|
|
@ -229,6 +220,11 @@ export type PendingImageEditReference = {
|
|||
openaiResponseId?: string;
|
||||
openaiReasoningItem?: unknown;
|
||||
};
|
||||
export type LoadingModelPick = {
|
||||
id: string;
|
||||
ggufVariant: string | null;
|
||||
nativePathToken: string | null;
|
||||
};
|
||||
export type ReasoningEffort =
|
||||
| "none"
|
||||
| "minimal"
|
||||
|
|
@ -680,70 +676,7 @@ export function loadedGpuMemoryFields(resp: {
|
|||
};
|
||||
}
|
||||
|
||||
/** loadedGpuMemoryFields (plus any seedExtras), unless a staged pick is open.
|
||||
*
|
||||
* With a staged pick open (the load fired mid-staging), preserve its editable
|
||||
* GPU knobs and seedExtras, but still advance every loaded baseline. Otherwise
|
||||
* cancelling the stage restores its edits onto the newly loaded model. The
|
||||
* status reseed cannot repair that while pendingSelection holds it off.
|
||||
*/
|
||||
export function loadedGpuMemoryFieldsUnlessStaged<T extends object>(
|
||||
resp: Parameters<typeof loadedGpuMemoryFields>[0],
|
||||
seedExtras?: T,
|
||||
) {
|
||||
const fields = loadedGpuMemoryFields(resp);
|
||||
if (useChatRuntimeStore.getState().pendingSelection != null) {
|
||||
return {
|
||||
loadedGpuMemoryMode: fields.loadedGpuMemoryMode,
|
||||
loadedGpuLayers: fields.loadedGpuLayers,
|
||||
loadedNCpuMoe: fields.loadedNCpuMoe,
|
||||
loadedSplitRatio: fields.loadedSplitRatio,
|
||||
loadedGpuIds: fields.loadedGpuIds,
|
||||
// These are metadata ceilings for the model that actually loaded, not
|
||||
// editable values from the open stage. Advance them with the baselines
|
||||
// so abandoning the stage cannot expose the previous model's limits.
|
||||
ggufLayerCount: fields.ggufLayerCount,
|
||||
moeLayerCount: fields.moeLayerCount,
|
||||
};
|
||||
}
|
||||
return { ...fields, ...seedExtras };
|
||||
}
|
||||
|
||||
/** A local model staged for a deferred load (see `pendingSelection`). Shape is
|
||||
* a subset of the load hook's `SelectedModelInput`, structurally assignable. */
|
||||
export type PendingModelSelection = {
|
||||
id: string;
|
||||
isLora?: boolean;
|
||||
ggufVariant?: string;
|
||||
isDownloaded?: boolean;
|
||||
expectedBytes?: number;
|
||||
/** Native (drag-drop / picked-from-disk) GGUF: the path token used to read
|
||||
* the header and to load. Absent for HF-repo models. */
|
||||
nativePathToken?: string;
|
||||
/** Direct local .gguf file (custom folder / LM Studio): a GGUF source even
|
||||
* though it carries neither an HF variant nor a native path token. */
|
||||
isGguf?: boolean;
|
||||
/** Native context length read from the GGUF header once the file is local.
|
||||
* Scoped here (not the shared `ggufContextLength`) so a staged model's
|
||||
* metadata never pollutes the currently-loaded model's context display. */
|
||||
contextLength?: number | null;
|
||||
/** Total layer count (GGUF block_count); the manual gpu-layers ceiling is
|
||||
* this + 1 (llama.cpp counts the output layer as offloadable too);
|
||||
* scoped here like contextLength. */
|
||||
layerCount?: number | null;
|
||||
/** MoE expert-layer count from the GGUF header (manual --n-cpu-moe ceiling);
|
||||
* 0 for dense models, scoped here like contextLength. */
|
||||
moeLayerCount?: number | null;
|
||||
/** "Load on selection" on + un-cached GGUF: download via the manager (global
|
||||
* indicator) without opening the sheet, then load once the download finishes. */
|
||||
autoLoad?: boolean;
|
||||
/** Uncached non-GGUF HF repo: download the full snapshot via the manager
|
||||
* (variant null) the same way GGUF picks download a variant. */
|
||||
isHubRepo?: boolean;
|
||||
};
|
||||
|
||||
/** A pick is a GGUF (HF variant, native file, or a direct local .gguf) and so
|
||||
* has pre-load options worth staging. Works on a selection or a staged pick. */
|
||||
/** A pick is a GGUF: HF variant, native file, or a direct local .gguf. */
|
||||
export function hasGgufSource(x: {
|
||||
ggufVariant?: string;
|
||||
nativePathToken?: string;
|
||||
|
|
@ -781,30 +714,6 @@ export function isDownloadableHubRepo(x: {
|
|||
);
|
||||
}
|
||||
|
||||
export function isPendingGguf(pending: PendingModelSelection | null): boolean {
|
||||
return pending != null && hasGgufSource(pending);
|
||||
}
|
||||
|
||||
/** Whether `pending` refers to the same model as `pick` (id + GGUF variant +
|
||||
* native path token, optionals null-normalized). Native ids are display labels
|
||||
* that can collide, so the token must match too — id alone can land on the
|
||||
* wrong file. */
|
||||
export function pendingSelectionMatches(
|
||||
pending: PendingModelSelection | null,
|
||||
pick: {
|
||||
id: string;
|
||||
ggufVariant?: string | null;
|
||||
nativePathToken?: string | null;
|
||||
},
|
||||
): boolean {
|
||||
return (
|
||||
pending != null &&
|
||||
pending.id === pick.id &&
|
||||
(pending.ggufVariant ?? null) === (pick.ggufVariant ?? null) &&
|
||||
(pending.nativePathToken ?? null) === (pick.nativePathToken ?? null)
|
||||
);
|
||||
}
|
||||
|
||||
type ChatRuntimeStore = {
|
||||
settingsHydrated: boolean;
|
||||
params: InferenceParams;
|
||||
|
|
@ -988,10 +897,6 @@ type ChatRuntimeStore = {
|
|||
/** Picked physical GPU indices (null = use all / automatic). */
|
||||
selectedGpuIds: number[] | null;
|
||||
loadedGpuIds: number[] | null;
|
||||
/** Persisted: when false, picking a local model stages it as
|
||||
* `pendingSelection` (and opens settings) instead of loading immediately,
|
||||
* so load settings can be set before the single load. */
|
||||
loadOnSelection: boolean;
|
||||
/** Persisted: expand every On Device GGUF repo's quantizations by default
|
||||
* instead of waiting for a click. */
|
||||
expandQuantizations: boolean;
|
||||
|
|
@ -1000,9 +905,6 @@ type ChatRuntimeStore = {
|
|||
/** Persisted, shared by the chat model selector and the Hub page: list only
|
||||
* models whose size fits this device's memory budget. */
|
||||
fitOnDeviceOnly: boolean;
|
||||
/** A local model picked while `loadOnSelection` is off: staged, not loaded.
|
||||
* The settings sheet shows its load knobs and a Load button. */
|
||||
pendingSelection: PendingModelSelection | null;
|
||||
loadedIsMultimodal: boolean;
|
||||
/** Active model is a block-diffusion model (DiffusionGemma): drives the
|
||||
* denoising-canvas artifact auto-render. */
|
||||
|
|
@ -1041,9 +943,16 @@ type ChatRuntimeStore = {
|
|||
cacheWriteTokens?: number;
|
||||
} | null;
|
||||
modelLoading: boolean;
|
||||
loadingModelPick: LoadingModelPick | null;
|
||||
activeNativePathToken: string | null;
|
||||
// Wall-clock expiry (ms) of the active native path token. The desktop host
|
||||
// prunes file leases after a TTL, so a reload checks this to prompt
|
||||
// re-selection instead of reusing a dead token.
|
||||
activeNativePathExpiresAtMs: number | null;
|
||||
hydratePersistedSettings: () => Promise<void>;
|
||||
setModelLoading: (loading: boolean) => void;
|
||||
setLoadingModelPick: (pick: LoadingModelPick | null) => void;
|
||||
clearLoadingModelPick: (expected: LoadingModelPick) => void;
|
||||
setModelRequiresTrustRemoteCode: (required: boolean) => void;
|
||||
setParams: (params: InferenceParams) => void;
|
||||
setCustomPresets: (presets: Preset[]) => void;
|
||||
|
|
@ -1119,38 +1028,14 @@ type ChatRuntimeStore = {
|
|||
setNudgeToolCalls: (enabled: boolean) => void;
|
||||
setMaxToolCallsPerMessage: (value: number) => void;
|
||||
setToolCallTimeout: (value: number) => void;
|
||||
setKvCacheDtype: (dtype: string | null) => void;
|
||||
setSpeculativeType: (type: string | null) => void;
|
||||
setSpecDraftNMax: (value: number | null) => void;
|
||||
/** Revert the editable load knobs to the loaded model's baseline (or defaults
|
||||
* when nothing is loaded). Used by the settings-sheet Reset button and to
|
||||
* start each deferred-staging session clean so one staged pick's settings
|
||||
* don't leak onto the next. */
|
||||
resetModelSettingsToLoaded: () => void;
|
||||
/** Seed the editable load knobs from a model's remembered settings. Shared by
|
||||
* the settings sheet's restore effect and the "Load on selection" paths,
|
||||
* which skip the sheet but must still honor a saved config. */
|
||||
applyRememberedLoadSettings: (settings: RememberedLoadSettings) => void;
|
||||
setTensorParallel: (value: boolean) => void;
|
||||
setGpuMemoryMode: (mode: "auto" | "manual") => void;
|
||||
setGpuLayers: (value: number) => void;
|
||||
setNCpuMoe: (value: number) => void;
|
||||
setSplitRatio: (value: number[] | null) => void;
|
||||
setSelectedGpuIds: (ids: number[] | null) => void;
|
||||
setLoadOnSelection: (value: boolean) => void;
|
||||
setExpandQuantizations: (value: boolean) => void;
|
||||
setShowAllQuantizations: (value: boolean) => void;
|
||||
setFitOnDeviceOnly: (value: boolean) => void;
|
||||
setPendingSelection: (selection: PendingModelSelection | null) => void;
|
||||
/** Stage a pick for a deferred load: revert knobs to the loaded baseline,
|
||||
* record the selection, and open the settings sheet. */
|
||||
stageModel: (selection: PendingModelSelection) => void;
|
||||
/** Abandon a staged pick without loading: revert knobs to the loaded baseline
|
||||
* and clear the pending selection. Cancels its in-flight download too, unless
|
||||
* `keepDownload` is set (navigation keeps the transfer running, like Hub). */
|
||||
abandonStagedModel: (opts?: { keepDownload?: boolean }) => void;
|
||||
setCustomContextLength: (v: number | null) => void;
|
||||
setChatTemplateOverride: (template: string | null) => void;
|
||||
setPendingAudio: (base64: string, name: string) => void;
|
||||
clearPendingAudio: () => void;
|
||||
setPendingImageEditReference: (
|
||||
|
|
@ -1352,38 +1237,6 @@ function setScalarSettingVersion<K extends ScalarSettingKey>(
|
|||
saveSettingsPatch({ [key]: value });
|
||||
}
|
||||
|
||||
/** The "revert to the loaded model" baseline for the editable load knobs.
|
||||
* Shared by resetModelSettingsToLoaded (full revert) and stageModel (which
|
||||
* overrides speculative and the per-model GPU knobs to start a fresh pick). */
|
||||
function loadedBaselineSettings(s: ChatRuntimeStore) {
|
||||
const hasLoadedModel = Boolean(s.params.checkpoint);
|
||||
return {
|
||||
// Revert to the loaded model's pin (null = Auto), not a blanket Auto.
|
||||
customContextLength: s.loadedCustomContextLength,
|
||||
kvCacheDtype: s.loadedKvCacheDtype,
|
||||
tensorParallel: s.loadedTensorParallel ?? false,
|
||||
speculativeType: hasLoadedModel
|
||||
? s.loadedSpeculativeType
|
||||
: readPersistedSpeculativeType(),
|
||||
specDraftNMax: hasLoadedModel ? s.loadedSpecDraftNMax : null,
|
||||
chatTemplateOverride: s.loadedChatTemplateOverride,
|
||||
// GPU memory mode is a standing preference; revert to the loaded model's
|
||||
// mode (or the persisted default when nothing is loaded). Manual knobs and
|
||||
// the GPU pick are per-model and revert to their loaded baseline. A loaded
|
||||
// model with no applicable mode -- diffusion ("auto" baseline) or non-GGUF
|
||||
// (null baseline) -- keeps the live preference so Reset can't drop it.
|
||||
gpuMemoryMode: !hasLoadedModel
|
||||
? readPersistedGpuMemoryMode()
|
||||
: s.loadedIsDiffusion
|
||||
? s.gpuMemoryMode
|
||||
: (s.loadedGpuMemoryMode ?? s.gpuMemoryMode),
|
||||
gpuLayers: s.loadedGpuLayers ?? GPU_LAYERS_AUTO,
|
||||
nCpuMoe: s.loadedNCpuMoe ?? 0,
|
||||
splitRatio: s.loadedSplitRatio ?? null,
|
||||
selectedGpuIds: s.loadedGpuIds,
|
||||
};
|
||||
}
|
||||
|
||||
export const useChatRuntimeStore = create<ChatRuntimeStore>((set, get) => ({
|
||||
settingsHydrated: false,
|
||||
// Hydrate the last external checkpoint so the external picker survives a
|
||||
|
|
@ -1493,11 +1346,9 @@ export const useChatRuntimeStore = create<ChatRuntimeStore>((set, get) => ({
|
|||
moeLayerCount: null,
|
||||
selectedGpuIds: null,
|
||||
loadedGpuIds: null,
|
||||
loadOnSelection: loadBool(CHAT_LOAD_ON_SELECTION_KEY, true),
|
||||
expandQuantizations: loadBool(CHAT_EXPAND_QUANTIZATIONS_KEY, false),
|
||||
showAllQuantizations: loadBool(CHAT_SHOW_ALL_QUANTIZATIONS_KEY, true),
|
||||
fitOnDeviceOnly: loadBool(MODELS_FIT_ON_DEVICE_ONLY_KEY, false),
|
||||
pendingSelection: null,
|
||||
loadedIsMultimodal: false,
|
||||
loadedIsDiffusion: false,
|
||||
customContextLength: null,
|
||||
|
|
@ -1515,7 +1366,9 @@ export const useChatRuntimeStore = create<ChatRuntimeStore>((set, get) => ({
|
|||
pendingImageEditReference: null,
|
||||
contextUsage: null,
|
||||
modelLoading: false,
|
||||
loadingModelPick: null,
|
||||
activeNativePathToken: null,
|
||||
activeNativePathExpiresAtMs: null,
|
||||
hydratePersistedSettings: async () => {
|
||||
if (get().settingsHydrated) {
|
||||
return;
|
||||
|
|
@ -1554,6 +1407,20 @@ export const useChatRuntimeStore = create<ChatRuntimeStore>((set, get) => ({
|
|||
return settingsHydrationPromise;
|
||||
},
|
||||
setModelLoading: (loading) => set({ modelLoading: loading }),
|
||||
setLoadingModelPick: (pick) => set({ loadingModelPick: pick }),
|
||||
clearLoadingModelPick: (expected) =>
|
||||
set((state) => {
|
||||
const current = state.loadingModelPick;
|
||||
if (
|
||||
!current ||
|
||||
current.id !== expected.id ||
|
||||
current.ggufVariant !== expected.ggufVariant ||
|
||||
current.nativePathToken !== expected.nativePathToken
|
||||
) {
|
||||
return state;
|
||||
}
|
||||
return { loadingModelPick: null };
|
||||
}),
|
||||
setModelRequiresTrustRemoteCode: (modelRequiresTrustRemoteCode) =>
|
||||
set({ modelRequiresTrustRemoteCode }),
|
||||
setParams: (params) =>
|
||||
|
|
@ -1634,13 +1501,6 @@ export const useChatRuntimeStore = create<ChatRuntimeStore>((set, get) => ({
|
|||
// Clear stale per-turn usage on model change; the relaxed external-provider
|
||||
// render gate would otherwise show old counters until the next completion.
|
||||
const checkpointChanged = state.params.checkpoint !== modelId;
|
||||
const pendingToClear =
|
||||
checkpointChanged && state.params.checkpoint
|
||||
? state.pendingSelection
|
||||
: null;
|
||||
if (pendingToClear) {
|
||||
cancelStagedModelDownload(pendingToClear);
|
||||
}
|
||||
// Clamp maxTokens to the new model's cap when switching into an external
|
||||
// model so a value carried over from a local session doesn't exceed the
|
||||
// slider's max.
|
||||
|
|
@ -1668,14 +1528,6 @@ export const useChatRuntimeStore = create<ChatRuntimeStore>((set, get) => ({
|
|||
},
|
||||
activeGgufVariant: ggufVariant ?? null,
|
||||
...(checkpointChanged ? { contextUsage: null } : {}),
|
||||
// Switching away from a loaded model (e.g. picking an external provider)
|
||||
// abandons any staged pick, so its Load button and edited knobs don't
|
||||
// linger over the newly active model. Same revert as abandonStagedModel.
|
||||
// Guarded on a non-empty current checkpoint: an establishing set from a
|
||||
// background status sync (empty -> active) must not wipe a fresh stage.
|
||||
...(pendingToClear
|
||||
? { ...loadedBaselineSettings(state), pendingSelection: null }
|
||||
: {}),
|
||||
};
|
||||
}),
|
||||
setActiveThreadId: (activeThreadId) =>
|
||||
|
|
@ -1689,7 +1541,6 @@ export const useChatRuntimeStore = create<ChatRuntimeStore>((set, get) => ({
|
|||
// clear any stored external selection so the next refresh doesn't snap
|
||||
// back to a model the user intentionally cleared.
|
||||
saveLastExternalCheckpoint(null);
|
||||
cancelStagedModelDownload(get().pendingSelection);
|
||||
return set((state) => ({
|
||||
params: {
|
||||
...state.params,
|
||||
|
|
@ -1697,7 +1548,7 @@ export const useChatRuntimeStore = create<ChatRuntimeStore>((set, get) => ({
|
|||
},
|
||||
activeGgufVariant: null,
|
||||
activeNativePathToken: null,
|
||||
pendingSelection: null,
|
||||
activeNativePathExpiresAtMs: null,
|
||||
ggufContextLength: null,
|
||||
ggufMaxContextLength: null,
|
||||
ggufNativeContextLength: null,
|
||||
|
|
@ -2044,10 +1895,6 @@ export const useChatRuntimeStore = create<ChatRuntimeStore>((set, get) => ({
|
|||
);
|
||||
return { toolCallTimeout };
|
||||
}),
|
||||
setKvCacheDtype: (kvCacheDtype) => set({ kvCacheDtype }),
|
||||
setSpeculativeType: (speculativeType) => set({ speculativeType }),
|
||||
setSpecDraftNMax: (specDraftNMax) => set({ specDraftNMax }),
|
||||
setTensorParallel: (tensorParallel) => set({ tensorParallel }),
|
||||
// Standing preference, but persisted only on a successful load (see
|
||||
// use-chat-model-runtime), not on selection -- so an unapplied pick the user
|
||||
// resets/abandons doesn't stick to the next session.
|
||||
|
|
@ -2056,63 +1903,6 @@ export const useChatRuntimeStore = create<ChatRuntimeStore>((set, get) => ({
|
|||
setNCpuMoe: (nCpuMoe) => set({ nCpuMoe }),
|
||||
setSplitRatio: (splitRatio) => set({ splitRatio }),
|
||||
setSelectedGpuIds: (selectedGpuIds) => set({ selectedGpuIds }),
|
||||
resetModelSettingsToLoaded: () => set((s) => loadedBaselineSettings(s)),
|
||||
applyRememberedLoadSettings: (settings) => {
|
||||
const gpuCacheWasCold = cachedPinnableGpuIndices() === null;
|
||||
const restoredGpuIds =
|
||||
settings.selectedGpuIds !== undefined
|
||||
? reconcilePersistedGpuIds(settings.selectedGpuIds)
|
||||
: undefined;
|
||||
// Coalesce every field: a blob persisted by an older/newer build can omit
|
||||
// keys, and a raw spread would push `undefined` into fields typed non-null.
|
||||
// The GPU knobs are spread only when present, but first reset the per-model
|
||||
// ones to defaults: this path (load-on-selection) starts from the loaded
|
||||
// model's baseline and skips the model-switch reset, so a blob omitting
|
||||
// gpuLayers/nCpuMoe/selectedGpuIds (older build) or splitRatio (never
|
||||
// remembered) must not inherit the previous model's placement. gpuMemoryMode
|
||||
// (standing preference) is NOT reset, only applied when the blob carries it;
|
||||
// selectedGpuIds keeps a meaningful null (all GPUs), so it keys off undefined.
|
||||
set({
|
||||
gpuLayers: GPU_LAYERS_AUTO,
|
||||
nCpuMoe: 0,
|
||||
splitRatio: null,
|
||||
selectedGpuIds: null,
|
||||
customContextLength: settings.contextLength ?? null,
|
||||
kvCacheDtype: settings.kvCacheDtype ?? null,
|
||||
speculativeType: settings.speculativeType ?? "auto",
|
||||
specDraftNMax: settings.specDraftNMax ?? null,
|
||||
tensorParallel: settings.tensorParallel ?? false,
|
||||
...(settings.gpuMemoryMode != null && {
|
||||
gpuMemoryMode: settings.gpuMemoryMode,
|
||||
}),
|
||||
...(settings.gpuLayers != null && { gpuLayers: settings.gpuLayers }),
|
||||
...(settings.nCpuMoe != null && { nCpuMoe: settings.nCpuMoe }),
|
||||
...(restoredGpuIds !== undefined && {
|
||||
// Reconcile against the GPUs present now (see reconcilePersistedGpuIds):
|
||||
// a saved [1] on a 1-GPU host (or under relative/UUID visibility) would
|
||||
// hide the picker yet still send gpu_ids, which the backend rejects.
|
||||
selectedGpuIds: restoredGpuIds,
|
||||
}),
|
||||
});
|
||||
// A cold cache makes the synchronous restore provisional. Reconcile again
|
||||
// when the shared fetch completes, but only if this exact restored array is
|
||||
// still current so a user edit, stage change, or load cannot be overwritten.
|
||||
if (gpuCacheWasCold && restoredGpuIds != null) {
|
||||
void ensureGpuDeviceCache().then(() => {
|
||||
set((state) => {
|
||||
if (state.selectedGpuIds !== restoredGpuIds) return state;
|
||||
const reconciled = reconcilePersistedGpuIds(restoredGpuIds);
|
||||
return reconciled === restoredGpuIds
|
||||
? state
|
||||
: { selectedGpuIds: reconciled };
|
||||
});
|
||||
});
|
||||
}
|
||||
},
|
||||
setLoadOnSelection: (loadOnSelection) => {
|
||||
saveBool(CHAT_LOAD_ON_SELECTION_KEY, loadOnSelection);
|
||||
set({ loadOnSelection });
|
||||
},
|
||||
setExpandQuantizations: (expandQuantizations) => {
|
||||
saveBool(CHAT_EXPAND_QUANTIZATIONS_KEY, expandQuantizations);
|
||||
set({ expandQuantizations });
|
||||
|
|
@ -2125,55 +1915,6 @@ export const useChatRuntimeStore = create<ChatRuntimeStore>((set, get) => ({
|
|||
saveBool(MODELS_FIT_ON_DEVICE_ONLY_KEY, fitOnDeviceOnly);
|
||||
set({ fitOnDeviceOnly });
|
||||
},
|
||||
setPendingSelection: (pendingSelection) => set({ pendingSelection }),
|
||||
stageModel: (selection) => {
|
||||
// Refuse staging mid-load: post-load cleanup would silently drop the queued
|
||||
// pick. stageOrLoad toasts first for callers that can.
|
||||
if (get().modelLoading) return;
|
||||
// Rebinding to a new pick keeps the prior pick's download running so the
|
||||
// user can queue multiple downloads at once (Hub-style).
|
||||
set((s) => {
|
||||
return {
|
||||
...loadedBaselineSettings(s),
|
||||
pendingSelection: selection,
|
||||
// autoLoad downloads silently and loads on completion, so keep the sheet shut.
|
||||
settingsPanelOpen: !selection.autoLoad,
|
||||
// Speculative starts from the standing default, not the loaded model's
|
||||
// mode, so a fresh pick doesn't inherit (and then carry, via the staged
|
||||
// Load's keepSpeculative) a forced MTP mode onto a model that may lack it.
|
||||
speculativeType: readPersistedSpeculativeType(),
|
||||
specDraftNMax: null,
|
||||
// Keep the on-screen GPU Memory selection (loadedBaselineSettings would
|
||||
// otherwise revert it to the loaded model's mode, dropping a Manual choice
|
||||
// just made). Use the live store value, not the persisted one, which can
|
||||
// lag a mode hydrated from an out-of-band load.
|
||||
gpuMemoryMode: s.gpuMemoryMode,
|
||||
// Per-model GPU knobs start from defaults too so a fresh pick doesn't
|
||||
// inherit the loaded model's layer/MoE/split/GPU choices, matching the
|
||||
// immediate-switch reset.
|
||||
gpuLayers: GPU_LAYERS_AUTO,
|
||||
nCpuMoe: 0,
|
||||
splitRatio: null,
|
||||
selectedGpuIds: null,
|
||||
// Fresh pick starts at Auto context (loadedBaselineSettings would
|
||||
// otherwise restore the current model's pin). Leaves the baseline
|
||||
// intact, like the GPU knobs, so abandoning restores the loaded pin.
|
||||
customContextLength: null,
|
||||
};
|
||||
});
|
||||
},
|
||||
abandonStagedModel: (opts) => {
|
||||
const { pendingSelection } = get();
|
||||
if (!pendingSelection) return;
|
||||
// Cancel the staged pick's in-flight download (centralized for every abandon
|
||||
// path: sheet close, thread switch, route exit, new chat). `keepDownload`
|
||||
// opts out so navigation leaves the transfer running, like a Hub download.
|
||||
if (!opts?.keepDownload) cancelStagedModelDownload(pendingSelection);
|
||||
set((s) => ({ ...loadedBaselineSettings(s), pendingSelection: null }));
|
||||
},
|
||||
setCustomContextLength: (customContextLength) => set({ customContextLength }),
|
||||
setChatTemplateOverride: (chatTemplateOverride) =>
|
||||
set({ chatTemplateOverride }),
|
||||
setPendingAudio: (base64, name) =>
|
||||
set({ pendingAudioBase64: base64, pendingAudioName: name }),
|
||||
clearPendingAudio: () =>
|
||||
|
|
|
|||
|
|
@ -99,6 +99,9 @@ export interface ValidateModelResponse {
|
|||
/** MoE expert-layer count from the GGUF header (manual --n-cpu-moe ceiling);
|
||||
* 0 for dense models, null until downloaded. */
|
||||
moe_layer_count?: number | null;
|
||||
/** Embedded GGUF chat template, returned when include_chat_template is set
|
||||
* (native lease-backed picks); null for non-GGUF, over-cap, or not read. */
|
||||
chat_template?: string | null;
|
||||
/** Architecture only shipped by a newer transformers; UI pauses on the upgrade dialog. */
|
||||
requires_transformers_upgrade?: boolean;
|
||||
/** Set only when requires_transformers_upgrade. */
|
||||
|
|
|
|||
|
|
@ -2,7 +2,6 @@
|
|||
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
import { Button } from "@/components/ui/button";
|
||||
import { FolderBrowser } from "@/components/assistant-ui/model-selector/folder-browser";
|
||||
import { Input } from "@/components/ui/input";
|
||||
import {
|
||||
InputGroup,
|
||||
|
|
@ -17,6 +16,7 @@ import {
|
|||
TooltipContent,
|
||||
TooltipTrigger,
|
||||
} from "@/components/ui/tooltip";
|
||||
import { FolderBrowser } from "@/features/model-picker";
|
||||
import {
|
||||
AlertCircleIcon,
|
||||
ArrowRight01Icon,
|
||||
|
|
@ -28,17 +28,17 @@ import {
|
|||
import { HugeiconsIcon } from "@hugeicons/react";
|
||||
import { useEffect, useRef, useState } from "react";
|
||||
import { useShallow } from "zustand/react/shallow";
|
||||
import type { ExportLogEntry } from "../api/export-api";
|
||||
import {
|
||||
EXPORT_METHODS,
|
||||
type ExportMethod,
|
||||
findMergedFormat,
|
||||
} from "../constants";
|
||||
import type { ExportLogEntry } from "../api/export-api";
|
||||
import { getExportLogLineClass } from "../lib/log-style";
|
||||
import {
|
||||
type ExportDestination,
|
||||
selectExportProgressPercent,
|
||||
useExportRuntimeStore,
|
||||
type ExportDestination,
|
||||
} from "../stores/export-runtime-store";
|
||||
|
||||
function useElapsedSeconds(startedAt: number | null, running: boolean): number {
|
||||
|
|
@ -165,7 +165,9 @@ export function ExportRunPanel(props: ExportRunPanelProps) {
|
|||
|
||||
const isExporting = run.isExporting;
|
||||
const isTerminal =
|
||||
run.phase === "success" || run.phase === "error" || run.phase === "canceled";
|
||||
run.phase === "success" ||
|
||||
run.phase === "error" ||
|
||||
run.phase === "canceled";
|
||||
const showConfig = run.phase === "idle";
|
||||
// Gate the log area on the active run's method (from the store) as well as the
|
||||
// local form selection, so it stays visible after navigating away and back
|
||||
|
|
@ -197,7 +199,9 @@ export function ExportRunPanel(props: ExportRunPanelProps) {
|
|||
setFollowTail(nearBottom);
|
||||
};
|
||||
|
||||
const methodTitle = EXPORT_METHODS.find((m) => m.value === exportMethod)?.title;
|
||||
const methodTitle = EXPORT_METHODS.find(
|
||||
(m) => m.value === exportMethod,
|
||||
)?.title;
|
||||
const summary = run.summary;
|
||||
const summaryBaseModel = summary?.baseModelName ?? baseModelName;
|
||||
const summaryCheckpoint = summary?.checkpointLabel ?? checkpoint;
|
||||
|
|
@ -290,7 +294,10 @@ export function ExportRunPanel(props: ExportRunPanelProps) {
|
|||
onClick={() => setFolderBrowserOpen(true)}
|
||||
aria-label="Browse save folder"
|
||||
>
|
||||
<HugeiconsIcon icon={FolderSearchIcon} className="size-4" />
|
||||
<HugeiconsIcon
|
||||
icon={FolderSearchIcon}
|
||||
className="size-4"
|
||||
/>
|
||||
</Button>
|
||||
</TooltipTrigger>
|
||||
<TooltipContent>Browse</TooltipContent>
|
||||
|
|
@ -301,8 +308,8 @@ export function ExportRunPanel(props: ExportRunPanelProps) {
|
|||
<>Default: {defaultSaveDirectory}</>
|
||||
) : (
|
||||
<>
|
||||
Paste an absolute path if the folder browser cannot reach the
|
||||
drive.
|
||||
Paste an absolute path if the folder browser cannot reach
|
||||
the drive.
|
||||
</>
|
||||
)}
|
||||
</p>
|
||||
|
|
@ -410,7 +417,10 @@ export function ExportRunPanel(props: ExportRunPanelProps) {
|
|||
: [];
|
||||
const showLabels = items.length > 1;
|
||||
return items.map((o, i) => (
|
||||
<div key={`${o.path}-${i}`} className="flex min-w-0 flex-col gap-0.5">
|
||||
<div
|
||||
key={`${o.path}-${i}`}
|
||||
className="flex min-w-0 flex-col gap-0.5"
|
||||
>
|
||||
{showLabels && o.label ? (
|
||||
<span className="text-xs text-emerald-700/80 dark:text-emerald-300/80">
|
||||
{o.label}
|
||||
|
|
@ -431,14 +441,22 @@ export function ExportRunPanel(props: ExportRunPanelProps) {
|
|||
|
||||
{run.phase === "canceled" && (
|
||||
<div className="flex items-start gap-2 rounded-lg bg-amber-500/10 p-3 text-sm text-amber-700 dark:text-amber-300">
|
||||
<HugeiconsIcon icon={CancelCircleIcon} className="mt-0.5 size-4 shrink-0" />
|
||||
<span>Export canceled. Training and inference were not affected.</span>
|
||||
<HugeiconsIcon
|
||||
icon={CancelCircleIcon}
|
||||
className="mt-0.5 size-4 shrink-0"
|
||||
/>
|
||||
<span>
|
||||
Export canceled. Training and inference were not affected.
|
||||
</span>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{run.phase === "error" && run.error && (
|
||||
<div className="flex items-start gap-2 rounded-lg bg-destructive/10 p-3 text-sm text-destructive">
|
||||
<HugeiconsIcon icon={AlertCircleIcon} className="size-4 mt-0.5 shrink-0" />
|
||||
<HugeiconsIcon
|
||||
icon={AlertCircleIcon}
|
||||
className="size-4 mt-0.5 shrink-0"
|
||||
/>
|
||||
<span>{run.error}</span>
|
||||
</div>
|
||||
)}
|
||||
|
|
@ -447,15 +465,21 @@ export function ExportRunPanel(props: ExportRunPanelProps) {
|
|||
<div className="rounded-xl bg-muted/50 p-3 text-xs text-muted-foreground flex flex-col gap-1">
|
||||
<div className="flex justify-between">
|
||||
<span>Base Model</span>
|
||||
<span className="font-medium text-foreground">{summaryBaseModel}</span>
|
||||
<span className="font-medium text-foreground">
|
||||
{summaryBaseModel}
|
||||
</span>
|
||||
</div>
|
||||
<div className="flex justify-between">
|
||||
<span>{isAdapter ? "Checkpoint" : "Model"}</span>
|
||||
<span className="font-medium text-foreground">{summaryCheckpoint}</span>
|
||||
<span className="font-medium text-foreground">
|
||||
{summaryCheckpoint}
|
||||
</span>
|
||||
</div>
|
||||
<div className="flex justify-between">
|
||||
<span>Export Method</span>
|
||||
<span className="font-medium text-foreground">{summaryMethodLabel}</span>
|
||||
<span className="font-medium text-foreground">
|
||||
{summaryMethodLabel}
|
||||
</span>
|
||||
</div>
|
||||
{summaryMethod === "merged" && summaryFormats.length > 0 && (
|
||||
<div className="flex justify-between gap-3">
|
||||
|
|
@ -484,7 +508,12 @@ export function ExportRunPanel(props: ExportRunPanelProps) {
|
|||
</span>
|
||||
{summaryMethod === "gguf" && run.quantTotal > 1 && (
|
||||
<span className="text-[10px] tabular-nums text-muted-foreground">
|
||||
Quant {Math.min(run.quantIndex + (isExporting ? 1 : 0), run.quantTotal)} of {run.quantTotal}
|
||||
Quant{" "}
|
||||
{Math.min(
|
||||
run.quantIndex + (isExporting ? 1 : 0),
|
||||
run.quantTotal,
|
||||
)}{" "}
|
||||
of {run.quantTotal}
|
||||
</span>
|
||||
)}
|
||||
<span className="rounded-full border border-border/60 px-2.5 py-1 text-[10px] font-medium tabular-nums text-muted-foreground">
|
||||
|
|
@ -506,7 +535,10 @@ export function ExportRunPanel(props: ExportRunPanelProps) {
|
|||
}
|
||||
/>
|
||||
{run.stage && (
|
||||
<p className="truncate text-[11px] text-muted-foreground/80" title={run.stage}>
|
||||
<p
|
||||
className="truncate text-[11px] text-muted-foreground/80"
|
||||
title={run.stage}
|
||||
>
|
||||
{run.stage}
|
||||
</p>
|
||||
)}
|
||||
|
|
@ -556,10 +588,7 @@ export function ExportRunPanel(props: ExportRunPanelProps) {
|
|||
) : (
|
||||
<div className="whitespace-pre-wrap break-words">
|
||||
{run.logLines.map((entry, idx) => (
|
||||
<div
|
||||
key={idx}
|
||||
className={getExportLogLineClass(entry)}
|
||||
>
|
||||
<div key={idx} className={getExportLogLineClass(entry)}>
|
||||
{formatLogLine(entry)}
|
||||
</div>
|
||||
))}
|
||||
|
|
|
|||
|
|
@ -10,6 +10,7 @@ import {
|
|||
} from "@hugeicons/core-free-icons";
|
||||
import type { IconSvgElement } from "@hugeicons/react";
|
||||
import { HugeiconsIcon } from "@hugeicons/react";
|
||||
import type { ReactNode } from "react";
|
||||
import { useLayoutEffect, useRef, useState } from "react";
|
||||
|
||||
export function NetworkErrorState({
|
||||
|
|
@ -199,10 +200,12 @@ export function EmptyState({
|
|||
title,
|
||||
body,
|
||||
icon = CubeIcon,
|
||||
action,
|
||||
}: {
|
||||
title: string;
|
||||
body: string;
|
||||
icon?: IconSvgElement;
|
||||
action?: ReactNode;
|
||||
}) {
|
||||
return (
|
||||
<div className="flex min-h-[220px] flex-col items-center justify-center gap-3 px-6 text-center">
|
||||
|
|
@ -217,6 +220,7 @@ export function EmptyState({
|
|||
{body}
|
||||
</p>
|
||||
</div>
|
||||
{action}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -152,11 +152,7 @@ export function DatasetDownloadSection({
|
|||
/>
|
||||
)}
|
||||
{isDownloaded && cachePath && (
|
||||
<PathInfoButton
|
||||
path={cachePath}
|
||||
title="On-device location"
|
||||
description={`Where ${repoId} lives on disk.`}
|
||||
/>
|
||||
<PathInfoButton path={cachePath} />
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
|
|
|
|||
|
|
@ -1,9 +1,9 @@
|
|||
// SPDX-License-Identifier: AGPL-3.0-only
|
||||
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
import type { ModelInventoryFormat } from "../inventory";
|
||||
import { GgufDownloadCard } from "./gguf-download-card";
|
||||
import { SafetensorsDownloadCard } from "./safetensors-download-card";
|
||||
import type { ModelInventoryFormat } from "../inventory";
|
||||
|
||||
export function DownloadSection({
|
||||
repoId,
|
||||
|
|
@ -22,6 +22,7 @@ export function DownloadSection({
|
|||
knownBytes,
|
||||
onLoad,
|
||||
onUseInChat,
|
||||
onEject,
|
||||
onTrain,
|
||||
onChange,
|
||||
}: {
|
||||
|
|
@ -41,6 +42,7 @@ export function DownloadSection({
|
|||
knownBytes?: number | null;
|
||||
onLoad: (opts: { ggufVariant?: string; expectedBytes?: number }) => void;
|
||||
onUseInChat?: () => void;
|
||||
onEject?: () => void;
|
||||
onTrain?: () => void;
|
||||
onChange?: () => void;
|
||||
}) {
|
||||
|
|
@ -57,6 +59,7 @@ export function DownloadSection({
|
|||
isPartial={isPartial}
|
||||
onLoad={onLoad}
|
||||
onUseInChat={onUseInChat}
|
||||
onEject={onEject}
|
||||
onChange={onChange}
|
||||
/>
|
||||
);
|
||||
|
|
@ -75,6 +78,7 @@ export function DownloadSection({
|
|||
knownBytes={knownBytes}
|
||||
onLoad={onLoad}
|
||||
onUseInChat={onUseInChat}
|
||||
onEject={onEject}
|
||||
onTrain={onTrain}
|
||||
onChange={onChange}
|
||||
/>
|
||||
|
|
|
|||
|
|
@ -1,6 +1,13 @@
|
|||
// SPDX-License-Identifier: AGPL-3.0-only
|
||||
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
import {
|
||||
DropdownMenu,
|
||||
DropdownMenuContent,
|
||||
DropdownMenuItem,
|
||||
DropdownMenuSeparator,
|
||||
DropdownMenuTrigger,
|
||||
} from "@/components/ui/dropdown-menu";
|
||||
import {
|
||||
Popover,
|
||||
PopoverContent,
|
||||
|
|
@ -12,49 +19,55 @@ import {
|
|||
TooltipContent,
|
||||
TooltipTrigger,
|
||||
} from "@/components/ui/tooltip";
|
||||
import {
|
||||
downloadManager,
|
||||
useDownloadManagerStore,
|
||||
useRepoDownload,
|
||||
} from "../download-manager";
|
||||
import {
|
||||
type GgufVariantDetail,
|
||||
deleteCachedModel,
|
||||
} from "../inventory";
|
||||
import { formatBytes } from "../lib/format";
|
||||
import { type GgufFitClass, classifyGgufFit } from "../lib/gguf-fit";
|
||||
import { HUB_GGUF_RUN_ACTIONS_VISIBLE } from "../lib/hub-feature-flags";
|
||||
import {
|
||||
ggufVariantsMatch,
|
||||
normalizeGgufVariantIdentity,
|
||||
} from "../lib/model-identity";
|
||||
import { usePlatformStore } from "@/config/env";
|
||||
import { getCachedModelPath, revealCachedModel } from "@/features/chat";
|
||||
import { pinKey, usePinnedModelsStore } from "@/features/model-picker";
|
||||
import { ChevronDownStandardIcon } from "@/lib/chevron-icons";
|
||||
import { copyToClipboard } from "@/lib/copy-to-clipboard";
|
||||
import { toast } from "@/lib/toast";
|
||||
import { cn } from "@/lib/utils";
|
||||
import { useHfTokenStore } from "../stores/hf-token-store";
|
||||
import { useOnlineStatus } from "../hooks/use-online-status";
|
||||
import {
|
||||
ArrowReloadHorizontalIcon,
|
||||
Copy01Icon,
|
||||
Delete02Icon,
|
||||
Download01Icon,
|
||||
Folder01Icon,
|
||||
InformationCircleIcon,
|
||||
PencilEdit02Icon,
|
||||
MoreVerticalIcon,
|
||||
PinIcon,
|
||||
PinOffIcon,
|
||||
PlayIcon,
|
||||
RemoveCircleIcon,
|
||||
} from "@hugeicons/core-free-icons";
|
||||
import { ChevronDownStandardIcon } from "@/lib/chevron-icons";
|
||||
import { HugeiconsIcon } from "@hugeicons/react";
|
||||
import {
|
||||
type KeyboardEventHandler,
|
||||
memo,
|
||||
useCallback,
|
||||
useEffect,
|
||||
useMemo,
|
||||
useState,
|
||||
type KeyboardEventHandler,
|
||||
type MouseEventHandler,
|
||||
} from "react";
|
||||
import {
|
||||
downloadManager,
|
||||
useDownloadManagerStore,
|
||||
useRepoDownload,
|
||||
} from "../download-manager";
|
||||
import { useOnlineStatus } from "../hooks/use-online-status";
|
||||
import { type GgufVariantDetail, deleteCachedModel } from "../inventory";
|
||||
import { formatBytes } from "../lib/format";
|
||||
import { type GgufFitClass, classifyGgufFit } from "../lib/gguf-fit";
|
||||
import {
|
||||
ggufVariantDisplayLabel,
|
||||
ggufVariantDownloadSizeBytes,
|
||||
sortDownloadableGgufVariants,
|
||||
} from "../lib/gguf-variant-sort";
|
||||
import { HUB_GGUF_RUN_ACTIONS_VISIBLE } from "../lib/hub-feature-flags";
|
||||
import {
|
||||
ggufVariantsMatch,
|
||||
normalizeGgufVariantIdentity,
|
||||
} from "../lib/model-identity";
|
||||
import { useHfTokenStore } from "../stores/hf-token-store";
|
||||
import { DotTag } from "./dot-tag";
|
||||
import { DownloadCancelIndicator } from "./download-cancel-indicator";
|
||||
import {
|
||||
|
|
@ -72,7 +85,6 @@ import {
|
|||
GgufDownloadStatusCard,
|
||||
GgufDownloadingFallbackCard,
|
||||
} from "./gguf-status-cards";
|
||||
import { PathInfoButton } from "./path-info-button";
|
||||
import { useDeleteConfirmAction } from "./use-delete-confirm-action";
|
||||
import { useDownloadCardState } from "./use-download-card-state";
|
||||
import { useGgufVariantFetchState } from "./use-gguf-variant-fetch-state";
|
||||
|
|
@ -204,7 +216,7 @@ function QuantBadge({
|
|||
onOpenChange={tooltipMode === "lazy" ? setTooltipOpen : undefined}
|
||||
>
|
||||
<TooltipTrigger
|
||||
asChild
|
||||
asChild={true}
|
||||
onFocusCapture={tooltipMode === "lazy" ? armTooltip : undefined}
|
||||
onPointerEnter={tooltipMode === "lazy" ? armTooltip : undefined}
|
||||
>
|
||||
|
|
@ -245,7 +257,181 @@ function createGgufVariantMenuItems(
|
|||
}));
|
||||
}
|
||||
|
||||
// Shared options menu: used on every variant row, the run bar, and the
|
||||
// single-model (non-GGUF) run bar. Omit `quant` for a repo-level model. The
|
||||
// identifier uses llama.cpp's repo:quant syntax so it pastes into `-hf`.
|
||||
export function QuantOptionsMenu({
|
||||
repoId,
|
||||
quant,
|
||||
label,
|
||||
downloaded,
|
||||
canDelete,
|
||||
onDelete,
|
||||
showPin = true,
|
||||
buttonClassName,
|
||||
iconClassName,
|
||||
}: {
|
||||
repoId: string;
|
||||
quant?: string;
|
||||
label: string;
|
||||
downloaded: boolean;
|
||||
canDelete: boolean;
|
||||
onDelete: (quant?: string) => void;
|
||||
// Hidden in the run bar; pinning belongs to the On Device list.
|
||||
showPin?: boolean;
|
||||
buttonClassName?: string;
|
||||
iconClassName?: string;
|
||||
}) {
|
||||
const pinnedKeys = usePinnedModelsStore((s) => s.pinned);
|
||||
const togglePinned = usePinnedModelsStore((s) => s.togglePinned);
|
||||
const pinned = pinnedKeys.includes(pinKey(repoId, quant));
|
||||
const deviceType = usePlatformStore((s) => s.deviceType);
|
||||
const revealLabel =
|
||||
deviceType === "mac"
|
||||
? "Reveal in Finder"
|
||||
: deviceType === "windows"
|
||||
? "Reveal in File Explorer"
|
||||
: "Reveal in File Manager";
|
||||
const handleCopyPath = useCallback(async () => {
|
||||
try {
|
||||
const { path } = await getCachedModelPath(repoId, quant);
|
||||
if (await copyToClipboard(path)) {
|
||||
toast.success("Copied path");
|
||||
} else {
|
||||
toast.error("Failed to copy");
|
||||
}
|
||||
} catch (err) {
|
||||
toast.error(
|
||||
err instanceof Error ? err.message : "Failed to resolve model path",
|
||||
);
|
||||
}
|
||||
}, [repoId, quant]);
|
||||
const handleCopyId = useCallback(async () => {
|
||||
const id = quant ? `${repoId}:${quant}` : repoId;
|
||||
if (await copyToClipboard(id)) {
|
||||
toast.success("Copied identifier");
|
||||
} else {
|
||||
toast.error("Failed to copy");
|
||||
}
|
||||
}, [repoId, quant]);
|
||||
|
||||
return (
|
||||
<DropdownMenu>
|
||||
<DropdownMenuTrigger asChild={true}>
|
||||
<button
|
||||
type="button"
|
||||
onClick={(e) => e.stopPropagation()}
|
||||
aria-label={`More options for ${label}`}
|
||||
className={cn(
|
||||
"inline-flex size-6 shrink-0 cursor-pointer items-center justify-center rounded-full",
|
||||
"text-muted-foreground transition-colors hover:bg-muted hover:text-foreground",
|
||||
"data-[state=open]:bg-muted data-[state=open]:text-foreground",
|
||||
buttonClassName,
|
||||
)}
|
||||
>
|
||||
<HugeiconsIcon
|
||||
icon={MoreVerticalIcon}
|
||||
strokeWidth={1.75}
|
||||
className={cn("size-3.5", iconClassName)}
|
||||
/>
|
||||
</button>
|
||||
</DropdownMenuTrigger>
|
||||
<DropdownMenuContent
|
||||
side="bottom"
|
||||
align="end"
|
||||
sideOffset={2}
|
||||
className="unsloth-plus-menu menu-flat-destructive w-48"
|
||||
>
|
||||
{showPin && downloaded && (
|
||||
<DropdownMenuItem
|
||||
onSelect={(e) => {
|
||||
e.stopPropagation();
|
||||
togglePinned(repoId, quant);
|
||||
}}
|
||||
>
|
||||
<HugeiconsIcon
|
||||
icon={pinned ? PinOffIcon : PinIcon}
|
||||
strokeWidth={1.75}
|
||||
className="size-icon"
|
||||
/>
|
||||
<span>{pinned ? "Unpin" : "Pin to top"}</span>
|
||||
</DropdownMenuItem>
|
||||
)}
|
||||
{downloaded && (
|
||||
<DropdownMenuItem
|
||||
onSelect={(e) => {
|
||||
e.stopPropagation();
|
||||
revealCachedModel(repoId, quant).catch((err) => {
|
||||
toast.error(
|
||||
err instanceof Error
|
||||
? err.message
|
||||
: "Failed to open file manager",
|
||||
);
|
||||
});
|
||||
}}
|
||||
>
|
||||
<HugeiconsIcon
|
||||
icon={Folder01Icon}
|
||||
strokeWidth={1.75}
|
||||
className="size-icon"
|
||||
/>
|
||||
<span>{revealLabel}</span>
|
||||
</DropdownMenuItem>
|
||||
)}
|
||||
<DropdownMenuItem
|
||||
onSelect={(e) => {
|
||||
e.stopPropagation();
|
||||
void handleCopyId();
|
||||
}}
|
||||
>
|
||||
<HugeiconsIcon
|
||||
icon={Copy01Icon}
|
||||
strokeWidth={1.75}
|
||||
className="size-icon"
|
||||
/>
|
||||
<span>Copy identifier</span>
|
||||
</DropdownMenuItem>
|
||||
{downloaded && (
|
||||
<DropdownMenuItem
|
||||
onSelect={(e) => {
|
||||
e.stopPropagation();
|
||||
void handleCopyPath();
|
||||
}}
|
||||
>
|
||||
<HugeiconsIcon
|
||||
icon={Copy01Icon}
|
||||
strokeWidth={1.75}
|
||||
className="size-icon"
|
||||
/>
|
||||
<span>Copy path</span>
|
||||
</DropdownMenuItem>
|
||||
)}
|
||||
{canDelete && (
|
||||
<>
|
||||
<DropdownMenuSeparator />
|
||||
<DropdownMenuItem
|
||||
variant="destructive"
|
||||
onSelect={(e) => {
|
||||
e.stopPropagation();
|
||||
onDelete(quant);
|
||||
}}
|
||||
>
|
||||
<HugeiconsIcon
|
||||
icon={Delete02Icon}
|
||||
strokeWidth={1.75}
|
||||
className="size-icon"
|
||||
/>
|
||||
<span>Delete</span>
|
||||
</DropdownMenuItem>
|
||||
</>
|
||||
)}
|
||||
</DropdownMenuContent>
|
||||
</DropdownMenu>
|
||||
);
|
||||
}
|
||||
|
||||
const GgufVariantMenuRow = memo(function GgufVariantMenuRow({
|
||||
repoId,
|
||||
item,
|
||||
selected,
|
||||
loaded,
|
||||
|
|
@ -254,6 +440,7 @@ const GgufVariantMenuRow = memo(function GgufVariantMenuRow({
|
|||
onSelect,
|
||||
onDelete,
|
||||
}: {
|
||||
repoId: string;
|
||||
item: GgufVariantMenuItem;
|
||||
selected: boolean;
|
||||
loaded: boolean;
|
||||
|
|
@ -275,13 +462,6 @@ const GgufVariantMenuRow = memo(function GgufVariantMenuRow({
|
|||
},
|
||||
[selectVariant],
|
||||
);
|
||||
const handleDelete = useCallback<MouseEventHandler<HTMLButtonElement>>(
|
||||
(e) => {
|
||||
e.stopPropagation();
|
||||
onDelete(item.quant);
|
||||
},
|
||||
[item.quant, onDelete],
|
||||
);
|
||||
const canDelete = (item.downloaded || item.partial) && !loaded && !liveActive;
|
||||
|
||||
return (
|
||||
|
|
@ -317,7 +497,7 @@ const GgufVariantMenuRow = memo(function GgufVariantMenuRow({
|
|||
)}
|
||||
{!item.downloaded && item.partial && (
|
||||
<Tooltip>
|
||||
<TooltipTrigger asChild>
|
||||
<TooltipTrigger asChild={true}>
|
||||
<span className="inline-flex">
|
||||
<DotTag
|
||||
tone="warning"
|
||||
|
|
@ -334,30 +514,23 @@ const GgufVariantMenuRow = memo(function GgufVariantMenuRow({
|
|||
)}
|
||||
</span>
|
||||
<span className="ml-auto flex shrink-0 items-center gap-1.5">
|
||||
<span className="relative">
|
||||
<span className={cn(CHIP_BASE, CHIP_DEFAULT)}>
|
||||
{item.downloadSizeLabel}
|
||||
</span>
|
||||
{canDelete && (
|
||||
<button
|
||||
type="button"
|
||||
onClick={handleDelete}
|
||||
aria-label={`Delete ${item.label}${item.partial && !item.downloaded ? " (partial)" : ""}`}
|
||||
className={cn(
|
||||
"absolute inset-0 inline-flex cursor-pointer items-center justify-center rounded-full",
|
||||
"bg-popover text-foreground/70 ring-1 ring-border transition-colors",
|
||||
"opacity-0 group-hover:opacity-100 focus-visible:opacity-100",
|
||||
"hover:text-destructive hover:ring-destructive/40",
|
||||
)}
|
||||
>
|
||||
<HugeiconsIcon
|
||||
icon={Delete02Icon}
|
||||
strokeWidth={1.75}
|
||||
className="size-3"
|
||||
/>
|
||||
</button>
|
||||
)}
|
||||
<span className={cn(CHIP_BASE, CHIP_DEFAULT)}>
|
||||
{item.downloadSizeLabel}
|
||||
</span>
|
||||
{/* Options only apply to files on disk; placeholder keeps the size
|
||||
chips column-aligned across rows. */}
|
||||
{item.downloaded || item.partial ? (
|
||||
<QuantOptionsMenu
|
||||
repoId={repoId}
|
||||
quant={item.quant}
|
||||
label={item.label}
|
||||
downloaded={Boolean(item.downloaded)}
|
||||
canDelete={canDelete}
|
||||
onDelete={(q) => q && onDelete(q)}
|
||||
/>
|
||||
) : (
|
||||
<span aria-hidden={true} className="size-6 shrink-0" />
|
||||
)}
|
||||
</span>
|
||||
</div>
|
||||
);
|
||||
|
|
@ -374,7 +547,7 @@ export function GgufDownloadCard({
|
|||
preferLocalCache = false,
|
||||
isPartial = false,
|
||||
onLoad,
|
||||
onUseInChat,
|
||||
onEject,
|
||||
onChange,
|
||||
}: {
|
||||
repoId: string;
|
||||
|
|
@ -387,7 +560,9 @@ export function GgufDownloadCard({
|
|||
preferLocalCache?: boolean;
|
||||
isPartial?: boolean;
|
||||
onLoad: (opts: { ggufVariant?: string; expectedBytes?: number }) => void;
|
||||
/** Accepted for API parity; the run bar ejects instead of opening chat. */
|
||||
onUseInChat?: () => void;
|
||||
onEject?: () => void;
|
||||
onChange?: () => void;
|
||||
}) {
|
||||
const hfToken = useHfTokenStore((s) => s.token);
|
||||
|
|
@ -422,7 +597,9 @@ export function GgufDownloadCard({
|
|||
() => createLiveGgufVariantStatesSelector(repoId),
|
||||
[repoId],
|
||||
);
|
||||
const liveVariantStates = useDownloadManagerStore(selectLiveGgufVariantStates);
|
||||
const liveVariantStates = useDownloadManagerStore(
|
||||
selectLiveGgufVariantStates,
|
||||
);
|
||||
const sortedVariants = useMemo(() => {
|
||||
if (!rawSortedVariants) return null;
|
||||
const withLive = applyLiveGgufVariantStates(
|
||||
|
|
@ -499,12 +676,7 @@ export function GgufDownloadCard({
|
|||
if (expectedBytes > progress.expectedBytes) {
|
||||
setExpectedBytes(expectedBytes, progress.variant);
|
||||
}
|
||||
}, [
|
||||
variants,
|
||||
progress?.variant,
|
||||
progress?.expectedBytes,
|
||||
setExpectedBytes,
|
||||
]);
|
||||
}, [variants, progress?.variant, progress?.expectedBytes, setExpectedBytes]);
|
||||
|
||||
useEffect(() => {
|
||||
setCompletedVariantKeys(new Set<string>());
|
||||
|
|
@ -595,7 +767,8 @@ export function GgufDownloadCard({
|
|||
if (!deleteTarget) return;
|
||||
await deleteCachedModel(repoId, deleteTarget, hfToken || undefined);
|
||||
},
|
||||
successMessage: () => `Deleted ${repoId} ${deleteTargetLabel ?? deleteTarget}`,
|
||||
successMessage: () =>
|
||||
`Deleted ${repoId} ${deleteTargetLabel ?? deleteTarget}`,
|
||||
errorToast: (err) => ({
|
||||
title: err instanceof Error ? err.message : "Failed to delete",
|
||||
}),
|
||||
|
|
@ -654,7 +827,7 @@ export function GgufDownloadCard({
|
|||
return (
|
||||
<GgufDownloadStatusCard
|
||||
job={job}
|
||||
loading
|
||||
loading={true}
|
||||
message="Loading available quantizations…"
|
||||
/>
|
||||
);
|
||||
|
|
@ -666,7 +839,7 @@ export function GgufDownloadCard({
|
|||
<GgufDownloadStatusCard
|
||||
job={job}
|
||||
tone="muted"
|
||||
partial
|
||||
partial={true}
|
||||
message="Partial download present. Couldn't load quantizations."
|
||||
actionLabel="Reload"
|
||||
onAction={() => void refresh()}
|
||||
|
|
@ -729,7 +902,7 @@ export function GgufDownloadCard({
|
|||
}
|
||||
>
|
||||
<Popover open={open} onOpenChange={setOpen}>
|
||||
<PopoverTrigger asChild>
|
||||
<PopoverTrigger asChild={true}>
|
||||
<button
|
||||
type="button"
|
||||
onClick={(e) => {
|
||||
|
|
@ -762,7 +935,7 @@ export function GgufDownloadCard({
|
|||
)}
|
||||
{selected && !selected.downloaded && selected.partial && (
|
||||
<Tooltip>
|
||||
<TooltipTrigger asChild>
|
||||
<TooltipTrigger asChild={true}>
|
||||
<span className="inline-flex">
|
||||
<DotTag
|
||||
tone="warning"
|
||||
|
|
@ -806,6 +979,7 @@ export function GgufDownloadCard({
|
|||
return (
|
||||
<GgufVariantMenuRow
|
||||
key={item.filename}
|
||||
repoId={repoId}
|
||||
item={item}
|
||||
selected={item.key === selectedVariantKey}
|
||||
loaded={isActive && item.key === activeVariantKey}
|
||||
|
|
@ -820,14 +994,29 @@ export function GgufDownloadCard({
|
|||
</PopoverContent>
|
||||
</Popover>
|
||||
|
||||
{selected?.downloaded && cachePath && (
|
||||
<PathInfoButton
|
||||
path={cachePath}
|
||||
title="On-device location"
|
||||
description={`Where ${repoId} (${selectedLabel}) lives on disk.`}
|
||||
className="ml-0.5"
|
||||
/>
|
||||
)}
|
||||
{/* TODO: inference settings gear hidden for now, work on it in a future PR. */}
|
||||
{/* Options only resolve managed HF-cache repos, so skip local paths;
|
||||
they also only apply to quants actually on disk. */}
|
||||
{selected &&
|
||||
Boolean(selected.downloaded || selected.partial) &&
|
||||
!/^([/\\~.]|[A-Za-z]:)/.test(repoId) && (
|
||||
<QuantOptionsMenu
|
||||
repoId={repoId}
|
||||
quant={selected.quant}
|
||||
label={`${repoId} ${selectedLabel}`}
|
||||
downloaded={Boolean(selected.downloaded)}
|
||||
canDelete={
|
||||
Boolean(selected.downloaded || selected.partial) &&
|
||||
!selectedIsActive &&
|
||||
!downloadingThisVariant &&
|
||||
!isLoadingThisModel
|
||||
}
|
||||
onDelete={(q) => q && handleDeleteVariant(q)}
|
||||
showPin={false}
|
||||
buttonClassName="ml-0.5 size-7"
|
||||
iconClassName="size-4"
|
||||
/>
|
||||
)}
|
||||
|
||||
{!isGgufRunCta && <CardDivider />}
|
||||
|
||||
|
|
@ -859,7 +1048,7 @@ export function GgufDownloadCard({
|
|||
return;
|
||||
}
|
||||
if (selectedIsActive) {
|
||||
onUseInChat?.();
|
||||
onEject?.();
|
||||
return;
|
||||
}
|
||||
if (!selected) return;
|
||||
|
|
@ -874,7 +1063,7 @@ export function GgufDownloadCard({
|
|||
}}
|
||||
aria-label={downloadAction.ariaLabel}
|
||||
className={cn(
|
||||
isGgufRunCta ? "hub-run-action-btn w-28" : "hub-action-btn w-28",
|
||||
isGgufRunCta ? "hub-run-action-btn w-24" : "hub-action-btn w-24",
|
||||
isGgufRunCta && "ml-2",
|
||||
ctaDisabled &&
|
||||
!selectedIsActive &&
|
||||
|
|
@ -917,8 +1106,8 @@ export function GgufDownloadCard({
|
|||
</span>
|
||||
) : selectedIsActive ? (
|
||||
<>
|
||||
<HugeiconsIcon icon={PencilEdit02Icon} strokeWidth={1.75} />
|
||||
New Chat
|
||||
<HugeiconsIcon icon={RemoveCircleIcon} strokeWidth={1.75} />
|
||||
Eject
|
||||
</>
|
||||
) : selected?.downloaded ? (
|
||||
<>
|
||||
|
|
|
|||
|
|
@ -6,9 +6,9 @@ import {
|
|||
PopoverContent,
|
||||
PopoverTrigger,
|
||||
} from "@/components/ui/popover";
|
||||
import { cn } from "@/lib/utils";
|
||||
import { Tick02Icon } from "@/lib/tick-icon";
|
||||
import { ChevronDownStandardIcon } from "@/lib/chevron-icons";
|
||||
import { Tick02Icon } from "@/lib/tick-icon";
|
||||
import { cn } from "@/lib/utils";
|
||||
import { HugeiconsIcon } from "@hugeicons/react";
|
||||
import {
|
||||
type KeyboardEvent,
|
||||
|
|
@ -71,7 +71,9 @@ export function HubOptionMenu<T extends string>({
|
|||
? -1
|
||||
: Math.min(activeIndex, options.length - 1);
|
||||
const activeOptionId =
|
||||
resolvedActiveIndex >= 0 ? `${idBase}-option-${resolvedActiveIndex}` : undefined;
|
||||
resolvedActiveIndex >= 0
|
||||
? `${idBase}-option-${resolvedActiveIndex}`
|
||||
: undefined;
|
||||
|
||||
const activateIndex = useCallback((index: number) => {
|
||||
setActiveIndex((current) => (current === index ? current : index));
|
||||
|
|
@ -161,7 +163,7 @@ export function HubOptionMenu<T extends string>({
|
|||
|
||||
return (
|
||||
<Popover open={open} onOpenChange={handleOpenChange}>
|
||||
<PopoverTrigger asChild>
|
||||
<PopoverTrigger asChild={true}>
|
||||
<button
|
||||
ref={triggerRef}
|
||||
type="button"
|
||||
|
|
@ -171,7 +173,7 @@ export function HubOptionMenu<T extends string>({
|
|||
aria-label={ariaLabel}
|
||||
title={title}
|
||||
className={cn(
|
||||
"field-trigger hub-menu-trigger field-soft field-filter inline-flex h-9 shrink-0 cursor-pointer items-center justify-between gap-0.5 rounded-full pl-3 pr-2 text-[12.5px] transition-colors",
|
||||
"field-trigger hub-menu-trigger field-soft field-filter inline-flex h-9 shrink-0 cursor-pointer items-center justify-between gap-2.5 rounded-full pl-3 pr-2.5 text-[12.5px] transition-colors",
|
||||
"focus-visible:border-border focus-visible:ring-0 focus-visible:ring-offset-0",
|
||||
className,
|
||||
)}
|
||||
|
|
@ -183,7 +185,10 @@ export function HubOptionMenu<T extends string>({
|
|||
}}
|
||||
>
|
||||
<span className="min-w-0 flex-1 truncate text-left">
|
||||
{triggerContent ?? selected?.triggerLabel ?? selected?.label ?? value}
|
||||
{triggerContent ??
|
||||
selected?.triggerLabel ??
|
||||
selected?.label ??
|
||||
value}
|
||||
</span>
|
||||
{showChevron && (
|
||||
<HugeiconsIcon
|
||||
|
|
|
|||
|
|
@ -34,11 +34,7 @@ export function LocalDatasetCard({
|
|||
)}
|
||||
</span>
|
||||
<div className="ml-auto flex items-center gap-0.5">
|
||||
<PathInfoButton
|
||||
path={path}
|
||||
title={sourceLabel}
|
||||
description="Where this dataset lives on disk."
|
||||
/>
|
||||
<PathInfoButton path={path} />
|
||||
</div>
|
||||
</div>
|
||||
{HUB_POST_DOWNLOAD_ACTIONS_VISIBLE && (
|
||||
|
|
|
|||
|
|
@ -1,12 +1,6 @@
|
|||
// SPDX-License-Identifier: AGPL-3.0-only
|
||||
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
import { TrainIcon } from "../components/train-icon";
|
||||
import {
|
||||
HUB_GGUF_RUN_ACTIONS_VISIBLE,
|
||||
HUB_NON_GGUF_RUN_ACTIONS_VISIBLE,
|
||||
HUB_POST_DOWNLOAD_ACTIONS_VISIBLE,
|
||||
} from "../lib/hub-feature-flags";
|
||||
import {
|
||||
Popover,
|
||||
PopoverContent,
|
||||
|
|
@ -18,37 +12,44 @@ import {
|
|||
TooltipContent,
|
||||
TooltipTrigger,
|
||||
} from "@/components/ui/tooltip";
|
||||
import { ChevronDownStandardIcon } from "@/lib/chevron-icons";
|
||||
import { cn } from "@/lib/utils";
|
||||
import {
|
||||
type BaseModelSource,
|
||||
type LocalModelInfo,
|
||||
type ModelInventoryFormat,
|
||||
deleteCachedModel,
|
||||
} from "../inventory";
|
||||
Alert02Icon,
|
||||
CubeIcon,
|
||||
PlayIcon,
|
||||
RemoveCircleIcon,
|
||||
Share05Icon,
|
||||
} from "@hugeicons/core-free-icons";
|
||||
import { HugeiconsIcon } from "@hugeicons/react";
|
||||
import { useCallback, useMemo, useState } from "react";
|
||||
import { TrainIcon } from "../components/train-icon";
|
||||
import {
|
||||
downloadManager,
|
||||
jobKeyOf,
|
||||
selectActiveJob,
|
||||
useDownloadManagerStore,
|
||||
} from "../download-manager";
|
||||
import { formatBytes } from "../lib/format";
|
||||
import { ggufVariantsMatch } from "../lib/model-identity";
|
||||
import { cn } from "@/lib/utils";
|
||||
import { confirmExternalLink } from "../stores/external-link-confirm";
|
||||
import { useHfTokenStore } from "../stores/hf-token-store";
|
||||
import { useOnlineStatus } from "../hooks/use-online-status";
|
||||
import {
|
||||
Alert02Icon,
|
||||
CubeIcon,
|
||||
PencilEdit02Icon,
|
||||
PlayIcon,
|
||||
Share05Icon,
|
||||
} from "@hugeicons/core-free-icons";
|
||||
import { ChevronDownStandardIcon } from "@/lib/chevron-icons";
|
||||
import { HugeiconsIcon } from "@hugeicons/react";
|
||||
import { useCallback, useMemo, useState } from "react";
|
||||
type BaseModelSource,
|
||||
type LocalModelInfo,
|
||||
type ModelInventoryFormat,
|
||||
deleteCachedModel,
|
||||
} from "../inventory";
|
||||
import { formatBytes } from "../lib/format";
|
||||
import {
|
||||
ggufVariantDisplayLabel,
|
||||
sortLocalGgufVariants,
|
||||
} from "../lib/gguf-variant-sort";
|
||||
import {
|
||||
HUB_GGUF_RUN_ACTIONS_VISIBLE,
|
||||
HUB_NON_GGUF_RUN_ACTIONS_VISIBLE,
|
||||
HUB_POST_DOWNLOAD_ACTIONS_VISIBLE,
|
||||
} from "../lib/hub-feature-flags";
|
||||
import { ggufVariantsMatch } from "../lib/model-identity";
|
||||
import { confirmExternalLink } from "../stores/external-link-confirm";
|
||||
import { useHfTokenStore } from "../stores/hf-token-store";
|
||||
import { DotTag } from "./dot-tag";
|
||||
import {
|
||||
CardDeleteButton,
|
||||
|
|
@ -60,7 +61,6 @@ import { PathInfoButton } from "./path-info-button";
|
|||
import { TransportConflictDialog } from "./transport-conflict-dialog";
|
||||
import { useCardDelete } from "./use-card-delete";
|
||||
import { useGgufVariantFetchState } from "./use-gguf-variant-fetch-state";
|
||||
import { useOnlineStatus } from "../hooks/use-online-status";
|
||||
|
||||
type LocalLoadOptions = {
|
||||
ggufVariant?: string;
|
||||
|
|
@ -91,7 +91,9 @@ interface LocalOnDeviceCardProps {
|
|||
systemRamGb?: number;
|
||||
unsupportedReason?: string | null;
|
||||
onLoad: (opts?: LocalLoadOptions) => void;
|
||||
/** Accepted for API parity; the run bar ejects instead of opening chat. */
|
||||
onUseInChat: () => void;
|
||||
onEject?: () => void;
|
||||
onTrain?: () => void;
|
||||
onChange?: () => void;
|
||||
}
|
||||
|
|
@ -151,7 +153,7 @@ function BaseModelReference({
|
|||
</div>
|
||||
{canOpenHub && (
|
||||
<Tooltip>
|
||||
<TooltipTrigger asChild>
|
||||
<TooltipTrigger asChild={true}>
|
||||
<a
|
||||
href={`https://huggingface.co/${baseModelHubId}`}
|
||||
target="_blank"
|
||||
|
|
@ -160,7 +162,11 @@ function BaseModelReference({
|
|||
className="inline-flex size-7 shrink-0 items-center justify-center rounded-full text-muted-foreground transition-colors hover:bg-background hover:text-foreground"
|
||||
onClick={(event) => {
|
||||
event.stopPropagation();
|
||||
if (confirmExternalLink(`https://huggingface.co/${baseModelHubId}`)) {
|
||||
if (
|
||||
confirmExternalLink(
|
||||
`https://huggingface.co/${baseModelHubId}`,
|
||||
)
|
||||
) {
|
||||
event.preventDefault();
|
||||
}
|
||||
}}
|
||||
|
|
@ -205,7 +211,7 @@ export function LocalOnDeviceCard({
|
|||
systemRamGb,
|
||||
unsupportedReason,
|
||||
onLoad,
|
||||
onUseInChat,
|
||||
onEject,
|
||||
onTrain,
|
||||
onChange,
|
||||
}: LocalOnDeviceCardProps) {
|
||||
|
|
@ -371,18 +377,20 @@ export function LocalOnDeviceCard({
|
|||
const handleConfirmUpdate = () => {
|
||||
if (!repoId || !updateTargetVariant) return;
|
||||
setUpdateOpen(false);
|
||||
void downloadManager.requestStart({
|
||||
kind: "model",
|
||||
repoId,
|
||||
variant: updateTargetVariant,
|
||||
expectedBytes: updateExpectedBytes,
|
||||
}).then((outcome) => {
|
||||
if (outcome === "conflict") {
|
||||
setUpdateConflictKey(jobKeyOf("model", repoId, updateTargetVariant));
|
||||
}
|
||||
void currentVariantState.refresh();
|
||||
void remoteVariantState.refresh();
|
||||
});
|
||||
void downloadManager
|
||||
.requestStart({
|
||||
kind: "model",
|
||||
repoId,
|
||||
variant: updateTargetVariant,
|
||||
expectedBytes: updateExpectedBytes,
|
||||
})
|
||||
.then((outcome) => {
|
||||
if (outcome === "conflict") {
|
||||
setUpdateConflictKey(jobKeyOf("model", repoId, updateTargetVariant));
|
||||
}
|
||||
void currentVariantState.refresh();
|
||||
void remoteVariantState.refresh();
|
||||
});
|
||||
};
|
||||
const selectedVariantIsActive =
|
||||
needsVariantSelection && selectedQuant
|
||||
|
|
@ -428,7 +436,8 @@ export function LocalOnDeviceCard({
|
|||
can still keep it on disk, or delete it to free space.
|
||||
</span>
|
||||
</div>
|
||||
)}<div className="hub-download-card">
|
||||
)}
|
||||
<div className="hub-download-card">
|
||||
<div className="group/dl flex items-center">
|
||||
<div className="relative flex h-9 min-w-0 flex-1 items-center pl-3 pr-2">
|
||||
<span className="flex min-w-0 items-center gap-1.5 text-[12px] text-muted-foreground">
|
||||
|
|
@ -540,7 +549,7 @@ export function LocalOnDeviceCard({
|
|||
{canUpdate && (
|
||||
<CardUpdateButton
|
||||
label={`Update ${repoId}`}
|
||||
emphasized
|
||||
emphasized={true}
|
||||
onClick={() => setUpdateOpen(true)}
|
||||
/>
|
||||
)}
|
||||
|
|
@ -550,11 +559,7 @@ export function LocalOnDeviceCard({
|
|||
onClick={() => setDeleteOpen(true)}
|
||||
/>
|
||||
)}
|
||||
<PathInfoButton
|
||||
path={path}
|
||||
title={sourceLabel}
|
||||
description="Where this model lives on disk."
|
||||
/>
|
||||
<PathInfoButton path={path} />
|
||||
</div>
|
||||
</div>
|
||||
{onTrain && HUB_POST_DOWNLOAD_ACTIONS_VISIBLE && (
|
||||
|
|
@ -585,7 +590,7 @@ export function LocalOnDeviceCard({
|
|||
onClick={() => {
|
||||
if (!canRun) return;
|
||||
if (selectedVariantIsActive) {
|
||||
onUseInChat();
|
||||
onEject?.();
|
||||
return;
|
||||
}
|
||||
if (needsVariantSelection) {
|
||||
|
|
@ -615,24 +620,24 @@ export function LocalOnDeviceCard({
|
|||
</>
|
||||
) : selectedVariantIsActive ? (
|
||||
<>
|
||||
<HugeiconsIcon icon={PencilEdit02Icon} strokeWidth={1.75} />
|
||||
Chat
|
||||
<HugeiconsIcon icon={RemoveCircleIcon} strokeWidth={1.75} />
|
||||
Eject
|
||||
</>
|
||||
) : variantActionPending ? (
|
||||
<>
|
||||
<Spinner />
|
||||
Loading…
|
||||
</>
|
||||
) : !canRun ? (
|
||||
<>
|
||||
<HugeiconsIcon icon={Alert02Icon} strokeWidth={1.75} />
|
||||
No run
|
||||
</>
|
||||
) : (
|
||||
) : canRun ? (
|
||||
<>
|
||||
<HugeiconsIcon icon={PlayIcon} strokeWidth={1.75} />
|
||||
Run
|
||||
</>
|
||||
) : (
|
||||
<>
|
||||
<HugeiconsIcon icon={Alert02Icon} strokeWidth={1.75} />
|
||||
No run
|
||||
</>
|
||||
)}
|
||||
</button>
|
||||
</div>
|
||||
|
|
|
|||
|
|
@ -17,9 +17,9 @@ import {
|
|||
formatRelativeShort,
|
||||
formatShortDate,
|
||||
} from "@/features/hub/lib/format";
|
||||
import { cn, formatCompact } from "@/lib/utils";
|
||||
import { confirmExternalLink } from "../stores/external-link-confirm";
|
||||
import { useHfTokenStore } from "@/features/hub/stores/hf-token-store";
|
||||
import { Tick02Icon } from "@/lib/tick-icon";
|
||||
import { cn, formatCompact } from "@/lib/utils";
|
||||
import {
|
||||
Calendar03Icon,
|
||||
CalendarAdd01Icon,
|
||||
|
|
@ -37,10 +37,10 @@ import {
|
|||
RamMemoryIcon,
|
||||
Share05Icon,
|
||||
} from "@hugeicons/core-free-icons";
|
||||
import { Tick02Icon } from "@/lib/tick-icon";
|
||||
import type { IconSvgElement } from "@hugeicons/react";
|
||||
import { HugeiconsIcon } from "@hugeicons/react";
|
||||
import { memo, useDeferredValue, useMemo } from "react";
|
||||
import { selectActiveJob, useDownloadManagerStore } from "../download-manager";
|
||||
import { useCopyFeedback } from "../hooks/use-copy-feedback";
|
||||
import { useDatasetSize } from "../hooks/use-dataset-size";
|
||||
import {
|
||||
|
|
@ -49,8 +49,8 @@ import {
|
|||
formatPipelineTag,
|
||||
parseLanguageTags,
|
||||
} from "../lib/view-models";
|
||||
import { confirmExternalLink } from "../stores/external-link-confirm";
|
||||
import type { SelectedModelView } from "../types";
|
||||
import { selectActiveJob, useDownloadManagerStore } from "../download-manager";
|
||||
import { DatasetDownloadSection } from "./dataset-download-section";
|
||||
import { DownloadSection } from "./download-section";
|
||||
import { LocalDatasetCard } from "./local-dataset-card";
|
||||
|
|
@ -399,6 +399,7 @@ export type ModelInspectorActions = {
|
|||
expectedBytes?: number;
|
||||
}) => void;
|
||||
onUseInChat: () => void;
|
||||
onEject?: () => void;
|
||||
onTrain?: () => void;
|
||||
onInventoryChange?: () => void;
|
||||
onSearchHub?: (query: string) => void;
|
||||
|
|
@ -433,6 +434,7 @@ export const ModelInspector = memo(function ModelInspector({
|
|||
onLoad,
|
||||
onLoadLocal,
|
||||
onUseInChat,
|
||||
onEject,
|
||||
onTrain,
|
||||
onInventoryChange,
|
||||
onSearchHub,
|
||||
|
|
@ -517,7 +519,9 @@ export const ModelInspector = memo(function ModelInspector({
|
|||
? formatRelativeShort(model.updatedAt)
|
||||
: formatLocalUpdated(model.localUpdatedAt);
|
||||
const updatedLabel = updatedRaw === "Unknown update" ? "N/A" : updatedRaw;
|
||||
const createdLabel = model.createdAt ? formatShortDate(model.createdAt) : null;
|
||||
const createdLabel = model.createdAt
|
||||
? formatShortDate(model.createdAt)
|
||||
: null;
|
||||
const libraryLabel = isDataset ? null : formatLibrary(model.libraryName);
|
||||
const gatedAccess = model.gated !== false && model.gated !== undefined;
|
||||
const downloadsTooltip =
|
||||
|
|
@ -696,6 +700,7 @@ export const ModelInspector = memo(function ModelInspector({
|
|||
}
|
||||
onLoad={onLoadLocal}
|
||||
onUseInChat={onUseInChat}
|
||||
onEject={onEject}
|
||||
onTrain={
|
||||
model.isDownloaded && canTrainModel ? onTrain : undefined
|
||||
}
|
||||
|
|
@ -719,6 +724,7 @@ export const ModelInspector = memo(function ModelInspector({
|
|||
knownBytes={model.cachedBytes}
|
||||
onLoad={model.isLocal ? onLoadLocal : onLoad}
|
||||
onUseInChat={onUseInChat}
|
||||
onEject={onEject}
|
||||
onTrain={
|
||||
model.isDownloaded && canTrainModel ? onTrain : undefined
|
||||
}
|
||||
|
|
|
|||
|
|
@ -2,13 +2,20 @@
|
|||
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
import { Spinner } from "@/components/ui/spinner";
|
||||
import {
|
||||
makePinRank,
|
||||
pinKey,
|
||||
usePinnedModelsStore,
|
||||
} from "@/features/model-picker";
|
||||
import {
|
||||
CubeIcon,
|
||||
DownloadCircle02Icon,
|
||||
FolderSearchIcon,
|
||||
PinIcon,
|
||||
Search01Icon,
|
||||
} from "@hugeicons/core-free-icons";
|
||||
import { HugeiconsIcon } from "@hugeicons/react";
|
||||
import type { RefObject } from "react";
|
||||
import { useMemo } from "react";
|
||||
import { useLayoutEffect, useMemo, useState } from "react";
|
||||
import {
|
||||
inventoryRowMatches,
|
||||
scoreInventoryRow,
|
||||
|
|
@ -208,7 +215,7 @@ export function DiscoverList({
|
|||
<SkeletonList />
|
||||
) : (
|
||||
<EmptyState
|
||||
icon={query.trim() ? FolderSearchIcon : CubeIcon}
|
||||
icon={query.trim() ? Search01Icon : CubeIcon}
|
||||
title={
|
||||
query.trim()
|
||||
? `No matching ${isDataset ? "datasets" : "models"}`
|
||||
|
|
@ -244,6 +251,8 @@ export function DownloadedList({
|
|||
downloadedReady,
|
||||
inventoryError,
|
||||
query,
|
||||
typeFilterActive = false,
|
||||
onClearFilters,
|
||||
scrollElement,
|
||||
columns = 1,
|
||||
activeCheckpoint,
|
||||
|
|
@ -262,6 +271,8 @@ export function DownloadedList({
|
|||
downloadedReady: boolean;
|
||||
inventoryError: boolean;
|
||||
query: string;
|
||||
typeFilterActive?: boolean;
|
||||
onClearFilters?: () => void;
|
||||
scrollElement: HTMLDivElement | null;
|
||||
columns?: number;
|
||||
activeCheckpoint: string | null;
|
||||
|
|
@ -274,11 +285,20 @@ export function DownloadedList({
|
|||
sort: InventorySort;
|
||||
onInventoryChange?: () => void;
|
||||
}) {
|
||||
// Pinned repos surface first regardless of the active sort; the chosen sort
|
||||
// still orders rows within the pinned and unpinned groups.
|
||||
const pinnedIds = usePinnedModelsStore((s) => s.pinned);
|
||||
const pinnedSet = useMemo(() => new Set(pinnedIds), [pinnedIds]);
|
||||
const inventoryItems = useMemo<InventoryItem[]>(() => {
|
||||
const merged: InventoryItem[] = [
|
||||
...cachedRows.map((row) => ({ variant: "cached" as const, row })),
|
||||
...localRows.map((row) => ({ variant: "local" as const, row })),
|
||||
];
|
||||
// Pinned rows order by pin recency (newest pin first), not the active
|
||||
// sort, so "Pin to top" puts the row exactly where the user expects.
|
||||
const rank = makePinRank(pinnedIds);
|
||||
const pinRank = (item: InventoryItem) =>
|
||||
item.row.repoId ? rank(pinKey(item.row.repoId)) : Number.MAX_SAFE_INTEGER;
|
||||
if (inventoryTokens.length > 0) {
|
||||
return merged
|
||||
.map((item, index) => ({
|
||||
|
|
@ -286,25 +306,85 @@ export function DownloadedList({
|
|||
index,
|
||||
score: scoreInventoryRow(item.row, inventoryTokens),
|
||||
}))
|
||||
.sort((a, b) => b.score - a.score || a.index - b.index)
|
||||
.sort(
|
||||
(a, b) =>
|
||||
pinRank(a.item) - pinRank(b.item) ||
|
||||
b.score - a.score ||
|
||||
a.index - b.index,
|
||||
)
|
||||
.map((entry) => entry.item);
|
||||
}
|
||||
if (sort === "recent") {
|
||||
return merged;
|
||||
return merged
|
||||
.map((item, index) => ({ item, index }))
|
||||
.sort((a, b) => pinRank(a.item) - pinRank(b.item) || a.index - b.index)
|
||||
.map((entry) => entry.item);
|
||||
}
|
||||
return merged
|
||||
.map((item, index) => ({ item, index }))
|
||||
.sort((a, b) =>
|
||||
sort === "name"
|
||||
? inventoryItemTitle(a.item).localeCompare(
|
||||
inventoryItemTitle(b.item),
|
||||
) || a.index - b.index
|
||||
: inventoryItemSize(b.item) - inventoryItemSize(a.item) ||
|
||||
a.index - b.index,
|
||||
.sort(
|
||||
(a, b) =>
|
||||
pinRank(a.item) - pinRank(b.item) ||
|
||||
(sort === "name"
|
||||
? inventoryItemTitle(a.item).localeCompare(
|
||||
inventoryItemTitle(b.item),
|
||||
) || a.index - b.index
|
||||
: inventoryItemSize(b.item) - inventoryItemSize(a.item) ||
|
||||
a.index - b.index),
|
||||
)
|
||||
.map((entry) => entry.item);
|
||||
}, [cachedRows, localRows, inventoryTokens, sort]);
|
||||
}, [cachedRows, localRows, inventoryTokens, sort, pinnedIds]);
|
||||
const hasInventoryRows = cachedRows.length > 0 || localRows.length > 0;
|
||||
// Pinned repos get their own labelled section so it's clear why they lead
|
||||
// the list; inventoryItems already sorts them first, so this is a prefix.
|
||||
const pinnedCount = useMemo(
|
||||
() =>
|
||||
inventoryItems.filter(
|
||||
(item) => item.row.repoId && pinnedSet.has(pinKey(item.row.repoId)),
|
||||
).length,
|
||||
[inventoryItems, pinnedSet],
|
||||
);
|
||||
const pinnedItems = inventoryItems.slice(0, pinnedCount);
|
||||
const unpinnedItems = inventoryItems.slice(pinnedCount);
|
||||
const [virtualRowsWrapper, setVirtualRowsWrapper] =
|
||||
useState<HTMLDivElement | null>(null);
|
||||
const [scrollMargin, setScrollMargin] = useState(0);
|
||||
useLayoutEffect(() => {
|
||||
if (!virtualRowsWrapper || !scrollElement) return;
|
||||
const measure = () => {
|
||||
const margin = Math.max(
|
||||
0,
|
||||
Math.round(
|
||||
virtualRowsWrapper.getBoundingClientRect().top -
|
||||
scrollElement.getBoundingClientRect().top +
|
||||
scrollElement.scrollTop,
|
||||
),
|
||||
);
|
||||
setScrollMargin((current) => (current === margin ? current : margin));
|
||||
};
|
||||
measure();
|
||||
const observer = new ResizeObserver(measure);
|
||||
observer.observe(virtualRowsWrapper.parentElement ?? scrollElement);
|
||||
return () => observer.disconnect();
|
||||
}, [virtualRowsWrapper, scrollElement]);
|
||||
const rowHeightPx = compact
|
||||
? RESULT_SPLIT_ROW_HEIGHT_PX
|
||||
: RESULT_GRID_ROW_HEIGHT_PX;
|
||||
const cellHeightPx = compact ? RESULT_SPLIT_HEIGHT_PX : RESULT_GRID_HEIGHT_PX;
|
||||
const renderInventoryRow = (item: InventoryItem) => (
|
||||
<InventoryRow
|
||||
row={item.row}
|
||||
selected={selectedId === item.row.id}
|
||||
activeCheckpoint={activeCheckpoint}
|
||||
activeGgufVariant={activeGgufVariant}
|
||||
isDataset={isDataset}
|
||||
dimmed={!inventoryRowMatches(item.row, inventoryTokens)}
|
||||
deviceType={deviceType}
|
||||
compact={compact}
|
||||
onSelect={onSelect}
|
||||
onChange={onInventoryChange}
|
||||
/>
|
||||
);
|
||||
|
||||
if (!downloadedReady && !hasInventoryRows) {
|
||||
return (
|
||||
|
|
@ -325,9 +405,29 @@ export function DownloadedList({
|
|||
}
|
||||
|
||||
if (cachedRows.length === 0 && localRows.length === 0) {
|
||||
if (!query.trim() && typeFilterActive) {
|
||||
return (
|
||||
<EmptyState
|
||||
icon={Search01Icon}
|
||||
title="No matching models on device"
|
||||
body="No downloaded or local model matches the selected type filter."
|
||||
action={
|
||||
onClearFilters && (
|
||||
<button
|
||||
type="button"
|
||||
onClick={onClearFilters}
|
||||
className="inline-flex h-8 items-center gap-1.5 rounded-full bg-transparent px-3 text-[12px] font-medium text-foreground transition-colors hover:bg-foreground/[0.04] dark:hover:bg-white/[0.05]"
|
||||
>
|
||||
Show all types
|
||||
</button>
|
||||
)
|
||||
}
|
||||
/>
|
||||
);
|
||||
}
|
||||
return (
|
||||
<EmptyState
|
||||
icon={query.trim() ? FolderSearchIcon : DownloadCircle02Icon}
|
||||
icon={query.trim() ? Search01Icon : DownloadCircle02Icon}
|
||||
title={query.trim() ? "No matches on device" : "Nothing on device yet"}
|
||||
body={
|
||||
query.trim()
|
||||
|
|
@ -341,27 +441,57 @@ export function DownloadedList({
|
|||
}
|
||||
|
||||
return (
|
||||
<VirtualRows
|
||||
items={inventoryItems}
|
||||
scrollElement={scrollElement}
|
||||
columns={columns}
|
||||
rowHeight={compact ? RESULT_SPLIT_ROW_HEIGHT_PX : RESULT_GRID_ROW_HEIGHT_PX}
|
||||
cellHeight={compact ? RESULT_SPLIT_HEIGHT_PX : RESULT_GRID_HEIGHT_PX}
|
||||
getKey={(item) => `${item.variant}-${item.row.id}`}
|
||||
renderRow={(item) => (
|
||||
<InventoryRow
|
||||
row={item.row}
|
||||
selected={selectedId === item.row.id}
|
||||
activeCheckpoint={activeCheckpoint}
|
||||
activeGgufVariant={activeGgufVariant}
|
||||
isDataset={isDataset}
|
||||
dimmed={!inventoryRowMatches(item.row, inventoryTokens)}
|
||||
deviceType={deviceType}
|
||||
compact={compact}
|
||||
onSelect={onSelect}
|
||||
onChange={onInventoryChange}
|
||||
/>
|
||||
<>
|
||||
{pinnedItems.length > 0 && (
|
||||
<>
|
||||
<div className="flex items-center gap-1.5 px-1 pb-2 pt-3 text-[11px] font-semibold uppercase tracking-wider text-muted-foreground">
|
||||
<HugeiconsIcon
|
||||
icon={PinIcon}
|
||||
strokeWidth={1.75}
|
||||
className="size-3.5"
|
||||
/>
|
||||
Pinned
|
||||
</div>
|
||||
{/* Pinned rows are few, so render them as a plain grid matching the
|
||||
virtualized list's lane count and row spacing. */}
|
||||
<div
|
||||
style={{
|
||||
display: "grid",
|
||||
gridTemplateColumns: `repeat(${Math.max(1, columns)}, minmax(0, 1fr))`,
|
||||
columnGap: 12,
|
||||
rowGap: rowHeightPx - cellHeightPx,
|
||||
paddingBottom: rowHeightPx - cellHeightPx,
|
||||
}}
|
||||
>
|
||||
{pinnedItems.map((item) => (
|
||||
<div
|
||||
key={`${item.variant}-${item.row.id}`}
|
||||
className="min-w-0"
|
||||
style={{ height: cellHeightPx }}
|
||||
>
|
||||
{renderInventoryRow(item)}
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
{unpinnedItems.length > 0 && (
|
||||
<div className="px-1 pb-2 pt-2 text-[11px] font-semibold uppercase tracking-wider text-muted-foreground">
|
||||
All {isDataset ? "datasets" : "models"}
|
||||
</div>
|
||||
)}
|
||||
</>
|
||||
)}
|
||||
/>
|
||||
<div ref={setVirtualRowsWrapper}>
|
||||
<VirtualRows
|
||||
items={unpinnedItems}
|
||||
scrollElement={scrollElement}
|
||||
scrollMargin={scrollMargin}
|
||||
columns={columns}
|
||||
rowHeight={rowHeightPx}
|
||||
cellHeight={cellHeightPx}
|
||||
getKey={(item) => `${item.variant}-${item.row.id}`}
|
||||
renderRow={renderInventoryRow}
|
||||
/>
|
||||
</div>
|
||||
</>
|
||||
);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,7 +1,6 @@
|
|||
// SPDX-License-Identifier: AGPL-3.0-only
|
||||
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
import { ModelDeleteAction } from "@/components/assistant-ui/model-selector/model-delete-action";
|
||||
import {
|
||||
Tooltip,
|
||||
TooltipContent,
|
||||
|
|
@ -9,18 +8,26 @@ import {
|
|||
} from "@/components/ui/tooltip";
|
||||
import {
|
||||
type GgufVariantDetail,
|
||||
deleteCachedModel,
|
||||
deleteCachedDataset,
|
||||
deleteCachedModel,
|
||||
formatLocalUpdated,
|
||||
listGgufVariants,
|
||||
useGgufVariantsCacheVersion,
|
||||
} from "@/features/hub/inventory";
|
||||
import { classifyUnslothSupport } from "@/features/hub/hooks/use-hub-model-search";
|
||||
import { formatBytes, formatRelativeShort } from "@/features/hub/lib/format";
|
||||
import { ggufVariantDisplayLabel } from "@/features/hub/lib/gguf-variant-sort";
|
||||
import { modelIdsMatch } from "@/features/hub/lib/model-identity";
|
||||
} from "../inventory";
|
||||
import {
|
||||
classifyUnslothSupport,
|
||||
formatBytes,
|
||||
formatRelativeShort,
|
||||
ggufVariantDisplayLabel,
|
||||
useHfTokenStore,
|
||||
} from "@/features/hub";
|
||||
import { modelIdsMatch } from "../lib/model-identity";
|
||||
import {
|
||||
ModelRowMenu,
|
||||
pinKey,
|
||||
usePinnedModelsStore,
|
||||
} from "@/features/model-picker";
|
||||
import { cn, formatCompact } from "@/lib/utils";
|
||||
import { useHfTokenStore } from "@/features/hub/stores/hf-token-store";
|
||||
import {
|
||||
Download01Icon,
|
||||
FavouriteIcon,
|
||||
|
|
@ -39,6 +46,7 @@ import {
|
|||
useRef,
|
||||
useState,
|
||||
} from "react";
|
||||
import { paramLabelFromId } from "../lib/view-models";
|
||||
import type {
|
||||
CachedInventoryRow,
|
||||
DiscoverRow,
|
||||
|
|
@ -46,7 +54,6 @@ import type {
|
|||
} from "../types";
|
||||
import { OwnerAvatar } from "./owner-avatar";
|
||||
import { AccessGlyphs } from "./shared";
|
||||
import { paramLabelFromId } from "../lib/view-models";
|
||||
|
||||
const COARSE_POINTER =
|
||||
typeof window !== "undefined" &&
|
||||
|
|
@ -142,15 +149,15 @@ function CachedSizeChipLive({
|
|||
);
|
||||
|
||||
const rows: Array<{ label: string; size_bytes: number }> | null =
|
||||
!needsVariantFetch
|
||||
? [{ label: repoId, size_bytes: totalBytes }]
|
||||
: currentVariantState.status === "loaded" &&
|
||||
currentVariantState.variants.length > 0
|
||||
needsVariantFetch
|
||||
? currentVariantState.status === "loaded" &&
|
||||
currentVariantState.variants.length > 0
|
||||
? currentVariantState.variants.map((variant) => ({
|
||||
label: ggufVariantDisplayLabel(variant),
|
||||
size_bytes: variant.size_bytes,
|
||||
}))
|
||||
: null;
|
||||
: null
|
||||
: [{ label: repoId, size_bytes: totalBytes }];
|
||||
const variantMessage =
|
||||
currentVariantState.status === "loading"
|
||||
? "Loading downloaded variants..."
|
||||
|
|
@ -275,7 +282,9 @@ function CatalogRow({
|
|||
)}
|
||||
/>
|
||||
<CatalogRowInteractiveContext.Provider value={interactive}>
|
||||
<div className={cn("pointer-events-none relative", card && "z-[1] w-full")}>
|
||||
<div
|
||||
className={cn("pointer-events-none relative", card && "z-[1] w-full")}
|
||||
>
|
||||
{children}
|
||||
</div>
|
||||
</CatalogRowInteractiveContext.Provider>
|
||||
|
|
@ -653,7 +662,9 @@ export const InventoryRow = memo(function InventoryRow({
|
|||
<div className="hidden shrink-0 items-center gap-1.5 sm:flex">
|
||||
{/* Format already shows as the status dot, so the pill stays neutral. */}
|
||||
{formatLabel && <span className="hub-chip">{formatLabel}</span>}
|
||||
{paramLabel && <span className="hub-chip tabular-nums">{paramLabel}</span>}
|
||||
{paramLabel && (
|
||||
<span className="hub-chip tabular-nums">{paramLabel}</span>
|
||||
)}
|
||||
{quantLabel && (
|
||||
<span className="hub-chip font-mono text-[10.5px] uppercase">
|
||||
{quantLabel}
|
||||
|
|
@ -697,9 +708,7 @@ export const InventoryRow = memo(function InventoryRow({
|
|||
const compactMarkers =
|
||||
partialRepoId || unsupported ? (
|
||||
<span className="flex shrink-0 items-center gap-1">
|
||||
{partialRepoId && (
|
||||
<StatusDot tone="warning" label="Partial download" />
|
||||
)}
|
||||
{partialRepoId && <StatusDot tone="warning" label="Partial download" />}
|
||||
{unsupported && (
|
||||
<StatusDot tone="danger" label="May not be supported yet" />
|
||||
)}
|
||||
|
|
@ -718,37 +727,72 @@ export const InventoryRow = memo(function InventoryRow({
|
|||
</span>
|
||||
);
|
||||
|
||||
const pinnedKeys = usePinnedModelsStore((s) => s.pinned);
|
||||
const togglePinned = usePinnedModelsStore((s) => s.togglePinned);
|
||||
const rowPinned =
|
||||
cacheDeletableRepoId != null &&
|
||||
pinnedKeys.includes(pinKey(cacheDeletableRepoId));
|
||||
const deleteAction =
|
||||
canDelete && cacheDeletableRepoId ? (
|
||||
<ModelDeleteAction
|
||||
ariaLabel={`Delete ${cacheDeletableRepoId}`}
|
||||
title={isDataset ? "Delete cached dataset?" : "Delete cached model?"}
|
||||
description={
|
||||
<>
|
||||
This will remove{" "}
|
||||
<span className="font-medium text-foreground">
|
||||
{cacheDeletableRepoId}
|
||||
</span>{" "}
|
||||
{isDataset
|
||||
? "and its downloaded files"
|
||||
: row.isGguf
|
||||
? "and all of its downloaded quantizations"
|
||||
: "and all of its downloaded files"}
|
||||
{row.kind === "cache" ? ` (${formatBytes(row.bytes)})` : ""} from
|
||||
disk. You can re-download it later.
|
||||
</>
|
||||
}
|
||||
successMessage={`Deleted ${cacheDeletableRepoId}`}
|
||||
<ModelRowMenu
|
||||
ariaLabel={`More options for ${cacheDeletableRepoId}`}
|
||||
buttonClassName="pointer-events-auto hub-modal-pe-guard p-2 opacity-0 transition-opacity group-hover/row:opacity-100 focus-visible:opacity-100 data-[state=open]:opacity-100 [@media(pointer:coarse)]:opacity-100"
|
||||
iconClassName="size-4"
|
||||
onConfirm={async () => {
|
||||
if (isDataset) {
|
||||
await deleteCachedDataset(cacheDeletableRepoId);
|
||||
} else {
|
||||
await deleteCachedModel(cacheDeletableRepoId);
|
||||
}
|
||||
pin={
|
||||
isDataset
|
||||
? undefined
|
||||
: {
|
||||
pinned: rowPinned,
|
||||
pinLabel: "Pin to top",
|
||||
unpinLabel: "Unpin",
|
||||
onToggle: () => togglePinned(cacheDeletableRepoId),
|
||||
}
|
||||
}
|
||||
cachePath={isDataset ? undefined : { repoId: cacheDeletableRepoId }}
|
||||
del={{
|
||||
title: isDataset ? "Delete cached dataset?" : "Delete cached model?",
|
||||
description: (
|
||||
<>
|
||||
This will remove{" "}
|
||||
<span className="font-medium text-foreground">
|
||||
{cacheDeletableRepoId}
|
||||
</span>{" "}
|
||||
{isDataset
|
||||
? "and its downloaded files"
|
||||
: row.isGguf
|
||||
? "and all of its downloaded quantizations"
|
||||
: "and all of its downloaded files"}
|
||||
{row.kind === "cache" ? ` (${formatBytes(row.bytes)})` : ""} from
|
||||
disk. You can re-download it later.
|
||||
</>
|
||||
),
|
||||
successMessage: `Deleted ${cacheDeletableRepoId}`,
|
||||
onConfirm: async () => {
|
||||
if (isDataset) {
|
||||
await deleteCachedDataset(cacheDeletableRepoId);
|
||||
} else {
|
||||
await deleteCachedModel(cacheDeletableRepoId);
|
||||
// Deleted repos can't stay pinned: drop the repo pin and any of
|
||||
// its per-quant pins so stale rows don't linger up top.
|
||||
const { pinned, togglePinned: toggle } =
|
||||
usePinnedModelsStore.getState();
|
||||
for (const key of pinned) {
|
||||
if (
|
||||
key === pinKey(cacheDeletableRepoId) ||
|
||||
key.startsWith(`${cacheDeletableRepoId}::`)
|
||||
) {
|
||||
toggle(
|
||||
cacheDeletableRepoId,
|
||||
key.includes("::")
|
||||
? key.slice(key.indexOf("::") + 2)
|
||||
: undefined,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
},
|
||||
onDeleted: onChange,
|
||||
}}
|
||||
onDeleted={onChange}
|
||||
/>
|
||||
) : null;
|
||||
|
||||
|
|
|
|||
|
|
@ -54,6 +54,7 @@ export interface ModelsCatalogState {
|
|||
hasMore: boolean;
|
||||
manualFetchAvailable: boolean;
|
||||
hasActiveFilters: boolean;
|
||||
typeFilterActive: boolean;
|
||||
}
|
||||
|
||||
export interface ModelsCatalogPagination {
|
||||
|
|
@ -117,6 +118,7 @@ export const ModelsCatalog = memo(function ModelsCatalog({
|
|||
loadingIntentCount,
|
||||
hasMore,
|
||||
hasActiveFilters,
|
||||
typeFilterActive,
|
||||
} = state;
|
||||
const { scrollRef, sentinelRef, isLoadingMore } = pagination;
|
||||
const {
|
||||
|
|
@ -469,6 +471,8 @@ export const ModelsCatalog = memo(function ModelsCatalog({
|
|||
downloadedReady={downloadedReady}
|
||||
inventoryError={inventoryError}
|
||||
query={query}
|
||||
typeFilterActive={typeFilterActive}
|
||||
onClearFilters={onClearFilters}
|
||||
scrollElement={downloadedScrollEl}
|
||||
activeCheckpoint={activeCheckpoint}
|
||||
activeGgufVariant={activeGgufVariant}
|
||||
|
|
|
|||
|
|
@ -17,6 +17,10 @@ import {
|
|||
formatRelativeLong,
|
||||
formatRelativeShort,
|
||||
} from "@/features/hub/lib/format";
|
||||
import {
|
||||
MODEL_TYPE_FILTER_OPTIONS,
|
||||
type ModelTypeFilter,
|
||||
} from "@/features/hub/lib/model-type-filter";
|
||||
import {
|
||||
formatModelParamLabel,
|
||||
formatPipelineTag,
|
||||
|
|
@ -25,6 +29,7 @@ import { copyToClipboard } from "@/lib/copy-to-clipboard";
|
|||
import { cn, formatCompact } from "@/lib/utils";
|
||||
import {
|
||||
ArrowLeft01Icon,
|
||||
ArrowUpDownIcon,
|
||||
Copy01Icon,
|
||||
Download01Icon,
|
||||
FavouriteIcon,
|
||||
|
|
@ -124,6 +129,7 @@ export function InventorySortControl({
|
|||
value: InventorySort;
|
||||
onChange: (value: InventorySort) => void;
|
||||
}) {
|
||||
const selected = INVENTORY_SORTS.find((option) => option.value === value);
|
||||
return (
|
||||
<HubOptionMenu<InventorySort>
|
||||
value={value}
|
||||
|
|
@ -131,7 +137,46 @@ export function InventorySortControl({
|
|||
onValueChange={onChange}
|
||||
ariaLabel="Sort downloads"
|
||||
align="end"
|
||||
className="h-8 text-[11.5px]"
|
||||
title={selected?.label}
|
||||
// Capped and shrinkable so a long label truncates instead of wrapping
|
||||
// the "On device" heading beside these pills in the narrow split pane.
|
||||
className="h-8 min-w-[72px] max-w-[124px] shrink text-[11.5px]"
|
||||
triggerContent={
|
||||
<span className="flex min-w-0 items-center gap-1">
|
||||
<HugeiconsIcon
|
||||
icon={ArrowUpDownIcon}
|
||||
strokeWidth={1.75}
|
||||
className="size-3.5 shrink-0 text-muted-foreground"
|
||||
/>
|
||||
<span className="truncate">{selected?.label ?? value}</span>
|
||||
</span>
|
||||
}
|
||||
/>
|
||||
);
|
||||
}
|
||||
|
||||
// Model-type filter pill (Text / Vision / Embedding / …) beside the sort pill.
|
||||
export function InventoryTypeFilterControl({
|
||||
value,
|
||||
onChange,
|
||||
}: {
|
||||
value: ModelTypeFilter;
|
||||
onChange: (value: ModelTypeFilter) => void;
|
||||
}) {
|
||||
const selected = MODEL_TYPE_FILTER_OPTIONS.find(
|
||||
(option) => option.value === value,
|
||||
);
|
||||
return (
|
||||
<HubOptionMenu<ModelTypeFilter>
|
||||
value={value}
|
||||
options={MODEL_TYPE_FILTER_OPTIONS}
|
||||
onValueChange={onChange}
|
||||
ariaLabel="Filter by model type"
|
||||
align="end"
|
||||
title={selected?.label}
|
||||
// Capped and shrinkable so a long label ("Speech to text") truncates
|
||||
// instead of wrapping the "On device" heading beside these pills.
|
||||
className="h-8 min-w-[72px] max-w-[124px] shrink text-[11.5px]"
|
||||
/>
|
||||
);
|
||||
}
|
||||
|
|
@ -183,7 +228,9 @@ export function HubListHeader({
|
|||
</button>
|
||||
)}
|
||||
<div className="min-w-0 space-y-0.5">
|
||||
<h2 className="text-[18px] font-semibold tracking-[-0.02em] text-foreground">
|
||||
{/* truncate keeps the heading on one line and clips a long search
|
||||
query with an ellipsis instead of overflowing the pills. */}
|
||||
<h2 className="truncate text-[18px] font-semibold tracking-[-0.02em] text-foreground">
|
||||
{title}
|
||||
</h2>
|
||||
{subtitle && (
|
||||
|
|
@ -216,7 +263,9 @@ export function HubListHeader({
|
|||
)}
|
||||
</div>
|
||||
{(actions || onViewChange) && (
|
||||
<div className="flex shrink-0 items-center gap-2">
|
||||
// min-w-0 (not shrink-0) so shrinkable actions (the On-device filter
|
||||
// pills) compress before the title is forced onto two lines.
|
||||
<div className="flex min-w-0 items-center gap-2">
|
||||
{actions}
|
||||
{onViewChange && (
|
||||
<div
|
||||
|
|
@ -524,7 +573,9 @@ function useResultRowModel(
|
|||
[isDataset, row.id, row.result, deviceType],
|
||||
);
|
||||
const sizeLabel = formatModelParamLabel(row.repo, row.result.totalParams);
|
||||
const taskLabel = isDataset ? null : formatPipelineTag(row.result.pipelineTag);
|
||||
const taskLabel = isDataset
|
||||
? null
|
||||
: formatPipelineTag(row.result.pipelineTag);
|
||||
const unsupported = support?.status === "unsupported";
|
||||
return {
|
||||
support,
|
||||
|
|
|
|||
|
|
@ -9,6 +9,7 @@ import {
|
|||
TooltipContent,
|
||||
TooltipTrigger,
|
||||
} from "@/components/ui/tooltip";
|
||||
import type { HfSortKey } from "@/features/hub/hooks/use-hub-model-search";
|
||||
import { cn } from "@/lib/utils";
|
||||
import {
|
||||
AiChipIcon,
|
||||
|
|
@ -19,26 +20,25 @@ import {
|
|||
SlidersHorizontalIcon,
|
||||
} from "@hugeicons/core-free-icons";
|
||||
import { HugeiconsIcon } from "@hugeicons/react";
|
||||
import type { HfSortKey } from "@/features/hub/hooks/use-hub-model-search";
|
||||
import type {
|
||||
CapabilityFilter,
|
||||
ModelFormatFilter,
|
||||
ModelsTab,
|
||||
ResourceTypeFilter,
|
||||
} from "../types";
|
||||
import {
|
||||
CAPABILITY_FILTER_OPTIONS,
|
||||
FORMAT_FILTER_OPTIONS,
|
||||
} from "../lib/view-models";
|
||||
import { HubOptionMenu, type HubOption } from "./hub-option-menu";
|
||||
import { memo, useMemo, useState } from "react";
|
||||
import {
|
||||
clearRecentSearches,
|
||||
recordRecentSearch,
|
||||
removeRecentSearch,
|
||||
useRecentSearches,
|
||||
} from "../lib/recent-searches";
|
||||
import {
|
||||
CAPABILITY_FILTER_OPTIONS,
|
||||
FORMAT_FILTER_OPTIONS,
|
||||
} from "../lib/view-models";
|
||||
import type {
|
||||
CapabilityFilter,
|
||||
ModelFormatFilter,
|
||||
ModelsTab,
|
||||
ResourceTypeFilter,
|
||||
} from "../types";
|
||||
import { type HubOption, HubOptionMenu } from "./hub-option-menu";
|
||||
import { RecentSearches } from "./recent-searches";
|
||||
import { memo, useMemo, useState } from "react";
|
||||
|
||||
// Widened so the format dropdown can carry the "Fine-tune ready" pseudo-option,
|
||||
// which opens the curated channel instead of becoming the active format filter.
|
||||
|
|
@ -175,7 +175,7 @@ export const ModelsToolbar = memo(function ModelsToolbar({
|
|||
<div className="flex min-w-0 flex-col gap-2 lg:flex-row lg:flex-nowrap lg:items-center">
|
||||
<div
|
||||
className={cn(
|
||||
"hub-menu-trigger hub-tab-toggle relative inline-flex h-9 w-full shrink-0 items-center rounded-full lg:w-[240px]",
|
||||
"hub-menu-trigger hub-tab-toggle relative inline-flex h-9 w-full shrink-0 items-center rounded-full lg:w-[280px]",
|
||||
)}
|
||||
role="radiogroup"
|
||||
aria-label="View"
|
||||
|
|
@ -219,88 +219,88 @@ export const ModelsToolbar = memo(function ModelsToolbar({
|
|||
</div>
|
||||
|
||||
<div className="relative min-w-0 flex-1 lg:flex-[1_1_360px]">
|
||||
<HugeiconsIcon
|
||||
icon={Search01Icon}
|
||||
strokeWidth={1.8}
|
||||
className="pointer-events-none absolute left-3.5 top-1/2 size-4 -translate-y-1/2 text-muted-foreground"
|
||||
/>
|
||||
<Input
|
||||
// `type="search"` plus these flags stop password managers and noisy
|
||||
// text assistance from acting on this field.
|
||||
type="search"
|
||||
name="hub-search"
|
||||
autoComplete="off"
|
||||
autoCorrect="off"
|
||||
autoCapitalize="off"
|
||||
spellCheck={false}
|
||||
enterKeyHint="search"
|
||||
data-1p-ignore={true}
|
||||
data-lpignore={true}
|
||||
data-form-type="other"
|
||||
value={query}
|
||||
onChange={(event) => onQueryChange(event.target.value)}
|
||||
onFocus={() => setSearchFocused(true)}
|
||||
onBlur={() => {
|
||||
setSearchFocused(false);
|
||||
if (isDiscover) {
|
||||
recordRecentSearch(query);
|
||||
}
|
||||
}}
|
||||
onKeyDown={(event) => {
|
||||
if (event.key === "Enter" && isDiscover) {
|
||||
recordRecentSearch(query);
|
||||
} else if (event.key === "Escape" && showRecentSearches) {
|
||||
event.currentTarget.blur();
|
||||
}
|
||||
}}
|
||||
placeholder={
|
||||
tab === "downloaded"
|
||||
? `Search on-device ${isDataset ? "datasets" : "models"}`
|
||||
: isDataset
|
||||
? "Search datasets"
|
||||
: "Search all models"
|
||||
<HugeiconsIcon
|
||||
icon={Search01Icon}
|
||||
strokeWidth={1.8}
|
||||
className="pointer-events-none absolute left-3.5 top-1/2 size-4 -translate-y-1/2 text-muted-foreground"
|
||||
/>
|
||||
<Input
|
||||
// `type="search"` plus these flags stop password managers and noisy
|
||||
// text assistance from acting on this field.
|
||||
type="search"
|
||||
name="hub-search"
|
||||
autoComplete="off"
|
||||
autoCorrect="off"
|
||||
autoCapitalize="off"
|
||||
spellCheck={false}
|
||||
enterKeyHint="search"
|
||||
data-1p-ignore={true}
|
||||
data-lpignore={true}
|
||||
data-form-type="other"
|
||||
value={query}
|
||||
onChange={(event) => onQueryChange(event.target.value)}
|
||||
onFocus={() => setSearchFocused(true)}
|
||||
onBlur={() => {
|
||||
setSearchFocused(false);
|
||||
if (isDiscover) {
|
||||
recordRecentSearch(query);
|
||||
}
|
||||
className={cn(
|
||||
"field-soft h-9 rounded-full !border-0 pl-10 text-[13px] placeholder:text-muted-foreground/80 focus-visible:!ring-0",
|
||||
hasTrailing ? "pr-10" : "pr-4",
|
||||
)}
|
||||
/>
|
||||
{query ? (
|
||||
<button
|
||||
type="button"
|
||||
aria-label="Clear search"
|
||||
// Keep focus on the input so clearing reveals recent searches
|
||||
// rather than dismissing the field.
|
||||
onMouseDown={(event) => event.preventDefault()}
|
||||
onClick={() => onQueryChange("")}
|
||||
className="absolute right-2.5 top-1/2 inline-flex size-6 -translate-y-1/2 items-center justify-center rounded-full text-muted-foreground/70 transition-colors hover:text-foreground"
|
||||
>
|
||||
<HugeiconsIcon
|
||||
icon={CancelCircleIcon}
|
||||
strokeWidth={1.75}
|
||||
className="size-[18px]"
|
||||
/>
|
||||
</button>
|
||||
) : isDiscover && isLoading ? (
|
||||
<Spinner className="pointer-events-none absolute right-3.5 top-1/2 size-4 -translate-y-1/2 text-muted-foreground" />
|
||||
) : null}
|
||||
{showRecentSearches && (
|
||||
<RecentSearches
|
||||
searches={recentSearches}
|
||||
onSelect={(value) => {
|
||||
recordRecentSearch(value);
|
||||
onQueryChange(value);
|
||||
}}
|
||||
onRemove={removeRecentSearch}
|
||||
onClear={clearRecentSearches}
|
||||
/>
|
||||
}}
|
||||
onKeyDown={(event) => {
|
||||
if (event.key === "Enter" && isDiscover) {
|
||||
recordRecentSearch(query);
|
||||
} else if (event.key === "Escape" && showRecentSearches) {
|
||||
event.currentTarget.blur();
|
||||
}
|
||||
}}
|
||||
placeholder={
|
||||
tab === "downloaded"
|
||||
? `Search on-device ${isDataset ? "datasets" : "models"}`
|
||||
: isDataset
|
||||
? "Search datasets"
|
||||
: "Search all models"
|
||||
}
|
||||
className={cn(
|
||||
"field-soft h-9 rounded-full !border-0 pl-10 text-[13px] placeholder:text-muted-foreground/80 focus-visible:!ring-0",
|
||||
hasTrailing ? "pr-10" : "pr-4",
|
||||
)}
|
||||
</div>
|
||||
/>
|
||||
{query ? (
|
||||
<button
|
||||
type="button"
|
||||
aria-label="Clear search"
|
||||
// Keep focus on the input so clearing reveals recent searches
|
||||
// rather than dismissing the field.
|
||||
onMouseDown={(event) => event.preventDefault()}
|
||||
onClick={() => onQueryChange("")}
|
||||
className="absolute right-2.5 top-1/2 inline-flex size-6 -translate-y-1/2 items-center justify-center rounded-full text-muted-foreground/70 transition-colors hover:text-foreground"
|
||||
>
|
||||
<HugeiconsIcon
|
||||
icon={CancelCircleIcon}
|
||||
strokeWidth={1.75}
|
||||
className="size-[18px]"
|
||||
/>
|
||||
</button>
|
||||
) : isDiscover && isLoading ? (
|
||||
<Spinner className="pointer-events-none absolute right-3.5 top-1/2 size-4 -translate-y-1/2 text-muted-foreground" />
|
||||
) : null}
|
||||
{showRecentSearches && (
|
||||
<RecentSearches
|
||||
searches={recentSearches}
|
||||
onSelect={(value) => {
|
||||
recordRecentSearch(value);
|
||||
onQueryChange(value);
|
||||
}}
|
||||
onRemove={removeRecentSearch}
|
||||
onClear={clearRecentSearches}
|
||||
/>
|
||||
)}
|
||||
</div>
|
||||
|
||||
<div className="flex min-w-0 flex-wrap items-center gap-2 lg:flex-[0_0_auto] lg:flex-nowrap lg:justify-end">
|
||||
{tab === "downloaded" && !isDataset && (
|
||||
<Tooltip>
|
||||
<TooltipTrigger asChild>
|
||||
<TooltipTrigger asChild={true}>
|
||||
<button
|
||||
type="button"
|
||||
onClick={onManageLocalFolders}
|
||||
|
|
@ -359,7 +359,7 @@ export const ModelsToolbar = memo(function ModelsToolbar({
|
|||
footer={
|
||||
isDataset ? undefined : (
|
||||
<Tooltip>
|
||||
<TooltipTrigger asChild>
|
||||
<TooltipTrigger asChild={true}>
|
||||
<button
|
||||
type="button"
|
||||
role="checkbox"
|
||||
|
|
@ -370,7 +370,7 @@ export const ModelsToolbar = memo(function ModelsToolbar({
|
|||
<Checkbox
|
||||
checked={fitOnDeviceOnly}
|
||||
tabIndex={-1}
|
||||
aria-hidden
|
||||
aria-hidden={true}
|
||||
className="pointer-events-none size-3.5 rounded-full [&_svg]:!size-2.5"
|
||||
/>
|
||||
Only show models that fit
|
||||
|
|
@ -402,7 +402,7 @@ export const ModelsToolbar = memo(function ModelsToolbar({
|
|||
)}
|
||||
/>
|
||||
<Tooltip>
|
||||
<TooltipTrigger asChild>
|
||||
<TooltipTrigger asChild={true}>
|
||||
<button
|
||||
type="button"
|
||||
role="radio"
|
||||
|
|
@ -428,7 +428,7 @@ export const ModelsToolbar = memo(function ModelsToolbar({
|
|||
</TooltipContent>
|
||||
</Tooltip>
|
||||
<Tooltip>
|
||||
<TooltipTrigger asChild>
|
||||
<TooltipTrigger asChild={true}>
|
||||
<button
|
||||
type="button"
|
||||
role="radio"
|
||||
|
|
|
|||
|
|
@ -1,7 +1,6 @@
|
|||
// SPDX-License-Identifier: AGPL-3.0-only
|
||||
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
import { FolderBrowser } from "@/components/assistant-ui/model-selector/folder-browser";
|
||||
import { Button } from "@/components/ui/button";
|
||||
import {
|
||||
Dialog,
|
||||
|
|
@ -22,9 +21,11 @@ import {
|
|||
addScanFolder,
|
||||
listScanFolders,
|
||||
removeScanFolder,
|
||||
} from "@/features/hub/inventory";
|
||||
import { openModelsDir } from "@/features/native-intents/api";
|
||||
} from "@/features/hub";
|
||||
import { FolderBrowser } from "@/features/model-picker";
|
||||
import { openModelsDir } from "@/features/native-intents";
|
||||
import { isTauri } from "@/lib/api-base";
|
||||
import { toast } from "@/lib/toast";
|
||||
import { cn } from "@/lib/utils";
|
||||
import {
|
||||
Delete02Icon,
|
||||
|
|
@ -38,7 +39,6 @@ import {
|
|||
} from "@hugeicons/core-free-icons";
|
||||
import { HugeiconsIcon } from "@hugeicons/react";
|
||||
import { useCallback, useEffect, useMemo, useRef, useState } from "react";
|
||||
import { toast } from "@/lib/toast";
|
||||
|
||||
function pathTail(path: string): string {
|
||||
const parts = path.split(/[\\/]/).filter(Boolean);
|
||||
|
|
@ -123,7 +123,9 @@ export function OnDeviceFoldersDialog({
|
|||
setPath("");
|
||||
mutationVersionRef.current += 1;
|
||||
setFolders((current) => {
|
||||
const withoutDuplicate = current.filter((row) => row.id !== folder.id);
|
||||
const withoutDuplicate = current.filter(
|
||||
(row) => row.id !== folder.id,
|
||||
);
|
||||
return [...withoutDuplicate, folder];
|
||||
});
|
||||
toast.success("Location added", {
|
||||
|
|
@ -184,9 +186,12 @@ export function OnDeviceFoldersDialog({
|
|||
overlayClassName="bg-black/20 backdrop-blur-none"
|
||||
>
|
||||
<DialogHeader className="border-b border-border/60 px-5 py-4">
|
||||
<DialogTitle className="text-[15px]">On-device locations</DialogTitle>
|
||||
<DialogTitle className="text-[15px]">
|
||||
On-device locations
|
||||
</DialogTitle>
|
||||
<DialogDescription className="sr-only">
|
||||
Hugging Face model folders, GGUF files, and adapters are indexed here.
|
||||
Hugging Face model folders, GGUF files, and adapters are indexed
|
||||
here.
|
||||
</DialogDescription>
|
||||
</DialogHeader>
|
||||
|
||||
|
|
@ -342,9 +347,7 @@ export function OnDeviceFoldersDialog({
|
|||
</p>
|
||||
<Tooltip>
|
||||
<TooltipTrigger asChild={true}>
|
||||
<p
|
||||
className="block w-full truncate font-mono text-[10.5px] text-muted-foreground"
|
||||
>
|
||||
<p className="block w-full truncate font-mono text-[10.5px] text-muted-foreground">
|
||||
{folder.path}
|
||||
</p>
|
||||
</TooltipTrigger>
|
||||
|
|
@ -372,7 +375,10 @@ export function OnDeviceFoldersDialog({
|
|||
/>
|
||||
</button>
|
||||
</TooltipTrigger>
|
||||
<TooltipContent side="left" className="tooltip-compact">
|
||||
<TooltipContent
|
||||
side="left"
|
||||
className="tooltip-compact"
|
||||
>
|
||||
Open in file manager
|
||||
</TooltipContent>
|
||||
</Tooltip>
|
||||
|
|
@ -397,7 +403,10 @@ export function OnDeviceFoldersDialog({
|
|||
)}
|
||||
</button>
|
||||
</TooltipTrigger>
|
||||
<TooltipContent side="left" className="tooltip-compact">
|
||||
<TooltipContent
|
||||
side="left"
|
||||
className="tooltip-compact"
|
||||
>
|
||||
Remove from list
|
||||
</TooltipContent>
|
||||
</Tooltip>
|
||||
|
|
|
|||
|
|
@ -1,38 +1,83 @@
|
|||
// SPDX-License-Identifier: AGPL-3.0-only
|
||||
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
import {
|
||||
Dialog,
|
||||
DialogContent,
|
||||
DialogDescription,
|
||||
DialogHeader,
|
||||
DialogTitle,
|
||||
} from "@/components/ui/dialog";
|
||||
import {
|
||||
Tooltip,
|
||||
TooltipContent,
|
||||
TooltipTrigger,
|
||||
} from "@/components/ui/tooltip";
|
||||
import { usePlatformStore } from "@/config/env";
|
||||
import { revealCachedModel } from "@/features/chat";
|
||||
import { toast } from "@/lib/toast";
|
||||
import { cn } from "@/lib/utils";
|
||||
import { Copy01Icon, FolderSearchIcon } from "@hugeicons/core-free-icons";
|
||||
import { Copy01Icon, Folder01Icon } from "@hugeicons/core-free-icons";
|
||||
import { Tick02Icon } from "@/lib/tick-icon";
|
||||
import { HugeiconsIcon } from "@hugeicons/react";
|
||||
import type { MouseEvent } from "react";
|
||||
import { useState } from "react";
|
||||
import { useCopyFeedback } from "../hooks/use-copy-feedback";
|
||||
|
||||
/** Reveal a cached repo (or one GGUF variant's file) in the OS file manager.
|
||||
* Resolved server-side from the HF cache, so only managed repos qualify. */
|
||||
export function RevealPathButton({
|
||||
repoId,
|
||||
variant,
|
||||
className,
|
||||
}: {
|
||||
repoId: string;
|
||||
variant?: string | null;
|
||||
className?: string;
|
||||
}) {
|
||||
const deviceType = usePlatformStore((s) => s.deviceType);
|
||||
const revealLabel =
|
||||
deviceType === "mac"
|
||||
? "Reveal in Finder"
|
||||
: deviceType === "windows"
|
||||
? "Reveal in File Explorer"
|
||||
: "Reveal in File Manager";
|
||||
|
||||
return (
|
||||
<Tooltip>
|
||||
<TooltipTrigger asChild={true}>
|
||||
<button
|
||||
type="button"
|
||||
aria-label={revealLabel}
|
||||
onClick={(e) => {
|
||||
e.stopPropagation();
|
||||
revealCachedModel(repoId, variant ?? undefined).catch((err) => {
|
||||
toast.error(
|
||||
err instanceof Error
|
||||
? err.message
|
||||
: "Failed to open file manager",
|
||||
);
|
||||
});
|
||||
}}
|
||||
className={cn(
|
||||
"inline-flex size-7 shrink-0 cursor-pointer items-center justify-center rounded-full text-muted-foreground opacity-0 transition-[opacity,background-color,color] duration-150 hover:bg-muted hover:text-foreground focus-visible:opacity-100 group-hover/dl:opacity-100",
|
||||
className,
|
||||
)}
|
||||
>
|
||||
<HugeiconsIcon
|
||||
icon={Folder01Icon}
|
||||
strokeWidth={1.75}
|
||||
className="size-4"
|
||||
/>
|
||||
</button>
|
||||
</TooltipTrigger>
|
||||
<TooltipContent side="bottom" className="tooltip-compact">
|
||||
{revealLabel}
|
||||
</TooltipContent>
|
||||
</Tooltip>
|
||||
);
|
||||
}
|
||||
|
||||
/** Copies the on-disk path straight to the clipboard, no dialog. */
|
||||
export function PathInfoButton({
|
||||
path,
|
||||
title = "On-device location",
|
||||
description = "Where this model lives on disk.",
|
||||
className,
|
||||
}: {
|
||||
path: string;
|
||||
title?: string;
|
||||
description?: string;
|
||||
className?: string;
|
||||
}) {
|
||||
const [open, setOpen] = useState(false);
|
||||
const { copied, copy } = useCopyFeedback();
|
||||
|
||||
const handleCopy = async (event: MouseEvent<HTMLButtonElement>) => {
|
||||
|
|
@ -42,67 +87,27 @@ export function PathInfoButton({
|
|||
};
|
||||
|
||||
return (
|
||||
<>
|
||||
<Tooltip>
|
||||
<TooltipTrigger asChild={true}>
|
||||
<button
|
||||
type="button"
|
||||
aria-label="Show on-device path"
|
||||
onClick={(e) => {
|
||||
e.stopPropagation();
|
||||
setOpen(true);
|
||||
}}
|
||||
className={cn(
|
||||
"inline-flex size-7 shrink-0 cursor-pointer items-center justify-center rounded-full text-muted-foreground opacity-0 transition-[opacity,background-color,color] duration-150 hover:bg-muted hover:text-foreground focus-visible:opacity-100 group-hover/dl:opacity-100",
|
||||
className,
|
||||
)}
|
||||
>
|
||||
<HugeiconsIcon
|
||||
icon={FolderSearchIcon}
|
||||
strokeWidth={1.75}
|
||||
className="size-4"
|
||||
/>
|
||||
</button>
|
||||
</TooltipTrigger>
|
||||
<TooltipContent side="bottom" className="tooltip-compact">
|
||||
Show path
|
||||
</TooltipContent>
|
||||
</Tooltip>
|
||||
<Dialog open={open} onOpenChange={setOpen}>
|
||||
<DialogContent
|
||||
className="sm:max-w-[520px]"
|
||||
onClick={(e) => e.stopPropagation()}
|
||||
<Tooltip>
|
||||
<TooltipTrigger asChild={true}>
|
||||
<button
|
||||
type="button"
|
||||
aria-label="Copy on-device path"
|
||||
onClick={handleCopy}
|
||||
className={cn(
|
||||
"inline-flex size-7 shrink-0 cursor-pointer items-center justify-center rounded-full text-muted-foreground opacity-0 transition-[opacity,background-color,color] duration-150 hover:bg-muted hover:text-foreground focus-visible:opacity-100 group-hover/dl:opacity-100",
|
||||
className,
|
||||
)}
|
||||
>
|
||||
<DialogHeader>
|
||||
<DialogTitle>{title}</DialogTitle>
|
||||
<DialogDescription>{description}</DialogDescription>
|
||||
</DialogHeader>
|
||||
<div className="flex items-stretch gap-2">
|
||||
<div className="flex-1 select-text rounded-[10px] border border-border bg-muted/30 px-3 py-2.5 text-[12px] leading-5 text-foreground/85 break-all">
|
||||
{path}
|
||||
</div>
|
||||
<Tooltip>
|
||||
<TooltipTrigger asChild={true}>
|
||||
<button
|
||||
type="button"
|
||||
aria-label="Copy path"
|
||||
onClick={handleCopy}
|
||||
className="inline-flex shrink-0 cursor-pointer items-center justify-center rounded-[10px] border border-border bg-muted/30 px-3 text-muted-foreground transition-colors hover:bg-muted hover:text-foreground"
|
||||
>
|
||||
<HugeiconsIcon
|
||||
icon={copied ? Tick02Icon : Copy01Icon}
|
||||
strokeWidth={1.75}
|
||||
className="size-4"
|
||||
/>
|
||||
</button>
|
||||
</TooltipTrigger>
|
||||
<TooltipContent side="bottom" className="tooltip-compact">
|
||||
{copied ? "Copied" : "Copy path"}
|
||||
</TooltipContent>
|
||||
</Tooltip>
|
||||
</div>
|
||||
</DialogContent>
|
||||
</Dialog>
|
||||
</>
|
||||
<HugeiconsIcon
|
||||
icon={copied ? Tick02Icon : Copy01Icon}
|
||||
strokeWidth={1.75}
|
||||
className="size-4"
|
||||
/>
|
||||
</button>
|
||||
</TooltipTrigger>
|
||||
<TooltipContent side="bottom" className="tooltip-compact">
|
||||
{copied ? "Copied" : "Copy path"}
|
||||
</TooltipContent>
|
||||
</Tooltip>
|
||||
);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -7,37 +7,36 @@ import {
|
|||
TooltipContent,
|
||||
TooltipTrigger,
|
||||
} from "@/components/ui/tooltip";
|
||||
import { useRepoDownload } from "../download-manager";
|
||||
import { deleteCachedModel } from "../inventory";
|
||||
import { cn } from "@/lib/utils";
|
||||
import {
|
||||
Alert02Icon,
|
||||
PencilEdit02Icon,
|
||||
PlayIcon,
|
||||
RemoveCircleIcon,
|
||||
} from "@hugeicons/core-free-icons";
|
||||
import { HugeiconsIcon } from "@hugeicons/react";
|
||||
import { useEffect, useState } from "react";
|
||||
import { TrainIcon } from "../components/train-icon";
|
||||
import { useRepoDownload } from "../download-manager";
|
||||
import { useOnlineStatus } from "../hooks/use-online-status";
|
||||
import { deleteCachedModel } from "../inventory";
|
||||
import type { ModelInventoryFormat } from "../inventory";
|
||||
import { fetchModelSize } from "../lib/dataset-size";
|
||||
import { formatBytes } from "../lib/format";
|
||||
import {
|
||||
HUB_NON_GGUF_RUN_ACTIONS_VISIBLE,
|
||||
HUB_POST_DOWNLOAD_ACTIONS_VISIBLE,
|
||||
} from "../lib/hub-feature-flags";
|
||||
import { HugeiconsIcon } from "@hugeicons/react";
|
||||
import { useEffect, useState } from "react";
|
||||
import { useHfTokenStore } from "../stores/hf-token-store";
|
||||
import { fetchModelSize } from "../lib/dataset-size";
|
||||
import { formatBytes } from "../lib/format";
|
||||
import { fingerprintToken } from "../lib/token-fingerprint";
|
||||
import { useOnlineStatus } from "../hooks/use-online-status";
|
||||
import { useHfTokenStore } from "../stores/hf-token-store";
|
||||
import { DotTag } from "./dot-tag";
|
||||
import {
|
||||
CardDivider,
|
||||
CardDeleteButton,
|
||||
DeleteConfirmDialog,
|
||||
DownloadActionButton,
|
||||
DownloadCard,
|
||||
} from "./download-card";
|
||||
import { DotTag } from "./dot-tag";
|
||||
import { PathInfoButton } from "./path-info-button";
|
||||
import { QuantOptionsMenu } from "./gguf-download-card";
|
||||
import { useCardDelete } from "./use-card-delete";
|
||||
import type { ModelInventoryFormat } from "../inventory";
|
||||
import { useDownloadCardState } from "./use-download-card-state";
|
||||
|
||||
function formatModelLabel(modelFormat?: ModelInventoryFormat | null): string {
|
||||
|
|
@ -62,10 +61,9 @@ export function SafetensorsDownloadCard({
|
|||
canRun = true,
|
||||
isActive,
|
||||
isLoadingThisModel,
|
||||
cachePath,
|
||||
knownBytes,
|
||||
onLoad,
|
||||
onUseInChat,
|
||||
onEject,
|
||||
onTrain,
|
||||
onChange,
|
||||
}: {
|
||||
|
|
@ -77,10 +75,13 @@ export function SafetensorsDownloadCard({
|
|||
canRun?: boolean;
|
||||
isActive: boolean;
|
||||
isLoadingThisModel: boolean;
|
||||
/** Accepted for API parity; the options menu resolves the path itself. */
|
||||
cachePath?: string | null;
|
||||
knownBytes?: number | null;
|
||||
onLoad: (opts: { ggufVariant?: string; expectedBytes?: number }) => void;
|
||||
/** Accepted for API parity; the run bar ejects instead of opening chat. */
|
||||
onUseInChat?: () => void;
|
||||
onEject?: () => void;
|
||||
onTrain?: () => void;
|
||||
onChange?: () => void;
|
||||
}) {
|
||||
|
|
@ -95,8 +96,8 @@ export function SafetensorsDownloadCard({
|
|||
knownBytes && knownBytes > 0
|
||||
? knownBytes
|
||||
: modelSize.key === sizeKey
|
||||
? modelSize.bytes
|
||||
: null;
|
||||
? modelSize.bytes
|
||||
: null;
|
||||
const [deleteRepoOpen, setDeleteRepoOpen] = useState(false);
|
||||
const { deleting, runDelete } = useCardDelete({
|
||||
action: () => deleteCachedModel(repoId, undefined, hfToken || undefined),
|
||||
|
|
@ -170,7 +171,8 @@ export function SafetensorsDownloadCard({
|
|||
!isLoadingThisModel;
|
||||
|
||||
return (
|
||||
<div className="flex w-full flex-col gap-2"><DownloadCard
|
||||
<div className="flex w-full flex-col gap-2">
|
||||
<DownloadCard
|
||||
job={job}
|
||||
progress={downloading ? progress : null}
|
||||
dialogs={
|
||||
|
|
@ -206,7 +208,7 @@ export function SafetensorsDownloadCard({
|
|||
)}
|
||||
{!isDownloaded && !isActive && isPartial && !downloading && (
|
||||
<Tooltip>
|
||||
<TooltipTrigger asChild>
|
||||
<TooltipTrigger asChild={true}>
|
||||
<span className="inline-flex">
|
||||
<DotTag tone="warning" label="Partial" />
|
||||
</span>
|
||||
|
|
@ -227,17 +229,20 @@ export function SafetensorsDownloadCard({
|
|||
)}
|
||||
</span>
|
||||
<div className="ml-auto flex items-center gap-0.5">
|
||||
{canDelete && (
|
||||
<CardDeleteButton
|
||||
label={`Delete ${repoId}`}
|
||||
onClick={() => setDeleteRepoOpen(true)}
|
||||
/>
|
||||
)}
|
||||
{isDownloaded && cachePath && (
|
||||
<PathInfoButton
|
||||
path={cachePath}
|
||||
title="On-device location"
|
||||
description={`Where ${repoId} lives on disk.`}
|
||||
{/* TODO: inference settings gear hidden for now, work on it in a future PR. */}
|
||||
{/* Same 3-dots menu as GGUF, at repo level (no quant); pinning is
|
||||
omitted in the run bar. Managed HF-cache repos only. */}
|
||||
{(isDownloaded || (isPartial && !downloading)) &&
|
||||
!/^([/\\~.]|[A-Za-z]:)/.test(repoId) && (
|
||||
<QuantOptionsMenu
|
||||
repoId={repoId}
|
||||
label={repoId}
|
||||
downloaded={isDownloaded}
|
||||
canDelete={canDelete}
|
||||
onDelete={() => setDeleteRepoOpen(true)}
|
||||
showPin={false}
|
||||
buttonClassName="ml-0.5 size-7"
|
||||
iconClassName="size-4"
|
||||
/>
|
||||
)}
|
||||
</div>
|
||||
|
|
@ -268,7 +273,7 @@ export function SafetensorsDownloadCard({
|
|||
onClick={() => {
|
||||
if (!canRun) return;
|
||||
if (isActive) {
|
||||
onUseInChat?.();
|
||||
onEject?.();
|
||||
return;
|
||||
}
|
||||
onLoad({});
|
||||
|
|
@ -287,26 +292,26 @@ export function SafetensorsDownloadCard({
|
|||
</>
|
||||
) : isActive ? (
|
||||
<>
|
||||
<HugeiconsIcon icon={PencilEdit02Icon} strokeWidth={1.75} />
|
||||
Chat
|
||||
<HugeiconsIcon icon={RemoveCircleIcon} strokeWidth={1.75} />
|
||||
Eject
|
||||
</>
|
||||
) : !canRun ? (
|
||||
<>
|
||||
<HugeiconsIcon icon={Alert02Icon} strokeWidth={1.75} />
|
||||
No run
|
||||
</>
|
||||
) : (
|
||||
) : canRun ? (
|
||||
<>
|
||||
<HugeiconsIcon icon={PlayIcon} strokeWidth={1.75} />
|
||||
Run
|
||||
</>
|
||||
) : (
|
||||
<>
|
||||
<HugeiconsIcon icon={Alert02Icon} strokeWidth={1.75} />
|
||||
No run
|
||||
</>
|
||||
)}
|
||||
</button>
|
||||
</div>
|
||||
) : showUnavailableAction ? (
|
||||
<button
|
||||
type="button"
|
||||
disabled
|
||||
disabled={true}
|
||||
className="hub-action-btn w-28 opacity-70"
|
||||
>
|
||||
<HugeiconsIcon icon={Alert02Icon} strokeWidth={1.75} />
|
||||
|
|
|
|||
|
|
@ -0,0 +1,435 @@
|
|||
// SPDX-License-Identifier: AGPL-3.0-only
|
||||
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
// Gear button for the GGUF run bar: model config, system prompt, reasoning,
|
||||
// sampling, tools and retrieval, using the same controls as the chat page's
|
||||
// Run settings. Edits write to the chat runtime store's persisted state.
|
||||
|
||||
import {
|
||||
Dialog,
|
||||
DialogContent,
|
||||
DialogDescription,
|
||||
DialogHeader,
|
||||
DialogTitle,
|
||||
} from "@/components/ui/dialog";
|
||||
import { InfoHint } from "@/components/ui/info-hint";
|
||||
import { Switch } from "@/components/ui/switch";
|
||||
import { Textarea } from "@/components/ui/textarea";
|
||||
import {
|
||||
Tooltip,
|
||||
TooltipContent,
|
||||
TooltipTrigger,
|
||||
} from "@/components/ui/tooltip";
|
||||
import {
|
||||
ParamSlider,
|
||||
useChatModelRuntime,
|
||||
useChatRuntimeStore,
|
||||
} from "@/features/chat";
|
||||
import {
|
||||
type PerModelConfig,
|
||||
SidebarModelConfig,
|
||||
applyPerModelConfigToRuntime,
|
||||
currentRuntimePerModelConfig,
|
||||
useActiveModelConfig,
|
||||
} from "@/features/model-picker";
|
||||
import { RetrievalSettingsSection } from "@/features/rag";
|
||||
import { toast } from "@/lib/toast";
|
||||
import { cn } from "@/lib/utils";
|
||||
import { Settings02Icon } from "@hugeicons/core-free-icons";
|
||||
import { HugeiconsIcon } from "@hugeicons/react";
|
||||
import { type ReactNode, useCallback, useState } from "react";
|
||||
import { HubOptionMenu } from "./hub-option-menu";
|
||||
|
||||
function SettingsSection({
|
||||
label,
|
||||
labelClassName,
|
||||
children,
|
||||
}: {
|
||||
label: string;
|
||||
labelClassName?: string;
|
||||
children: ReactNode;
|
||||
}) {
|
||||
return (
|
||||
<div className="border-t border-border/50 pb-5 pt-5 first:border-t-0 first:pt-0">
|
||||
<div
|
||||
className={cn(
|
||||
"pb-4 text-[11px] font-semibold uppercase tracking-wider text-muted-foreground",
|
||||
labelClassName,
|
||||
)}
|
||||
>
|
||||
{label}
|
||||
</div>
|
||||
<div className="flex flex-col gap-5">{children}</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function ToggleRow({
|
||||
label,
|
||||
info,
|
||||
checked,
|
||||
disabled,
|
||||
onCheckedChange,
|
||||
}: {
|
||||
label: string;
|
||||
info?: string;
|
||||
checked: boolean;
|
||||
disabled?: boolean;
|
||||
onCheckedChange: (checked: boolean) => void;
|
||||
}) {
|
||||
return (
|
||||
<div className="flex items-center justify-between gap-3">
|
||||
<div className="flex min-w-0 items-center gap-1.5">
|
||||
<span className="text-[13px] font-medium text-nav-fg">{label}</span>
|
||||
{info && <InfoHint>{info}</InfoHint>}
|
||||
</div>
|
||||
<Switch
|
||||
checked={checked}
|
||||
disabled={disabled}
|
||||
onCheckedChange={(value) => onCheckedChange(value === true)}
|
||||
aria-label={label}
|
||||
/>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
export function SamplingSettingsButton({ className }: { className?: string }) {
|
||||
const [open, setOpen] = useState(false);
|
||||
const params = useChatRuntimeStore((s) => s.params);
|
||||
const setParams = useChatRuntimeStore((s) => s.setParams);
|
||||
|
||||
// Loaded model's per-model config (context length etc.), mirroring the chat
|
||||
// page's Run settings Model section.
|
||||
const { selectModel } = useChatModelRuntime();
|
||||
const modelLoading = useChatRuntimeStore((s) => s.modelLoading);
|
||||
const activeGgufVariant = useChatRuntimeStore((s) => s.activeGgufVariant);
|
||||
const ggufContextLength = useChatRuntimeStore((s) => s.ggufContextLength);
|
||||
const ggufNativeContextLength = useChatRuntimeStore(
|
||||
(s) => s.ggufNativeContextLength,
|
||||
);
|
||||
const {
|
||||
checkpoint,
|
||||
isGguf: activeModelIsGguf,
|
||||
config: activeModelConfig,
|
||||
} = useActiveModelConfig();
|
||||
const handleReloadActiveModel = useCallback(
|
||||
(config: PerModelConfig) => {
|
||||
const runtime = useChatRuntimeStore.getState();
|
||||
const activeCheckpoint = runtime.params.checkpoint;
|
||||
if (!activeCheckpoint) return;
|
||||
const nativeToken = runtime.activeNativePathToken;
|
||||
const nativeExpiry = runtime.activeNativePathExpiresAtMs;
|
||||
// Mirrors the chat page: an expired native-path token can't reload.
|
||||
if (nativeToken && nativeExpiry != null && Date.now() >= nativeExpiry) {
|
||||
toast.error("This local model file's access has expired.", {
|
||||
description: "Re-select the model file to reload it.",
|
||||
});
|
||||
return;
|
||||
}
|
||||
// selectModel reads config from the runtime store, not the selection, so
|
||||
// apply it first (snapshotting the current one for rollback).
|
||||
const previousConfig = currentRuntimePerModelConfig({
|
||||
includeMaxSeqLength: true,
|
||||
});
|
||||
applyPerModelConfigToRuntime(config);
|
||||
void selectModel({
|
||||
id: activeCheckpoint,
|
||||
source: "local",
|
||||
ggufVariant: runtime.activeGgufVariant ?? undefined,
|
||||
nativePathToken: nativeToken ?? undefined,
|
||||
nativePathExpiresAtMs: nativeExpiry,
|
||||
isGguf: activeModelIsGguf,
|
||||
isDownloaded: true,
|
||||
keepSpeculative: true,
|
||||
previousConfig,
|
||||
forceReload: true,
|
||||
});
|
||||
},
|
||||
[selectModel, activeModelIsGguf],
|
||||
);
|
||||
|
||||
// Reasoning + tools: same store bindings as the chat page.
|
||||
const reasoningEnabled = useChatRuntimeStore((s) => s.reasoningEnabled);
|
||||
const reasoningAlwaysOn = useChatRuntimeStore((s) => s.reasoningAlwaysOn);
|
||||
const setReasoningEnabled = useChatRuntimeStore((s) => s.setReasoningEnabled);
|
||||
const reasoningEffort = useChatRuntimeStore((s) => s.reasoningEffort);
|
||||
const reasoningEffortLevels = useChatRuntimeStore(
|
||||
(s) => s.reasoningEffortLevels,
|
||||
);
|
||||
const setReasoningEffort = useChatRuntimeStore((s) => s.setReasoningEffort);
|
||||
const supportsPreserveThinking = useChatRuntimeStore(
|
||||
(s) => s.supportsPreserveThinking,
|
||||
);
|
||||
const preserveThinking = useChatRuntimeStore((s) => s.preserveThinking);
|
||||
const setPreserveThinking = useChatRuntimeStore((s) => s.setPreserveThinking);
|
||||
const maxToolCalls = useChatRuntimeStore((s) => s.maxToolCallsPerMessage);
|
||||
const setMaxToolCalls = useChatRuntimeStore(
|
||||
(s) => s.setMaxToolCallsPerMessage,
|
||||
);
|
||||
const toolCallTimeout = useChatRuntimeStore((s) => s.toolCallTimeout);
|
||||
const setToolCallTimeout = useChatRuntimeStore((s) => s.setToolCallTimeout);
|
||||
const autoHealToolCalls = useChatRuntimeStore((s) => s.autoHealToolCalls);
|
||||
const setAutoHealToolCalls = useChatRuntimeStore(
|
||||
(s) => s.setAutoHealToolCalls,
|
||||
);
|
||||
const nudgeToolCalls = useChatRuntimeStore((s) => s.nudgeToolCalls);
|
||||
const setNudgeToolCalls = useChatRuntimeStore((s) => s.setNudgeToolCalls);
|
||||
const permissionMode = useChatRuntimeStore((s) => s.permissionMode);
|
||||
const setConfirmToolCalls = useChatRuntimeStore((s) => s.setConfirmToolCalls);
|
||||
|
||||
const toolsEnabled = useChatRuntimeStore((s) => s.toolsEnabled);
|
||||
const setToolsEnabled = useChatRuntimeStore((s) => s.setToolsEnabled);
|
||||
|
||||
const set = (key: keyof typeof params) => (value: number) =>
|
||||
setParams({ ...params, [key]: value });
|
||||
|
||||
// Slider 0-41; 41 maps to 9999 ("Max"), mirroring the chat page.
|
||||
const toolCallsSliderValue =
|
||||
maxToolCalls >= 9999 ? 41 : Math.min(maxToolCalls, 40);
|
||||
// Slider 1-31; 31 maps to 9999 ("Max").
|
||||
const timeoutSliderValue =
|
||||
toolCallTimeout >= 9999 ? 31 : Math.min(Math.max(toolCallTimeout, 1), 30);
|
||||
|
||||
return (
|
||||
<>
|
||||
<Tooltip>
|
||||
<TooltipTrigger asChild={true}>
|
||||
<button
|
||||
type="button"
|
||||
aria-label="Inference settings"
|
||||
onClick={(e) => {
|
||||
e.stopPropagation();
|
||||
setOpen(true);
|
||||
}}
|
||||
className={cn(
|
||||
"inline-flex size-7 shrink-0 cursor-pointer items-center justify-center rounded-full text-muted-foreground transition-colors hover:bg-muted hover:text-foreground",
|
||||
className,
|
||||
)}
|
||||
>
|
||||
<HugeiconsIcon
|
||||
icon={Settings02Icon}
|
||||
strokeWidth={1.75}
|
||||
className="size-4"
|
||||
/>
|
||||
</button>
|
||||
</TooltipTrigger>
|
||||
<TooltipContent side="bottom" className="tooltip-compact">
|
||||
Inference settings
|
||||
</TooltipContent>
|
||||
</Tooltip>
|
||||
<Dialog open={open} onOpenChange={setOpen}>
|
||||
<DialogContent
|
||||
className="flex max-h-[85vh] flex-col gap-5 sm:max-w-[580px]"
|
||||
onClick={(e) => e.stopPropagation()}
|
||||
>
|
||||
<DialogHeader>
|
||||
<DialogTitle>Inference settings</DialogTitle>
|
||||
<DialogDescription>
|
||||
Applies to chats with local models.
|
||||
</DialogDescription>
|
||||
</DialogHeader>
|
||||
<div className="-mr-2 flex min-h-0 flex-col overflow-y-auto overflow-x-hidden pr-2 [scrollbar-width:thin]">
|
||||
{checkpoint && activeModelConfig && !modelLoading && (
|
||||
<SettingsSection label="Model">
|
||||
<SidebarModelConfig
|
||||
modelId={checkpoint}
|
||||
ggufVariant={activeGgufVariant ?? null}
|
||||
isGguf={activeModelIsGguf}
|
||||
nativeContextLength={ggufNativeContextLength}
|
||||
loadedContextLength={ggufContextLength}
|
||||
loadedConfig={activeModelConfig}
|
||||
onReload={handleReloadActiveModel}
|
||||
/>
|
||||
</SettingsSection>
|
||||
)}
|
||||
|
||||
<SettingsSection label="System Prompt">
|
||||
<Textarea
|
||||
value={params.systemPrompt}
|
||||
onChange={(e) =>
|
||||
setParams({ ...params, systemPrompt: e.target.value })
|
||||
}
|
||||
placeholder="Instructions sent before every conversation."
|
||||
aria-label="System prompt"
|
||||
className="min-h-[84px] resize-y text-[13px]"
|
||||
/>
|
||||
</SettingsSection>
|
||||
|
||||
<SettingsSection label="Reasoning" labelClassName="pb-5">
|
||||
<ToggleRow
|
||||
label="Enable reasoning"
|
||||
checked={reasoningAlwaysOn || reasoningEnabled}
|
||||
disabled={reasoningAlwaysOn}
|
||||
onCheckedChange={setReasoningEnabled}
|
||||
/>
|
||||
{reasoningEffortLevels.length > 0 && (
|
||||
<div className="flex items-center justify-between gap-3">
|
||||
<span className="text-[13px] font-medium text-nav-fg">
|
||||
Reasoning effort
|
||||
</span>
|
||||
<HubOptionMenu
|
||||
value={reasoningEffort}
|
||||
options={reasoningEffortLevels.map((level) => ({
|
||||
value: level,
|
||||
label: level.charAt(0).toUpperCase() + level.slice(1),
|
||||
}))}
|
||||
onValueChange={setReasoningEffort}
|
||||
ariaLabel="Reasoning effort"
|
||||
align="end"
|
||||
className="h-8 text-[11.5px]"
|
||||
/>
|
||||
</div>
|
||||
)}
|
||||
{supportsPreserveThinking && (
|
||||
<ToggleRow
|
||||
label="Preserve thinking"
|
||||
info="Keep earlier turns' reasoning in context so the model can build on it. Uses more context."
|
||||
checked={preserveThinking}
|
||||
onCheckedChange={setPreserveThinking}
|
||||
/>
|
||||
)}
|
||||
</SettingsSection>
|
||||
|
||||
<SettingsSection label="Sampling">
|
||||
<ParamSlider
|
||||
label="Temperature"
|
||||
value={params.temperature}
|
||||
min={0}
|
||||
max={2}
|
||||
step={0.01}
|
||||
onChange={set("temperature")}
|
||||
info="Controls randomness. Lower values make output focused and deterministic; higher values increase variety and creativity."
|
||||
/>
|
||||
<ParamSlider
|
||||
label="Top P"
|
||||
value={params.topP}
|
||||
min={0}
|
||||
max={1}
|
||||
step={0.05}
|
||||
onChange={set("topP")}
|
||||
displayValue={params.topP === 1 ? "Off" : undefined}
|
||||
info="Nucleus sampling. Restricts choices to the smallest set of tokens whose cumulative probability reaches this threshold. 1.0 = off."
|
||||
/>
|
||||
<ParamSlider
|
||||
label="Top K"
|
||||
value={params.topK}
|
||||
min={0}
|
||||
max={100}
|
||||
step={1}
|
||||
onChange={set("topK")}
|
||||
displayValue={params.topK === 0 ? "Off" : undefined}
|
||||
info="Limits sampling to the K most likely tokens at each step. 0 = off."
|
||||
/>
|
||||
<ParamSlider
|
||||
label="Min P"
|
||||
value={params.minP}
|
||||
min={0}
|
||||
max={1}
|
||||
step={0.01}
|
||||
onChange={set("minP")}
|
||||
info="Drops tokens whose probability is below this fraction of the top token's probability. Filters unlikely candidates."
|
||||
/>
|
||||
<ParamSlider
|
||||
label="Repetition Penalty"
|
||||
value={params.repetitionPenalty}
|
||||
min={1}
|
||||
max={2}
|
||||
step={0.05}
|
||||
onChange={set("repetitionPenalty")}
|
||||
displayValue={
|
||||
params.repetitionPenalty === 1 ? "Off" : undefined
|
||||
}
|
||||
info="Down-weights tokens that have already appeared, reducing repetition. 1.0 = off; higher values penalize more strongly."
|
||||
/>
|
||||
<ParamSlider
|
||||
label="Presence Penalty"
|
||||
value={params.presencePenalty}
|
||||
min={0}
|
||||
max={2}
|
||||
step={0.1}
|
||||
onChange={set("presencePenalty")}
|
||||
displayValue={params.presencePenalty === 0 ? "Off" : undefined}
|
||||
info="Penalizes any token that has already appeared at least once, encouraging the model to introduce new topics. 0 = off."
|
||||
/>
|
||||
<ParamSlider
|
||||
label="Max Tokens"
|
||||
value={params.maxTokens}
|
||||
min={64}
|
||||
max={131072}
|
||||
step={64}
|
||||
onChange={set("maxTokens")}
|
||||
valueSize={6}
|
||||
info="Maximum number of tokens to generate per response. Generation stops at this limit or when the model emits an end-of-sequence token."
|
||||
/>
|
||||
</SettingsSection>
|
||||
|
||||
<SettingsSection label="Tools">
|
||||
<ToggleRow
|
||||
label="Enable tools"
|
||||
info="Master switch for tool use in chat. When off, the model answers without calling any tools."
|
||||
checked={toolsEnabled}
|
||||
onCheckedChange={setToolsEnabled}
|
||||
/>
|
||||
<ToggleRow
|
||||
label="Auto-healing tool calls"
|
||||
info="Unsloth auto-fixes broken tool calls so inference output is never broken."
|
||||
checked={autoHealToolCalls}
|
||||
onCheckedChange={setAutoHealToolCalls}
|
||||
/>
|
||||
<ToggleRow
|
||||
label="Nudge tool calls"
|
||||
info="When a tool call cannot be repaired, re-ask the model once so the intended tool still runs."
|
||||
checked={nudgeToolCalls}
|
||||
onCheckedChange={setNudgeToolCalls}
|
||||
/>
|
||||
<ToggleRow
|
||||
label="Confirm tool calls"
|
||||
info="Pause every local tool call for your approval before it runs. Overridden by Full access."
|
||||
checked={permissionMode === "ask"}
|
||||
disabled={permissionMode === "full"}
|
||||
onCheckedChange={setConfirmToolCalls}
|
||||
/>
|
||||
<ParamSlider
|
||||
label="Max Tool Calls Per Message"
|
||||
value={toolCallsSliderValue}
|
||||
min={0}
|
||||
max={41}
|
||||
step={1}
|
||||
onChange={(v) => setMaxToolCalls(v >= 41 ? 9999 : v)}
|
||||
displayValue={
|
||||
toolCallsSliderValue >= 41
|
||||
? "Max"
|
||||
: toolCallsSliderValue === 0
|
||||
? "Off"
|
||||
: undefined
|
||||
}
|
||||
info="Cap on tool/function calls the model may invoke within a single response. 0 disables tool use; Max removes the cap."
|
||||
/>
|
||||
<ParamSlider
|
||||
label="Max Tool Call Duration"
|
||||
value={timeoutSliderValue}
|
||||
min={1}
|
||||
max={31}
|
||||
step={1}
|
||||
onChange={(v) => setToolCallTimeout(v >= 31 ? 9999 : v)}
|
||||
displayValue={
|
||||
timeoutSliderValue >= 31
|
||||
? "Max"
|
||||
: timeoutSliderValue === 1
|
||||
? "1 minute"
|
||||
: `${timeoutSliderValue} minutes`
|
||||
}
|
||||
valueSize={10}
|
||||
info="Per-call wall-clock limit. Long-running tool executions are terminated when this elapses; the model continues with what completed."
|
||||
/>
|
||||
</SettingsSection>
|
||||
|
||||
<SettingsSection label="Retrieval">
|
||||
<RetrievalSettingsSection />
|
||||
</SettingsSection>
|
||||
</div>
|
||||
</DialogContent>
|
||||
</Dialog>
|
||||
</>
|
||||
);
|
||||
}
|
||||
|
|
@ -1,12 +1,19 @@
|
|||
// SPDX-License-Identifier: AGPL-3.0-only
|
||||
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
import {
|
||||
Tooltip,
|
||||
TooltipContent,
|
||||
TooltipTrigger,
|
||||
} from "@/components/ui/tooltip";
|
||||
import { cn } from "@/lib/utils";
|
||||
import {
|
||||
BrainIcon,
|
||||
Chat01Icon,
|
||||
CodeIcon,
|
||||
GlobeIcon,
|
||||
HeadphonesIcon,
|
||||
ImageIcon,
|
||||
LockIcon,
|
||||
LockKeyIcon,
|
||||
SparklesIcon,
|
||||
|
|
@ -15,12 +22,6 @@ import {
|
|||
} from "@hugeicons/core-free-icons";
|
||||
import type { IconSvgElement } from "@hugeicons/react";
|
||||
import { HugeiconsIcon } from "@hugeicons/react";
|
||||
import {
|
||||
Tooltip,
|
||||
TooltipContent,
|
||||
TooltipTrigger,
|
||||
} from "@/components/ui/tooltip";
|
||||
import { cn } from "@/lib/utils";
|
||||
import type { Capability, CapabilityKey } from "../lib/model-capabilities";
|
||||
|
||||
const CAPABILITY_ICON: Record<CapabilityKey, IconSvgElement> = {
|
||||
|
|
@ -30,6 +31,7 @@ const CAPABILITY_ICON: Record<CapabilityKey, IconSvgElement> = {
|
|||
reasoning: BrainIcon,
|
||||
code: CodeIcon,
|
||||
embedding: SparklesIcon,
|
||||
diffusion: ImageIcon,
|
||||
multilingual: GlobeIcon,
|
||||
conversational: Chat01Icon,
|
||||
};
|
||||
|
|
@ -45,6 +47,8 @@ const CAPABILITY_TONE: Record<CapabilityKey, string> = {
|
|||
code: "bg-cyan-500/10 text-cyan-800 dark:bg-cyan-400/20 dark:text-cyan-300",
|
||||
embedding:
|
||||
"bg-emerald-500/10 text-emerald-700 dark:bg-emerald-400/20 dark:text-emerald-300",
|
||||
diffusion:
|
||||
"bg-pink-500/10 text-pink-700 dark:bg-pink-400/20 dark:text-pink-300",
|
||||
multilingual:
|
||||
"bg-sky-500/10 text-sky-700 dark:bg-sky-400/20 dark:text-sky-300",
|
||||
conversational:
|
||||
|
|
@ -59,9 +63,7 @@ export function AccessChip({ label }: { label: string }) {
|
|||
);
|
||||
}
|
||||
|
||||
function isGatedAccess(
|
||||
gated: false | "auto" | "manual" | undefined,
|
||||
): boolean {
|
||||
function isGatedAccess(gated: false | "auto" | "manual" | undefined): boolean {
|
||||
return gated !== false && gated !== undefined;
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -1,14 +1,10 @@
|
|||
// SPDX-License-Identifier: AGPL-3.0-only
|
||||
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
import { DOWNLOAD_KIND } from "./constants";
|
||||
import {
|
||||
createDownloadManagerInitialState,
|
||||
jobKeyOf,
|
||||
removeJob,
|
||||
selectActiveJob,
|
||||
setState,
|
||||
useDownloadManagerStore,
|
||||
} from "./download-manager-state";
|
||||
import { resetDownloadApiAdapterState } from "./download-api-adapter";
|
||||
import {
|
||||
|
|
@ -69,25 +65,6 @@ export const downloadManager: DownloadManagerController = {
|
|||
dismiss: removeJob,
|
||||
};
|
||||
|
||||
/** Cancel the in-flight download for a staged model pick. No-op when nothing is
|
||||
* downloading (e.g. a native/local file that was never fetched). Lets non-React
|
||||
* callers (the chat store's abandon paths) stop a staged transfer without the
|
||||
* useRepoDownload hook. */
|
||||
export function cancelStagedModelDownload(
|
||||
pending: { id: string; ggufVariant?: string | null } | null,
|
||||
): void {
|
||||
if (!pending) return;
|
||||
const variant = pending.ggufVariant ?? null;
|
||||
const activeJob = selectActiveJob(
|
||||
useDownloadManagerStore.getState(),
|
||||
DOWNLOAD_KIND.MODEL,
|
||||
pending.id,
|
||||
variant,
|
||||
);
|
||||
void downloadManager.cancel(
|
||||
activeJob?.key ?? jobKeyOf(DOWNLOAD_KIND.MODEL, pending.id, variant),
|
||||
);
|
||||
}
|
||||
|
||||
if (import.meta.hot) {
|
||||
import.meta.hot.dispose(() => {
|
||||
|
|
|
|||
|
|
@ -20,7 +20,6 @@ export {
|
|||
} from "./constants";
|
||||
export {
|
||||
__resetDownloadManagerForTests,
|
||||
cancelStagedModelDownload,
|
||||
clearCompletedInventoryHint,
|
||||
downloadManager,
|
||||
hydrateDownloadManager,
|
||||
|
|
|
|||
|
|
@ -1,28 +1,7 @@
|
|||
// SPDX-License-Identifier: AGPL-3.0-only
|
||||
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
import {
|
||||
loadRememberedLoadSettings,
|
||||
rememberedLoadSettingsKey,
|
||||
} from "@/components/assistant-ui/model-selector/remembered-load-settings";
|
||||
import { hfModelFitsDevice } from "@/components/assistant-ui/model-selector/recommended-fit";
|
||||
import { useHubInventory } from "@/features/hub/inventory";
|
||||
import { useDebouncedValue } from "@/hooks/use-debounced-value";
|
||||
import { useGpuInfo } from "@/hooks/use-gpu-info";
|
||||
import {
|
||||
type HfModelSearchChannel,
|
||||
type HfSortDirection,
|
||||
type HfSortKey,
|
||||
} from "@/features/hub/hooks/use-hub-model-search";
|
||||
import { useOnlineStatus } from "@/features/hub/hooks/use-online-status";
|
||||
import { useHubInfiniteScroll } from "@/features/hub/hooks/use-hub-infinite-scroll";
|
||||
import { ggufVariantsMatch, modelIdsMatch } from "@/features/hub/lib/model-identity";
|
||||
import { cn } from "@/lib/utils";
|
||||
import { usePlatformStore } from "@/config/env";
|
||||
import {
|
||||
hfApiToken,
|
||||
useHfTokenStore,
|
||||
} from "@/features/hub/stores/hf-token-store";
|
||||
import {
|
||||
isChannelEntryFresh,
|
||||
useHubFeedStore,
|
||||
|
|
@ -33,6 +12,25 @@ import {
|
|||
useChatModelRuntime,
|
||||
useChatRuntimeStore,
|
||||
} from "@/features/chat";
|
||||
import { useHubInventory } from "./inventory";
|
||||
import type {
|
||||
HfModelSearchChannel,
|
||||
HfSortDirection,
|
||||
HfSortKey,
|
||||
} from "./hooks/use-hub-model-search";
|
||||
import { useOnlineStatus } from "@/features/hub";
|
||||
import { useHubInfiniteScroll } from "@/features/hub";
|
||||
import { ggufVariantsMatch, modelIdsMatch } from "./lib/model-identity";
|
||||
import { hfApiToken, useHfTokenStore } from "./stores/hf-token-store";
|
||||
import {
|
||||
applyModelLoadConfigToRuntime,
|
||||
currentRuntimePerModelConfig,
|
||||
hfModelFitsDevice,
|
||||
resolveInitialConfig,
|
||||
} from "@/features/model-picker";
|
||||
import { useDebouncedValue } from "@/hooks/use-debounced-value";
|
||||
import { useGpuInfo } from "@/hooks/use-gpu-info";
|
||||
import { cn } from "@/lib/utils";
|
||||
import { useNavigate, useSearch } from "@tanstack/react-router";
|
||||
import {
|
||||
useCallback,
|
||||
|
|
@ -42,17 +40,10 @@ import {
|
|||
useRef,
|
||||
useState,
|
||||
} from "react";
|
||||
import { ExternalLinkConfirmDialog } from "./catalog/external-link-confirm-dialog";
|
||||
import { HubDetailView } from "./catalog/hub-detail-view";
|
||||
import { HubTopBar } from "./catalog/hub-top-bar";
|
||||
import { HubFeed } from "./catalog/hub-feed";
|
||||
import { OwnerScopeToggle } from "./catalog/owner-scope-toggle";
|
||||
import {
|
||||
type AllModelsView,
|
||||
HubListHeader,
|
||||
type InventorySort,
|
||||
InventorySortControl,
|
||||
ResultListHeader,
|
||||
} from "./catalog/models-table";
|
||||
import { HubTopBar } from "./catalog/hub-top-bar";
|
||||
import {
|
||||
ModelsCatalog,
|
||||
type ModelsCatalogHandlers,
|
||||
|
|
@ -60,14 +51,22 @@ import {
|
|||
type ModelsCatalogState,
|
||||
} from "./catalog/models-catalog";
|
||||
import { ModelsHeader } from "./catalog/models-header";
|
||||
import {
|
||||
type AllModelsView,
|
||||
HubListHeader,
|
||||
type InventorySort,
|
||||
InventorySortControl,
|
||||
InventoryTypeFilterControl,
|
||||
ResultListHeader,
|
||||
} from "./catalog/models-table";
|
||||
import { ModelsToolbar } from "./catalog/models-toolbar";
|
||||
import { ExternalLinkConfirmDialog } from "./catalog/external-link-confirm-dialog";
|
||||
import { OnDeviceFoldersDialog } from "./catalog/on-device-folders-dialog";
|
||||
import { OwnerScopeToggle } from "./catalog/owner-scope-toggle";
|
||||
import { useDiscoverSearch } from "./hooks/use-discover-search";
|
||||
import { useFeedWriteBack } from "./hooks/use-feed-write-back";
|
||||
import { useHiddenEmbeddingModelIds } from "./hooks/use-hidden-embedding-models";
|
||||
import { useHubFeed } from "./hooks/use-hub-feed";
|
||||
import { useHubModelVram } from "./hooks/use-hub-model-vram";
|
||||
import { useHiddenEmbeddingModelIds } from "./hooks/use-hidden-embedding-models";
|
||||
import { useModelsSelection } from "./hooks/use-models-selection";
|
||||
import {
|
||||
CHANNEL_TO_SECTION,
|
||||
|
|
@ -83,6 +82,10 @@ import {
|
|||
isHiddenModelId,
|
||||
} from "./lib/hidden-models";
|
||||
import { inventoryRowMatches, tokenizeQuery } from "./lib/inventory-search";
|
||||
import {
|
||||
type ModelTypeFilter,
|
||||
matchesModelType,
|
||||
} from "./lib/model-type-filter";
|
||||
import { resolveOwnerProviderLogo } from "./lib/provider-logos";
|
||||
import { fingerprintToken } from "./lib/token-fingerprint";
|
||||
import {
|
||||
|
|
@ -456,6 +459,8 @@ export function ModelsPage() {
|
|||
setInventorySortState(sort);
|
||||
writeInventorySortPreference(sort);
|
||||
}, []);
|
||||
const [inventoryTypeFilter, setInventoryTypeFilter] =
|
||||
useState<ModelTypeFilter>("all");
|
||||
const [foldersDialogOpen, setFoldersDialogOpen] = useState(false);
|
||||
const [discoverFetchIntent, setDiscoverFetchIntent] = useState(0);
|
||||
const [sortBrowseActive, setSortBrowseActive] = useState(false);
|
||||
|
|
@ -582,15 +587,15 @@ export function ModelsPage() {
|
|||
const deferredCapabilityFilter = useDeferredValue(capabilityFilter);
|
||||
|
||||
const hasQuery = deferredDebouncedQuery.trim() !== "";
|
||||
const mode: DiscoverMode = !isModelDiscover
|
||||
? "search"
|
||||
: hasQuery
|
||||
const mode: DiscoverMode = isModelDiscover
|
||||
? hasQuery
|
||||
? "search"
|
||||
: urlSection != null
|
||||
? "channel-list"
|
||||
: sortBrowseActive
|
||||
? "search"
|
||||
: "feed";
|
||||
: "feed"
|
||||
: "search";
|
||||
const isFeedMode = mode === "feed";
|
||||
const isChannelListMode = mode === "channel-list";
|
||||
const isSortBrowseMode =
|
||||
|
|
@ -737,7 +742,10 @@ export function ModelsPage() {
|
|||
// The default feed only shows models with a provider logo.
|
||||
(!isFeedMode ||
|
||||
resolveOwnerProviderLogo(row.owner, row.repo) !== null) &&
|
||||
matchesFormat(detectResultFormat(row.result), effectiveDiscoverFormat) &&
|
||||
matchesFormat(
|
||||
detectResultFormat(row.result),
|
||||
effectiveDiscoverFormat,
|
||||
) &&
|
||||
matchesCapability(row.capabilities, deferredCapabilityFilter) &&
|
||||
(!activeChannel?.finetunableOnly || isUnslothFinetunable(row.result)) &&
|
||||
// Models already on disk stay visible regardless of device fit,
|
||||
|
|
@ -807,7 +815,10 @@ export function ModelsPage() {
|
|||
}
|
||||
return merged;
|
||||
}, [isFeedMode, feedTrendingRows, filteredDiscoverRows]);
|
||||
const feedResults = useMemo(() => feedRows.map((row) => row.result), [feedRows]);
|
||||
const feedResults = useMemo(
|
||||
() => feedRows.map((row) => row.result),
|
||||
[feedRows],
|
||||
);
|
||||
const selectionDiscoverRows = isFeedMode ? feedRows : discoverRows;
|
||||
const selectionFilteredDiscoverRows = isFeedMode
|
||||
? feedRows
|
||||
|
|
@ -837,7 +848,8 @@ export function ModelsPage() {
|
|||
// Local rows may lack a repo id, so also check path and title.
|
||||
return (
|
||||
!isHiddenModelId(row.id, row.repoId, row.path, row.title) ||
|
||||
(inventoryTokens.length > 0 && inventoryRowMatches(row, inventoryTokens))
|
||||
(inventoryTokens.length > 0 &&
|
||||
inventoryRowMatches(row, inventoryTokens))
|
||||
);
|
||||
},
|
||||
[hiddenEmbeddingModelIds, inventoryTokens],
|
||||
|
|
@ -855,6 +867,7 @@ export function ModelsPage() {
|
|||
// id/title/path happens to contain an infra needle is not dropped.
|
||||
isDatasetMode ||
|
||||
(matchesFormat(row.modelFormat, deferredFormatFilter) &&
|
||||
matchesModelType(row, inventoryTypeFilter) &&
|
||||
isVisibleInventoryRow(row)),
|
||||
),
|
||||
inventoryTokens,
|
||||
|
|
@ -863,6 +876,7 @@ export function ModelsPage() {
|
|||
effectiveCachedRows,
|
||||
isDatasetMode,
|
||||
deferredFormatFilter,
|
||||
inventoryTypeFilter,
|
||||
inventoryTokens,
|
||||
isVisibleInventoryRow,
|
||||
],
|
||||
|
|
@ -878,6 +892,7 @@ export function ModelsPage() {
|
|||
// id/title/path happens to contain an infra needle is not dropped.
|
||||
isDatasetMode ||
|
||||
(matchesFormat(row.modelFormat, deferredFormatFilter) &&
|
||||
matchesModelType(row, inventoryTypeFilter) &&
|
||||
isVisibleInventoryRow(row)),
|
||||
),
|
||||
inventoryTokens,
|
||||
|
|
@ -886,6 +901,7 @@ export function ModelsPage() {
|
|||
effectiveLocalRows,
|
||||
isDatasetMode,
|
||||
deferredFormatFilter,
|
||||
inventoryTypeFilter,
|
||||
inventoryTokens,
|
||||
isVisibleInventoryRow,
|
||||
],
|
||||
|
|
@ -918,6 +934,7 @@ export function ModelsPage() {
|
|||
resourceType,
|
||||
deferredFormatFilter,
|
||||
deferredCapabilityFilter,
|
||||
inventoryTypeFilter,
|
||||
effectiveSort,
|
||||
effectiveDirection,
|
||||
activeChannelId,
|
||||
|
|
@ -928,6 +945,7 @@ export function ModelsPage() {
|
|||
resourceType,
|
||||
deferredFormatFilter,
|
||||
deferredCapabilityFilter,
|
||||
inventoryTypeFilter,
|
||||
effectiveSort,
|
||||
effectiveDirection,
|
||||
activeChannelId,
|
||||
|
|
@ -946,6 +964,7 @@ export function ModelsPage() {
|
|||
}
|
||||
} else {
|
||||
setDownloadedFormat("all");
|
||||
setInventoryTypeFilter("all");
|
||||
}
|
||||
setCapabilityFilter("all");
|
||||
}, [isDiscoverTab, urlSection, navigate]);
|
||||
|
|
@ -1155,50 +1174,22 @@ export function ModelsPage() {
|
|||
(opts: ModelLoadOptions, isDownloaded: boolean) => {
|
||||
if (!selectedModel) return;
|
||||
const runId = selectedModel.resource.runId;
|
||||
// "Load on selection" off: stage GGUF picks instead of loading, so the
|
||||
// chat page's staging flow can read the header and show the load options.
|
||||
// Non-GGUF models have nothing to configure pre-load, so they load now.
|
||||
if (
|
||||
!useChatRuntimeStore.getState().loadOnSelection &&
|
||||
(opts.ggufVariant != null || selectedModel.isGguf)
|
||||
) {
|
||||
useChatRuntimeStore.getState().stageModel({
|
||||
id: runId,
|
||||
ggufVariant: opts.ggufVariant,
|
||||
isGguf: selectedModel.isGguf,
|
||||
isDownloaded,
|
||||
expectedBytes: opts.expectedBytes,
|
||||
});
|
||||
openNewChat();
|
||||
return;
|
||||
}
|
||||
// Detach any leftover staged pick first so its edited knobs (e.g. a custom
|
||||
// context length) don't leak into this load -- mirrors the chat page's
|
||||
// detachStaged(); keepDownload keeps any staged download running.
|
||||
useChatRuntimeStore.getState().abandonStagedModel({ keepDownload: true });
|
||||
// Load-on-selection skips the chat sheet, so seed this GGUF pick's saved
|
||||
// load knobs here the way the sheet's restore effect would; otherwise the
|
||||
// remembered config is silently ignored on the Hub run path. keepSpeculative
|
||||
// then honors the restored speculative choice across the switch.
|
||||
const remembered =
|
||||
opts.ggufVariant != null || selectedModel.isGguf
|
||||
? loadRememberedLoadSettings(
|
||||
rememberedLoadSettingsKey({
|
||||
id: runId,
|
||||
ggufVariant: opts.ggufVariant,
|
||||
}),
|
||||
)
|
||||
: null;
|
||||
if (remembered) {
|
||||
useChatRuntimeStore.getState().applyRememberedLoadSettings(remembered);
|
||||
}
|
||||
const resolvedConfig = resolveInitialConfig(runId, opts.ggufVariant);
|
||||
const rememberedConfig = resolvedConfig.remembered
|
||||
? resolvedConfig.config
|
||||
: null;
|
||||
const previousConfig = currentRuntimePerModelConfig({
|
||||
includeMaxSeqLength: true,
|
||||
});
|
||||
const hasAppliedConfig = applyModelLoadConfigToRuntime(rememberedConfig);
|
||||
void selectModel({
|
||||
id: runId,
|
||||
ggufVariant: opts.ggufVariant,
|
||||
isDownloaded,
|
||||
expectedBytes: opts.expectedBytes,
|
||||
keepSpeculative: remembered != null,
|
||||
keepSpeculative: hasAppliedConfig,
|
||||
throwOnError: true,
|
||||
previousConfig,
|
||||
})
|
||||
.then(() => {
|
||||
// Read fresh: the load is async, so the checkpoint may have changed.
|
||||
|
|
@ -1280,6 +1271,7 @@ export function ModelsPage() {
|
|||
onLoad: handleLoad,
|
||||
onLoadLocal: handleLoadLocal,
|
||||
onUseInChat: openNewChat,
|
||||
onEject: () => void ejectModel(),
|
||||
onTrain: handleTrain,
|
||||
onInventoryChange: refreshInventory,
|
||||
onSearchHub: handleSearchHub,
|
||||
|
|
@ -1288,6 +1280,7 @@ export function ModelsPage() {
|
|||
handleLoad,
|
||||
handleLoadLocal,
|
||||
openNewChat,
|
||||
ejectModel,
|
||||
handleTrain,
|
||||
handleSearchHub,
|
||||
refreshInventory,
|
||||
|
|
@ -1295,31 +1288,38 @@ export function ModelsPage() {
|
|||
);
|
||||
|
||||
const catalogState = useMemo<ModelsCatalogState>(
|
||||
() => ({
|
||||
tab,
|
||||
discoverRows: listRows,
|
||||
cachedRows: filteredCachedRows,
|
||||
localRows: filteredLocalRows,
|
||||
selectedId,
|
||||
isLoading,
|
||||
downloadedReady,
|
||||
inventoryError,
|
||||
inventoryWarning,
|
||||
query,
|
||||
activeCheckpoint,
|
||||
activeGgufVariant,
|
||||
searchError,
|
||||
online,
|
||||
isDataset: isDatasetMode,
|
||||
inventoryTokens,
|
||||
scannedCount,
|
||||
loadingIntentCount: discoverFetchIntent,
|
||||
hasMore,
|
||||
manualFetchAvailable: discoverManualFetchAvailable,
|
||||
hasActiveFilters:
|
||||
!isFeedMode &&
|
||||
(deferredFormatFilter !== "all" || deferredCapabilityFilter !== "all"),
|
||||
}),
|
||||
() => {
|
||||
const typeFilterActive =
|
||||
!isDatasetMode && inventoryTypeFilter !== "all";
|
||||
return {
|
||||
tab,
|
||||
discoverRows: listRows,
|
||||
cachedRows: filteredCachedRows,
|
||||
localRows: filteredLocalRows,
|
||||
selectedId,
|
||||
isLoading,
|
||||
downloadedReady,
|
||||
inventoryError,
|
||||
inventoryWarning,
|
||||
query,
|
||||
activeCheckpoint,
|
||||
activeGgufVariant,
|
||||
searchError,
|
||||
online,
|
||||
isDataset: isDatasetMode,
|
||||
inventoryTokens,
|
||||
scannedCount,
|
||||
loadingIntentCount: discoverFetchIntent,
|
||||
hasMore,
|
||||
manualFetchAvailable: discoverManualFetchAvailable,
|
||||
hasActiveFilters:
|
||||
!isFeedMode &&
|
||||
(deferredFormatFilter !== "all" ||
|
||||
deferredCapabilityFilter !== "all" ||
|
||||
(tab === "downloaded" && typeFilterActive)),
|
||||
typeFilterActive,
|
||||
};
|
||||
},
|
||||
[
|
||||
tab,
|
||||
isFeedMode,
|
||||
|
|
@ -1344,6 +1344,7 @@ export function ModelsPage() {
|
|||
discoverManualFetchAvailable,
|
||||
deferredFormatFilter,
|
||||
deferredCapabilityFilter,
|
||||
inventoryTypeFilter,
|
||||
],
|
||||
);
|
||||
|
||||
|
|
@ -1422,16 +1423,18 @@ export function ModelsPage() {
|
|||
</div>
|
||||
);
|
||||
}
|
||||
const ownerToggle = !isDatasetMode ? (
|
||||
const ownerToggle = isDatasetMode ? undefined : (
|
||||
<OwnerScopeToggle value={ownerScope} onChange={setOwnerScope} />
|
||||
) : undefined;
|
||||
);
|
||||
// Compact pill so it stays beside the view-mode tabs even in the narrow
|
||||
// split pane instead of dropping to its own row.
|
||||
return (
|
||||
<div className="flex flex-col gap-3 pt-6">
|
||||
{isChannelListMode ? (
|
||||
<HubListHeader
|
||||
title={channelSection ? HUB_SECTION_TITLE[channelSection] : "Models"}
|
||||
title={
|
||||
channelSection ? HUB_SECTION_TITLE[channelSection] : "Models"
|
||||
}
|
||||
count={listCount}
|
||||
view={allModelsView}
|
||||
onViewChange={setAllModelsView}
|
||||
|
|
@ -1474,27 +1477,40 @@ export function ModelsPage() {
|
|||
]);
|
||||
|
||||
const downloadedHeader = useMemo(() => {
|
||||
const sortControl = (
|
||||
<InventorySortControl value={inventorySort} onChange={setInventorySort} />
|
||||
// Compact pills so they stay beside the view-mode tabs even in the narrow
|
||||
// split pane instead of dropping to their own row.
|
||||
const controls = (
|
||||
<div className="flex min-w-0 items-center gap-1.5">
|
||||
{!isDatasetMode && (
|
||||
<InventoryTypeFilterControl
|
||||
value={inventoryTypeFilter}
|
||||
onChange={setInventoryTypeFilter}
|
||||
/>
|
||||
)}
|
||||
<InventorySortControl
|
||||
value={inventorySort}
|
||||
onChange={setInventorySort}
|
||||
/>
|
||||
</div>
|
||||
);
|
||||
// Compact pill so it stays beside the view-mode tabs even in the narrow
|
||||
// split pane instead of dropping to its own row.
|
||||
return (
|
||||
<HubListHeader
|
||||
title="On device"
|
||||
count={visibleCachedCount + visibleLocalCount}
|
||||
count={filteredCachedRows.length + filteredLocalRows.length}
|
||||
view={allModelsView}
|
||||
onViewChange={setAllModelsView}
|
||||
actions={sortControl}
|
||||
actions={controls}
|
||||
/>
|
||||
);
|
||||
}, [
|
||||
visibleCachedCount,
|
||||
visibleLocalCount,
|
||||
filteredCachedRows,
|
||||
filteredLocalRows,
|
||||
allModelsView,
|
||||
setAllModelsView,
|
||||
inventorySort,
|
||||
setInventorySort,
|
||||
inventoryTypeFilter,
|
||||
isDatasetMode,
|
||||
]);
|
||||
|
||||
const detailOpen = urlModel !== null;
|
||||
|
|
|
|||
|
|
@ -1,10 +1,57 @@
|
|||
// SPDX-License-Identifier: AGPL-3.0-only
|
||||
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
export { cancelStagedModelDownload } from "./download-manager";
|
||||
export {
|
||||
downloadManager,
|
||||
jobKeyOf,
|
||||
subscribeJobListeners,
|
||||
useDownloadManagerStore,
|
||||
} from "./download-manager";
|
||||
export {
|
||||
useHubInventory,
|
||||
type CachedInventoryRow,
|
||||
type GgufVariantDetail,
|
||||
type LocalInventoryRow,
|
||||
type LocalSource,
|
||||
type ScanFolderInfo,
|
||||
addScanFolder,
|
||||
deleteCachedModel,
|
||||
invalidateGgufVariantsCache,
|
||||
listGgufVariants,
|
||||
listScanFolders,
|
||||
removeScanFolder,
|
||||
} from "./inventory";
|
||||
export {
|
||||
type HfModelResult,
|
||||
type HfSortKey,
|
||||
useHubModelSearch,
|
||||
} from "./hooks/use-hub-model-search";
|
||||
export { useOnlineStatus } from "./hooks/use-online-status";
|
||||
export { useHubInfiniteScroll } from "./hooks/use-hub-infinite-scroll";
|
||||
export { bumpInventoryVersion } from "./stores/inventory-events";
|
||||
export {
|
||||
getHfToken,
|
||||
hfApiToken,
|
||||
mirrorHfTokenInto,
|
||||
useHfTokenStore,
|
||||
} from "./stores/hf-token-store";
|
||||
export { useInventoryVersion } from "./stores/inventory-events";
|
||||
export { looksLikeLocalPath } from "./lib/local-path";
|
||||
export { hubTokenHeader } from "./lib/hub-token-header";
|
||||
export {
|
||||
ggufVariantsMatch,
|
||||
normalizeGgufVariantIdentity,
|
||||
normalizeModelIdentity,
|
||||
} from "./lib/model-identity";
|
||||
export { formatBytes, formatRelativeShort } from "./lib/format";
|
||||
export { ggufVariantDisplayLabel } from "./lib/gguf-variant-sort";
|
||||
export {
|
||||
DeleteConfirmDialog,
|
||||
UpdateConfirmDialog,
|
||||
} from "./catalog/download-card";
|
||||
export { HubOptionMenu, type HubOption } from "./catalog/hub-option-menu";
|
||||
export { DotTag } from "./catalog/dot-tag";
|
||||
export { TransportConflictDialog } from "./catalog/transport-conflict-dialog";
|
||||
export { TrainIcon } from "./components/train-icon";
|
||||
export { isHiddenModelId } from "./lib/hidden-models";
|
||||
export { classifyUnslothSupport } from "./lib/unsloth-support";
|
||||
|
|
|
|||
|
|
@ -48,6 +48,7 @@ export interface CachedGgufRepo {
|
|||
capabilities?: BackendModelCapabilities | null;
|
||||
size_bytes: number;
|
||||
cache_path?: string;
|
||||
last_modified?: number | null;
|
||||
partial?: boolean;
|
||||
partial_transport?: string | null;
|
||||
pipeline_tag?: string | null;
|
||||
|
|
@ -65,6 +66,7 @@ export interface CachedModelRepo {
|
|||
capabilities?: BackendModelCapabilities | null;
|
||||
size_bytes: number;
|
||||
cache_path?: string;
|
||||
last_modified?: number | null;
|
||||
partial?: boolean;
|
||||
partial_transport?: string | null;
|
||||
pipeline_tag?: string | null;
|
||||
|
|
|
|||
|
|
@ -47,6 +47,7 @@ export interface CachedInventoryRow {
|
|||
capabilities: ModelInventoryCapabilities;
|
||||
bytes: number;
|
||||
cachePath?: string | null;
|
||||
lastModified?: number | null;
|
||||
partial?: boolean;
|
||||
partialTransport?: string | null;
|
||||
pipelineTag?: string | null;
|
||||
|
|
@ -66,6 +67,8 @@ export interface LocalInventoryRow {
|
|||
title: string;
|
||||
source: LocalSource;
|
||||
sourceLabel: string;
|
||||
modelId?: string | null;
|
||||
displayName?: string;
|
||||
path: string;
|
||||
isGguf: boolean;
|
||||
modelFormat: ModelInventoryFormat;
|
||||
|
|
|
|||
|
|
@ -14,6 +14,7 @@ import {
|
|||
listLocalModels,
|
||||
} from "./api";
|
||||
import { useDebouncedValue } from "@/hooks/use-debounced-value";
|
||||
import { ensureHiddenModelMatchers } from "../lib/hidden-models";
|
||||
import { fingerprintToken } from "@/features/hub/lib/token-fingerprint";
|
||||
import { useInventoryVersion } from "@/features/hub/stores/inventory-events";
|
||||
import { useCallback, useEffect, useMemo } from "react";
|
||||
|
|
@ -146,12 +147,15 @@ async function runSourceFetch<K extends DeviceInventorySource>(
|
|||
): Promise<DeviceInventoryRows[K]> {
|
||||
switch (source) {
|
||||
case "cachedGguf":
|
||||
await ensureHiddenModelMatchers();
|
||||
return (await listCachedGguf(hfToken)) as DeviceInventoryRows[K];
|
||||
case "cachedModels":
|
||||
await ensureHiddenModelMatchers();
|
||||
return (await listCachedModels(hfToken)) as DeviceInventoryRows[K];
|
||||
case "cachedDatasets":
|
||||
return (await listCachedDatasets()) as DeviceInventoryRows[K];
|
||||
case "localModels":
|
||||
await ensureHiddenModelMatchers();
|
||||
return (await listLocalModels()).models as DeviceInventoryRows[K];
|
||||
case "localDatasets":
|
||||
return (await listLocalDatasets()).datasets as DeviceInventoryRows[K];
|
||||
|
|
|
|||
|
|
@ -176,6 +176,7 @@ export function buildCachedInventoryRow(
|
|||
runtime?: string | null;
|
||||
format_variant?: string | null;
|
||||
capabilities?: BackendModelCapabilities | null;
|
||||
last_modified?: number | null;
|
||||
optimistic?: boolean;
|
||||
},
|
||||
fallbackFormat: ModelInventoryFormat,
|
||||
|
|
@ -215,6 +216,12 @@ export function buildCachedInventoryRow(
|
|||
capabilities,
|
||||
bytes: row.size_bytes,
|
||||
cachePath: row.cache_path ?? null,
|
||||
lastModified:
|
||||
typeof row.last_modified === "number" &&
|
||||
Number.isFinite(row.last_modified) &&
|
||||
row.last_modified > 0
|
||||
? row.last_modified
|
||||
: null,
|
||||
partial: row.partial ?? false,
|
||||
partialTransport: row.partial_transport ?? null,
|
||||
pipelineTag: row.pipeline_tag ?? null,
|
||||
|
|
@ -278,6 +285,8 @@ export function buildLocalInventoryRows(
|
|||
title,
|
||||
source: model.source,
|
||||
sourceLabel: localSourceLabel(model.source),
|
||||
modelId: model.model_id ?? null,
|
||||
displayName: model.display_name,
|
||||
path: model.path,
|
||||
isGguf: modelFormat === "gguf",
|
||||
modelFormat,
|
||||
|
|
|
|||
|
|
@ -1,19 +1,77 @@
|
|||
// SPDX-License-Identifier: AGPL-3.0-only
|
||||
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
import { authFetch } from "@/features/auth";
|
||||
import { getInventoryVersion } from "../stores/inventory-events";
|
||||
|
||||
// Infra models hidden from browse/preview lists (Hub Discover, the chat model
|
||||
// selector, and local on-device rows). Mirrors the backend
|
||||
// `utils.hidden_models`: the RAG embedding model and the llama.cpp validation
|
||||
// probe are not usable chat models. Server-confirmed cache rows are trusted
|
||||
// because the backend applies variant-aware filtering. Optimistic cache rows
|
||||
// still use these needles until the server confirms them. Per-repo views are
|
||||
// not filtered, so reinstall flows still show downloaded files.
|
||||
// still use these needles until the server confirms them. The dynamic matchers
|
||||
// fetched from `/api/hub/hidden-models` add the user's configured embedder as
|
||||
// exact repo ids and exact resolved paths, never substring needles. Per-repo
|
||||
// views are not filtered, so reinstall flows still show downloaded files.
|
||||
const HIDDEN_NEEDLES = [
|
||||
"bge-small-en-v1.5", // RAG embedder: unsloth/bge-small-en-v1.5[-GGUF]
|
||||
"ggml-org/models", // llama.cpp validation probe repo
|
||||
"stories260k.gguf", // probe filename (carries .gguf so it stays specific)
|
||||
];
|
||||
|
||||
let dynamicNeedles: readonly string[] = [];
|
||||
let dynamicExactIds: readonly string[] = [];
|
||||
let dynamicExactPaths: readonly string[] = [];
|
||||
let matchersFetch: Promise<void> | null = null;
|
||||
let matchersFetchVersion = -1;
|
||||
|
||||
function toLowerStrings(value: unknown): string[] {
|
||||
if (!Array.isArray(value)) {
|
||||
return [];
|
||||
}
|
||||
return value
|
||||
.filter((v): v is string => typeof v === "string" && v.length > 0)
|
||||
.map((v) => v.toLowerCase());
|
||||
}
|
||||
|
||||
export function ensureHiddenModelMatchers(): Promise<void> {
|
||||
const version = getInventoryVersion();
|
||||
if (matchersFetch && matchersFetchVersion === version) {
|
||||
return matchersFetch;
|
||||
}
|
||||
matchersFetchVersion = version;
|
||||
matchersFetch = (async () => {
|
||||
try {
|
||||
const response = await authFetch("/api/hub/hidden-models");
|
||||
if (!response.ok) {
|
||||
throw new Error(`HTTP ${response.status}`);
|
||||
}
|
||||
const data = (await response.json()) as {
|
||||
needles?: unknown;
|
||||
exact_ids?: unknown;
|
||||
exact_paths?: unknown;
|
||||
};
|
||||
if (
|
||||
getInventoryVersion() !== version ||
|
||||
matchersFetchVersion !== version
|
||||
) {
|
||||
return;
|
||||
}
|
||||
dynamicNeedles = toLowerStrings(data.needles);
|
||||
dynamicExactIds = toLowerStrings(data.exact_ids);
|
||||
dynamicExactPaths = toLowerStrings(data.exact_paths);
|
||||
} catch {
|
||||
if (
|
||||
getInventoryVersion() === version &&
|
||||
matchersFetchVersion === version
|
||||
) {
|
||||
matchersFetch = null;
|
||||
}
|
||||
}
|
||||
})();
|
||||
return matchersFetch;
|
||||
}
|
||||
|
||||
/** True if any id/path is a hidden infra model. */
|
||||
export function isHiddenModelId(
|
||||
...values: (string | null | undefined)[]
|
||||
|
|
@ -23,7 +81,12 @@ export function isHiddenModelId(
|
|||
return false;
|
||||
}
|
||||
const lower = v.toLowerCase();
|
||||
return HIDDEN_NEEDLES.some((needle) => lower.includes(needle));
|
||||
return (
|
||||
HIDDEN_NEEDLES.some((needle) => lower.includes(needle)) ||
|
||||
dynamicNeedles.some((needle) => lower.includes(needle)) ||
|
||||
dynamicExactIds.includes(lower) ||
|
||||
dynamicExactPaths.includes(lower)
|
||||
);
|
||||
});
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -10,6 +10,7 @@ export type CapabilityKey =
|
|||
| "reasoning"
|
||||
| "code"
|
||||
| "embedding"
|
||||
| "diffusion"
|
||||
| "multilingual"
|
||||
| "conversational";
|
||||
|
||||
|
|
@ -62,6 +63,20 @@ const REASONING_TAGS = new Set([
|
|||
"step-by-step",
|
||||
]);
|
||||
|
||||
// Image generation / diffusion (surfaced as "Image generation" in filters).
|
||||
const DIFFUSION_TAGS = new Set([
|
||||
"diffusers",
|
||||
"diffusion",
|
||||
"stable-diffusion",
|
||||
"latent-diffusion",
|
||||
"flux",
|
||||
"text-to-image",
|
||||
"image-to-image",
|
||||
"text-to-video",
|
||||
"image-to-video",
|
||||
"unconditional-image-generation",
|
||||
]);
|
||||
|
||||
const CODE_TAGS = new Set([
|
||||
"code",
|
||||
"code-generation",
|
||||
|
|
@ -186,10 +201,7 @@ export function detectCapabilities(
|
|||
) {
|
||||
out.push({ key: "code", label: "Code" });
|
||||
}
|
||||
if (
|
||||
hasAny(CONVERSATIONAL_TAGS) ||
|
||||
CONVERSATIONAL_ID_RE.test(lowerId)
|
||||
) {
|
||||
if (hasAny(CONVERSATIONAL_TAGS) || CONVERSATIONAL_ID_RE.test(lowerId)) {
|
||||
out.push({ key: "conversational", label: "Conversational" });
|
||||
}
|
||||
if (
|
||||
|
|
@ -200,6 +212,14 @@ export function detectCapabilities(
|
|||
) {
|
||||
out.push({ key: "embedding", label: "Embeddings" });
|
||||
}
|
||||
if (
|
||||
hasAny(DIFFUSION_TAGS) ||
|
||||
/stable[-_]?diffusion|\bsdxl\b|\bflux\b|qwen[-_]?image|hunyuan[-_]?(?:video|image)|wan2|latent[-_]?consistency|[-_]lcm\b|dreamshaper/.test(
|
||||
lowerId,
|
||||
)
|
||||
) {
|
||||
out.push({ key: "diffusion", label: "Image generation" });
|
||||
}
|
||||
const languageCodes = new Set<string>();
|
||||
for (const tag of tags ?? []) {
|
||||
const lower = tag.toLowerCase();
|
||||
|
|
|
|||
52
studio/frontend/src/features/hub/lib/model-type-filter.ts
Normal file
52
studio/frontend/src/features/hub/lib/model-type-filter.ts
Normal file
|
|
@ -0,0 +1,52 @@
|
|||
// SPDX-License-Identifier: AGPL-3.0-only
|
||||
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
// Type filter for the On Device list. Mirrors the hub Discover capability
|
||||
// options and shares its detection, so both dropdowns behave the same.
|
||||
|
||||
import type {
|
||||
CachedInventoryRow,
|
||||
LocalInventoryRow,
|
||||
} from "@/features/hub/inventory/types";
|
||||
import { type CapabilityKey, detectCapabilities } from "./model-capabilities";
|
||||
|
||||
export type ModelTypeFilter =
|
||||
| "all"
|
||||
| "reasoning"
|
||||
| "vision"
|
||||
| "audio"
|
||||
| "embedding"
|
||||
| "diffusion";
|
||||
|
||||
export const MODEL_TYPE_FILTER_OPTIONS: ReadonlyArray<{
|
||||
value: ModelTypeFilter;
|
||||
label: string;
|
||||
}> = [
|
||||
{ value: "all", label: "All types" },
|
||||
{ value: "reasoning", label: "Reasoning" },
|
||||
{ value: "vision", label: "Vision" },
|
||||
{ value: "audio", label: "Audio" },
|
||||
{ value: "embedding", label: "Embeddings" },
|
||||
{ value: "diffusion", label: "Image generation" },
|
||||
];
|
||||
|
||||
function rowName(row: CachedInventoryRow | LocalInventoryRow): string {
|
||||
return row.kind === "local"
|
||||
? `${row.id} ${row.repoId ?? ""} ${row.title} ${row.modelId ?? ""}`
|
||||
: `${row.id} ${row.repoId}`;
|
||||
}
|
||||
|
||||
export function matchesModelType(
|
||||
row: CachedInventoryRow | LocalInventoryRow,
|
||||
filter: ModelTypeFilter,
|
||||
): boolean {
|
||||
if (filter === "all") return true;
|
||||
// Honor the row's own vision flag before falling back to tag detection.
|
||||
if (filter === "vision" && row.capabilities.supportsVision) return true;
|
||||
const caps = detectCapabilities(
|
||||
row.tags ?? undefined,
|
||||
row.pipelineTag ?? undefined,
|
||||
rowName(row),
|
||||
);
|
||||
return caps.some((cap: { key: CapabilityKey }) => cap.key === filter);
|
||||
}
|
||||
|
|
@ -7,18 +7,18 @@ import type {
|
|||
CachedInventoryRow,
|
||||
LocalInventoryRow,
|
||||
} from "@/features/hub/inventory/types";
|
||||
import { ownerOf, repoOf } from "@/features/hub/lib/format";
|
||||
import type {
|
||||
CapabilityFilter,
|
||||
DiscoverRow,
|
||||
ModelFormatFilter,
|
||||
} from "../types";
|
||||
import { estimateSizeFromDtypes, isGgufLike } from "./hf-model-meta";
|
||||
import {
|
||||
type CapabilityKey,
|
||||
detectBaseModel,
|
||||
detectCapabilities,
|
||||
type CapabilityKey,
|
||||
} from "./model-capabilities";
|
||||
import { ownerOf, repoOf } from "@/features/hub/lib/format";
|
||||
import { estimateSizeFromDtypes, isGgufLike } from "./hf-model-meta";
|
||||
export {
|
||||
detectResultFormat,
|
||||
isUnslothFinetunable,
|
||||
|
|
@ -39,6 +39,7 @@ export const CAPABILITY_FILTER_OPTIONS: ReadonlyArray<{
|
|||
{ value: "vision", label: "Vision" },
|
||||
{ value: "audio", label: "Audio" },
|
||||
{ value: "embedding", label: "Embeddings" },
|
||||
{ value: "diffusion", label: "Image generation" },
|
||||
];
|
||||
|
||||
export const FORMAT_FILTER_OPTIONS: ReadonlyArray<{
|
||||
|
|
|
|||
|
|
@ -0,0 +1,20 @@
|
|||
// SPDX-License-Identifier: AGPL-3.0-only
|
||||
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
import { getModelConfig } from "@/features/training";
|
||||
|
||||
export async function fetchModelMaxPositionEmbeddings(
|
||||
modelName: string,
|
||||
hfToken?: string | null,
|
||||
signal?: AbortSignal,
|
||||
): Promise<number | null> {
|
||||
const config = await getModelConfig(
|
||||
modelName,
|
||||
signal,
|
||||
hfToken?.trim() || undefined,
|
||||
);
|
||||
const value = config.max_position_embeddings;
|
||||
return typeof value === "number" && Number.isFinite(value) && value > 0
|
||||
? Math.floor(value)
|
||||
: null;
|
||||
}
|
||||
87
studio/frontend/src/features/model-picker/api/templates.ts
Normal file
87
studio/frontend/src/features/model-picker/api/templates.ts
Normal file
|
|
@ -0,0 +1,87 @@
|
|||
// SPDX-License-Identifier: AGPL-3.0-only
|
||||
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
import { authFetch } from "@/features/auth";
|
||||
import { hubTokenHeader } from "@/features/hub";
|
||||
import { consumeNativePathToken } from "@/features/native-intents/api";
|
||||
import { readFastApiError } from "@/lib/format-fastapi-error";
|
||||
|
||||
export interface ValidateChatTemplateResult {
|
||||
valid: boolean;
|
||||
error: string | null;
|
||||
}
|
||||
|
||||
async function parseJsonOrThrow<T>(response: Response): Promise<T> {
|
||||
if (!response.ok) {
|
||||
throw new Error(await readFastApiError(response));
|
||||
}
|
||||
return response.json();
|
||||
}
|
||||
|
||||
export async function validateChatTemplate(
|
||||
template: string,
|
||||
signal?: AbortSignal,
|
||||
): Promise<ValidateChatTemplateResult> {
|
||||
const response = await authFetch("/api/picker/validate-chat-template", {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ template }),
|
||||
signal,
|
||||
});
|
||||
return parseJsonOrThrow<ValidateChatTemplateResult>(response);
|
||||
}
|
||||
|
||||
export async function fetchDefaultChatTemplate(
|
||||
modelName: string,
|
||||
ggufVariant?: string | null,
|
||||
hfToken?: string | null,
|
||||
signal?: AbortSignal,
|
||||
nativePathToken?: string | null,
|
||||
): Promise<string | null> {
|
||||
// A native (picked / drag-drop) GGUF lives at a path only its signed lease
|
||||
// knows, and the picker chat-template GET has no lease plumbing, so redeem a
|
||||
// one-shot validate-model lease and read the embedded template through the
|
||||
// lease-aware /api/inference/validate probe instead (mirrors the staged
|
||||
// header-dims fetch). Non-native models keep the plain GET path.
|
||||
if (nativePathToken) {
|
||||
let nativePathLease: string | null = null;
|
||||
try {
|
||||
nativePathLease = (
|
||||
await consumeNativePathToken(nativePathToken, "validate-model")
|
||||
).nativePathLease;
|
||||
} catch {
|
||||
// Lease expired / revoked: no readable path, so no default template (the
|
||||
// subsequent load re-mints its own lease).
|
||||
return null;
|
||||
}
|
||||
const response = await authFetch("/api/inference/validate", {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({
|
||||
model_path: modelName,
|
||||
gguf_variant: ggufVariant ?? null,
|
||||
hf_token: hfToken ?? null,
|
||||
native_path_lease: nativePathLease,
|
||||
include_chat_template: true,
|
||||
}),
|
||||
signal,
|
||||
});
|
||||
const data = await parseJsonOrThrow<{ chat_template?: string | null }>(
|
||||
response,
|
||||
);
|
||||
return data.chat_template ?? null;
|
||||
}
|
||||
|
||||
const query = ggufVariant
|
||||
? `?gguf_variant=${encodeURIComponent(ggufVariant)}`
|
||||
: "";
|
||||
const response = await authFetch(
|
||||
`/api/picker/chat-template/${encodeURIComponent(modelName)}${query}`,
|
||||
{ headers: hubTokenHeader(hfToken), signal },
|
||||
);
|
||||
const data = await parseJsonOrThrow<{
|
||||
model_name: string;
|
||||
chat_template: string | null;
|
||||
}>(response);
|
||||
return data.chat_template ?? null;
|
||||
}
|
||||
|
|
@ -0,0 +1,191 @@
|
|||
// SPDX-License-Identifier: AGPL-3.0-only
|
||||
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
import { Button } from "@/components/ui/button";
|
||||
import {
|
||||
Dialog,
|
||||
DialogContent,
|
||||
DialogDescription,
|
||||
DialogFooter,
|
||||
DialogHeader,
|
||||
DialogTitle,
|
||||
} from "@/components/ui/dialog";
|
||||
import { Spinner } from "@/components/ui/spinner";
|
||||
import { Textarea } from "@/components/ui/textarea";
|
||||
import { useRef, useState } from "react";
|
||||
import { validateChatTemplate } from "../api/templates";
|
||||
import {
|
||||
MAX_CHAT_TEMPLATE_BYTES,
|
||||
chatTemplateByteLength,
|
||||
isChatTemplateWithinLimit,
|
||||
} from "../model-config/per-model-config";
|
||||
|
||||
interface ChatTemplateEditorDialogProps {
|
||||
open: boolean;
|
||||
onOpenChange: (open: boolean) => void;
|
||||
value: string | null;
|
||||
defaultTemplate: string | null;
|
||||
defaultLoading: boolean;
|
||||
onSave: (override: string | null) => void;
|
||||
readOnly?: boolean;
|
||||
}
|
||||
|
||||
export function ChatTemplateEditorDialog({
|
||||
open,
|
||||
onOpenChange,
|
||||
value,
|
||||
defaultTemplate,
|
||||
defaultLoading,
|
||||
onSave,
|
||||
readOnly = false,
|
||||
}: ChatTemplateEditorDialogProps) {
|
||||
const [draft, setDraft] = useState<string | null>(null);
|
||||
const [error, setError] = useState<string | null>(null);
|
||||
const [validating, setValidating] = useState(false);
|
||||
// Bumped whenever the dialog closes so a validation still in flight cannot
|
||||
// apply a template the user has already dismissed.
|
||||
const validationToken = useRef(0);
|
||||
const renderedDraft = draft ?? value ?? defaultTemplate ?? "";
|
||||
|
||||
const byteLength = chatTemplateByteLength(renderedDraft);
|
||||
const overLimit = !isChatTemplateWithinLimit(renderedDraft);
|
||||
const matchesDefault =
|
||||
defaultTemplate != null && renderedDraft === defaultTemplate;
|
||||
|
||||
const handleClose = () => {
|
||||
validationToken.current += 1;
|
||||
setDraft(null);
|
||||
setError(null);
|
||||
setValidating(false);
|
||||
onOpenChange(false);
|
||||
};
|
||||
|
||||
const handleSave = async () => {
|
||||
if (renderedDraft.trim().length === 0 || matchesDefault) {
|
||||
onSave(null);
|
||||
handleClose();
|
||||
return;
|
||||
}
|
||||
if (overLimit) {
|
||||
setError("Template exceeds the size limit.");
|
||||
return;
|
||||
}
|
||||
setValidating(true);
|
||||
const token = validationToken.current;
|
||||
try {
|
||||
const result = await validateChatTemplate(renderedDraft);
|
||||
// Dialog was closed (or reopened) while validating; drop the result so a
|
||||
// discarded template is never applied.
|
||||
if (token !== validationToken.current) {
|
||||
return;
|
||||
}
|
||||
if (!result.valid) {
|
||||
setError(result.error ?? "Invalid Jinja template.");
|
||||
return;
|
||||
}
|
||||
onSave(renderedDraft);
|
||||
handleClose();
|
||||
} catch {
|
||||
if (token === validationToken.current) {
|
||||
setError("Could not validate the template.");
|
||||
}
|
||||
} finally {
|
||||
if (token === validationToken.current) {
|
||||
setValidating(false);
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
return (
|
||||
<Dialog
|
||||
open={open}
|
||||
onOpenChange={(nextOpen) => {
|
||||
if (nextOpen) {
|
||||
onOpenChange(true);
|
||||
return;
|
||||
}
|
||||
handleClose();
|
||||
}}
|
||||
>
|
||||
<DialogContent className="corner-squircle dialog-soft-surface sm:max-w-3xl">
|
||||
<DialogHeader>
|
||||
<DialogTitle>
|
||||
{readOnly ? "Chat Template" : "Edit Chat Template"}
|
||||
</DialogTitle>
|
||||
<DialogDescription>
|
||||
{readOnly
|
||||
? "This is the model's chat template. Custom templates apply to GGUF models for now, so it is view only for safetensors models."
|
||||
: "Override the model's chat template with custom Jinja. The change applies when the model loads. Saving an empty template or one that matches the default clears the override."}
|
||||
</DialogDescription>
|
||||
</DialogHeader>
|
||||
<Textarea
|
||||
value={renderedDraft}
|
||||
onChange={(event) => {
|
||||
if (readOnly) return;
|
||||
setDraft(event.target.value);
|
||||
setError(null);
|
||||
}}
|
||||
readOnly={readOnly}
|
||||
className="min-h-[20rem] max-h-[50vh] overflow-y-auto border-0 font-mono text-xs leading-5 corner-squircle focus-visible:ring-0"
|
||||
rows={14}
|
||||
spellCheck={false}
|
||||
placeholder={defaultLoading ? "Loading model default..." : ""}
|
||||
/>
|
||||
{readOnly ? null : (
|
||||
<div className="flex items-center justify-between gap-3 px-0.5 text-[11px]">
|
||||
<span
|
||||
className={overLimit ? "text-amber-500" : "text-muted-foreground"}
|
||||
>
|
||||
{byteLength.toLocaleString()} /{" "}
|
||||
{MAX_CHAT_TEMPLATE_BYTES.toLocaleString()} bytes
|
||||
</span>
|
||||
{error ? (
|
||||
<span className="truncate text-red-500" title={error}>
|
||||
{error}
|
||||
</span>
|
||||
) : null}
|
||||
</div>
|
||||
)}
|
||||
<DialogFooter className="flex-wrap gap-2 sm:justify-between">
|
||||
{readOnly ? (
|
||||
<div className="flex w-full justify-end">
|
||||
<Button type="button" onClick={handleClose}>
|
||||
Close
|
||||
</Button>
|
||||
</div>
|
||||
) : (
|
||||
<>
|
||||
<Button
|
||||
type="button"
|
||||
variant="ghost"
|
||||
onClick={() => setDraft(defaultTemplate ?? "")}
|
||||
disabled={
|
||||
defaultLoading || renderedDraft === (defaultTemplate ?? "")
|
||||
}
|
||||
className="text-muted-foreground"
|
||||
>
|
||||
{defaultLoading ? (
|
||||
<Spinner className="size-3.5" />
|
||||
) : (
|
||||
"Reset to default"
|
||||
)}
|
||||
</Button>
|
||||
<div className="flex gap-2">
|
||||
<Button type="button" variant="ghost" onClick={handleClose}>
|
||||
Cancel
|
||||
</Button>
|
||||
<Button
|
||||
type="button"
|
||||
onClick={handleSave}
|
||||
disabled={validating || overLimit}
|
||||
>
|
||||
{validating ? "Validating..." : "Save"}
|
||||
</Button>
|
||||
</div>
|
||||
</>
|
||||
)}
|
||||
</DialogFooter>
|
||||
</DialogContent>
|
||||
</Dialog>
|
||||
);
|
||||
}
|
||||
|
|
@ -0,0 +1,991 @@
|
|||
// SPDX-License-Identifier: AGPL-3.0-only
|
||||
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
import { Button } from "@/components/ui/button";
|
||||
import { Checkbox } from "@/components/ui/checkbox";
|
||||
import { InfoHint } from "@/components/ui/info-hint";
|
||||
import {
|
||||
Select,
|
||||
SelectContent,
|
||||
SelectItem,
|
||||
SelectTrigger,
|
||||
SelectValue,
|
||||
} from "@/components/ui/select";
|
||||
import { Slider } from "@/components/ui/slider";
|
||||
import { Switch } from "@/components/ui/switch";
|
||||
import {
|
||||
GPU_LAYERS_AUTO,
|
||||
fetchGgufStagedMetadata,
|
||||
readPersistedSpeculativeType,
|
||||
useChatRuntimeStore,
|
||||
} from "@/features/chat";
|
||||
import { useGpuDevices } from "@/hooks/use-gpu-info";
|
||||
import { ChevronDownStandardIcon } from "@/lib/chevron-icons";
|
||||
import { toast } from "@/lib/toast";
|
||||
import { ArrowLeft01Icon } from "@hugeicons/core-free-icons";
|
||||
import { HugeiconsIcon } from "@hugeicons/react";
|
||||
import { type ReactNode, useEffect, useId, useState } from "react";
|
||||
import {
|
||||
useDefaultChatTemplate,
|
||||
useModelMaxPositionEmbeddings,
|
||||
} from "../hooks/use-model-defaults";
|
||||
import { perModelConfigsEqual } from "../model-config/apply-per-model-config";
|
||||
import {
|
||||
CONTEXT_LENGTH_MIN,
|
||||
DEFAULT_MAX_SEQ_LENGTH,
|
||||
DEFAULT_PER_MODEL_CONFIG,
|
||||
KV_CACHE_DTYPES,
|
||||
MAX_SEQ_LENGTH_MAX,
|
||||
MAX_SEQ_LENGTH_MIN,
|
||||
MAX_SEQ_LENGTH_STEP,
|
||||
MTP_SPECULATIVE_TYPES,
|
||||
type PerModelConfig,
|
||||
SPECULATIVE_TYPES,
|
||||
deletePerModelConfig,
|
||||
floorMaxSeqLength,
|
||||
isDefaultConfig,
|
||||
normalizeMaxSeqLength,
|
||||
resolveInitialConfig,
|
||||
savePerModelConfig,
|
||||
} from "../model-config/per-model-config";
|
||||
import { ChatTemplateEditorDialog } from "./chat-template-editor-dialog";
|
||||
import type { ModelPickTarget } from "./model-selector/types";
|
||||
import { NumericValueInput } from "./numeric-value-input";
|
||||
|
||||
const ROW_CLASS = "flex min-h-8 items-center justify-between gap-3";
|
||||
const LABEL_CLASS =
|
||||
"min-w-0 truncate text-[13px] font-medium leading-[1.25] tracking-nav text-nav-fg";
|
||||
const LABEL_CLASS_WRAP =
|
||||
"min-w-0 text-[13px] font-medium leading-[1.25] tracking-nav text-nav-fg";
|
||||
const CONTROL_SURFACE =
|
||||
"rounded-full border-transparent bg-black/[0.04] dark:bg-white/[0.05] hover:bg-black/[0.06] dark:hover:bg-white/[0.1]";
|
||||
const SELECT_TRIGGER_CLASS = `grid h-8 min-w-0 grid-cols-[minmax(0,1fr)_auto] items-center gap-1 ${CONTROL_SURFACE} pl-3 pr-2 py-0 text-[13px]! font-medium text-nav-fg focus-visible:ring-0 focus-visible:border-transparent [&_[data-slot=select-value]]:min-w-0 [&_[data-slot=select-value]]:truncate [&>svg]:shrink-0`;
|
||||
const NUMBER_INPUT_CLASS = `h-8 w-[92px] ${CONTROL_SURFACE} pl-3 pr-2 py-0 text-right text-[13px] font-medium text-nav-fg outline-none focus-visible:ring-0`;
|
||||
|
||||
const KV_CACHE_DTYPE_DEFAULT = "f16";
|
||||
const SPECULATIVE_TYPE_LABELS: Record<(typeof SPECULATIVE_TYPES)[number], string> =
|
||||
{
|
||||
auto: "Auto",
|
||||
mtp: "MTP",
|
||||
ngram: "Ngram",
|
||||
"mtp+ngram": "MTP+Ngram",
|
||||
off: "Off",
|
||||
};
|
||||
|
||||
function hasNonDefaultAdvanced(config: PerModelConfig): boolean {
|
||||
return (
|
||||
config.kvCacheDtype != null ||
|
||||
(config.speculativeType ?? "auto") !== "auto" ||
|
||||
config.specDraftNMax != null ||
|
||||
config.tensorParallel ||
|
||||
config.chatTemplateOverride != null ||
|
||||
(config.gpuMemoryMode ?? "auto") !== "auto" ||
|
||||
(config.gpuLayers != null && config.gpuLayers >= 0) ||
|
||||
(config.nCpuMoe ?? 0) > 0 ||
|
||||
config.selectedGpuIds != null
|
||||
);
|
||||
}
|
||||
|
||||
function ChatTemplateSetting({
|
||||
config,
|
||||
onEditTemplate,
|
||||
readOnly = false,
|
||||
}: {
|
||||
config: PerModelConfig;
|
||||
onEditTemplate: () => void;
|
||||
readOnly?: boolean;
|
||||
}) {
|
||||
return (
|
||||
<div className={ROW_CLASS}>
|
||||
<div className="flex min-w-0 items-center gap-1.5">
|
||||
<span className={LABEL_CLASS}>Chat Template</span>
|
||||
<InfoHint>
|
||||
{readOnly
|
||||
? "Preview the model's chat template. Custom overrides apply to GGUF models for now."
|
||||
: "Override the model's chat template with custom Jinja. Applies when the model loads."}
|
||||
</InfoHint>
|
||||
</div>
|
||||
<div className="flex shrink-0 items-center gap-2">
|
||||
{readOnly ? null : (
|
||||
<span className="text-[12px] text-muted-foreground">
|
||||
{config.chatTemplateOverride ? "Custom" : "Default"}
|
||||
</span>
|
||||
)}
|
||||
<Button
|
||||
type="button"
|
||||
size="sm"
|
||||
variant="ghost"
|
||||
className={`h-8 px-3 text-[13px] ${CONTROL_SURFACE}`}
|
||||
onClick={onEditTemplate}
|
||||
>
|
||||
{readOnly ? "View" : "Edit"}
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function MaxSeqLengthSetting({
|
||||
value,
|
||||
max,
|
||||
inputMax,
|
||||
onChange,
|
||||
}: {
|
||||
value: number;
|
||||
max: number;
|
||||
inputMax: number;
|
||||
onChange: (value: number) => void;
|
||||
}) {
|
||||
return (
|
||||
<div className="space-y-3">
|
||||
<div className={ROW_CLASS}>
|
||||
<div className="flex min-w-0 items-center gap-1.5">
|
||||
<span className={LABEL_CLASS}>Max Seq Length</span>
|
||||
<InfoHint>
|
||||
Maximum context window size in tokens. Applies when the model loads.
|
||||
</InfoHint>
|
||||
</div>
|
||||
<NumericValueInput
|
||||
value={value}
|
||||
min={MAX_SEQ_LENGTH_MIN}
|
||||
max={inputMax}
|
||||
step={MAX_SEQ_LENGTH_STEP}
|
||||
onChange={onChange}
|
||||
ariaLabel="Max Seq Length"
|
||||
className={NUMBER_INPUT_CLASS}
|
||||
size={8}
|
||||
/>
|
||||
</div>
|
||||
<Slider
|
||||
min={MAX_SEQ_LENGTH_MIN}
|
||||
max={max}
|
||||
step={MAX_SEQ_LENGTH_STEP}
|
||||
value={[value]}
|
||||
onValueChange={([next]) => onChange(next)}
|
||||
className="panel-slider"
|
||||
aria-label="Max Seq Length"
|
||||
/>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function clampMaxSeqLength(value: number, max: number): number {
|
||||
const normalized = normalizeMaxSeqLength(value) ?? MAX_SEQ_LENGTH_MIN;
|
||||
return Math.max(MAX_SEQ_LENGTH_MIN, Math.min(max, normalized));
|
||||
}
|
||||
|
||||
function AdvancedGpuSlider({
|
||||
label,
|
||||
value,
|
||||
min,
|
||||
max,
|
||||
onChange,
|
||||
displayValue,
|
||||
info,
|
||||
}: {
|
||||
label: string;
|
||||
value: number;
|
||||
min: number;
|
||||
max: number;
|
||||
onChange: (value: number) => void;
|
||||
displayValue?: string;
|
||||
info?: ReactNode;
|
||||
}) {
|
||||
return (
|
||||
<div className="space-y-3">
|
||||
<div className={ROW_CLASS}>
|
||||
<div className="flex min-w-0 items-center gap-1.5">
|
||||
<span className={LABEL_CLASS}>{label}</span>
|
||||
{info && <InfoHint>{info}</InfoHint>}
|
||||
</div>
|
||||
<NumericValueInput
|
||||
value={value}
|
||||
min={min}
|
||||
max={max}
|
||||
step={1}
|
||||
onChange={onChange}
|
||||
displayValue={displayValue}
|
||||
ariaLabel={label}
|
||||
className={NUMBER_INPUT_CLASS}
|
||||
size={8}
|
||||
/>
|
||||
</div>
|
||||
<Slider
|
||||
min={min}
|
||||
max={max}
|
||||
step={1}
|
||||
value={[value]}
|
||||
onValueChange={([next]) => onChange(next)}
|
||||
className="panel-slider"
|
||||
aria-label={label}
|
||||
/>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
// GPU Memory placement controls (mode / GPU Layers / MoE offload / GPU picker),
|
||||
// GGUF only. Slider ceilings come from the GGUF header dims, the picker from the
|
||||
// live device list. --tensor-split is not persisted per model, so not exposed here.
|
||||
function GpuMemorySettings({
|
||||
config,
|
||||
update,
|
||||
layerCount,
|
||||
moeLayerCount,
|
||||
}: {
|
||||
config: PerModelConfig;
|
||||
update: (patch: Partial<PerModelConfig>) => void;
|
||||
layerCount: number | null;
|
||||
moeLayerCount: number | null;
|
||||
}) {
|
||||
const gpuDevices = useGpuDevices();
|
||||
const mode = config.gpuMemoryMode ?? "auto";
|
||||
const isManual = mode === "manual";
|
||||
const gpuLayers = config.gpuLayers ?? GPU_LAYERS_AUTO;
|
||||
// Slider at Auto: llama.cpp --fit owns the layout, so MoE-offload doesn't apply.
|
||||
const autoLayers = isManual && gpuLayers < 0;
|
||||
// Ceiling = layer count + 1 (llama.cpp counts the output layer as offloadable),
|
||||
// else a safe fallback.
|
||||
const gpuLayersMax = layerCount != null ? layerCount + 1 : 256;
|
||||
const nCpuMoe = config.nCpuMoe ?? 0;
|
||||
const moeLayersMax = moeLayerCount ?? 0;
|
||||
const showMoeSlider = isManual && !autoLayers && moeLayersMax > 0;
|
||||
const selectedGpuIds = config.selectedGpuIds ?? null;
|
||||
const singleGpuInUse =
|
||||
(selectedGpuIds ?? gpuDevices.map((device) => device.index)).length <= 1;
|
||||
// Multi-GPU only, and only with physical indices (relative ordinals from a
|
||||
// CUDA_VISIBLE_DEVICES mask can't be mapped back to pin a device). null = all (auto).
|
||||
const showGpuPicker =
|
||||
gpuDevices.length > 1 && gpuDevices.every((d) => d.physicalIndex);
|
||||
const isGpuChecked = (index: number) =>
|
||||
selectedGpuIds === null || selectedGpuIds.includes(index);
|
||||
const toggleGpu = (index: number) => {
|
||||
const all = gpuDevices.map((d) => d.index);
|
||||
const current = selectedGpuIds ?? all;
|
||||
const next = current.includes(index)
|
||||
? current.filter((i) => i !== index)
|
||||
: [...current, index].sort((a, b) => a - b);
|
||||
if (next.length === 0) return; // keep at least one GPU selected
|
||||
update({ selectedGpuIds: next.length === all.length ? null : next });
|
||||
};
|
||||
return (
|
||||
<>
|
||||
<div className={ROW_CLASS}>
|
||||
<div className="flex min-w-0 items-center gap-1.5">
|
||||
<span className={LABEL_CLASS}>GPU Memory</span>
|
||||
<InfoHint>
|
||||
<div className="flex flex-col gap-1.5">
|
||||
<div>
|
||||
<span className="font-medium">Default:</span> Unsloth fits the
|
||||
model and context to your GPUs.
|
||||
</div>
|
||||
<div>
|
||||
<span className="font-medium">Manual:</span> set GPU Layers
|
||||
yourself. Leave it on Auto to let llama.cpp size the context and
|
||||
offload overflow (including MoE experts) to RAM.
|
||||
</div>
|
||||
</div>
|
||||
</InfoHint>
|
||||
</div>
|
||||
<Select
|
||||
value={mode}
|
||||
onValueChange={(v) =>
|
||||
// Returning to Default must clear the Manual-only knobs, else a
|
||||
// remembered config keeps stale gpuLayers/nCpuMoe/GPU pick that a
|
||||
// later load re-applies while the page shows Default.
|
||||
update(
|
||||
v === "manual"
|
||||
? { gpuMemoryMode: "manual" }
|
||||
: {
|
||||
gpuMemoryMode: "auto",
|
||||
gpuLayers: undefined,
|
||||
nCpuMoe: undefined,
|
||||
selectedGpuIds: undefined,
|
||||
},
|
||||
)
|
||||
}
|
||||
>
|
||||
<SelectTrigger
|
||||
animateRadius={false}
|
||||
icon={ChevronDownStandardIcon}
|
||||
iconClassName="size-3.5"
|
||||
className={`w-[124px] shrink-0 ${SELECT_TRIGGER_CLASS}`}
|
||||
>
|
||||
<SelectValue />
|
||||
</SelectTrigger>
|
||||
<SelectContent className="menu-soft-surface ring-0 border-0 rounded-lg">
|
||||
<SelectItem value="auto">Default</SelectItem>
|
||||
<SelectItem value="manual">Manual</SelectItem>
|
||||
</SelectContent>
|
||||
</Select>
|
||||
</div>
|
||||
{isManual && (
|
||||
<>
|
||||
<AdvancedGpuSlider
|
||||
label="GPU Layers"
|
||||
value={Math.max(GPU_LAYERS_AUTO, Math.min(gpuLayers, gpuLayersMax))}
|
||||
min={GPU_LAYERS_AUTO}
|
||||
max={gpuLayersMax}
|
||||
onChange={(v) => update({ gpuLayers: v })}
|
||||
displayValue={autoLayers ? "Auto" : undefined}
|
||||
info={
|
||||
<>
|
||||
Layers to keep on the GPU (--gpu-layers); the rest run on CPU.
|
||||
Auto lets llama.cpp size the split (and the context) to fit VRAM.
|
||||
At the maximum, the whole model is on the GPU.
|
||||
</>
|
||||
}
|
||||
/>
|
||||
{showMoeSlider && (
|
||||
<AdvancedGpuSlider
|
||||
label="MoE Layers on CPU"
|
||||
value={Math.min(nCpuMoe, moeLayersMax)}
|
||||
min={0}
|
||||
max={moeLayersMax}
|
||||
onChange={(v) => update({ nCpuMoe: v })}
|
||||
info={
|
||||
<>
|
||||
Keep the experts of this many MoE layers on the CPU
|
||||
(--n-cpu-moe) to save VRAM. 0 = all experts on the GPU; at the
|
||||
maximum, all are on the CPU.
|
||||
</>
|
||||
}
|
||||
/>
|
||||
)}
|
||||
</>
|
||||
)}
|
||||
{showGpuPicker && (
|
||||
<div className="space-y-2">
|
||||
<div className="flex min-w-0 items-center gap-1.5">
|
||||
<span className={LABEL_CLASS}>GPUs</span>
|
||||
<InfoHint>
|
||||
Which GPUs this model may use. Unchecked GPUs are hidden from
|
||||
llama.cpp (CUDA_VISIBLE_DEVICES, or HIP_VISIBLE_DEVICES on ROCm).
|
||||
Leave all checked to use every GPU. At least one GPU must stay
|
||||
selected.
|
||||
</InfoHint>
|
||||
</div>
|
||||
<div className="flex flex-col gap-2">
|
||||
{gpuDevices.map((d) => (
|
||||
<div
|
||||
key={d.index}
|
||||
className="flex items-center justify-between gap-3"
|
||||
>
|
||||
<span className="min-w-0 truncate text-[12px] text-nav-fg/80">
|
||||
GPU {d.index}: {d.name}
|
||||
{d.memoryTotalGb
|
||||
? ` · ${Math.round(d.memoryTotalGb)} GB`
|
||||
: ""}
|
||||
</span>
|
||||
<Switch
|
||||
className="panel-switch shrink-0"
|
||||
checked={isGpuChecked(d.index)}
|
||||
onCheckedChange={() => toggleGpu(d.index)}
|
||||
disabled={isGpuChecked(d.index) && singleGpuInUse}
|
||||
/>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
</>
|
||||
);
|
||||
}
|
||||
|
||||
function GgufAdvancedSettings({
|
||||
config,
|
||||
update,
|
||||
isMtp,
|
||||
speculativeFallback,
|
||||
onEditTemplate,
|
||||
layerCount,
|
||||
moeLayerCount,
|
||||
}: {
|
||||
config: PerModelConfig;
|
||||
update: (patch: Partial<PerModelConfig>) => void;
|
||||
isMtp: boolean;
|
||||
speculativeFallback: string;
|
||||
onEditTemplate: () => void;
|
||||
layerCount: number | null;
|
||||
moeLayerCount: number | null;
|
||||
}) {
|
||||
return (
|
||||
<>
|
||||
<div className={ROW_CLASS}>
|
||||
<div className="flex min-w-0 items-center gap-1.5">
|
||||
<span className={LABEL_CLASS}>KV Cache Dtype</span>
|
||||
<InfoHint>
|
||||
Lower KV cache precision to save VRAM at the cost of some quality.
|
||||
f16/bf16 are full precision; q8_0/q5_1/q4_1 are quantized.
|
||||
</InfoHint>
|
||||
</div>
|
||||
<Select
|
||||
value={config.kvCacheDtype ?? KV_CACHE_DTYPE_DEFAULT}
|
||||
onValueChange={(v) =>
|
||||
update({ kvCacheDtype: v === KV_CACHE_DTYPE_DEFAULT ? null : v })
|
||||
}
|
||||
>
|
||||
<SelectTrigger
|
||||
animateRadius={false}
|
||||
icon={ChevronDownStandardIcon}
|
||||
iconClassName="size-3.5"
|
||||
className={`w-[92px] ${SELECT_TRIGGER_CLASS}`}
|
||||
>
|
||||
<SelectValue />
|
||||
</SelectTrigger>
|
||||
<SelectContent className="menu-soft-surface ring-0 border-0 rounded-lg">
|
||||
<SelectItem value={KV_CACHE_DTYPE_DEFAULT}>
|
||||
{KV_CACHE_DTYPE_DEFAULT}
|
||||
</SelectItem>
|
||||
{KV_CACHE_DTYPES.map((dtype) => (
|
||||
<SelectItem key={dtype} value={dtype}>
|
||||
{dtype}
|
||||
</SelectItem>
|
||||
))}
|
||||
</SelectContent>
|
||||
</Select>
|
||||
</div>
|
||||
|
||||
<div className={ROW_CLASS}>
|
||||
<div className="flex min-w-0 items-center gap-1.5">
|
||||
<span className={LABEL_CLASS_WRAP}>Speculative Decoding</span>
|
||||
<InfoHint>
|
||||
Faster generation with no accuracy hit. Auto picks MTP / ngram based
|
||||
on the model and platform. Pick a strategy to force it.
|
||||
</InfoHint>
|
||||
</div>
|
||||
<Select
|
||||
value={config.speculativeType ?? speculativeFallback}
|
||||
onValueChange={(v) =>
|
||||
update({
|
||||
speculativeType: v,
|
||||
specDraftNMax:
|
||||
v === "mtp" || v === "mtp+ngram" ? config.specDraftNMax : null,
|
||||
})
|
||||
}
|
||||
>
|
||||
<SelectTrigger
|
||||
animateRadius={false}
|
||||
icon={ChevronDownStandardIcon}
|
||||
iconClassName="size-3.5"
|
||||
className={`w-[124px] shrink-0 ${SELECT_TRIGGER_CLASS}`}
|
||||
>
|
||||
<SelectValue />
|
||||
</SelectTrigger>
|
||||
<SelectContent className="menu-soft-surface ring-0 border-0 rounded-lg">
|
||||
{SPECULATIVE_TYPES.map((type) => (
|
||||
<SelectItem key={type} value={type}>
|
||||
{SPECULATIVE_TYPE_LABELS[type]}
|
||||
</SelectItem>
|
||||
))}
|
||||
</SelectContent>
|
||||
</Select>
|
||||
</div>
|
||||
|
||||
{isMtp && (
|
||||
<div className={ROW_CLASS}>
|
||||
<div className="flex min-w-0 items-center gap-1.5">
|
||||
<span className={LABEL_CLASS}>Draft Tokens</span>
|
||||
<InfoHint>
|
||||
Max MTP draft tokens per step. Leave blank for the platform
|
||||
default (2 on GPU, 3 on CPU/Mac).
|
||||
</InfoHint>
|
||||
</div>
|
||||
<input
|
||||
type="number"
|
||||
min={1}
|
||||
max={16}
|
||||
step={1}
|
||||
value={config.specDraftNMax ?? ""}
|
||||
placeholder="auto"
|
||||
onChange={(event) => {
|
||||
const raw = event.target.value;
|
||||
if (raw === "") {
|
||||
update({ specDraftNMax: null });
|
||||
return;
|
||||
}
|
||||
const parsed = Number.parseInt(raw, 10);
|
||||
if (Number.isFinite(parsed)) {
|
||||
update({ specDraftNMax: Math.max(1, Math.min(16, parsed)) });
|
||||
}
|
||||
}}
|
||||
aria-label="Speculative decoding draft tokens"
|
||||
className={NUMBER_INPUT_CLASS}
|
||||
/>
|
||||
</div>
|
||||
)}
|
||||
|
||||
<div className={ROW_CLASS}>
|
||||
<div className="flex min-w-0 items-center gap-1.5">
|
||||
<span className={LABEL_CLASS}>Tensor Parallelism</span>
|
||||
<InfoHint>
|
||||
No effect on a single GPU. On multi-GPU setups, improves tokens/sec
|
||||
for dense models. MoE models don't benefit.
|
||||
</InfoHint>
|
||||
</div>
|
||||
<Switch
|
||||
className="panel-switch shrink-0"
|
||||
checked={config.tensorParallel}
|
||||
onCheckedChange={(checked) => update({ tensorParallel: checked })}
|
||||
/>
|
||||
</div>
|
||||
|
||||
<GpuMemorySettings
|
||||
config={config}
|
||||
update={update}
|
||||
layerCount={layerCount}
|
||||
moeLayerCount={moeLayerCount}
|
||||
/>
|
||||
|
||||
<ChatTemplateSetting config={config} onEditTemplate={onEditTemplate} />
|
||||
</>
|
||||
);
|
||||
}
|
||||
|
||||
interface ModelConfigPageProps {
|
||||
target: ModelPickTarget;
|
||||
onBack?: () => void;
|
||||
onRun: (config: PerModelConfig) => void;
|
||||
loadedConfig?: PerModelConfig | null;
|
||||
loadedContextLength?: number | null;
|
||||
initialConfig?: PerModelConfig | null;
|
||||
variant?: "page" | "sidebar";
|
||||
}
|
||||
|
||||
export function ModelConfigPage({
|
||||
target,
|
||||
onBack,
|
||||
onRun,
|
||||
loadedConfig = null,
|
||||
loadedContextLength = null,
|
||||
initialConfig = null,
|
||||
variant = "page",
|
||||
}: ModelConfigPageProps) {
|
||||
const rememberId = useId();
|
||||
const isActiveModel = loadedConfig != null;
|
||||
const hfToken = useChatRuntimeStore((s) => s.hfToken);
|
||||
const activeNativePathToken = useChatRuntimeStore(
|
||||
(s) => s.activeNativePathToken,
|
||||
);
|
||||
const loadedDefaultChatTemplate = useChatRuntimeStore(
|
||||
(s) => s.defaultChatTemplate,
|
||||
);
|
||||
const loadedMaxContextLength = useChatRuntimeStore(
|
||||
(s) => s.ggufMaxContextLength,
|
||||
);
|
||||
const resolveInitial = () => {
|
||||
const resolved = resolveInitialConfig(target.id, target.ggufVariant);
|
||||
if (loadedConfig) {
|
||||
return { config: loadedConfig, remembered: resolved.remembered };
|
||||
}
|
||||
if (initialConfig) {
|
||||
return {
|
||||
config: initialConfig,
|
||||
remembered:
|
||||
resolved.remembered &&
|
||||
perModelConfigsEqual(initialConfig, resolved.config),
|
||||
};
|
||||
}
|
||||
return resolved;
|
||||
};
|
||||
const [initial] = useState(resolveInitial);
|
||||
const [config, setConfig] = useState<PerModelConfig>(() => initial.config);
|
||||
const [remember, setRemember] = useState(() => initial.remembered);
|
||||
const [savedRemember, setSavedRemember] = useState(() => initial.remembered);
|
||||
const [speculativeFallback] = useState(readPersistedSpeculativeType);
|
||||
const [templateOpen, setTemplateOpen] = useState(false);
|
||||
const [showAdvanced, setShowAdvanced] = useState(() =>
|
||||
hasNonDefaultAdvanced(config),
|
||||
);
|
||||
const nativePathToken =
|
||||
target.meta.nativePathToken ??
|
||||
(isActiveModel ? activeNativePathToken : null);
|
||||
const templateDefaults = useDefaultChatTemplate(
|
||||
target.id,
|
||||
target.ggufVariant,
|
||||
templateOpen,
|
||||
nativePathToken,
|
||||
);
|
||||
const modelMaxPosition = useModelMaxPositionEmbeddings(
|
||||
target.id,
|
||||
!target.isGguf,
|
||||
);
|
||||
const hasLoadedDefaultTemplate =
|
||||
isActiveModel && loadedDefaultChatTemplate != null;
|
||||
const resolvedDefaultTemplate = hasLoadedDefaultTemplate
|
||||
? loadedDefaultChatTemplate
|
||||
: templateDefaults.template;
|
||||
const resolvedDefaultLoading = hasLoadedDefaultTemplate
|
||||
? false
|
||||
: templateDefaults.loading;
|
||||
|
||||
const update = (patch: Partial<PerModelConfig>) =>
|
||||
setConfig((current) => ({ ...current, ...patch }));
|
||||
|
||||
// Fetch GGUF header dims (context + layer/MoE counts) to size the GPU Memory
|
||||
// sliders; the context also fills in below when target.meta lacks it.
|
||||
const contextFetchKey = target.isGguf
|
||||
? `${target.id}\n${target.ggufVariant ?? ""}\n${hfToken || ""}\n${nativePathToken ?? ""}`
|
||||
: null;
|
||||
const [fetchedStagedDims, setFetchedStagedDims] = useState<{
|
||||
key: string;
|
||||
contextLength: number | null;
|
||||
layerCount: number | null;
|
||||
moeLayerCount: number | null;
|
||||
} | null>(null);
|
||||
useEffect(() => {
|
||||
if (contextFetchKey == null) {
|
||||
return;
|
||||
}
|
||||
let cancelled = false;
|
||||
void fetchGgufStagedMetadata({
|
||||
model_path: target.id,
|
||||
gguf_variant: target.ggufVariant ?? null,
|
||||
hf_token: hfToken || null,
|
||||
nativePathToken,
|
||||
})
|
||||
.then((dims) => {
|
||||
if (!cancelled) {
|
||||
setFetchedStagedDims({ key: contextFetchKey, ...dims });
|
||||
}
|
||||
})
|
||||
.catch(() => {
|
||||
if (!cancelled) {
|
||||
setFetchedStagedDims({
|
||||
key: contextFetchKey,
|
||||
contextLength: null,
|
||||
layerCount: null,
|
||||
moeLayerCount: null,
|
||||
});
|
||||
}
|
||||
});
|
||||
return () => {
|
||||
cancelled = true;
|
||||
};
|
||||
}, [
|
||||
contextFetchKey,
|
||||
target.id,
|
||||
target.ggufVariant,
|
||||
hfToken,
|
||||
nativePathToken,
|
||||
]);
|
||||
const stagedDims =
|
||||
fetchedStagedDims?.key === contextFetchKey ? fetchedStagedDims : null;
|
||||
|
||||
const isMtp =
|
||||
config.speculativeType != null &&
|
||||
MTP_SPECULATIVE_TYPES.has(config.speculativeType);
|
||||
const nativeContextLength =
|
||||
target.meta.contextLength ?? stagedDims?.contextLength ?? null;
|
||||
const activeLoadedContext =
|
||||
isActiveModel && target.isGguf ? loadedContextLength : null;
|
||||
const minContext = CONTEXT_LENGTH_MIN;
|
||||
const maxContext = Math.max(
|
||||
minContext,
|
||||
Math.max(
|
||||
nativeContextLength ?? 0,
|
||||
activeLoadedContext ?? 0,
|
||||
config.customContextLength ?? 0,
|
||||
) || 32768,
|
||||
);
|
||||
const contextValue = Math.min(
|
||||
Math.max(
|
||||
config.customContextLength ??
|
||||
activeLoadedContext ??
|
||||
nativeContextLength ??
|
||||
maxContext,
|
||||
minContext,
|
||||
),
|
||||
maxContext,
|
||||
);
|
||||
const setContextLength = (v: number) =>
|
||||
update({ customContextLength: v });
|
||||
const baseline = loadedConfig ?? DEFAULT_PER_MODEL_CONFIG;
|
||||
const atBaseline = perModelConfigsEqual(config, baseline);
|
||||
// An explicit customContextLength equal to the native ceiling is still an
|
||||
// override (Reset stays enabled). "At default" means no override at all AND the
|
||||
// shown context matches native (or no native context length is exposed).
|
||||
const contextAtDefault =
|
||||
!target.isGguf ||
|
||||
(config.customContextLength == null &&
|
||||
(nativeContextLength == null || contextValue === nativeContextLength));
|
||||
const atDefault =
|
||||
contextAtDefault &&
|
||||
perModelConfigsEqual(
|
||||
{ ...config, customContextLength: null },
|
||||
DEFAULT_PER_MODEL_CONFIG,
|
||||
);
|
||||
const nativeMaxSeqLength =
|
||||
floorMaxSeqLength(modelMaxPosition.maxPositionEmbeddings) ??
|
||||
MAX_SEQ_LENGTH_MAX;
|
||||
// A non-GGUF active model seeds maxSeqLength from its loaded value. Once cleared
|
||||
// (Reset sets null), fall back to the app default, not the loaded runtime value,
|
||||
// else a remembered/active override can never be cleared.
|
||||
const maxSeqLengthValue =
|
||||
normalizeMaxSeqLength(config.maxSeqLength) ??
|
||||
clampMaxSeqLength(DEFAULT_MAX_SEQ_LENGTH, nativeMaxSeqLength);
|
||||
const maxSeqLengthMax = Math.max(nativeMaxSeqLength, maxSeqLengthValue);
|
||||
// An auto-fit-below-native GGUF shows activeLoadedContext while
|
||||
// customContextLength stays null. If the user fixes GPU Layers (Manual) and
|
||||
// remembers, pin that shown context so a later fresh load keeps the fitted
|
||||
// placement instead of sending native/0 for fixed layers and recreating the OOM.
|
||||
const pinFixedLayerContext =
|
||||
target.isGguf &&
|
||||
config.gpuMemoryMode === "manual" &&
|
||||
config.gpuLayers != null &&
|
||||
config.gpuLayers >= 0 &&
|
||||
config.customContextLength == null &&
|
||||
activeLoadedContext != null;
|
||||
// Persisted record: keep config as-is (non-GGUF keeps maxSeqLength null) so
|
||||
// isDefaultConfig recognises it and clears a remembered override instead of
|
||||
// pinning the app default.
|
||||
const runtimeConfig = target.isGguf
|
||||
? pinFixedLayerContext
|
||||
? { ...config, customContextLength: activeLoadedContext }
|
||||
: config
|
||||
: config;
|
||||
// Load request needs a concrete max length; substitute the fallback here only,
|
||||
// never in the persisted runtimeConfig.
|
||||
const loadConfig = target.isGguf
|
||||
? runtimeConfig
|
||||
: { ...runtimeConfig, maxSeqLength: maxSeqLengthValue };
|
||||
const rememberChanged = remember !== savedRemember;
|
||||
const persistenceOnly = isActiveModel && atBaseline && rememberChanged;
|
||||
const primaryActionLabel = persistenceOnly
|
||||
? remember
|
||||
? "Save settings"
|
||||
: "Forget settings"
|
||||
: isActiveModel
|
||||
? "Reload model"
|
||||
: "Load model";
|
||||
|
||||
const handleRun = () => {
|
||||
const defaultConfig = isDefaultConfig(runtimeConfig);
|
||||
let saveFailed = false;
|
||||
if (remember) {
|
||||
saveFailed = !savePerModelConfig(
|
||||
target.id,
|
||||
target.ggufVariant,
|
||||
runtimeConfig,
|
||||
);
|
||||
} else {
|
||||
saveFailed = !deletePerModelConfig(target.id, target.ggufVariant);
|
||||
}
|
||||
if (persistenceOnly) {
|
||||
if (saveFailed) {
|
||||
toast.error("Couldn't save settings for this model.");
|
||||
return;
|
||||
}
|
||||
const nextRemember = remember && !defaultConfig;
|
||||
setSavedRemember(nextRemember);
|
||||
setRemember(nextRemember);
|
||||
toast.success(
|
||||
nextRemember
|
||||
? "Settings saved."
|
||||
: remember
|
||||
? "Default settings kept."
|
||||
: "Settings forgotten.",
|
||||
);
|
||||
return;
|
||||
}
|
||||
if (saveFailed) {
|
||||
toast.error("Couldn't save these settings, loading with them anyway.");
|
||||
}
|
||||
onRun(loadConfig);
|
||||
};
|
||||
|
||||
return (
|
||||
<div className="flex flex-col">
|
||||
{variant === "page" && (
|
||||
<div className="flex items-center gap-2.5 pb-4">
|
||||
{onBack && (
|
||||
<button
|
||||
type="button"
|
||||
onClick={onBack}
|
||||
className="nav-icon-btn shrink-0 text-nav-icon-idle hover:bg-panel-surface-hover hover:text-black dark:hover:text-white"
|
||||
aria-label="Back to model list"
|
||||
>
|
||||
<HugeiconsIcon
|
||||
icon={ArrowLeft01Icon}
|
||||
className="size-4"
|
||||
strokeWidth={1.75}
|
||||
/>
|
||||
</button>
|
||||
)}
|
||||
<div className="min-w-0 flex-1">
|
||||
<div className="text-[10px] font-semibold uppercase leading-none tracking-wider text-muted-foreground">
|
||||
Run settings
|
||||
</div>
|
||||
<div className="mt-1.5 truncate text-[14px] font-semibold leading-tight text-nav-fg">
|
||||
{target.displayName}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
|
||||
<div className="space-y-3.5">
|
||||
{target.isGguf && (
|
||||
<>
|
||||
<div className="space-y-3">
|
||||
<div className={ROW_CLASS}>
|
||||
<div className="flex min-w-0 items-center gap-1.5">
|
||||
<span className={LABEL_CLASS}>Context Length</span>
|
||||
<InfoHint>
|
||||
Tokens of context to allocate. Higher uses more VRAM.
|
||||
{nativeContextLength != null
|
||||
? ` This model's native context is ${nativeContextLength.toLocaleString()} tokens.`
|
||||
: ""}
|
||||
</InfoHint>
|
||||
</div>
|
||||
<NumericValueInput
|
||||
value={contextValue}
|
||||
min={minContext}
|
||||
max={maxContext}
|
||||
step={1}
|
||||
onChange={setContextLength}
|
||||
displayValue={
|
||||
config.customContextLength == null &&
|
||||
nativeContextLength == null &&
|
||||
activeLoadedContext == null
|
||||
? "Auto"
|
||||
: undefined
|
||||
}
|
||||
ariaLabel="Context Length"
|
||||
className={NUMBER_INPUT_CLASS}
|
||||
size={8}
|
||||
/>
|
||||
</div>
|
||||
{nativeContextLength != null ? (
|
||||
<Slider
|
||||
min={minContext}
|
||||
max={maxContext}
|
||||
step={128}
|
||||
value={[contextValue]}
|
||||
onValueChange={([v]) => setContextLength(v)}
|
||||
className="panel-slider"
|
||||
aria-label="Context Length"
|
||||
/>
|
||||
) : null}
|
||||
{isActiveModel &&
|
||||
loadedMaxContextLength != null &&
|
||||
contextValue > loadedMaxContextLength && (
|
||||
<p className="text-[11px] text-amber-500">
|
||||
Exceeds estimated VRAM capacity (
|
||||
{loadedMaxContextLength.toLocaleString()} tokens). The model
|
||||
may use system RAM.
|
||||
</p>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{showAdvanced && (
|
||||
<GgufAdvancedSettings
|
||||
config={config}
|
||||
update={update}
|
||||
isMtp={isMtp}
|
||||
speculativeFallback={speculativeFallback}
|
||||
onEditTemplate={() => setTemplateOpen(true)}
|
||||
layerCount={stagedDims?.layerCount ?? null}
|
||||
moeLayerCount={stagedDims?.moeLayerCount ?? null}
|
||||
/>
|
||||
)}
|
||||
|
||||
<div className={ROW_CLASS}>
|
||||
<div className="flex min-w-0 items-center gap-1.5">
|
||||
<span className="min-w-0 text-[13px] font-medium leading-[1.25] tracking-nav text-muted-foreground">
|
||||
Advanced settings
|
||||
</span>
|
||||
<InfoHint>
|
||||
Extra options for how the model loads. Most setups don't need
|
||||
these.
|
||||
</InfoHint>
|
||||
</div>
|
||||
<Switch
|
||||
className="panel-switch shrink-0"
|
||||
checked={showAdvanced}
|
||||
onCheckedChange={setShowAdvanced}
|
||||
aria-label="Show advanced settings"
|
||||
/>
|
||||
</div>
|
||||
</>
|
||||
)}
|
||||
{!target.isGguf && (
|
||||
<>
|
||||
<MaxSeqLengthSetting
|
||||
value={maxSeqLengthValue}
|
||||
max={maxSeqLengthMax}
|
||||
inputMax={MAX_SEQ_LENGTH_MAX}
|
||||
onChange={(value) =>
|
||||
update({
|
||||
maxSeqLength: clampMaxSeqLength(value, MAX_SEQ_LENGTH_MAX),
|
||||
})
|
||||
}
|
||||
/>
|
||||
<ChatTemplateSetting
|
||||
config={config}
|
||||
onEditTemplate={() => setTemplateOpen(true)}
|
||||
readOnly={true}
|
||||
/>
|
||||
</>
|
||||
)}
|
||||
</div>
|
||||
|
||||
<div
|
||||
className={
|
||||
variant === "sidebar"
|
||||
? "mt-4 flex flex-col gap-3 border-t border-border/60 pt-4"
|
||||
: "mt-4 flex items-center justify-between gap-3 border-t border-border/60 pt-4"
|
||||
}
|
||||
>
|
||||
<div className="flex min-w-0 items-center gap-2">
|
||||
<Checkbox
|
||||
id={rememberId}
|
||||
checked={remember}
|
||||
onCheckedChange={(checked) => setRemember(checked === true)}
|
||||
/>
|
||||
<label
|
||||
htmlFor={rememberId}
|
||||
className="cursor-pointer select-none truncate text-[13px] text-nav-fg"
|
||||
>
|
||||
Remember for this model
|
||||
</label>
|
||||
</div>
|
||||
<div
|
||||
className={
|
||||
variant === "sidebar"
|
||||
? "flex items-center justify-end gap-2"
|
||||
: "flex shrink-0 items-center gap-2"
|
||||
}
|
||||
>
|
||||
<Button
|
||||
type="button"
|
||||
variant="ghost"
|
||||
size="sm"
|
||||
className="h-8"
|
||||
disabled={atDefault}
|
||||
onClick={() => setConfig({ ...DEFAULT_PER_MODEL_CONFIG })}
|
||||
>
|
||||
Reset
|
||||
</Button>
|
||||
<Button
|
||||
type="button"
|
||||
size="sm"
|
||||
className="h-8"
|
||||
disabled={isActiveModel && atBaseline && !rememberChanged}
|
||||
onClick={handleRun}
|
||||
>
|
||||
{primaryActionLabel}
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<ChatTemplateEditorDialog
|
||||
open={templateOpen}
|
||||
onOpenChange={setTemplateOpen}
|
||||
value={config.chatTemplateOverride}
|
||||
defaultTemplate={resolvedDefaultTemplate}
|
||||
defaultLoading={resolvedDefaultLoading}
|
||||
readOnly={!target.isGguf}
|
||||
onSave={(override) => update({ chatTemplateOverride: override })}
|
||||
/>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
|
@ -3,6 +3,7 @@
|
|||
|
||||
"use client";
|
||||
|
||||
import { Input } from "@/components/ui/input";
|
||||
import {
|
||||
Popover,
|
||||
PopoverContent,
|
||||
|
|
@ -10,7 +11,7 @@ import {
|
|||
} from "@/components/ui/popover";
|
||||
import { TooltipProvider } from "@/components/ui/tooltip";
|
||||
import { usePlatformStore } from "@/config/env";
|
||||
import { isCustomProviderType } from "@/features/chat/external-providers";
|
||||
import { isCustomProviderType } from "@/features/chat";
|
||||
import { ChevronDownStandardIcon } from "@/lib/chevron-icons";
|
||||
import { cn } from "@/lib/utils";
|
||||
import {
|
||||
|
|
@ -33,7 +34,11 @@ import {
|
|||
useRef,
|
||||
useState,
|
||||
} from "react";
|
||||
import { Input } from "../ui/input";
|
||||
import {
|
||||
type PerModelConfig,
|
||||
resolveInitialConfig,
|
||||
} from "../model-config/per-model-config";
|
||||
import { ModelConfigPage } from "./model-config-page";
|
||||
import { HubModelPicker, hasDownloadedModels } from "./model-selector/pickers";
|
||||
import { PillTabs } from "./model-selector/pill-tabs";
|
||||
import {
|
||||
|
|
@ -45,6 +50,7 @@ import type {
|
|||
ExternalModelOption,
|
||||
LoraModelOption,
|
||||
ModelOption,
|
||||
ModelPickTarget,
|
||||
ModelSelectorChangeMeta,
|
||||
} from "./model-selector/types";
|
||||
|
||||
|
|
@ -122,6 +128,10 @@ interface ModelSelectorProps {
|
|||
value?: string;
|
||||
defaultValue?: string;
|
||||
activeGgufVariant?: string | null;
|
||||
activeModelConfig?: PerModelConfig | null;
|
||||
activeGgufContextLength?: number | null;
|
||||
selectedConfig?: PerModelConfig | null;
|
||||
selectedGgufVariant?: string | null;
|
||||
onValueChange?: (value: string, meta: ModelSelectorChangeMeta) => void;
|
||||
onEject?: () => void;
|
||||
onFoldersChange?: () => void;
|
||||
|
|
@ -183,15 +193,12 @@ function ModelSelectorTrigger({
|
|||
>
|
||||
{isLoaded &&
|
||||
(onEject ? (
|
||||
// Loaded status doubles as a mouse eject shortcut: green checkmark
|
||||
// at rest, red eject icon on pill hover, click to eject. A plain
|
||||
// span (no role/tabIndex) keeps it out of the trigger button's
|
||||
// content model, which forbids focusable descendants. Keyboard and
|
||||
// screen-reader users eject via the picker's "Eject model" button.
|
||||
// aria-hidden marks it decorative; stopPropagation stops the
|
||||
// popover from toggling. On touch (no hover) the eject icon and
|
||||
// tooltip never reveal, so pointer-events-none disables the
|
||||
// shortcut there and taps open the picker instead of ejecting.
|
||||
// Loaded status doubles as a mouse eject shortcut (checkmark at rest,
|
||||
// eject icon on hover). A plain span keeps it out of the trigger
|
||||
// button's content model (no focusable descendants); keyboard/SR users
|
||||
// eject via the "Eject model" button. aria-hidden marks it decorative;
|
||||
// stopPropagation stops the popover toggling. On touch (no hover)
|
||||
// pointer-events-none disables it so taps open the picker instead.
|
||||
<span
|
||||
aria-hidden={true}
|
||||
title="Eject model"
|
||||
|
|
@ -285,7 +292,8 @@ function saveLastHubSection(section: HubSection): void {
|
|||
// when they have downloads, else Recommended.
|
||||
function defaultHubSection(): HubSection {
|
||||
return (
|
||||
loadLastHubSection() ?? (hasDownloadedModels() ? "downloaded" : "recommended")
|
||||
loadLastHubSection() ??
|
||||
(hasDownloadedModels() ? "downloaded" : "recommended")
|
||||
);
|
||||
}
|
||||
|
||||
|
|
@ -308,6 +316,11 @@ function ModelSelectorContent({
|
|||
loraModels,
|
||||
externalModels,
|
||||
value,
|
||||
activeGgufVariant,
|
||||
activeModelConfig,
|
||||
activeGgufContextLength,
|
||||
selectedConfig,
|
||||
selectedGgufVariant,
|
||||
onSelect,
|
||||
onEject,
|
||||
onFoldersChange,
|
||||
|
|
@ -323,6 +336,11 @@ function ModelSelectorContent({
|
|||
loraModels: LoraModelOption[];
|
||||
externalModels: ExternalModelOption[];
|
||||
value?: string;
|
||||
activeGgufVariant?: string | null;
|
||||
activeModelConfig?: PerModelConfig | null;
|
||||
activeGgufContextLength?: number | null;
|
||||
selectedConfig?: PerModelConfig | null;
|
||||
selectedGgufVariant?: string | null;
|
||||
onSelect: (id: string, meta: ModelSelectorChangeMeta) => void;
|
||||
onEject?: () => void;
|
||||
onFoldersChange?: () => void;
|
||||
|
|
@ -337,8 +355,7 @@ function ModelSelectorContent({
|
|||
const chatOnly = usePlatformStore((s) => s.isChatOnly());
|
||||
const hasExternal = externalModels.length > 0;
|
||||
// The Fine-tuned tab is for fine-tuned models only. Local models (LM Studio,
|
||||
// Ollama, custom folders) carry source "local" and live in the Hub tab's
|
||||
// Downloaded / Custom sections instead.
|
||||
// Ollama, custom folders) carry source "local" and live in the Hub tab instead.
|
||||
const fineTunedModels = useMemo(
|
||||
() => loraModels.filter((model) => isFineTunedSource(model.source)),
|
||||
[loraModels],
|
||||
|
|
@ -391,9 +408,12 @@ function ModelSelectorContent({
|
|||
const effectiveHubSection: HubSection =
|
||||
hubSection === "connected" && !hasExternal ? "recommended" : hubSection;
|
||||
|
||||
// The picker below remounts on each open, but this tab state does not, so a
|
||||
// persisted selection that lands in lora/external after async load would
|
||||
// reopen on Hub. Re-derive the default tab on the open edge.
|
||||
const [configTarget, setConfigTarget] = useState<ModelPickTarget | null>(
|
||||
null,
|
||||
);
|
||||
|
||||
// The picker remounts on each open but this tab state does not, so re-derive
|
||||
// the default tab on the open edge (else a lora/external selection reopens on Hub).
|
||||
const wasOpen = useRef(open);
|
||||
useEffect(() => {
|
||||
if (open && !wasOpen.current) {
|
||||
|
|
@ -402,6 +422,9 @@ function ModelSelectorContent({
|
|||
// user has downloads, else their last section.
|
||||
setHubSection(wantsConnectedDefault ? "connected" : defaultHubSection());
|
||||
}
|
||||
if (!open && wasOpen.current) {
|
||||
setConfigTarget(null);
|
||||
}
|
||||
wasOpen.current = open;
|
||||
}, [
|
||||
open,
|
||||
|
|
@ -452,6 +475,29 @@ function ModelSelectorContent({
|
|||
}
|
||||
}
|
||||
|
||||
const visibleConfigTarget = open ? configTarget : null;
|
||||
const openConfigPage = (id: string, meta: ModelSelectorChangeMeta) => {
|
||||
const leaf = id.includes("/") ? id.slice(id.lastIndexOf("/") + 1) : id;
|
||||
setConfigTarget({
|
||||
id,
|
||||
displayName: meta.ggufVariant ? `${leaf} · ${meta.ggufVariant}` : leaf,
|
||||
ggufVariant: meta.ggufVariant ?? null,
|
||||
isGguf: meta.isGguf ?? Boolean(meta.ggufVariant),
|
||||
meta,
|
||||
});
|
||||
};
|
||||
const handlePick = (id: string, meta: ModelSelectorChangeMeta) => {
|
||||
if (meta.source === "external") {
|
||||
onSelect(id, meta);
|
||||
return;
|
||||
}
|
||||
const resolved = resolveInitialConfig(id, meta.ggufVariant);
|
||||
onSelect(id, {
|
||||
...meta,
|
||||
...(resolved.remembered ? { config: resolved.config } : {}),
|
||||
});
|
||||
};
|
||||
|
||||
return (
|
||||
<PopoverContent
|
||||
align="start"
|
||||
|
|
@ -459,12 +505,17 @@ function ModelSelectorContent({
|
|||
data-tour={dataTour}
|
||||
onKeyDown={handlePickerEntryKeyDown}
|
||||
className={cn(
|
||||
"unsloth-model-selector-menu menu-soft-surface ring-0 max-w-[calc(100vw-1rem)] min-w-0 gap-0 pt-4 pb-0 pl-4",
|
||||
// Sized so the left-packed row keeps uniform gaps and the last dropdown's
|
||||
// right gap matches the pill's left gap (pl-4 vs pr-4).
|
||||
hasExternal
|
||||
? "w-[min(614px,calc(100vw-1rem))] pr-4"
|
||||
: "w-[min(506px,calc(100vw-1rem))] pr-2",
|
||||
"unsloth-model-selector-menu menu-soft-surface ring-0 max-w-[calc(100vw-1rem)] min-w-0 gap-0",
|
||||
visibleConfigTarget
|
||||
? "w-[min(468px,calc(100vw-1rem))] px-4 pt-4 pb-4"
|
||||
: cn(
|
||||
"pt-4 pb-0 pl-4",
|
||||
// Sized so the left-packed row keeps uniform gaps and the last
|
||||
// dropdown's right gap matches the pill's left gap (pl-4 vs pr-4).
|
||||
hasExternal
|
||||
? "w-[min(614px,calc(100vw-1rem))] pr-4"
|
||||
: "w-[min(506px,calc(100vw-1rem))] pr-2",
|
||||
),
|
||||
className,
|
||||
)}
|
||||
>
|
||||
|
|
@ -477,6 +528,42 @@ function ModelSelectorContent({
|
|||
skipDelayDuration={0}
|
||||
disableHoverableContent={true}
|
||||
>
|
||||
{visibleConfigTarget ? (
|
||||
<ModelConfigPage
|
||||
key={`${visibleConfigTarget.id}::${visibleConfigTarget.ggufVariant ?? ""}`}
|
||||
target={visibleConfigTarget}
|
||||
onBack={() => setConfigTarget(null)}
|
||||
onRun={(config) =>
|
||||
onSelect(visibleConfigTarget.id, {
|
||||
...visibleConfigTarget.meta,
|
||||
config,
|
||||
forceReload: true,
|
||||
})
|
||||
}
|
||||
loadedConfig={
|
||||
value === visibleConfigTarget.id &&
|
||||
(activeGgufVariant ?? null) ===
|
||||
(visibleConfigTarget.ggufVariant ?? null)
|
||||
? (activeModelConfig ?? null)
|
||||
: null
|
||||
}
|
||||
loadedContextLength={
|
||||
value === visibleConfigTarget.id &&
|
||||
(activeGgufVariant ?? null) ===
|
||||
(visibleConfigTarget.ggufVariant ?? null)
|
||||
? (activeGgufContextLength ?? null)
|
||||
: null
|
||||
}
|
||||
initialConfig={
|
||||
value === visibleConfigTarget.id &&
|
||||
(selectedGgufVariant ?? null) ===
|
||||
(visibleConfigTarget.ggufVariant ?? null)
|
||||
? (selectedConfig ?? null)
|
||||
: null
|
||||
}
|
||||
/>
|
||||
) : (
|
||||
<>
|
||||
{tabs.length > 1 ? (
|
||||
<PillTabs
|
||||
ariaLabel="Model source"
|
||||
|
|
@ -494,13 +581,14 @@ function ModelSelectorContent({
|
|||
loraModels={fineTunedModels}
|
||||
externalModels={externalModels}
|
||||
value={value}
|
||||
onSelect={onSelect}
|
||||
onSelect={handlePick}
|
||||
onFoldersChange={onFoldersChange}
|
||||
onBrowseHub={onBrowseHub}
|
||||
onModelsChange={onModelsChange}
|
||||
onConfigure={openConfigPage}
|
||||
deleteDisabled={deleteDisabled}
|
||||
section={effectiveHubSection}
|
||||
onEject={hasSelection && onEject ? onEject : undefined}
|
||||
section={effectiveHubSection}
|
||||
sectionToggle={
|
||||
<PillTabs
|
||||
ariaLabel="Hub section"
|
||||
|
|
@ -538,10 +626,8 @@ function ModelSelectorContent({
|
|||
</button>
|
||||
</div>
|
||||
) : null}
|
||||
{/* Hub renders Eject inline as the last list row; other tabs keep the
|
||||
footer button. */}
|
||||
{effectiveTab !== "hub" && hasSelection && onEject ? (
|
||||
<div className="mt-1.5 pt-1.5">
|
||||
<div className="mt-1.5 border-t border-border/70 pt-1.5 pb-2">
|
||||
<button
|
||||
type="button"
|
||||
onClick={onEject}
|
||||
|
|
@ -553,6 +639,8 @@ function ModelSelectorContent({
|
|||
</button>
|
||||
</div>
|
||||
) : null}
|
||||
</>
|
||||
)}
|
||||
</TooltipProvider>
|
||||
</PopoverContent>
|
||||
);
|
||||
|
|
@ -565,6 +653,10 @@ export function ModelSelector({
|
|||
value,
|
||||
defaultValue,
|
||||
activeGgufVariant,
|
||||
activeModelConfig,
|
||||
activeGgufContextLength,
|
||||
selectedConfig,
|
||||
selectedGgufVariant,
|
||||
onValueChange,
|
||||
onEject,
|
||||
onFoldersChange,
|
||||
|
|
@ -693,6 +785,11 @@ export function ModelSelector({
|
|||
loraModels={loraModels}
|
||||
externalModels={externalModels}
|
||||
value={selected}
|
||||
activeGgufVariant={activeGgufVariant}
|
||||
activeModelConfig={activeModelConfig}
|
||||
activeGgufContextLength={activeGgufContextLength}
|
||||
selectedConfig={selectedConfig}
|
||||
selectedGgufVariant={selectedGgufVariant}
|
||||
onSelect={handleSelect}
|
||||
onEject={onEject ? handleEject : undefined}
|
||||
onFoldersChange={onFoldersChange}
|
||||
|
|
@ -14,10 +14,7 @@ import {
|
|||
DialogTitle,
|
||||
} from "@/components/ui/dialog";
|
||||
import { Spinner } from "@/components/ui/spinner";
|
||||
import {
|
||||
type BrowseFoldersResponse,
|
||||
browseFolders,
|
||||
} from "@/features/chat/api/chat-api";
|
||||
import { type BrowseFoldersResponse, browseFolders } from "@/features/chat";
|
||||
import { ChevronUpStandardIcon } from "@/lib/chevron-icons";
|
||||
import { cn } from "@/lib/utils";
|
||||
import { Folder02Icon } from "@hugeicons/core-free-icons";
|
||||
|
|
@ -35,9 +32,9 @@ export interface FolderBrowserProps {
|
|||
|
||||
function splitBreadcrumb(path: string): { label: string; value: string }[] {
|
||||
if (!path) return [];
|
||||
// Detect path style BEFORE normalizing: on POSIX, `\` is a valid filename
|
||||
// char, so blindly rewriting `\` -> `/` mangles names like `my\backup` into
|
||||
// 404ing breadcrumbs. Only Windows-style paths (drive letter, or UNC) convert.
|
||||
// Detect path style BEFORE normalizing: on POSIX `\` is a valid filename char,
|
||||
// so rewriting `\` -> `/` would mangle names like `my\backup`. Only Windows
|
||||
// paths (drive letter or UNC) convert.
|
||||
const isWindowsDrive =
|
||||
/^[A-Za-z]:[\\/]/.test(path) || /^[A-Za-z]:$/.test(path);
|
||||
const isUnc = /^\\\\/.test(path);
|
||||
|
|
@ -58,9 +55,8 @@ function splitBreadcrumb(path: string): { label: string; value: string }[] {
|
|||
return parts;
|
||||
}
|
||||
|
||||
// Windows drive path (C:, D:): first segment is the drive. Use `C:/` as the
|
||||
// crumb value so clicking the drive root navigates to the drive root, not the
|
||||
// drive-relative CWD (`C:` alone resolves to CWD-on-C, not `C:\`).
|
||||
// Windows drive path: use `C:/` as the crumb value so clicking the drive root
|
||||
// goes to the drive root, not the drive-relative CWD (`C:` alone is CWD-on-C).
|
||||
if (/^[A-Za-z]:$/.test(segments[0])) {
|
||||
const driveRoot = `${segments[0]}/`;
|
||||
let cur = driveRoot;
|
||||
|
|
@ -90,47 +86,43 @@ export function FolderBrowser({
|
|||
const [error, setError] = useState<string | null>(null);
|
||||
const abortRef = useRef<AbortController | null>(null);
|
||||
|
||||
const navigate = useCallback(
|
||||
(
|
||||
target: string | undefined,
|
||||
hidden: boolean,
|
||||
opts?: { fallbackOnError?: boolean },
|
||||
) => {
|
||||
abortRef.current?.abort();
|
||||
const ctrl = new AbortController();
|
||||
abortRef.current = ctrl;
|
||||
setLoading(true);
|
||||
setError(null);
|
||||
// Forward the signal so cancelled navigation aborts the backend
|
||||
// enumeration, not just the response.
|
||||
browseFolders(target, hidden, ctrl.signal)
|
||||
.then((res) => {
|
||||
if (ctrl.signal.aborted) return;
|
||||
setData(res);
|
||||
setPath(res.current);
|
||||
})
|
||||
.catch((err) => {
|
||||
if (ctrl.signal.aborted) return;
|
||||
// Surface the error; if the first request (e.g. a bad initialPath)
|
||||
// fails, fall back to HOME so the modal stays navigable.
|
||||
const message = err instanceof Error ? err.message : String(err);
|
||||
setError(message);
|
||||
if (opts?.fallbackOnError && target !== undefined) {
|
||||
// Re-issue without a target -> backend defaults to HOME.
|
||||
// Don't recurse if HOME itself fails (allowlist always has HOME).
|
||||
queueMicrotask(() => navigate(undefined, hidden));
|
||||
}
|
||||
})
|
||||
.finally(() => {
|
||||
if (!ctrl.signal.aborted) setLoading(false);
|
||||
});
|
||||
},
|
||||
[],
|
||||
);
|
||||
function navigate(
|
||||
target: string | undefined,
|
||||
hidden: boolean,
|
||||
opts?: { fallbackOnError?: boolean },
|
||||
) {
|
||||
abortRef.current?.abort();
|
||||
const ctrl = new AbortController();
|
||||
abortRef.current = ctrl;
|
||||
setLoading(true);
|
||||
setError(null);
|
||||
// Forward the signal so cancelled navigation aborts the backend
|
||||
// enumeration, not just the response.
|
||||
browseFolders(target, hidden, ctrl.signal)
|
||||
.then((res) => {
|
||||
if (ctrl.signal.aborted) return;
|
||||
setData(res);
|
||||
setPath(res.current);
|
||||
})
|
||||
.catch((err) => {
|
||||
if (ctrl.signal.aborted) return;
|
||||
// Surface the error; if the first request (e.g. a bad initialPath)
|
||||
// fails, fall back to HOME so the modal stays navigable.
|
||||
const message = err instanceof Error ? err.message : String(err);
|
||||
setError(message);
|
||||
if (opts?.fallbackOnError && target !== undefined) {
|
||||
// Re-issue without a target -> backend defaults to HOME.
|
||||
// Don't recurse if HOME itself fails (allowlist always has HOME).
|
||||
queueMicrotask(() => navigate(undefined, hidden));
|
||||
}
|
||||
})
|
||||
.finally(() => {
|
||||
if (!ctrl.signal.aborted) setLoading(false);
|
||||
});
|
||||
}
|
||||
|
||||
// Fetch only on closed -> open; later navigation is driven by `navigate()`,
|
||||
// so `path` is deliberately kept out of the dependency list.
|
||||
// eslint-disable-next-line react-hooks/exhaustive-deps
|
||||
useEffect(() => {
|
||||
if (!open) return;
|
||||
// fallbackOnError: recover into HOME if initialPath is bad, rather than
|
||||
|
|
@ -147,7 +139,7 @@ export function FolderBrowser({
|
|||
|
||||
const crumbs = useMemo(
|
||||
() => (data?.current ? splitBreadcrumb(data.current) : []),
|
||||
[data?.current],
|
||||
[data],
|
||||
);
|
||||
|
||||
return (
|
||||
|
|
@ -1,12 +1,12 @@
|
|||
// SPDX-License-Identifier: AGPL-3.0-only
|
||||
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
import { DeleteConfirmDialog } from "@/features/hub/catalog/download-card";
|
||||
import { DeleteConfirmDialog } from "@/features/hub";
|
||||
import { toast } from "@/lib/toast";
|
||||
import { cn } from "@/lib/utils";
|
||||
import { Delete02Icon } from "@hugeicons/core-free-icons";
|
||||
import { HugeiconsIcon } from "@hugeicons/react";
|
||||
import { useCallback, useState, type ReactNode } from "react";
|
||||
import { toast } from "@/lib/toast";
|
||||
import { type ReactNode, useCallback, useState } from "react";
|
||||
|
||||
interface ModelDeleteActionProps {
|
||||
ariaLabel: string;
|
||||
|
|
@ -63,7 +63,8 @@ export function ModelDeleteAction({
|
|||
disabled={disabled}
|
||||
className={cn(
|
||||
"shrink-0 rounded-md p-1.5 text-muted-foreground/60 transition-colors hover:bg-destructive/10 hover:text-destructive",
|
||||
disabled && "cursor-not-allowed opacity-40 hover:bg-transparent hover:text-muted-foreground/60",
|
||||
disabled &&
|
||||
"cursor-not-allowed opacity-40 hover:bg-transparent hover:text-muted-foreground/60",
|
||||
buttonClassName,
|
||||
)}
|
||||
>
|
||||
|
|
@ -6,24 +6,18 @@ import {
|
|||
TooltipContent,
|
||||
TooltipTrigger,
|
||||
} from "@/components/ui/tooltip";
|
||||
import { useChatRuntimeStore } from "@/features/chat/stores/chat-runtime-store";
|
||||
import { cn } from "@/lib/utils";
|
||||
import { Settings02Icon } from "@hugeicons/core-free-icons";
|
||||
import { HugeiconsIcon } from "@hugeicons/react";
|
||||
|
||||
/** Gear button on a downloaded quant row. Stages the model into the Run
|
||||
* settings sidebar (always, regardless of the Load-on-selection toggle) so the
|
||||
* user can set load options, then click Load model. */
|
||||
export function ModelLoadSettingsAction({
|
||||
ariaLabel,
|
||||
repoId,
|
||||
quant,
|
||||
maxContext,
|
||||
onConfigure,
|
||||
className,
|
||||
}: {
|
||||
ariaLabel: string;
|
||||
repoId: string;
|
||||
quant: string;
|
||||
maxContext?: number | null;
|
||||
onConfigure: () => void;
|
||||
className?: string;
|
||||
}) {
|
||||
return (
|
||||
<Tooltip delayDuration={0}>
|
||||
|
|
@ -32,16 +26,12 @@ export function ModelLoadSettingsAction({
|
|||
type="button"
|
||||
onClick={(e) => {
|
||||
e.stopPropagation();
|
||||
useChatRuntimeStore.getState().stageModel({
|
||||
id: repoId,
|
||||
ggufVariant: quant,
|
||||
isDownloaded: true,
|
||||
contextLength: maxContext ?? null,
|
||||
});
|
||||
onConfigure();
|
||||
}}
|
||||
aria-label={ariaLabel}
|
||||
className={cn(
|
||||
"shrink-0 rounded-md p-1 text-muted-foreground/60 transition-colors hover:bg-muted hover:text-foreground",
|
||||
"shrink-0 rounded-md p-1 text-muted-foreground/60 transition-colors hover:bg-black/5 hover:text-foreground dark:hover:bg-white/10",
|
||||
className,
|
||||
)}
|
||||
>
|
||||
<HugeiconsIcon
|
||||
|
|
@ -0,0 +1,289 @@
|
|||
// SPDX-License-Identifier: AGPL-3.0-only
|
||||
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
// Condensed row actions for model rows: everything except the run-settings
|
||||
// gear collapses into one dots menu (pin, update, delete) so rows don't grow
|
||||
// an icon strip. Mirrors the sidebar chat rows' MoreVertical menu pattern.
|
||||
|
||||
import {
|
||||
DropdownMenu,
|
||||
DropdownMenuContent,
|
||||
DropdownMenuItem,
|
||||
DropdownMenuSeparator,
|
||||
DropdownMenuTrigger,
|
||||
} from "@/components/ui/dropdown-menu";
|
||||
import { usePlatformStore } from "@/config/env";
|
||||
import { revealCachedModel } from "@/features/chat";
|
||||
import {
|
||||
DeleteConfirmDialog,
|
||||
UpdateConfirmDialog,
|
||||
ggufVariantsMatch,
|
||||
subscribeJobListeners,
|
||||
} from "@/features/hub";
|
||||
import { toast } from "@/lib/toast";
|
||||
import { cn } from "@/lib/utils";
|
||||
import {
|
||||
Delete02Icon,
|
||||
Folder01Icon,
|
||||
MoreVerticalIcon,
|
||||
PinIcon,
|
||||
PinOffIcon,
|
||||
} from "@hugeicons/core-free-icons";
|
||||
import { HugeiconsIcon } from "@hugeicons/react";
|
||||
import { RefreshCw } from "lucide-react";
|
||||
import {
|
||||
type ReactNode,
|
||||
useCallback,
|
||||
useEffect,
|
||||
useRef,
|
||||
useState,
|
||||
} from "react";
|
||||
|
||||
interface ModelRowMenuPin {
|
||||
pinned: boolean;
|
||||
/** Menu item labels, e.g. "Pin quant to the top" / "Unpin quant". */
|
||||
pinLabel: string;
|
||||
unpinLabel: string;
|
||||
onToggle: () => void;
|
||||
}
|
||||
|
||||
interface ModelRowMenuUpdate {
|
||||
title: string;
|
||||
description: ReactNode;
|
||||
/** Repo + variant the update targets (see ModelUpdateAction). */
|
||||
repoId: string;
|
||||
variant?: string | null;
|
||||
disabled?: boolean;
|
||||
onConfirm: () => Promise<void> | void;
|
||||
onUpdated?: () => void;
|
||||
}
|
||||
|
||||
interface ModelRowMenuDelete {
|
||||
title: string;
|
||||
description: ReactNode;
|
||||
successMessage: string;
|
||||
disabled?: boolean;
|
||||
onConfirm: () => Promise<void> | void;
|
||||
onDeleted?: () => void;
|
||||
}
|
||||
|
||||
/** Managed-cache location for "Reveal in Finder" (resolved server-side). */
|
||||
interface ModelRowMenuCachePath {
|
||||
repoId: string;
|
||||
variant?: string;
|
||||
}
|
||||
|
||||
export function ModelRowMenu({
|
||||
ariaLabel,
|
||||
buttonClassName,
|
||||
iconClassName,
|
||||
cachePath,
|
||||
pin,
|
||||
update,
|
||||
del,
|
||||
}: {
|
||||
ariaLabel: string;
|
||||
buttonClassName?: string;
|
||||
iconClassName?: string;
|
||||
/** Enables "Reveal in Finder" for cached repos. */
|
||||
cachePath?: ModelRowMenuCachePath;
|
||||
pin?: ModelRowMenuPin;
|
||||
update?: ModelRowMenuUpdate;
|
||||
del?: ModelRowMenuDelete;
|
||||
}) {
|
||||
const deviceType = usePlatformStore((s) => s.deviceType);
|
||||
const revealLabel =
|
||||
deviceType === "mac"
|
||||
? "Reveal in Finder"
|
||||
: deviceType === "windows"
|
||||
? "Reveal in File Explorer"
|
||||
: "Reveal in File Manager";
|
||||
const [deleteOpen, setDeleteOpen] = useState(false);
|
||||
const [deleting, setDeleting] = useState(false);
|
||||
const [updateOpen, setUpdateOpen] = useState(false);
|
||||
|
||||
// Refresh the caller when this repo+variant's managed update completes
|
||||
// (mirrors ModelUpdateAction).
|
||||
const onUpdatedRef = useRef(update?.onUpdated);
|
||||
useEffect(() => {
|
||||
onUpdatedRef.current = update?.onUpdated;
|
||||
}, [update?.onUpdated]);
|
||||
const updateRepoId = update?.repoId;
|
||||
const updateVariant = update?.variant ?? null;
|
||||
useEffect(() => {
|
||||
if (!updateRepoId) return;
|
||||
return subscribeJobListeners("model", updateRepoId, {
|
||||
onComplete: (completedVariant) => {
|
||||
const matches = updateVariant
|
||||
? ggufVariantsMatch(completedVariant, updateVariant)
|
||||
: !completedVariant;
|
||||
if (matches) onUpdatedRef.current?.();
|
||||
},
|
||||
});
|
||||
}, [updateRepoId, updateVariant]);
|
||||
|
||||
const onDeleteConfirm = del?.onConfirm;
|
||||
const onDeleted = del?.onDeleted;
|
||||
const deleteSuccessMessage = del?.successMessage;
|
||||
const handleDeleteConfirm = useCallback(async () => {
|
||||
if (!onDeleteConfirm) return;
|
||||
setDeleting(true);
|
||||
try {
|
||||
await onDeleteConfirm();
|
||||
if (deleteSuccessMessage) toast.success(deleteSuccessMessage);
|
||||
onDeleted?.();
|
||||
setDeleteOpen(false);
|
||||
} catch (err) {
|
||||
toast.error(
|
||||
err instanceof Error ? err.message : "Failed to delete model",
|
||||
);
|
||||
} finally {
|
||||
setDeleting(false);
|
||||
}
|
||||
}, [onDeleteConfirm, onDeleted, deleteSuccessMessage]);
|
||||
|
||||
const onUpdateConfirm = update?.onConfirm;
|
||||
const handleUpdateConfirm = useCallback(() => {
|
||||
// Start the re-download and close the dialog; the Downloads panel owns
|
||||
// progress + cancel. Only a failure to START toasts.
|
||||
void Promise.resolve()
|
||||
.then(onUpdateConfirm)
|
||||
.catch((err) => {
|
||||
toast.error(
|
||||
err instanceof Error ? err.message : "Failed to start update",
|
||||
);
|
||||
});
|
||||
setUpdateOpen(false);
|
||||
}, [onUpdateConfirm]);
|
||||
|
||||
const cachePathRepoId = cachePath?.repoId;
|
||||
const cachePathVariant = cachePath?.variant;
|
||||
const handleReveal = useCallback(() => {
|
||||
if (!cachePathRepoId) return;
|
||||
revealCachedModel(cachePathRepoId, cachePathVariant).catch((err) => {
|
||||
toast.error(
|
||||
err instanceof Error ? err.message : "Failed to open file manager",
|
||||
);
|
||||
});
|
||||
}, [cachePathRepoId, cachePathVariant]);
|
||||
|
||||
if (!pin && !update && !del && !cachePath) return null;
|
||||
|
||||
return (
|
||||
<>
|
||||
<DropdownMenu>
|
||||
<DropdownMenuTrigger asChild={true}>
|
||||
<button
|
||||
type="button"
|
||||
onClick={(e) => e.stopPropagation()}
|
||||
aria-label={ariaLabel}
|
||||
className={cn(
|
||||
"shrink-0 rounded-md p-1 text-muted-foreground/60 transition-colors hover:bg-black/5 hover:text-foreground dark:hover:bg-white/10",
|
||||
buttonClassName,
|
||||
)}
|
||||
>
|
||||
<HugeiconsIcon
|
||||
icon={MoreVerticalIcon}
|
||||
strokeWidth={1.75}
|
||||
className={cn("size-3.5", iconClassName)}
|
||||
/>
|
||||
</button>
|
||||
</DropdownMenuTrigger>
|
||||
<DropdownMenuContent
|
||||
side="bottom"
|
||||
align="end"
|
||||
sideOffset={2}
|
||||
className="unsloth-plus-menu menu-flat-destructive w-48"
|
||||
>
|
||||
{pin && (
|
||||
<DropdownMenuItem
|
||||
onSelect={(e) => {
|
||||
e.stopPropagation();
|
||||
pin.onToggle();
|
||||
}}
|
||||
>
|
||||
<HugeiconsIcon
|
||||
icon={pin.pinned ? PinOffIcon : PinIcon}
|
||||
strokeWidth={1.75}
|
||||
className="size-icon"
|
||||
/>
|
||||
<span>{pin.pinned ? pin.unpinLabel : pin.pinLabel}</span>
|
||||
</DropdownMenuItem>
|
||||
)}
|
||||
{cachePath && (
|
||||
<DropdownMenuItem
|
||||
onSelect={(e) => {
|
||||
e.stopPropagation();
|
||||
handleReveal();
|
||||
}}
|
||||
>
|
||||
<HugeiconsIcon
|
||||
icon={Folder01Icon}
|
||||
strokeWidth={1.75}
|
||||
className="size-icon"
|
||||
/>
|
||||
<span>{revealLabel}</span>
|
||||
</DropdownMenuItem>
|
||||
)}
|
||||
{update && (
|
||||
<DropdownMenuItem
|
||||
disabled={update.disabled}
|
||||
onSelect={(e) => {
|
||||
e.stopPropagation();
|
||||
setUpdateOpen(true);
|
||||
}}
|
||||
>
|
||||
<RefreshCw className="size-icon" />
|
||||
<span>Update</span>
|
||||
</DropdownMenuItem>
|
||||
)}
|
||||
{del && (
|
||||
<>
|
||||
{(cachePath || pin || update) && <DropdownMenuSeparator />}
|
||||
<DropdownMenuItem
|
||||
variant="destructive"
|
||||
disabled={del.disabled}
|
||||
onSelect={(e) => {
|
||||
e.stopPropagation();
|
||||
setDeleteOpen(true);
|
||||
}}
|
||||
>
|
||||
<HugeiconsIcon
|
||||
icon={Delete02Icon}
|
||||
strokeWidth={1.75}
|
||||
className="size-icon"
|
||||
/>
|
||||
<span>Delete</span>
|
||||
</DropdownMenuItem>
|
||||
</>
|
||||
)}
|
||||
</DropdownMenuContent>
|
||||
</DropdownMenu>
|
||||
|
||||
{del && (
|
||||
<DeleteConfirmDialog
|
||||
open={deleteOpen}
|
||||
onOpenChange={(nextOpen) => {
|
||||
if (!nextOpen && deleting) return;
|
||||
setDeleteOpen(nextOpen);
|
||||
}}
|
||||
title={del.title}
|
||||
description={del.description}
|
||||
deleting={deleting}
|
||||
onConfirm={() => void handleDeleteConfirm()}
|
||||
/>
|
||||
)}
|
||||
|
||||
{update && (
|
||||
<UpdateConfirmDialog
|
||||
open={updateOpen}
|
||||
onOpenChange={setUpdateOpen}
|
||||
title={update.title}
|
||||
description={update.description}
|
||||
updating={false}
|
||||
onConfirm={handleUpdateConfirm}
|
||||
/>
|
||||
)}
|
||||
</>
|
||||
);
|
||||
}
|
||||
|
|
@ -1,12 +1,20 @@
|
|||
// SPDX-License-Identifier: AGPL-3.0-only
|
||||
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
import { subscribeJobListeners } from "@/features/hub/download-manager";
|
||||
import { UpdateConfirmDialog } from "@/features/hub/catalog/download-card";
|
||||
import { ggufVariantsMatch } from "@/features/hub/lib/model-identity";
|
||||
import {
|
||||
UpdateConfirmDialog,
|
||||
ggufVariantsMatch,
|
||||
subscribeJobListeners,
|
||||
} from "@/features/hub";
|
||||
import { cn } from "@/lib/utils";
|
||||
import { RefreshCw } from "lucide-react";
|
||||
import { useCallback, useEffect, useRef, useState, type ReactNode } from "react";
|
||||
import {
|
||||
type ReactNode,
|
||||
useCallback,
|
||||
useEffect,
|
||||
useRef,
|
||||
useState,
|
||||
} from "react";
|
||||
import { toast } from "sonner";
|
||||
|
||||
interface ModelUpdateActionProps {
|
||||
|
|
@ -42,10 +50,10 @@ export function ModelUpdateAction({
|
|||
}: ModelUpdateActionProps) {
|
||||
const [open, setOpen] = useState(false);
|
||||
|
||||
// Refresh the caller when this repo+variant's download finishes so the "update available" cue
|
||||
// clears. A ref keeps the subscription stable across renders.
|
||||
const onUpdatedRef = useRef(onUpdated);
|
||||
onUpdatedRef.current = onUpdated;
|
||||
useEffect(() => {
|
||||
onUpdatedRef.current = onUpdated;
|
||||
}, [onUpdated]);
|
||||
useEffect(() => {
|
||||
return subscribeJobListeners("model", repoId, {
|
||||
onComplete: (completedVariant) => {
|
||||
|
|
@ -83,7 +91,8 @@ export function ModelUpdateAction({
|
|||
disabled={disabled}
|
||||
className={cn(
|
||||
"shrink-0 rounded-md p-1.5 text-muted-foreground/60 transition-colors hover:bg-amber-500/10 hover:text-amber-700 dark:hover:bg-amber-500/15 dark:hover:text-amber-300",
|
||||
disabled && "cursor-not-allowed opacity-40 hover:bg-transparent hover:text-muted-foreground/60",
|
||||
disabled &&
|
||||
"cursor-not-allowed opacity-40 hover:bg-transparent hover:text-muted-foreground/60",
|
||||
buttonClassName,
|
||||
)}
|
||||
>
|
||||
|
|
@ -40,7 +40,8 @@ export function loadedAt(times: ModelLoadTimes, id: string): number {
|
|||
export function useModelLoadTimes(currentValue?: string): ModelLoadTimes {
|
||||
const [times, setTimes] = useState<ModelLoadTimes>(() => readLoadTimes());
|
||||
useEffect(() => {
|
||||
if (currentValue) setTimes(recordModelLoaded(currentValue));
|
||||
if (!currentValue) return;
|
||||
queueMicrotask(() => setTimes(recordModelLoaded(currentValue)));
|
||||
}, [currentValue]);
|
||||
return times;
|
||||
}
|
||||
File diff suppressed because it is too large
Load diff
|
|
@ -78,7 +78,8 @@ export function PillTabs({
|
|||
onValueChange(tabs[next].value);
|
||||
e.currentTarget.parentElement
|
||||
?.querySelectorAll<HTMLElement>('button[role="tab"]')
|
||||
[next]?.focus();
|
||||
.item(next)
|
||||
?.focus();
|
||||
}}
|
||||
onClick={() => onValueChange(tab.value)}
|
||||
className={cn(
|
||||
|
|
@ -21,6 +21,13 @@ export interface PinnedQuantEntry {
|
|||
quant: string;
|
||||
}
|
||||
|
||||
export function makePinRank(
|
||||
pinned: readonly string[],
|
||||
): (key: string) => number {
|
||||
const pinIndex = new Map(pinned.map((key, index) => [key, index]));
|
||||
return (key) => pinIndex.get(key) ?? Number.MAX_SAFE_INTEGER;
|
||||
}
|
||||
|
||||
/** The pinned GGUF quants, in pin order. Plain repo pins are excluded. */
|
||||
export function pinnedQuantEntries(pinned: string[]): PinnedQuantEntry[] {
|
||||
const out: PinnedQuantEntry[] = [];
|
||||
|
|
@ -63,10 +70,20 @@ export const usePinnedModelsStore = create<PinnedModelsState>((set) => ({
|
|||
togglePinned: (repoId, quant) =>
|
||||
set((state) => {
|
||||
const key = pinKey(repoId, quant);
|
||||
// Newest pin first, so "Pin to top" literally lands on top of the
|
||||
// pinned group rather than under earlier pins.
|
||||
const next = state.pinned.includes(key)
|
||||
? state.pinned.filter((id) => id !== key)
|
||||
: [...state.pinned, key];
|
||||
: [key, ...state.pinned];
|
||||
writePinned(next);
|
||||
return { pinned: next };
|
||||
}),
|
||||
}));
|
||||
|
||||
if (typeof window !== "undefined") {
|
||||
window.addEventListener("storage", (event) => {
|
||||
if (event.key === KEY || event.key === null) {
|
||||
usePinnedModelsStore.setState({ pinned: readPinned() });
|
||||
}
|
||||
});
|
||||
}
|
||||
|
|
@ -64,9 +64,8 @@ export function matchesFormatFilter(
|
|||
}
|
||||
}
|
||||
|
||||
// First "<n>B" token in a repo id, e.g. "Qwen3-4B-GGUF" -> 4, "gpt-oss-20b" ->
|
||||
// 20, "Qwen3-30B-A3B" -> 30 (MoE total), "gemma-4-E4B" -> 4 (effective-param
|
||||
// "E" series). The digits must be bounded by a separator so we never read "16"
|
||||
// First "<n>B" token in a repo id, e.g. "Qwen3-30B-A3B" -> 30 (MoE total),
|
||||
// "gemma-4-E4B" -> 4. Digits must be separator-bounded so we never read "16"
|
||||
// from "bf16" or the "2" in "Kimi-K2".
|
||||
const PARAM_RE = /(?:^|[-_/. ])[eE]?(\d+(?:\.\d+)?)\s*[bB](?=$|[-_./ ])/;
|
||||
|
||||
|
|
@ -79,9 +78,8 @@ export function paramsFromId(id: string): number | undefined {
|
|||
return Number.isFinite(billions) && billions > 0 ? billions * 1e9 : undefined;
|
||||
}
|
||||
|
||||
// Smallest practical GGUF/MLX quant (~Q2_K, low-bit). The fit check asks whether
|
||||
// a model can run at all, so it uses this rather than a default 4-bit size; a
|
||||
// user with a smaller device can still pick a low-bit variant.
|
||||
// Smallest practical GGUF/MLX quant (~Q2_K). The fit check asks whether a model
|
||||
// can run at all, so it uses this rather than a default 4-bit size.
|
||||
const MIN_QUANT_BYTES_PER_PARAM = 0.4;
|
||||
|
||||
/** Rough on-disk bytes for the smallest practical quant of `params` weights. */
|
||||
|
|
@ -2,6 +2,7 @@
|
|||
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
import type { ReactNode } from "react";
|
||||
import type { PerModelConfig } from "../../model-config/per-model-config";
|
||||
|
||||
export interface ModelOption {
|
||||
id: string;
|
||||
|
|
@ -36,6 +37,19 @@ export interface ModelSelectorChangeMeta {
|
|||
/** Direct local .gguf file picked without a variant (custom folder / LM
|
||||
* Studio). Marks it as a GGUF source for the deferred-load staging flow. */
|
||||
isGguf?: boolean;
|
||||
config?: PerModelConfig;
|
||||
forceReload?: boolean;
|
||||
/** Native path token so an active-model reload can reopen a file-picked GGUF. */
|
||||
nativePathToken?: string;
|
||||
nativePathExpiresAtMs?: number | null;
|
||||
}
|
||||
|
||||
export interface ModelPickTarget {
|
||||
id: string;
|
||||
displayName: string;
|
||||
ggufVariant?: string | null;
|
||||
isGguf: boolean;
|
||||
meta: ModelSelectorChangeMeta;
|
||||
}
|
||||
|
||||
export interface DeletedModelRef {
|
||||
|
|
@ -0,0 +1,113 @@
|
|||
// SPDX-License-Identifier: AGPL-3.0-only
|
||||
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
import { cn } from "@/lib/utils";
|
||||
import { useRef, useState } from "react";
|
||||
|
||||
export function snapToStep(
|
||||
value: number,
|
||||
step: number,
|
||||
min?: number,
|
||||
max?: number,
|
||||
): number {
|
||||
const lo = min ?? Number.NEGATIVE_INFINITY;
|
||||
const hi = max ?? Number.POSITIVE_INFINITY;
|
||||
const clamped = Math.min(Math.max(value, lo), hi);
|
||||
const stepStr = String(step);
|
||||
const decimals = stepStr.includes(".") ? stepStr.split(".")[1].length : 0;
|
||||
const base = Number.isFinite(lo) ? lo : 0;
|
||||
const snapped = base + Math.round((clamped - base) / step) * step;
|
||||
const reclamped = Math.min(Math.max(snapped, lo), hi);
|
||||
return Number(reclamped.toFixed(decimals));
|
||||
}
|
||||
|
||||
function sanitizeNumeric(raw: string, allowNegative: boolean): string {
|
||||
const sign = allowNegative && raw.startsWith("-") ? "-" : "";
|
||||
const [head, ...rest] = raw.replace(/[^\d.]/g, "").split(".");
|
||||
const tail = rest.length > 0 ? `.${rest.join("")}` : "";
|
||||
return `${sign}${head}${tail}`;
|
||||
}
|
||||
|
||||
export function NumericValueInput({
|
||||
value,
|
||||
min,
|
||||
max,
|
||||
step,
|
||||
onChange,
|
||||
displayValue,
|
||||
className,
|
||||
ariaLabel,
|
||||
size: sizeAttr,
|
||||
disabled = false,
|
||||
}: {
|
||||
value: number;
|
||||
min?: number;
|
||||
max?: number;
|
||||
step: number;
|
||||
onChange: (v: number) => void;
|
||||
displayValue?: string;
|
||||
className?: string;
|
||||
ariaLabel?: string;
|
||||
size?: number;
|
||||
disabled?: boolean;
|
||||
}) {
|
||||
const [focused, setFocused] = useState(false);
|
||||
const [draft, setDraft] = useState("");
|
||||
const cancelBlurCommitRef = useRef(false);
|
||||
|
||||
const commit = (raw: string) => {
|
||||
const parsed = Number.parseFloat(raw);
|
||||
if (!Number.isFinite(parsed)) {
|
||||
return;
|
||||
}
|
||||
const final = snapToStep(parsed, step, min, max);
|
||||
if (final !== value) {
|
||||
onChange(final);
|
||||
}
|
||||
};
|
||||
|
||||
const displayed = focused ? draft : (displayValue ?? String(value));
|
||||
|
||||
return (
|
||||
<input
|
||||
type="text"
|
||||
inputMode="decimal"
|
||||
disabled={disabled}
|
||||
size={sizeAttr}
|
||||
style={{
|
||||
boxSizing: "content-box",
|
||||
width: `calc(${Math.max(displayed.length, 4)}ch + 2px)`,
|
||||
}}
|
||||
value={displayed}
|
||||
aria-label={ariaLabel}
|
||||
onFocus={(e) => {
|
||||
cancelBlurCommitRef.current = false;
|
||||
setDraft(String(value));
|
||||
setFocused(true);
|
||||
const target = e.currentTarget;
|
||||
requestAnimationFrame(() => target.select());
|
||||
}}
|
||||
onBlur={() => {
|
||||
if (cancelBlurCommitRef.current) {
|
||||
cancelBlurCommitRef.current = false;
|
||||
} else {
|
||||
commit(draft);
|
||||
}
|
||||
setFocused(false);
|
||||
}}
|
||||
onChange={(e) =>
|
||||
setDraft(sanitizeNumeric(e.target.value, (min ?? 0) < 0))
|
||||
}
|
||||
onKeyDown={(e) => {
|
||||
if (e.key === "Enter") {
|
||||
e.currentTarget.blur();
|
||||
} else if (e.key === "Escape") {
|
||||
cancelBlurCommitRef.current = true;
|
||||
setDraft(String(value));
|
||||
e.currentTarget.blur();
|
||||
}
|
||||
}}
|
||||
className={cn(className)}
|
||||
/>
|
||||
);
|
||||
}
|
||||
|
|
@ -0,0 +1,91 @@
|
|||
// SPDX-License-Identifier: AGPL-3.0-only
|
||||
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
import { useMemo } from "react";
|
||||
import { gpuFieldsSignature } from "../model-config/apply-per-model-config";
|
||||
import type { PerModelConfig } from "../model-config/per-model-config";
|
||||
import { ModelConfigPage } from "./model-config-page";
|
||||
import type { ModelPickTarget } from "./model-selector/types";
|
||||
|
||||
interface SidebarModelConfigProps {
|
||||
modelId: string;
|
||||
ggufVariant: string | null;
|
||||
isGguf: boolean;
|
||||
nativeContextLength: number | null;
|
||||
loadedContextLength: number | null;
|
||||
loadedConfig: PerModelConfig;
|
||||
onReload: (config: PerModelConfig) => void;
|
||||
}
|
||||
|
||||
const TRAILING_SEPARATORS = /[\\/]+$/;
|
||||
|
||||
function leafName(id: string): string {
|
||||
const trimmed = id.replace(TRAILING_SEPARATORS, "");
|
||||
const separator = Math.max(
|
||||
trimmed.lastIndexOf("/"),
|
||||
trimmed.lastIndexOf("\\"),
|
||||
);
|
||||
return separator >= 0 ? trimmed.slice(separator + 1) : trimmed;
|
||||
}
|
||||
|
||||
function hashString(value: string): number {
|
||||
let hash = 5381;
|
||||
for (let i = 0; i < value.length; i += 1) {
|
||||
hash = (Math.imul(hash, 33) ^ value.charCodeAt(i)) >>> 0;
|
||||
}
|
||||
return hash;
|
||||
}
|
||||
|
||||
function configSignature(config: PerModelConfig): string {
|
||||
return [
|
||||
config.customContextLength ?? "",
|
||||
config.maxSeqLength ?? "",
|
||||
config.kvCacheDtype ?? "",
|
||||
config.speculativeType ?? "",
|
||||
config.specDraftNMax ?? "",
|
||||
config.tensorParallel ? "1" : "0",
|
||||
config.chatTemplateOverride == null
|
||||
? ""
|
||||
: `${config.chatTemplateOverride.length}:${hashString(config.chatTemplateOverride)}`,
|
||||
gpuFieldsSignature(config),
|
||||
].join("|");
|
||||
}
|
||||
|
||||
export function SidebarModelConfig({
|
||||
modelId,
|
||||
ggufVariant,
|
||||
isGguf,
|
||||
nativeContextLength,
|
||||
loadedContextLength,
|
||||
loadedConfig,
|
||||
onReload,
|
||||
}: SidebarModelConfigProps) {
|
||||
const target = useMemo<ModelPickTarget>(() => {
|
||||
const leaf = leafName(modelId);
|
||||
return {
|
||||
id: modelId,
|
||||
displayName: ggufVariant ? `${leaf} · ${ggufVariant}` : leaf,
|
||||
ggufVariant,
|
||||
isGguf,
|
||||
meta: {
|
||||
source: "local",
|
||||
isLora: false,
|
||||
ggufVariant: ggufVariant ?? undefined,
|
||||
isGguf,
|
||||
isDownloaded: true,
|
||||
contextLength: nativeContextLength,
|
||||
},
|
||||
};
|
||||
}, [modelId, ggufVariant, isGguf, nativeContextLength]);
|
||||
|
||||
return (
|
||||
<ModelConfigPage
|
||||
key={`${modelId}::${ggufVariant ?? ""}::${configSignature(loadedConfig)}`}
|
||||
target={target}
|
||||
onRun={onReload}
|
||||
loadedConfig={loadedConfig}
|
||||
loadedContextLength={loadedContextLength}
|
||||
variant="sidebar"
|
||||
/>
|
||||
);
|
||||
}
|
||||
|
|
@ -0,0 +1,77 @@
|
|||
// SPDX-License-Identifier: AGPL-3.0-only
|
||||
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
import { isExternalModelId, useChatRuntimeStore } from "@/features/chat";
|
||||
import { useMemo } from "react";
|
||||
import type { PerModelConfig } from "../model-config/per-model-config";
|
||||
|
||||
export interface ActiveModelConfigState {
|
||||
checkpoint: string | null;
|
||||
isGguf: boolean;
|
||||
config: PerModelConfig | null;
|
||||
}
|
||||
|
||||
export function useActiveModelConfig(): ActiveModelConfigState {
|
||||
const checkpoint = useChatRuntimeStore((s) => s.params.checkpoint) || null;
|
||||
const maxSeqLength = useChatRuntimeStore((s) => s.params.maxSeqLength);
|
||||
const activeGgufVariant = useChatRuntimeStore((s) => s.activeGgufVariant);
|
||||
const ggufContextLength = useChatRuntimeStore((s) => s.ggufContextLength);
|
||||
const customContextLength = useChatRuntimeStore((s) => s.customContextLength);
|
||||
const kvCacheDtype = useChatRuntimeStore((s) => s.kvCacheDtype);
|
||||
const speculativeType = useChatRuntimeStore((s) => s.speculativeType);
|
||||
const specDraftNMax = useChatRuntimeStore((s) => s.specDraftNMax);
|
||||
const tensorParallel = useChatRuntimeStore((s) => s.tensorParallel);
|
||||
const chatTemplateOverride = useChatRuntimeStore(
|
||||
(s) => s.chatTemplateOverride,
|
||||
);
|
||||
const gpuMemoryMode = useChatRuntimeStore((s) => s.gpuMemoryMode);
|
||||
const gpuLayers = useChatRuntimeStore((s) => s.gpuLayers);
|
||||
const nCpuMoe = useChatRuntimeStore((s) => s.nCpuMoe);
|
||||
const selectedGpuIds = useChatRuntimeStore((s) => s.selectedGpuIds);
|
||||
|
||||
const isGguf =
|
||||
activeGgufVariant != null ||
|
||||
ggufContextLength != null ||
|
||||
(checkpoint?.toLowerCase().endsWith(".gguf") ?? false);
|
||||
|
||||
const config = useMemo<PerModelConfig | null>(() => {
|
||||
if (!checkpoint || isExternalModelId(checkpoint)) {
|
||||
return null;
|
||||
}
|
||||
const base: PerModelConfig = {
|
||||
customContextLength: customContextLength ?? null,
|
||||
maxSeqLength: isGguf ? null : maxSeqLength,
|
||||
kvCacheDtype: kvCacheDtype ?? null,
|
||||
speculativeType: speculativeType ?? "auto",
|
||||
specDraftNMax: specDraftNMax ?? null,
|
||||
tensorParallel: tensorParallel ?? false,
|
||||
chatTemplateOverride: chatTemplateOverride ?? null,
|
||||
};
|
||||
if (!isGguf) {
|
||||
return base;
|
||||
}
|
||||
return {
|
||||
...base,
|
||||
gpuMemoryMode,
|
||||
gpuLayers,
|
||||
nCpuMoe,
|
||||
selectedGpuIds,
|
||||
};
|
||||
}, [
|
||||
checkpoint,
|
||||
isGguf,
|
||||
maxSeqLength,
|
||||
customContextLength,
|
||||
kvCacheDtype,
|
||||
speculativeType,
|
||||
specDraftNMax,
|
||||
tensorParallel,
|
||||
chatTemplateOverride,
|
||||
gpuMemoryMode,
|
||||
gpuLayers,
|
||||
nCpuMoe,
|
||||
selectedGpuIds,
|
||||
]);
|
||||
|
||||
return { checkpoint, isGguf, config };
|
||||
}
|
||||
|
|
@ -0,0 +1,191 @@
|
|||
// SPDX-License-Identifier: AGPL-3.0-only
|
||||
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
import { useHfTokenStore, useInventoryVersion } from "@/features/hub";
|
||||
import { useEffect, useState } from "react";
|
||||
import { fetchModelMaxPositionEmbeddings } from "../api/model-metadata";
|
||||
import { fetchDefaultChatTemplate } from "../api/templates";
|
||||
|
||||
export interface DefaultChatTemplateState {
|
||||
template: string | null;
|
||||
loading: boolean;
|
||||
error: string | null;
|
||||
}
|
||||
|
||||
export interface ModelMaxPositionState {
|
||||
maxPositionEmbeddings: number | null;
|
||||
loading: boolean;
|
||||
error: string | null;
|
||||
}
|
||||
|
||||
const TEMPLATE_CACHE_MAX_ENTRIES = 50;
|
||||
const templateCache = new Map<string, string | null>();
|
||||
const maxPositionCache = new Map<string, number | null>();
|
||||
|
||||
function cacheTemplate(key: string, template: string | null): void {
|
||||
templateCache.delete(key);
|
||||
templateCache.set(key, template);
|
||||
while (templateCache.size > TEMPLATE_CACHE_MAX_ENTRIES) {
|
||||
const oldest = templateCache.keys().next().value;
|
||||
if (oldest === undefined) {
|
||||
break;
|
||||
}
|
||||
templateCache.delete(oldest);
|
||||
}
|
||||
}
|
||||
|
||||
function cacheMaxPosition(key: string, value: number | null): void {
|
||||
maxPositionCache.delete(key);
|
||||
maxPositionCache.set(key, value);
|
||||
while (maxPositionCache.size > TEMPLATE_CACHE_MAX_ENTRIES) {
|
||||
const oldest = maxPositionCache.keys().next().value;
|
||||
if (oldest === undefined) {
|
||||
break;
|
||||
}
|
||||
maxPositionCache.delete(oldest);
|
||||
}
|
||||
}
|
||||
|
||||
export function useDefaultChatTemplate(
|
||||
modelId: string | null,
|
||||
ggufVariant: string | null | undefined,
|
||||
enabled: boolean,
|
||||
nativePathToken?: string | null,
|
||||
): DefaultChatTemplateState {
|
||||
const token = useHfTokenStore((s) => s.token);
|
||||
const inventoryVersion = useInventoryVersion();
|
||||
// The native token is part of the identity: a picked GGUF resolves its
|
||||
// template through the lease, not the model id, so two picks of the same
|
||||
// basename must not share a cache entry.
|
||||
const cacheKey =
|
||||
enabled && modelId
|
||||
? `${modelId}::${ggufVariant ?? ""}::${token}::${inventoryVersion}::${nativePathToken ?? ""}`
|
||||
: null;
|
||||
const [fetched, setFetched] = useState<{
|
||||
key: string;
|
||||
state: DefaultChatTemplateState;
|
||||
} | null>(null);
|
||||
|
||||
useEffect(() => {
|
||||
if (cacheKey == null || !modelId || templateCache.has(cacheKey)) {
|
||||
return;
|
||||
}
|
||||
const controller = new AbortController();
|
||||
fetchDefaultChatTemplate(
|
||||
modelId,
|
||||
ggufVariant,
|
||||
token,
|
||||
controller.signal,
|
||||
nativePathToken,
|
||||
)
|
||||
.then((template) => {
|
||||
if (controller.signal.aborted) {
|
||||
return;
|
||||
}
|
||||
// Cache the terminal result, including a null "no default template",
|
||||
// so reopening the viewer for such a model reuses it instead of
|
||||
// re-running the backend/Hugging Face lookup every time.
|
||||
cacheTemplate(cacheKey, template);
|
||||
setFetched({
|
||||
key: cacheKey,
|
||||
state: { template, loading: false, error: null },
|
||||
});
|
||||
})
|
||||
.catch((err: unknown) => {
|
||||
if (controller.signal.aborted) {
|
||||
return;
|
||||
}
|
||||
setFetched({
|
||||
key: cacheKey,
|
||||
state: {
|
||||
template: null,
|
||||
loading: false,
|
||||
error:
|
||||
err instanceof Error ? err.message : "Failed to load template",
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
return () => controller.abort();
|
||||
}, [cacheKey, modelId, ggufVariant, token, nativePathToken]);
|
||||
|
||||
if (cacheKey == null) {
|
||||
return { template: null, loading: false, error: null };
|
||||
}
|
||||
if (templateCache.has(cacheKey)) {
|
||||
return {
|
||||
template: templateCache.get(cacheKey) ?? null,
|
||||
loading: false,
|
||||
error: null,
|
||||
};
|
||||
}
|
||||
if (fetched?.key === cacheKey) {
|
||||
return fetched.state;
|
||||
}
|
||||
return { template: null, loading: true, error: null };
|
||||
}
|
||||
|
||||
export function useModelMaxPositionEmbeddings(
|
||||
modelId: string | null,
|
||||
enabled: boolean,
|
||||
): ModelMaxPositionState {
|
||||
const token = useHfTokenStore((s) => s.token);
|
||||
const inventoryVersion = useInventoryVersion();
|
||||
const cacheKey =
|
||||
enabled && modelId ? `${modelId}::${token}::${inventoryVersion}` : null;
|
||||
const [fetched, setFetched] = useState<{
|
||||
key: string;
|
||||
state: ModelMaxPositionState;
|
||||
} | null>(null);
|
||||
|
||||
useEffect(() => {
|
||||
if (cacheKey == null || !modelId || maxPositionCache.has(cacheKey)) {
|
||||
return;
|
||||
}
|
||||
const controller = new AbortController();
|
||||
fetchModelMaxPositionEmbeddings(modelId, token, controller.signal)
|
||||
.then((maxPositionEmbeddings) => {
|
||||
if (controller.signal.aborted) {
|
||||
return;
|
||||
}
|
||||
cacheMaxPosition(cacheKey, maxPositionEmbeddings);
|
||||
setFetched({
|
||||
key: cacheKey,
|
||||
state: { maxPositionEmbeddings, loading: false, error: null },
|
||||
});
|
||||
})
|
||||
.catch((err: unknown) => {
|
||||
if (controller.signal.aborted) {
|
||||
return;
|
||||
}
|
||||
setFetched({
|
||||
key: cacheKey,
|
||||
state: {
|
||||
maxPositionEmbeddings: null,
|
||||
loading: false,
|
||||
error:
|
||||
err instanceof Error
|
||||
? err.message
|
||||
: "Failed to load model metadata",
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
return () => controller.abort();
|
||||
}, [cacheKey, modelId, token]);
|
||||
|
||||
if (cacheKey == null) {
|
||||
return { maxPositionEmbeddings: null, loading: false, error: null };
|
||||
}
|
||||
if (maxPositionCache.has(cacheKey)) {
|
||||
return {
|
||||
maxPositionEmbeddings: maxPositionCache.get(cacheKey) ?? null,
|
||||
loading: false,
|
||||
error: null,
|
||||
};
|
||||
}
|
||||
if (fetched?.key === cacheKey) {
|
||||
return fetched.state;
|
||||
}
|
||||
return { maxPositionEmbeddings: null, loading: true, error: null };
|
||||
}
|
||||
39
studio/frontend/src/features/model-picker/index.ts
Normal file
39
studio/frontend/src/features/model-picker/index.ts
Normal file
|
|
@ -0,0 +1,39 @@
|
|||
// SPDX-License-Identifier: AGPL-3.0-only
|
||||
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
export { ModelSelector } from "./components/model-selector";
|
||||
export { FolderBrowser } from "./components/model-selector/folder-browser";
|
||||
export { ModelRowMenu } from "./components/model-selector/model-row-menu";
|
||||
export {
|
||||
makePinRank,
|
||||
pinKey,
|
||||
usePinnedModelsStore,
|
||||
} from "./components/model-selector/pinned-models";
|
||||
export { hfModelFitsDevice } from "./components/model-selector/recommended-fit";
|
||||
export {
|
||||
NumericValueInput,
|
||||
snapToStep,
|
||||
} from "./components/numeric-value-input";
|
||||
export { SidebarModelConfig } from "./components/sidebar-model-config";
|
||||
export {
|
||||
useActiveModelConfig,
|
||||
} from "./hooks/use-active-model-config";
|
||||
export type {
|
||||
DeletedModelRef,
|
||||
ExternalModelOption,
|
||||
LoraModelOption,
|
||||
ModelOption,
|
||||
ModelSelectorChangeMeta,
|
||||
} from "./components/model-selector";
|
||||
export {
|
||||
applyModelLoadConfigToRuntime,
|
||||
applyPerModelConfigToRuntime,
|
||||
currentRuntimePerModelConfig,
|
||||
perModelConfigsEqual,
|
||||
} from "./model-config/apply-per-model-config";
|
||||
export {
|
||||
DEFAULT_MAX_SEQ_LENGTH,
|
||||
normalizeMaxSeqLength,
|
||||
type PerModelConfig,
|
||||
resolveInitialConfig,
|
||||
} from "./model-config/per-model-config";
|
||||
|
|
@ -0,0 +1,123 @@
|
|||
// SPDX-License-Identifier: AGPL-3.0-only
|
||||
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
import type {
|
||||
CachedGgufRepo,
|
||||
CachedModelRepo,
|
||||
LocalModelInfo,
|
||||
} from "@/features/chat";
|
||||
import {
|
||||
type CachedInventoryRow,
|
||||
type LocalInventoryRow,
|
||||
type LocalSource,
|
||||
isHiddenModelId,
|
||||
useHubInventory,
|
||||
} from "@/features/hub";
|
||||
import { useMemo } from "react";
|
||||
|
||||
const PICKER_LOCAL_SOURCES: ReadonlySet<LocalSource> = new Set([
|
||||
"lmstudio",
|
||||
"models_dir",
|
||||
"custom",
|
||||
]);
|
||||
|
||||
function isCompleteCachedRow(row: CachedInventoryRow): boolean {
|
||||
return !row.partial && !row.liveDownload;
|
||||
}
|
||||
|
||||
function toCachedGgufRepo(row: CachedInventoryRow): CachedGgufRepo {
|
||||
return {
|
||||
repo_id: row.repoId,
|
||||
size_bytes: row.bytes,
|
||||
cache_path: row.cachePath ?? "",
|
||||
last_modified: row.lastModified ?? undefined,
|
||||
has_vision: row.capabilities.supportsVision,
|
||||
};
|
||||
}
|
||||
|
||||
function toCachedModelRepo(row: CachedInventoryRow): CachedModelRepo {
|
||||
return {
|
||||
repo_id: row.repoId,
|
||||
size_bytes: row.bytes,
|
||||
last_modified: row.lastModified ?? undefined,
|
||||
};
|
||||
}
|
||||
|
||||
function toLocalModelInfo(row: LocalInventoryRow): LocalModelInfo {
|
||||
return {
|
||||
id: row.loadId,
|
||||
display_name: row.displayName ?? row.title,
|
||||
path: row.path,
|
||||
source: row.source as LocalModelInfo["source"],
|
||||
model_id: row.modelId ?? row.repoId,
|
||||
model_format: row.modelFormat,
|
||||
updated_at: row.updatedAt,
|
||||
};
|
||||
}
|
||||
|
||||
export interface ChatPickerInventory {
|
||||
cachedGguf: CachedGgufRepo[];
|
||||
cachedModels: CachedModelRepo[];
|
||||
cachedReady: boolean;
|
||||
localModels: LocalModelInfo[];
|
||||
refreshInventory: () => Promise<void>;
|
||||
}
|
||||
|
||||
export function useChatPickerInventory(
|
||||
options: { enabled?: boolean } = {},
|
||||
): ChatPickerInventory {
|
||||
const inventory = useHubInventory({
|
||||
kind: "models",
|
||||
enabled: options.enabled,
|
||||
includeLocal: true,
|
||||
});
|
||||
|
||||
const cachedGguf = useMemo(
|
||||
() =>
|
||||
inventory.cachedRows
|
||||
.filter(
|
||||
(row) =>
|
||||
row.modelFormat === "gguf" &&
|
||||
isCompleteCachedRow(row) &&
|
||||
!isHiddenModelId(row.repoId),
|
||||
)
|
||||
.map(toCachedGgufRepo),
|
||||
[inventory.cachedRows],
|
||||
);
|
||||
const cachedModels = useMemo(
|
||||
() =>
|
||||
inventory.cachedRows
|
||||
.filter(
|
||||
(row) =>
|
||||
row.modelFormat !== "gguf" &&
|
||||
isCompleteCachedRow(row) &&
|
||||
!isHiddenModelId(row.repoId),
|
||||
)
|
||||
.map(toCachedModelRepo),
|
||||
[inventory.cachedRows],
|
||||
);
|
||||
const localModels = useMemo(
|
||||
() =>
|
||||
inventory.localRows
|
||||
.filter(
|
||||
(row) =>
|
||||
PICKER_LOCAL_SOURCES.has(row.source) &&
|
||||
// Skip non-chat rows (e.g. a folder with only config.json is
|
||||
// classified "unknown" -> canChat false); selecting one would try to
|
||||
// load a weightless path. toLocalModelInfo drops capabilities, so
|
||||
// this is the only place the guard can live.
|
||||
row.capabilities.canChat &&
|
||||
!isHiddenModelId(row.modelId, row.repoId, row.path),
|
||||
)
|
||||
.map(toLocalModelInfo),
|
||||
[inventory.localRows],
|
||||
);
|
||||
|
||||
return {
|
||||
cachedGguf,
|
||||
cachedModels,
|
||||
cachedReady: inventory.downloadedReady,
|
||||
localModels,
|
||||
refreshInventory: inventory.refreshInventory,
|
||||
};
|
||||
}
|
||||
|
|
@ -0,0 +1,127 @@
|
|||
// SPDX-License-Identifier: AGPL-3.0-only
|
||||
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
import {
|
||||
GPU_LAYERS_AUTO,
|
||||
defaultInferenceParams,
|
||||
normalizeSpeculativeType,
|
||||
readPersistedGpuMemoryMode,
|
||||
readPersistedSpeculativeType,
|
||||
reconcilePersistedGpuIds,
|
||||
useChatRuntimeStore,
|
||||
} from "@/features/chat";
|
||||
import {
|
||||
DEFAULT_PER_MODEL_CONFIG,
|
||||
type PerModelConfig,
|
||||
normalizeMaxSeqLength,
|
||||
} from "./per-model-config";
|
||||
|
||||
function cleanTemplate(value: string | null | undefined): string | null {
|
||||
return value?.trim() ? value : null;
|
||||
}
|
||||
|
||||
export function applyPerModelConfigToRuntime(config: PerModelConfig): void {
|
||||
// Fall back to the standing default when the model has no saved
|
||||
// maxSeqLength. maxSeqLength is the only per-model field carried on
|
||||
// params (the rest are reset below), so without this a model with no
|
||||
// remembered config would inherit the previously loaded model's value.
|
||||
const maxSeqLength =
|
||||
normalizeMaxSeqLength(config.maxSeqLength) ??
|
||||
defaultInferenceParams.maxSeqLength;
|
||||
const store = useChatRuntimeStore.getState();
|
||||
if (maxSeqLength !== store.params.maxSeqLength) {
|
||||
store.setParams({ ...store.params, maxSeqLength });
|
||||
}
|
||||
useChatRuntimeStore.setState({
|
||||
customContextLength: config.customContextLength ?? null,
|
||||
kvCacheDtype: config.kvCacheDtype ?? null,
|
||||
speculativeType:
|
||||
normalizeSpeculativeType(config.speculativeType) ??
|
||||
readPersistedSpeculativeType(),
|
||||
specDraftNMax: config.specDraftNMax ?? null,
|
||||
tensorParallel: config.tensorParallel ?? false,
|
||||
chatTemplateOverride: cleanTemplate(config.chatTemplateOverride),
|
||||
// GPU Memory knobs are per-model (GGUF-only). Absent = defaults; the mode is
|
||||
// a standing preference so an absent mode falls back to the persisted one.
|
||||
// The per-GPU split ratio is never remembered, so it always resets. The GPU
|
||||
// pick is reconciled against the GPUs present now (a saved [1] on a 1-GPU
|
||||
// host would otherwise be sent and rejected).
|
||||
gpuMemoryMode: config.gpuMemoryMode ?? readPersistedGpuMemoryMode(),
|
||||
gpuLayers: config.gpuLayers ?? GPU_LAYERS_AUTO,
|
||||
nCpuMoe: config.nCpuMoe ?? 0,
|
||||
splitRatio: null,
|
||||
selectedGpuIds:
|
||||
config.selectedGpuIds !== undefined
|
||||
? reconcilePersistedGpuIds(config.selectedGpuIds)
|
||||
: null,
|
||||
});
|
||||
}
|
||||
|
||||
export function applyModelLoadConfigToRuntime(
|
||||
config: PerModelConfig | null | undefined,
|
||||
): boolean {
|
||||
const hasConfig = config != null;
|
||||
applyPerModelConfigToRuntime(config ?? DEFAULT_PER_MODEL_CONFIG);
|
||||
return hasConfig;
|
||||
}
|
||||
|
||||
export function currentRuntimePerModelConfig(
|
||||
options: { includeMaxSeqLength?: boolean } = {},
|
||||
): PerModelConfig {
|
||||
const s = useChatRuntimeStore.getState();
|
||||
return {
|
||||
customContextLength: s.customContextLength ?? null,
|
||||
maxSeqLength: options.includeMaxSeqLength
|
||||
? normalizeMaxSeqLength(s.params.maxSeqLength)
|
||||
: null,
|
||||
kvCacheDtype: s.kvCacheDtype ?? null,
|
||||
speculativeType: normalizeSpeculativeType(s.speculativeType),
|
||||
specDraftNMax: s.specDraftNMax ?? null,
|
||||
tensorParallel: s.tensorParallel ?? false,
|
||||
chatTemplateOverride: cleanTemplate(s.chatTemplateOverride),
|
||||
// Snapshot the live GPU knobs too so a failed switch rolls the previous
|
||||
// model's GPU Memory settings back (see applyPerModelConfigToRuntime). The
|
||||
// split ratio is intentionally never remembered.
|
||||
gpuMemoryMode: s.gpuMemoryMode,
|
||||
gpuLayers: s.gpuLayers,
|
||||
nCpuMoe: s.nCpuMoe,
|
||||
selectedGpuIds: s.selectedGpuIds,
|
||||
};
|
||||
}
|
||||
|
||||
export function perModelConfigsEqual(
|
||||
a: PerModelConfig,
|
||||
b: PerModelConfig,
|
||||
): boolean {
|
||||
return (
|
||||
(a.customContextLength ?? null) === (b.customContextLength ?? null) &&
|
||||
normalizeMaxSeqLength(a.maxSeqLength) ===
|
||||
normalizeMaxSeqLength(b.maxSeqLength) &&
|
||||
(a.kvCacheDtype ?? null) === (b.kvCacheDtype ?? null) &&
|
||||
normalizeSpeculativeType(a.speculativeType) ===
|
||||
normalizeSpeculativeType(b.speculativeType) &&
|
||||
(a.specDraftNMax ?? null) === (b.specDraftNMax ?? null) &&
|
||||
Boolean(a.tensorParallel) === Boolean(b.tensorParallel) &&
|
||||
cleanTemplate(a.chatTemplateOverride) ===
|
||||
cleanTemplate(b.chatTemplateOverride) &&
|
||||
gpuFieldsEqual(a, b)
|
||||
);
|
||||
}
|
||||
|
||||
// Serialize the per-model GPU knobs with the same "absent == default"
|
||||
// coalescing the store applies: mode auto/absent, gpuLayers Auto (< 0) /
|
||||
// absent, nCpuMoe 0 / absent, and the GPU pick (null / absent = all GPUs).
|
||||
export function gpuFieldsSignature(config: PerModelConfig): string {
|
||||
return [
|
||||
config.gpuMemoryMode ?? "auto",
|
||||
config.gpuLayers == null || config.gpuLayers < 0 ? -1 : config.gpuLayers,
|
||||
config.nCpuMoe ?? 0,
|
||||
config.selectedGpuIds == null
|
||||
? "all"
|
||||
: [...config.selectedGpuIds].sort((a, b) => a - b).join(","),
|
||||
].join("|");
|
||||
}
|
||||
|
||||
function gpuFieldsEqual(a: PerModelConfig, b: PerModelConfig): boolean {
|
||||
return gpuFieldsSignature(a) === gpuFieldsSignature(b);
|
||||
}
|
||||
|
|
@ -0,0 +1,69 @@
|
|||
// SPDX-License-Identifier: AGPL-3.0-only
|
||||
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
import {
|
||||
normalizeGgufVariantIdentity,
|
||||
normalizeModelIdentity,
|
||||
} from "@/features/hub";
|
||||
|
||||
export {
|
||||
normalizeGgufVariantIdentity,
|
||||
normalizeModelIdentity,
|
||||
} from "@/features/hub";
|
||||
|
||||
const MODEL_STORAGE_KEY_PREFIX = "v2:";
|
||||
|
||||
type ParsedModelStorageKey = {
|
||||
modelId: string;
|
||||
ggufVariant: string;
|
||||
};
|
||||
|
||||
function parseVersionedModelStorageKey(
|
||||
key: string,
|
||||
): ParsedModelStorageKey | null {
|
||||
if (!key.startsWith(MODEL_STORAGE_KEY_PREFIX)) {
|
||||
return null;
|
||||
}
|
||||
try {
|
||||
const parsed = JSON.parse(key.slice(MODEL_STORAGE_KEY_PREFIX.length));
|
||||
if (
|
||||
!Array.isArray(parsed) ||
|
||||
parsed.length !== 2 ||
|
||||
typeof parsed[0] !== "string" ||
|
||||
typeof parsed[1] !== "string"
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
return { modelId: parsed[0], ggufVariant: parsed[1] };
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
export function modelStorageKey(
|
||||
modelId: string,
|
||||
ggufVariant?: string | null,
|
||||
): string {
|
||||
return `${MODEL_STORAGE_KEY_PREFIX}${JSON.stringify([
|
||||
normalizeModelIdentity(modelId),
|
||||
normalizeGgufVariantIdentity(ggufVariant),
|
||||
])}`;
|
||||
}
|
||||
|
||||
export function modelIdFromStorageKey(key: string): string | null {
|
||||
const parsed = parseVersionedModelStorageKey(key);
|
||||
if (parsed) {
|
||||
return parsed.modelId;
|
||||
}
|
||||
const separator = key.lastIndexOf("::");
|
||||
return separator >= 0 ? key.slice(0, separator) : null;
|
||||
}
|
||||
|
||||
export function ggufVariantFromStorageKey(key: string): string | null {
|
||||
const parsed = parseVersionedModelStorageKey(key);
|
||||
if (parsed) {
|
||||
return parsed.ggufVariant;
|
||||
}
|
||||
const separator = key.lastIndexOf("::");
|
||||
return separator >= 0 ? key.slice(separator + 2) : null;
|
||||
}
|
||||
|
|
@ -0,0 +1,665 @@
|
|||
// SPDX-License-Identifier: AGPL-3.0-only
|
||||
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
import {
|
||||
ggufVariantFromStorageKey,
|
||||
modelIdFromStorageKey,
|
||||
modelStorageKey,
|
||||
normalizeGgufVariantIdentity,
|
||||
normalizeModelIdentity,
|
||||
} from "./model-identity";
|
||||
|
||||
export interface PerModelConfig {
|
||||
customContextLength: number | null;
|
||||
maxSeqLength: number | null;
|
||||
kvCacheDtype: string | null;
|
||||
speculativeType: string | null;
|
||||
specDraftNMax: number | null;
|
||||
tensorParallel: boolean;
|
||||
chatTemplateOverride: string | null;
|
||||
// GPU Memory controls (per-model, GGUF-only), optional so older blobs still
|
||||
// parse. null selectedGpuIds (all GPUs) is distinct from absent. The --tensor-split
|
||||
// ratio is deliberately not remembered: it is positionally bound to the exact
|
||||
// GPU set/order and unvalidated.
|
||||
gpuMemoryMode?: "auto" | "manual";
|
||||
gpuLayers?: number;
|
||||
nCpuMoe?: number;
|
||||
selectedGpuIds?: number[] | null;
|
||||
}
|
||||
|
||||
export const DEFAULT_PER_MODEL_CONFIG: PerModelConfig = {
|
||||
customContextLength: null,
|
||||
maxSeqLength: null,
|
||||
kvCacheDtype: null,
|
||||
speculativeType: null,
|
||||
specDraftNMax: null,
|
||||
tensorParallel: false,
|
||||
chatTemplateOverride: null,
|
||||
};
|
||||
|
||||
export const MAX_SEQ_LENGTH_MIN = 128;
|
||||
export const MAX_SEQ_LENGTH_MAX = 1048576;
|
||||
export const MAX_SEQ_LENGTH_STEP = 128;
|
||||
// App-default max sequence length when a non-GGUF model has no override. Both
|
||||
// paths fall back to this rather than an active model's runtime value, so an
|
||||
// unconfigured pane never inherits another model's larger context and OOMs.
|
||||
export const DEFAULT_MAX_SEQ_LENGTH = 4096;
|
||||
export const CONTEXT_LENGTH_MIN = 128;
|
||||
|
||||
export const KV_CACHE_DTYPES = ["bf16", "q8_0", "q5_1", "q4_1"] as const;
|
||||
const VALID_KV_CACHE_DTYPES = new Set<string>(KV_CACHE_DTYPES);
|
||||
|
||||
export const SPECULATIVE_TYPES = [
|
||||
"auto",
|
||||
"mtp",
|
||||
"ngram",
|
||||
"mtp+ngram",
|
||||
"off",
|
||||
] as const;
|
||||
export const MTP_SPECULATIVE_TYPES: ReadonlySet<string> = new Set([
|
||||
"mtp",
|
||||
"mtp+ngram",
|
||||
]);
|
||||
|
||||
const STORAGE_KEY = "unsloth_model_configs";
|
||||
const LEGACY_STORAGE_KEY = "unsloth_load_settings";
|
||||
const LEGACY_MIGRATION_FLAG = "unsloth_model_configs_migrated";
|
||||
const STORAGE_SCHEMA_VERSION = 1;
|
||||
const MAX_ENTRIES = 500;
|
||||
const MAX_PER_MODEL_CONFIG_STORAGE_BYTES = 1024 * 1024;
|
||||
export const MAX_CHAT_TEMPLATE_BYTES = 65_536;
|
||||
|
||||
type StoredPerModelConfig = PerModelConfig & {
|
||||
version: typeof STORAGE_SCHEMA_VERSION;
|
||||
};
|
||||
type StoredMap = Record<string, PerModelConfig | StoredPerModelConfig>;
|
||||
type RawConfig = Partial<PerModelConfig> & { version?: unknown };
|
||||
|
||||
const STORED_CONFIG_FIELDS = new Set([
|
||||
"version",
|
||||
"customContextLength",
|
||||
"maxSeqLength",
|
||||
"kvCacheDtype",
|
||||
"speculativeType",
|
||||
"specDraftNMax",
|
||||
"tensorParallel",
|
||||
"chatTemplateOverride",
|
||||
"gpuMemoryMode",
|
||||
"gpuLayers",
|
||||
"nCpuMoe",
|
||||
"selectedGpuIds",
|
||||
]);
|
||||
|
||||
function normalizeGpuFields(partial: RawConfig): {
|
||||
gpuMemoryMode?: "auto" | "manual";
|
||||
gpuLayers?: number;
|
||||
nCpuMoe?: number;
|
||||
selectedGpuIds?: number[] | null;
|
||||
} {
|
||||
const out: {
|
||||
gpuMemoryMode?: "auto" | "manual";
|
||||
gpuLayers?: number;
|
||||
nCpuMoe?: number;
|
||||
selectedGpuIds?: number[] | null;
|
||||
} = {};
|
||||
// Only "manual" is a real override; persisting "auto" would pin the model and
|
||||
// stop it following later changes to the global GPU Memory preference.
|
||||
if (partial.gpuMemoryMode === "manual") {
|
||||
out.gpuMemoryMode = "manual";
|
||||
}
|
||||
if (
|
||||
typeof partial.gpuLayers === "number" &&
|
||||
Number.isFinite(partial.gpuLayers)
|
||||
) {
|
||||
out.gpuLayers = Math.trunc(partial.gpuLayers);
|
||||
}
|
||||
if (
|
||||
typeof partial.nCpuMoe === "number" &&
|
||||
Number.isFinite(partial.nCpuMoe) &&
|
||||
partial.nCpuMoe >= 0
|
||||
) {
|
||||
out.nCpuMoe = Math.trunc(partial.nCpuMoe);
|
||||
}
|
||||
if (partial.selectedGpuIds === null) {
|
||||
out.selectedGpuIds = null;
|
||||
} else if (
|
||||
Array.isArray(partial.selectedGpuIds) &&
|
||||
partial.selectedGpuIds.every(
|
||||
(n) => typeof n === "number" && Number.isFinite(n),
|
||||
)
|
||||
) {
|
||||
out.selectedGpuIds = partial.selectedGpuIds.map((n) => Math.trunc(n));
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
function canonicalizeSpeculativeType(value: string): string | null {
|
||||
const s = value.trim().toLowerCase();
|
||||
if (!s) {
|
||||
return null;
|
||||
}
|
||||
// "auto"/"default" is the follow-global sentinel; store as null so it is never
|
||||
// persisted as an override and global speculative-decoding changes keep applying.
|
||||
if (s === "auto" || s === "default") {
|
||||
return null;
|
||||
}
|
||||
if (s === "off") {
|
||||
return "off";
|
||||
}
|
||||
if (s === "mtp" || s === "draft-mtp") {
|
||||
return "mtp";
|
||||
}
|
||||
if (s === "ngram" || s === "ngram-mod" || s === "ngram-simple") {
|
||||
return "ngram";
|
||||
}
|
||||
if (s === "mtp+ngram") {
|
||||
return "mtp+ngram";
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
export function normalizeMaxSeqLength(value: unknown): number | null {
|
||||
if (typeof value !== "number" || !Number.isFinite(value) || value <= 0) {
|
||||
return null;
|
||||
}
|
||||
const snapped = Math.round(value / MAX_SEQ_LENGTH_STEP) * MAX_SEQ_LENGTH_STEP;
|
||||
return Math.max(MAX_SEQ_LENGTH_MIN, Math.min(MAX_SEQ_LENGTH_MAX, snapped));
|
||||
}
|
||||
|
||||
export function floorMaxSeqLength(value: unknown): number | null {
|
||||
if (typeof value !== "number" || !Number.isFinite(value) || value <= 0) {
|
||||
return null;
|
||||
}
|
||||
const snapped = Math.floor(value / MAX_SEQ_LENGTH_STEP) * MAX_SEQ_LENGTH_STEP;
|
||||
return Math.max(MAX_SEQ_LENGTH_MIN, Math.min(MAX_SEQ_LENGTH_MAX, snapped));
|
||||
}
|
||||
|
||||
function canUseStorage(): boolean {
|
||||
return typeof window !== "undefined";
|
||||
}
|
||||
|
||||
function serializedByteLength(value: string): number {
|
||||
return typeof TextEncoder !== "undefined"
|
||||
? new TextEncoder().encode(value).byteLength
|
||||
: value.length;
|
||||
}
|
||||
|
||||
export function chatTemplateByteLength(value: string): number {
|
||||
return serializedByteLength(value);
|
||||
}
|
||||
|
||||
export function isChatTemplateWithinLimit(value: string): boolean {
|
||||
return chatTemplateByteLength(value) <= MAX_CHAT_TEMPLATE_BYTES;
|
||||
}
|
||||
|
||||
function serializedMapSize(map: StoredMap): number {
|
||||
return serializedByteLength(JSON.stringify(map));
|
||||
}
|
||||
|
||||
function serializedMapEntrySize(key: string, value: StoredMap[string]): number {
|
||||
return (
|
||||
serializedByteLength(JSON.stringify(key)) +
|
||||
1 +
|
||||
serializedByteLength(JSON.stringify(value))
|
||||
);
|
||||
}
|
||||
|
||||
function deleteOldestEvictableEntry(
|
||||
map: StoredMap,
|
||||
protectedKeys?: ReadonlySet<string>,
|
||||
): { key: string; value: StoredMap[string] } | null {
|
||||
for (const key of Object.keys(map)) {
|
||||
// Never evict a future-schema entry an older client cannot interpret.
|
||||
if (
|
||||
protectedKeys?.has(key) ||
|
||||
storedConfigVersion(map[key]) > STORAGE_SCHEMA_VERSION
|
||||
) {
|
||||
continue;
|
||||
}
|
||||
const value = map[key];
|
||||
delete map[key];
|
||||
return { key, value };
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
function enforceStorageBudget(
|
||||
map: StoredMap,
|
||||
protectedKeys?: ReadonlySet<string>,
|
||||
): boolean {
|
||||
let entryCount = Object.keys(map).length;
|
||||
while (entryCount > MAX_ENTRIES) {
|
||||
if (!deleteOldestEvictableEntry(map, protectedKeys)) {
|
||||
return false;
|
||||
}
|
||||
entryCount -= 1;
|
||||
}
|
||||
let bytes = serializedMapSize(map);
|
||||
while (bytes > MAX_PER_MODEL_CONFIG_STORAGE_BYTES) {
|
||||
const removed = deleteOldestEvictableEntry(map, protectedKeys);
|
||||
if (!removed) {
|
||||
return false;
|
||||
}
|
||||
bytes -=
|
||||
serializedMapEntrySize(removed.key, removed.value) +
|
||||
(entryCount > 1 ? 1 : 0);
|
||||
entryCount -= 1;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
function storedConfigVersion(raw: unknown): number {
|
||||
if (!raw || typeof raw !== "object" || Array.isArray(raw)) {
|
||||
return 0;
|
||||
}
|
||||
const version = (raw as RawConfig).version;
|
||||
return typeof version === "number" && Number.isFinite(version) ? version : 0;
|
||||
}
|
||||
|
||||
let legacyMigrationChecked = false;
|
||||
|
||||
function parseLegacyModelKey(
|
||||
key: string,
|
||||
): { modelId: string; ggufVariant: string | null } | null {
|
||||
const separator = key.lastIndexOf("::");
|
||||
if (separator >= 0) {
|
||||
const modelId = key.slice(0, separator);
|
||||
return modelId
|
||||
? { modelId, ggufVariant: key.slice(separator + 2) || null }
|
||||
: null;
|
||||
}
|
||||
return key ? { modelId: key, ggufVariant: null } : null;
|
||||
}
|
||||
|
||||
function legacyEntryToConfig(raw: Record<string, unknown>): PerModelConfig {
|
||||
return normalizeV1({
|
||||
customContextLength:
|
||||
typeof raw.contextLength === "number" ? raw.contextLength : null,
|
||||
maxSeqLength: null,
|
||||
kvCacheDtype:
|
||||
typeof raw.kvCacheDtype === "string" ? raw.kvCacheDtype : null,
|
||||
speculativeType:
|
||||
typeof raw.speculativeType === "string" ? raw.speculativeType : null,
|
||||
specDraftNMax:
|
||||
typeof raw.specDraftNMax === "number" ? raw.specDraftNMax : null,
|
||||
tensorParallel:
|
||||
typeof raw.tensorParallel === "boolean" ? raw.tensorParallel : false,
|
||||
chatTemplateOverride: null,
|
||||
// Carry legacy GPU Memory knobs; normalizeGpuFields drops anything malformed.
|
||||
gpuMemoryMode:
|
||||
raw.gpuMemoryMode === "auto" || raw.gpuMemoryMode === "manual"
|
||||
? raw.gpuMemoryMode
|
||||
: undefined,
|
||||
gpuLayers: typeof raw.gpuLayers === "number" ? raw.gpuLayers : undefined,
|
||||
nCpuMoe: typeof raw.nCpuMoe === "number" ? raw.nCpuMoe : undefined,
|
||||
selectedGpuIds:
|
||||
raw.selectedGpuIds === null
|
||||
? null
|
||||
: Array.isArray(raw.selectedGpuIds)
|
||||
? (raw.selectedGpuIds as number[])
|
||||
: undefined,
|
||||
});
|
||||
}
|
||||
|
||||
function mergeLegacyEntries(
|
||||
map: StoredMap,
|
||||
legacy: Record<string, unknown>,
|
||||
): string[] {
|
||||
const addedKeys: string[] = [];
|
||||
for (const [legacyKey, value] of Object.entries(legacy)) {
|
||||
if (!value || typeof value !== "object") {
|
||||
continue;
|
||||
}
|
||||
const parsedKey = parseLegacyModelKey(legacyKey);
|
||||
if (!parsedKey) {
|
||||
continue;
|
||||
}
|
||||
const migrated = legacyEntryToConfig(value as Record<string, unknown>);
|
||||
const key = modelStorageKey(parsedKey.modelId, parsedKey.ggufVariant);
|
||||
if (isDefaultConfig(migrated) || Object.hasOwn(map, key)) {
|
||||
continue;
|
||||
}
|
||||
map[key] = toStoredConfig(migrated);
|
||||
addedKeys.push(key);
|
||||
}
|
||||
return addedKeys;
|
||||
}
|
||||
|
||||
function migrateLegacyLoadSettingsOnce(): void {
|
||||
if (legacyMigrationChecked || !canUseStorage()) {
|
||||
return;
|
||||
}
|
||||
legacyMigrationChecked = true;
|
||||
try {
|
||||
if (localStorage.getItem(LEGACY_MIGRATION_FLAG)) {
|
||||
return;
|
||||
}
|
||||
let legacy: unknown = null;
|
||||
try {
|
||||
legacy = JSON.parse(localStorage.getItem(LEGACY_STORAGE_KEY) ?? "null");
|
||||
} catch {
|
||||
legacy = null;
|
||||
}
|
||||
if (!legacy || typeof legacy !== "object" || Array.isArray(legacy)) {
|
||||
localStorage.setItem(LEGACY_MIGRATION_FLAG, "1");
|
||||
return;
|
||||
}
|
||||
const map = readMapRaw();
|
||||
// Snapshot existing entries so eviction can protect them: importing old load
|
||||
// settings must never discard a newer per-model config the user already has.
|
||||
const existingKeys = new Set(Object.keys(map));
|
||||
const migratedKeys = mergeLegacyEntries(
|
||||
map,
|
||||
legacy as Record<string, unknown>,
|
||||
);
|
||||
if (migratedKeys.length === 0) {
|
||||
localStorage.setItem(LEGACY_MIGRATION_FLAG, "1");
|
||||
return;
|
||||
}
|
||||
// Protect pre-existing entries so only just-migrated legacy entries are
|
||||
// dropped when over budget.
|
||||
if (!enforceStorageBudget(map, existingKeys)) {
|
||||
return;
|
||||
}
|
||||
if (writeMap(map)) {
|
||||
localStorage.setItem(LEGACY_MIGRATION_FLAG, "1");
|
||||
}
|
||||
} catch (err) {
|
||||
console.warn("Failed to migrate legacy load settings:", err);
|
||||
}
|
||||
}
|
||||
|
||||
function readMapRaw(): StoredMap {
|
||||
if (!canUseStorage()) {
|
||||
return {};
|
||||
}
|
||||
try {
|
||||
const raw = localStorage.getItem(STORAGE_KEY);
|
||||
if (!raw) {
|
||||
return {};
|
||||
}
|
||||
const parsed = JSON.parse(raw);
|
||||
if (!parsed || typeof parsed !== "object" || Array.isArray(parsed)) {
|
||||
return {};
|
||||
}
|
||||
return parsed as StoredMap;
|
||||
} catch {
|
||||
return {};
|
||||
}
|
||||
}
|
||||
|
||||
function readMap(): StoredMap {
|
||||
migrateLegacyLoadSettingsOnce();
|
||||
return readMapRaw();
|
||||
}
|
||||
|
||||
function writeMap(map: StoredMap): boolean {
|
||||
if (!canUseStorage()) {
|
||||
return false;
|
||||
}
|
||||
try {
|
||||
localStorage.setItem(STORAGE_KEY, JSON.stringify(map));
|
||||
return true;
|
||||
} catch (err) {
|
||||
console.warn("Failed to persist per-model config:", err);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
function warnDroppedFields(raw: Record<string, unknown>, version: number): void {
|
||||
if (!import.meta.env?.DEV) {
|
||||
return;
|
||||
}
|
||||
const dropped = Object.keys(raw).filter(
|
||||
(key) => !STORED_CONFIG_FIELDS.has(key),
|
||||
);
|
||||
if (dropped.length > 0) {
|
||||
console.warn("Dropped unknown per-model config fields:", dropped);
|
||||
}
|
||||
if (version > STORAGE_SCHEMA_VERSION) {
|
||||
console.warn("Per-model config schema is newer than this app:", version);
|
||||
}
|
||||
}
|
||||
|
||||
function normalizeV1(partial: RawConfig): PerModelConfig {
|
||||
const rawSpecType =
|
||||
typeof partial.speculativeType === "string"
|
||||
? canonicalizeSpeculativeType(partial.speculativeType)
|
||||
: null;
|
||||
const speculativeType = rawSpecType ?? DEFAULT_PER_MODEL_CONFIG.speculativeType;
|
||||
const specDraftNMax =
|
||||
speculativeType != null &&
|
||||
MTP_SPECULATIVE_TYPES.has(speculativeType) &&
|
||||
typeof partial.specDraftNMax === "number" &&
|
||||
Number.isFinite(partial.specDraftNMax)
|
||||
? Math.max(1, Math.min(16, Math.round(partial.specDraftNMax)))
|
||||
: null;
|
||||
return {
|
||||
customContextLength:
|
||||
typeof partial.customContextLength === "number" &&
|
||||
Number.isFinite(partial.customContextLength) &&
|
||||
partial.customContextLength > 0
|
||||
? Math.max(CONTEXT_LENGTH_MIN, Math.floor(partial.customContextLength))
|
||||
: null,
|
||||
maxSeqLength: normalizeMaxSeqLength(partial.maxSeqLength),
|
||||
kvCacheDtype:
|
||||
typeof partial.kvCacheDtype === "string" &&
|
||||
VALID_KV_CACHE_DTYPES.has(partial.kvCacheDtype)
|
||||
? partial.kvCacheDtype
|
||||
: null,
|
||||
speculativeType,
|
||||
specDraftNMax,
|
||||
tensorParallel:
|
||||
typeof partial.tensorParallel === "boolean"
|
||||
? partial.tensorParallel
|
||||
: DEFAULT_PER_MODEL_CONFIG.tensorParallel,
|
||||
chatTemplateOverride:
|
||||
typeof partial.chatTemplateOverride === "string" &&
|
||||
isChatTemplateWithinLimit(partial.chatTemplateOverride)
|
||||
? partial.chatTemplateOverride
|
||||
: null,
|
||||
...normalizeGpuFields(partial),
|
||||
};
|
||||
}
|
||||
|
||||
function normalize(raw: unknown): PerModelConfig {
|
||||
if (!raw || typeof raw !== "object" || Array.isArray(raw)) {
|
||||
return normalizeV1({});
|
||||
}
|
||||
const partial = raw as RawConfig;
|
||||
const version =
|
||||
typeof partial.version === "number" && Number.isFinite(partial.version)
|
||||
? partial.version
|
||||
: 0;
|
||||
warnDroppedFields(raw as Record<string, unknown>, version);
|
||||
return normalizeV1(partial);
|
||||
}
|
||||
|
||||
function toStoredConfig(config: PerModelConfig): StoredPerModelConfig {
|
||||
return {
|
||||
version: STORAGE_SCHEMA_VERSION,
|
||||
...normalize(config),
|
||||
};
|
||||
}
|
||||
|
||||
function legacyModelStorageKey(
|
||||
modelId: string,
|
||||
ggufVariant?: string | null,
|
||||
): string {
|
||||
return `${modelId}::${ggufVariant ?? ""}`;
|
||||
}
|
||||
|
||||
function storageKeysForModelVariant(
|
||||
modelId: string,
|
||||
ggufVariant?: string | null,
|
||||
): string[] {
|
||||
const key = modelStorageKey(modelId, ggufVariant);
|
||||
const legacyKey = legacyModelStorageKey(modelId, ggufVariant);
|
||||
return key === legacyKey ? [key] : [key, legacyKey];
|
||||
}
|
||||
|
||||
function configKeyMatchesModelVariant(
|
||||
key: string,
|
||||
modelId: string,
|
||||
ggufVariant?: string | null,
|
||||
): boolean {
|
||||
const storedModelId = modelIdFromStorageKey(key);
|
||||
if (!storedModelId) {
|
||||
return false;
|
||||
}
|
||||
return (
|
||||
normalizeModelIdentity(storedModelId) === normalizeModelIdentity(modelId) &&
|
||||
normalizeGgufVariantIdentity(ggufVariantFromStorageKey(key)) ===
|
||||
normalizeGgufVariantIdentity(ggufVariant)
|
||||
);
|
||||
}
|
||||
|
||||
function findConfigKeyForModelVariant(
|
||||
map: StoredMap,
|
||||
modelId: string,
|
||||
ggufVariant?: string | null,
|
||||
): string | null {
|
||||
for (const key of storageKeysForModelVariant(modelId, ggufVariant)) {
|
||||
if (Object.hasOwn(map, key)) {
|
||||
return key;
|
||||
}
|
||||
}
|
||||
for (const key of Object.keys(map)) {
|
||||
if (configKeyMatchesModelVariant(key, modelId, ggufVariant)) {
|
||||
return key;
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
function hasFutureConfigForModelVariant(
|
||||
map: StoredMap,
|
||||
modelId: string,
|
||||
ggufVariant?: string | null,
|
||||
): boolean {
|
||||
for (const key of Object.keys(map)) {
|
||||
if (
|
||||
configKeyMatchesModelVariant(key, modelId, ggufVariant) &&
|
||||
storedConfigVersion(map[key]) > STORAGE_SCHEMA_VERSION
|
||||
) {
|
||||
return true;
|
||||
}
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
function deleteConfigEntriesForModelVariant(
|
||||
map: StoredMap,
|
||||
modelId: string,
|
||||
ggufVariant?: string | null,
|
||||
): boolean {
|
||||
let changed = false;
|
||||
for (const key of Object.keys(map)) {
|
||||
if (!configKeyMatchesModelVariant(key, modelId, ggufVariant)) {
|
||||
continue;
|
||||
}
|
||||
delete map[key];
|
||||
changed = true;
|
||||
}
|
||||
return changed;
|
||||
}
|
||||
|
||||
function loadPerModelConfig(
|
||||
modelId: string,
|
||||
ggufVariant?: string | null,
|
||||
): PerModelConfig | null {
|
||||
const map = readMap();
|
||||
const key = findConfigKeyForModelVariant(map, modelId, ggufVariant);
|
||||
if (!key) {
|
||||
return null;
|
||||
}
|
||||
// Never apply a future-schema record an older client cannot interpret.
|
||||
if (storedConfigVersion(map[key]) > STORAGE_SCHEMA_VERSION) {
|
||||
return null;
|
||||
}
|
||||
return normalize(map[key]);
|
||||
}
|
||||
|
||||
export function isDefaultConfig(config: PerModelConfig): boolean {
|
||||
return (
|
||||
config.customContextLength == null &&
|
||||
config.maxSeqLength == null &&
|
||||
(config.kvCacheDtype ?? null) === DEFAULT_PER_MODEL_CONFIG.kvCacheDtype &&
|
||||
config.speculativeType === DEFAULT_PER_MODEL_CONFIG.speculativeType &&
|
||||
config.specDraftNMax == null &&
|
||||
Boolean(config.tensorParallel) ===
|
||||
Boolean(DEFAULT_PER_MODEL_CONFIG.tensorParallel) &&
|
||||
(config.chatTemplateOverride ?? null) === null &&
|
||||
gpuFieldsAtDefault(config)
|
||||
);
|
||||
}
|
||||
|
||||
// GPU knobs are "default" when mode is Auto with no explicit choice: mode
|
||||
// auto/absent, gpuLayers < 0/absent, nCpuMoe 0/absent, selectedGpuIds null/absent.
|
||||
function gpuFieldsAtDefault(config: PerModelConfig): boolean {
|
||||
return (
|
||||
(config.gpuMemoryMode ?? "auto") === "auto" &&
|
||||
(config.gpuLayers == null || config.gpuLayers < 0) &&
|
||||
(config.nCpuMoe == null || config.nCpuMoe === 0) &&
|
||||
config.selectedGpuIds == null
|
||||
);
|
||||
}
|
||||
|
||||
export function savePerModelConfig(
|
||||
modelId: string,
|
||||
ggufVariant: string | null | undefined,
|
||||
config: PerModelConfig,
|
||||
): boolean {
|
||||
if (
|
||||
typeof config.chatTemplateOverride === "string" &&
|
||||
!isChatTemplateWithinLimit(config.chatTemplateOverride)
|
||||
) {
|
||||
return false;
|
||||
}
|
||||
const normalized = normalize(config);
|
||||
const map = readMap();
|
||||
if (hasFutureConfigForModelVariant(map, modelId, ggufVariant)) {
|
||||
return false;
|
||||
}
|
||||
if (isDefaultConfig(normalized)) {
|
||||
const changed = deleteConfigEntriesForModelVariant(
|
||||
map,
|
||||
modelId,
|
||||
ggufVariant,
|
||||
);
|
||||
return changed ? writeMap(map) : true;
|
||||
}
|
||||
const [key] = storageKeysForModelVariant(modelId, ggufVariant);
|
||||
deleteConfigEntriesForModelVariant(map, modelId, ggufVariant);
|
||||
map[key] = toStoredConfig(normalized);
|
||||
if (!enforceStorageBudget(map, new Set([key]))) {
|
||||
return false;
|
||||
}
|
||||
return writeMap(map);
|
||||
}
|
||||
|
||||
export function deletePerModelConfig(
|
||||
modelId: string,
|
||||
ggufVariant?: string | null,
|
||||
): boolean {
|
||||
const map = readMap();
|
||||
// Mirror savePerModelConfig: never let an older client destroy a future-schema entry.
|
||||
if (hasFutureConfigForModelVariant(map, modelId, ggufVariant)) {
|
||||
return false;
|
||||
}
|
||||
if (!deleteConfigEntriesForModelVariant(map, modelId, ggufVariant)) {
|
||||
return true;
|
||||
}
|
||||
return writeMap(map);
|
||||
}
|
||||
|
||||
export function resolveInitialConfig(
|
||||
modelId: string,
|
||||
ggufVariant?: string | null,
|
||||
): { config: PerModelConfig; remembered: boolean } {
|
||||
const saved = loadPerModelConfig(modelId, ggufVariant);
|
||||
if (saved) {
|
||||
return { config: saved, remembered: true };
|
||||
}
|
||||
return { config: { ...DEFAULT_PER_MODEL_CONFIG }, remembered: false };
|
||||
}
|
||||
|
|
@ -16,7 +16,6 @@ import {
|
|||
Folder01Icon,
|
||||
McpServerIcon,
|
||||
PencilRulerIcon,
|
||||
Settings02Icon,
|
||||
ShieldBanIcon,
|
||||
} from "@hugeicons/core-free-icons";
|
||||
import { HugeiconsIcon } from "@hugeicons/react";
|
||||
|
|
@ -148,10 +147,6 @@ export function ChatTab() {
|
|||
const hydratePersistedSettings = useChatRuntimeStore(
|
||||
(state) => state.hydratePersistedSettings,
|
||||
);
|
||||
const loadOnSelection = useChatRuntimeStore((state) => state.loadOnSelection);
|
||||
const setLoadOnSelection = useChatRuntimeStore(
|
||||
(state) => state.setLoadOnSelection,
|
||||
);
|
||||
const expandQuantizations = useChatRuntimeStore(
|
||||
(state) => state.expandQuantizations,
|
||||
);
|
||||
|
|
@ -193,40 +188,6 @@ export function ChatTab() {
|
|||
</header>
|
||||
|
||||
<SettingsSection title="Select model settings">
|
||||
<SettingsRow
|
||||
label="Load on selection"
|
||||
alignTop={true}
|
||||
description={
|
||||
<span>
|
||||
On: Unsloth auto-picks the best settings and loads it.
|
||||
<br />
|
||||
Off: opens Run settings to customize, then load.
|
||||
<br />
|
||||
The gear always opens Run settings:{" "}
|
||||
<span className="ml-2 inline-flex items-center gap-3 align-middle">
|
||||
<span className="font-mono text-xs text-foreground">
|
||||
Q4_K_M
|
||||
</span>
|
||||
<span className="text-[9px] font-medium text-green-600/90 dark:text-green-400/80">
|
||||
downloaded
|
||||
</span>
|
||||
<span className="text-[10px] text-muted-foreground">16 GB</span>
|
||||
<span className="inline-flex size-4 items-center justify-center rounded bg-black/[0.06] dark:bg-white/[0.08]">
|
||||
<HugeiconsIcon
|
||||
icon={Settings02Icon}
|
||||
strokeWidth={1.75}
|
||||
className="size-2.5 text-muted-foreground/80"
|
||||
/>
|
||||
</span>
|
||||
</span>
|
||||
</span>
|
||||
}
|
||||
>
|
||||
<Switch
|
||||
checked={loadOnSelection}
|
||||
onCheckedChange={setLoadOnSelection}
|
||||
/>
|
||||
</SettingsRow>
|
||||
<SettingsRow
|
||||
label="Expand quantizations"
|
||||
description={
|
||||
|
|
|
|||
|
|
@ -93,9 +93,11 @@ const PREFS_KEYS: string[] = [
|
|||
"unsloth_chat_inference_params",
|
||||
"unsloth_chat_collapsible_state",
|
||||
"unsloth_chat_preferences",
|
||||
"unsloth_model_configs",
|
||||
"unsloth_model_configs_migrated",
|
||||
"unsloth_load_settings",
|
||||
// Model selector settings ("Select model settings" group)
|
||||
"unsloth_chat_load_on_selection",
|
||||
// Model selector settings ("Select model settings" group)
|
||||
"unsloth_chat_expand_quantizations",
|
||||
"unsloth_chat_show_all_quantizations",
|
||||
"unsloth_models_fit_on_device_only",
|
||||
|
|
|
|||
|
|
@ -2,6 +2,7 @@
|
|||
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
import { authFetch } from "@/features/auth";
|
||||
import { hubTokenHeader } from "@/features/hub";
|
||||
|
||||
interface VisionCheckResponse {
|
||||
model_name: string;
|
||||
|
|
@ -98,8 +99,9 @@ export async function checkVisionModel(
|
|||
hfToken?: string | null,
|
||||
): Promise<boolean> {
|
||||
const encoded = encodeURIComponent(modelName);
|
||||
const query = hfToken?.trim() ? `?hf_token=${encodeURIComponent(hfToken.trim())}` : "";
|
||||
const response = await authFetch(`/api/models/check-vision/${encoded}${query}`);
|
||||
const response = await authFetch(`/api/models/check-vision/${encoded}`, {
|
||||
headers: hubTokenHeader(hfToken?.trim() || null),
|
||||
});
|
||||
if (!response.ok) {
|
||||
// If the check fails (e.g. network error), default to non-vision
|
||||
return false;
|
||||
|
|
@ -114,8 +116,9 @@ export async function checkEmbeddingModel(
|
|||
hfToken?: string | null,
|
||||
): Promise<boolean> {
|
||||
const encoded = encodeURIComponent(modelName);
|
||||
const query = hfToken?.trim() ? `?hf_token=${encodeURIComponent(hfToken.trim())}` : "";
|
||||
const response = await authFetch(`/api/models/check-embedding/${encoded}${query}`);
|
||||
const response = await authFetch(`/api/models/check-embedding/${encoded}`, {
|
||||
headers: hubTokenHeader(hfToken?.trim() || null),
|
||||
});
|
||||
if (!response.ok) {
|
||||
// If the check fails (e.g. network error), default to non-embedding
|
||||
return false;
|
||||
|
|
@ -130,8 +133,10 @@ export async function getModelConfig(
|
|||
hfToken?: string,
|
||||
): Promise<ModelConfigResponse> {
|
||||
const encoded = encodeURIComponent(modelName);
|
||||
const params = hfToken ? `?hf_token=${encodeURIComponent(hfToken)}` : "";
|
||||
const response = await authFetch(`/api/models/config/${encoded}${params}`, { signal });
|
||||
const response = await authFetch(`/api/models/config/${encoded}`, {
|
||||
headers: hubTokenHeader(hfToken?.trim() || null),
|
||||
signal,
|
||||
});
|
||||
if (!response.ok) {
|
||||
throw new Error(`Failed to fetch model config (${response.status})`);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -25,8 +25,8 @@ export { HfDatasetSubsetSplitSelectors } from "./components/hf-dataset-subset-sp
|
|||
export { useDatasetPreviewDialogStore } from "./stores/dataset-preview-dialog-store";
|
||||
export { listLocalDatasets, uploadTrainingDataset } from "./api/datasets-api";
|
||||
export type { LocalDatasetInfo } from "./types/datasets";
|
||||
export { listLocalModels } from "./api/models-api";
|
||||
export type { LocalModelInfo } from "./api/models-api";
|
||||
export { getModelConfig, listLocalModels } from "./api/models-api";
|
||||
export type { LocalModelInfo, ModelConfigResponse } from "./api/models-api";
|
||||
export type {
|
||||
TrainingPhase,
|
||||
TrainingViewData,
|
||||
|
|
|
|||
|
|
@ -1557,7 +1557,7 @@ class TestWorkerRocmMambaSsm:
|
|||
assert "getattr(torch.version, 'hip', None)" in source
|
||||
|
||||
def test_direct_wheel_url_returns_none_without_cuda_major(self, monkeypatch):
|
||||
"""_direct_wheel_url should return None when cuda_major is empty (ROCm)."""
|
||||
"""direct_wheel_url should return None when cuda_major is empty (ROCm)."""
|
||||
_worker_spec = importlib.util.spec_from_file_location("test_worker", _WORKER_PATH)
|
||||
assert _worker_spec is not None and _worker_spec.loader is not None
|
||||
worker_mod = importlib.util.module_from_spec(_worker_spec)
|
||||
|
|
@ -1583,7 +1583,7 @@ class TestWorkerRocmMambaSsm:
|
|||
"hip_version": "7.1.12345",
|
||||
"cxx11abi": "TRUE",
|
||||
}
|
||||
result = worker_mod._direct_wheel_url(
|
||||
result = worker_mod.direct_wheel_url(
|
||||
filename_prefix = "causal_conv1d",
|
||||
package_version = "1.6.1",
|
||||
release_tag = "v1.6.1.post4",
|
||||
|
|
|
|||
|
|
@ -821,10 +821,16 @@ with sync_playwright() as p:
|
|||
last_assistant = page.locator('[data-role="assistant"]').last
|
||||
last_assistant.hover()
|
||||
page.wait_for_timeout(400)
|
||||
regen_btn = page.get_by_role(
|
||||
"button",
|
||||
name = re.compile(r"(reload|regenerate)", re.I),
|
||||
).first
|
||||
# Exclude disabled controls: the picker's new disabled "Reload model"
|
||||
# button also matches and sorts first, so .first would target it.
|
||||
regen_btn = (
|
||||
page.get_by_role(
|
||||
"button",
|
||||
name = re.compile(r"(reload|regenerate)", re.I),
|
||||
)
|
||||
.and_(page.locator("button:not([disabled])"))
|
||||
.first
|
||||
)
|
||||
if regen_btn.count() > 0:
|
||||
regen_btn.click()
|
||||
try:
|
||||
|
|
|
|||
740
tests/studio/playwright_model_config.py
Normal file
740
tests/studio/playwright_model_config.py
Normal file
|
|
@ -0,0 +1,740 @@
|
|||
# SPDX-License-Identifier: AGPL-3.0-only
|
||||
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
"""Model-picker per-model-config Playwright regression test (GPU-free, CPU gemma).
|
||||
|
||||
Guards, end to end against the real frontend, the exact regressions that got the
|
||||
predecessor PR reverted:
|
||||
|
||||
- Context Length persists: set a distinctive per-model Context Length + tick
|
||||
"Remember for this model" + Load; the value reaches the /api/inference/load
|
||||
request (max_seq_length) AND lands in localStorage (unsloth_model_configs),
|
||||
and survives a full browser reload (HARD).
|
||||
- Reset clears: after customizing, Reset must clear the stored override, never
|
||||
pin the context to a fixed number (the "Reset pins context" regression) (HARD).
|
||||
- Hidden infra models absent: the RAG embedder (bge-small-en-v1.5) and the
|
||||
llama.cpp validation probe (stories260K) never appear in the picker. The
|
||||
probe GGUF is primed into the HF cache by the CI job, so "absent" proves
|
||||
"hidden", not "not downloaded" (HARD).
|
||||
- Legacy migration is idempotent: a pre-feature unsloth_load_settings store
|
||||
migrates once into the versioned unsloth_model_configs map with the value
|
||||
preserved, and a second reload with a fresh legacy seed present does not
|
||||
re-migrate, duplicate, or clobber (gates under STUDIO_UI_STRICT via soft_fail).
|
||||
- Advanced settings persist: KV cache dtype / tensor-parallel toggled under
|
||||
Advanced + Remember land in unsloth_model_configs (best-effort).
|
||||
|
||||
Runs as a plain script (not via pytest), mirroring tests/studio/playwright_extra_ui.py:
|
||||
accumulate failures in `_failed`, exit non-zero if any HARD gate failed. With
|
||||
STUDIO_UI_STRICT=1 (as CI sets), soft_fail also gates; genuinely-optional checks
|
||||
use runtime_warn so they never flake the merge gate.
|
||||
"""
|
||||
|
||||
import json
|
||||
import re
|
||||
import sys
|
||||
import os
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
from playwright.sync_api import sync_playwright
|
||||
|
||||
# Run as a plain script (not via pytest), so prepend the dir to sys.path.
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent))
|
||||
from _playwright_robust import ( # noqa: E402
|
||||
chromium_launch_args,
|
||||
click_and_wait_for_response,
|
||||
evaluate_fetch,
|
||||
install_view_transition_killer,
|
||||
install_wall_clock_watchdog,
|
||||
is_benign_page_error,
|
||||
recover_or_replace_page,
|
||||
robust_evaluate,
|
||||
wait_for_health,
|
||||
)
|
||||
|
||||
BASE = os.environ["BASE_URL"]
|
||||
NEW = os.environ.get("STUDIO_NEW_PW", "ModelCfg-NEW-2026!")
|
||||
# Attach mode: log into an already-provisioned Studio with an existing password
|
||||
# instead of the first-boot change-password dance. CI leaves STUDIO_LOGIN_PW unset
|
||||
# to exercise the real change-password flow; local runs can set it to skip re-provisioning.
|
||||
LOGIN_PW = os.environ.get("STUDIO_LOGIN_PW")
|
||||
LOGIN_USER = os.environ.get("STUDIO_LOGIN_USER", "unsloth")
|
||||
GGUF_REPO = os.environ.get("GGUF_REPO", "unsloth/gemma-3-270m-it-GGUF")
|
||||
GGUF_VARIANT = os.environ.get("GGUF_VARIANT", "UD-Q4_K_XL")
|
||||
# Substring of the On Device picker row for the loaded model.
|
||||
MODEL_HINT = os.environ.get("STUDIO_MODEL_HINT", "gemma-3-270m")
|
||||
# A distinctive valid (>=128, multiple of 128, below the model's 32768 ceiling)
|
||||
# Context Length, clearly not a default, so persistence is unambiguous.
|
||||
DISTINCT_CTX = int(os.environ.get("STUDIO_DISTINCT_CTX", "4096"))
|
||||
ART_DIR = os.environ.get("PW_ART_DIR", "logs/playwright_modelcfg")
|
||||
ART = Path(ART_DIR)
|
||||
ART.mkdir(parents = True, exist_ok = True)
|
||||
STRICT = os.environ.get("STUDIO_UI_STRICT", "0") == "1"
|
||||
TURN_TIMEOUT_MS = int(os.environ.get("STUDIO_UI_TURN_TIMEOUT_MS", "180000"))
|
||||
WALL_TIMEOUT_S = float(os.environ.get("STUDIO_UI_WALL_TIMEOUT_S", "720"))
|
||||
FETCH_TIMEOUT_MS = int(os.environ.get("STUDIO_UI_FETCH_TIMEOUT_MS", "30000"))
|
||||
LOAD_FETCH_TIMEOUT_MS = int(os.environ.get("STUDIO_UI_LOAD_TIMEOUT_MS", "180000"))
|
||||
|
||||
_n = [0]
|
||||
_failed: list[str] = []
|
||||
|
||||
|
||||
def step(s: str) -> None:
|
||||
print(f"[ui-modelcfg] STEP {s}", flush = True)
|
||||
|
||||
|
||||
def info(s: str) -> None:
|
||||
print(f"[ui-modelcfg] {s}", flush = True)
|
||||
|
||||
|
||||
def fail(m: str) -> None:
|
||||
print(f"[ui-modelcfg] FAIL: {m}", flush = True)
|
||||
_failed.append(m)
|
||||
|
||||
|
||||
def soft_fail(m: str) -> None:
|
||||
if STRICT:
|
||||
fail(m)
|
||||
else:
|
||||
info(f"WARN (strict-off): {m}")
|
||||
|
||||
|
||||
def runtime_warn(m: str) -> None:
|
||||
"""Warn about a genuinely-optional check that STRICT does not gate."""
|
||||
info(f"WARN (runtime): {m}")
|
||||
|
||||
|
||||
def _count(loc) -> int:
|
||||
try:
|
||||
return loc.count()
|
||||
except Exception:
|
||||
return 0
|
||||
|
||||
|
||||
def _as_int(value) -> int | None:
|
||||
"""Parse an input value to int, tolerating commas/whitespace. Comparisons
|
||||
must be numeric, never substring: '40960' (a model's native default) would
|
||||
spuriously "contain" '4096'."""
|
||||
if value is None:
|
||||
return None
|
||||
try:
|
||||
return int(str(value).replace(",", "").strip())
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
def _login_token_via_api(base: str, user: str, pw: str) -> str:
|
||||
"""POST /api/auth/login -> access_token (attach-mode helper, stdlib only)."""
|
||||
import urllib.request
|
||||
|
||||
req = urllib.request.Request(
|
||||
f"{base}/api/auth/login",
|
||||
data = json.dumps({"username": user, "password": pw}).encode(),
|
||||
headers = {"Content-Type": "application/json"},
|
||||
method = "POST",
|
||||
)
|
||||
with urllib.request.urlopen(req, timeout = 15) as r:
|
||||
return json.loads(r.read().decode())["access_token"]
|
||||
|
||||
|
||||
with sync_playwright() as p:
|
||||
_watchdog = install_wall_clock_watchdog(
|
||||
WALL_TIMEOUT_S,
|
||||
label = "ui-modelcfg",
|
||||
info = info,
|
||||
)
|
||||
# Health pre-flight: bash-side health wait can pass before the auth DB migrates.
|
||||
wait_for_health(BASE, timeout = 30.0, info = info)
|
||||
browser = p.chromium.launch(
|
||||
headless = True,
|
||||
args = chromium_launch_args(),
|
||||
)
|
||||
ctx = browser.new_context(
|
||||
viewport = {"width": 1280, "height": 900},
|
||||
reduced_motion = "reduce",
|
||||
)
|
||||
install_view_transition_killer(ctx)
|
||||
page = ctx.new_page()
|
||||
page.set_default_timeout(60_000)
|
||||
page_errors = []
|
||||
|
||||
def _on_pageerror(e):
|
||||
msg = str(e)
|
||||
if is_benign_page_error(msg):
|
||||
info(f"WARN ignoring benign pageerror: {msg!r}")
|
||||
return
|
||||
page_errors.append(msg)
|
||||
|
||||
page.on("pageerror", _on_pageerror)
|
||||
|
||||
# Record every /api/inference/load POST payload so the persistence gate can
|
||||
# assert max_seq_length.
|
||||
load_posts: list[str] = []
|
||||
|
||||
def _on_request(req):
|
||||
try:
|
||||
if req.method == "POST" and "/api/inference/load" in req.url:
|
||||
load_posts.append(req.post_data or "")
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
page.on("request", _on_request)
|
||||
|
||||
def shoot(name: str) -> None:
|
||||
_n[0] += 1
|
||||
try:
|
||||
page.screenshot(
|
||||
path = str(ART / f"{_n[0]:02d}-{name}.png"),
|
||||
full_page = True,
|
||||
timeout = 90_000,
|
||||
animations = "disabled",
|
||||
)
|
||||
except Exception as _shoot_err:
|
||||
info(f"WARN: screenshot {name} failed: {_shoot_err}")
|
||||
|
||||
def read_configs() -> dict:
|
||||
"""Return the parsed unsloth_model_configs map (or {} if absent/invalid)."""
|
||||
raw = robust_evaluate(page, "() => localStorage.getItem('unsloth_model_configs')")
|
||||
if not raw:
|
||||
return {}
|
||||
try:
|
||||
data = json.loads(raw)
|
||||
return data if isinstance(data, dict) else {}
|
||||
except Exception:
|
||||
return {}
|
||||
|
||||
def config_entries(cfg: dict) -> list[dict]:
|
||||
"""The per-model entries (dict values) of the stored map, schema-tolerant."""
|
||||
return [v for v in cfg.values() if isinstance(v, dict)]
|
||||
|
||||
# ─────────────────────────────────────────────────────
|
||||
# Setup: authenticate + model load.
|
||||
# ─────────────────────────────────────────────────────
|
||||
if LOGIN_PW:
|
||||
# Attach mode: log in via the API and seed the token before navigation,
|
||||
# skipping the first-boot change-password dance.
|
||||
step("setup: API login + token seed (attach to running Studio)")
|
||||
_tok = _login_token_via_api(BASE, LOGIN_USER, LOGIN_PW)
|
||||
ctx.add_init_script(
|
||||
f"try{{localStorage.setItem('unsloth_auth_token', {json.dumps(_tok)});}}"
|
||||
f"catch(e){{}}"
|
||||
)
|
||||
page.goto(BASE, wait_until = "domcontentloaded", timeout = 60_000)
|
||||
else:
|
||||
step("setup: change-password")
|
||||
# 3-attempt retry: the form can re-render mid-fill on slow runners and
|
||||
# detach the password fields; each retry re-navigates with a fresh page.
|
||||
form_err: Exception | None = None
|
||||
for _form_attempt in range(3):
|
||||
try:
|
||||
page.goto(f"{BASE}/change-password", wait_until = "domcontentloaded", timeout = 60_000)
|
||||
try:
|
||||
page.wait_for_load_state("networkidle", timeout = 30_000)
|
||||
except Exception:
|
||||
pass
|
||||
pw_field = page.locator("#new-password")
|
||||
pw_field.wait_for(state = "visible", timeout = 60_000)
|
||||
pw_field.fill(NEW, timeout = 60_000)
|
||||
page.fill("#confirm-password", NEW, timeout = 60_000)
|
||||
status, _ = click_and_wait_for_response(
|
||||
page,
|
||||
url_substr = "/api/auth/change-password",
|
||||
method = "POST",
|
||||
do_click = lambda: page.locator('button[type="submit"]').click(),
|
||||
timeout_ms = 30_000,
|
||||
info = lambda m: print(f"[ui-modelcfg] {m}", flush = True),
|
||||
)
|
||||
if status is not None and status >= 400:
|
||||
raise AssertionError(
|
||||
f"change-password POST returned {status}; page_errors={page_errors[:1]!r}"
|
||||
)
|
||||
form_err = None
|
||||
break
|
||||
except Exception as e:
|
||||
form_err = e
|
||||
try:
|
||||
cur_url = page.url
|
||||
except Exception:
|
||||
cur_url = "<page closed>"
|
||||
print(
|
||||
f"[ui-modelcfg] change-password attempt {_form_attempt + 1} failed: "
|
||||
f"{type(e).__name__}: {str(e)[:200]}; page.url={cur_url}; "
|
||||
f"page_errors={len(page_errors)}",
|
||||
flush = True,
|
||||
)
|
||||
if _form_attempt < 2:
|
||||
if "ERR_NO_BUFFER_SPACE" in str(e):
|
||||
backoff_s = 5 if _form_attempt == 0 else 15
|
||||
time.sleep(backoff_s)
|
||||
page = recover_or_replace_page(
|
||||
page,
|
||||
ctx,
|
||||
default_timeout_ms = 60_000,
|
||||
info = lambda m: print(f"[ui-modelcfg] recovery: {m}", flush = True),
|
||||
)
|
||||
page.on("request", _on_request)
|
||||
if form_err is not None:
|
||||
raise form_err
|
||||
|
||||
try:
|
||||
page.wait_for_load_state("networkidle", timeout = 30_000)
|
||||
except Exception:
|
||||
pass
|
||||
composer = page.locator('textarea[aria-label="Message input"]')
|
||||
last_err: Exception | None = None
|
||||
for _attempt in range(2):
|
||||
try:
|
||||
composer.wait_for(state = "visible", timeout = 60_000)
|
||||
last_err = None
|
||||
break
|
||||
except Exception as e:
|
||||
last_err = e
|
||||
try:
|
||||
shoot(f"00-composer-wait-attempt-{_attempt + 1}-fail")
|
||||
except Exception:
|
||||
pass
|
||||
if _attempt == 0:
|
||||
page = recover_or_replace_page(
|
||||
page,
|
||||
ctx,
|
||||
default_timeout_ms = 60_000,
|
||||
goto_url = BASE,
|
||||
settle_networkidle = True,
|
||||
info = lambda m: print(f"[ui-modelcfg] recovery: {m}", flush = True),
|
||||
)
|
||||
page.on("request", _on_request)
|
||||
composer = page.locator('textarea[aria-label="Message input"]')
|
||||
if last_err is not None:
|
||||
raise last_err
|
||||
shoot("01-chat-loaded")
|
||||
|
||||
token = robust_evaluate(page, "() => localStorage.getItem('unsloth_auth_token')")
|
||||
if not token:
|
||||
fail("no access token after auth setup")
|
||||
sys.exit(1)
|
||||
|
||||
# Load the tiny GGUF so it is a live "On Device" model in the picker.
|
||||
load_resp = evaluate_fetch(
|
||||
page,
|
||||
f"{BASE}/api/inference/load",
|
||||
method = "POST",
|
||||
headers = {
|
||||
"Authorization": f"Bearer {token}",
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
body = {
|
||||
"model_path": GGUF_REPO,
|
||||
"gguf_variant": GGUF_VARIANT,
|
||||
"is_lora": False,
|
||||
"max_seq_length": 2048,
|
||||
},
|
||||
timeout_ms = LOAD_FETCH_TIMEOUT_MS,
|
||||
)
|
||||
if load_resp.get("error"):
|
||||
fail(f"/api/inference/load wedged: {load_resp['error']!r}")
|
||||
sys.exit(1)
|
||||
if load_resp["status"] != 200:
|
||||
fail(f"/api/inference/load -> {load_resp['status']}: {load_resp.get('body')!r}")
|
||||
sys.exit(1)
|
||||
info(f"loaded model: {(load_resp['body'] or {}).get('display_name')}")
|
||||
page.reload()
|
||||
composer = page.locator('textarea[aria-label="Message input"]')
|
||||
composer.wait_for(state = "visible", timeout = 60_000)
|
||||
load_posts.clear() # drop the setup load; keep only UI-driven loads below.
|
||||
|
||||
# ─────────────────────────────────────────────────────
|
||||
# Picker helpers (proven selectors).
|
||||
# ─────────────────────────────────────────────────────
|
||||
POPOVER = '[data-tour="chat-model-selector-popover"]'
|
||||
TRIGGER = '[data-tour="chat-model-selector"]'
|
||||
|
||||
def open_picker():
|
||||
popover = page.locator(POPOVER).first
|
||||
if _count(popover) == 0 or not popover.is_visible():
|
||||
page.locator(TRIGGER).first.click()
|
||||
page.wait_for_timeout(900)
|
||||
popover = page.locator(POPOVER).first
|
||||
popover.wait_for(state = "visible", timeout = 30_000)
|
||||
return popover
|
||||
|
||||
def close_picker():
|
||||
try:
|
||||
page.keyboard.press("Escape")
|
||||
page.wait_for_timeout(400)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
def select_on_device_row(popover, hint):
|
||||
od = page.get_by_role("tab", name = "On Device").first
|
||||
if _count(od):
|
||||
od.click()
|
||||
page.wait_for_timeout(700)
|
||||
row = popover.locator("[data-model-picker-option]", has_text = hint).first
|
||||
if _count(row) == 0:
|
||||
# Fall back to search filtering.
|
||||
search = popover.locator("[data-model-picker-search-input]").first
|
||||
if _count(search):
|
||||
search.click()
|
||||
search.fill(hint)
|
||||
page.wait_for_timeout(700)
|
||||
row = popover.locator("[data-model-picker-option]", has_text = hint).first
|
||||
if _count(row) == 0:
|
||||
return None
|
||||
row.click()
|
||||
page.wait_for_timeout(800)
|
||||
return row
|
||||
|
||||
def open_config(popover, hint):
|
||||
if select_on_device_row(popover, hint) is None:
|
||||
return None
|
||||
gear = popover.locator('button[aria-label^="Inference settings for"]').first
|
||||
if _count(gear) == 0:
|
||||
return None
|
||||
gear.click()
|
||||
page.wait_for_timeout(800)
|
||||
return popover
|
||||
|
||||
def context_input(popover):
|
||||
for role in ("textbox", "spinbutton"):
|
||||
loc = popover.get_by_role(role, name = "Context Length").first
|
||||
if _count(loc):
|
||||
return loc
|
||||
loc = popover.locator('input[aria-label="Context Length"]').first
|
||||
return loc if _count(loc) else None
|
||||
|
||||
def primary_button(popover):
|
||||
for name in ("Load model", "Reload model", "Save settings", "Forget settings"):
|
||||
b = popover.get_by_role("button", name = name).first
|
||||
if _count(b):
|
||||
return b
|
||||
return None
|
||||
|
||||
# ─────────────────────────────────────────────────────
|
||||
# 1. Hidden infra models absent from the picker (HARD).
|
||||
# ─────────────────────────────────────────────────────
|
||||
step("hidden infra models absent from picker")
|
||||
popover = open_picker()
|
||||
shoot("02-picker-open")
|
||||
needles = ["bge-small-en-v1.5", "stories260"]
|
||||
tabs = ["Recommended", "On Device", "Connected"]
|
||||
hidden_ok = True
|
||||
for needle in needles:
|
||||
for tab_name in tabs:
|
||||
tab = page.get_by_role("tab", name = tab_name).first
|
||||
if _count(tab) == 0:
|
||||
continue
|
||||
try:
|
||||
tab.click()
|
||||
page.wait_for_timeout(400)
|
||||
except Exception:
|
||||
continue
|
||||
search = popover.locator("[data-model-picker-search-input]").first
|
||||
if _count(search):
|
||||
search.click()
|
||||
search.fill(needle)
|
||||
page.wait_for_timeout(600)
|
||||
hit = popover.locator(
|
||||
"[data-model-picker-option]",
|
||||
has_text = re.compile(re.escape(needle), re.I),
|
||||
)
|
||||
c = _count(hit)
|
||||
if c > 0:
|
||||
hidden_ok = False
|
||||
fail(f"infra model {needle!r} visible in picker '{tab_name}' tab ({c} rows)")
|
||||
if _count(search):
|
||||
search.fill("")
|
||||
page.wait_for_timeout(300)
|
||||
if hidden_ok:
|
||||
info("OK hidden: bge-small-en-v1.5 + stories260K absent from every picker tab")
|
||||
shoot("03-hidden-check")
|
||||
close_picker()
|
||||
|
||||
# ─────────────────────────────────────────────────────
|
||||
# 2. Context Length persists (load + request + reload) (HARD).
|
||||
# ─────────────────────────────────────────────────────
|
||||
step(f"context length {DISTINCT_CTX} persists")
|
||||
popover = open_picker()
|
||||
if open_config(popover, MODEL_HINT) is None:
|
||||
fail(f"could not open run-settings for a model matching {MODEL_HINT!r}")
|
||||
else:
|
||||
shoot("04-config-open")
|
||||
ctx_in = context_input(popover)
|
||||
if ctx_in is None:
|
||||
fail("Context Length input not found in run-settings")
|
||||
else:
|
||||
default_ctx = ctx_in.input_value()
|
||||
info(f"default Context Length shown: {default_ctx!r}")
|
||||
ctx_in.click()
|
||||
ctx_in.fill(str(DISTINCT_CTX))
|
||||
page.wait_for_timeout(300)
|
||||
page.keyboard.press("Tab") # blur to commit
|
||||
page.wait_for_timeout(300)
|
||||
remember = popover.get_by_label("Remember for this model").first
|
||||
if _count(remember):
|
||||
try:
|
||||
remember.check()
|
||||
except Exception:
|
||||
remember.click()
|
||||
else:
|
||||
fail("'Remember for this model' checkbox not found")
|
||||
page.wait_for_timeout(300)
|
||||
shoot("05-ctx-set")
|
||||
btn = primary_button(popover)
|
||||
if btn is None:
|
||||
fail("primary Load/Save button not found in run-settings")
|
||||
else:
|
||||
btn.click()
|
||||
page.wait_for_timeout(2500)
|
||||
shoot("06-after-load")
|
||||
|
||||
# (a) localStorage stored the distinctive context.
|
||||
cfg = read_configs()
|
||||
entries = config_entries(cfg)
|
||||
got_ls = any(e.get("customContextLength") == DISTINCT_CTX for e in entries)
|
||||
if got_ls:
|
||||
info(f"OK persist(localStorage): customContextLength={DISTINCT_CTX} stored")
|
||||
else:
|
||||
fail(
|
||||
"context not stored in unsloth_model_configs "
|
||||
f"(entries={json.dumps(entries)[:400]})"
|
||||
)
|
||||
|
||||
# (b) the load request carried max_seq_length == distinctive value.
|
||||
got_req = False
|
||||
for body in load_posts:
|
||||
try:
|
||||
payload = json.loads(body) if body else {}
|
||||
except Exception:
|
||||
payload = {}
|
||||
if payload.get("max_seq_length") == DISTINCT_CTX:
|
||||
got_req = True
|
||||
break
|
||||
if got_req:
|
||||
info(f"OK persist(request): /api/inference/load max_seq_length={DISTINCT_CTX}")
|
||||
else:
|
||||
# The UI may debounce the load; localStorage is the primary
|
||||
# proof, so only warn if the request was missed.
|
||||
runtime_warn(
|
||||
"no /api/inference/load carried "
|
||||
f"max_seq_length={DISTINCT_CTX}; posts={load_posts!r}"
|
||||
)
|
||||
|
||||
# (c) survives a full browser reload.
|
||||
close_picker()
|
||||
page.reload()
|
||||
composer = page.locator('textarea[aria-label="Message input"]')
|
||||
composer.wait_for(state = "visible", timeout = 60_000)
|
||||
popover = open_picker()
|
||||
if open_config(popover, MODEL_HINT) is None:
|
||||
fail("could not reopen run-settings after reload")
|
||||
else:
|
||||
ctx_in = context_input(popover)
|
||||
val = ctx_in.input_value() if ctx_in else None
|
||||
if _as_int(val) == DISTINCT_CTX:
|
||||
info(f"OK persist(reload): Context Length still {val!r} after reload")
|
||||
else:
|
||||
fail(f"Context Length did not persist across reload (got {val!r})")
|
||||
shoot("07-after-reload")
|
||||
|
||||
# ─────────────────────────────────────────────────────
|
||||
# 3. Reset clears the override (never pins context) (HARD).
|
||||
# ─────────────────────────────────────────────────────
|
||||
step("reset clears the per-model override")
|
||||
# (popover + config still open from the reload check.)
|
||||
reset_btn = popover.get_by_role("button", name = "Reset").first
|
||||
if _count(reset_btn) == 0:
|
||||
fail("Reset button not found in run-settings")
|
||||
else:
|
||||
try:
|
||||
reset_btn.click()
|
||||
page.wait_for_timeout(500)
|
||||
except Exception as e:
|
||||
fail(f"Reset click failed: {e}")
|
||||
# The input after Reset is informational only: a live-loaded model can still
|
||||
# echo its context even with the stored override gone. The regression we
|
||||
# guard ("Reset PINS the override") lives in localStorage, asserted below.
|
||||
ctx_in = context_input(popover)
|
||||
after_reset = ctx_in.input_value() if ctx_in else None
|
||||
info(f"reset: Context Length input now shows {after_reset!r}")
|
||||
# Commit the reset so the stored override is dropped, then assert storage.
|
||||
btn = primary_button(popover)
|
||||
if btn is not None and btn.is_enabled():
|
||||
btn.click()
|
||||
page.wait_for_timeout(1500)
|
||||
cfg = read_configs()
|
||||
pinned = any(
|
||||
_as_int(e.get("customContextLength")) == DISTINCT_CTX for e in config_entries(cfg)
|
||||
)
|
||||
if pinned:
|
||||
fail("Reset left the distinctive context pinned in unsloth_model_configs")
|
||||
else:
|
||||
info("OK reset: distinctive context cleared from unsloth_model_configs")
|
||||
shoot("08-after-reset")
|
||||
close_picker()
|
||||
|
||||
# ─────────────────────────────────────────────────────
|
||||
# 4. Advanced settings persist (best-effort, never gates).
|
||||
# ─────────────────────────────────────────────────────
|
||||
step("advanced (KV cache dtype / tensor parallel) persists")
|
||||
try:
|
||||
popover = open_picker()
|
||||
if open_config(popover, MODEL_HINT) is not None:
|
||||
adv = popover.get_by_role("switch", name = re.compile("advanced settings", re.I)).first
|
||||
if _count(adv):
|
||||
try:
|
||||
adv.check()
|
||||
except Exception:
|
||||
adv.click()
|
||||
page.wait_for_timeout(500)
|
||||
# The Tensor Parallelism Radix Switch has no aria-label, so target the
|
||||
# first switch after the "Tensor Parallelism" text.
|
||||
tp = popover.locator(
|
||||
'xpath=.//span[contains(text(),"Tensor Parallelism")]'
|
||||
'/following::*[@role="switch"][1]'
|
||||
).first
|
||||
toggled = False
|
||||
if _count(tp):
|
||||
try:
|
||||
tp.click()
|
||||
toggled = True
|
||||
except Exception:
|
||||
pass
|
||||
remember = popover.get_by_label("Remember for this model").first
|
||||
if _count(remember):
|
||||
try:
|
||||
remember.check()
|
||||
except Exception:
|
||||
remember.click()
|
||||
btn = primary_button(popover)
|
||||
if btn is not None and btn.is_enabled():
|
||||
btn.click()
|
||||
page.wait_for_timeout(1500)
|
||||
cfg = read_configs()
|
||||
has_adv = any(
|
||||
e.get("tensorParallel") or e.get("kvCacheDtype") for e in config_entries(cfg)
|
||||
)
|
||||
if toggled and has_adv:
|
||||
info("OK advanced: tensorParallel/kvCacheDtype persisted")
|
||||
else:
|
||||
runtime_warn(
|
||||
f"advanced persistence not observed (toggled={toggled}, "
|
||||
f"entries={json.dumps(config_entries(cfg))[:300]})"
|
||||
)
|
||||
else:
|
||||
runtime_warn("could not open run-settings for the advanced-persist check")
|
||||
close_picker()
|
||||
except Exception as e:
|
||||
runtime_warn(f"advanced-persist check errored: {e}")
|
||||
|
||||
# ─────────────────────────────────────────────────────
|
||||
# 5. Legacy migration is idempotent (gates in CI via soft_fail).
|
||||
# Seed a pre-feature unsloth_load_settings store, confirm it migrates once
|
||||
# with the value preserved, then reload with a fresh legacy seed and confirm
|
||||
# the migration does not re-run, duplicate, or clobber. Re-running on every
|
||||
# reload was the regression that reverted the predecessor PR.
|
||||
# ─────────────────────────────────────────────────────
|
||||
step("legacy unsloth_load_settings migrates once and stays idempotent")
|
||||
try:
|
||||
legacy_key = f"{GGUF_REPO}::{GGUF_VARIANT}"
|
||||
legacy = {
|
||||
legacy_key: {
|
||||
"contextLength": DISTINCT_CTX,
|
||||
"kvCacheDtype": "q8_0",
|
||||
"tensorParallel": True,
|
||||
}
|
||||
}
|
||||
robust_evaluate(
|
||||
page,
|
||||
"(seed) => {"
|
||||
" localStorage.setItem('unsloth_load_settings', JSON.stringify(seed));"
|
||||
" localStorage.removeItem('unsloth_model_configs');"
|
||||
" localStorage.removeItem('unsloth_model_configs_migrated');"
|
||||
" return true;"
|
||||
"}",
|
||||
arg = legacy,
|
||||
)
|
||||
page.reload()
|
||||
composer = page.locator('textarea[aria-label="Message input"]')
|
||||
composer.wait_for(state = "visible", timeout = 60_000)
|
||||
# Opening the picker config forces the store to read (which migrates).
|
||||
popover = open_picker()
|
||||
open_config(popover, MODEL_HINT)
|
||||
page.wait_for_timeout(800)
|
||||
cfg_first = read_configs()
|
||||
migrated_ctx = any(
|
||||
e.get("customContextLength") == DISTINCT_CTX for e in config_entries(cfg_first)
|
||||
)
|
||||
if migrated_ctx:
|
||||
info(f"OK migration: legacy context {DISTINCT_CTX} preserved after migrating")
|
||||
else:
|
||||
soft_fail(
|
||||
f"legacy context {DISTINCT_CTX} not migrated into unsloth_model_configs "
|
||||
f"(got {json.dumps(cfg_first)[:400]})"
|
||||
)
|
||||
flag_first = robust_evaluate(
|
||||
page, "() => localStorage.getItem('unsloth_model_configs_migrated')"
|
||||
)
|
||||
if flag_first != "1":
|
||||
soft_fail(f"migration flag not set after migrating (got {flag_first!r})")
|
||||
shoot("09-after-migration")
|
||||
close_picker()
|
||||
|
||||
# Idempotency: a second reload with a DIFFERENT legacy entry must not re-run
|
||||
# the migration (the persistent flag blocks it), so the new key must not leak
|
||||
# in, nothing duplicates, and the migrated value is untouched.
|
||||
if migrated_ctx:
|
||||
probe_key = "unsloth/__idem_probe__::Q4_K_M"
|
||||
robust_evaluate(
|
||||
page,
|
||||
"(seed) => {"
|
||||
" localStorage.setItem('unsloth_load_settings', JSON.stringify(seed));"
|
||||
" return true;"
|
||||
"}",
|
||||
arg = {probe_key: {"contextLength": DISTINCT_CTX + 2048, "tensorParallel": True}},
|
||||
)
|
||||
page.reload()
|
||||
composer.wait_for(state = "visible", timeout = 60_000)
|
||||
popover = open_picker()
|
||||
open_config(popover, MODEL_HINT)
|
||||
page.wait_for_timeout(800)
|
||||
cfg_second = read_configs()
|
||||
keys_first = set(cfg_first.keys())
|
||||
keys_second = set(cfg_second.keys())
|
||||
new_keys = keys_second - keys_first
|
||||
still_has_ctx = any(
|
||||
e.get("customContextLength") == DISTINCT_CTX for e in config_entries(cfg_second)
|
||||
)
|
||||
if new_keys:
|
||||
soft_fail(
|
||||
"legacy migration re-ran on a second reload (persistent flag "
|
||||
f"ignored): new keys {sorted(new_keys)}"
|
||||
)
|
||||
elif keys_second != keys_first:
|
||||
soft_fail(
|
||||
"legacy migration dropped entries on a second reload: "
|
||||
f"{sorted(keys_first)} -> {sorted(keys_second)}"
|
||||
)
|
||||
elif not still_has_ctx:
|
||||
soft_fail("legacy migration clobbered the migrated context on a second reload")
|
||||
else:
|
||||
info(
|
||||
"OK migration idempotent: second reload did not re-migrate, duplicate, or clobber"
|
||||
)
|
||||
shoot("10-after-second-reload")
|
||||
close_picker()
|
||||
except Exception as e:
|
||||
soft_fail(f"migration idempotency check errored: {e}")
|
||||
|
||||
# ─────────────────────────────────────────────────────
|
||||
if page_errors:
|
||||
fail(f"page errors during run: {page_errors[:3]!r}")
|
||||
|
||||
browser.close()
|
||||
|
||||
if _failed:
|
||||
print(f"[ui-modelcfg] RESULT: FAIL ({len(_failed)} issue(s))", flush = True)
|
||||
for m in _failed:
|
||||
print(f"[ui-modelcfg] - {m}", flush = True)
|
||||
sys.exit(1)
|
||||
print("[ui-modelcfg] RESULT: PASS", flush = True)
|
||||
sys.exit(0)
|
||||
241
tests/studio/test_cached_model_path_selection.py
Normal file
241
tests/studio/test_cached_model_path_selection.py
Normal file
|
|
@ -0,0 +1,241 @@
|
|||
# SPDX-License-Identifier: AGPL-3.0-only
|
||||
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
"""Variant-file selection guards for the cached-model-path endpoint.
|
||||
|
||||
The Copy path / Reveal endpoint must resolve a quant label to the same file
|
||||
the variant menus offer: MTP drafters, mmproj vision adapters, and big-endian
|
||||
builds are excluded, and directory layouts (``BF16/model-00001-of-....gguf``)
|
||||
resolve their label from the snapshot-relative path, not the basename.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
|
||||
|
||||
def _find_repo_root() -> Path | None:
|
||||
env = os.environ.get("UNSLOTH_REPO_ROOT")
|
||||
if env:
|
||||
p = Path(env).resolve()
|
||||
if (p / "studio" / "backend").is_dir():
|
||||
return p
|
||||
here = Path(__file__).resolve()
|
||||
for parent in (here, *here.parents):
|
||||
if (parent / "studio" / "backend").is_dir():
|
||||
return parent
|
||||
return None
|
||||
|
||||
|
||||
_REPO_ROOT = _find_repo_root()
|
||||
if _REPO_ROOT is None:
|
||||
pytest.skip(
|
||||
"Could not locate studio/backend. Set UNSLOTH_REPO_ROOT or run from "
|
||||
"the repository checkout.",
|
||||
allow_module_level = True,
|
||||
)
|
||||
|
||||
_STUDIO_BACKEND = _REPO_ROOT / "studio" / "backend"
|
||||
if str(_STUDIO_BACKEND) not in sys.path:
|
||||
sys.path.insert(0, str(_STUDIO_BACKEND))
|
||||
|
||||
pytest.importorskip("fastapi")
|
||||
pytest.importorskip("huggingface_hub")
|
||||
|
||||
try:
|
||||
from routes import models as routes_models
|
||||
except Exception as exc:
|
||||
pytest.skip(f"studio backend import unavailable: {exc}", allow_module_level = True)
|
||||
|
||||
from fastapi import HTTPException
|
||||
|
||||
|
||||
def test_plain_quant_label_resolves():
|
||||
assert routes_models._main_variant_gguf_label("Model-Q8_0.gguf") == "Q8_0"
|
||||
|
||||
|
||||
def test_mtp_drafter_in_subdir_is_excluded():
|
||||
assert routes_models._main_variant_gguf_label("MTP/Model-Q8_0-MTP.gguf") is None
|
||||
|
||||
|
||||
def test_mtp_drafter_root_prefix_is_excluded():
|
||||
assert routes_models._main_variant_gguf_label("mtp-Model-Q8_0.gguf") is None
|
||||
|
||||
|
||||
def test_mmproj_adapter_is_excluded():
|
||||
assert routes_models._main_variant_gguf_label("mmproj-Model-F16.gguf") is None
|
||||
|
||||
|
||||
def test_directory_layout_quant_resolves_from_parent_dir():
|
||||
assert routes_models._main_variant_gguf_label("BF16/Model-00001-of-00002.gguf") == "BF16"
|
||||
|
||||
|
||||
def test_big_endian_build_is_excluded():
|
||||
assert routes_models._main_variant_gguf_label("Model-Q8_0-BE.gguf") is None
|
||||
|
||||
|
||||
def test_non_gguf_file_is_excluded():
|
||||
assert routes_models._main_variant_gguf_label("config.json") is None
|
||||
|
||||
|
||||
def test_normalized_quant_label_ignores_separators():
|
||||
assert routes_models._normalized_quant_label("UD-Q4_K_XL") == "udq4kxl"
|
||||
assert routes_models._normalized_quant_label("Q8-0") == routes_models._normalized_quant_label(
|
||||
"Q8_0"
|
||||
)
|
||||
|
||||
|
||||
def _revision(
|
||||
snapshot: Path,
|
||||
last_modified: float,
|
||||
names: list[str],
|
||||
size_on_disk: int = 4,
|
||||
) -> SimpleNamespace:
|
||||
files = []
|
||||
for name in names:
|
||||
path = snapshot / name
|
||||
path.parent.mkdir(parents = True, exist_ok = True)
|
||||
path.write_bytes(b"x" * size_on_disk)
|
||||
files.append(
|
||||
SimpleNamespace(
|
||||
file_name = name,
|
||||
file_path = path,
|
||||
blob_path = path,
|
||||
size_on_disk = size_on_disk,
|
||||
)
|
||||
)
|
||||
return SimpleNamespace(snapshot_path = snapshot, last_modified = last_modified, files = files)
|
||||
|
||||
|
||||
def _patch_cache(monkeypatch, tmp_path: Path, revisions: list[SimpleNamespace]) -> None:
|
||||
repo = SimpleNamespace(
|
||||
repo_id = "Org/Repo",
|
||||
repo_type = "model",
|
||||
repo_path = tmp_path,
|
||||
revisions = revisions,
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
routes_models, "_all_hf_cache_scans", lambda: [SimpleNamespace(repos = [repo])]
|
||||
)
|
||||
|
||||
|
||||
def _repo(root: Path, revisions: list[SimpleNamespace]) -> SimpleNamespace:
|
||||
return SimpleNamespace(
|
||||
repo_id = "Org/Repo",
|
||||
repo_type = "model",
|
||||
repo_path = root,
|
||||
revisions = revisions,
|
||||
)
|
||||
|
||||
|
||||
def _patch_caches(monkeypatch, repos: list[SimpleNamespace]) -> None:
|
||||
monkeypatch.setattr(
|
||||
routes_models,
|
||||
"_all_hf_cache_scans",
|
||||
lambda: [SimpleNamespace(repos = [repo]) for repo in repos],
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("newest_first", [True, False])
|
||||
def test_variant_resolves_from_newest_revision(monkeypatch, tmp_path, newest_first):
|
||||
old = _revision(tmp_path / "snapshots" / "aaa", 1_000.0, ["Model-Q4_K_M.gguf"])
|
||||
new = _revision(tmp_path / "snapshots" / "bbb", 2_000.0, ["Model-Q4_K_M.gguf"])
|
||||
_patch_cache(monkeypatch, tmp_path, [new, old] if newest_first else [old, new])
|
||||
resolved = routes_models._resolve_cached_model_path("Org/Repo", "Q4_K_M")
|
||||
assert resolved == tmp_path / "snapshots" / "bbb" / "Model-Q4_K_M.gguf"
|
||||
|
||||
|
||||
def test_sharded_variant_resolves_first_split(monkeypatch, tmp_path):
|
||||
rev = _revision(
|
||||
tmp_path / "snapshots" / "aaa",
|
||||
1_000.0,
|
||||
["Model-Q4_K_M-00002-of-00002.gguf", "Model-Q4_K_M-00001-of-00002.gguf"],
|
||||
)
|
||||
_patch_cache(monkeypatch, tmp_path, [rev])
|
||||
resolved = routes_models._resolve_cached_model_path("Org/Repo", "Q4_K_M")
|
||||
assert resolved.name == "Model-Q4_K_M-00001-of-00002.gguf"
|
||||
|
||||
|
||||
def test_variant_only_in_older_revision_resolves(monkeypatch, tmp_path):
|
||||
old = _revision(tmp_path / "snapshots" / "aaa", 1_000.0, ["Model-Q4_K_M.gguf"])
|
||||
new = _revision(tmp_path / "snapshots" / "bbb", 2_000.0, ["Model-Q8_0.gguf"])
|
||||
_patch_cache(monkeypatch, tmp_path, [new, old])
|
||||
resolved = routes_models._resolve_cached_model_path("Org/Repo", "Q4_K_M")
|
||||
assert resolved == tmp_path / "snapshots" / "aaa" / "Model-Q4_K_M.gguf"
|
||||
|
||||
|
||||
def test_missing_newest_file_falls_back_to_older_revision(monkeypatch, tmp_path):
|
||||
old = _revision(tmp_path / "snapshots" / "aaa", 1_000.0, ["Model-Q4_K_M.gguf"])
|
||||
new_snapshot = tmp_path / "snapshots" / "bbb"
|
||||
new_snapshot.mkdir(parents = True)
|
||||
new = SimpleNamespace(
|
||||
snapshot_path = new_snapshot,
|
||||
last_modified = 2_000.0,
|
||||
files = [
|
||||
SimpleNamespace(
|
||||
file_name = "Model-Q4_K_M.gguf",
|
||||
file_path = new_snapshot / "Model-Q4_K_M.gguf",
|
||||
)
|
||||
],
|
||||
)
|
||||
_patch_cache(monkeypatch, tmp_path, [new, old])
|
||||
resolved = routes_models._resolve_cached_model_path("Org/Repo", "Q4_K_M")
|
||||
assert resolved == tmp_path / "snapshots" / "aaa" / "Model-Q4_K_M.gguf"
|
||||
|
||||
|
||||
def test_variant_resolves_across_all_cache_roots(monkeypatch, tmp_path):
|
||||
first_root = tmp_path / "active"
|
||||
second_root = tmp_path / "default"
|
||||
old = _revision(
|
||||
first_root / "snapshots" / "aaa",
|
||||
1_000.0,
|
||||
["Model-Q4_K_M.gguf"],
|
||||
)
|
||||
new = _revision(
|
||||
second_root / "snapshots" / "bbb",
|
||||
2_000.0,
|
||||
["Model-Q4_K_M.gguf"],
|
||||
)
|
||||
_patch_caches(
|
||||
monkeypatch,
|
||||
[_repo(first_root, [old]), _repo(second_root, [new])],
|
||||
)
|
||||
resolved = routes_models._resolve_cached_model_path("Org/Repo", "Q4_K_M")
|
||||
assert resolved == second_root / "snapshots" / "bbb" / "Model-Q4_K_M.gguf"
|
||||
|
||||
|
||||
def test_repo_path_matches_largest_visible_cache_entry(monkeypatch, tmp_path):
|
||||
first_root = tmp_path / "active"
|
||||
second_root = tmp_path / "default"
|
||||
small = _revision(
|
||||
first_root / "snapshots" / "aaa",
|
||||
2_000.0,
|
||||
["Model-Q8_0.gguf"],
|
||||
size_on_disk = 4,
|
||||
)
|
||||
large = _revision(
|
||||
second_root / "snapshots" / "bbb",
|
||||
1_000.0,
|
||||
["Model-Q8_0.gguf"],
|
||||
size_on_disk = 8,
|
||||
)
|
||||
_patch_caches(
|
||||
monkeypatch,
|
||||
[_repo(first_root, [small]), _repo(second_root, [large])],
|
||||
)
|
||||
resolved = routes_models._resolve_cached_model_path("Org/Repo", None)
|
||||
assert resolved == second_root / "snapshots" / "bbb"
|
||||
|
||||
|
||||
def test_unknown_variant_raises_404(monkeypatch, tmp_path):
|
||||
rev = _revision(tmp_path / "snapshots" / "aaa", 1_000.0, ["Model-Q8_0.gguf"])
|
||||
_patch_cache(monkeypatch, tmp_path, [rev])
|
||||
with pytest.raises(HTTPException) as excinfo:
|
||||
routes_models._resolve_cached_model_path("Org/Repo", "Q4_K_M")
|
||||
assert excinfo.value.status_code == 404
|
||||
assert "Q4_K_M" in excinfo.value.detail
|
||||
Some files were not shown because too many files have changed in this diff Show more
Loading…
Add table
Add a link
Reference in a new issue