The pill wired the request end of the loop but the response was lost
on the client: the backend emits a `tool_end` _toolEvent carrying the
base64 PNG on `image_b64` / `image_mime`, but the chat-adapter only
read the `result` string and the generic ToolFallback printed the
prompt as JSON args with an empty Result block -- the "I see no
image" symptom in the chat.
- chat-adapter: when the closing `tool_end` is for `image_generation`,
repackage `image_b64` + `image_mime` (+ size/quality/background)
into a structured result object instead of dropping them.
- New `ImageGenerationToolUI` reads that result and renders the image
inline via `<img src="data:image/...;base64,...">` with the prompt
as a caption. Falls back to a spinner while the request is still
running.
- Register the component under `image_generation` in thread.tsx's
tools.by_name map so it preempts ToolFallback for this tool only.
#5685 wired the backend to honor `prompt_cache_ttl` on the request,
but there was no UI to actually pick it -- every Studio chat ended up
on Anthropic's default 5 minute pool. This adds a Cache TTL selector
to the chat settings sheet's Provider section, visible only when the
provider supports the choice (Anthropic today) and Prompt caching is
on.
- New `promptCacheTtl?: "5m" | "1h"` on `ExternalProviderConfig`.
Normalizer drops the field on providers that don't support the
choice so localStorage stays clean across provider swaps.
- `supportsProviderPromptCacheTtl` + `isPromptCacheTtl` helpers so
the picker, normalizer, and adapter all agree on which values are
valid.
- Settings sheet renders a small Select (5 minutes / 1 hour) right
under the Prompt caching switch when the toggle is on; flipping
it persists on the provider config like the other per-provider
knobs.
- chat-adapter passes `prompt_cache_ttl` on outbound requests when
the value is valid; omitted otherwise so the backend keeps
inheriting Anthropic's 5m default.
The backend already wires OpenAI's Responses-API image_generation
server tool: when `enabled_tools` carries "image_generation" on an
OpenAI cloud request, _stream_openai_responses appends
`{type: "image_generation"}` to the request's tools array and emits
`image_generation_call` output items back to the assistant stream
(see backend/core/inference/external_provider.py and
backend/tests/test_openai_image_generation.py for the round-trip).
This wires the frontend half so a user can actually opt into it from
the composer next to the Search and Code pills, instead of the tool
sitting dormant.
- `providerSupportsBuiltinImageGeneration` gates on OpenAI cloud
(`api.openai.com`) + a Responses-API model prefix (gpt-5.x, o3).
Mirror of the backend's `is_openai_cloud` guard so the pill is hidden
on custom OpenAI-compat backends (ollama / llama.cpp / vLLM) that
report `provider_type="openai"` but would 400 on the tool.
- New `imageToolsEnabled` flag in chat-runtime-store, persisted under
`unsloth_chat_image_tools_enabled` and reset on model change in
chat-page exactly like `codeToolsEnabled`.
- `chat-adapter` appends "image_generation" to `enabled_tools` and
flips `enable_tools: true` when the pill is on, so the existing
backend dispatch picks it up.
- Composer renders an Images pill (lucide `ImageIcon`) immediately
after the Code pill, only when the active model advertises the
capability. The in-thread composer (assistant-ui/thread.tsx) gets
the matching `ImagesToggle` for parity.
The first pass only wired the localStorage mirror into `setCheckpoint`,
but the main chat-page picker actually selects an external model by
calling `setParams({ ...store.params, checkpoint: value })`. That path
never hit `setCheckpoint`, so the persisted slot stayed empty and a
refresh fell back to whatever `/api/inference/status.active_model`
returned -- the previously loaded local model (Qwen3.5 etc) or null
("Select model") when nothing was loaded locally.
Mirror the persistence in `setParams` whenever the checkpoint changes
so every entry point converges on the same behavior. `setCheckpoint`
still does it directly so the load path (compare, GGUF auto-load,
gemma fallback in chat-adapter) keeps working.
* Add Anthropic prompt guards for disabled tools
* fix: merge Anthropic tool guard into structured system prompts
* fix: scope Anthropic disabled-tool guard wording
* chore: adjust claude guard prompt
* chore: add openai to list of prompt guarded providers
* Studio: include web_fetch in the per-turn disabled-tool guard
Add webFetchEnabledForThisTurn alongside webSearchEnabledForThisTurn
and codeExecEnabledForThisTurn. Use it in the enabled_tools payload
so web_fetch follows the Search pill the same way web_search does,
and mention "web fetch" in the disabled-tool guard prose on providers
that ship the tool (Anthropic today; other providers stay inert via
providerSupportsBuiltinWebFetch).
---------
Co-authored-by: Roland Tannous <115670425+rolandtannous@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Selecting a connected external provider (Anthropic, OpenAI, Google, etc.)
and refreshing the page reverted the picker back to no selection. Root
cause is that `PersistedInferenceParams` in `chat-settings-api.ts`
excludes `checkpoint` from the server-side settings payload by design.
Local model selections survive refresh because the backend re-derives
them from `/api/inference/status.active_model`, but external selections
have no backend mirror, so they were lost.
Fix: persist `external::*` checkpoints to a small dedicated
`localStorage` key (`unsloth_chat_last_external_checkpoint`) and hydrate
from it on store init. Local checkpoints continue to come from the
backend status as before; only external ids are mirrored client-side.
`setCheckpoint` writes the key when an external id is selected and
clears it when switching back to a local id, and `clearCheckpoint`
clears it so the picker does not snap back after an explicit reset.
Deleting a connection in one browser left the same connection stuck in
every other browser/tab. The user could not delete or edit it from there
because the local state never caught up with the server, and clicks
either no-op'd or threw on a missing-row backend response.
Two pieces caused the bug:
1. `ChatProvidersSettings` ran its backend sync once on mount and then
silently kept localStorage providers whenever `listProviderConfigs`
returned an empty array, on the assumption that an empty server
response had to be a transient glitch. That assumption is wrong when
another browser removed the last connection. With the guard gone,
trust any successful API response, including an empty list. A focus /
visibilitychange listener now triggers a silent re-sync so the dialog
does not need to be closed and reopened to pick up remote deletes.
2. `deleteProviderConfig` threw on HTTP 404, so once Browser A deleted a
connection, Browser B's "Delete" click failed and the local row stuck
around. Treat 404 as success: the server's job is already done and
the local cache only needs to be pruned.
* feat: Persist chat history in backend storage
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address chat tombstone batching review
* fix: update desktop auth routes stub
* chat db settings storage
* chat db settings routes
* chat db settings client
* chat db settings store
* chat db settings wiring
* chat db history storage
* chat db settings migration
* chat db settings fallback
* chat db container metadata
* chat db legacy migration fixes
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* chat ci auth background reads
* chat auth storage fixes
* chat migration final fixes
* chat export batch message lookup
* chat history review fixes
* chat prune sync fix
* chat settings hydration retry
* gate settings persistence
* Scope chat-history rows by subject; fix hijack, clear-confirm, hydrate race
Backend storage and routes:
- chat_threads / chat_messages / chat_settings carry a NOT NULL subject
column with composite PRIMARY KEY (id, subject). Two authenticated
identities can no longer see or wipe each other's data.
- Pre-existing rows on an existing studio.db migrate under sentinel
subject __legacy_unscoped__ via rename + rebuild + copy; single-user
installs see no behavior change.
- ON CONFLICT(id, subject) DO UPDATE ... WHERE chat_messages.thread_id =
excluded.thread_id refuses cross-thread re-parenting via upsert.
upsert_chat_message + sync_chat_messages now raise
ChatMessageThreadMismatch which the routes map to HTTP 409.
- replace_thread_messages rejects body messages whose threadId does not
match the URL thread (HTTP 400) instead of silently rewriting them.
- DELETE /api/chat requires ?confirm=true, returns row count, logs the
subject and count.
- upsert_chat_settings_merge does read + deep-merge + write inside a
single BEGIN IMMEDIATE so concurrent writers no longer drop each
other's updates. The route delegates to this helper.
- New POST /api/chat/messages:batch returns {thread_id -> messages[]}
for many threads in one HTTP call. Subject-scoped. Unknown ids return
empty lists instead of 404 so the sidebar/search caller can rebuild
atomically.
Frontend:
- chat-runtime-store: hydrate-failure catch sets settingsHydrated:true
so a transient backend blip no longer permanently disables
persistence. setParams bumps inferenceParamMutationVersions
unconditionally so a slow hydration response cannot clobber a
pre-hydrate user edit. saveSettingsPatch replaces the serial chain
with a debounced pendingPatch + deep merge; flush on beforeunload.
- chat-history-storage: clearStoredChats returns ClearStoredChatsResult
distinguishing backend / legacy / both outcomes.
listStoredChatThreadsWithMessages uses the batched fetch (one HTTP
call) instead of Promise.all per-thread; legacy Dexie fallback only
fires when the batch result is empty.
- chat-api: batchListChatMessages with graceful 404 / 405 fallback to
per-thread listChatMessages for older servers.
- chat-thread-tombstones: store {id, deletedAt} tuples with 90-day GC
and a 5000-entry cap so localStorage stays bounded. Back-compat reads
pre-fix plain strings. Adds removeChatThreadTombstones (rollback) and
clearAllChatThreadTombstones (post-legacy-purge clean-up).
- use-chat-sidebar-items: deleteChatItem tombstones synchronously
BEFORE the backend round-trip and rolls back on failure (restores
pre-PR optimistic UX). 300 ms trailing debounce on
CHAT_HISTORY_UPDATED_EVENT plus requestSeq guard so stream-time event
bursts produce at most one fetch per quiet window.
Tests:
- studio/backend/tests/pr5272_sim/ adds 64 regression tests covering
schema migration from pre-fix shape, subject scoping, cross-thread
hijack, bulk-replace mismatch, clear-confirm, concurrent settings,
unicode + 2MB content + SQL-injection-safe binding, chunking
boundary at 900 and 901 ids, batched endpoint (multi-subject + 1200
ids + per-thread order), and grep contracts for the frontend patches.
test_chat_history_storage.py updated to pass subject.
Verified locally on Linux + macOS + Windows GitHub Actions runners
(staging fork): 64 pass + 2 from the PR's own backend test on all
three OSes.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Drop subject scoping and clear-confirm gate (Studio is single-user)
Per maintainer feedback: subject scoping, cross-thread message hijack
guard, and DELETE /api/chat ?confirm=true gate are unnecessary because
Studio is intentionally single-user (the client already shows a confirm
dialog before clear-all).
This commit reverts those backend changes and keeps only the
non-multi-user pieces from the earlier fix commit:
- studio_db.py: restored to pre-fix shape; adds upsert_chat_settings_merge
which does atomic read + deep-merge + write under BEGIN IMMEDIATE so
two concurrent slider drags cannot drop one another's updates.
- routes/chat_history.py: restored; put_settings now calls the atomic
merge instead of doing the read-merge-write across three separate
connections. Adds POST /api/chat/messages:batch to collapse the
sidebar/search rebuild from N round-trips to 1.
- frontend/api/chat-api.ts: align batchListChatMessages request and
response keys with the backend (threadIds / messagesByThreadId).
- tests/test_chat_history_storage.py: add atomic-merge concurrency test,
deep-merge nested-key test, and 901-id chunking-boundary test.
- Drop the pr5272_sim test directory (those tests covered the reverted
subject-scoping/hijack/confirm behavior).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix sidebar delete crash, keepalive on settings beforeunload flush, search rebuild race
Two correctness bugs and one perf race surfaced by a fresh code review of
the prior fix commit:
- chat-api.ts: notifyChatHistoryUpdated was declared as a non-exported
function, but use-chat-sidebar-items.ts imports it. The import would
fail tsc with TS2305 and at runtime the optimistic-delete and
delete-failure rollback paths would both throw.
- chat-runtime-store.ts + chat-settings-api.ts + chat-settings-storage.ts:
the beforeunload settings flush is now actually keepalive. Without it
the browser cancels the in-flight PUT on tab close, so the last slider
drag is silently dropped (which is exactly the case the
debounce+beforeunload combination was meant to protect against).
- use-chat-search-index.ts: rebuilds now coalesce with a 300ms trailing
debounce and discard out-of-order responses via a requestSeq guard.
Matches the sibling pattern in use-chat-sidebar-items.ts so two rapid
CHAT_HISTORY_UPDATED_EVENTs (run-start + run-end save during a turn)
cannot land with stale data winning.
- chat-thread-tombstones.ts: drop dead clearAllChatThreadTombstones with
no call sites; Dexie is never wiped so the function has no use.
* fix(studio): protect chat persistence writes
* fix(studio): align chat history clear semantics
* fix(studio): show partial chat clear feedback
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio): preserve chat persistence fallbacks
* fix(studio): harden chat thread persistence checks
* Preserve chat message timestamps
* Gate chat stream on history save
* Make chat thread backfill best effort
* Avoid chat message 404 probe
* Tighten chat legacy fallbacks
* chat: server-side ledger so legacy Dexie import is recoverable
The boolean localStorage sentinel
(unsloth_chat_legacy_imported_to_studio_db) made importLegacyChatsIfNeeded
non-recoverable: deleting studio.db while the browser keeps the flag
silently hides every legacy Dexie thread from the sidebar (verified by
the 3-GPU validation probe; matches the third review comment on PR
#5272). Same trap fires for browser-profile sync to a fresh machine
and any other path that wipes studio.db while keeping IndexedDB.
Source of truth moves into studio.db itself via a new
chat_legacy_import_log table keyed by legacy thread id. The ledger
disappears together with studio.db, so the next launch re-runs the
import from whatever Dexie still holds. localStorage stays as a
per-session perf hint only.
Performance, all bounded by the three new fast-paths before any
backend work:
A) localStorage hint says "imported earlier in this session" -- 0
network, ~0 ms. Covers the warm sidebar mount.
B) indexedDB.databases() reports no "unsloth-chat" DB -- 0 network,
~1 ms. Covers every new user who never had the old browser-only
Studio (the common case after launch).
C) db.threads.count() + db.messages.count() are both 0 -- 0 network,
~5 ms. Covers returning users who migrated long ago and Dexie was
never repopulated.
Only when all three miss does the code talk to the backend
(GET /api/chat/import-ledger -> diff vs Dexie -> existing import path
-> POST /api/chat/import-ledger to record what was just imported).
Per-thread tracking is enough because Dexie is read-only after this
PR; a thread's message set does not grow.
Backend deployments that predate the import-ledger routes are
handled transparently: the client treats 404/405 as an empty ledger
and re-runs the (idempotent via UPSERT) import on next launch.
Changes:
- storage/studio_db.py: new chat_legacy_import_log table (WITHOUT
ROWID, PK on legacy_thread_id) + list_chat_legacy_import_log() +
record_chat_legacy_import_log() (idempotent batch UPSERT).
- routes/chat_history.py: GET + POST /api/chat/import-ledger with the
obvious request/response models.
- frontend api/chat-api.ts: listChatImportLedger() (returns a Set for
O(1) diff) + recordChatImportLedger(), both with 404/405 fallback.
- frontend utils/chat-history-storage.ts: importLegacyChatsIfNeeded
gains three fast-paths, ledger fetch on the slow path, and writes
the ledger after a successful import. The localStorage helper is
unchanged on the surface; it just stops being authoritative.
- tests: 5 new test_legacy_import_log_* cases (empty default, record
+ list round-trip, idempotency, input dedup, empty/null ignore).
All 9 pre-existing tests still pass.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Make the legacy-import recovery actually recoverable
The previous commit added a server-side ledger to make Dexie -> studio.db
import recoverable after a studio.db wipe, but the localStorage perf hint
still short-circuited the import gate before the ledger was ever consulted.
After a wipe, the hint stayed "true" and the bulk re-import never ran -- the
ledger sat empty and only the per-thread lazy materialize-on-continue path
restored data.
Changes:
- Remove the localStorage short-circuit from importLegacyChatsIfNeeded so
the ledger is checked on every fresh tab. legacyChatImportPromise keeps
the per-session cache; the hint now only matters for the listing paths.
- Batch the slow path: one db.messages.where().anyOf().toArray() and one
batchListChatMessages() instead of 2N round-trips. At 1k threads this
drops a multi-second blocking import to a single request pair.
- recordChatImportLedger returns {accepted, inserted, supported}. The
localStorage hint is only flipped when supported is true, so old
backends (404 / 405 / 501) no longer permanently poison recovery.
- Ledger backfill: threads already present in chat_threads but missing
from the ledger now get added too, so old-FE-then-new-FE deployments
don't redo the diff every launch.
- Backend response field renamed recorded -> {accepted, inserted}.
accepted is the deduped non-empty input count; inserted is the rows
actually new (via INSERT ... RETURNING). Bounded by Field(max_length=
10_000) on the request payload.
- Storage helpers renamed: chat_legacy_import_log -> chat_legacy_imports,
record_* -> upsert_* to match the existing noun/verb conventions.
- DEXIE_DB_NAME exported from db.ts; duplicate constant in
chat-history-storage.ts removed.
- 3 new route-level tests for /api/chat/import-ledger covering the
round-trip, the (accepted, inserted) split, and the 10k payload cap.
All 18 chat-history tests pass.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: shine1i <wasimysdev@gmail.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Studio: wire Anthropic web_fetch server-side tool
Studio's Anthropic passthrough only forwarded web_search and
code_execution when enabled_tools was set. Asking Claude through Studio
to fetch a URL produced no fetch (the tool was not in the outbound
tools array), so users had to fall back to web_search even when they
already had the exact URL they wanted.
This change opts in web_fetch_20250910 when enabled_tools contains
"web_fetch". The new tool entry is appended alongside any existing
web_search / code_execution entries:
{"type": "web_fetch_20250910", "name": "web_fetch", "max_uses": 5}
No anthropic-beta header is required (web_fetch is GA); the existing
code-execution-2025-08-25 flag continues to merge cleanly when both
tools are enabled in the same turn.
SSE translation mirrors the web_search path. A `server_tool_use` block
with name="web_fetch" emits a `tool_start` _toolEvent carrying the
URL the model asked to fetch; the matching `web_fetch_tool_result`
block emits a `tool_end` _toolEvent whose result string follows the
Title / URL / Snippet shape parseSourcesFromResult on the frontend
already expects, so the source pill renders identically. Error blocks
(`web_fetch_tool_error`) are surfaced as "Error: <error_code>" matching
the code_execution error path.
The final "Anthropic stream complete" log line picks up web_fetch_
requested / web_fetch_invocations / web_fetch_urls so support reports
of "the model did not fetch anything" can be triaged from the log.
Verified end to end against claude-haiku-4-5 with
`enabled_tools=["web_fetch"]`: the model emitted tool_start with
url=https://example.com and tool_end with the page Title + URL +
Snippet, plus the assistant message correctly read back "Example
Domain" as the title.
Tests:
- 5 new unit tests in test_anthropic_web_fetch.py covering tool
registration, the combined web_search + web_fetch + code_execution
request body, the pill-off case, and SSE translation for both
success and error paths.
- All 242 existing Anthropic + OpenAI provider tests still pass.
The enabled_tools field description in models/inference.py is updated
so OpenAPI consumers see the new option.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* web_fetch: title fallback to URL, log parse failures, drop dead checks
Three review nits on the previous commit:
1. `_format_web_fetch_result` left `title` empty when Anthropic omitted
`document.title`. The frontend `parseSourcesFromResult` only emits
a source pill when both `Title:` and `URL:` lines are present, so
fetches against pages without an HTML title tag silently lost
their citation in the UI. Fall back to `title = title or url`,
matching the web_search formatter.
2. The broad `except Exception` around `json.loads(buffer)` for the
web_fetch input swallowed the failure with no trace. Log at debug
so a malformed partial_json buffer can be triaged from the server
log without changing behavior.
3. `inner` was already sanitised to a dict at the matching
content_block_start and `_format_web_fetch_result` always returns
a non-empty string (defaulting to "(fetch complete)"), so the
`isinstance(inner, dict) else {}` guard and the
`result_text or "(fetch complete)"` fallback at the emit site
were dead code. Removed.
Added a test exercising the titleless path so the fallback stays
covered.
* chat-adapter: emit source pills for web_fetch tool calls
`parseSourcesFromResult` was only wired up for tool calls where
`toolName === "web_search"`, so the Title / URL / Snippet block the
backend formatter emits for `web_fetch_tool_result` never reached the
source-pill renderer. Users saw the raw tool result in the tool card
but the dedicated source-pill row at the message tail stayed empty.
Both web_search and web_fetch ship the same text shape today, so the
fix is to broaden the gate.
* Address review: wire web_fetch from Search pill + fix pause_turn truncation
Two reviewer follow-ups on the Anthropic web_fetch PR:
1. The backend tool wiring landed but the frontend chat-adapter
never put `web_fetch` in `enabled_tools`, so toggling the Search
pill only ever attached `web_search` -- web_fetch was unreachable
from the UI. Added providerSupportsBuiltinWebFetch() (Anthropic
today) and paired the entry with the existing Search pill, since
the canonical workflow is "search returns URLs, fetch reads
them" and there is no separate UI toggle yet.
2. `pause_turn` from Anthropic's stop_reason vocabulary fell through
the finish_reason map's "stop" default, which the OpenAI-format
client renders as end-of-message and truncates the answer. Per
the docs pause_turn means "Claude paused a long server-tool
turn (web_search / web_fetch) and will resume". Mapped to None
and skipped the chunk emission so the SSE stream still ends with
[DONE] on message_stop but no terminal finish_reason lands on
the client. While there: added explicit mappings for `tool_use`
(-> tool_calls) and `refusal` (-> content_filter) which were
also falling through to "stop".
Tests added: pause_turn emits no finish_reason, end_turn still
emits "stop", refusal maps to "content_filter".
Sourcing: https://platform.claude.com/docs/en/api/messages#response-stop-reason
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* studio/frontend: set per-route document.title
The browser tab title was hardcoded to "Unsloth Studio" in
index.html and never updated. Users running multiple Studio
installs (or browsing several threads in separate tabs) saw the
same tab label everywhere, making the OS / browser tab strip
useless for switching between them.
Map known route prefixes (Chat, Train, Data Recipes, Export,
Settings, Login, Onboarding, Change Password) to a "Label -
Unsloth Studio" tab title and update document.title from a small
effect inside RootLayout. Unknown routes keep the original
"Unsloth Studio".
Resolves#5659.
* studio/frontend: per-route document.title via staticData + useMatches
Address review feedback on #5660 (gemini-code-assist): move titles from
the centralized ROUTE_TITLES map in __root.tsx into each route's
`staticData: { title }` and read the deepest matched route's title via
`useMatches`. This co-locates the title with the route definition, so
renames or new routes only have to touch one file, and drops the
pathname.startsWith(...) string matching.
Routes given a title (everything that actually renders chrome):
- /chat -> "Chat"
- /studio -> "Train"
- /data-recipes -> "Data Recipes"
- /data-recipes/$recipeId -> "Data Recipes"
- /export -> "Export"
- /login -> "Login"
- /onboarding -> "Onboarding"
- /change-password -> "Change Password"
/settings and / both redirect on `beforeLoad`, so they never render and
don't need a title; they fall through to the default "Unsloth Studio".
The previous PR's ROUTE_TITLES + routeTitle() helper are removed from
__root.tsx. tsc + vite build clean; bundle confirms every route carries
its `staticData:{title:...}` and __root.tsx's useMatches selector walks
matches deepest-first.
* studio/frontend: type staticData.title via module augmentation + useLayoutEffect
- Augment `StaticDataRouteOption` so `createRoute({ staticData: { title } })` is typed at the leaves and the layout reads `match.staticData.title` without the inline cast.
- Switch the title-writing effect to `useLayoutEffect` so the tab title updates synchronously and doesn't flash the previous route's title for a frame during in-app navigation.
- Use " | " separator (web convention) for the document title.
* studio/frontend: Settings dialog drives document.title + revert separator to PR contract
12/12 reviewers flagged that /settings is a modal deep link whose route throws redirect in beforeLoad, so useMatches resolves to the post-auth route (usually /chat). The tab title therefore showed "Chat - Unsloth Studio" while the user was actually looking at the Settings dialog.
Fix:
- Subscribe to useSettingsDialogStore.open in __root.tsx and prefer "Settings" as the document title while the dialog is visible.
- Add staticData.title = "Settings" on /settings for the rare case beforeLoad returns without throwing (future refactor); the live source-of-truth is the dialog store since the redirect means the route never matches.
Also revert the document title separator from " | " back to " - " to match the PR description / acceptance contract that the previous round inadvertently broke.
* studio/frontend: tighten document-title comments
---------
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
* studio/frontend: fix onboarding CSP violations
Two onboarding-only CSP violations were showing up in the browser
console on a default install:
* `WizardSidebar` rendered the brand sticker from
`https://unsloth.ai/cgi/image/unsloth_sticker_no_shadow_*.png`,
which is not in the Studio CSP `img-src` allowlist. The sticker
rendered as a broken image.
* `Confetti` defaulted `globalOptions.useWorker` to `true`, so
`canvas-confetti` tried to spawn an OffscreenCanvas worker from
a `blob:` URL. CSP `script-src 'self'` blocks it; three blocked-
worker errors fired on the final wizard step.
Use the bundled `/sticker.png` for the brand image, and default
the Confetti wrapper to the main-thread fallback. CSP stays tight.
Resolves#5657.
* studio/frontend: harden CSP confetti fix + BASE_URL sticker
Address review feedback on #5658:
1. confetti.tsx
- Hoist the default globalOptions to a module-scope constant so the
prop default has a stable identity across renders (canvasRef's
dependency array no longer churns every render).
- Always force useWorker:false at the confetti.create site, regardless
of what the caller passed in globalOptions. Previously a caller that
set `{ resize: true }` would silently re-enable the worker and trip
the CSP block again.
- Add a lazily-mounted, module-scoped CSP-safe instance and route
ConfettiButton through it instead of the global confetti() (which
defaults to useWorker:true and would otherwise violate CSP).
2. confetti-fireworks.ts
- Replace the direct confetti(...) calls (global instance, default
worker on) with calls to a shared confetti.create instance with
useWorker:false. The guided-tour completion confetti no longer
trips the CSP block.
3. wizard-sidebar.tsx
- Use import.meta.env.BASE_URL prefix on the sticker src so the asset
still resolves when Studio is deployed under a subpath (e.g.
/studio/). Defaults to "/" so single-host installs are unchanged.
tsc clean, bun run build clean, bundle confirms the changes
(`{resize:!0,useWorker:!1}` appears in every relevant call site).
* studio/tour: preserve opts.zIndex on shared confetti fireworks canvas
Address chatgpt-codex-connector inline review on #5658 follow-up:
When canvas-confetti runs against a caller-provided canvas (which is
what we need for the CSP fix), the per-fire `zIndex` option is ignored
for stacking purposes -- the canvas element's own CSS `z-index` is what
the browser uses. The previous follow-up hard-coded the shared canvas
to `z-index:99999`, so callers that pass `opts.zIndex` (or expect the
old global-confetti behavior of being able to lower fireworks under an
overlay) silently lost that knob.
Apply `opts.zIndex` to the shared canvas's `style.zIndex` on each call
(default 99999 still used when omitted). Same default; behavior is now
restored for the lower/raise case.
The current only caller (`guided-tour.tsx` invoking
`fireConfettiFireworks()` with no args) is unaffected since it never
provided `opts.zIndex`. Public API contract is preserved.
* studio/frontend: drop dead ConfettiButton + BASE_URL onboarding mascots
- confetti.tsx: remove unused ConfettiButton + getSharedConfettiFire singleton (0 callsites)
- splash-screen.tsx, wizard-content.tsx: prefix sloth mascot paths with import.meta.env.BASE_URL so onboarding works under non-root subpaths
- confetti-fireworks.ts: drop dead per-fire zIndex from defaults (caller-provided canvas ignores it; we already drive stacking via canvas style)
* studio/frontend: BASE_URL on HF icon + race-safe shared fireworks init
- dataset-step.tsx: prefix the Hugging Face dataset-source icon with import.meta.env.BASE_URL so it resolves correctly under non-root deployments. Last onboarding asset that was still root-relative after the earlier BASE_URL sweep.
- confetti-fireworks.ts: cache the in-flight init promise in getSharedFire so two same-tick callers share the dynamic import and the appended overlay canvas. Previously two concurrent fireConfettiFireworks() calls each appended a fixed full-screen canvas and orphaned the first one.
* studio/frontend: tighten confetti CSP comments
---------
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
* studio/frontend: correct Think pill aria-label before model loads
`reasoningEnabled` defaults to true in the chat-runtime store, so on a
fresh /chat with no model the Think pill renders disabled + visually
off (LightbulbOffIcon, data-active="false"), but its aria-label still
reads "Disable thinking" -- screen readers announce it as if the
button is currently on. Add a `disabled` branch between
reasoningLockedOn and effectiveReasoningEnabled so the label reads
"Thinking (model not loaded)" while the button is unreachable, then
falls through to the normal enable/disable copy once a model is
loaded. Apply the same fix to the equivalent pill in shared-composer
(where the disabled flag is named `reasoningDisabled`).
* studio/frontend: Think pill distinguishes !modelLoaded vs unsupported reasoning
Address review feedback on #5655 (chatgpt-codex-connector + gemini-code-assist
both flagged the same edge case):
The previous `disabled` branch labeled the Think pill "Thinking (model not
loaded)" whenever the button was disabled, but `disabled` is defined as
`!(modelLoaded && effectiveSupportsReasoning)` (in thread.tsx) and
`!modelLoaded || !effectiveSupportsReasoning` (in shared-composer.tsx).
Both cover the second case where a model IS loaded but does not support
reasoning at all (e.g. Llama-3.2-1B-Instruct), which mislabeled the pill
for screen-reader users.
Split the branch so the no-model case keeps "Thinking (model not loaded)"
and the loaded-but-unsupported case reads "Thinking (not supported by this
model)". Locked-on / enabled / disabled labels are unchanged.
Verified by re-running the Playwright probe:
- no model -> aria-label "Thinking (model not loaded)"
- Llama-3.2-1B loaded -> aria-label "Thinking (not supported by this model)"
- reasoning-capable loaded, OFF -> "Enable thinking"
- reasoning-capable loaded, ON -> "Disable thinking"
- locked-on model -> "Thinking is required for this model"
* studio/frontend: extract Think pill aria-label helper, fix effort dropdown pre-load mislabel
Address review consensus on #5655:
1. Extract the duplicate 5-branch aria-label conditional into a shared
helper `thinkToggleAriaLabel` (plus a parallel `thinkEffortAriaLabel`
for the reasoning-effort dropdown). Both `thread.tsx` and
`shared-composer.tsx` now import from
`components/assistant-ui/think-aria-label.ts`.
2. While reviewing the diff, an Opus reviewer noticed the same
conceptual bug existed in the reasoning-effort dropdown branch in
`thread.tsx:627` (the alternate render path used by Claude-style
models with effort levels): before a model loaded, the aria-label
announced e.g. "Reasoning effort: medium" on a disabled, grayed-out
button. Same contradiction as the original bug for the on/off
toggle. Now routed through `thinkEffortAriaLabel`, which falls back
to "Thinking (model not loaded)" / "Thinking (not supported by this
model)" while the button is unreachable and only emits the effort
label when the model is loaded and actually supports reasoning.
3. Locked-on stays intentionally absent from `thinkEffortAriaLabel`:
the dropdown remains interactive in that case (users can still pick
an effort level), so the per-level label is the right announcement.
Verified by bun run typecheck (clean) and bun run build (clean). Bundle
confirms all six label strings still ship.
* studio/frontend: route shared composer effort dropdown through thinkEffortAriaLabel
12/12 reviewers flagged that the earlier think-aria-label helper was only wired into thread.tsx; the parallel reasoning-effort dropdown in shared-composer.tsx still hard-coded the raw "Reasoning effort: medium" label, so screen readers heard a stale effort value when the control was disabled (no model loaded, unsupported reasoning).
Route shared-composer's effort button through the same helper, matching thread.tsx.
* studio/frontend: shorten think-aria-label helper comments
---------
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
* studio/frontend: friendlier 404 fallback for unknown routes
TanStack Router defaults to a bare "Not Found" string when no
route matches. With Studio's root layout that string sits alone
in the main content area while the sidebar still renders, which
looks broken when the user hits a typo'd path, a stale share
link, or a chat URL with an extra path segment.
Provide a small DefaultNotFound component to createRouter:
sloth mascot, "Page not found" heading, the offending pathname,
and a Back to chat button. Studio chrome continues to render
around it, so the user gets the same sidebar nav for free.
Resolves#5663.
* studio/frontend: 404 fallback uses useRouterState + URL-encoded sloth path
Address review feedback on #5664:
- Read pathname via useRouterState({ select: s => s.location.pathname })
instead of window.location.pathname. Matches the pattern already used
in __root.tsx, drops the window-typeof guard, and stays consistent with
the router store on subsequent client navigations.
- URL-encode the sloth mascot src so the space-containing path resolves
cleanly without relying on the browser to encode it.
- Add break-all on the pathname paragraph so long offending URLs wrap
instead of pushing the card wider than the viewport.
---------
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
* feat: add custom model v1/model loading
* fix: require base URL for local model catalog loading
* ux/studio-provider-model-loading-controls
* fix: normalize local provider base URLs
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Roland Tannous <115670425+rolandtannous@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* feat: add custom model v1/model loading
* fix: require base URL for local model catalog loading
---------
Co-authored-by: Roland Tannous <115670425+rolandtannous@users.noreply.github.com>
* studio/frontend: show Generation stopped placeholder when cancelled mid-thinking
Closes#5563.
When the user clicks Stop before any visible content has streamed in,
the running indicator disappears but no Parts have rendered yet, leaving
just the AssistantActionBar floating below the user prompt. That looks
broken (and is the exact failure mode behind the 'tools work, but I
don't see anything happening' bucket of reports).
Add a sibling CancelledIndicator next to GeneratingIndicator that fires
when content is empty AND status is incomplete with reason cancelled,
rendering a muted 'Generation stopped.' italic. The terminal-state
label is consistent with tool-fallback's existing 'Cancelled tool'
treatment and with reasoning's 'Thought for N seconds' summary.
* studio/frontend: shorten CancelledIndicator comment
Trim the 3-line explanation to a single line describing what the
placeholder is for.
* studio/frontend: use 'Cancelled.' to match tool-fallback wording
---------
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
* studio/frontend: settings dialog fits viewport at tablet widths
The dialog used a fixed w-[820px] with sm:w-[820px] override, so any
viewport between 640px and 820px (iPad portrait at 768px is the
canonical case) saw the dialog overflow horizontally by 26px on each
side -- the right-edge scroll arrow and the active-tab chevron got
clipped against the viewport.
Replace the hard 820 with min(820px, calc(100vw-2rem)) on both max-w
and w so the dialog caps at the original 820px on desktop and shrinks
to fit (with a 1rem gutter) on narrower screens. max-sm: still drives
the full-bleed h-dvh/w-dvw layout under 640px.
* studio/frontend: keep mobile full-bleed override !important
Bot review: base !max-w-[min(...)] is !important so the regular
max-sm:max-w-none never wins, leaving a 1rem gutter on phones where
the previous code rendered a true full-bleed dialog. Bump the mobile
override to !important too.
---------
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
The composer's mic icon buttons used tooltip="Dictate" /
"Stop dictation" but no aria-label, so screen-reader users heard
only the empty SVG-only button. Every other composer icon button
(Send, Add Attachment, audio buttons, composer pills) carries an
explicit aria-label; the shared-composer.tsx implementation already
does too. Mirror that here for parity.
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
The settings dialog opens via a global Ctrl+, keydown handler in
__root.tsx, not via a <DialogTrigger>. Radix's FocusScope tries to
capture document.activeElement at mount as the focus-restore target,
but settings-dialog.tsx schedules a requestAnimationFrame that focuses
the active tab button right after mount, racing FocusScope's previous-
focus capture. On Escape or close-button click, focus then lands on
<body> instead of the textarea (or button, or wherever the user was).
A Playwright focus-management probe confirmed: open dialog, press Tab
15 times (trap holds), press Escape, document.activeElement === BODY.
This is a WCAG 2.4.3 (Focus Order) violation: keyboard-only users
have to re-Tab from the start of the page after every settings visit.
Fix: capture document.activeElement in the Zustand store at the moment
openDialog() runs, then restore via onCloseAutoFocus on DialogContent.
Use opener.isConnected so a stale node from a re-rendered tree falls
back to Radix's default. closeDialog deliberately does NOT clear the
opener slot - onCloseAutoFocus reads it on the render after open=false,
so clearing in the same set() would null it before restoration.
Probe re-run confirms focus restored to the TEXTAREA opener after
Escape, after close-button click, on both repeats. Tab + Shift+Tab
trap still holds (unchanged Radix behaviour).
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
* studio/frontend: compare composer blocks send when no model picked
Closes the racing-handle half of #5569. In Compare mode (GeneralCompare
shell with model1/model2 props), if the user sends a prompt before
picking models in either pane, the SharedComposer used to fall through
to the per-handle append branch. Both panes then raced
createOpenAIStreamAdapter -> autoLoadSmallestModel, one won, the other
dispatched into an unloaded slot and produced an empty bubble with a
1000000.0 tok/s readout. The per-pane picker state never observed the
global checkpoint change either, so both pickers stayed at
"Select model".
Add a guard before the content build: when handlesRef has model1/model2
keys but both selections are empty, surface a toast asking the user to
pick models first, leave the text in the composer for retry, and never
enter the racing dispatch path. Keeps the per-pane picker state as the
source of truth for which model is on each side.
The unphysical tok/s readout that the same path produced is separately
covered by PR #5570 (display guard).
* studio/frontend: tighten compare-mode guard to require both panes
Review feedback on #5574:
- Gemini: the redundant `model1 !== undefined && model2 !== undefined`
checks let the racing-handle dispatch slip through whenever the
Compare props arrive as undefined, which is the exact case the
guard is trying to block.
- Codex: with `isGeneralizedCompare` keyed on `model1?.id || model2?.id`,
a half-selected Compare (one model picked, one empty) still falls
into the generalized branch. The composer clears, the empty pane
gets the user message appended, and `startRun` only fires for the
side with an id, leaving the empty pane with a dangling prompt
and no response.
Switch `isGeneralizedCompare` to require BOTH panes (`&&`), drop the
undefined gate, and surface the "Pick a model in each pane" toast for
either the fully-empty or half-selected case. `hasCompareHandles` is
true only inside GeneralCompareContent, so LoraCompare and the
single-pane path stay unchanged.
* studio/frontend: shorten compare-mode no-model-guard comment
* studio/frontend: clarify compare-pane toast wording
---------
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
* studio/frontend: include filename in attachment aria-label and img alt
When a chat has multiple attachments of the same kind, the rendered
tiles all share the generic accessible name "Image attachment" or
"Document attachment". Sighted users get the filename from the Radix
tooltip that pops on hover, but:
- screen-reader users hear "Image attachment, Image attachment,
Image attachment" with no way to distinguish three PNGs;
- touch-device users (no hover) lose the filename entirely;
- keyboard-only users would have to focus and read a tooltip that
isn't always announced.
Fold the filename into both the button's aria-label and the thumbnail
<img alt>, falling back to the existing labels when the attachment has
no filename. Sighted UX is unchanged: the Radix tooltip already shows
the same name on hover, and the visible aria-label has no rendered
counterpart.
Found while running a multi-image attach probe in the autonomous Studio
UX loop (cycle 8). Repro:
await page.evaluate(`Array.from(document.querySelectorAll(
'button[aria-label*="attachment" i]'
)).map(b => b.getAttribute('aria-label'))`)
Before: ["Image attachment", "Document attachment", "Add Attachment"]
After: ["Image attachment: test_red_circle.png",
"Document attachment: notes.txt",
"Add Attachment"]
* studio/frontend: shorten attachment a11y comment
---------
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
* studio/frontend: show Loading fallback instead of blank pane on lazy route navigation
Closes#5567.
Train, Recipes and Export pages are imported via React.lazy() in their
respective createRoute calls, and the Suspense boundary around <Outlet />
in __root.tsx passes fallback={null}. The result is a 1-3 second
completely white pane between sidebar click and content paint, which is
the exact failure mode behind reports that those pages look broken or
stuck. /chat does not suffer from this because chat.tsx imports its
ChatPage synchronously.
Replace fallback={null} on both Suspense boundaries (hideNavbar and
sidebar layouts) with a small centered 'Loading...' label using the
same muted-foreground style as elsewhere in the app. Synchronous routes
(/chat) never suspend so they are unaffected; lazy routes now have a
visible terminal-state placeholder while their chunk loads.
* studio/frontend: also apply RouteFallback to the sidebar Suspense
The first revision only replaced the fallback={null} inside the
hideNavbar branch (used for onboarding / login). The primary lazy
boundary that wraps Train / Recipes / Export is inside the SidebarInset
branch at the other Suspense site, which kept rendering null and made
the page look stuck for the same window the original bug describes
(per bot review feedback on #5568).
Replace both Suspense fallbacks with RouteFallback so the "Loading..."
placeholder fires on every lazy route, not just on the auth flows.
* studio/frontend: shorten RouteFallback comment
---------
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
* studio/frontend: keep theme classes mutually exclusive on <html>
The Sonner Toaster reads next-themes (mounted at provider.tsx with
attribute="class" defaultTheme="light"), so on first mount next-themes
adds a "light" class to <html>. Studio's own setTheme path
(features/settings/stores/theme-store.ts) only toggled "dark", so
after the user picked Dark in settings the document ended up with
html.className = "light dark". Harmless in CSS cascade because the
dark variables override, but reads as a UI defect in devtools and trips
CSS-aware tooling that branches on class lists.
Toggle "light" alongside "dark" in applyToDocument so the two classes
stay mutually exclusive regardless of how next-themes seeded the
initial class.
* studio/frontend: shorten theme-toggle comment
---------
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
* studio/web: distinguish "offline" from "studio crashed" in error toast
When the user's browser loses network mid-request, authFetch caught the
fetch TypeError and surfaced "Studio isn't running -- please relaunch it."
That is a correct diagnosis in the Tauri desktop app (the supervisor died
in-process), but it is a misleading diagnosis in the web build where the
backend lives elsewhere: the user will start hunting for a dead process
when the actual problem is connectivity.
Branch on navigator.onLine === false (web build only) and surface
"You appear to be offline. Check your network connection and try again."
instead. Tauri keeps the original wording so it stays accurate there.
Found while running a slow-network UX probe and toggling
Network.emulateNetworkConditions {offline: true} mid-stream.
* studio/frontend: shorten offline-error wording comment
---------
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
* studio/frontend: guard message-timing badge against unphysical tok/s
llama.cpp can report `predicted_ms == 0` and `predicted_n == 0` on turns
that effectively produced no generation (most reliably reproduced today
on a Compare-mode pane that loses the auto-load race and dispatches a
generate against an unloaded slot, see issue #5569). The current display
trusts `predicted_per_second` verbatim, which turns into `Infinity` /
`1000000.0 tok/s` on the action toolbar of an otherwise empty bubble
and reads like a UI defect even when the underlying request did happen.
Require at least one predicted token, at least one millisecond of
generation time, and a finite rate before rendering. Falls back to the
total stream time formatter, which already handles the zero case
gracefully.
* studio/frontend: shorten predictedRate guard comment
* studio/frontend: tighten timing guard threshold and hide Generation row when suppressed
Raise the decode-window floor from 1ms to 10ms so race-lost panes that
emit a stray token in 1-2ms (still giving 1000-5000 tok/s) drop out
alongside the predicted_ms=0 case. Gate the tooltip's Generation row
on the same hasPredicted predicate as Speed so the tooltip never shows
'Generation: 0ms' with no Speed underneath.
* studio/frontend: accept sub-10ms decode windows in timing guard
Cycle-15 codex P2 flagged that the >= 10ms threshold hid legitimate
fast generation (cached single-token, small models). The original
Infinity-blocker was predicted_ms=0, so use >0 instead. predicted_n
>= 1 and Number.isFinite() still keep the no-op race-lost cases out.
---------
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
* studio: respect prefers-reduced-motion across animations
Tailwind animate-in/out, Radix dialog/popover zoom-in/slide-in transforms,
and the infinite shine / shiny-text / icon-pop keyframes all run at their
full duration regardless of the user's OS-level reduced-motion preference.
A Playwright probe that emulated the media query confirmed every measured
transition was identical between no-preference and reduce, so users with
vestibular triggers see the same scaling overlays and continuous shimmers.
Add the canonical universal-selector override so animation-duration,
animation-iteration-count, and transition-duration collapse to ~0ms when
the preference is set, leaving end states intact. Probe re-run shows
settings-dialog animationDuration drop from 0.1s to 1e-05s and the 50ms
mid-open screenshot is byte-identical to the settled one.
* studio: exempt .animate-spin from reduced-motion collapse
The universal-selector rule from the previous commit froze every
animation including .animate-spin, which is used as the canonical
in-progress indicator across Studio: tool execution loaders
(tool-ui-python/terminal/web-search/code-execution/fallback/group),
sonner toast spinners, Tauri startup + update screens, and the
generic <Spinner /> primitive in components/ui/spinner.tsx.
Freezing those leaves reduced-motion users with no visual signal
that work is in flight, which trades one accessibility win for
another. WCAG treats progress indicators as "essential motion"
that should keep moving.
Restore .animate-spin with a 1.5s cadence (instead of the default
1s) so the rotation is still perceptible but less aggressive than
the no-preference path. animation-iteration-count goes back to
`infinite` so the spinner doesn't halt after one rotation.
Verified via a focused probe that injects a .animate-spin element
and a .animate-in fade element side by side:
no-preference spin=1s infinite fade=0.15s
reduce spin=1.5s infinite fade=1e-05s
---------
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
The settings dialog sidebar was fixed at w-[200px], which left only
~92px of horizontal space for tab labels after icon, gap, and the
'New' badge for Connections/API. 'Connections' (11 chars at the
14.5px font weight medium) overflowed and rendered as 'Connectio...',
matching the paper-cut reported in issue #5572.
Bump the sidebar to w-[216px] -- 16 more pixels of label space, fully
within the existing dialog width and unchanged on mobile
(max-sm:w-full still drives the responsive layout).
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
Adds tools, thinking blocks, code execution, and web search support to the safetensors / transformers and MLX inference backends in Studio, bringing them to parity with the GGUF path.
What ships
- safetensors / transformers agentic tool loop with cumulative-text state machine, tool-call XML parser, and template kwarg forwarding (tools / enable_thinking / reasoning_effort / preserve_thinking).
- MLX backend: same kwargs accepted on Apple Silicon; chat_template_info shipped through worker IPC; pills enable for Qwen / Qwen3 / Qwen3.5 / Gemma reasoning.
- Capability classifier (_detect_safetensors_features) gates supports_tools on actual parser-compatible emission markers (<tool_call> / <function=) so Llama-3 / Mistral / Gemma 4 do not advertise toggles the parser cannot honour.
- gpt-oss override stays: reasoning on, tools off (Harmony channel, not <tool_call> XML).
- CWE-209 hygiene: safetensors SSE error path emits a constant message and logs the trace server-side.
Validation
- 256 unit tests green (43 tool-loop, 11 capability advertise, 7 MLX backend, 5 main-added, 190 adjacent inference / anthropic / openai regression).
- Cross-OS staging CI green on ubuntu-latest / macos-14 / windows-latest plus a dedicated MLX cartesian probe against real unsloth/Qwen3.5-0.8B on macos-14 (CI 26098107440).
- Capability parity verified across Qwen3 / Qwen3.5 / Llama-3 / Mistral / Gemma / DeepSeek-R1 / gpt-oss (incl. BF16).
- Manual confirmation from Imagineer99 on Qwen3.5-2B: think + search + code exec working.
Closes the safetensors / MLX gap with the GGUF backend.
* studio: add --spec-draft-n-max toggle for MTP speculative decoding
Surface llama-server's --spec-draft-n-max as a first-class
LoadRequest field so users can tune the MTP draft tree size from
the chat settings panel. Default behaviour is unchanged: when the
caller omits spec_draft_n_max, the existing platform defaults still
apply (6 on GPU, 3 on CPU/Mac).
Why this matters: on context-constrained loads the draft KV cache
competes with the target model's KV cache for VRAM. Lowering
spec_draft_n_max reduces that pressure, lets a larger user context
fit, and recovers throughput; raising it pays off when draft
acceptance is high enough to amortise the extra cache.
Backend
- LoadRequest gains an optional spec_draft_n_max: int (1..16).
- LlamaCppBackend.load_model accepts and persists the override on
self._spec_draft_n_max, used in place of the hardcoded 6/3 in the
MTP emit branch.
- LoadResponse and InferenceStatusResponse echo the active value
(None when the platform default is in effect) so the UI can
hydrate the input on refresh.
- _already_in_target_state and _request_matches_loaded_settings
compare spec_draft_n_max alongside speculative_type so a value
change triggers a reload rather than no-op'ing.
- strip_shadowing_flags now strips inherited --spec-* extras when
either speculative_type or spec_draft_n_max is in fields_set, so
an inherited --spec-draft-n-max cannot last-wins-override a fresh
request's first-class field.
Frontend
- LoadModelRequest, LoadModelResponse, InferenceStatusResponse
TypeScript shapes get spec_draft_n_max.
- chat-runtime-store gains specDraftNMax / loadedSpecDraftNMax and
a setter, hydrated from /v1/status and /v1/load.
- chat-settings-sheet renders a "Draft Tokens" numeric input
directly under the Speculative Decoding switch when that switch
is on. Toggling the switch off clears the override; the Reset
button restores the loaded value.
Tests
- Four new regression tests cover _already_in_target_state with
matching / mismatching / non-MTP / unset spec_draft_n_max.
- Existing test_llama_server_args.py and test_llama_cpp_mtp_detection.py
green: 141 passed locally.
* studio: add --spec-draft-p-min and --spec-draft-p-split to spec strip set
llama.cpp server documents --spec-draft-p-min (default 0.75, min draft
acceptance probability) and --spec-draft-p-split (default 0.10). Both
are first-class spec-decoding knobs that should travel with the rest
of the --spec-* family when an Apply re-sets speculative_type, so an
inherited override doesn't leak across a fresh load.
* studio/tests: skip MTP capability-probe tests on Windows
The four probe_server_capabilities tests use a bash stub written to
tmp_path/llama-server, which Windows' subprocess can't execute
directly (no shebang resolution, .bat / .cmd would be needed). Mark
them skipif sys.platform == 'win32' so the rest of the MTP plumbing
suite stays green on Windows CI. Unix coverage is unchanged.
* studio: lower MTP GPU default --spec-draft-n-max from 6 to 2
Bench on B200 / Qwen3.6-27B-MTP-GGUF UD-Q4_K_XL across five prompt
types (essay, code, story, math, science) with greedy temp=0:
prompt OFF n=1 n=2 n=3 n=6
essay 79.1 93.4 93.8 84.7 64.6
code 79.1 104.4 116.6 113.5 103.0
story 79.1 99.2 105.7 101.8 88.9
math 79.1 100.8 110.8 111.8 98.2
science 79.1 100.1 110.8 110.8 102.9
The previous hardcoded GPU default of 6 was 17% SLOWER than spec-off
on the essay prompt (64.6 vs 79.1 t/s) and 11-50% slower than n=2 on
the rest. n=2 wins on 4/5 prompts with a 1.18x-1.47x speedup vs OFF;
n=3 wins on the math prompt by a hair. n=6 collapses once acceptance
rate drops past n=3 -- wasted draft decode dominates the per-step
budget.
Matches the dataset README ("n_max=2 is the sweet spot for 36 of 42
quants"). Keeps CPU/Mac default at 3, which empirically tracks the
narrower ngram+MTP chained budget on those platforms.
Users who want the old behaviour can pass spec_draft_n_max in
LoadRequest (the toggle this PR also adds) or --spec-draft-n-max via
llama_extra_args.
* studio: skip MTP auto-promote on sub-2B models, backfill chat usage
Two MTP-visibility fixes uncovered while bisecting llama.cpp post-#22673
on Qwen3.6-27B-MTP-GGUF UD-Q4_K_XL on B200.
Size gate. Direct llama-server bench (no Studio measurement loop) at
n_predict=192 across 9 prompts shows MTP regresses vs spec-off on
sub-2B dense models because draft cost exceeds savings:
Qwen3.5-0.8B Q4_K_XL GPU: 452.0 OFF -> 283.4 t/s n=2 (0.63x)
CPU: 84.5 OFF -> 64.9 t/s n=3 (0.77x)
Qwen3.5-4B Q4_K_XL GPU: 241.0 OFF -> 258.2 t/s n=2 (1.07x)
Qwen3.5-9B Q4_K_XL GPU: 201.6 OFF -> 228.9 t/s n=2 (1.14x)
Qwen3.5-27B Q4_K_XL GPU: 78.8 OFF -> 113.6 t/s n=2 (1.44x)
Qwen3.6-27B Q4_K_XL GPU: 78.8 OFF -> 113.6 t/s n=2 (1.44x)
Qwen3.6-35B-A3B Q4 GPU: 192.3 OFF -> 223.2 t/s n=2 (1.16x)
The 2B inflection is sharp. Skip auto-promote to draft-mtp when the
identifier reports <2.0B params; users can still force via --spec-type
or the Speculative Decoding toggle. Mirror the gate in the
reload-skip check so a sub-2B reload-with-default does not bounce a
spec-off backend.
Chat-completions usage. llama-server's final SSE chunk emits both an
OpenAI-style usage block and a custom timings block. timings.predicted_n
is always populated, but usage.completion_tokens is zero on some
server builds. The Studio chat UI computes generation t/s from
meta.usage.completion_tokens / totalStreamTime, so a zero
completion_tokens makes the UI fall back to wall-clock time
(including SSE / proxy / template overhead) which dilutes MTP gains and
makes ON look the same as OFF.
Add _backfill_usage_from_timings: if usage.completion_tokens is missing
or zero AND timings has predicted_n/prompt_n, synthesize a complete
usage dict. Apply at the streaming metadata yield in
generate_chat_completion and at the three accumulator/yield sites in
generate_chat_completion_with_tools so per-iteration counts are not
silently lost across tool calls.
Tests cover both the gate (sub-2B skips, 2B+ promotes) and the
backfill (zero usage filled, real usage preserved, empty timings
passthrough).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: probe + emit legacy ngram-mod flags for pre-rename llama-server
llama.cpp upstream renamed the ngram-mod tuning knobs:
--draft-max -> --spec-ngram-mod-n-max (and --spec-draft-n-max)
--draft-min -> --spec-ngram-mod-n-min (and --spec-draft-n-min)
--spec-ngram-size-n -> --spec-ngram-mod-n-match
The new names are real flags on post-rename builds and stub removal
entries on the same builds (with description "argument has been
removed"). Pre-rename builds only carry the legacy names as real
flags. Studio was emitting the new names unconditionally, so a user
running a pre-rename llama-server (e.g. an older prebuilt or a
hand-installed binary) would see "unknown argument" errors when the
ngram-mod path engages, or silent drop of the ngram knobs.
Extend `probe_server_capabilities` to parse the help text into
per-flag description blocks and tell real flags apart from removal
stubs by the "argument has been removed" marker. Add three new probe
fields: `ngram_mod_flavor` ("new" / "legacy" / None),
`supports_ngram_mod`, and `spec_draft_n_max_flag` (the actual n_max
flag the binary accepts). Cached by (path, mtime) the same way as
`mtp_token`.
Add `_build_ngram_mod_flags(caps, ...)` that picks the right flag
set, returning [] when neither is usable so callers can drop ngram
chaining entirely on minimal binaries.
Wire both call sites to use the probe-driven flag set:
- CPU/Mac MTP comma-chain (--spec-type ngram-mod,draft-mtp) emits
legacy or new knobs as appropriate. If neither set is available,
degrade to MTP-only (warn but still engage spec).
- Standalone --spec-type ngram-mod branch uses the same helper.
Tests cover post-rename detection, legacy detection, removal-stub
discrimination, minimal-binary case, and all three branches of
`_build_ngram_mod_flags` plus custom n_match/n_min/n_max values.
Verified against three real binaries (Studio bundled 726704a, my
build of 45b455e HEAD, and the MTP merge baseline 2555826) all
correctly reporting ngram_mod_flavor=new.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: sub-3B MTP falls back to ngram-mod, not off
Earlier sub-2B gate disabled speculative decoding entirely for tiny
dense MTP models because the MTP draft head's per-token cost exceeds
the acceptance savings at that scale. The "fully off" fallback was
conservative -- ngram-mod has near-zero idle cost on diverse content
and consistently outperforms both off and draft-mtp at sub-3B.
Clean-methodology bench (each of 9 distinct prompts run once after
two unrelated warmup prompts so the ngram-mod hash pool is
realistically populated but never holds the exact deterministic
output we're about to measure):
Q4_K_XL on B200:
0.8B OFF=451 draft-mtp n=2=263 (0.58x) ngram-only=498 (1.10x)
2B OFF=377 draft-mtp n=2=308 (0.82x) ngram-only=369 (1.00x)
4B OFF=240 draft-mtp n=2=260 (1.08x) -- 4B+ wins with MTP
Q4_K_XL on x86 48 cores:
0.8B OFF= 80 chained n=2= 69 (0.86x) ngram-only= 95 (1.19x)
2B OFF= 62 chained n=2= 51 (0.83x) ngram-only= 63 (1.01x)
4B OFF= 31 chained n=2= 41 (1.33x)
Change:
- Raise the MTP-skip threshold from 2.0B to 3.0B (2B falls below it).
- When skipping the MTP head, fall back to --spec-type ngram-mod via
the probe-driven _build_ngram_mod_flags helper. Works on both
post-rename and pre-rename llama-server builds.
- If the binary advertises neither ngram-mod flavor, fall back to
spec-off (older binaries that don't support ngram-mod at all).
- Mirror the same fallback in _already_in_target_state so a sub-3B
reload-with-default does not bounce a ngram-mod backend.
Tests updated: monkeypatch probe_server_capabilities so the gate
behavior is deterministic regardless of which llama-server happens
to be on the host. +1 new test for the "binary has no ngram-mod
support" branch; renamed prior 2B/0.8B tests to reflect new semantics.
This generalizes the size gate to be probe-driven instead of a hard
"disable spec" branch.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: 5-mode Speculative Decoding dropdown (Auto / MTP / Ngram / MTP+Ngram / Off)
Replace the Chat Settings Speculative Decoding on/off Switch with a 5-option
Select. Auto preserves today's platform-aware resolver (MTP on MTP GGUFs,
ngram-mod fallback for sub-3B, --spec-default for non-MTP). The other 3 modes
force the user's choice on BOTH GPU and CPU: MTP emits draft-mtp only (no
ngram chain on CPU), Ngram emits ngram-mod only, MTP+Ngram emits the
ngram-mod,draft-mtp chain on both platforms. Off is the existing fully-off
state, kept so the Switch's "disable" capability isn't lost.
Backend
- New module-level _canonicalize_spec_mode(value) maps any accepted input
(canonical, legacy "default" / "draft-mtp" / "ngram-mod" / "ngram-simple",
or comma-chained "ngram-mod,draft-mtp") onto one of auto / mtp / ngram /
mtp+ngram / off / ngram-simple / None. Lets external callers and old
persisted UI state round-trip without breaking.
- LlamaCppBackend grows a _requested_spec_mode field + requested_spec_mode
property storing the canonical UI mode the user requested. Status
responses round-trip this instead of the resolved internal flag, so the
dropdown restores the picked value after reload / refresh (Auto on a 27B
MTP GGUF resolves to draft-mtp internally but the dropdown stays on
"Auto").
- The resolver block in load_model is extracted into a unit-testable
_build_speculative_flags method. Forced MTP / MTP+Ngram on a sub-3B or
non-MTP GGUF logs a warning and engages anyway (user override > the
Auto-path sub-3B fallback).
- _already_in_target_state and routes/inference._request_matches_loaded_settings
now compare canonical-requested mode, dropping the old auto-promotion
mirror. spec_draft_n_max still gates on the resolved spec so Auto + a
changed n_max still bounces a reload.
Frontend
- chat-settings-sheet.tsx: Switch swapped for Select modeled on the KV
Cache Dtype Select. Items: Auto / MTP / Ngram / MTP+Ngram / Off. Draft
Tokens input only visible when speculativeType is "mtp" or "mtp+ngram".
- chat-runtime-store.ts: initial value flips from "default" to "auto".
- use-chat-model-runtime.ts normalizeSpeculativeType mirrors the backend
canonicaliser so persisted "default" / "draft-mtp" / "ngram-mod" / chain
values hydrate to the right dropdown option.
- types/api.ts: docs the canonical wire vocabulary.
Tests
- 53 new assertions in test_llama_cpp_mtp_detection.py: full
_canonicalize_spec_mode table, a 23-row resolver matrix across
(requested mode) x (GPU/CPU) x (model size class), plus n_max override,
user-extra-args precedence, requested-mode round-trip, and graceful
degrade on an outdated llama-server without an MTP token.
- 165 existing backend tests still green. 218 total in the MTP /
server-args / reload-inheritance suite.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: reset Speculative Decoding to Auto on model switch
When the user switches from model A to a different model B, clear the
runtime store's speculativeType + specDraftNMax (and their loaded*
shadows). The new load request then carries null, the backend
canonicalises that to "auto", and its platform-aware resolver runs
fresh for the new model.
Without this, a non-MTP model loaded with "Off" carried the Off choice
into a subsequent MTP load, suppressing MTP auto-promotion (and the
sub-3B ngram-mod fallback) until the user manually opened settings and
flipped the dropdown back to Auto. The clean-sweep deep probe caught
it as anomaly A-1.
The reset only fires when currentCheckpoint != modelId, so a
same-model reapply or forceReload still honours the user's current
spec choice. End-to-end probe on Qwen3.5-4B-GGUF (non-MTP, Off) ->
Qwen3.5-0.8B-MTP confirms: dropdown shows Auto, /api/inference/status
returns speculative_type=auto, studio.log shows the Auto sub-3B
fallback emitted --spec-type ngram-mod.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* studio/frontend: cap auto-load cascade attempts
autoLoadSmallestModel walks every cached GGUF and safetensors repo with a
try/catch + continue, so a folder of broken caches (missing files, stale
llama.cpp prebuilt, GPU OOM) can fire dozens of failing POST /api/inference/load
calls in a row. Each call costs ~5 seconds (HF metadata probe + DNS guard
inside inference.py), so the user sees a runaway sequence of request_completed
log lines after sending one message that needed an auto-load.
Cap the total loadModel calls inside autoLoadSmallestModel at 3 (GGUF cascade
plus safetensors fallback share the same counter). Caching that fails three
times in a row is almost certainly an environment problem, not "we haven't
found the working one yet"; the default-Gemma download path still runs.
No behavior change on the happy path: success returns after the first hit
exactly like today, and the trust-remote-code skip path does not consume an
attempt slot.
* shorter comment on auto-load cap
* studio chat: extend autoload cap to default Gemma fallback
Cached cascade respected MAX_AUTO_LOAD_ATTEMPTS but the default-Gemma
download path skipped the budget, so a broken cache could still emit a
fourth /api/inference/load. Gate the fallback on the same cap (and bump
loadAttempts when we do call loadModel) so the total cross-path budget
is 3, matching the cap's intent.
* studio/frontend: reconcile stale must_change_password localStorage flag
The client OR's a localStorage flag against /api/auth/status everywhere it
gates change-password routing, but never clears the flag when the server
flips requires_password_change back to false. A user whose default admin
password was already rotated (change-password from another browser, the
CLI reset-password command, or a recreated auth DB) keeps that flag, so:
1. requirePasswordChangeFlow lets them sit on the change-password route.
2. Back to login bounces via requireGuest, hasActiveSession (which only
checks key presence, not validity), then getPostAuthRoute, which sends
the user back because the flag is still set.
End result: the user is pinned on change-password and cannot escape without
clearing localStorage by hand.
Fix the three places that compare server status to the flag:
- auth-guards.ts fetchAuthStatus: clear the local flag whenever the server
reports requires_password_change = false.
- auth-guards.ts requireGuest: call fetchAuthStatus before routing so a stale
flag cannot decide getPostAuthRoute.
- auth-form.tsx initializeAuthForm: same reconcile inside the page so the
change-password page redirects to login as soon as it loads when the
server no longer requires a change.
- api.ts redirectToAuth: same reconcile in the fetch wrapper's auth redirect.
After the reconcile the redundant mustChangePassword() OR clauses are no
longer load bearing for the change-password gates; the server's
fetchAuthStatus is now the single source of truth.
* shorter comments around auth-status reconcile
* studio/auth: make localStorage reconcile bidirectional
Two related issues on the chat toasts:
1. Close X did nothing. The lib/toast.ts wrapper defaulted every toast
to `dismissible: false` (originally to keep swipe capture from
stealing text selection). In sonner v2, `dismissible: false` makes
the close-button onClick a no-op, so the X looked clickable but
never dismissed the toast. The Toaster already sets
`swipeDirections={[]}` in components/ui/sonner.tsx, so the
per-toast swipe workaround is unnecessary and harmful. Replace the
wrapper with a thin re-export of sonner.
2. Close X hover collapsed to a near-black circle in light mode.
Sonner's default close-button styling uses fixed gray-scale tokens
(--gray2 hover, --gray12 text) that ignore the theme attribute.
Once the Toaster's inline style overrides --normal-bg with
var(--popover), the base background follows the app theme but the
hover state does not, so the hover bg lands on a color that has no
contrast with the X glyph. Pin both base and hover to theme tokens
(--popover, --muted, --popover-foreground, --border) so contrast
stays visible in both light and dark modes.
Repro: open chat, load any cached model, hover the X on the
"<name> loaded" toast in light mode -- before this change the circle
turned dark and the click did nothing; after, the circle stays light
and the click dismisses the toast.
* studio: engage draft-mtp on vision MTP GGUFs
The draft-mtp auto-promotion in LlamaCppBackend.load_model was gated on
not effective_is_vision, and the spec-emit branch repeated the same
guard. Every Unsloth -MTP GGUF repo ships an mmproj projector, so
effective_is_vision was always True for those repos and the MTP speedup
silently never engaged out of the box.
llama.cpp #22673 explicitly states MTP is compatible with vision input.
The bundled b9204 server happily loads both: a manual run with
--mmproj ... --spec-type draft-mtp --spec-draft-n-max 6 logs
"loaded multimodal model" followed by
"adding speculative implementation 'draft-mtp'".
Drop the vision gate from both sites and rewrite the matching short
circuit in _already_in_target_state so reload checks reach the auto
promotion path on vision MTP loads. Add three regression tests covering
vision MTP match (auto and default), and non MTP vision repo unaffected.
Verified on a B200 with unsloth/Qwen3.6-35B-A3B-MTP-GGUF:UD-Q4_K_XL:
base decode 179.7 t/s vs MTP decode 253.8 t/s, draft acceptance 0.57,
1.41x speedup on a 255 token completion. mmproj still loads and image
input remains available.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: prefer Qwen3.5 -MTP GGUF variants in default model lists
With the vision gate dropped in the previous commit, draft-mtp now
auto-engages on -MTP GGUF repos out of the box. Swap the four Qwen3.5
recommended entries in DEFAULT_MODELS_GGUF and DEFAULT_MODELS_STANDARD
to their -MTP-GGUF counterparts so new users get the speedup by default:
unsloth/Qwen3.5-4B-GGUF -> unsloth/Qwen3.5-4B-MTP-GGUF
unsloth/Qwen3.5-9B-GGUF -> unsloth/Qwen3.5-9B-MTP-GGUF
unsloth/Qwen3.5-35B-A3B-GGUF -> unsloth/Qwen3.5-35B-A3B-MTP-GGUF
unsloth/Qwen3.5-0.8B-GGUF -> unsloth/Qwen3.5-0.8B-MTP-GGUF
All four HF repos exist (HEAD 200) and ship the same UD-Q4_K_XL quant
layout as the non-MTP variants. Non-Qwen3.5 entries are untouched.
* bump version to 2026.5.4
Picks up the studio MTP vision-gate fix and the Qwen3.5 -MTP default
swap in this PR.
* studio: prefer Qwen3.6-35B-A3B-MTP-GGUF in default model lists
Same rationale as the previous Qwen3.5 swap. The Qwen3.6 MTP variant
exists at unsloth/Qwen3.6-35B-A3B-MTP-GGUF (HF HEAD 200) and now
auto-engages draft-mtp out of the box with the gate fix.
* studio: drop --spec-draft-n-max from 6 to 3 for draft-mtp
n=6 is too greedy: on Qwen3.6 the draft has to guess 6 tokens ahead
and acceptance crashes to ~0.45, leaving only ~14% throughput gain.
PR ggml-org/llama.cpp#22673's author benched n=3 at ~0.72 acceptance
and 2 to 3x speedup on the same Qwen3.6 family, and the README sample
command uses n=2 or n=3. Match that.
CPU/Mac branch already uses n=3, so this aligns both paths.
* studio: set --spec-draft-n-max back to 6 for draft-mtp on GPU
Reverts the n=3 tuning. n=6 is the original default; user-side comparisons
hold the larger draft window steady so the toggle (next commit) is the
primary on/off lever.
* studio: add Speculative Decoding toggle under Max Tokens
Adds a top-level kill switch (panel-switch under Max Tokens, mirroring
Auto-Healing Tool Calls) that forces the /load request's
speculative_type to "off" when disabled. The backend "off" branch in
LlamaCppBackend.load_model skips both the draft-mtp auto-promotion and
the spec-emit branch, so neither --spec-type draft-mtp nor
--spec-default reaches llama-server.
Wiring:
- chat-runtime-store: new speculativeDecodingEnabled bool, default
true, persisted to localStorage under unsloth_speculative_decoding,
plus a setSpeculativeDecodingEnabled setter.
- chat-settings-sheet: SpeculativeDecodingToggle rendered immediately
beneath the Max Tokens slider for non-external models.
- use-chat-model-runtime: when speculativeDecodingEnabled is false,
override speculative_type to "off" in the loadModel call so the
switch wins over any pre-existing speculativeType state (including
the existing per-model toggle in Model Settings).
Verified end to end on unsloth/Qwen3.6-35B-A3B-MTP-GGUF:UD-Q4_K_XL:
toggle ON emits --spec-type draft-mtp --spec-draft-n-max 6; toggle
OFF emits zero --spec-* flags on the same MTP GGUF.
* studio: relocate Speculative Decoding toggle into Model Settings
Move the toggle out from under Max Tokens and back into the Model
Settings section, directly beneath KV Cache Dtype, where the existing
Apply/Reset workflow already drives a reload on dirty. This way flipping
the switch in the UI actually picks up: the section becomes dirty,
Apply re-runs /load with the new speculative_type.
Drop the !currentModelIsMultimodal gate so vision MTP GGUFs can also
disable speculative decoding from the UI.
Switch the toggle's off-value from null to "off" so the backend's "off"
short-circuit fires for MTP models too (null normalises to None which
re-triggers the draft-mtp auto-promotion).
Tooltip now reads "Faster generation with 0% accuracy hit".
Remove the now-redundant speculativeDecodingEnabled bool + setter from
the runtime store and the load-time override in use-chat-model-runtime;
the toggle binds directly to speculativeType.
* studio: restore OOM/TIGHT badge on recommended GGUF rows
The recommended-list row passed vramStatus=null for any GGUF repo
because the existing useRecommendedModelVram hook reads safetensors
totals from HF model info, which GGUF-only repos do not expose. As a
result, an OOM Q-quant repo would render with only a "GGUF" badge and
no visual signal that nothing in it fits.
Add useGgufRecommendedFit: per repo, fetch the variant list via the
existing /api/models/gguf-variants endpoint, take the smallest
variant's size_bytes, and classify with the same 0.7*GPU + 0.7*RAM
thresholds as GgufVariantExpander. Session-scoped cache + in-flight
dedup so a repo is requested at most once.
Wire the result into the three GGUF row sites in pickers.tsx so OOM
and TIGHT badges show on the collapsed cards.
* Revert "studio: restore OOM/TIGHT badge on recommended GGUF rows"
This reverts commit 07793b1240df72b13e51d6dc15f63c4ee8c6cba9.
The new useGgufRecommendedFit hook was treating the symptom. PR #5561
identified the real root cause: useGpuInfo was calling /api/system
with plain fetch instead of authFetch, so the session-auth check
failed silently and gpu.available stayed false everywhere. With no
GPU info, every fit check (variant expander, recommended carousel)
fell back to "no signal" and dropped the OOM/TIGHT badges.
Reverting the over-engineered hook and applying the authFetch fix
in the next commit, which restores the existing badges with one line.
* chore: replace qwen suggested with MTP variant
* fix: restore GPU info auth for GGUF fit badges
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: imagineer99 <samleejackson0@gmail.com>
* studio/chat: release stuck IME flag when compositionend never fires
Chrome on Windows talking to a WSL-hosted Studio (issue #5546) fires
compositionstart + compositionupdate but no compositionend after the
IME commits. The earlier hardening in #5327 cleared the stale flag on
the next non-composing input event, which never arrives in this
sequence, so composingRef stays true forever and the Send button stays
disabled even though the committed CJK text is already in the textarea.
Add a watchdog in both useImeComposerInputHandlers (main + edit
composer) and SharedComposer (compare mode) that runs the same reset
the missing compositionend would have done. The timer is rearmed on
every compositionupdate and on every non-composing input so it only
fires when the IME pipeline has actually gone quiet — normal candidate
selection keeps it alive, the WSL stuck case lets it expire.
Extends the existing IME Playwright smoke with a stuck-compositionend
repro and adds a static guard so the watchdog can't be removed without
the regression tests catching it.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio/chat: re-pin composing flag on IME keydown to close#5546 watchdog gap
The stuck-compositionend watchdog (PR #5551) releases composingRef after
2500 ms of IME silence so Send unwedges in the WSL+Chrome case. The same
release also fires during a long candidate-window pause in healthy IMEs,
which lets a subsequent IME-confirm Enter slip preedit text through
handleSubmit (main composer) or click-Send through send() (compare composer).
Add a keydown gate to both composers: when the browser still reports
nativeEvent.isComposing or keyCode 229, re-pin composingRef and cancel
any pending watchdog so the next form-submit / send() guard refuses.
The Send button stays visually enabled (avoids re-introducing the
stuck-UI bug) but the submit path is blocked until a real compositionend
or non-composing input arrives. Mirrors the existing isComposing guard
shape in shared-composer.onKeyDown.
Tests:
- tests/studio/test_composer_rtl_bidi_attribute.py: two new static
guards asserting the keydown gate wiring in both composer files.
- tests/studio/playwright_chat_ime_i18n.py: new section 6c repro that
fires the IME-confirm keydown after the watchdog has cleared, then
triggers form.requestSubmit() and asserts the preedit text is not
cleared (would indicate a leaked submit).
Verified across Chromium / Firefox / WebKit via a side-by-side pre-PR
vs post-PR simulation (54 scenarios, zero pageerror or console.error).
The #5546 stuck-end repro still passes (Send re-enables 2.5-3 s after
the silent commit) and the new keydown-repin probe confirms the submit
gate refuses on all three engines.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio/chat: re-arm IME watchdog after keydown re-pin (Codex P1)
The keydown re-pin added in 2c3c9793 closed the watchdog-race for
healthy IMEs, but on the same WSL+Chrome no-compositionend path this
PR targets it would re-lock Send permanently: setting composingRef=true
and only *clearing* the watchdog leaves the flag pinned forever if no
follow-up compositionend or non-composing input ever arrives.
Swap clearStuckTimer/clearStuckImeTimer for refreshStuckTimer/
refreshStuckImeTimer in both composer keydown gates so the watchdog
fires once more after every IME keypress. Same visual contract — Send
stays enabled — the submit gate just keeps a 2.5s window before
re-releasing instead of staying locked.
Extends the playwright IME smoke with section 6d: clears composing via
the watchdog, fires an IME keydown, then waits past the re-armed
watchdog window and asserts the form submit actually flushes the
textarea. Two new static guards in test_composer_rtl_bidi_attribute
lock the refresh call into both keydown handlers.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* studio/frontend: hide Current password input on first boot
PR #5490 added a third Current password input to the change-password form
so the admin-forced must_change_password reset path could supply a current
password (the bootstrap is empty in that path). The side effect is that the
dominant first-boot UX, which has window.__UNSLOTH_BOOTSTRAP__ present and
silently fed into currentPassword, now shows three visible inputs instead
of the two it had before.
Render the Current password input only when window.__UNSLOTH_BOOTSTRAP__
is absent. The loadBootstrap effect already seeds the password state from
the bootstrap and currentPassword keeps the bootstrap fallback, so
handleSubmit sees the same value as before. On admin-forced resets where
the bootstrap is undefined, the Current password input still appears so
the user can type their actual current password.
Verified end-to-end against a local install via UNSLOTH_STUDIO_HOME +
install.sh --local with Playwright driving the page: bootstrap present
renders two inputs (New, Confirm) and completes change-password into
/chat; bootstrap suppressed via a non-configurable property descriptor
init script renders the three inputs (Current, New, Confirm) and keeps
the #5490 fix intact.
* studio/frontend: add deterministic input-count tests for auth-form
Pure-source pytest covering the change-password JSX contract. No
browser, no Studio boot, no JS toolchain -- runs on any CI runner.
Complements the Playwright probe in tests/studio/playwright_chat_ui.py
which exercises the same contract end to end.
Pins seven invariants with explicit failure reasons:
1. hasBootstrapPassword is derived from window.__UNSLOTH_BOOTSTRAP__
so a future swap to a localStorage flag or prop cannot silently
drift from the backend's _inject_bootstrap contract in
studio/backend/main.py.
2. Exactly one !hasBootstrapPassword conditional exists; multiple
would split rendering into branches these tests cannot reason
about.
3. The Current password input sits inside that conditional, so it
never renders on first boot (the regression PR #5490 introduced
and that this fix reverses).
4. The New password input sits outside it, so it always renders in
change-password mode (admin-forced reset still works).
5. Confirm password: same as New.
6. The change-password JSX subtree declares exactly current /
new / confirm; a fourth password input would almost certainly
break the 2-input first-boot contract.
7. The login JSX subtree declares exactly one password input.
Verified the tests fail loudly on the pre-fix auth-form.tsx at
c4575ca0 (5/7 fail with descriptive reasons) and pass on the fixed
version (7/7).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* studio/frontend: soften toast shadow and tighten vertical padding
Sonner's defaults felt heavy in the chat header surface: a 16px
all-around padding made the box taller than the two-line content
warranted, and the 4/12/0.10 drop shadow read as a hard slab
against the light background. Trim padding to 10px vertical
(horizontal unchanged at 16px) and dial the shadow back to
0 2px 6px / 0.08 so the toast still lifts off the surface without
casting a heavy halo.
* studio/frontend: annotate why toast override needs !important
Sonner injects its base styles at runtime from inside its JS bundle,
so a plain cascade tie can lose depending on injection order. One
short comment above the override saves the next reader the dig.
* studio/frontend: boost toast shadow opacity in dark mode
Sonner's lighter 0.08 shadow disappears on the dark popover surface:
quantitative measurement of the shadow band (10px below the toast)
across Chromium / Firefox / WebKit showed only a ~3% luminance drop
vs background, well below perceptual threshold. Bump the dark-mode
opacity to 0.3, matching the existing .shadow-border light/dark ratio
(0.1 -> 0.3) and bringing the toast in line with .menu-soft-surface's
dark-mode shadow (0.28). Light mode keeps the original 0.08.
* studio: add dismissable toasts with corner close button
- Enable Sonner's close button globally on the Toaster, so every toast
(model load progress, model loaded, load failure, etc.) gets an X that
users can click to dismiss without waiting for the auto-dismiss timer.
This matches the Claude desktop notification behavior.
- Drop the per-toast 'closeButton: false' overrides in the model load
runtime so they inherit the global default. The existing 'onDismiss'
handler already flips state to show an inline header status, so the
X on the loading toast hides the toast without canceling the load
(Cancel still aborts).
- Pin the close button to the top-right corner inside the toast box.
Overrides Sonner's left-side default placement, outside-corner
translate, and hardcoded 'top: 0'. Top is set via a small rule in
index.css because Sonner does not expose it as a CSS variable.
- Add a small offset on the Toaster so toasts sit at the chat header
line, shifted left of the parameters and settings buttons on the
right edge instead of stacking on top of them.
- Bump the post-load success and failure durations from 2s and 5s to
8s so users actually have time to read and click the new close X
before the toast auto-dismisses.
* studio: explicit boolean for closeButton prop to satisfy biome
* studio: keep close button X visible in dark mode
Two defensive fixes for the dark-mode close button visibility:
- Use resolvedTheme so sonner's data-sonner-theme always matches the
class next-themes applies to <html>. Passing theme can be 'system',
which makes sonner resolve via its own media query; that can disagree
with next-themes (Tauri webview, hydration races, OS quirks), leaving
CSS vars dark while sonner still applies its light close-button colors
(dark X on dark background).
- Bump the close-icon stroke from sonner's default 1.5 to 2.25 so the X
is readable on a 12x12 svg sitting on dark backgrounds.
---------
Co-authored-by: shimmyshimmer <datta_mike@hotmail.com>
* studio/frontend: make toast and inline error text selectable and copyable
Sonner toasts and the inline model-load error in the chat header were
showing copyable content (backend tracebacks, model-load failures, log
lines) that users could not actually select with the mouse.
Two underlying issues:
1. Sonner's swipe-to-dismiss handler calls `setPointerCapture` in
`onPointerDown`, which preempts the browser's text-selection
gesture. The capture only happens when `dismissible` is true. CSS
alone cannot work around this.
2. The inline model-load error truncated with `text-overflow: ellipsis`
and parked the full string in a native `title=` tooltip, which
browsers render as an OS tooltip that cannot be selected.
Fixes:
- New `@/lib/toast` wrapper that defaults `dismissible: false` on every
toast (callable plus `.success` / `.error` / `.info` / `.warning` /
`.loading` / `.message` / `.custom`). API is identical to sonner's
`toast`, so the 18 call sites just swap their import path. Callers
can opt back into swipe-to-dismiss with `dismissible: true`.
- `<Toaster>` sets `swipeDirections={[]}` to make the intent explicit.
- `index.css` forces `user-select: text` on toast text content and
keeps `user-select: none` on toast buttons.
- New `<CopyableErrorChip>` component replaces the truncated inline
error in the chat header. The chip shows the truncated message
inline and opens a popover with the full, wrap-friendly, selectable
message and a one-click Copy button.
Toasts still auto-dismiss after their `duration`, close buttons and
action buttons still work.
* studio/frontend: tighten code comments in selectable-toast change
* studio/frontend: address PR review on selectable-toast change
Three review-driven fixes:
1. CopyableErrorChip clears the copied->reset setTimeout on unmount via
a useRef + useEffect cleanup so setState cannot fire on an unmounted
component.
2. index.css restricts `cursor: text` to text-bearing toast nodes
(`[data-title]`, `[data-description]`, `p`, `span`). The toast
container keeps its default cursor and no longer pretends to be an
editable surface. `user-select: text` still applies to the full toast
tree so a drag-select starting on padding still works.
3. Toast wrapper now also injects `dismissible: false` into the second
argument of `toast.promise(p, data?)`, covering the loading /
success / error toasts created from a single promise call. Explicit
`dismissible: true` in the data continues to win.
A fourth review point asked us to drop the wrapper and instead pass
`toastOptions={{ dismissible: false }}` to <Toaster>. Sonner v2.0.7's
Toaster only forwards `duration`, `className`, `descriptionClassName`,
`closeButton`, `style`, `unstyled`, `classNames`, `cancelButtonStyle`,
`actionButtonStyle`, and `closeButtonAriaLabel` from `toastOptions`
(see index.mjs lines 1144-1164). `dismissible` is not forwarded, so the
global-option approach is a runtime no-op (verified empirically across
Chromium / Firefox / WebKit). Wrapper is required.
* studio/frontend: drop chip aria-label override so message reads via SR
The CopyableErrorChip trigger set a fixed `aria-label`, which overrides
the visible message in the accessibility tree. Inside the chat header's
`role="status"` region this caused screen readers to announce the
generic label instead of the actual model-load error, a regression
versus the old plain-text status div.
Removed the `ariaLabel` prop and the default override. The button's
visible message text is now its accessible name, so the full
(untruncated) error is announced. Truncation stays purely visual via
CSS. Caller in chat-page.tsx dropped the prop too.
Added a Playwright assertion that the trigger's accessible name
contains the error message across Chromium, Firefox, and WebKit.
---------
Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local>
* studio/frontend: grow chat composer to 16 rows and inset scrollbar
Raise the composer textarea cap from 6 to 16 rows so the input keeps
expanding as you type longer prompts. Also nudge the textarea in with
mt-2 / mr-3 so the internal scrollbar no longer sits flush against
the rounded edges of the chat composer surface.
* studio/frontend: lower composer cap from 16 to 12 rows
Keeps the composer growing past the previous 6-row cap while staying
conservative enough that a fully expanded textarea does not cover the
scroll-to-bottom button or a large slice of recent messages.
* studio/frontend: use symmetric mx-3 inset on composer-input
Replaces mr-3 with mx-3 (and width calc(100%-1.5rem)) so the textarea
sits inset from both edges of the chat composer surface. Keeps the
scrollbar tucked in regardless of writing direction: LTR scrolls on
the right, RTL scrolls on the left, and both edges are now ~16px in
from the surface (4px surface px-1 + 12px mx-3).
The spinner inside ThreadWelcome subscribed to the global
generatingStatus from chat-runtime-store, so any in-flight warmup
(or stale leak from a prior run) surfaced Generating on the empty
Chat with your model surface, even while the user was still typing.
On the normal path the welcome view is gone the moment a message
is submitted, and the assistant bubble already renders its own
per-message GeneratingIndicator. Remove the welcome-screen spinner,
its component, and the now-unused LoaderIcon import.
* studio: register /settings route that opens the settings dialog
Navigating to /settings used to render Not Found because the route
was never registered. The settings dialog only opened via the user
menu, so /settings was a broken deep link if shared. Add a route
that calls useSettingsDialogStore.openDialog() and redirects to the
post-auth landing page so the modal appears on top of the chat.
* studio: harden Connections dialog provider sync and allow manual model IDs
Two related fixes for the Connections panel.
1. Keep localStorage providers when the server returns an empty list.
The dialog used to sync from /api/providers/ on mount and unconditionally
overwrite the Zustand provider store with the server result. When the
server had no enabled configs but the local store had entries (legacy
users, fresh dev installs, or providers created via earlier paths),
opening the dialog silently wiped them. The model picker reads from the
same store, so the chat header reverted from 'gpt-4o . OpenAI' to the
raw 'external::openai-1::gpt-4o' key. Treat the server as authoritative
only when it actually has rows; otherwise keep the local view.
2. Accept manual model IDs alongside the live catalog for remote-mode
providers (DeepSeek, OpenAI, etc.). Previously the only way to save was
to load the available-models catalog via a live API call, which fails
in air-gapped setups, behind 502s, or when the user already knows the
exact model ID. Add a Textarea fallback in the same render block, and
relax the validation to accept manual IDs even when availableModels is
empty. The validation message now points users at the manual path.
* studio: restrict manual model ID entry to openrouter among remote providers
Address review feedback: major remote providers (openai, anthropic,
gemini, mistral, cohere, deepseek, ...) expose large per-model
parameter surfaces that differ across models, so accepting pasted
model IDs leads to mismatched parameter expectations and frustrating
runtime errors. Keep their catalog curated by hiding the manual
textarea and falling back to the prior 'Load available models first'
validation toast for them.
OpenRouter drops unsupported parameters server-side, so manual entry
remains useful there; keep the textarea and the union save path for
it. Custom and curated backends already gated via isCustomProvider /
isCuratedModelList and continue to require manual entry as before.
* studio: shorten code comments in chat-providers-dialog.tsx
Trim three multi-line comment blocks to single lines per review.
Daniel Han
