* Unsloth: appearance palettes, customization options, and control restyle
Adds Standard, Classic, and Minimal color palettes to Appearance settings,
each adapting to light and dark mode. Classic is a neutral enterprise look
that reserves its blue accent for toggles, badges, and focus rings; Minimal
is strictly black, grey, and white.
Adds customization options scoped to the active mode: accent, background,
and foreground colors with an in-app color picker, UI and code fonts with a
searchable dropdown covering bundled, device, and imported fonts, font file
import, UI and code font sizes, contrast, pointer cursors, reduce motion,
font smoothing, and translucent sidebar. Settings persist through the
personalization API with backend validation and sync across devices.
Restyles core controls for a cleaner, flatter look in both modes: bordered
white input fields, fully rounded pills for single-row controls, no drop
shadows, simple straight-line chevrons replacing all rounded arrow icons,
and consistent hover tones in dropdown menus. Popovers now portal into the
open dialog so their lists scroll correctly inside modal dialogs.
Moves Language into General settings and Chat defaults into the Chat tab
above the Canvas section.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Unsloth: appearance follow-ups, font options, and settings search
Neutralizes focus and selection rings across all palettes so highlighted
elements, including typing boxes and the selected palette card, never take
the accent color. The custom accent no longer recolors rings.
Restyles the color controls as filled pills showing the hex value inside,
with text and border contrast picked from the color's luminance. Menus in
popovers now match the app's dropdown menus: rounded-lg corners, tighter
padding, accent hover rows, and a bordered search field. Popovers inside
modal dialogs are modal so their lists scroll with the wheel. Outline
buttons share the same dark fills as dropdown triggers.
Adds heading and chat font options next to the UI and code fonts, each
using the searchable font dropdown and persisting through the
personalization API. Removes the translucent sidebar option end to end.
Adds settings search: a search field at the top of the settings sidebar
that filters setting names across every tab, grouped by tab with icons,
and jumps to the tab on click.
* Unsloth: use the shared accent token for dark hover fills
The settings dialog nav, its close button, the model selector, and the
project switcher hovered with hardcoded blue tinted greys (#3a3d43,
#2d2e32) in dark mode while every menu and sidebar uses --accent. All
hover and active pill fills now use the accent token so dark hovers are
the same everywhere and adapt to the active palette.
* Unsloth: settings search polish and jump to matched setting
Widens the settings dialog to 880px and the sidebar column to 248px so
the search field has more room. The search pill aligns with the left
start of the Settings title, gets more spacing above and below, and its
icon and placeholder sit slightly further left.
Search results now jump to the exact setting: rows and sections expose
their label as a data attribute, and picking a result opens the tab,
scrolls the matched row into view, and flashes it briefly.
* Unsloth: settings search bar spans the full nav pill width
The search field now starts and ends at the same edges as the nav hover
pills instead of being inset to the title text.
* Unsloth: address review findings on motion, sync, and font limits
Reduce motion Off now opts back out of the OS reduced-motion preference
for CSS animations via a force-motion class that the media rules skip,
and forcing reduce motion On keeps the loader exceptions (spinners,
loading dots, progress bars) animating.
When the color scheme follows the system, the resolved mode is now part
of the theme store snapshot, so an OS scheme flip re-renders consumers
and reapplies per-mode custom colors instead of leaving stale inline
variables from the previous mode.
Imported fonts get an aggregate size cap (4.4M characters) on both the
frontend sanitizer and the backend model so the persisted store always
fits browser localStorage quotas, with a clear error toast when an
import would exceed it. Backend validation also tightens imported font
names (rejects CSS delimiter characters) and requires strict base64
font data URLs, matching the frontend patterns.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Unsloth: profile toggle to hide the sloth in the chat greeting
Adds a Show greeting sloth switch to Settings > Profile. The chat welcome
hides the mascot when it is off. The preference persists locally and
through the personalization API, with backend validation and tests, and
the row is reachable from settings search in all four locales.
* Unsloth: control restyle, dropdown scrolling, and palette consistency
Settings sidebar puts search on top with the tab list under a small
Settings label. Combobox popups scroll with the wheel inside dialogs by
falling back to manual list scrolling while a dialog scroll lock is
active, and the local model selector popover became modal for the same
reason. Number inputs swap native spinners for a shared grey stepper
that clamps to min, max, and step. Run settings fields in light mode use
the same white fill and border as the settings dialog. Selection and
focus rings derive from each palette's border color instead of near
black, hover borders soften the same way, the Classic sidebar stays
white like Standard, decorative greens follow the palette accent, and
meaning-carrying marks like the hub verified badge keep the brand green
in every palette.
* Unsloth: palette card selection keyed off the palette attribute
Switching palettes restyles the whole page the moment data-palette lands
on the html element, but the React re-render that moves the selection
classes arrives later, so the ring and check briefly stayed on the
previous card with the new palette's colors. The active ring and check
now key off html[data-palette] in CSS, so they swap in the same style
pass that swaps the tokens. Also adds breathing room around the settings
search bar and under the Settings label, shortens the greeting sloth
description, and renames the avatar section to Or pick a sloth profile
picture in all locales.
* Unsloth: restore neutral rings, drop the palette check, sidebar spacing
Puts the ring tokens back to their fixed per palette values and removes
the hover border darkening, undoing the derived border experiment. The
selected palette card no longer shows a check since the ring already
marks it. The settings sidebar search bar, nav pills, and search results
get a little side padding, and the Settings label lines up with the pill
text.
* Unsloth: indicator restyle, sidebar menu customization, edge fade toggle
- Derive focus and selection rings from the border color so indicators
stay 1px and adapt to every theme and palette
- Suppress mouse focus rings except on pressed controls to remove the
selection flash on the avatar and palette pickers
- Defer settings panel rendering so the active nav pill updates instantly
- Customizable sidebar user menu with drag to reorder and shortcuts to
the settings tabs
- Grey hover for the standard light palette instead of green
- Borderless controls in dark mode with fill based focus states
- Profile picture: no picture option, pencil edit icon, atomic selection
- Font dropdowns: narrower triggers and the resolved default shown as
Inter Variable (Default)
- System prompt border darkens on focus
- New appearance setting to swap edge fades for thin divider lines
- Move the theme bootstrap to an external script to satisfy CSP
* Unsloth: harden theme boot and Firefox scroll container focus
- Guard the theme and palette storage reads separately so a blocked
localStorage (private browsing) still resolves a mode from the OS
preference instead of skipping the boot entirely
- Firefox makes scrollable containers keyboard focusable and drew its
3px UA outline on them; swap it for the app's soft 1px indicator
* Unsloth: make the UI and code font settings reach the font utilities
The theme block declared the sans and mono stacks as literals, so
Tailwind inlined them into every font-sans and font-mono utility at
build time and the runtime overrides from Settings > Appearance never
applied. Reference the :root tokens instead, matching how the color
tokens already work.
* Unsloth: in-dropdown font upload, accent meters and avatar, naming cleanup
- Move font importing into each font dropdown: Upload and Select folder
sit side by side under the list, imported fonts get an inline remove,
and the standalone Import font row is gone
- Uploads reuse fonts the user already has (bundled, imported, or
installed, matched by file name with style suffixes stripped) instead
of embedding a duplicate copy; only new fonts are embedded
- Folder scan lists font files from a picked folder in every dropdown
for the session; picking one imports it through the same path
- Fallback avatar uses the control accent with a readable foreground
instead of the neutral primary that rendered black outside standard
- Monitor bars, progress defaults, sliders, and usage meters use the
control accent; warning and danger tiers stay amber and red
- User facing strings that called the app just Studio now say Unsloth
in all four locales, keeping Unsloth Studio and LM Studio intact
* Unsloth: left align the font upload actions and divide them
Upload and Select folder now read from the left like the list items,
with a short vertical rule between the two.
* Unsloth: keep sliders neutral and the chat greeting on Hellix
- Sliders are controls, not meters, so their fill goes back to the
neutral primary instead of the palette accent
- The base h1 rule reads --font-heading with !important and the chat
thread root resets that variable to the sans stack, which pulled the
greeting off Hellix; restore the stack on the greeting element
* Unsloth: move the None avatar cell last and keep footer actions on one line
- None sits after the sloth pictures instead of leading the grid
- Upload shrinks to its label so Select folder no longer wraps
* Unsloth: size the folder action to its label
Both footer actions now hug their content so the hover pill does not
stretch across the leftover row width.
* Unsloth: separators only between unrelated settings clusters
Rows inside a titled section are related, so the per row divide-y is
gone from SettingsSection. A SettingsGroupDivider marks the two real
boundaries in the theme section (colors to fonts, fonts to contrast)
and the Clear all chats row gets its destructive border back now that
divide-y no longer draws one for it.
* Unsloth: balance the two font upload actions
Both actions share the footer row evenly again; nowrap keeps Select
folder on one line at the narrower width.
* Unsloth: drop the theme section dividers and split the chat menu groups
The colors, fonts, and contrast rows read fine without rules, and the
chat menu gains its one real boundary between the pin toggles and the
disclaimer rows.
* Unsloth: normalize oversized sidebar menus and reject newline font data URLs
Two backend validation fixes in PersonalizationCustomization:
- sidebarMenu refused any list longer than the number of distinct ids
because Field(max_length) is enforced before the dedupe validator runs.
A stale or duplicated payload that would normalize to one entry per id
was rejected outright, defeating the normalizer that exists for exactly
that case. Cap the incoming list at a generous multiple so it reaches
the validator; a pathologically long list is still refused.
- The imported font dataUrl validator used re.match on a pattern ending
in $, which also matches just before a trailing newline, so
"data:font/woff2;base64,AAAA\n" passed even though the frontend JS
pattern rejects it. Use re.fullmatch for parity.
Adds covering tests for both.
* Unsloth: preview fonts in their own typeface and slim the color pills
- Every font dropdown entry, the default item, and the closed trigger
render in the font they name, falling back to the UI stack for
families the browser cannot resolve
- Color swatch pills drop from 36px to 28px so they sit closer to the
row label height
* Unsloth: drop the font row and theme section descriptions
The labels carry the meaning on their own; the mode switching note in
particular read long and confusing.
* Unsloth: let the chat greeting follow the heading font setting
The greeting stays on Hellix by default but adopts a chosen heading
font through a --custom-heading-font variable the applier sets only
while an override exists, so the thread root's sans reset for chat
prose no longer hides the user's pick from the greeting.
* Unsloth: divide the theme section clusters and align the color pill height
Separators return between colors and fonts and between fonts and
contrast, and the color pills share the 32px height of the font
dropdown triggers.
* Unsloth: color pills at half the dropdown width
Fixed w-24 against the w-48 font triggers, with tighter padding so the
hex value still fits.
* Studio: update dep-removal test after next-themes was replaced
The frontend no longer declares next-themes or imports it in src (it was
replaced by the custom theme store and boot script), so the checker now
reports its removal as a safe no-op. The C1 and C8 fixtures in
test_frontend_dep_removal.py still asserted next-themes was a used
dependency, which fails the studio frontend CI dependency-removal safety
check. Update C1 to expect a no-op PASS and drop next-themes from the C8
expected failures so the suite matches the checker's correct output.
* Studio: remove unused ageLabel and exportCollectionJsonl helpers
* Studio: fix blocked-storage theme desync, search jump race, font validation
- theme-store.ts: keep an in-memory currentTheme/currentPalette so a selected
value survives when localStorage is blocked (private browsing). The snapshots
previously re-read empty storage and reverted React state to the default while
the DOM already changed. The matchMedia handler no longer re-reads storage, so
it cannot clobber the in-memory choice; cross-tab storage events still adopt.
- settings-dialog.tsx: the search jump waited a single fixed 60ms for the
deferred tab panel to render, then silently missed under render lag. Retry
across animation frames until the target row exists, then scroll and flash.
- settings.py: apply the font-name character check to the four selected-font
fields (uiFont/headingFont/chatFont/codeFont), and forbid backslash, comma,
slash and control characters so a name cannot escape the quoted CSS
font-family or smuggle extra fallbacks. Adds covering tests.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix appearance customization edge cases for PR #7077
- Reset all local preferences now also clears palette and appearance customization
- Number input wrapper keeps full width so fields fill their flex/grid cell, and the stepper stays pinned to the field edge
- Number stepper snaps to the min anchored step grid like the native spinner instead of leaving a step-invalid value
- Code font now applies to chat code fences and inline code via a dedicated token
- Reduce motion (on/off) is honored by onboarding/tour confetti and the theme toggle view transition
- Re-importing a font under the same name with new bytes now swaps the FontFace
- Keep local customization when a synced record predates the customization field, and re-push it
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Align client font name sanitization with server validation for PR #7077
sanitizeFont now strips the same characters the backend _FONT_NAME_FORBIDDEN
rejects (backslash, slash, comma, backtick) plus control chars, so a locally
chosen font name can no longer pass the client but fail the personalization PUT
and silently stall appearance sync.
* Address follow-up review items for PR #7077
- Number input wrapper carries React Flow interaction classes (nodrag/nopan/nowheel) so clicking the stepper arrows increments instead of dragging the node
- Preserve local palette and greeting-sloth toggle when the synced record predates those fields, and re-push them, mirroring the customization handling (new paletteSaved and greetingSlothSaved response flags)
- Add settings-search scroll targets (data-settings-label) for the Profile title, description, display name, nickname, and avatar shape rows
* Preserve absent personalization fields on PUT for PR #7077
A stale client that omits palette or customization previously had those
defaults materialized by model_dump() and persisted, which flipped
paletteSaved/customizationSaved to true and defeated the legacy detection.
The PUT now dumps only the request's set fields and merges them onto the
stored record, so omitted fields keep whatever was already stored.
* Persist theme and palette via a fixed allow-list for PR #7077
The theme/palette values reach setTheme/setPalette from the authenticated
personalization sync, which made the CodeQL clear-text-storage query treat
writing them to localStorage as storing sensitive data. Store a re-derived
literal from a constant map instead, so a plain UI preference is not tracked
as sensitive; behavior is unchanged.
* Harden imported-font handling for PR #7077
- syncImportedFonts: a rejected FontFace.load() only clears the registry entry
if it still points at that face, so a same-name re-import while the old load
was pending is no longer untracked/leaked.
- Cap imported-font names to the backend length (100) so an over-long name can
no longer pass the client but fail the personalization PUT and stall sync.
- Add a backend test that a stale PUT preserves an existing stored palette and
customization (not just that absent fields stay absent).
* Return the merged personalization record from PUT
The PUT /personalization handler returned the request payload, which
Pydantic had already filled with defaults for any field the client
omitted. A partial or stale write (for example a client sending only
theme) therefore got back a response that contradicted both storage and
the next GET: preserved fields like palette and the custom font showed
their defaults instead of the stored values.
Return model_validate(merged) so the response mirrors what was stored.
The stored record is still the full merged dict, so legacy fields the
model does not know about are preserved as before.
* Fix small UI and keyboard-focus defects in appearance settings
- Settings search now scrolls to the result within its destination tab
instead of a same-named row in the previously rendered deferred tab
(for example "Storage" and "Models folder" appear in both General and
Resources).
- The reduce-motion segmented control honors its own Off/On/System choice
by reading useReducedMotionConfig instead of the OS-only useReducedMotion.
- The color picker saturation/value area is operable by keyboard, so the
role="slider" surface responds to the arrow keys it advertises.
- Profile avatars and palette cards show a visible keyboard focus ring
again.
- Guard the persisted appearance-customization write so a blocked or full
localStorage does not throw out of a store action, matching the theme
store.
- Import the appearance store symbols from the settings feature barrel.
* Tighten appearance fix comments
---------
Co-authored-by: shimmyshimmer <shimmyshimmer@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Studio: persistent stdio MCP sessions so server state survives across tool calls
call_tool_sync spawned a fresh stdio subprocess per tool call
(keep_alive=False) and tore it down when the call returned, so any stateful
MCP server lost its state between calls: with @playwright/mcp,
browser_navigate opened the page in one subprocess and
browser_take_screenshot ran in a brand-new one, screenshotting about:blank.
Keep one connected client per (command, env) on a dedicated event-loop
thread and reuse it across calls:
- idle sessions are reaped after 5 minutes (in-flight calls excluded) and
everything closes at exit, preserving the old design's no-orphans property
- a dead subprocess is detected via is_connected() and retried once on a
fresh session; tool-level errors leave the session alone
- cancel and timeout semantics are unchanged, and a timed-out call does not
tear the session down
- updating a server's endpoint/env/enabled state or deleting it closes its
live session
- HTTP/SSE servers stay one-shot per call
* address review feedback
* fix stdio session cleanup
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* address review: per-thread MCP scope, close-during-connect and abort races
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* address review: unblock no-limit calls on close, drain borrowers before close, scope closes to url+env
* don't retry sessions closed by config changes, re-verify server row before caching, keep env secrets out of generation keys
* fail fast on connect errors and make the stdio key-lock wait cancellable
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* quote MCP scope parts so IDs with colons can't collide
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* serialize per-session stdio calls, span one timeout budget across connect and call, hash urls in generation keys
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Harden persistent stdio MCP sessions: crash recovery, concurrency, scoping
- Evict a stdio session on any transport-level (non-ToolError) call failure and
do not replay it, so a mid-call subprocess crash can no longer poison the scope.
Never gate liveness on Client.is_connected() (it only reports that a session
object exists, not that the subprocess is alive); add a version-adaptive
dead-transport probe that works on fastmcp 3.0.2 and newer.
- Re-check closed/defunct/config and transport liveness after acquiring the call
lock, and retire a session before releasing the lock, so a queued same-scope
caller never reuses a session that another caller's timeout already retired.
- Force a ProactorEventLoop on Windows so the stdio transport can always spawn
subprocesses regardless of the active event-loop policy.
- Scope stdio sessions per conversation: require thread_id to persist, and tag
the fields so a session_id and a thread_id with the same value cannot collide.
A session_id alone is project-wide, so it now falls back to a safe one-shot
session instead of sharing browser/DB/REPL state across conversations.
- Forward thread_id on the Anthropic Messages path.
- Treat timeout=None as unlimited on connect and the key lock (was capped at 60s).
- Bound the session cache (default 32, override via
UNSLOTH_STUDIO_MAX_STDIO_MCP_SESSIONS) with LRU eviction of idle sessions.
- Run config_check on cache hits, and log a redacted exe#digest label instead of
the raw command so credentials in argv never reach the logs.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Trim the stdio MCP session cache on release and skip close-generation for HTTP servers
Two fixes from review of the persistent stdio session lifecycle:
- Re-enforce the session cap when a session goes idle. A concurrent burst of
distinct-scope calls can overshoot the cap while every cached session is busy
(insert-time eviction only reclaims idle sessions), and the overshoot used to
persist until the 5-minute idle reaper. _release_stdio_session now trims the
idle overshoot back within the cap, without ever evicting an in-flight call.
- close_stdio_sessions() now no-ops for a specific non-stdio (HTTP/SSE) url.
Those transports are never cached as stdio sessions, so calling it on every
HTTP server update or delete used to accrue an unbounded close-generation entry.
Both are covered by regression tests that fail before the change and pass after.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Keep the live stdio MCP session across a display-name rename
The edit dialog resends url, headers, and use_oauth unchanged whenever a
server is saved, so gating the tool-cache invalidation and stdio session
close on field presence dropped the persistent process on a plain rename
or any no-op edit. Gate on a real value change against the stored row so
only a genuine endpoint, auth, or enable change closes the session.
Regression tests: a rename that resends unchanged url/headers/oauth keeps
the session; a real command change still closes it.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten comments in the stdio MCP session lifecycle
Collapse a few verbose comments to fewer lines with the wording preserved,
and drop one that restated the clear_oauth_tokens_async docstring. Comments
only; no code change.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
* Studio: add 7 display languages, complete and fix existing locales
Adds fully translated French, German, Spanish, Hindi, Arabic, Russian and
Korean locales. Fills in all missing keys for zh-CN (113), ja (71) and
pt-BR (47), fixes translation errors found in review, and reorders the
language dropdown by popularity. All overlays pass check-parity with zero
missing keys and zero placeholder mismatches.
* Studio: default display language to auto detect
The language preference now defaults to auto and resolves against the
browser language list, with exact tag match first and language subtag
match second (pt-PT resolves to pt-BR, zh-TW to zh-CN). Auto detect is
the first dropdown option and is translated in every locale. Explicit
choices still persist and sync; personalization sync now round trips
the preference instead of the resolved locale so auto stays auto across
devices. Auto mode also follows browser languagechange events.
* Studio: guard import.meta.env in translate for non-Vite contexts
translate() read import.meta.env.DEV directly, which throws when the
module runs outside Vite (SSR or Node tooling). Optional-chain it so the
dev-only warning is skipped and translation still works everywhere.
* Studio: RTL for Arabic, translate recipes, keep Traditional Chinese off zh-CN
- Sync document dir from a per-locale dir field so Arabic mirrors the
layout instead of rendering RTL text in an LTR shell.
- Translate the recipes nav label in fr, de, ko and hi to match the
other locales (Recettes, Rezepte, and native forms).
- Detection no longer maps Traditional Chinese (zh-Hant / zh-TW / zh-HK /
zh-MO) to Simplified zh-CN; those tags fall through to the next
preferred language. Simplified tags (zh, zh-CN, zh-SG, zh-Hans) still
resolve to zh-CN.
* Studio: don't treat legacy synced English as an explicit language pick
The old sync serialized the resolved locale on every save, so existing
profiles carry appearance.language 'en' even when the user never chose a
language. Hydrating that as a pinned locale forced non-English browsers
back to English under the new Auto detect default. Payloads now carry
version 2 (the preference itself); on hydrate a version 1 'en' maps to
auto, while explicit picks and all version 2 values are kept as-is.
* Studio: persist only known language codes from the locale table
normalizePreference now returns a value re-derived from the LOCALES keys
instead of the raw input. It stays functionally identical (the stored
value was already whitelisted) but makes it explicit that only known,
non-sensitive language codes are written to localStorage, and clears a
false-positive clear-text-storage scan on the persistence path.
* Studio i18n: fix Train label transliteration and tidy locale consistency
- ja and hi: the nav and route Train label used the railway transliteration
(トレイン and ट्रेन); switch to the training term already used everywhere
else in each file (トレーニング, ट्रेनिंग).
- zh-CN: keep VRAM in English to match every other locale and the PR's own
keep-English rule, and drop an extra clause added to the upload size hint
so it matches the English source.
- hi: translate Recents to हाल के in the export and import section to match
the sidebar label, and point users to the Configure tab by its translated
name (कॉन्फ़िगर).
- ru: reword the preview sharing hint to avoid the "disable to disable"
repetition.
i18n parity and the type checked build stay green.
* Studio i18n: keep Arabic layout LTR until physical-direction CSS is converted
Setting ar to dir rtl only mirrors the flex based shell, sidebar and
settings dialog. The shared select, dialog and dropdown primitives use
physical-direction utilities (right-2, top-5 right-5, ml-auto) that do not
flip under dir rtl, so chevrons, close buttons and check marks land on the
wrong side. Keep Arabic on an LTR layout for now, matching the original plan
in this PR. Arabic text still renders right to left per element via bidi and
chat content keeps dir auto, so nothing regresses. Full layout mirroring can
follow once the physical-direction classes are converted to logical ones.
* Studio i18n: do not let a generic zh after a Traditional tag pick Simplified
navigator.languages can be a list like ['zh-TW', 'zh', 'en-US']. The zh-TW
pass already falls through, but the bare zh then reached the language-subtag
match and selected zh-CN, so Traditional Chinese users still got Simplified
and the guard was defeated. detectLocale now remembers when a Traditional
tag was seen and skips a later bare zh, so detection keeps falling through to
the next non-Chinese language. A lone bare zh, and explicit zh-CN or zh-Hans
fallbacks, still resolve to Simplified as before.
* Studio i18n: collapse two locale comments to a single line
The Arabic dir note in messages.ts and the bare-zh note in
locale-store.ts were two lines each; tighten each to one. Comment
only, no behavior change.
* Studio i18n: translate Hindi strings that were left in English
Seventeen hi.ts labels stayed in English while all the other locales
translated them: the training parameter labels (Grad Accum, Grad Norm,
Grad Checkpoint, Eval Loss, Clip p95/p99, Seed, Continued Pretraining),
the API example labels (curl/Python/JavaScript + tools/advanced),
Hugging Face token, the VRAM estimate and the training terminal start
line. Parity only checks key/placeholder presence so it did not catch
these. Brand and technical tokens (curl, Python, VRAM, Loss, p95/p99,
Hugging Face, unsloth) stay in English as elsewhere.
---------
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
* MCP image handling
* clean upg
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: return MCP error results so image content is not dropped
FastMCP client.call_tool raises ToolError by default on an is_error
result, so it never reaches _flatten_result and any returned image is
dropped. Pass raise_on_error=False so error results flow through
_flatten_result and keep their images. Transport failures still raise
and hit the existing handler. Add a regression test for the real path.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: accept raise_on_error kwarg in MCP test fake clients
The call_tool_sync fix passes raise_on_error=False to client.call_tool.
Update the fake MCP clients patched into mcp_client._client so their
call_tool signatures accept the keyword, keeping the stdio/servers MCP
test suites green.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: tighten MCP raise_on_error rationale comments
* Studio: only strip MCP image sentinel when suffix is a valid image envelope
* Studio: validate MCP image envelope in chat adapter and keep base64 out of exports
* Studio: sanitize MCP images in all export formats and fall through to sandbox parser on invalid marker
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
* Studio: remove AGENTS.md from install artifacts
* Studio: prune CLAUDE.md from install artifacts
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix Studio instruction cleanup edge cases
* Trim Studio cleanup comments
* Make Studio cleanup safe on PowerShell 5.1
* Fix Studio cleanup ownership boundaries
* Simplify Windows link detection
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Stabilize floating monitor drag
* Restore floating monitor exit animation
* Harden Windows Studio smoke checks
* Keep API menu badge removed
* Apply no-build-tools env overrides in-script
The runner does not apply step-level env keys containing parentheses,
so ProgramFiles(x86) kept its real value and Find-VsBuildTools still
detected VS through vswhere. Set the overrides inside each pwsh step
instead; child processes inherit them. The resolver step moves to pwsh
because bash cannot export a variable named ProgramFiles(x86).
* Reset chat UI session without a second browser context
macOS runs Chromium with --single-process, where closing the last
context tears down the whole browser, so the shutdown re-login died
with TargetClosedError on new_page. Clear cookies and swap pages
inside the same context instead, opening the replacement page before
closing the old one.
* Keep the no-build-tools Path filtered across session refreshes
install.ps1's Refresh-SessionPath and setup.ps1's Refresh-Environment
rebuild the session Path from the Machine and User registry scopes, so
the process-level filter could be undone mid-install and re-expose
CMake. Filter those scopes in the Prepare step with normalized dir
matching and restore them in cleanup.
* Drop stale localStorage auth tokens before re-login
Auth tokens live in localStorage, not cookies, and the login guest
guard redirects on their mere presence. Remove them during the session
reset so the /login navigation is deterministic instead of relying on
the tolerated redirect bounce.
* unstructured block removal
* Enhance unstructured block handling
* Restrict block cleanup to upload UIDs
* cleanup for seed block uploads
* upload cleanup queue for unstructured blocks in recipe studio
* Fix unstructured upload cleanup edge cases
* Fix unstructured upload import ownership
* Fix-unstructured-import-path-ownership
* Guard failed-delete restore against stale block in unstructured drop zone
* Drain queued upload cleanups when autosave is skipped
---------
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: imagineer99 <samleejackson0@gmail.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* feat: detect installed coding agent CLIs in Studio settings
The API-keys panel only ever showed the "claude" flavor of the
`unsloth start` command, so anyone using Codex, OpenCode, OpenClaw,
Hermes, or Pi had to manually rewrite the copied command by hand.
Add a backend check that looks for each agent's CLI binary on PATH
(shutil.which, mirroring the pattern already used elsewhere in
studio/backend/utils) and expose it as GET /api/settings/coding-agents.
The API-keys panel now renders a picker for all six supported agents,
marks the ones it finds installed, and defaults to one of those instead
of always falling back to claude.
Includes unit tests for the detection helper.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* address review feedback on coding-agent detection
Three fixes from PR review:
- detect_installed_coding_agents now treats a PATH lookup failure as
"not installed" instead of letting it bubble up and break the
settings endpoint; added a regression test for it.
- CodingAgentsResponse.agents is now typed as an immutable tuple
instead of a list built from one, matching CODING_AGENTS itself.
- Fixed a race in the API-keys panel: picking an agent while the
installed-CLI check is still in flight could get silently overwritten
once that check resolved. A ref now tracks whether the user has made
a manual choice, so the auto-detected default only applies before
that happens.
* Address Codex feedback: GGUF gating and remote-detection scope
- codex refuses to launch against a non-GGUF (transformers-backed) model
(unsloth_cli's _require_gguf_for_codex), so auto-defaulting to it produced
a copy-pasteable command that fails immediately whenever the loaded model
isn't GGUF. Add useActiveModelIsGguf() (looks up the active checkpoint in
the chat runtime store) and a correction effect that steers the auto-pick
away from codex unless the loaded model qualifies, without ever touching a
choice the user made by hand.
- Detection runs via shutil.which on the Studio backend host, which isn't
the same machine as the browser in a tunnel/remote session. Reword the
'installed'/'detected' copy to say so explicitly when the tunnel URL is
in use, instead of implying the check ran on the viewer's own device.
* Rework auto-default per review: loopback gating + inline GGUF check
Replaces the previous approach with the exact shape discussed on the PR:
- Export isLoopbackHost/normalizeHost from agent-command.ts. The detection
endpoint runs shutil.which on the Studio backend, which only describes the
browser's own machine when the base this panel targets resolves to
loopback. For a LAN or tunnel/remote base, gate the whole thing off --
don't mark anything as "detected" and don't let it drive the default --
instead of just relabeling the copy.
- Drop the separate GGUF-correction effect and useActiveModelIsGguf hook.
Read useChatRuntimeStore.getState().activeGgufVariant inline inside the
existing detection effect's .then() (so it doesn't need to sit in the
effect's deps), and pick the first detected agent that isn't codex unless
the loaded model is GGUF, leaving the existing default untouched when no
compatible agent is detected.
Verified both branches (loopback vs LAN/tunnel base, gguf vs non-gguf,
manual pick preserved, no-compatible-agent fallback) with a standalone
port of the .then() logic.
* Address latest Codex findings: stale detection, model swap, cache
- Clear detectedAgents (and skip the network call entirely) when the panel
leaves a loopback base, instead of leaving a previous loopback detection
result marked 'installed' for a command that now targets a LAN/tunnel/
remote host.
- Add a separate, network-free correction effect keyed on the live
activeGgufVariant: if codex was auto-picked while a GGUF model was loaded
and the user then switches to a transformers-backed model while this panel
stays mounted, steer away from codex instead of leaving a command that
unsloth_cli's _require_gguf_for_codex will now reject. Never touches a
manual pick.
- Drop coding-agents.ts's module-lifetime cache. Installed-CLI detection is
environment state, not a persisted setting, so a stale positive/negative
from before the user installed something (or reopened the tab) is worse
than one extra cheap local API call per mount; keep only the in-flight
de-dupe for concurrent callers.
Verified the correction-effect logic (gguf->non-gguf swap with/without a
fallback, still-gguf no-op, manual pick never overridden) with a standalone
port of the effect.
* Make the codex/GGUF auto-pick symmetric in both directions
The correction effect only steered away from codex when the model stopped
being GGUF; it never steered back toward codex if the model became GGUF
*after* a non-GGUF-gated fallback had already picked something else (e.g.
codex is the only detected CLI, a transformers model is loaded so the
selection correctly falls back to the claude default, then the user loads a
GGUF model while the panel stays mounted -- codex never gets reconsidered).
Consolidate into one effect that re-derives the preferred detected agent
from scratch whenever detectedAgents or activeGgufVariant changes, in either
direction, instead of only reacting to the codex-specific downgrade case.
The fetch effect now only populates detectedAgents/availableAgents; this
effect is the single source of truth for what gets auto-picked from that
list. Never overrides a manual choice.
Verified both transition directions plus the manual-pick-survives and
initial-detection cases with a standalone port of the derivation logic.
* Reset the auto-pick to the default when it stops being trustworthy
Two more real gaps from the latest Codex pass on d988f52:
- The unified derivation effect only handled the case where a *different*
detected agent could take over. If codex was the only detected agent and
auto-picked while a GGUF model was loaded, then the model stopped being
GGUF, 'preferred' came back undefined and the effect silently left the
selection on codex -- exactly the command unsloth_cli's
_require_gguf_for_codex now rejects. Fall back to DEFAULT_AGENT in that
case instead of leaving it untouched.
- Leaving a loopback base cleared detectedAgents (so the 'installed' badges
correctly disappear) but left whatever agent had been auto-picked from
that now-stale, server-side-only detection still selected. Reset to
DEFAULT_AGENT there too, unless the user picked by hand.
Introduces a shared DEFAULT_AGENT constant instead of repeating the "claude"
literal at each reset site. Verified all five cases (both new resets, both
manual-pick-survives variants, and the existing multi-detected-agent
fallback still preferring another compatible agent over resetting) with a
standalone port of the effects.
* Derive GGUF-ness from the actual loaded state, not just the variant string
activeGgufVariant only covers an HF-repo GGUF pick (a specific quant
variant string). A direct local .gguf file -- custom folder, LM
Studio, or drag-drop -- is just as much a GGUF the codex preflight
(unsloth_cli's _require_gguf_for_codex) would accept, but it never has
a "variant" to report, so it read as non-GGUF here even though
/api/inference/status correctly reports is_gguf: true for it. That
mismatch could leave a Codex-only install not auto-selected, or reset
an auto-picked Codex, for a model that actually supports it.
Combined activeGgufVariant with activeNativePathToken (covers the
drag-drop/picked-file case) and ggufContextLength (only ever populated
when the backend last reported is_gguf: true for the active model, see
applyActiveModelStatusToStore) so all three paths a model can be GGUF
through are covered, matching the same is_gguf-or-equivalent check
hasGgufSource already applies to a staged pick elsewhere in this
codebase.
* Clear stale native-path token on a non-GGUF status refresh
When a native (drag-dropped or picked) GGUF was loaded and the backend later
switches to a transformers model outside the UI load path, refresh() adopts the
new /api/inference/status via setCheckpoint and applyActiveModelStatusToStore.
Those reset activeGgufVariant and ggufContextLength but never clear
activeNativePathToken, so the isGguf OR stays true after the switch and a
Codex-only detection auto-selects unsloth start codex for a non-GGUF model its
preflight rejects.
Drop activeNativePathToken in applyActiveModelStatusToStore whenever the status
is non-GGUF. A real GGUF load reports is_gguf: true, so its token is preserved
(the load path owns it); only a non-GGUF status clears it.
* Add the AGPL-3.0 header to the new studio contract test
* Fix/adjust agent detection for PR #6909
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
Co-authored-by: wasimysaid <112766706+wasimysaid@users.noreply.github.com>
* Studio: fix link, currency and indentation edge cases in LaTeX rendering
Follow-up to #6914. Three fixes to studio/frontend/src/lib/latex.ts:
- Skip reference-link definition URLs ([id]: url) during delimiter
conversion, so escaped parens in such URLs are not rewritten as math.
- Preserve the opener line's indentation when emitting a display $$ block,
so a \[...\] inside a list item stays part of the list.
- Stop a currency amount from pairing with a converted span's opening $,
which swallowed the price into math (for example $5 + x \(y\)).
* Exclude GFM footnote definitions from the reference-URL skip
A footnote definition like [^1]: \(x\) had its body treated as a link
destination, so leading math was left literal. Skip [^...] labels.
* Merge overlapping link destination regions
A reference-def token can nest inline-link spans (for example
[1]: http://h/[a](b)/foo\(x\)), so the combined spans could overlap and
isInRegion's binary search missed the outer one, rewriting the URL. Merge
overlapping spans before the search.
* Guard lineStart when the display opener is at index 0
Behavior is unchanged (lastIndexOf clamps a negative fromIndex to 0), but
the explicit guard avoids relying on that implicit clamp.
* Scope to indentation and currency fixes
Drop the reference-link URL protection added earlier. It guards a case
models effectively never emit (escaped parens in a reference-style URL),
and approximating CommonMark reference definitions with a regex needs
open-ended special-casing. Keep the two high-value fixes: preserve display
math indentation (including multi-line bodies) inside a list item, and stop
a currency amount from pairing with a converted span's opening dollar sign.
* Studio: show Hugging Face address on hover for Hub and online model rows
The model selector already shows an on-disk path tooltip on local rows,
but Hub and online rows showed only the bare repo id, and nothing at all
when there was no VRAM estimate. Add an optional hubUrl prop and a
hubRepoUrl helper that mirrors localPathTooltip, and surface
huggingface.co/<repo_id> on hover for the Discover, search, and
downloaded Hub rows. Local and VRAM tooltips are unchanged; the VRAM
tooltip now also appends the address line.
Closes#6382
* Studio: use a 700ms hover delay before the model-row tooltip
Give the model-row hover tooltip (the Hugging Face address, plus the VRAM
and local-path lines it shares) a 700ms open delay instead of showing it
instantly, so it does not flash while sweeping the mouse down the list.
* Fix/adjust GGUF tooltips for PR #6928
---------
Co-authored-by: wasimysaid <112766706+wasimysaid@users.noreply.github.com>
Co-authored-by: Wasim Yousef Said <wasimysdev@gmail.com>
* Move New badge to System settings tab
Show the "New" badge on the System tab and drop it from Connections.
* Stabilize refresh revocation UI test
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Polish assistant message actions menu
Use the circle question mark (HelpCircleIcon) for the "See response
details" action instead of the file-database icon, and lowercase the
"Export as markdown" label.
* Align response details sheet icon
* Speed up Studio startup path
* Studio: recheck managed binary executability on preflight cache hit and ignore stale unauthenticated platform fetches
Preflight: a matching capability cache fingerprint no longer skips the
runnability check when the managed binary's executable bit was cleared
(size and mtime unchanged, since chmod bumps ctime not mtime). The cache
fast path now confirms the binary is still executable, otherwise it falls
back to the CLI help probe so preflight reports Stale and can repair,
instead of returning Ready and failing later at backend start. Adds a
regression test.
Frontend: now that first render is no longer gated on fetchDeviceType,
the initial unauthenticated health call can resolve after an
authenticated platform fetch. Guard the store so a late unauthenticated
or failed non-forced response cannot overwrite an already authoritative
device type, tunnel URL, or secure flag. Forced refreshes and the first
unauthenticated load are unaffected.
* Studio: use access(X_OK) for the preflight cache executability guard
A mode bitmask treats any execute bit as launchable, but the executable
bits can be set only for another owner or group, or be denied by an ACL,
so the current user could still hit PermissionDenied at launch and the
cached fast path would wrongly return Ready. access(X_OK) checks real
executability for the calling user, so an ownership or permission change
correctly falls back to the CLI help probe and the Stale repair path.
* Studio: ignore any stale non-forced platform fetch once authoritative
Extend the platform store guard so a non-forced health response never
overwrites an already authoritative result, not only unauthenticated
ones. With a saved token the post-render non-forced request can be
authenticated but older than a later forced refresh that already picked
up the tunnel URL and secure flag; if that earlier request resolves last
it would null those fields. Now any non-forced response is dropped once
the store holds a server-reported platform. Forced refreshes and the
first authoritative write are unaffected.
* Studio: run the managed CLI help probe before trusting the preflight cache
Restore running the managed CLI help probe before returning Ready from
the desktop capability cache, so a managed install whose venv interpreter
or a runtime dependency is broken (while path, size, mtime, and markers
are unchanged) is reported Stale for repair rather than proceeding to a
backend start that cannot spawn. The capability cache still skips the
heavier desktop-capabilities probe on a hit, so a warm cache runs one
probe instead of two. Removes the executable-access shortcut, which the
help probe now subsumes.
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* show chat by by last activity
* Update chat thread updated_at logic and enhance sidebar chat item handling
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: add assistant response details panel
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Hide model badge by default, show on hover/focus
Wrap MessageResponseModelBadge in a span with hidden/group-hover visibility classes to reduce visual clutter. The badge now only displays when hovering or focusing on the assistant message, improving the UI presentation. Updated corresponding tests to verify the new CSS classes.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Run the malware gate on the RAG embedding model before it loads
Setting the RAG embedding model through PUT /api/settings/embedding-model
persisted an arbitrary repo and later handed it straight to
SentenceTransformer, which deserializes pickle weights. Unlike the normal
model-load paths, this route never ran evaluate_file_security, and force
skipped verification entirely, so a repo Hugging Face flags as unsafe (or
any repo under force) could be downloaded and loaded in the backend
process without a scan.
Run the malware/pickle scan at both ends: the settings endpoint now scans
before persisting and returns 409 on a flagged repo even under force
(force still only skips the is-embedding-model type check for offline or
local repos), and the embedder scans again at the load sink so a name that
arrives via env or default is covered too. Local paths and unreachable
scans fail open inside evaluate_file_security, and the sink never bricks
the embedder on a gate error.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Thread the load token into the embedding scan and hard-fail on a block
The load-sink scan ran without a token, so evaluate_file_security (which
passes token=False when none is given) could not reach a gated or private
repo and failed open for exactly the model SentenceTransformer would still
load. Resolve the loader's own token (HF_TOKEN env or the cached login)
and pass it to the sink scan, and fall back to it in the settings endpoint
when the request omits one.
The sink previously raised a plain RuntimeError, which the llama-server
fallback in encode() and _build_st_backend_or_fallback() swallowed as a
routine ST failure, silently switching backends instead of blocking. Raise
a distinct UnsafeEmbeddingModelError that both fallback paths re-raise, so
a flagged model hard-fails.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Scan sentence-transformers module dirs and scope the embedding pickle gate to the ST backend
Extend the RAG embedding malware gate so a poisoned pickle under a SentenceTransformer
module dir (for example 0_Transformer/pytorch_model.bin) blocks. Those dirs are read
from the repo's modules.json and passed as load roots to evaluate_file_security at both
the settings endpoint and the load sink, so such a pickle is treated as root-level there
instead of an unreferenced nested shard that was previously allowed.
Scope the ST pickle scan to the sentence-transformers backend. On the llama-server
backend the embedder loads GGUF files (inert) from the -GGUF companion repo, never the
ST repo's pickle, so a custom ST repo with a flagged pickle and a clean GGUF companion
is no longer rejected. The existing GGUF availability checks already cover that path.
Return 403 for the hard security block instead of 409. The settings UI routes every 409
into the forceable save-anyway flow, but this block cannot be bypassed by force, so it
now uses a distinct status the client treats as non-forceable.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Base the embedding pickle scan on the actual backend, not just the resolver
_llama_backend_active only consulted the auto resolver, so on a GPU box
where auto resolves to sentence-transformers but the process already fell
back to the llama-server backend at runtime (a torch or CUDA load/encode
failure), it returned False and the settings endpoint hard-blocked a save
whose ST pickle is flagged even though the process loads only inert GGUF.
Add active_backend_is_llama, which reflects the actual built backend (True
when the cached backend is a LlamaServerBackend, including a runtime
fallback) and otherwise defers to the resolver as a fresh process would,
and delegate _llama_backend_active to it.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Report the cached embedding backend verbatim, not the resolver
active_backend_is_llama() fell through to the config resolver whenever a
backend was already built but was not llama-server, so a live
sentence-transformers backend could report llama=True once the resolver
picked llama (GPU heuristic or a runtime config change) and wrongly skip
its pickle scan. Once a backend exists, return isinstance(backend,
LlamaServerBackend) directly; only defer to the resolver before any
backend is built.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio chat: tool-call nudging on by default (API stays opt-in)
Healing is already default-on everywhere and the nudge retry from the
client-tool passthrough is opt-in on the API. Studio chat had neither
signal: the frontend never sent nudge_tool_calls, and the safetensors
and MLX server-side loop lacked the GGUF loop's plan-without-action
re-prompt entirely.
Backend: the re-prompt helpers move from llama_cpp.py into
tool_call_parser.py (shared, cycle-free; the GGUF loop imports them
under its old names with zero behavior change) and
run_safetensors_tool_loop now re-prompts once at the streaming
no-tool-call exit, gated on Auto-Heal, active tools, nothing executed
yet, and short forward-looking text. Re-prompts do not consume tool
iterations.
Frontend: the chat adapter sends nudge_tool_calls from a new
nudgeToolCalls runtime setting (default true) with the same
persistence, hydration, and settings toggle plumbing as Auto-Heal.
Request-model defaults are untouched, so raw API callers stay opt-in.
* Address review: persist the nudge setting, consume the flag in the loops, skip the re-prompt after RAG autoinject
ChatSettingsPayload uses extra forbid, so a settings patch containing
nudgeToolCalls failed to persist any settings; the field is now typed
and round-trips. nudge_tool_calls now plumbs into both server-side tool
loops and gates the plan-without-action re-prompt with None meaning on,
so API callers keep today's behavior, explicit false disables it, and
Studio's default-on flag actually controls the path Studio chat runs.
The safetensors loop no longer re-prompts after RAG autoinject: the
injected retrieval bypasses the tool controller, so the nothing-executed
gate saw an empty history and re-asked after a successful retrieval.
* Safetensors loop: the plan-without-action retry requires an explicit nudge flag
The retry is new on this loop, so an omitted nudge_tool_calls must not
change existing API behavior; Studio opts in explicitly. The GGUF loop
keeps None as on because its re-prompt predates the flag.
* Suppress the plan-without-action re-prompt after a denied tool confirmation
A denial appends TOOL_REJECTED_MESSAGE but records nothing in the tool
controller history, so the nothing-executed gate re-prompted the model
to call the tool the user had just rejected, producing another
confirmation prompt. A denial now suppresses the re-prompt for the rest
of the request, mirroring the RAG autoinject handling.
* Tighten plan-without-action re-prompt comments
* Tighten plan-without-action re-prompt comments
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: match unified plan-without-action nudge cap to GGUF default of 3
The shared MAX_ACT_REPROMPTS was set to 1, but GGUF's established default
(llama_cpp.py) has re-prompted a stalling model up to 3 times since #5620.
Restore the GGUF-matched cap so safetensors and MLX inherit the same
behavior, and update the safetensors cap test to assert the cap dynamically.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
The live resource monitor and GPU readouts derive memory from binary byte
counts (bytes / 1024**3 for torch and psutil, MiB / 1024 for the nvidia-smi
path), which is GiB, but the UI labeled the values "GB". On a B200 this
showed "178.35 GB" for a card whose nvidia-smi total is 183359 MiB
(179 GiB), so it looked like memory was missing.
Relabel the measured RAM and VRAM readouts to GiB across the floating
monitor, the resources tab, the studio live GPU panel, the hub header, the
about tab and the onboarding summary. The numeric values are unchanged, so
the training GPU selection and memory-fit logic that read the same fields
are unaffected. Disk stays labeled GB because the backend reports it in
decimal GB (bytes / 1e9), and model file sizes and download progress keep
their decimal GB labels to match Hugging Face.
* Studio: expose full compressed-tensors scheme set in an export formats dropdown
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: multi-select export formats, portable torchao FP8/INT8, GGUF LoRA, source parity
Export page overhaul on top of the formats dropdown:
- Unify merged precision into one sorted multi-select list (16-bit first, then
8-bit, then 4-bit). Drop "vLLM" from labels, add INT8 (W8A8), INT8 (W8A16),
INT4 (W4A16), MXFP4, MXFP8. Quick formats render as toggle pills; the rest live
in a multi-select "More formats" dropdown, so several formats export in one run.
- Add a portable torchao FP8/INT8 save path (Float8WeightOnlyConfig /
Int8WeightOnlyConfig) that needs no NVIDIA GPU to produce and loads in vLLM.
FP8 serializes to safetensors, INT8 to .bin. Wired into save_pretrained_merged
and push_to_hub_merged via a TORCHAO_EXPORT_SCHEMES registry and
_unsloth_save_torchao, parallel to the compressed-tensors path.
- Hide NVIDIA-only compressed-tensors formats when no NVIDIA GPU is present; keep
16-bit and portable FP8/INT8. The backend also rejects a compressed request on
non-NVIDIA hardware so it stays authoritative.
- Relax merged export to non-PEFT models so Local Model and Hugging Face sources
get the same 16-bit / compressed / portable options.
- GGUF: send the whole quant list in one call (merge once, quantize many).
- LoRA: add a GGUF adapter option (convert_lora_to_gguf.py) with an outtype
select (f16/bf16/f32/q8_0/auto), alongside the safetensors adapter.
- Thread the new fields through models, routes, orchestrator, and worker; extend
the export tests.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: gate export by accelerator with a torch-aware reason; fix export save dir naming
Export runs through Unsloth, which requires a compute accelerator (NVIDIA/AMD/Intel
GPU or Apple MLX) and has no CPU code path, so a bare-CPU host cannot export even
with PyTorch installed. Add export_capability() in utils/hardware that reports
export_supported plus a precise reason so the UI stops showing a generic "no GPU":
- pytorch_not_installed: a --no-torch install (even a physical GPU is unusable)
- no_accelerator: PyTorch present but no supported accelerator (bare CPU)
- mlx_unavailable: Apple Silicon where the MLX stack is missing or too old
Expose the fields on /api/system/hardware and /api/system, and guard the mutating
export routes (load-checkpoint, export/merged|base|gguf|lora) with HTTP 400 and the
reason, leaving read-only endpoints usable so the Export page still renders.
Make core/export/export.py import without PyTorch and without a usable accelerator
(the Unsloth import is caught) so the export worker degrades to a clear message
instead of crashing at import.
Frontend: keep /export reachable on chat-only hosts and gray out the method and
format options with the backend reason (Alert plus disabled MethodPicker) instead
of silently redirecting to /chat, so users see why export is unavailable.
Also fix the export save directory producing "model/null" for Local Model and
Hugging Face sources that have no run/checkpoint, naming the folder from the model id.
* CI: validate Studio export capability gating on Linux, Windows and macOS
Add a small pytest matrix that runs studio/backend/tests/test_export_capability.py
on ubuntu-latest, windows-latest and macos-latest. It confirms, on each real OS,
that hardware.export_capability() reports the right decision and reason
(pytorch_not_installed, no_accelerator, or mlx_unavailable) and that the export
backend imports without PyTorch and degrades to a clear message instead of crashing.
Hosted runners have no GPU/MLX, so this covers the "export unavailable, here is why"
path a Mac/Windows user without an accelerator sees; a real accelerator export is
validated separately. The job installs only a CPU PyTorch plus the backend import
deps (no unsloth, triton, or llama.cpp), so it runs in seconds with no GPU.
* Studio export: address Codex review (source-aware gating, GGUF LoRA token/MLX/guard)
Frontend (export-page):
- Gate LoRA and quantized-model restrictions on the active source. isAdapter /
isQuantized come from the selected checkpoint; in Local Model / Hugging Face
("model") source mode they were stale, so LoRA stayed wrongly enabled for a
direct base model (backend then rejects "No adapter to export") and a stale
"quantized" flag disabled every method for an unrelated, exportable model. Add
effectiveIsAdapter / effectiveIsQuantized (false outside checkpoint mode) and use
them in the method-reset effect and the MethodPicker disabled state.
- Hide the GGUF LoRA option on a macOS/MLX host (the backend rejects GGUF LoRA on
MLX), so users no longer pick it, wait through the load, and always fail. Disable
the "GGUF adapter" button on a Mac host and never send loraGguf there.
Backend (core/export/export.py):
- Pass the HF token into the GGUF LoRA conversion (save_pretrained_gguf), so a
gated/private base model's config fetch in convert_lora_to_gguf.py is
authenticated; without it the load can succeed but the conversion fails.
- Guard the save_pretrained_gguf capability check with getattr so an older Unsloth
model that lacks the method returns the clean "not supported" message instead of
an AttributeError that surfaces as a generic 500.
* Studio export: address 2nd Codex review (CI index, empty merged, test import)
- studio-export-capability-ci.yml: add --extra-index-url https://pypi.org/simple to
the torch install so torch's transitive deps still resolve; --index-url alone
replaces PyPI with only the CPU wheel index, which does not serve all of them.
- export-page handleStart: reject an empty merged selection (mirrors canExport), so
clicking the panel's Start button with every precision pill deselected no longer
submits mergedSelections: [] and launches an unintended default 16-bit export.
- test_export_imatrix_compressed: the torchao-registry test now reads unsloth/save.py
as text (like the other ast/string checks) instead of `import unsloth.save`, which
raised ModuleNotFoundError in the CPU studio-backend suite that has no unsloth
installed.
* Studio export: make comments succinct across the export changes
* Studio export: use load token for local GGUF LoRA export of gated bases
* Studio export: harden portable torchao path and gate multi-format Hub push
torchao (_unsloth_save_torchao):
- merge to an isolated temp staging dir so a co-selected 16-bit output at save_directory is not deleted
- narrow VLM detection to vision_config / ForVisionText2Text so T5/BART/Whisper are not misrouted
- forward trust_remote_code (from auto_map) to the reload so custom-code models export
Export UI:
- hide portable torchao formats on macOS/MLX (backend rejects quantized export there)
- restrict a Hub merged export to a single format (each writes to the repo root)
* Studio export: torchao tokenizer remote-code + XPU offload, scale GGUF timeout
torchao (_unsloth_save_torchao):
- honor auto_map in the staged tokenizer/processor configs (not just model.config) when
deriving trust_remote_code, so custom-code tokenizers reload after the merge
- offload single-device XPU models to CPU (and empty the XPU cache) before the reload, matching
the CUDA path, so an Intel GPU that fits the model once does not OOM on the second copy
Export orchestrator:
- scale the GGUF wait timeout by the number of requested quants so a multi-quant list export of a
large model does not time out at a flat 3600s
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio export: show portable torchao formats only on non-NVIDIA (CPU) hosts
Portable torchao FP8/INT8 is the fallback for hosts without the NVIDIA compressed-tensors path.
On an NVIDIA GPU the compressed-tensors FP8/FP4/INT formats are the intended path (llm-compressor
auto-installs), so hide the portable duplicates there; keep them on CPU / non-NVIDIA hosts and
continue hiding them on macOS/MLX.
* Studio export: report all output folders and the exported formats
- Multi-format merged export now collects every sibling output directory (one per selected
precision) instead of only the last; the success banner lists them all.
- Show the selected precision formats in the run summary (a Formats row, like GGUF Quantizations),
so the panel says what is being exported rather than just 'Merged Model'.
- Persist the selected formats in the run summary and seed them on mount, so navigating away and
back (or toggling the export method) restores the selection instead of resetting to 16-bit.
* Studio export: list all output formats, add GGUF LoRA target, default Q8_0, auto-select newest checkpoint
- Progress/summary panel now shows a Formats row with the selected merged
formats, and the success banner lists every output folder a multi-format
merged run creates (one line per format) instead of only the last one.
- Merged format selection is seeded from the active run, so navigating away
and back (or switching method cards) no longer resets it to 16-bit.
- GGUF / Llama.cpp now offers an Export target toggle (Full model or LoRA
adapter) for adapter checkpoints, reusing the LoRA GGUF export path.
- Removed the Auto GGUF LoRA output type and defaulted to Q8_0 in the UI,
the request model, and the backend defaults; the outtype list is now
Q8_0/F16/BF16/F32. Core save.py still accepts auto for external callers.
- When a finetune has no checkpoint selected, auto-select the newest one.
* Studio torchao export: robust reload class + optional VLM import
Two fixes to the portable torchao FP8/INT8 export reload, from review of the
narrowed VLM detection:
- Encoder-decoder seq2seq checkpoints (T5/BART/Whisper) are not causal LMs.
With the narrowed is_vlm test they now correctly skip the image-text class,
but fell through to AutoModelForCausalLM and failed to reload after the merge.
Reload them with their own architecture class from the config instead.
- AutoModelForImageTextToText was imported unconditionally at the top of the
torchao path, so on Transformers builds without that class the import aborted
every torchao export (even text-only). Import it lazily only for a VLM, with
the AutoModelForVision2Seq fallback used elsewhere in Unsloth.
* Studio: enable FP8/FP4 compressed export for newer-transformers models
The shipped llm-compressor 0.10.x pins transformers<=4.57.6, so FP8/FP4 export failed
for models needing a transformers 5.x sidecar (Qwen3.5, Gemma-4, Qwen3-Next): the
quantization subprocess crashed importing the removed TORCH_INIT_FUNCTIONS.
Run the quantization against a dedicated llm-compressor-main "shadow": a --target
package dir (transformers 5.10.2 + llm-compressor main + compressed-tensors) layered
over the existing torch. It installs --no-deps so torch is never touched (works on any
Studio torch build), is provisioned lazily and fingerprint-cached, and can be turned
off with UNSLOTH_DISABLE_LLMCOMPRESSOR_MAIN.
- transformers_version.py: provision + validate .venv_llmcompressor.
- export.py: route all compressed exports through the shadow when available; else keep
the workspace 0.10.x path and fail fast past its transformers ceiling.
- save.py: launch _compressed_quantize.py with a clean PYTHONPATH = shadow.
- _compressed_quantize.py: skip linear_attn / vision tower / MTP modules (matches the
RedHatAI and NVIDIA reference quants, and is required by the grouped schemes).
Verified all four schemes (fp8, w8a8, w4a16, mxfp4) on Qwen3.5-9B and Llama-3.2-1B, and
fp8 on Gemma-4, end to end through Studio.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix GGUF LoRA export tests
* Fix export CI expectations
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: wasimysaid <112766706+wasimysaid@users.noreply.github.com>
Co-authored-by: Wasim Yousef Said <wasimysdev@gmail.com>
* replaced connect with start
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix
* Studio: build the coding-agent command from the selected server
The API keys panel showed a hardcoded `unsloth start claude`. `unsloth start`
defaults to 127.0.0.1:8888 and only mints a key for a loopback server, so a
non-default port or a tunnel/remote base would target the wrong server or fail
to mint. Build the command from the panel base/key (and emit a key for
non-loopback), matching the other snippets in the panel.
* CLI: keep `unsloth connect` as a hidden alias for `unsloth start`
Avoids breaking existing scripts and docs that still call `unsloth connect`.
* Tests: stub _unstarted_cleanup in same-task disconnect test
The test builds _SameTaskStreamingResponse via __new__, so set the attribute
that __call__ now reads.
* Match coding-agent command loopback check to the CLI 127.0.0.0/8 rule (#6613)
* Keep unsloth_cli.commands.connect importable as a deprecated shim (#6613)
* Format the new coding-agents panel strings and import per biome (#6613)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Drop the unsloth connect alias and shim; unsloth start is the only command (#6613)
* Route unsloth connect to unsloth start as a hidden backward-compatible alias (#6613)
* Forward unsloth run model-load flags to unsloth start (gguf-variant, context-length, load-in-4bit, tensor-parallel) (#6613)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Session-scope coding agent config in unsloth start
Configure each agent for the current session instead of writing the Studio endpoint, key, and default model into the user's own config. Codex, OpenCode, OpenClaw, and Hermes get a private config relocated through their config-path env vars (CODEX_HOME, OPENCODE_CONFIG overlay, OPENCLAW_CONFIG_PATH plus OPENCLAW_STATE_DIR, HERMES_HOME). Claude Code suppresses the attribution header for the session via the CLAUDE_CODE_ATTRIBUTION_HEADER env var plus a --settings overlay, with no ~/.claude write. --launch uses an ephemeral temp dir removed after the agent exits; --no-launch uses a stable Unsloth-owned dir and prints the matching export lines.
* Read relocated agent session config in Local Agent Guides CI
The contract crosscheck and the openclaw/hermes patch helpers now read each agent's config from the relocated path printed by unsloth start --no-launch (CODEX_HOME, OPENCODE_CONFIG, OPENCLAW_CONFIG_PATH, HERMES_HOME) instead of fixed home paths. The Claude attribution A/B toggles the header for the session only (shipped-config HIT vs vanilla MISS) instead of editing ~/.claude/settings.json.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Skip the POSIX-only --no-launch parser test on Windows
test_no_launch_output_is_parseable mirrors the #6547 bash CI parser, which greps export/unset lines and only runs on Linux/macOS runners. On Windows --no-launch prints PowerShell ($env: / Remove-Item), so the export-line assertion does not apply there. Cross-OS staging CI surfaced this.
* Size Claude Code's auto-compact window to the loaded model's context
Claude Code auto-compacts against its native (~600k token) window, so against a smaller local model it overflows the server's context (silent truncation) long before it compacts. Set CLAUDE_CODE_AUTO_COMPACT_WINDOW to the loaded model's real context length (the value codex/openclaw already get via model_context_window / contextWindow). Omitted when the model reports no context length.
* Pin OpenCode/Hermes context window and set 90% compaction across agents
Feed every agent the server-determined sequence length (the value /v1/models reports from runtime_context_length) and a ~90% compaction threshold. OpenCode: a custom-provider model with no limit defaults to context 0, which silently disables auto-compaction, so set limit.context/output and scale the compaction buffer to 10% of the window. Hermes: pin model.context_length (it otherwise falls back to a 256k default when the server's /v1/models omits the field) and set compression.threshold 0.9. Claude: add CLAUDE_AUTOCOMPACT_PCT_OVERRIDE=90 alongside the window. Codex (model_context_window) and OpenClaw (contextWindow) already carried the window and auto-manage off it.
* Add `unsloth start pi` recipe
Pi was the only agent without a built-in recipe, so the agent-guides CI
hand-wrote ~/.pi/agent/models.json. Add a first-class `pi` command mirroring
the others:
- write_pi_config writes the session-scoped OpenAI-compatible provider config
(key in the config, like openclaw/opencode).
- pi() launches `pi --provider unsloth --model <id>` (Pi defaults to the google
provider, so the provider/model are pinned on the command line) with HOME
relocated for the session. Pi has no config-dir env var and resolves ~/.pi off
$HOME, so HOME-scoping keeps the user's ~/.pi untouched.
Migrate the agent-guides CI off the hand-written config onto the
`unsloth start pi --no-launch` path (connection + file-edit), with a crosscheck
for the provider api, so the documented recipe is exercised.
* Harden unsloth start for Windows and WSL agent launches
Address the Codex review on PR 6613:
- write_pi_config now pins the loaded contextWindow and a sane maxTokens so Pi
compacts instead of overflowing a small Studio context (it otherwise assumes
its 128000 default), matching the other agents.
- pi() sets USERPROFILE (and HOMEDRIVE/HOMEPATH when present) alongside HOME on
native Windows, where Node resolves ~/.pi via USERPROFILE rather than HOME, so
the session no longer reads or writes the user's real ~/.pi.
- The WSLENV bridge flags path-valued vars with /p so a Windows npm shim under
/mnt receives translated paths, while scalar vars (the numeric context window)
pass through untranslated. WSLENV is deduped on the bare name.
- _print_env prints the launch command with PowerShell-safe quoting so the inline
--settings JSON survives copy-paste on native Windows --no-launch.
Add tests for the WSLENV path flagging, PowerShell quoting, the Pi context
window, and the Pi USERPROFILE relocation.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Set CLAUDE_CODE_NO_FLICKER for the Claude session
A local server streams in bursts, so Claude Code's full-screen TUI redraw
flickers between tokens. Disable it for the session via CLAUDE_CODE_NO_FLICKER,
alongside the other CLAUDE_CODE_* session env knobs.
* Add a normalized --yolo flag routed to each agent's auto-approve mode
It is easy to forget which agent spells "run tools without prompting" which way,
so `unsloth start` now accepts all three spellings as one option (--yolo,
--dangerously-skip-permissions, --dangerously-bypass-approvals-and-sandbox) and
routes to the agent's own mechanism:
- claude: --dangerously-skip-permissions
- codex: --dangerously-bypass-approvals-and-sandbox
- hermes: --yolo
- pi: --approve (Pi's only approval gate is project trust)
- opencode: a permission allow block in opencode.json (no CLI flag exists)
- openclaw: tools.exec security=full / ask=off / host=gateway (no CLI flag exists)
Because the option is parsed by `unsloth start`, the "wrong" spelling for an
agent still routes correctly instead of leaking through to the agent and erroring.
IS_SANDBOX is deliberately left unset for Claude so its root/sandbox safety gate
still applies. Adds routing, cross-routing, and per-config tests.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix review findings: IPv6 loopback command, pi USERPROFILE under WSL, yolo guard
From a 10-reviewer pass over the PR:
- studio/frontend agent-command.ts: normalize bracketed IPv6 hosts. URL.hostname
returns "[::1]" for http://[::1]:8888, which never matched the "::1" loopback
checks, so the copied command embedded the placeholder API key for a local IPv6
server instead of the bare auto-minting command. Now [::1] is treated as loopback
like the CLI's is_loopback_url, so the command matches the CLI contract.
- pi(): also relocate USERPROFILE (and HOMEDRIVE/HOMEPATH) when running under WSL
against a /mnt Windows shim, not just on native Windows. Windows Node resolves
~/.pi via USERPROFILE, and the WSLENV bridge translates the path, so pi no longer
falls back to the user's real ~/.pi in that case.
- _yolo_command_flags: use .get so a config-based agent (or a typo) yields no flag
instead of a latent KeyError.
Adds tests for the WSL pi USERPROFILE relocation, the yolo unmapped-agent guard,
and that opencode/openclaw --yolo stays config-only (no argv flag).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix round-2 review findings: WSLENV /p upgrade, agent help text
- _merge_wslenv now upgrades a user's pre-existing unflagged WSLENV entry (e.g. a
bare HOME or USERPROFILE) to the path-translated form (HOME/p) instead of leaving
it as-is, so a Windows agent shim under WSL receives the translated session path
rather than the raw Linux path.
- Generalize the `unsloth start` registration help to list all six agents (was only
"Claude Code, Codex").
Adds a test for the WSLENV unflagged-entry upgrade.
* Fix round-3 review findings: complete openclaw --yolo, refresh stale copy
- openclaw --yolo now also writes the host approvals file (exec-approvals.json with
defaults security=full / ask=off / askFallback=full) alongside the tools.exec
config. OpenClaw gates tool execution on both layers (the stricter wins), so the
config alone could still leave it prompting or denying. Mirrors `openclaw
exec-policy preset yolo`. ask=off means nothing is ever prompted, so the runtime
socket block is unnecessary.
- Studio API panel copy: clarify that a local server auto-mints the key while a
remote one embeds it in the command, and add pi to the swap hint.
- Local Agent Guides CI: drop the stale "pi has no start.py recipe" note now that
all six agents are driven via `unsloth start <agent> --no-launch`.
Adds the openclaw approvals-file assertions and a no-yolo openclaw test.
* start: parse claude --version with a regex so a format change does not drop optimization flags
* start: offer to install a missing agent (prompt then run its install command)
* start: auto-start a Studio server for --model when none is running, and stop it on exit
* inference: surface an actionable message when llama-server cannot compile a tool grammar
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix review findings: kill the auto-started server tree on Windows; apply the tool-grammar message to the OpenAI passthrough too
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* start: split --model org/repo:variant so a running session is not evicted
`unsloth start <agent> --model org/repo:QUANT` failed against an already-running
Studio server and, worse, killed whatever model another session had loaded.
/v1/models lists a loaded GGUF under its bare repo id (e.g. unsloth/Qwen3-1.7B-GGUF),
so _resolve_model never matched the `:QUANT`-suffixed request. It then POSTed
/api/inference/load with model_path=org/repo:QUANT, which (a) Hugging Face rejects
("Repo id must use alphanumeric chars, '-', '_' or '.'") and (b) evicts the model the
other session was using, so a second 'unsloth start' in a new tmux/terminal tore down
the first. Re-running the command then attached to the now-empty server, which is why
it 'worked the second time'.
Mirror the org/repo:QUANT -> org/repo + --gguf-variant QUANT shorthand that
'unsloth run' and llama.cpp already accept, splitting it in _connect before we match or
serve. Matching now resolves against the loaded bare repo id (no spurious reload, no
eviction), and any real load uses a valid repo id plus gguf_variant. An explicit
--gguf-variant still wins; local paths and Windows drive letters pass through untouched.
The auto-serve path likewise spawns 'unsloth run --model org/repo --gguf-variant QUANT'.
* start: harden auth-key handling, codex teardown, and CI transcript redaction
Three review findings:
1. CI could leak a live key. agent-guides-drive.sh printed the raw
'unsloth start --no-launch' transcript (which carries export UNSLOTH_API_KEY /
ANTHROPIC_AUTH_TOKEN lines) to the Actions log on both the failure path and the
success path before redact() ran. Add cat_redacted() and use it for those two
prints, so the key is scrubbed on the way to the log while the on-disk file stays
intact for the env parsing that follows.
2. Outages masqueraded as bad keys. _key_accepted caught a broad Exception and
returned False, so a 5xx or timeout while checking a cached key looked like a
rejection: it discarded a good key and minted extra ones (local) or reported 'no
saved key' (remote). Only treat HTTP 401/403 as a rejection; let other errors
propagate so a real outage surfaces.
3. Codex preflight could leave the auto-started server up. _require_gguf_for_codex
runs after _connect may have auto-started Studio but before _run installs its
teardown finally, so a preflight rejection (e.g. a transformers-backend model) left
the server holding the port/GPU until the atexit backstop. Tear it down explicitly
at the point of failure.
Tests: a 5xx on a saved key surfaces without minting; a non-GGUF codex preflight
tears down the auto-served server.
* start: fix IPv6/portless studio URLs, Pi config-dir isolation, and Pi install recipe
Four review findings:
1. Pi ignored the session config when PI_CODING_AGENT_DIR was already set. Pi's
getAgentDir() reads process.env.PI_CODING_AGENT_DIR before falling back to
$HOME/.pi/agent, so a value inherited from the user's shell sent Pi to their real
config and skipped our provider/key (the HOME relocation alone was not enough). Pin
PI_CODING_AGENT_DIR at the session's .pi/agent dir; it is path-valued so the WSL
bridge translates it automatically.
2. Pi install hint dropped Pi's documented --ignore-scripts. Pi's README installs with
'npm install -g --ignore-scripts @earendil-works/pi-coding-agent' and notes it needs
no install scripts, so accepting the prompt now follows that safe recipe.
3. Auto-start ignored a portless UNSLOTH_STUDIO_URL. unsloth run binds to
'parsed.port or 8888', so http://127.0.0.1 launched the child on 8888 but the health
poll (and the returned base) still used port 80, stalling until the startup timeout.
Normalize the base to host:8888 (IPv6-safe) before starting and polling.
4. API-panel command mistook IPv6 loopback for the bare default. The bare 'unsloth
start' only probes 127.0.0.1:8888 on the IPv4 stack, so http://[::1]:8888 must carry
an explicit UNSLOTH_STUDIO_URL. Drop ::1 from the bare-default host set while keeping
it a loopback host (URL emitted, no key needed).
Tests: PI_CODING_AGENT_DIR is set to the session dir; _effective_base normalizes
portless/IPv6 bases; a portless UNSLOTH_STUDIO_URL auto-serves on :8888.
* start: apply fresh-review findings across CLI, CI, and the API-panel command
From a fresh multi-reviewer pass over the merged head plus the latest Codex bot review:
1. Load knobs now always consult the server. _resolve_model matched on model id alone,
so --gguf-variant / --context-length / --no-load-in-4bit / --tensor-parallel were
silently ignored whenever the id was already loaded (asking for UD-Q4_K_XL kept a
Q8_0 serving). With any explicit knob the CLI defers to /api/inference/load, whose
already-loaded dedup answers without reloading when variant and settings match, so a
second session running the same command still attaches without evicting the first.
2. OpenCode --yolo and the session model pin now ride in OPENCODE_CONFIG_CONTENT. A
project's own opencode.json outranks OPENCODE_CONFIG, so a repo config could silently
override the session model and the --yolo permission block; OPENCODE_CONFIG_CONTENT
outranks project config. The API key stays in the private file, never in printed env.
3. The --no-launch recipe's last line is a self-contained one-liner (inline VAR=value
assignments before the command, conflicting vars blanked). People copy just the last
line, and a bare codex/claude there ran against the user's real ~/.codex or Anthropic
credentials with zero isolation, e.g. inheriting a pre-existing damaged ~/.codex
state DB and blaming the recipe. The CI drive script scrubs the key from the one
'invoking:' echo this adds.
4. The auto-serve log is 0600 and the parent handle is closed. It sat world-readable in
the shared tempdir under a predictable name while carrying the minted sk-unsloth-
key from the unsloth run banner.
5. _key_accepted fails with a clean message on outages. Non-auth errors (5xx, network,
timeout) surfaced as a raw traceback; 401/403 still mean a rejected key.
6. _effective_base strips URL paths, and https loopback targets never auto-serve.
http://127.0.0.1:8888/studio polled /studio/api/health (404) and https://127.0.0.1
polled the wrong scheme, both spinning until the 15-minute startup timeout.
7. API-panel command: only literal 127.0.0.1:8888 earns the bare command. localhost can
resolve to ::1, which the bare CLI never probes, so it keeps UNSLOTH_STUDIO_URL.
8. CI artifact sweep covers redacted-configs/ and agent-workdir/, not just logs/.
Tests: 125 CLI tests pass (new coverage for each fix), 156 backend tests pass, ruff
clean. Adds an unsloth connect alias regression test.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* start: hand Pi a clean screen at launch
Pi paints inline from wherever the cursor sits: its first render assumes a
clean screen instead of clearing or entering the alternate screen itself
(current Pi never emits a clear at startup). Launched under unsloth start,
that left the session starting mid-scroll beneath the connection output.
Clear the screen (click.clear, cross-platform, no-op without a TTY) right
before the Studio banner so Pi opens exactly one line down on a clean
viewport. Launch path only: --no-launch recipes and piped output are never
wiped, and alternate-screen agents are left alone.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* start: auto-override hermes' 64K context floor for small model windows
Hermes refuses to initialize when the served model's context window is
under 64,000 tokens, and a second copy of the same check rejects the
compression model mid-session. write_hermes_config previously pinned the
real window, so any small local model (e.g. 40,960) failed at startup
with manual config.yaml instructions.
For windows below the floor the recipe now claims 65,536 in
model.context_length, scales compression.threshold so compaction still
fires at 90% of the real window, and sets
auxiliary.compression.context_length to cover the mid-session check.
Windows at or above the floor keep the exact previous behavior.
* ci: install pi with --ignore-scripts, matching the start.py hint
The pi cell predates the pi recipe in start.py and still installed the
package with lifecycle scripts enabled, so CI stopped exercising the
exact command users are prompted to run. npm_retry now passes extra
flags through, the pi branch mirrors the install hint verbatim, and the
stale no-recipe comment is refreshed.
* ci: fail loudly when a relocation var is missing from connect output
The empty-string guards ran after appending /config.toml or /config.yaml,
so they could never fire: crosscheck_contract silently skipped its
contract checks and patch_hermes_tools died on the root path with a bare
traceback. Check the raw variable first and guide_fail with the real
cause.
* staging: 6613 round 6 (https elision, no-launch home reuse, auto-start key fallback)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: shimmyshimmer <107991372+shimmyshimmer@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: Wasim Yousef Said <wasimysdev@gmail.com>
* feat: Implementation of the Portuguese (Brazil) language and VRAM/RAM monitor.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update studio/frontend/src/hooks/use-gpu-utilization.ts
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* Update studio/backend/main.py
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* Update studio/backend/main.py
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update studio/backend/utils/hardware/hardware.py
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* Update studio/backend/utils/hardware/hardware.py
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update studio/frontend/src/features/settings/components/usage-examples.tsx
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update studio/backend/main.py
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* Update studio/backend/utils/hardware/hardware.py
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* Update studio/frontend/src/features/studio/sections/progress-section.tsx
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* fix: resolve automated review feedback on API shape
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix review issues for PR #6509: Cpu icon, VRAM percent, system polling
- model-inspector: use the exported CpuIcon (Cpu is not a Hugeicons export)
- app-sidebar: guard the VRAM percent on totalVram to avoid Infinity, and
reset the system poll cache only after each request settles so a slow probe
is reused instead of stacking overlapping requests
- use-gpu-info: populate CPU/RAM on hosts without a GPU
- progress-section: label GPUs by visible_ordinal instead of array index
- hub-page: base the RAM label on systemRamTotalGb
- usage-examples: emit JS sampling and tool options at the top level instead
of nesting them under extra_body (the JS SDK does not unwrap extra_body)
- main: read torch and transformers versions from package metadata instead of
importing the libraries on every system poll, and guard the VRAM math
against null values
- hardware: translate a leftover comment to English
* Harden /api/system: guard psutil.boot_time for PR #6509
Simulating restricted containers and some VMs (where psutil.boot_time can raise)
showed the /api/system endpoint would 500 on the unguarded boot_time call, the
same failure class already handled for cpu_freq, disk_usage, and Process. Wrap
boot_time and return uptime_seconds as null when it is unavailable so the sidebar
monitor degrades gracefully instead of breaking. Widen the uptime_seconds type to
number | null to match.
* Studio: make the sidebar hardware monitor a toggle (default on) for PR #6509
Adds a "Show hardware monitor" switch under Settings > Appearance > Layout,
backed by a localStorage preference (default on), mirroring the existing
useSidebarPin pattern. When turned off, the sidebar hides the VRAM/RAM meters
and useSystemInfo stops the 3s /api/system poll entirely, so no nvidia-smi /
SMI probes run while the monitor is disabled. Adds the en and pt-BR strings.
* Studio: default the sidebar hardware monitor to off (opt-in) for PR #6509
* Studio pt-BR: fix three small translation defects for PR #6509
- learningRateDescription: "5e-5 for CPT" -> "5e-5 para CPT" (leftover English)
- exportScopeRecents: "Recents" -> "Recentes" (untranslated)
- relativeMonthsAgo/relativeYearsAgo: add the missing space ("há {count} meses"/
"há {count} anos") so they no longer render as "há 3meses"
* Studio pt-BR: translate the last 10 fallback keys for PR #6509
Adds the settings.general.storage block (Armazenamento) and the
settings.chat.modelDisclaimer pair, so pt-BR now covers all en keys
(679/679) with no English fallbacks.
* Studio: hide sidebar VRAM row on CPU-only hosts for PR #6509
* Studio: tighten and trim code comments for PR #6509
* fix: UI issue in the stop button dialog box (fine-tuning)
* Studio pt-BR: translate 18 new keys from main merge (password dialog, GGUF export, dataset streaming) for PR #6509
* Rounding to GB
* Fix/adjust System resources tab for PR #6509
* Fix/adjust GPU monitor review items for PR #6509
* Fix/adjust remaining GPU monitor review items for PR #6509
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix/adjust MLX resource fallback for PR #6509
* floating window implementation
* resize for floating window
* Fix resource monitor review items
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Restore frontend optional dependency lock entries
* Make GPU selection tests hermetic
* Fix GPU monitor CI test failures
* Bound MLX GGUF reload smoke
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix MLX GGUF reload smoke exit
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
* Add customizable RAG embedding model setting and reorganize settings tabs
Chat with files, project sources, and knowledge bases previously always
embedded with unsloth/bge-small-en-v1.5. This adds a Settings option to
pick any Hugging Face embedding model (or local path), with HF search
autocomplete, server-side verification that the repo is actually an
embedding model, and a save anyway escape hatch for offline or local
models. The setting persists in app_settings and applies at runtime to
both the sentence-transformers and llama-server GGUF embedder backends
without a restart.
Also reorganizes the General settings tab: Documents & RAG sits above
Uploads, Helper LLM moved above the danger zone, and Model auto-switch
(OpenAI API) moved to the bottom of the API tab.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Support local model paths on the GGUF embedder and normalize default saves
Found by simulation testing of the embedding model setting:
Local paths saved as the embedding model now work on the llama-server
GGUF backend (the default backend on macOS and CPU). A path to a .gguf
file is used directly and a directory is scanned for a variant-matching
non-mmproj .gguf, with a clear error when none exists. Previously a
local path was sent to the HF hub API and failed with a repo lookup
error.
Saving the default model explicitly no longer stores an override, so
is_custom stays false and the UI does not show a reset button for the
default value.
* Address review: stale-vector handling, GGUF derivation, save-time guards
Review follow-ups, each verified by new tests:
Re-uploading a document after an embedding model change now re-indexes
instead of deduping by content hash. Documents record the embedder that
produced their vectors (lazy embedding_model column, NULL legacy rows
keep deduping) and a mismatch replaces the old document.
A vector width change no longer bricks the dense index. ensure_vec
drops and recreates chunks_vec when the dim changes (old vectors are in
a foreign space and only block inserts) and search_dense returns empty
on a width mismatch instead of surfacing a vec0 error, so lexical
search keeps working until documents are re-uploaded.
Saving a local sentence-transformers folder with no .gguf now returns
409 with a clear message when the install embeds via llama-server,
instead of failing at first index. force still saves.
A custom RAG_EMBEDDING_MODEL env without RAG_EMBED_GGUF_REPO now
derives the -GGUF companion repo instead of silently keeping the bge
GGUF on CPU and macOS installs.
The resolved GGUF path is tagged with the repo captured at entry, so a
setting change during a download cannot mark the old model as current.
GGUF repo detection matches gguf as a whole name segment rather than a
substring, hf_token is trimmed before verification, and the settings
combobox drops a redundant state mirror of its controlled value.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Shrink embedding model font to 11px in the input and dropdown
The combobox wrapper applies className to the outer input group, so the
size utility must target the inner input element; the previous text-xs
never reached it and the field rendered at the browser default.
* Show curated unsloth embedding models when the search field is empty
The empty-query listing was the global top-downloads page, which holds
no unsloth mirrors for the unsloth-first float to reorder, so the
dropdown opened on third-party models. Match the model picker: curated
unsloth listing when empty, whole-Hub search once a query is typed.
* Address review: settings resilience and index consistency
Keep the last known embedding model on settings store errors, remove the
re-entrant dim lock in the llama-server backend, accept local GGUF saves
and verify GGUF availability for HF repos on that backend, match local
path embedders exactly in model list filters, drop same-width stale
vectors from dense search, pin the embedder per ingestion job, and only
replace completed documents after the re-index succeeds.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Consolidate the GGUF repo derivation tests
* Trim to a single core embedding-model test
* Address review: GGUF repo saves and cache race
Accept a GGUF-named HF repo on the llama-server backend by verifying GGUF
availability instead of the sentence-transformers metadata gate, and guard
the settings cache with a generation counter so a read overlapping a save
cannot repopulate it with the pre-save value.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Add a shared fits-on-device filter to the model selects
The chat model selector gains an Only show models that fit on this
device tick under its filter row, and the Hub page gains a matching
Fits device pill next to the sort menu. Both read one persisted
preference (unsloth_models_fit_on_device_only), so toggling either
applies to both.
The filter reuses the Recommended sort's existing fit math, extracted
into hfModelFitsDevice: size from safetensors metadata, GGUF param
count, or the repo name, against the 0.7 GPU + 0.7 RAM budget, with
unsizable models hidden. In the chat selector it extends the fit
filtering to the Trending and Recent sorts and to search results;
downloaded models stay visible regardless. An unknown device budget
keeps everything. The preference is cleared by Reset all local
preferences like the other picker toggles.
* Move the device-fit toggle into the sort dropdowns
* Tighten sort menu footer spacing and shorten the label
* Align the footer checkbox with the option text
* Make the footer checkbox circular with a smaller tick
* Clear menu highlight when the pointer leaves the options
* Address review: fit filter coverage and sizing
Exempt on-disk models from the Hub fit filter, apply it to the feed
trending rows and curated search results, size safetensors and MLX rows
by the quantized load estimate instead of checkpoint bytes, and replace
the native title hint with the app Tooltip.
* Make the whole device-fit row toggle the filter
* Studio: opt-in OpenAI /v1 model auto-switch and idle keep-warm
The OpenAI-compatible endpoints serve whichever GGUF is loaded and ignore the
request model field, so an OpenAI client that changes model never reloads. Add
an opt-in setting that, when a /v1 request names a downloaded local GGUF
different from the loaded one, loads it before serving by reusing the existing
/load path (its dedup, tensor fallback, and threading apply). Unknown names
still serve the loaded model, so drop-in compatibility is preserved and no
remote download is triggered.
Also add an optional idle auto-unload (TTL keep-warm): a pure-ASGI middleware
tracks in-flight inference requests so a stream is never unloaded mid-response,
and a lifespan loop unloads the model after the configured idle seconds. Both
settings default off and live in the app_settings store, exposed via
GET/PUT /api/settings/openai-auto-switch.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: variant-aware auto-switch, /v1/responses coverage, keep-warm load stamp
Follow-ups from review of the opt-in OpenAI auto-switch path:
1. Variant-aware dedup. _maybe_auto_switch_model compared only the repo id, so
requesting another quant of the loaded repo (e.g. Q4_K_M loaded, Q8_0 asked)
was served by the old quant. Compare hf_variant too, matching /load dedup.
2. Streaming /v1/responses now calls the auto-switch hook. It went straight into
_responses_stream and only checked is_loaded, so stream=True could serve the
old model or 400. Non-streaming already routed through chat completions; the
hook is idempotent once loaded.
3. resolve_local_gguf tries an exact id match before splitting a trailing
:VARIANT, so local ids that contain a colon (e.g. a Windows path) resolve
instead of being cut at the drive letter.
4. Idle keep-warm stamps activity on a load/swap transition. _last_active was
only refreshed by inference requests, so a model loaded after the server sat
idle past the TTL could be unloaded before its first request.
Tests cover each case.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: make the /v1/responses auto-switch test order-independent
The new streaming-responses test passed in isolation but failed under the CI's
randomized collection order with "object has no attribute 'state'": it passed a
bare object() as the request and stubbed only one dispatcher, so an ordering
where the real dispatcher ran hit request.state. Give the request a state and
stub both dispatchers; the test still asserts the hook fires before dispatch.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: assert /v1/responses auto-switch wiring on source, not at runtime
The behavioral version executed openai_responses and relied on stubbing its
callees, which a randomized collection order in CI could defeat (the real
dispatcher ran and hit request attributes). Assert on the function source that
the hook precedes both dispatchers instead; the hook's runtime behavior is
already covered by the direct _maybe_auto_switch_model tests.
* Studio: auto-switch on /v1/embeddings, GGUF-only targets, idle-unload race gate
Second-pass review follow-ups on the opt-in auto-switch path:
1. /v1/embeddings now calls the auto-switch hook before the loaded-state check,
matching the other model-bearing OpenAI endpoints (the keep-warm middleware
already treats embeddings as inference).
2. The resolver index is now GGUF-only. The local-model scanners also surface
Transformers/safetensors repos; without a filter, auto-switch could unload
the GGUF and route a request into the non-GGUF loader. _has_local_gguf checks
a direct .gguf, a models-dir folder, and the HF-cache snapshots layout.
3. Idle keep-warm now holds an asyncio gate across the idle check and the
unload, and a request bumps inflight under the same gate, so the loop can no
longer unload in the window between "looks idle" and the kill.
Tests cover each. Broader local-model source parity (LM Studio, Ollama, legacy
caches, custom scan folders) is a follow-up; missing one of those today just
falls through to the loaded model.
* Studio: variant-aware local resolver, count_tokens + audio auto-switch coverage
Third-pass review follow-ups on the opt-in auto-switch path:
1. The resolver is now variant-aware via list_local_gguf_variants. It indexes
only the quants actually on disk, recursing snapshots and quant subdirs such
as the nested per-quant folders, so a requested repo:VARIANT resolves only
when that quant is local and a bare repo resolves to a concrete local quant.
This fixes two gaps: the previous shallow glob rejected nested-variant GGUF
repos, and a request for an uncached quant could send /load down the remote
download path, breaking the local-only contract.
2. /v1/messages/count_tokens now auto-switches like its sibling /v1/messages, so
a count uses the requested model's tokenizer.
3. /api/inference/audio/generate (direct GGUF TTS) is now tracked as in-flight
inference, so the idle loop cannot unload the model mid-generation.
Tests cover each. Two reviewer items are left as follow-ups: indexing the
remaining local sources (LM Studio, Ollama, legacy/default caches, custom scan
folders), which fails safe today by falling through to the loaded model; and
fully serializing concurrent different-model requests, an inherent limit of the
single-slot llama backend that the opt-in feature is not designed around.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: make local GGUF resolver fail-safe so a bad model name cannot 500
The auto-switch hook calls resolve_local_gguf without its own guard, and
/v1/completions and /v1/embeddings pass body.get("model") through unchanged.
A non-string model (e.g. {"model": 123}) or any internal scan failure would
then raise out of the resolver and turn a request that would otherwise be
served by the loaded model into a 500, breaking the drop-in compatibility the
feature is built on.
Guard the resolver at its boundary: reject non-string input up front and wrap
the lookup so any failure returns None (fall through to the loaded model).
Add regression tests for both paths.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: per-model launch flags for auto-switched GGUF models
* Studio: list switch-eligible GGUFs in /v1/models when auto-switch is on
* Studio: settings UI for OpenAI model auto-switch and idle auto-unload
* Studio: show save error over the disabled-idle hint in auto-switch settings
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: address gemini review (case-insensitive /v1/models retrieve, idle-input empty guard)
* Studio: address codex review (deterministic override args, exclude probe/embedding models from discovery)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: keep-warm count_tokens, gate idle on auto-switch, drop hidden models
Three hardening fixes to the opt-in auto-switch path surfaced while reviewing
the work that builds on it:
1. count_tokens keep-warm. /v1/messages/count_tokens counts via the loaded
tokenizer and already auto-switches, but the keep-warm middleware did not
track it, so idle auto-unload could free the model mid-count. It is now a
tracked in-flight path.
2. "Off means unchanged" for idle unload. get_auto_unload_idle_seconds now
reports 0 while auto-switch is disabled. Idle unload only makes sense with
auto-switch on (an unloaded model returns only via the next request's swap),
so a stray TTL can no longer trigger a destructive unload while the feature
is off, keeping the disabled state identical to pre-feature behavior.
3. Hidden models are not switch targets. The resolver index now skips what
Studio hides from its own pickers (the llama.cpp validation probe, RAG
embedding weights) via _is_hidden_model, so they can never be auto-switched
to by name.
Tests added for each.
* Studio: bare-id reuse, responses validation order, in-flight tracking
Review follow-ups after folding in the per-model overrides and discovery work:
1. A bare model id (no :VARIANT) is now satisfied by any loaded quant of that
repo. Previously a bare name resolved to the largest local quant, so it could
force a slow reload when a different quant of the same repo was already
serving. An explicit repo:VARIANT request still honors the quant.
2. /v1/responses now runs the auto-switch hook after the empty-input validation
so a request that 400s can no longer trigger a multi-minute model load before
being rejected. The hook still precedes both dispatchers, so streaming
requests switch.
3. The keep-warm middleware now tracks in-flight requests whenever auto-switch
is enabled rather than only when the idle TTL is already positive, so a stream
that starts with the TTL at 0 is still protected if idle-unload is enabled
mid-stream. Off still passes straight through.
Tests added for each.
* Studio: tighten auto-switch code comments
Comment/docstring-only pass over the OpenAI auto-switch feature: collapse
multi-line blocks, drop a comment that restated the gate it sits next to, and
trim verbose docstrings on internal helpers while keeping the load-bearing
rationale (concurrency, API behavior, drop-in compat, gotchas). No logic
change: verified comment-only with the AST/printer signature check.
* Studio: bind auto-switch locks per running loop
Review follow-up. The auto-switch swap lock and the keep-warm unload gate were
module-level asyncio.Lock objects. That is safe under the single uvicorn loop
and on Python 3.10+ (the Lock resolves the running loop lazily on acquire), but
a module-level Lock binds to one loop on pre-3.10, which can raise a loop
mismatch in multi-loop runners. Resolve each lock through a per-loop accessor
backed by a WeakKeyDictionary so every running loop gets its own Lock and stale
loops are collected. No behavior change under the server's single loop.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: auto-switch re-review fixes (body codes, coverage, swap, alias, tracking)
Follow-ups from a second review pass over the opt-in OpenAI auto-switch feature:
1. OFF-state status codes: /v1/completions and /v1/embeddings moved the body
read ahead of the loaded-state check, so a malformed/empty body with no model
loaded returned 500 instead of the prior 503. A shared helper reads the body
defensively (an unparseable/non-dict body yields no model), and the handler
re-reads after the 503 gate to surface the original parse error exactly as
before. OFF behavior is unchanged.
2. Local-model coverage: the resolver index only scanned ./models and the active
HF cache, while the model picker also lists the legacy/default HF caches, LM
Studio dirs, and user scan folders. A request for one of those named models
silently served the loaded model instead. _build_index now scans the same
roots (Ollama's symlink-creating scanner is skipped on the request path), and
resolution is offloaded with asyncio.to_thread so the wider scan never blocks
the event loop.
3. Swap vs in-flight stream: a cross-model swap killed the llama-server while
another client was still streaming from it. The hook now tracks how many
requests are streaming on the loaded model (in-flight minus those still inside
the hook) and returns 409 instead of swapping while one is active. Concurrent
same-model requests never reach this path, so they are unaffected.
4. Idle-unload + alias: after idle-unload freed the model, an unknown/alias name
resolved to nothing and 503'd, though it served the active model before the
TTL. Idle-unload now remembers the freed id and an alias request reloads it
(only an already-local model, so no remote download), cleared once a model is
loaded again.
5. In-flight tracking: the keep-warm middleware tracked in-flight only while the
feature was on, so a stream started while off could be unloaded if idle-unload
was enabled mid-stream. It now tracks on every inference path; counting is
cheap and invisible to clients.
Tests added for each.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Remove stray async_task_outputs files committed by mistake
* Studio: auto-switch review round 3 (revert swap guard, hardening)
Addressing a third review pass:
- Revert the cross-model swap guard. It counted keep-warm in-flight (which
includes external-provider calls that never touch the local model) and so
could 409 a local swap spuriously, and it still left a same-model request able
to start streaming on the model a concurrent swap was unloading. A correct fix
needs a request-lifetime reader/writer barrier; a partial guard was worse than
the honest single-slot behavior, so concurrent different-model use is back to
being serialized (documented), like llama-swap's single slot.
- Non-string request model (e.g. {"model": 123} on a raw-body endpoint) is now
treated as absent, so it falls through instead of raising in the membership
checks once an idle-unload stash exists.
- Idle-unload now stashes and replays the freed quant: an alias reload restores
the exact (id, variant) that was freed rather than the largest local quant.
- Anthropic /v1/messages validates max_tokens before the auto-switch hook, so a
request that 400s never triggers a model load.
- Keep-warm tracks a pending count for requests waiting on the unload gate, so
the idle loop cannot unload the model out from under a request that is blocked
on the gate but not yet counted as in-flight.
- The idle-unload task is awaited after cancel on shutdown to avoid pending-task
warnings.
- The resolver's HF cache scan is None-safe and logs at debug instead of letting
a bad root abort the whole index build.
- upsert_app_setting_map_entry rolls back explicitly on error.
Tests updated/added for each.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: keep saved idle-unload seconds when auto-switch is toggled off
* Studio: auto-switch hardening (thread-safe lock maps, body validation)
Defensive fixes from review:
- Guard the per-loop WeakKeyDictionary get-or-create for both the unload gate and
the auto-switch lock with a threading lock, since WeakKeyDictionary mutation is
not thread-safe when two event loops run on different threads.
- Build the resolver index under the cache lock so concurrent callers with an
expired cache don't all run the multi-dir scan at once.
- /v1/completions and /v1/embeddings return a clean 400 for a valid JSON body
that is not an object (e.g. a list), instead of a 500 from body.get(...).
- The keep-warm middleware only tracks POST requests (inference is always POST),
so CORS preflight (OPTIONS) is not counted, and tolerates a None path.
Tests added for the list-body 400 and the non-POST skip.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: auto-switch review round 4 (local-path load, swap guard, idle fixes)
From a 10-reviewer pass:
- HF-cache entries now load by a concrete local path, not the bare repo id. The
resolver records a load_path (the snapshot dir for a models--* cache repo, the
file/dir otherwise) so /load takes the local branch and can never trigger a
download to satisfy a partial cache. The advertised loader_id (repo id) is kept
as the launch-override key. resolve_local_gguf now returns
(load_path, variant, loader_id).
- Re-add a single-slot swap guard: a cross-model swap returns 409 model_switch_busy
while another inference request is active rather than killing its stream (the
caller is excluded from the count), and holds the keep-warm gate across the load
so no new inference starts mid-swap. Concurrent same-model requests never reach
this path. A residual spurious 409 is possible while a concurrent or external-
provider request is active; that is the documented single-slot tradeoff.
- Idle keep-warm tracks (model_identifier, hf_variant): reloading the same repo at
a different quant counts as a fresh model, so it is not unloaded before one TTL.
- Track Studio's own /api/inference/generate/stream so the idle loop can't unload
the model mid-stream on that route.
- A successful manual /load clears the idle-unload reload stash synchronously, not
only on the next idle poll.
Also merged origin/main (the branch had fallen behind, which would have reverted
unrelated files on merge). Tests added/updated for each.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: auto-switch review round 5 (concurrency, identity, load gate)
From a 10-reviewer pass (9 request-changes, 1 approve):
- Concurrent same-target requests load once instead of each returning 409. The
count-based busy guard could not tell "another request wants the same model"
(safe, load once) from "another request is using the loaded model" (refuse).
Track in-flight auto-switch requests per (target, variant) and subtract
same-target waiters from the busy count; a cross-model swap still 409s while a
genuinely different request is active.
- Fix the identity confusion introduced when round 4 began loading by concrete
local path: the backend identifier became a filesystem path. Record the
advertised repo id on the backend after an auto-switch load and use it so
(a) a model loaded manually by repo id is recognized as already serving
(no spurious reswap/409), (b) /v1/models reports the repo id, never a host
path or a duplicate, and (c) the idle-unload stash keeps the override keyed by
the repo id, so an alias reload after TTL keeps the user's saved launch flags.
- Gate the manual /load route with the keep-warm lifecycle gate so idle
auto-unload can't unload a model mid-load. load_model now wraps _load_model_impl
in the gate; auto-switch calls _load_model_impl directly since it already holds
the gate.
- Restore default-off parity on Anthropic /v1/messages: an unloaded backend with
auto-switch disabled 503s before the max_tokens 400 check, as it did pre-feature.
When the feature is on, request-shape validation still runs before any load.
Tests added for each; full backend suite diff vs baseline is unchanged.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: auto-switch review round 6 (concurrency ordering, leaks, unload gate)
From a second 10-reviewer pass (8 request-changes, 2 approve):
- Same-target concurrency: register a waiter by the raw requested model before
the (slow) resolve, and exclude pending requests from the swap busy count. The
middleware counts a concurrent same-model request as in-flight before it
resolves and joins the resolved-target waiter map, so the prior fix could still
409 it. The guard now subtracts max(same resolved-target, same raw-request)
waiters and ignores pending (a pending request is blocked in the middleware,
not generating, so a swap can't interrupt it).
- External-provider requests no longer block a local swap. The keep-warm
middleware counts every inference-path POST, but external-provider chat returns
before the auto-switch hook and never touches the local GGUF. The chat handler
now untracks itself before proxying, so its in-flight stream can't trip
model_switch_busy on a concurrent local auto-switch. The middleware skips its
own end-decrement for an untracked request.
- Manual /unload is gated like load and idle-unload: it holds the lifecycle gate
and returns 409 rather than tearing down llama-server while an inference request
is in flight.
- Response model id no longer leaks the load path. /v1/models already advertised
the repo id; chat, completions, embeddings, Anthropic messages, and audio
response bodies now use the same _llama_public_model_id helper instead of the
concrete on-disk model_identifier.
- Chat completions validates the non-system-message requirement before the
auto-switch hook (as /responses and /messages already do), so an invalid
request can't swap the resident model before returning 400.
Tests added for each; full backend suite diff vs baseline is unchanged.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: auto-switch review round 7 (teardown policy, Unsloth-active swap, training)
From a third 10-reviewer pass (9 request-changes, 1 approve), all on the same
asymmetric-teardown theme. Resolved per the intended policy that only automatic
paths defer to an active stream; deliberate user actions stay interrupting:
- Revert the manual /unload in-flight guard added last round. A manual /load or
/unload is a deliberate action and tears down immediately, as before; only the
automatic idle-unload loop and auto-switch defer to an active request. This
removes the asymmetry the reviewers flagged (manual /load, the /unload Unsloth
branch, and the opposite-backend swaps inside _load_model_impl) by not
extending the guard to deliberate paths, rather than spreading it.
- Auto-switch now refuses a swap whenever another inference request is in flight,
not only when a GGUF is already loaded. _load_model_impl also unloads an active
Unsloth/transformers backend before loading a GGUF, so the busy guard must cover
that case too; otherwise an Unsloth stream could be killed by an auto-switch.
- Refuse API-initiated training while inference is active. When Studio is driven
as an inference API (sk-unsloth key auth), POST /api/training/start returns 409
if a request is in flight, since training frees VRAM by unloading the chat
model and would kill the stream. The Studio UI (session auth) still starts
training and coexists/frees VRAM as before. A mixed UI+API session is not yet
special-cased. Adds auth.authentication.authenticated_via_api_key.
Tests added/updated for each; full backend suite diff vs baseline is unchanged.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: add UNSLOTH_MODEL_IDLE_TTL env override for idle-unload
Borrowed from PR 6517: a startup env var that sets the idle-unload TTL without
the settings UI. Unlike the stored setting (gated on auto-switch), the env value
is a standalone default that enables idle-unload even with auto-switch off, for
headless/container deploys. An explicit UI/API value still overrides it and stays
gated. The settings GET reflects the env default when nothing is stored.
* Studio: auto-switch fixes from review (paths, embeddings input, env idle reload)
- /v1/models advertises a client-facing alias instead of a filesystem path:
the ./models and LM Studio scanners report the on-disk path as the model id,
so the index now prefers model_id/display_name as the advertised/override id
and keeps the concrete path internal as load_path, still resolvable by path.
- /v1/embeddings validates input before auto-switch: a request with a model but
no input now 400s before the hook (like chat/responses/messages), so an
invalid embeddings request cannot unload or swap the resident model.
- Standalone UNSLOTH_MODEL_IDLE_TTL reloads the freed model: the hook now runs
when auto-switch or idle-unload is active, and with auto-switch off it skips
the resolver and only restores the idle-unloaded model, so the first idle
timeout no longer leaves later /v1 requests with nothing loaded.
- Do not resurrect a stale GGUF over an active Unsloth model: the reload-stash
path bails when a non-GGUF backend is loaded, so an unknown /v1 name cannot
tear down a live Transformers/Unsloth model.
- Defensive HF cache scan: each cache root's resolve/dedup is wrapped so a
missing or malformed root skips that root rather than aborting the index.
- Single-model retrieve checks the id is a string before lowercasing.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: fix automatic-load asymmetry, audio reload, preview, idle timer
The standalone UNSLOTH_MODEL_IDLE_TTL reload is a second automatic-load
trigger, but several validate-before-switch guards and reload hooks only
checked the auto-switch toggle. Add a shared _automatic_model_load_may_run()
(auto-switch on, or idle TTL > 0) and route every guard through it.
- /v1/completions validates prompt before any automatic load (it was the one
model-bearing route with no pre-check).
- /v1/chat/completions and /v1/embeddings pre-checks gate on the shared
predicate so a standalone idle TTL cannot reload then reject.
- /v1/messages no longer 503s before the reload hook can restore an idle-freed
model when auto-switch is off.
- Raw completions/embeddings with no model field pass a non-empty sentinel so
the idle-stash reload runs, restoring the legacy "omit model, use loaded" path.
- /api/inference/audio/generate gains the reload hook (after message validation)
so an idle-freed audio GGUF is restored.
- Public preview opts out of auto-switch via a request-scope flag, so a caller's
model field cannot swap away from the pinned checkpoint; preview chat streams
are now matched by _is_inference_path so idle-unload cannot kill them.
- Keep-warm no longer stamps activity on request start, and external-provider
untracking decrements without restamping, so periodic external traffic can no
longer keep the local GGUF warm forever.
Merges origin/main (the branch had fallen behind, which also brought in the
preview route the review flagged).
* Studio: surface model auto-switch in the API tab and demo it in examples
The OpenAI auto-switch toggle previously lived only in Settings -> General.
Add the same toggle to the API tab's usage-examples panel (it shares the
settings cache), and make the examples reflect it: when on, the Python
examples append a second call naming a different downloaded GGUF (so the
model field visibly selects which model serves), and the curl examples gain
a one-line note. Reuses the existing settings API client and i18n keys.
* Studio: harden OpenAI auto-switch reload-only path and Anthropic tool validation
- Omitted-model raw-body requests pass a reload-only sentinel so the idle-stash
reload still restores an idle-freed model, but the resolver never matches a
downloaded GGUF literally named "default".
- Reject malformed Anthropic client tools before _maybe_auto_switch_model so an
invalid request can no longer evict the loaded model.
* Studio: extend auto-switch reload-only and tool validation to schema endpoints
- Schema-backed endpoints (chat completions, responses, count_tokens, messages,
audio) defaulted an omitted model to "default" and passed it to the switch
hook, so a downloaded GGUF named "default" could be swapped to. Route the hook
through a helper that switches only on an explicitly set model, else reload-only.
- Propagate the explicit-set status when building the chat request from a
Responses request, so the non-streaming chat re-check stays reload-only too.
- Validate Responses function tools before the switch hook so a malformed tool
returns 400 without evicting the loaded model.
* Studio: serialize auto-switch swaps across event loops with a process-wide gate
The auto-switch lock is a per-event-loop asyncio.Lock, so two /v1 swaps on
different loops in one process could both pass it and race the single model slot
(the backend and _load_model_impl are process-wide). Add a process-wide
threading gate around the swap, acquired off the loop so a cross-loop wait never
blocks it, layered with the existing per-loop lock. Add a cross-loop test that
fails without the gate (two slow loads overlap) and passes with it.
* Studio: make the auto-switch swap gate wait cancellation-safe
_acquire_swap_gate awaited asyncio.to_thread(lock.acquire) when another loop held
the process-wide gate. to_thread cancellation doesn't stop the worker thread, so a
/v1 request cancelled mid-wait (client disconnect during a cross-loop swap) would
have its thread acquire the gate after the fact, while the finally that releases it
never runs -- permanently deadlocking later auto-switch swaps.
Poll a non-blocking acquire off a short asyncio.sleep instead: it still keeps the
wait off the loop and serializes across loops, but a cancel now lands during the
sleep, when the gate is not held, so nothing leaks. Add a test that deadlocks the
to_thread variant (it times out) and passes with the poll.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: validate modality and tool-confirmation before auto-switch
Two more request shapes could load a named GGUF and only then 400, evicting the
resident model:
- An image request naming a different text-only GGUF. The switch hook now takes
require_vision and rejects a swap to a non-vision target before loading it; a
GGUF's vision capability is its companion mmproj, knowable without a load, and
matches the post-load guard. Only the resolver branch is checked, never the
reload-stash restore.
- confirm_tool_calls=true with stream=false and local tools. /v1/chat/completions
now rejects that shape before the hook, mirroring the local tool path's
bypass_permissions exemption and intent signal.
The vision probe threads the ambient HF token to keep the capability-probe
invariant. Reload-only and idle-reload paths are unaffected.
* Studio: extend validate-before-switch and make the lifecycle gate process-wide
- /v1/messages/count_tokens now rejects malformed client tools before the switch
hook, like /messages (shared _validate_anthropic_client_tools helper), so a
count request can't evict the loaded model.
- /v1/chat/completions rejects a malformed tool_choice forcing object (a
{"type":"function","function":{}} with no name) before the switch hook.
- The inference lifecycle gate that blocks new inference during a swap is now
process-wide (a poll-acquired threading lock, cancellation-safe), not a
per-loop asyncio lock, so a request on another event loop can't start inference
while a swap tears the single backend down.
- Usage examples no longer hard-code a switch-demo repo most users lack; the
model is an explicit placeholder the user replaces.
* Studio: extend the auto-switch modality guard to /v1/responses and /v1/messages
The pre-load vision check that guards /v1/chat/completions now also runs on
/v1/responses and /v1/messages, so an image request naming a text-only GGUF is
rejected before the swap and never evicts the resident vision model. Run the
vision capability probe off the event loop. Make the /v1/models retrieve
loaded fast-path case-insensitive, and never advertise a host path from the
resolver. Remove the dead list_switch_eligible_ids helper, superseded by the
/v1/models catalog.
* Studio: filter /v1/models to GGUF, per-loop catalog lock, reject system-only Responses
Address review findings on the auto-switch path:
- /v1/models advertises only GGUF models the API can actually switch to; a
safetensors/LoRA entry would be selectable but never loadable via llama.cpp.
- The /v1/models catalog cache uses a per-loop lock (like the auto-switch path)
so a second event loop awaiting it can't hang in a multi-loop process.
- /v1/responses rejects system/developer-only input before the switch, mirroring
chat, so an invalid request can't evict the resident model.
- _build_index guards each scan source on its own so one bad root drops only
that source; the vision probe logs a real detection failure instead of
swallowing it.
* Studio: list cached GGUFs in /v1/models by inspecting files, not model_format
The HF-cache scanner leaves model_format unset for GGUF snapshots, so the
previous model_format == "gguf" filter dropped every downloaded HF-cache GGUF
from /v1/models and the retrieve fallback. Decide GGUF-ness from the on-disk
files via the resolver (info_has_local_gguf) instead, run off the event loop, so
the catalog advertises exactly what /v1 can serve.
* Studio: fix /v1/messages/count_tokens route binding plus auto-switch review fixes
The @router.post decorator for /messages/count_tokens had been separated from
anthropic_count_tokens by the _validate_anthropic_client_tools helper, so the
route bound to the validator and dropped its auth dependency. Move the decorator
back onto the handler. Add route-binding tests asserting each /v1 endpoint maps
to its handler with the auth dependency, so a decorator/handler split is caught
at the route level (the direct-call tests missed it).
Also from review:
- update_openai_auto_switch writes both settings keys in one transaction so a PUT
can't leave one updated and the other stale (drop the now-unused single setters).
- max_seq_length override rejects 0 at the boundary (ge=1) instead of accepting
then silently dropping it.
- Document that embeddings auto-switch is best-effort: GGUF pooling has no cheap
pre-load probe like vision's mmproj, so a guard would false-reject GGUF embedders.
- Add a positive idle-unload test (loop frees the model and stashes it for reload).
* Studio: validate Responses tool_choice + Anthropic mixed tools before switch, filter Ollama from catalog
More auto-switch review findings:
- /v1/responses rejects a forcing-function tool_choice with no name before the
switch, mirroring chat, so a malformed request can't evict the resident model.
- /v1/messages rejects mixing Anthropic server tools with custom client tools
before the switch (the check depends only on the payload, so it moves up cleanly).
- /v1/models no longer advertises Ollama-link models: info_has_local_gguf excludes
.studio_links / ollama_links entries, which the resolver skips and can't switch
to, so an advertised id never silently falls through.
* Studio: guard chat audio input before switch; surface env-backed idle unload in settings UI
A chat request carrying audio_base64 rides the same companion mmproj
projector as a vision request, so a text-only target cannot serve it
either. Flag require_vision for audio input as well so the multimodal
probe runs before the switch and a rejected request never evicts the
working model. Generalize the reject message to cover image and audio.
The settings response now reports idle_unload_active (effective TTL > 0)
so the UI can distinguish idle-unload that is active via the
UNSLOTH_MODEL_IDLE_TTL env var from the case where it needs the toggle
enabled.
* Studio: harden auto-switch eviction guards (count_tokens vision, TTS reload-only, mmproj/stash)
Four eviction/correctness fixes on the opt-in /v1 auto-switch path:
- /v1/messages/count_tokens now carries the same require_vision guard as
/messages, so an image count naming a text-only GGUF can't evict a loaded
vision model for a swap that can't serve the request.
- /audio/generate is now reload-only. A local GGUF's audio-input capability
is not a cheap pre-load probe (the companion mmproj signal can't tell an
audio projector from a vision one, and codec TTS ships no projector), so
resolving the client model could load a text/vision-only target and evict
the working audio model before the audio check fails. Only the idle-stash
restore runs here; switching TTS models is an explicit /load.
- The resolver no longer treats a standalone mmproj .gguf as a servable
model. _scan_models_dir's standalone-file pass does not filter mmproj the
way its directory scan does, so /v1/models could advertise a projector and
a switch could load it over the real weights.
- A non-GGUF (Transformers/Unsloth) load and a deliberate /unload now clear
the idle reload stash, so a manual load/unload is never superseded by a
stale idle-freed GGUF that the next /v1 request resurrects.
* Studio: report advertised repo id consistently after an auto-switch
Two model-id reporting fixes so an auto-switched cached HF GGUF is named by
its repo id everywhere, not its snapshot path:
- Streamed /v1/responses envelopes now derive the model id from
_llama_public_model_id (which prefers _openai_advertised_id) instead of the
raw model_identifier. After an auto-switch the identifier is the snapshot
path while the repo id lives in _openai_advertised_id, so the stream used to
report a snapshot basename while /v1/models, chat completions, and
non-streaming Responses all reported the repo id.
- When an advertised alias already resolves to the loaded model (a model
loaded by local path, requested by its repo or LM Studio id), the
already-serving early return now records the alias as the advertised id, so
/v1/models and responses report the alias and mark it loaded instead of the
path-derived basename. Resolver branch only; safe lock-free because an
in-flight request blocks any concurrent swap via the single-slot busy guard.
* Studio: validate request shapes before auto-switch (prompt/input/audio/mcp confirm)
Four more validate-before-switch guards so a deterministic client error never
evicts the resident model on the opt-in /v1 auto-switch path:
- /v1/completions rejects an object/number prompt (only a string or array is
valid) before the switch, instead of loading the named GGUF and letting
llama-server reject the shape afterward.
- /v1/embeddings rejects an object/number input the same way.
- Chat rejects an oversized audio_base64 upload (413) before the switch. The
size cap is a cheap, target-independent length check; the decode itself
stays post-switch to avoid decoding a valid upload twice.
- The chat confirm-without-stream pre-switch guard now mirrors the tool loop's
actual enablement: _effective_enable_tools (honoring a CLI --enable-tools
policy) and mcp_enabled (which opens the tool loop on its own but defers to a
CLI --disable-tools policy). Previously a confirm+no-stream request with only
mcp_enabled slipped past and 400'd after the swap.
* Studio: fix model-id retrieval, streaming n>1, resolver cache TTL, keep-warm auth
Four fixes from review:
- GET /v1/models/{id} legacy raw-path fallback now maps the raw identifier to
the same public id its /v1/models entry uses. After an auto-switch load the
identifier is the snapshot path while the entry is keyed by the advertised
repo id, so a client that cached the old absolute path no longer 404s on a
model that is in fact loaded.
- stream=true with n>1 is now rejected before the switch. Only the
non-streaming GGUF path returns multiple choices, so streaming n>1 is invalid
on every local serving path; both fields are known pre-switch, so it must not
load model B only to 400 and evict model A. Non-streaming n>1 stays
post-switch where the serving path decides.
- The resolver index cache is stamped after _build_index, not with the pre-scan
timestamp. On installs with enough local models for the multi-root scan to
exceed the 5s TTL, the cache was stored already expired and every request
rebuilt it.
- The keep-warm middleware no longer stamps model activity for 401/403
responses. It runs before FastAPI auth, so unauthenticated probes used to
refresh the idle timer without touching llama.cpp; they now decrement the
in-flight count without keeping the model warm.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
* add models for /update endpoint
* add logic for identifying out of date hf models
* add endpoint for updating hf models
* add relevant field to GgufVariantDetail
* make exception handling better
* add update_available flag for cached_models, and moved /update endpoint from inference -> models
* hook up /update endpoint on the frontend
* implement update scenarios for the model picker
* fix bug where downloaded flag for an older revision was being wrongly set to false
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix import and make hf calls async
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* remove has_vision from UpdateRequest
* fix ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* clear cancel event before updating gguf variant
* set _cancel_event back if it was set initially
* add hf_token to get_paths_info
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: harden model update endpoint and update checks
- update_hf_model: pass snapshot_download local_dir (local_path is not a
valid kwarg and 500s when updating bicodec audio models)
- get_gguf_variants: wrap the remote update check so a network, rate-limit,
gated, or offline failure degrades to "no update info" instead of failing
the whole variant listing, matching list_cached_models
- add regression tests for both paths
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: HF model update detection and Update action for cached models
Surface an "Update available" cue and a managed Update action for cached
on-device models. /api/hub/update-status compares each cached main GGUF
file's local blobs against the remote main revision using set membership
across all cached revisions, so a repo that was already updated (and still
holds the old snapshot alongside the new one) is not falsely flagged.
The Update action re-downloads through the download manager so it shows in
the Downloads panel with progress and cancel. The frontend wires the Update
button into the GGUF, on-device, and model-selector cards and keeps the
quant label fully visible when the action buttons crowd the row.
Adds regression tests for the multi-revision update check.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: accept force_download kwarg in hf_xet_fallback test double
The download seam now passes force_download to the attempt callable; the _FakeAttempt mock did not accept it, failing 6 tests with TypeError. Add the keyword (default False) so the scripted-results double matches the seam.
* Fix Studio model update regressions
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address Studio update review feedback
* Address Studio update edge cases
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Share GGUF update status helper
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix GGUF update detection and cache cleanup
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix cached GGUF update badges
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: shimmyshimmer <107991372+shimmyshimmer@users.noreply.github.com>
Co-authored-by: Etherll <61019402+Etherll@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
* i18n: register Japanese language support in messages
* i18n: add Japanese locale support
* Update studio/frontend/src/i18n/locales/ja.ts
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* Update studio/frontend/src/i18n/locales/ja.ts
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* test(i18n): add ja locale to parity check
* i18n: fill remaining missing keys for Japanese locale
* i18n: fix terminal string localization in ja locale
---------
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
* fix(studio/llama_cpp): disable trust_env on the loopback health probe
_wait_for_health() polls http://127.0.0.1:<port>/health with the default
httpx trust_env=True, so an ambient HTTP(S)_PROXY in the environment is
applied to the loopback request. A proxy that returns 503 for 127.0.0.1
makes every probe fail, so the loop runs until timeout and Studio load
hangs (trust_env=False returns 200 immediately).
Pass trust_env=False so the local readiness probe never goes through a
proxy. This mirrors the existing trust_env=False handling in the sibling
llama_http / external_provider HTTP clients.
* test(offline_gguf_cache): accept trust_env kwarg in fake_get mock
_wait_for_health now calls httpx.get(..., trust_env=False); update the retry test's fake_get to accept the kwarg so it doesn't raise TypeError.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio/llama_cpp): bypass proxies for loopback clients
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio/routes): bypass proxies for llama streams
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
* Studio: quick eject from the model selector
Add a one-click eject shortcut to the loaded-model pill so users do not
have to open the picker to unload a model.
- The loaded-status indicator shows a green checkmark at rest and swaps to
a red eject icon on pill hover, with an "Eject model" tooltip. Clicking
it ejects without opening the picker.
- On Device tab now uses the placeholder "Search local models" instead of
"Search Unsloth models".
- The picker's "Eject model" button uses medium font weight.
* Studio: drop unused group/eject marker class on the eject control
* Studio: make the inline eject control valid HTML
The eject shortcut was a focusable span (role/tabIndex) nested inside the
trigger button. A button's content model forbids focusable descendants, so
make it a plain decorative span (aria-hidden, no role/tabIndex) that keeps
the mouse shortcut. Keyboard and screen-reader users eject via the picker's
"Eject model" button.
* Studio: disable the inline eject shortcut on touch devices
On touch (no hover) the red eject icon and title tooltip never reveal, so
tapping the loaded pill could unload the model with no visible affordance.
Add [@media(hover:none)]:pointer-events-none so taps fall through to the
trigger and open the picker; touch users eject from the picker instead.
---------
Co-authored-by: shimmyshimmer <shimmyshimmer@users.noreply.github.com>
Co-authored-by: Wasim Yousef Said <wasimysdev@gmail.com>
* Add whole-document context mode to RAG chat attachments
Thread-attached files are injected in full when they fit a token budget,
instead of only top-K retrieved chunks, so the model reads the entire file
for summarize/reason-over-document requests. Oversized files fall back to
top-K retrieval so the context window is never blown. KB and project
corpora are unchanged (still retrieval).
- core/rag/store.py: all_chunks_for_scope returns every completed-document
chunk for a scope, ordered document-then-index, joined with filename.
- core/rag/tool.py: whole_document_context renders the chunks as the same
<chunk> blocks + citation source-map retrieval produces, returns None
when empty or over budget.
- core/inference/tools.py: build_rag_autoinject tries whole-document first
for thread scopes, falls through to search_for_autoinject otherwise.
- core/rag/config.py: THREAD_WHOLE_DOC + WHOLE_DOC_MAX_TOKENS (env-tunable).
- tests/test_rag_whole_document.py: store ordering, whole-doc render +
budget cutoff, auto-inject whole-doc vs top-K fallback, KB never whole-doc.
* Add scanned-PDF OCR fallback to RAG ingestion
A PDF page with no extractable text layer (a scanned or image-only page)
previously ingested as empty, so image PDFs were invisible to retrieval and
whole-document context. Such pages are now rendered and transcribed by the
loaded vision model during ingestion, so they become searchable and readable
like any other page. This restores OCR for the RAG document flow without a
separate extraction pipeline.
- core/rag/parsers.py: render_pdf_pages renders whole pages (1-based) to PNG.
- core/rag/captioner.py: factor the shared vision call into _vision_complete;
add _ocr_one + ocr_pages (transcribe rendered pages, OCR_MAX_PAGES bound).
- core/rag/ingestion.py: _ocr_scanned_pages runs right after parse, replacing
text on near-empty PDF pages. No-op when OCR is off, no page is scanned, or
no vision model is loaded (degrades like figure captioning).
- core/rag/config.py: OCR_SCANNED, OCR_MIN_CHARS, OCR_MAX_PAGES, OCR_DPI,
OCR_TIMEOUT_S, OCR_MAX_TOKENS (env-tunable).
- tests/test_rag_ocr_fallback.py: page render, ocr_pages gating + cap, scanned
PDF end-to-end OCR into chunks + whole-doc, born-digital skips OCR, disabled
leaves the page empty.
* Broaden OCR prompt to figures/tables and guard against repetition runaway
The OCR prompt now asks the vision model to also transcribe text inside figures,
diagrams, charts and tables, so labels and table cells on scanned pages are
indexed rather than skipped. Verified on real documents that this does not
regress plain-text transcription.
Some vision models loop on sparse images (e.g. a title-only cover) and emit the
same line hundreds of times. _collapse_runaway caps any run of identical
consecutive lines so a pathological page cannot flood the index; legitimate
short repeats (a label appearing a few times) survive. Applied in ocr_pages.
* Restrict whole-document injection to thread attachments only
whole_document_context resolved the combined project+thread scope, so a project
chat (the frontend sends both thread_id and project_id) injected the entire
project corpus in full, contradicting the design that project and KB corpora stay
retrieval-only. A large project corpus could also push the total over budget and
drop a small thread attachment back to top-K.
Resolve the thread scope alone in whole_document_context, and in
build_rag_autoinject only enter whole-doc mode when a thread attachment is present
and no KB is selected (a KB pick is exclusive: search that corpus). Project
sources and KBs keep top-K retrieval. Adds regression tests for the mixed
project+thread payload, the budget isolation, and KB precedence.
* Address review: keep project retrieval, harden budget + OCR guards
Follow-up to the 8-reviewer pass on the whole-document + OCR work.
- Preserve project grounding in project chats. The thread-scope-only fix made
whole-doc exclusive of retrieval, so a thread attachment silently dropped the
project corpus for that turn. build_rag_autoinject now whole-docs the thread
attachment AND retrieves the project sources top-K, merged under one citation
numbering via tool.render_sources. KB selection stays exclusive.
- Budget: a NULL/zero token_count no longer bypasses the cap (length-based
fallback in _row_token_count), so a malformed huge doc can't inject in full.
- OCR runaway guard: _collapse_runaway now also caps each distinct line at a
generous total across the page (not just consecutive), bounding the
interleaved/alternating loops weak models emit; blank-line floods collapse too.
- OCR: warn when a scanned PDF exceeds OCR_MAX_PAGES (pages past the cap stay
untranscribed) instead of silently dropping them.
- Document the known limits: OCR'd pages have no PDF highlight regions; vision
models need a micro-batch >= image tokens (Gemma-family) or the server aborts.
- Tests for project-retrieval composition, NULL-token budget, and interleaved
runaway; drop the now-superseded exclude-project test.
* Add OCR toggle to RAG retrieval settings
Make scanned-PDF OCR user-controllable per upload instead of only via the
RAG_OCR_SCANNED config default. The retrieval settings panel gains an OCR
scanned pages switch (persisted in localStorage, on by default); the chosen
value is read fresh at upload time and sent with each document upload.
Backend: the three upload routes accept an optional ocr form field and pass it
through start_ingestion to _ocr_scanned_pages, which now treats None as use the
config default and an explicit bool as an override. The on/off policy lives only
in _ocr_scanned_pages now, so ocr_pages no longer re-checks the config (that
double gate would have blocked a per-upload ocr=True while the default was off).
Tests cover both override directions (force on while config off, force off while
config on).
* Add "Describe figures & charts" toggle with chart-aware captions
Surface RAG figure captioning as a user control and make it actually useful for
graphs and plots. The figure detection already clustered vector drawings and
raster images into regions and rendered them, but captioning was off by default,
had no UI, and used a thin generic prompt.
Accuracy: the caption prompt now asks for chart type, axis titles and units,
legend or series, salient trends and readable values, and table columns, while
forbidding invented numbers. The token budget is configurable (CAPTION_MAX_TOKENS)
and captions pass through the same runaway guard as OCR so a looping vision model
cannot flood the index.
Control: a per-upload caption override threads from the three upload routes through
start_ingestion and _run, with the on/off policy single-sourced in _run (caption
self-gating removed from caption_images, mirroring the OCR change) so a force-on
override works when the config default is off. The frontend adds a "Describe
figures & charts" switch in the retrieval settings, persisted in localStorage and
sent with each upload. Default on; it is a no-op without a vision model and bounded
to CAPTION_MAX_IMAGES figures per document.
Tests cover the new caption_images contract, the runaway guard on captions, the
chart-aware prompt and token budget (and that OCR keeps its own prompt and budget),
and both override directions end to end through ingestion.
* Generalize figure understanding: transcribe-first prompt + high-DPI tiling
Make figure/chart description work across any visual and any model strength, not
just a strong VLM on simple figures. Two changes, validated by a recall benchmark
on authoritative documents (ResNet/Attention papers, USDA, UN UDHR).
1. Transcribe-first caption prompt. The caption now asks the model to transcribe
every visible label verbatim (titles, axis labels and units, legends, every
box/node/arrow label, table cells, equations) and then add a one-line summary,
instead of only describing the figure. Transcription is the most model-robust
visual task, so weak models that cannot reason about a chart still recover its
labels.
2. High-DPI tiling of figure pages. Figure-bearing pages are rendered as an
overlapping grid of high-DPI tiles (plus a full-page pass for context); each
tile is transcribed, then merged and de-duplicated. This keeps small diagram
labels legible and covers every sub-figure without relying on exact region
detection, which previously missed sub-figures and small labels.
Supporting changes: figure render DPI 130 -> 200 with a clip margin so edge labels
are not lost; vision calls are deterministic (temperature 0) so transcription does
not randomly drop labels; the repetition guard now applies to captions too. New
config knobs: FIGURE_DPI, FIGURE_MARGIN_FRAC, FIGURE_TILE_ROWS/COLS, FIGURE_TILE_
OVERLAP, FIGURE_FULLPAGE, CAPTION_MAX_PAGES, larger CAPTION_MAX_TOKENS, and
CAPTION_MAX_IMAGES as a per-document tile budget.
Measured figure context recall (per-label, dense academic figures):
Qwen2.5-VL: 0.50 -> 0.83 (overall 0.81 -> 0.94)
Gemma-4-E2B (weak): ~0 with loops -> 0.83 (overall 0.91)
Born-digital text and scanned-page recall are unchanged (no regression).
parsers gains _figure_boxes (shared detection), pages_with_figures, and
render_pdf_figure_tiles; captioner gains merge_page_captions and a temperature
parameter; ingestion routes figure captioning through the tiled path.
* Fix RAG review issues: whole-doc budget pre-check, figure gating, empty re-ingest, vision auth
Whole-document context now runs a cheap token-sum pre-check (store.scope_token_estimate)
before hydrating every chunk's text, so an attachment that cannot fit the budget is
rejected without loading the whole corpus into memory. The estimate mirrors
all_chunks_for_scope's filter and the per-row token-count fallback exactly.
Ingestion skips all figure work (PDF rasterization and detection, not just the caption
call) unless a vision model is loaded, so a text-only deployment pays nothing. When OCR
is enabled, scanned/image-only pages are excluded from figure tiling since OCR already
transcribes them whole, avoiding double vision work and overlapping index entries; a
scanned figure page is still tiled when OCR is off.
start_ingestion no longer dedupes forever to a prior ingest that produced zero chunks
(e.g. a scanned PDF uploaded before a vision model was loaded): the empty record is
dropped and the content is re-ingested.
Vision OCR and caption requests now send the backend Authorization header, so they
match the chat endpoint and do not 401 under direct-stream (--api-key) mode.
Adds tests for the budget estimate, scanned-page exclusion, the vision-model gate, the
empty re-ingest path, and the auth-header passthrough.
* Trim RAG vision-ingestion comments and docstrings
Tighten the verbose multi-line docstrings and comments added across the RAG vision
ingestion work (captioner, config, parsers, ingestion, store, tool, build_rag_autoinject,
the RAG tests, and the chat-store/upload-hook frontend toggles) to one or two lines while
keeping their intent. No code changed: verified comment/docstring-only against the prior
commit, and the RAG test suite still passes.
* Fix figure-tiling exclusion and client dedupe for re-ingestable docs
Figure tiling now excludes only the pages OCR actually transcribed, not every
text-less page. _ocr_scanned_pages returns the set of pages it OCR'd, and _run passes
that to pages_with_figures as exclude_pages (replacing the ocr_on-keyed min_text_chars
heuristic). A scanned page that OCR skipped (past OCR_MAX_PAGES, or whose OCR returned
empty) is no longer dropped from captioning, so a chart on such a page still gets a
caption.
The document panel's upload dedupe no longer skips re-selecting a file whose only
matching doc completed with zero chunks. Such a doc is re-ingestable (e.g. a scan
attached before a vision model loaded), and the backend re-ingests on the same content
hash, so the client must let it reach the backend; healthy or still-indexing docs are
still skipped. The SSE complete frame's chunk count is recorded on the doc so the
check is exact.
Adds a regression test for the un-OCR'd scanned figure page and updates the
pages_with_figures test to the exclude_pages interface.
* Address review findings: whole-doc budget guard, job numChunks, dead code, upload cap
whole_document_context now treats a non-positive max_tokens as "never inject" instead
of injecting the whole corpus unbounded, so RAG_WHOLE_DOC_MAX_TOKENS=0 tightens rather
than disables the budget (the real off switch stays RAG_THREAD_WHOLE_DOC=0).
The job-status endpoint and get_job_status now expose num_chunks (joined from the
document), and the upload hook threads it through the SSE-fallback completion paths
(reconcile + poll). Previously a document that finished via the connection-cap fallback
had no chunk count client-side, so the re-ingest dedupe wrongly treated it as empty and
re-uploaded it. IndexJob/JobEvent gain the field and the untyped cast is dropped.
Removes the dead render_pdf_figures function (superseded by the tiling path), its test,
and the unused FIGURE_MARGIN_FRAC config knob.
Adds an upload size cap (RAG_MAX_UPLOAD_BYTES, default 200 MB; 413 on exceed with the
partial file cleaned up) so a pathological file can't drive unbounded parse + vision
work. render_pdf_figure_tiles clamps rows/cols to >= 1 (no ZeroDivisionError on a
misconfigured grid). Captioning progress is reported after OCR so the bar is monotonic.
sqlite connections set busy_timeout=5000 so a long figure/scan ingest holding its
connection doesn't make a concurrent ingest/read fail with "database is locked".
Adds tests for the non-positive budget, the zero-grid clamp, job-status num_chunks, and
the oversize-upload rejection.
* Extract PDF text as layout-aware Markdown via pymupdf4llm
parsers._pdf now extracts each PDF page as Markdown with pymupdf4llm.to_markdown
(page_chunks=True) instead of flat page.get_text("text"), so tables, headings and lists
keep their structure in the indexed chunks and retrieve far better (a table's cells stay
associated with their row instead of flattening into a token stream). Gated by
RAG_PDF_MARKDOWN (default on); falls back to plain PyMuPDF text when the toggle is off,
pymupdf4llm is missing, extraction fails, or a page yields no Markdown. The scanned-page
OCR and figure-tiling passes operate on rendered pixels and are unaffected; docx/html/txt
keep their existing extractors.
The preview-highlight locator already strips Markdown punctuation when building anchors;
it now also splits anchor tokens on pipes so a Markdown table row still anchors to the
raw PDF word stream.
Declares pymupdf4llm as a studio/RAG dependency (was only transitively present via the
data-designer plugin). Adds parser tests (Markdown table reaches the page text, the
plain-text fallback, the missing-lib fallback) and a locator test for table-pipe anchoring.
* Pin pymupdf4llm to 0.3.4 so the package scan does not pull onnxruntime
The lockstep pymupdf4llm 1.27.x line makes pymupdf-layout a hard dependency,
which in turn pulls onnxruntime (plus numpy/networkx/protobuf). The security-audit
pip scan-packages job resolves requirements --with-deps, so adding pymupdf4llm to
no-torch-runtime.txt and studio.txt surfaced onnxruntime's un-baselined CRITICAL
finding and flipped the hf-stack shard from pass to fail.
pymupdf4llm 0.3.x keeps pymupdf-layout behind an optional [layout] extra, so a plain
install resolves to pymupdf + tabulate only and never touches onnxruntime. 0.3.4
requires pymupdf>=1.27.1, satisfied by our pinned pymupdf==1.27.2.3, and to_markdown
(page_chunks=True) produces equivalent layout-aware Markdown on real PDFs (verified on
the Attention, ResNet and USDA documents). Production already installs these files
--no-deps, so onnxruntime was never shipped at runtime; this only fixes the scanner.
The parser test now asserts Markdown markup (heading or table pipes) rather than table
pipes specifically, since 0.3.4 emits a heading but not a pipe table on the tiny
borderless synthetic fixture; both markers are absent from the plain-text fallback.
* Fix RAG whole-doc review findings
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address RAG whole-doc review follow-ups
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address RAG review follow-up edge cases
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Reserve image budget for whole-document RAG
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: wire imatrix GGUF option and FP8/NVFP4 compressed export into the export UI
GGUF export gains an importance-matrix toggle. When enabled it auto-downloads the
upstream Unsloth imatrix for the base model (or uses a custom path), which unlocks
the IQ low-bit quants iq2_xxs, iq2_m, iq3_xxs and iq4_xs. Merged export gains an
FP8 / NVFP4 compressed-tensors precision selector that runs llm-compressor for vLLM.
Backend threads imatrix_file through routes -> orchestrator -> worker -> export_gguf
(both the local save and the hub push), and maps the new compressed format_type
values onto the fp8/nvfp4 save_method, reporting the "<dir>-<suffix>" sibling output
directory. Frontend adds the imatrix Switch on the GGUF card and a merged precision
picker on the merged card, threaded through the export runtime store.
Depends on unslothai/unsloth#6706 (save.py imatrix_file and compressed-tensors
export) and unslothai/unsloth-zoo#839 (quantize_gguf imatrix flag).
* Studio export: guard imatrix/compressed against older unsloth builds and force imatrix for IQ quants
Addresses review feedback on the export wiring:
- GGUF: pass imatrix_file only when set, so a plain no-imatrix export (e.g. Q4_K_M) no
longer fails with an unexpected-keyword error against an unsloth build that predates the
imatrix_file parameter. When imatrix is requested but unsupported, return a clear
upgrade message instead of a TypeError.
- Merged: gate FP8/NVFP4 compressed-tensors export on the installed unsloth actually
supporting it, returning a clear message rather than a cryptic save_method failure.
- Frontend: IQ quants (iq2_xxs, iq2_m, iq3_xxs, iq4_xs) are imatrix-only, so force the
imatrix on when one is selected and lock the toggle, instead of submitting an IQ quant
with no imatrix that llama.cpp would reject.
Extends the backend tests for the new capability guards and the conditional kwarg wiring.
* Studio: upload compressed merged models to the Hub without recompressing
For an FP8/NVFP4 Hub export the model is already produced locally in the "<dir>-<suffix>"
output. Uploading it directly with HfApi.upload_folder (mirroring export_base_model) avoids
re-running the expensive compressed-tensors quantization a second time inside
push_to_hub_merged, which for NVFP4 also re-runs calibration and risks OOM. Falls back to
push_to_hub_merged when there is no local compressed output to reuse.
* (feat) Add project names to studio training runs to avoid models being overwritten when doing similar training runs
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update studio/frontend/src/features/export/export-page.tsx
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* Update studio/frontend/src/features/export/export-page.tsx
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* Update studio/frontend/src/features/export/export-page.tsx
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* better project name sanitization, removed duplicated project name normalization
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* implement checkpoint scanning utilities and tests for base model inference
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Guard project_name against null and use leading important modifiers
* Fix/adjust training project names for PR #6512
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix/adjust training project names for PR #6512
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address project-name review feedback
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Show project names in training recents
* Keep GGUF export directories source-specific
---------
Co-authored-by: NZ-Linix <nz-linix@outlook.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: NZ-Linix <linus.ordowski@outlook.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
* Speed up Studio desktop startup
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address Studio startup review findings
* Keep orphaned run cleanup before readiness
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Nilay