* Studio: fix nested dropdown submenus clipped by the menu alignment nudge
The 3px alignment nudge on DropdownMenuContent used translate, which
makes the scroll container the containing block for nested
position:fixed submenu wrappers, so overflow clipped every submenu
(plus menu More and Projects, sidebar chat menus, Recents export).
Use margin for the same visual shift without a containing block.
* Portal dropdown submenus so ancestor styling can never clip them
Rendered inline, the fixed submenu wrapper lives inside the parent
menu's scroll container, so any transform there (like the 3px nudge)
re-enables overflow clipping. Portaling to body removes the submenu
from that subtree entirely, same as DropdownMenuContent.
* Studio: bulk export and import in Settings Chat Data, MCP pill off switch
- Settings -> Chat -> Data gains Export Recents and Projects (Recents or
Recents + Projects, Raw JSONL / CSV / ShareGPT, combined or per chat)
and Import chats, reusing the sidebar Recents menu actions
- Extract bulkExportConversationsByScope so the sidebar and Settings share
one implementation; expose the export and import helpers via the chat
feature index
- MCP composer pill icon now swaps to an X on hover like Search, Code and
RAG; clicking it turns MCP off without opening the server menu
- en and zh-CN locale strings added (parity check passes)
* Reveal the pill X on hover for off-switch icons regardless of active look
The X was gated on data-active, so an MCP pill with no servers enabled
(or a RAG pill without a model) closed on icon click but never showed
the affordance. Off-switch pills only render while their feature is on,
so hover now always reveals the X.
* Replace the Recents hamburger menu with an Export all chats link to Settings
Bulk export and import now live in Settings -> Chat -> Data, so the
sidebar Recents header menu is gone. Each chat's Export submenu gains
Export all chats, which opens Settings on the Chat tab.
* Studio: refine chat UI shadow, background and sidebar divider
- Give both chat composers a Gemini style drop shadow with a short
transition, and add a visible shadow in dark mode.
- Set the main chat surface to a soft off white (#fbfbfc) in light mode
so the white composer reads as a card; dark mode is unchanged.
- Remove the divider line between the side menu and the page.
* Studio: lighten chat background to #fcfcfd
* Studio: make dark mode composer shadow visible
* Studio: drop composer shadow in dark mode
* Studio: unify all light mode shadows on the chatbox shadow
* Studio: unify dark mode surface colors and make hover states visible
* Studio: fix barely visible dark mode hover in account and plus menus
* Studio: lift settings dialog off the page background in dark mode
* Studio: drop greeting periods and restyle projects page to match the chatbox
* Studio: chatbox shadow on search dialog, grey sort pill on projects
* Studio: slimmer sidebar profile row, borderless login card
* Studio: match select popups to their trigger, taller profile hover target
* Studio: bigger projects search pill with the original chatbox glow
* Studio: keep select trigger shape while open
* Studio: center and narrow the projects search pill like Gemini
* Studio: borderless export card with the chatbox shadow
* Studio: chatbox shadow for selectable pills, warmer composer, search pill tweaks
* Studio: one warm background token for every page, softer search glow
* Studio: keep off white depth when warming the page background
* Studio: distinct sidebar surface from the page background
* Studio: white sidebar on the warm page background
* Studio: soften projects search shadow
* Studio: nudge the chat zero state up 5px
* Studio: darker dark mode page background, 28.5vh welcome offset
* Studio: dark mode shadows match the chatbox geometry, borderless recipes empty state
* Studio: revert dark mode shadows, desaturate dark recipe cards
* Studio: lift dark recipe cards, no hover shadow in dark
* Studio: drop the pale hover halo on recipe cards
* Studio: lighter dark recipe text, flat dark menus, flush select popups
* Studio: page-bg shadow on model selector, flush popovers and dropdowns
* Studio: zero menu offsets so dropdowns sit flush against triggers
* Studio: one 14px radius for list menus and sidebar buttons
* Studio: 14px buttons, pill hover shapes, narrower slider inputs
* Studio: pill buttons, keep profile row rectangular
* Studio: pill Save and Delete, small gap under account menu
* Studio: skinnier dropdown popups, larger account menu gap
* Studio: dropdown popups slightly wider than their trigger
* Restore light mode sidebar separator line
* Size panel number input pills to their content
* Lighten light mode page background
* Pure white composer, nudge page background whiter
* Use inline ch width for panel number pills
* Fix panel number pills at uniform 4ch width
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* fix(studio): preserve system role in ShareGPT exports
System messages in ShareGPT conversation exports were serialized as
gpt turns, which changes the semantics of exported training data.
Map role system to from system in both the single-thread and bulk
export paths, matching the importer (sharegptToRecords), which
already maps from system back to a system role.
Extracted from #5606 by @LeoBorcherding (commit 7b277913).
* fix(studio): parse quoted newlines when importing conversation CSV
csvToRecords split the file on raw newlines before parsing quotes, so
any exported message containing a newline broke on re-import (the
record was cut mid-field and remainder lines were dropped). The module
already ships an RFC 4180 parser (parseCsv) used by the prompt and
list importers; use it for conversation CSV too.
Multi-line content, embedded quotes and commas, and CRLF files now
round-trip. Unquoted commas in hand-made CSV keep the previous
behavior (rest of line is the content). Flagged in #5606 review as
'Preserve quoted newlines when importing CSV'.
npm 12 (July 2026) stops running dependency install scripts unless they
are approved via allowScripts, and npm 11.16 already warns. Studio has
no git or remote URL deps anywhere, so script gating is the only
exposure:
- commit the allowScripts policy that npm approve-scripts writes for
@biomejs/biome and msw, plus a manual fsevents entry: the tooling
cannot match a darwin-only optional dep from Linux, but the strict
check walks the platform independent ideal tree and flags it anyway
- drop the minimum-release-age npmrc alias; npm >=11.16 flags it as an
unknown project config that stops working in npm 12
- approve bun's postinstall in the setup.sh / setup.ps1 bun bootstrap;
under npm 12 defaults npm install -g bun otherwise leaves a broken
stub and setup falls back to the slower npm install path
- fix the stale esbuild comment in studio-frontend-ci.yml: the vite 8
chain ships napi binaries with no install scripts
* fix(studio): block arbitrary external image URLs in markdown renderer
Model-emitted <img src="http://attacker.com/..."> tags were causing the
browser to issue HTTP requests to arbitrary origins, leaking the user's
IP address, User-Agent, and Referer header to any domain a prompt-injected
model could emit (tracking-pixel vector, issue #5596).
Add a urlTransform function passed to <Streamdown> that only allows:
- data: URIs (inline images, mermaid SVG, user attachments)
- blob: URIs (locally generated object URLs)
- relative paths without a scheme (same-origin assets)
All other schemes (http:, https:, ftp:, etc.) return null, causing
Streamdown to omit the <img> element entirely.
Existing iframes are already stripped by Streamdown's default sanitizer;
event-handler attributes (onerror, onload, etc.) are also stripped by
the default schema.
* fix(studio): strip control chars and block backslash URL variants
Two bypass vectors found after review:
1. Backslash-normalised URLs: \\attacker.com\pixel has no colon and does
not start with // so the earlier guards allowed it as a relative path.
Browsers normalise leading backslash pairs to // before resolving, so
the request still reaches the external origin.
2. Embedded control characters: /\n/attacker.com passes trim() unchanged,
startsWith("//") is false, and no-colon check passes it as relative.
Browsers strip ASCII controls (U+0000-U+001F, U+007F) before URL
resolution, so the value resolves to the attacker origin.
Fix: strip all ASCII control characters from the raw URL before any guard,
then block any URL whose normalized form starts with two chars from [/\\]
to cover //, \\, /\, and \/ in one regex.
* fix(studio): delegate non-image URLs to defaultUrlTransform
Returning the raw URL for non-img nodes bypassed Streamdown's built-in
link sanitization, allowing model-emitted javascript: hrefs to reach the
DOM unfiltered. Pass non-image URLs through defaultUrlTransform so the
library's own javascript:/data: sanitization stays active for links.
* fix(studio): use scheme regex instead of includes() for colon check
A colon anywhere in the URL (e.g. /api/image?id=model:v2 or
/snapshots/2026-06-04T12:00:00Z.png) was incorrectly treated as an
explicit scheme and the URL was dropped. Replace the includes(':') check
with a proper scheme regex that only matches when a valid scheme token
appears before any path separator.
* Studio: shorten safeImageUrl comments in markdown renderer
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* studio(ui): use the --primary brand token for the avatar fallback color
The fallback profile avatar hardcoded #14b789, a slightly different green
from the app's general brand color (--primary = #17b88b, used by the send
button and every other primary-colored control). Next to primary-colored UI
-- e.g. the artifact preview/code panel -- the avatar's off-brand shade looked
inconsistent ("changes color weirdly"). Point avatarBgStyle() at
var(--primary) so the avatar always renders the general brand green and
follows the theme token.
Verified live in Studio: the avatar was rgb(20,183,137) (#14b789) while
--primary resolves to rgb(23,184,139) (#17b88b); the fix unifies them. This
is the only hardcoded brand-green left in the frontend -- every other
brand-green element already uses --primary / bg-primary.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* studio(ui): add literal fallback to the avatar --primary token
UserAvatar is a reusable component; if it is ever rendered outside the theme
root (where --primary is undefined), var(--primary) alone would compute to
transparent. Use var(--primary, #17b88b) so the avatar stays branded in that
edge case. When --primary is defined (the normal case, app-wide) it always
wins, so this changes nothing in practice -- verified in a browser:
var(--primary)=rgb(23,184,139), and an undefined var correctly falls back to
the literal.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Adds a self-contained RAG stack to Studio: knowledge bases with chunked indexing, hybrid (dense + lexical) retrieval, and an automatic first-pass context inject into chat. Embeddings run through a local llama-server GGUF backend (default unsloth/bge-small-en-v1.5-GGUF) with a sentence-transformers fallback. The chat tool loop gains a search_knowledge_base tool, a per-turn re-search cap, and source citation, layered on top of the shared ToolLoopController.
* Studio: fix OpenAI- and Anthropic-compatible API spec compliance
* Studio: fix API spec-compliance gaps on passthrough and streaming paths
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: carry context_length_exceeded through the OpenAI passthrough error path
* Studio: count tool-schema tokens in the Anthropic server-tool stream, and small stream-handling guards
* Studio: guard message_delta usage against None and normalize developer role before proxying
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: honor max_completion_tokens on the external-provider proxy path
* Studio: forward llama-server cached_tokens into OpenAI prompt_tokens_details
* Studio: sanitize messages in count_tokens to match the /v1/messages prompt
* Studio: report max_tokens for truncated tool calls and guard null usage in metadata events
* Studio: drop the request-id middleware (headers aren't declared in either spec)
* Studio: include the required request_id field in Anthropic error bodies
* Studio: honor max_completion_tokens on the audio (TTS / audio-input) paths
* Studio: add the _effective_max_tokens helper and route all max-token sites through it
* Studio: align API compatibility edge cases
* Studio: clarify multi-choice chat support
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: clarify logprobs chat support
* Studio: opt the local chat UI into the streaming usage chunk so the context bar and tok/s repopulate
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: forward seed to llama-server, and fix Anthropic server-tool stop_reason, tool_result id correlation, and parallel-tool execution cap
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: align OpenAI chat completion spec edge cases
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: align backend API compatibility tests
* Studio: honor tool caps and internal stream usage
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: coerce nullable stream usage counts
* Studio: preserve system prompts with developer messages
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
Unify the Studio local tool-call loop (GGUF + safetensors) behind a shared ToolLoopController: ordered preface-then-tool-card rendering, duplicate-call de-looping with a forced final answer, XML-leak containment, and a parser fix that accepts closed <function=...> calls followed by trailing prose. Includes backend tests for the controller, strict parser, and GGUF route cursor reset.
* Studio: make Helper LLM startup pre-cache opt in
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: fix recipe dataset preview
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Adds the Studio Hub and download manager: browse Hugging Face models and datasets, download GGUF and safetensors with live progress and cancellation, and manage on-device inventory. The Hub does not require a GPU, so it is available on chat-only hosts.
CI: all substantive checks pass, including the three Core jobs after unsloth-zoo#736. The two red checks are non-code flakes, a transient npm-registry DNS resolution failure in the package scan and one quantized vision-model output assertion whose sibling shards passed.
Trim and tighten code comments across studio/frontend TS/JS. Comment-only: every changed file verified code-identical to main via the TypeScript printer signature comparison.
* Studio: keep chat in place when composer attachments resize it
Attaching or removing a file in the chat composer could yank the whole
conversation to the bottom, and the grown composer covered the end of
the chat with no way to scroll it back into view.
Root cause: the Viewport composes refs with an identity that changes on
re-render, so React re-runs our scroll ref on unrelated renders and the
autoscroll hook treated every rebind as a fresh mount, pinning to the
bottom. On top of that the viewport reserved a fixed 160px under the
last message regardless of composer size.
- Treat same-element ref rebinds as no-ops in the autoscroll hook; only
a genuinely new viewport element pins and resets detach state
- Size the bottom spacer from the measured composer height plus a 24px
gap so the chat can always be scrolled above the composer
- On composer growth, detach from the bottom instead of auto-scrolling;
the user scrolls down to reveal the covered lines
- On composer shrink, defer the spacer shrink until it cannot clamp
scrollTop, then release it invisibly on scroll or on bottom-pinning
moments (run start, thread switch, thread load)
* Studio: release deferred composer spacer when a run owns the bottom
Sending with attachments cleared the chips after thread.runStart had
already fired, so the spacer shrink was deferred while the user sat
pinned at the bottom, leaving a permanent extra gap above the composer.
Apply shrinks immediately while a run is active or within 1s of run
start; the run-start pin owns the bottom then, so the clamp is the
intended glide. Caught by a cross-engine Playwright pass (Chromium,
Firefox, WebKit) over the pre and post builds.
* Studio: track the viewport element in state so listeners survive remounts
The deferred-shrink scroll listener was attached once against a ref, but
the keyed overlay provider remounts the viewport subtree on thread
switches, leaving the listener bound to the unmounted element. Removing
an attachment near the bottom in the new thread then left the oversized
spacer stuck until a run started. Track the viewport element in state so
the listener and the clamp math follow the new element.
Reproduced and verified with a thread-switch scenario on Chromium,
Firefox and WebKit; full matrix re-run green.
* Studio: release deferred composer spacer shrink when at the bottom (#6070)
---------
Co-authored-by: shimmyshimmer <michael@unsloth.ai>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Studio: refine tool call and reasoning trigger UI
Tool call triggers:
- Chevron fades in on hover or keyboard focus and sits next to the
label instead of being pinned to the right edge, matching the other
collapsible triggers.
- Labels wrap instead of truncating so long tool names and search
queries stay fully readable.
- Smaller chevron for a lighter look.
Reasoning trigger:
- Smaller chevron to match.
- Thinking box drops its bottom padding and raises the streaming max
height so more of the thinking text is visible.
* Studio: pointer cursors and sidebar 3-dots polish
Collapsible triggers:
- Pointer cursor on the reasoning, tool call, and tool group triggers
so they read as clickable.
Chat sidebar:
- Swap the chat row 3-dots menu to the vertical more-vertical icon.
- Pointer cursor on the chat row and its menu button.
- Chat row right padding opens up on hover (pr-4 at rest, pr-8 on
hover) so the title keeps a comfortable gap and clears the menu.
* studio: refine tool-call spinner, chevron, and reasoning spacing
- Use the lucide arc spinner for running tool calls and the app-wide
Spinner, so loading states match the rest of the UI.
- Collapse long tool-call labels to a single line with an ellipsis,
reveal the full label when the row is expanded, and fix the clipped
descenders.
- Keep the collapse chevron next to the label and add top spacing above
the reasoning trigger.
- Remove the redundant nested spinner in the web search running state.
* Studio: drop tool call group background fill
The ghost tool call group used a translucent bg-muted/10 fill that read
as a faint lighter box around every group in dark mode. Remove the fill
and rounding so the group sits flush on the chat background.
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
The composer dock backdrop was a solid block with a hard top edge, so
chat text scrolling underneath got visibly clipped. Replace it with a
gradient that fades the top 28px to transparent.
Also shorten the plus to x rotation in the composer from 300ms to 250ms,
including the reduced motion override.
* Studio: enable audio input for Gemma 4 GGUF models
Audio file upload was disabled for Gemma 4 vision+audio GGUFs (e.g.
gemma-4-12b-it-GGUF) even though their mmproj carries an audio encoder
(clip.has_audio_encoder, gemma4ua). Two causes:
- Audio-input detection only matched Gemma 3n's <audio_soft_token>;
Gemma 4 uses <|audio|>, so audio_vlm was never detected.
- The GGUF load/status responses hardcoded has_audio_input=False, so the
flag was dropped even when audio_vlm was detected (affected Gemma 3n
GGUFs too).
Changes:
- Recognize <|audio|> alongside <audio_soft_token> in the llama-server
token probe and the tokenizer-config pattern.
- Read clip.has_audio_encoder from the mmproj as an independent,
model-agnostic signal (read_mmproj_audio_capability).
- Emit the computed has_audio_input on the GGUF load/status responses.
- Tests for the new pattern and the mmproj reader.
* Studio: default chat model and dataset helper to Qwen3.5-4B-MTP
Switch the auto-loaded chat default and the dataset-analysis helper GGUF
from gemma-4-E2B-it to unsloth/Qwen3.5-4B-MTP-GGUF (UD-Q4_K_XL).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* studio: redesign chat composer
Reworks the new-chat composer and the compare composer into a single
rounded pill surface with a softer, lighter look.
- New welcome screen with a time-of-day sloth mascot and a lighter
heading.
- One rounded composer surface with a soft drop shadow. The input grows
inline as you type and collapses back to a single row when cleared.
- Tools and attachments live in a single plus menu; the thinking control
is a compact pill with a reasoning-effort submenu.
- Inlined glyphs for the thinking, send, and dictate controls, kept in
sync across the main and compare composers.
- Toast notifications match the composer surface: no border line, the
same drop shadow, and the same dark surface color, with a ring-less
close button.
- Dark mode: the side-menu shadow blends into the background, hovered
menu rows read clearly, and their roundness matches light mode.
- Composer styles use dedicated unsloth- prefixed classes so compare
mode keeps its own stacked layout.
* studio: sync compare-composer reasoning state and harden compare id
- Compare composer: keep "Preserve thinking" consistent with reasoning,
matching the main composer. Enabling it now turns reasoning on, and
disabling reasoning (the None option or the Thinking toggle) turns it
off, so the invalid "preserve on while thinking off" state can't occur.
- Guard crypto.randomUUID in the Compare action. It is undefined in
non-secure contexts (HTTP over a LAN IP) and would throw; fall back to
a timestamped random id, matching createNavigationNonce.
* studio: reflect pre-selected Search/Code tools when no model is loaded
The Search and Code pills only lit up when the tool was usable right now
(a model loaded and capable), so a tool turned on from the + menu showed
as off in the pill while the menu showed it on. toolsEnabled is persisted
and takes effect once a capable model loads, so the pill should reflect it.
The pills now disable only when a loaded model lacks the capability, and
otherwise reflect the selected state. Applied to the main and compare
composers.
* Studio: link MCP Servers heading to its PR and fix composer pill cursors
Make the "MCP Servers" heading in the chat Configuration sheet link to the
MCP PR, keeping the chevron as the toggle. The label and chevron are rendered
as siblings so we don't nest an <a> inside a <button>.
Also add cursor-pointer to the composer pills and the thinking pill so hovering
a clickable pill shows the hand cursor instead of the default arrow.
* Studio: refine chat composer and add compare-mode parity
- Composer expands to two rows only once the input wraps to a second line,
not on the first keystroke. Re-measure the autosize textarea on the width
swap so expanding no longer leaves a stray blank row.
- Light-mode composer shadow now matches Gemini's soft elevation.
- Plus menu: replace Canvas with a More submenu (Canvas, Compare chat, RAG)
and add Code above MCP. Active Web search/Code items use medium weight.
- Compare mode: the plus side menu, Search/Code toggles, and a Compare exit
pill now match single chat, with the thinking control on the right.
- Projects menu entries link to their tracking PR (#5725).
- Add cursor-pointer to the composer plus button.
* studio: refine composer controls and chat search shadow
- Active tool pills show an x on hover to signal click-to-disable
- Plus button rotates into an x when the tools menu opens
- Composer surface uses a 32px radius and a taller single-line height
- Even, ChatGPT-style spacing between the plus and tool pills in both the single and compare composers
- Send and mic circles resized and spaced, with the arrow centered in the circle
- Chat search box gets a borderless, soft Gemini-style shadow
* studio: size the pill hover x to match the icon it replaces
Cross-engine checks (Chromium, Firefox, WebKit) flagged the active-pill
hover x as a fixed 14px, so it popped smaller than the 19px Code icon.
Fill the glyph slot instead so the x tracks whatever icon it covers.
* studio: do not persist Kimi search/thinking mutual-exclusion in single composer
The single-chat composer flipped the other control off when toggling
search or thinking on Kimi, but without { persist: false }, so it
overwrote the user's saved preference. Match shared-composer and keep
the side effect session-only.
* studio: pointer cursor on model selector trigger and menu items
Add scoped marker classes so the model picker trigger and every
clickable element in its menu (tabs, model rows, delete, eject) show a
pointer cursor; disabled items stay not-allowed.
* studio: pass baseUrl when resolving reasoning caps in single composer
The docked composer omitted baseUrl, so a custom Gemini OpenAI-compat
gateway still advertised the native thinking ladder the backend cannot
honor. Pass selectedExternalProvider.baseUrl like the compare composer
so the resolver hides it.
* studio: grey side-menu hover, green pill hover, thinking hover x
- Plus side-menu items hover grey in light mode, not the green accent
- Thinking pill hovers green like the Search and Code pills
- The plain Thinking toggle shows an x on hover when active, matching
Search and Code; the effort dropdown trigger keeps its bulb
* studio: make the pill hover x a uniform size
The x filled the icon slot, so the wider Code chevron gave a bigger x
than Search and Compare. Pin it to a fixed 15px, centered, so every
pill's x matches.
* studio: broaden chat attachments, fix active hover color, gemini shadow
- Accept svg, source code and many text/config files as drag-and-drop
or picked attachments, matched by extension since their MIME is
unreliable; html keeps its own adapter
- Active (green) side-menu items keep their text and icon color on
hover instead of switching to the accent color
- Composer surface uses Gemini's soft centered shadow 0 0 20px rgba(0,0,0,0.04)
* studio: keep the thinking pill full height when icon-only
The inactive thinking pill has no label, so its flex row collapsed to
the icon height and the hover box looked short. Reserve one text line
(min-height: 1lh + padding) so it matches the Search and Code pills.
* studio: refine composer menu, drop overlay and greetings
- Open the MCP servers dialog directly from the composer plus menu
- Redesign the drag-and-drop affordance Gemini style, drop the badge and border, make the whole chat page a drop target
- Swap in Hugeicons for the RAG, attachment chip and new project icons
- Add time-based randomized welcome greetings, each matched to a fitting sloth
* studio: rename artifacts toggle to Canvas and make it opt-in
- Label the toggle Canvas everywhere, matching the plus menu
- Stop greying out the Canvas menu item; it toggles like the other items
- Only show the Canvas pill in the composer row once it is turned on, since it is less central than Search and Code
* studio: wire Canvas and MCP composer toggles, even out the pill row
- Open the MCP servers dialog from the menu, or toggle MCP on/off once a server is enabled
- Force MCP off when no server is enabled, so the toggle stays honest
- Show Canvas and MCP as opt-in pills that appear in the order they were toggled on
- Expand the composer and light up the pill when Canvas or MCP is on, like Search and Code
- Keep Compare directly after Code in the compare composer
- Use the same Code icon on both composers and give every pill an even icon slot
* studio: tidy composer toggle row and fix MCP enable/disable lifecycle
- Enable MCP automatically after a server is configured via the toggle flow
- Force MCP off everywhere once the last enabled server is removed
- Collapse the pill labels to icons only when more than 4 pills show, keeping Compare labelled
- Order Compare first in compare mode, before Search and Code
- Use the same Code icon and an even 19px icon slot across both composers
- Match the compare composer surface padding and send button inset to normal chat
* studio: revert compare composer padding change that cramped the input
Matching the surface padding to normal chat clipped the textarea text and
left a white strip on top. Restore the compare composer's own padding, which
gives proper top spacing. The send button inset fix stays.
* studio: center welcome greeting and soften composer scrollbar
Center the sloth and title together over the composer instead of
shifting the row left, which left the greeting sitting off to the side.
Keep the composer textarea scroll thumb faint by default and only darken
it when the thumb is hovered or dragged, so a tall draft no longer shows
a heavy dark rail.
* studio: match composer plus-menu tool gating to the pills
The new plus-menu tool entries did not carry the gating the visible pills
already enforce, so the menu and pills could disagree about a loaded
model's capabilities.
- Web search and Code menu items now disable when a loaded model lacks
the capability, while still allowing preselection with no model loaded.
- Enabling Web search from the menu on a Kimi model now flips thinking
off as a session-only change, since Kimi forbids search and thinking
together. This matches the Search pill.
- Added an Images menu item, shown only for image-generation models and
disabled until a model loads, so a short prompt has an entry point.
Applied to both the single-chat and compare composers.
* studio: round the active-pill hover x and even out pill padding
The hover x sat bare and the trailing label was tighter to the pill edge
than the leading icon, so the pill looked lopsided.
- Give the hover x a soft circular background that fills the icon slot,
matching the ChatGPT-style toggle and the icon it replaces.
- Add a little more trailing padding so the label and the leading icon
have even breathing room, and keep icon-only compact pills symmetric.
* studio: nudge the thinking bulb icon up by 0.5px
Bump the thinking lightbulb from 15px to 15.5px in the single-chat and
compare composers so it sits a touch larger next to the other controls.
* studio: drop the hover x circle on icon-only pills
When pills collapse to icon-only, the circle around the hover x is too
cramped in the small chip, so show a bare x there and keep the circle
only on the full-width labelled pills.
* studio: space the compare send button like normal chat
In compare mode the Thinking control sat right against the send button.
Match the normal composer's control spacing (gap-1.5 plus a send margin)
so Thinking has the same breathing room before send. The send button
keeps its 14px inset, so its position is unchanged.
* studio: make collapsed pill hover a circle, not a wide pill
Icon-only pills were wider than tall, so their rounded-full hover
highlight read as a fat rounded rectangle. Make the compact button a
square and center the glyph so the hover (and the x it reveals) sits in
a clean circle.
* studio: fix compare pane drops and audio picker lifetime
- Skip the page-level drop handler when the composer is hidden, so files
dropped on a compare pane are not swallowed by a hidden composer; the
shared compare composer keeps handling drops through its own dropzone.
- Build the audio file input on document.body instead of inside the plus
menu, so the menu closing on select no longer unmounts the input before
the OS picker returns and drops the file.
* studio/chat: stop projects list from white-screening on older backends
The projects list API returned data.projects directly, so a backend that
omits the field handed back undefined. useChatProjects cached that value,
then the next mount read undefined.length and crashed the whole chat page.
Default the projects and threads list APIs to an empty array and keep the
hook null-safe so a bad response can never poison the cache.
* studio/chat: align MCP dropdown with the + menu and add a chevron
Reuse the + menu surface (unsloth-plus-menu) for the MCP dropdown: rounded
corners, narrower width, neutral grey hover, and enabled rows shown as green
text with a right-aligned check instead of the emerald underlay. Add a
chevron to the MCP pill so it reads as openable, matching the Thinking pill.
* studio/chat: make MCP an opt-in pill and fix its dropdown placement
- MCP is back in the + menu as a toggle. The pill now only shows in the
composer when MCP is on, matching Canvas, instead of always sitting there.
- The dropdown follows the composer side like the + menu (opens down in the
welcome composer, up when docked) rather than always opening upward.
- Drop the dropdown caret when pills collapse so the icon is not squished.
- Stop force-syncing mcpEnabledForChat to the server count; the + menu owns it.
* studio/chat: MCP expands the composer, drop sidebar Compare, tidy scrollbars
- Toggling MCP now expands the composer and shows the tool pills, the same as
Canvas, instead of leaving the row collapsed.
- Remove the Compare item from the sidebar now that it lives in the + menu, and
point the compare tour step at the side-by-side view instead of the old button.
- Both sidebars only show their scrollbar on hover, and run settings reserves
the scrollbar gutter so the close button no longer shifts when it appears.
* studio/chat: tighten toggle gap, fix run-settings close button, collapsed Train
- Reduce the composer toggle gap by 2px (gap-1 to gap-0.5) in both composers.
- Move the run settings header out of the scroll area so the close button keeps
its position whether or not the scrollbar shows, and sits flush with the
topbar open button again instead of shifting left.
- Surface Train as an icon in the collapsed sidebar (it already has a labelled
section when expanded).
* studio/chat: tighten Thinking pill X padding, create projects inline
- The Thinking pill used px-2.5, so the hover X sat further in than the left
pills. Match their pl-2 so the X lines up.
- The + menu New project now opens a create dialog and jumps straight to the
new project, instead of routing to the projects list. Shared by both
composers via a small NewProjectDialog.
* studio/chat: soften account menu, hover scrollbars, show collapsed chevrons
- Account menu drops its border ring for the composer's soft shadow and opens
centered over its trigger.
- Settings and search reuse the hover-only scrollbar via a shared
hover-scrollbar class, matching the sidebars.
- Train and Recents keep their chevron visible while collapsed so it is clear
they can be expanded.
* studio/chat: roomier, more rounded account menu
Widen the account menu, add more left and right padding on the rows, bump the
row height and text a touch, and round the corners more, closer to the GPT
account menu.
* studio/chat: trim account menu width and nudge it up 2px
Pull the account menu in slightly on the left and right (narrower box, a touch
less row padding) and lift it 2px higher above the trigger.
* studio/settings: drop outline ring, circular close hover, pointer cursors
- Remove the settings dialog outline ring, keeping just the soft shadow.
- The close button hover is now a circle instead of a rounded rectangle.
- Every clickable control in the settings dialog uses a pointer cursor.
* studio/chat: bump MCP pill icon to 14.5px
Nudge the MCP icon up 0.5px so it sits even with the other pill glyphs.
* studio/chat: bump MCP pill icon to 15px
Nudge the MCP icon up another 0.5px.
* studio/settings: add a Settings title above the tabs
Put a Settings heading at the top of the sidebar so the tabs sit below it,
matching the Claude settings layout. Hidden on mobile where the nav is a row.
* studio/settings: rounder tab hover, bigger title, less-round search dialog
* studio/sidebar: round nav row hover boxes 2px more (10px to 12px)
* studio: drop settings dark shadow + divider, add tab left padding, tune hover roundness
* studio/model-selector: roomier padding, borderless box, rounder hover rows; settings divider light-only
* studio/search: match chat box shadow (soft light, none dark)
* studio/sidebar: borderless chat context menus, rename submenu to Projects with folder-export icon
* studio/model-selector: match light corner radius in dark, drop dark shadow, more visible dark hover
* studio/sidebar: chat context menu matches + side menu styling; relabel submenu Move to project
* studio: borderless message export menu (no dark shadow), match dark corner radius to light on export menu and settings
* studio/sidebar: open chat options menu GPT-style (down-right) and widen so Move to project fits one line
* studio/chat: message export menu uses the chatbox shadow in light mode
* studio/sidebar: narrow chat options menu slightly (w-60 to w-56)
* studio: unify all download icons to Hugeicons download-01; round profile button hover 1px more
* studio/run-settings: bump header to 16px
* studio/sidebar: trim chat options menu width slightly (w-56 to 216px)
* studio/sidebar: trim chat options menu width to w-52
* studio/profile: camera-01 Hugeicons glyph and chatbox shadow on avatar button
* studio: match dark-mode corner radius to light globally (single --radius token)
* studio/recipes: borderless New Recipe menu with chatbox shadow in light, none in dark
* studio: borderless dropdowns globally, chatbox shadow in light, none in dark
* studio: extend borderless + chatbox/none shadow to select, combobox and popover overlays
* studio/mcp: nudge MCP dropdown radius to 20px so its wider box reads as round as the + menu
* studio: restore dark dropdown shadow to avoid same-color merge; greet name ~1/3 of lines; bigger sloth + more gap
* studio/train: active tab is a borderless pill (no underline), roomier padding, more tab gap and bottom spacing
* studio/chat: nudge welcome up ~5px (still vh-based) and trim sloth image to 44px
* studio/train: active tab pill is white with chatbox shadow in light, taller padding
* studio/chat: welcome offset to calc(30vh - 10px)
* studio/chat: welcome offset to 28vh (drop the -10px)
* studio/chat: tighten sloth-to-text gap by 1px (16px to 15px)
* studio/train: revert light active pill to grey fill, drop white bg + shadow
* studio: app-wide hand cursor on every clickable control (disabled excluded)
* studio/chat: welcome offset to 26vh
* studio/chat: welcome offset to 28vh
* studio/chat: harden project and thread list guards against non-array payloads
* studio/sidebar: give the profile row more height and breathing room
* studio/sidebar: trim the profile row top and bottom padding slightly
* studio/sidebar: reduce Train and Recents section label size slightly
* studio/sidebar: trim the profile row top and bottom padding a touch more
* studio/sidebar: enlarge the profile hover area top and bottom
* studio/sidebar: increase profile hover roundness by 1px
* studio/sidebar: trim the profile row top and bottom padding slightly
* studio/sidebar: trim the profile row top and bottom padding slightly
* studio/chat: cache composer line metrics so wrap detection runs once, not per keystroke
* studio/chat: restore the prior view when exiting compare opened from the + menu
* studio/tests: drive Compare from the composer + menu after it moved out of the sidebar
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio/tests: open Compare from the composer + menu in the extra UI suite too
* studio: fix chat dictation microphone access
* studio: snappier plus-to-x spin and steady composer expand gap
Speed up the composer plus icon morph from 480ms to 300ms.
Add row-gap on the expanded composer line so the space between the text
and the controls row stays the same whether the box expanded from
wrapped text or from a toggle being on. The gap sits on the line, not the
input, so the placeholder max-height clamp never crops it.
* studio: only show composer tool pills once a model is loaded
Persisted Search/Code/Canvas/MCP toggles were surfacing the composer pill row on a fresh page load before any model was selected, so an empty composer looked different from the clean just-ejected state. Gate the composerExpanded tool checks on modelLoaded so a model-less composer stays collapsed, while saved preferences still apply the moment a model loads.
* studio: hide RAG composer menu item temporarily
Hide the placeholder RAG entry from the composer plus menu in both single chat and compare until the feature is ready, and drop the now-unused DatabaseIcon import.
* studio: let composer tools pre-select before a model loads
Selecting Web search, Code, Canvas or MCP from the + menu with no model
loaded did nothing visible: the toggle turned on but the composer never
expanded, so the pill stayed hidden. Drop the model-loaded gate from the
expand check so an active tool always surfaces its pill.
Align MCP with the Search/Code pattern too: grey it out only when a loaded
model lacks tool support, so MCP stays toggleable and the pill stays
clickable before a model is loaded instead of looking disabled.
---------
Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: Daniel Han <23090290+danielhanchen@users.noreply.github.com>
The Studio login error rewrote the backend's PATH-based command into a relative Windows path (.\unsloth_studio\Scripts\unsloth.exe ...) that only resolves from inside the Studio home dir and fails with CommandNotFoundException elsewhere. Removes the Windows-only rewrite and the now-unused usePlatformStore import so the backend's unsloth studio reset-password command is shown as-is on all platforms.
Clears the safe set of Dependabot advisories.
Frontend (overrides, all transitive; npm audit now 0):
mermaid 11.14.0->11.15.0, hono 4.12.17->4.12.18, qs 6.15.1->6.15.2,
ip-address 10.1.0->10.1.1 (also clears express-rate-limit), and
brace-expansion 5.0.5->5.0.6 (scoped, the 1.1.14 line is untouched).
Desktop: tauri 2.10.3->2.11.1, pulling the runtime crates it requires
(tao, wry, tauri-runtime, tauri-build, tray-icon).
Backend (test-only): pytest <9.0 -> >=9.0.3,<10, and the pinned
pytest-rerunfailures==15.1 -> >=16.2,<17 (16.2 is the first release with
pytest 9 support). Verified pytest 9.0.3 + rerunfailures 16.3 +
pytest-json-report + pytest-xdist run reruns, fixtures, json reports and
xdist together.
Left out intentionally: transformers (stays 4.x for unsloth compat; patch
is 5.0.0rc3), sqlfluff (major 3->4), glib/rand (stack-coupled transitive).
Fixes#5572. Replaces cmdk's default fuzzy filter with a strict substring-token filter so threads that do not match the query are hidden and the empty state shows. Each item uses its unique thread id as the cmdk value, with the searchable title and preview supplied via keywords.
* Studio: persist Tauri window size and maximized state across launches
* Studio: keep window state under the app home dir, not ~/.config
* Undo an unnecessary change
* Address Gemini's feedback
* Revert to tauri-plugin-window-state implementation
* fix(Studio): restore saved window size before default layout
* Fix cross-platform window-state restore
* fix(Studio): avoid clobbering saved window size
---------
Co-authored-by: Wasim Yousef Said <wasimysdev@gmail.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
* studio: cap training dataset uploads
* studio: clean up failed dataset uploads
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: raise upload limits to 500MB
* studio: make upload limit configurable
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: stream upload routes
* studio: split recipe upload caps
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: tighten upload limit handling
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: import settings router directly
* studio: polish upload cap setting control
* studio: cap settings request bodies
* studio: stub settings route in desktop auth test
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix: toast cancel and style
* fix: align model load toast Cancel and dismiss on the right
* fix: show short cased model name in loaded toast and removed prefix org
* revert: chat load toast refactor to visual-only changes
* fix: align model load toast close button
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: guard empty toast label and dedupe toast padding CSS
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Roland Tannous <rolandtannous@gravityq.ai>
Co-authored-by: Roland Tannous <115670425+rolandtannous@users.noreply.github.com>
* feat: align project sidebar UX with ChatGPT
* feat: align project sidebar UX with ChatGPT
* feat(chat): load stored project list
* feat(chat): add project sidebar workflows
* fix: stabilize project page navigation
* fix: projects chat loading
* fix: show project chat thread
* style: sidebar project spacing and hover clipping
* style: add expandable project chat history and move-to-project submenu
* feat: polish project sidebar
* feat: persist project sandbox paths
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: only create sandbox project workspace dir
* feat: add optional project workspace deletion from delete dialog
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: stabilize chat projects CI failures
* fix: polish project chat navigation
* Studio: manage chat history with projects
Group chats into projects with a dedicated projects page and route.
Sidebar shows recents with per-row actions and a vertical more-vertical
menu, and the sidebar scrollbar stays hidden so rows never shift on
hover. Includes chat settings and composer refinements.
* Studio: projects sidebar and breadcrumb polish
Sidebar:
- Remove the Compare nav item.
- Widen the sidebar to match the projects layout.
- Replace the scroll-gated bottom fade with a static fade pinned above
the profile box, so it no longer attaches to Recents or lags the
collapse and expand animation.
Topbar breadcrumb (chat-page):
- On a project landing show "Projects" linking to the projects list.
- Inside a project chat show the project name and chat title, with the
project name linking back to that specific project page.
- Drop the divider between the model selector and the breadcrumb.
* Studio: make project workspace delete test cross-platform
test_chat_project_delete_files_removes_workspace rooted the project under
pytest tmp_path, which resolves to /private/tmp on macOS. The workspace
delete guard refuses paths under the system denylist by design, so the
test passed on Linux CI but failed on macOS.
Add a workspace_projects_home fixture that keeps tmp_path on Linux and
Windows (CI unchanged) and falls back to a home subdir only when the temp
root is on the platform denylist. Derive the workspace path from the
created project so it tracks the projects home.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: satisfy import-hoist check for new path re-exports
documents_root and project_workspaces_root are re-exported from
utils.paths but only referenced as __all__ string literals, which the
import-hoist safety net does not count as a use. It flagged the two newly
added re-exports as unused imports and failed Source lint.
Name-load both via a module-level _REEXPORTED tuple so the check sees
them used. No behaviour change; consumers still import them from
utils.paths.
* fix: avoid projects empty-state flash
* fix: batch chat search indexing
* Studio: polish chat sidebar, run settings, and search
- Use the native OS scrollbar for the chat sidebar, Run settings panel, and chat search list instead of a custom scrollbar
- Highlight the active run in the sidebar and keep chat search available during training
- Stop the training log view from replaying when navigating back to a run
- Rename the chat settings panel to Run settings and align its toggle icon and position
- Tighten heading and sidebar letter spacing and lighten the Train and Recents labels
- Match the search dialog corner style across light and dark and drop the stray border
- Make the MCP Servers section header plain text instead of a link
- Remove a stray .orig backup file
* studio/frontend: restore Compare entry point in the sidebar
The chat-projects sidebar redesign dropped the Compare nav item and moved
it to thread-sidebar.tsx, which is not imported or rendered anywhere. That
left no way for a user to start a new model comparison (enterCompare only
fired from the guided tour and the training handoff), and broke the
Compare/Recipes/Export UI smoke test that clicks [data-tour="chat-compare"].
Re-add the Compare NavItem to the New Chat / Search group, carrying
data-tour="chat-compare" and the same new-comparison navigation as before.
* studio/frontend: use Unsloth green for the fallback profile avatar
Switch the initials-avatar background from blue to #14b789 so the sidebar
and edit-profile avatar match the Unsloth brand colour.
* studio/frontend: turn project breadcrumb into a project switcher dropdown
* studio/frontend: stop project card kebab clicks from opening the project
* studio/frontend: hide project switcher outside projects
* studio/frontend: stabilize project switcher loading
* style: project switcher alignment
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: shimmyshimmer <107991372+shimmyshimmer@users.noreply.github.com>
Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local>
Co-authored-by: Roland Tannous <rolandtannous@gravityq.ai>
Co-authored-by: Roland Tannous <115670425+rolandtannous@users.noreply.github.com>
* fix: support connected models in compare mode
* fix: support connected models in compare mode
* studio/frontend: wait for compare prompt append
---------
Co-authored-by: Roland Tannous <115670425+rolandtannous@users.noreply.github.com>
Move MCP from the chat settings sidebar to a composer pill next to Artifacts. The dropdown lists keyless presets (Context7, Exa, Hugging Face), all off by default, created on first enable and deduped by URL. Add custom MCP opens the existing manage dialog. Enabled rows show a green underlay and a tick that becomes an X on hover to remove; enabling Exa turns off the built-in Web Search and shows a hover tooltip. Reconciles the per-chat MCP flag on load so already-enabled servers activate the pill. No backend changes.
The SourcesGroup measurement container renders every citation badge
off-screen with `invisible absolute` so we can measure how many fit in
two rows. Absolute descendants still contribute to the parent's
scrollable overflow region, so long source lists added hundreds of
pixels of phantom scroll space below the assistant message that the
user could scroll into.
Wrap the measurement container in an absolute, `h-0`, `overflow-hidden`
box so the off-screen pills are clipped out of the scrollable overflow
region. Measurement still works because offsetTop is read relative to
the positioned wrapper.
* Studio: keep web search/code pills off on model load if user disabled them
* Studio: avoid redundant localStorage reads when resolving tool pills on load
* style: remove dark mode upload circle
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* added remote MCP server support
* trim
* added tests
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* increased timeout
* disabling MCP chat toggle
* Fix MCP OpenAI function-name validation + cancel propagation for PR #5750
OpenAI requires function.name to match ^[a-zA-Z0-9_-]{1,64}$ before
streaming starts. The existing 64-char length check is necessary but
not sufficient: MCP servers can return tool names containing '.', '/',
spaces, etc. that would 400 the whole chat request. Validate the
composed mcp__<server_id>__<tool> name against the regex, skip + warn
on miss, and drop duplicate tool names from the same server (which
would also 400 the request as "duplicates").
Also propagate the agentic-loop cancel_event into MCP tool execution
so a /cancel POST during a long-running MCP call (e.g. GitHub MCP
search across a large repo) actually interrupts the in-flight HTTP
call instead of waiting out the 300 s timeout. The watcher polls the
threading.Event at 50 ms cadence inside the asyncio loop (matches
routes/inference.py's existing cancel-watcher cadence) and races
against the call task with asyncio.wait FIRST_COMPLETED.
Tests added:
- test_mcp_specs_skip_invalid_openai_function_names: drops bad chars
- test_mcp_specs_skip_empty_tool_name
- test_mcp_specs_drops_duplicate_names
- test_call_tool_sync_respects_pre_set_cancel_event
Also fix test_desktop_auth.py's router stub that listed every existing
router but missed mcp_servers_router, so importing main.py fails after
this PR adds it to routes/__init__.py.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* PR #5750 round 2: OAuth cleanup on delete/url-change + mcp_enabled standalone
Round 2 of cross-platform validation surfaced two more P1 findings:
1. OAuth tokens never get cleared. fastmcp keys tokens by MCP URL, not by
server row, and delete / URL change / use_oauth toggle only updated
the SQLite row. Re-registering the same URL would silently reuse the
old account's credentials. Adds clear_oauth_tokens_async() in
mcp_client.py and calls it from the delete + put route handlers when
the row had use_oauth=True and either the URL changes or OAuth is
turned off.
2. mcp_enabled=true was ignored unless the caller also sent
enable_tools=true. The frontend always sends both together so the UI
path was fine, but a direct API caller sending only mcp_enabled would
silently get no MCP tools, which contradicts the field's documented
"append tools from every enabled MCP server" behavior. Loosens the
use_tools gate in both the GGUF and safetensors paths so mcp_enabled
opens the tool loop on its own; when the caller did not also opt
into built-ins, the built-in list starts empty.
Tests added:
- test_clear_oauth_tokens_async_no_op_safe
- test_delete_server_calls_oauth_cleanup_when_oauth_was_on
- test_delete_server_skips_oauth_cleanup_when_oauth_off
- test_update_server_clears_oauth_on_url_change
- test_update_server_clears_oauth_when_oauth_disabled
26 backend MCP tests pass; full studio/backend suite 1710 passed locally.
Cross-platform CI (Linux, macOS, Windows) green on staging fork.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* PR #5750 round 3: reject null bool updates + /test surfaces 400
Round 3 of cross-platform validation:
1. PUT /api/mcp/servers/<id> would 500 with TypeError when the body
explicitly set is_enabled or use_oauth to null. Pydantic accepts
None for an Optional[bool] and _changes_from_payload then passed
None into mcp_servers_db.update_server, which int(None)d. Reject
explicit null at the validation layer with 400 instead.
2. POST /api/mcp/servers/test caught HTTPException under
"except Exception", so an invalid URL came back as HTTP 200 with
{"ok": false, "error": "400: ..."} instead of a real 400. The
create + update paths return 400 for the same input. Move
validation outside the transport try/except so it surfaces 400.
Tests added:
- test_changes_from_payload_rejects_null_is_enabled
- test_changes_from_payload_rejects_null_use_oauth
- test_test_endpoint_surfaces_url_validation_as_400
* PR #5750 round 4: hyphenated MCP tool names + empty-tool-list gate
Round 4 surfaces two more interaction bugs between the new MCP path
and existing safetensors tool plumbing:
1. OpenAI accepts ^[a-zA-Z0-9_-]{1,64}$ for function.name, and round 1
widened the MCP regex to that set, so MCP tools can now be advertised
as `mcp__srv__list-issues`. But the XML tool-call parser in
tool_call_parser.py used `\w+` (no hyphen), so the model could call
the tool but Studio could not parse the call. Same in
routes/inference.py's `_TOOL_XML_RE` stripper, which would leave
hyphenated tool-call XML in the visible content. Both regexes now
use `[\w-]+`.
2. safetensors_agentic treats `tools=[]` as "allow all" (documented
contract, exercised by test_empty_tools_list_does_not_enforce_allowlist).
When a caller sends `enable_tools=true` + `enabled_tools=[]` +
`mcp_enabled=true` and MCP discovery returns 0, the resolved tool
list is genuinely empty and built-in tools (web_search / python /
terminal) could execute via the model's emitted call. Fix at the
route gate instead of breaking the documented contract: set
`use_tools=False` when the resolved list is empty, in both GGUF and
safetensors paths. Existing callers who omit `enabled_tools` still
get ALL_TOOLS and are unaffected.
Tests added (32 total):
- test_tool_xml_parser_handles_hyphenated_function_names
- test_tool_xml_strip_handles_hyphenated_function_names
- test_safetensors_agentic_empty_allowlist_still_means_allow_all
(documents the contract round 4 preserved)
1716 passed locally; cross-platform CI on staging fork still green.
* PR #5750 round 5: GGUF allow-list + CLI policy + hyphenated params + cancel race
Round 5 of parallel-reviewer aggregation surfaced six additional
findings; five are real and fixed here:
1. Hyphenated MCP parameter names (`<parameter=issue-number>`) were
dropped by the XML parser's `\w+` regex. Extended to `[\w-]+` in
both core/inference/tool_call_parser.py and core/tool_healing.py.
The latter is GGUF's own copy of the parser/strip patterns and was
missed by round 4.
2. core/tool_healing.py's `strip_tool_call_markup` still used
`<function=\w+>` so hyphenated MCP tool-call XML leaked into the
GGUF visible content even after round 4 fixed the shared parser.
3+4. `mcp_enabled` re-opened the tool loop even when the operator
passed `unsloth run --disable-tools` (CLI policy False). Round 2's
`(_tools_on or payload.mcp_enabled)` gate ignored the raw process
policy. Now reads `state.tool_policy.get_tool_policy()` and gates
mcp_enabled on `_cli_policy is not False`. Applied to both GGUF
and safetensors paths.
5. GGUF's agentic loop called `execute_tool(tool_name, ...)` without
checking the model-emitted name against the per-request tool list,
while the safetensors loop already enforces this. Added the same
allow-list check so a model that hallucinates a filtered MCP name
or a built-in the caller opted out of returns "not enabled" instead
of executing.
Bonus P2 fixes:
- `call_tool_sync` now checks `cancel_event.is_set()` BEFORE
creating the call task, so a pre-set cancellation does not open
the HTTP transport.
- `clear_oauth_tokens_async` moved the OAuth import + construction
inside the protected try block; a fastmcp.client.auth load error
used to escape and 500 the delete / update route.
NOT fixed (verified false or out of scope):
- finding #10 "structured_content vs structuredContent": fastmcp's
CallToolResult dataclass uses snake_case (verified live against
structured-only tool result; fields are
`dict_keys(['content', 'structured_content', 'meta', 'data', 'is_error'])`).
- finding #11 "asyncio.run from running loop": call_tool_sync is
invoked from `asyncio.to_thread` worker threads which have no
event loop; asyncio.run() is safe there.
Tests added (37 total): hyphenated param names, tool_healing strip,
GGUF allow-list gate, cancel pre-set short-circuit, OAuth cleanup
constructor-error swallowing. 1721 passed locally, no regressions.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
* Studio: add Gemini provider with web_search, code_execution, prompt caching, and Nano Banana image generation
Wires Google's native Gemini API into Studio's external-provider stack
so users can pick gemini-2.5-pro / gemini-2.5-flash / gemini-2.5-flash-image
(Nano Banana) alongside the existing OpenAI / Anthropic / OpenRouter
providers. Gemini does not speak OpenAI Chat Completions on its primary
endpoint; the new `_stream_gemini` async generator translates between
the two shapes the same way `_stream_anthropic` handles the Messages API.
Backend:
- New `_stream_gemini` translator in external_provider.py. Converts
OpenAI messages -> Gemini `contents` + `systemInstruction`; maps
generationConfig (temperature / topP / topK / maxOutputTokens);
forwards `tools: [{googleSearch: {}}]` for web_search and
`{codeExecution: {}}` for code_execution; passes `cachedContent`
through for prompt caching; sets `responseModalities=[TEXT, IMAGE]`
for Nano Banana image generation.
- Translates streamed `GenerateContentResponse` SSE frames back into
OpenAI chat.completion.chunk frames (text deltas, function_call ->
tool_calls deltas, inlineData -> image_b64 tool_end envelope, usage
chunk before [DONE]).
- Registry entry switched to native base URL
`https://generativelanguage.googleapis.com/v1beta` with
`openai_compatible: False` and the `x-goog-api-key` auth header.
Model lineup curated to current 2.5 / 2.0 family + Nano Banana.
Frontend:
- Provider-capability matrix: Gemini supports temperature, top_p, top_k,
presence_penalty (matches generationConfig); min_p / repetition_penalty
hidden because the API does not accept them.
- `providerSupportsBuiltinWebSearch` / `providerSupportsBuiltinCodeExecution`
/ `providerSupportsBuiltinImageGeneration` extended for Gemini.
- Prompt caching toggle now also lit on Gemini.
Tests:
- 21 new tests in `test_gemini_provider.py` using httpx.MockTransport.
Cover request body shape conversion, URL/header wiring, web_search
forwarded as googleSearch, function-call translation both directions,
prompt caching passthrough, image generation emitting image_b64,
grounded-search citations -> tool_end, finish_reason mapping, and
vision data URL -> inlineData translation.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: forward presence_penalty to Gemini and recover function name from tool_call_id
Two follow-up fixes for the Gemini provider:
* Thread presence_penalty into _stream_gemini and set
generationConfig.presencePenalty when non-zero. The OpenAI-side
capability matrix already exposes the slider for Gemini, so the
value was being collected and silently dropped on the way out.
* When an OpenAI role=tool message omits 'name' and only carries
'tool_call_id', recover the function name from the matching
functionCall on the prior assistant turn. Gemini 400s on an empty
functionResponse name.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: surface Gemini code execution parts as code_execution tool events
The Gemini stream parser only handled text/functionCall/inlineData
parts, so when the user toggled the Code pill on a Gemini model the
sandbox output (executableCode + codeExecutionResult parts) was
dropped on the floor while adjacent text reached the UI. Reviewers
flagged this as the headline feature being silently broken.
Translate both parts into the existing code_execution tool envelope
that CodeExecutionToolUI already consumes for OpenAI / Anthropic:
* executableCode -> tool_start with kind=code_execution and the
source code under arguments.code. We mint a tool_call_id and
stash it so the matching result block can pair to it.
* codeExecutionResult -> tool_end on that id with the stdout under
result. Non-OK outcomes (OUTCOME_FAILED / OUTCOME_DEADLINE_EXCEEDED)
are prefixed onto the text so the failure is visible.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: native Gemini model catalog, function-call ids, and honest cache claim
Three follow-ups to the Gemini provider PR after the codex pass:
* list_models() now translates Gemini's native /v1beta/models
payload ({models[{name, baseModelId, displayName,
supportedGenerationMethods}]}) into the OpenAI-compatible shape
Studio expects. Without this the picker stayed empty for Gemini
and fell back to hardcoded defaults. Embedding-only models are
filtered out.
* Forward the OpenAI tool_call id into Gemini's functionCall.id
and mirror it onto functionResponse.id. Two parallel calls to
the same function name can now be paired unambiguously on the
follow-up turn.
* Drop Gemini from the prompt-caching capability set. The wire
flow requires a separate cachedContents POST first and the
boolean Studio emits today is a no-op; the toggle should not
advertise a feature it cannot apply. Leaves a pointer to the
docs for the eventual two-step orchestration.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: distinct tool_calls index per emitted Gemini function call
Codex flagged that the Gemini stream parser hardcoded
tool_calls[0].index to 0 on every emitted functionCall. OpenAI
reassemblers key tool_calls by index when joining deltas, so two
parallel function calls in one assistant turn collapsed onto a
single slot and the second call's arguments overwrote the first.
Track the running count via len(emitted_function_call_ids) - 1
and emit it as the per-call index. The dedupe guard above (skip
when fc_id already in the set) means the index is monotonic and
stable for the lifetime of the stream. Regression test asserts
[0, 1] across two parallel calls in one candidate parts list.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: surface Gemini 3.5/3.1/3 + Nano Banana 2/Pro and plumb thinking budget
`gemini-2.0-flash` / `gemini-2.0-flash-exp` were retired by Google in 2026
(`/v1beta/models/gemini-2.0-flash:streamGenerateContent` returns HTTP 404
"no longer available to new users"), and the picker had nothing past the
2.x family. Verified against the live ListModels catalog: drop the retired
ids from `default_models` + allowlist and surface the chat-capable
3.5 / 3.1 / 3 families plus the Nano Banana image trio.
Also plumb `enable_thinking` / `reasoning_effort` into Gemini's
`generationConfig.thinkingConfig`. Without this, Gemini 3.5 Flash,
gemini-pro-latest, and the 3.x previews silently spend the caller's
`max_tokens` budget on hidden "thoughts" before emitting any visible
answer -- the chat shows a truncated stub like "The capital of" and
streams stop. Mapping:
- enable_thinking=False / reasoning_effort=none -> thinkingBudget=0
(Flash tier; Pro tier coerces to a small positive budget because
the API 400s on 0 with "This model only works in thinking mode")
- minimal/low/medium/high -> 512/2048/8192/24576 budget tokens
- max/xhigh -> -1 (dynamic)
- default (neither knob set) -> thinkingConfig omitted, model decides
Frontend `getExternalReasoningCapabilities` now surfaces a
`reasoning_effort` picker for every Gemini chat id (Pro tier hides the
"none" option; image-tier ids stay knob-less). Adds 6 unit tests
covering Flash/Pro effort mapping, the off-toggle coercion on Pro,
default omission, and the nano-banana-pro-preview alias routing
through the image modalities path. 28 -> 34 tests in
`test_gemini_provider.py`, all green; full backend suite still passes
(1459/1460; the unrelated test_help_output flake is pre-existing and
not in any file this PR touches).
Live verification against generativelanguage.googleapis.com on
2026-05-24 with `_stream_gemini` directly:
text gemini-3.5-flash single PASS multi PASS
text gemini-3.1-pro-preview single PASS multi PASS
text gemini-3.1-flash-lite single PASS multi PASS
text gemini-3-pro-preview single PASS multi PASS
text gemini-3-flash-preview single PASS multi PASS
text gemini-2.5-pro single PASS multi PASS
text gemini-2.5-flash single PASS multi PASS
text gemini-2.5-flash-lite single PASS multi PASS
text gemini-flash-latest single PASS multi PASS
text gemini-flash-lite-latest single PASS multi PASS
text gemini-pro-latest single PASS multi PASS
image gemini-2.5-flash-image PASS (1082 KB png returned)
image gemini-3.1-flash-image-preview PASS (Nano Banana 2)
image gemini-3-pro-image-preview PASS (Nano Banana Pro)
tool web_search PASS
tool code_execution PASS
-> 16/16 e2e through the actual ExternalProviderClient code path.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: tighten Gemini provider after review (PR #5720)
Fixes a batch of bugs surfaced by a second-pass review on top of the
3.5/3.1/3 + Nano Banana 2/Pro additions in c6724dbd.
Backend (external_provider.py):
- Constructor normalises legacy /v1beta/openai base URLs to /v1beta so
Gemini providers saved before the native switch keep working without
a manual re-config.
- Skip thinkingConfig, googleSearch, and codeExecution on image-tier
models (-image / nano-banana). The image responseModalities path is
mutually exclusive with text-tool wiring and stale UI state would
otherwise 400 the turn.
- _PRO_THINKING_PREFIXES now includes gemini-3.5-pro and uses anchored
prefix matching (exact id or "<prefix>-...") so the image-tier
gemini-3-pro-image-preview cannot accidentally match the pro guard.
- Gemini 3 functionCall thoughtSignature is round-tripped through the
tool_calls envelope via extra_content.google.thought_signature on
emit, and replayed as a sibling of functionCall on the next request.
- finishReason swaps STOP -> tool_calls when any functionCall was
emitted on the same turn so OAI clients trigger tool execution
(matches the OpenAI Chat Completions contract).
- usageMetadata.thoughtsTokenCount is rolled into output_tokens and
surfaced on output_tokens_details.reasoning_tokens so total_tokens
reflects the full billable spend instead of dropping the hidden
reasoning slice.
Registry (providers.py):
- Drop gemini-3-pro-preview from default_models. Google shut it down
on 2026-03-09 and auto-redirects to gemini-3.1-pro-preview; we
surface the canonical id only.
- Add model_id_deny_exact = ("gemini-3-pro-preview",) so the live
ListModels fetch does not re-surface the redirect alias.
Route schema (models/inference.py):
- enable_prompt_caching widened to Optional[Union[bool, str]] so the
/v1/chat/completions caller can pass a Gemini cachedContent resource
name (e.g. cachedContents/abc123). Without this widening _stream_gemini
s string cachedContent passthrough was unreachable from the public
route (bool_parsing 422). stream_chat_completion signature mirrors.
Frontend (provider-capabilities.ts, chat-page.tsx, chat-adapter.ts):
- providerSupportsBuiltinImageGeneration now also recognises
nano-banana ids (nano-banana-pro-preview was hidden from the image
pill before).
- providerSupportsBuiltinWebSearch takes the model id so Gemini image
models hide the Search pill (mirrors the backend skip).
- providerSupportsBuiltinCodeExecution uses the same isGeminiImageModel
guard for nano-banana ids.
- GEMINI_THINKING_PRO_PREFIXES gains gemini-3.5-pro; gemini-3-pro
tightened to gemini-3-pro-preview to avoid the image-id overlap.
- Updated 3 callers of providerSupportsBuiltinWebSearch to thread the
selected model id through.
Tests (test_gemini_provider.py): 34 -> 42, all green
- test_image_models_skip_thinking_config
- test_image_models_drop_text_only_tools
- test_gemini_35_pro_recognized_as_pro_thinking
- test_legacy_openai_base_url_normalized
- test_finish_reason_swaps_to_tool_calls_when_function_call_emitted
- test_thought_signature_round_trips_into_gemini_function_call
- test_thought_signature_emitted_in_tool_call_delta
- test_usage_chunk_includes_thoughts_tokens
Verification:
- Backend pytest 1518/1519 passing (one unrelated Qwen3.5 flash-attn
test fails on main as well; nothing in this PR touches that path).
- Frontend npx tsc -b clean.
- Live e2e 16/16 against generativelanguage.googleapis.com through the
patched _stream_gemini code path (all 11 chat models single + multi
turn, all 3 image models returned image bytes, web_search and
code_execution tools both emit the expected envelope).
- Live /api/providers/models against the patched backend surfaces 16
ids (gemini-3-pro-preview correctly filtered via deny_exact).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: address second-pass review findings on Gemini (PR #5720)
Round-2 reviewer.py flagged a phantom web_search card on image
turns (12/12 reviewers), route-layer stripping of tool_calls /
tool_call_id / name, an over-narrow image-mode tool guard, and
silent safety blocks. This patch fixes all four.
Backend (external_provider.py):
- web_search_active is now derived from the outbound tools_array
(whether googleSearch was actually forwarded), not the raw
enabled_tools intent. Image-mode turns dropped the tool above so
the inbound stream no longer emits a phantom "search complete"
tool_start / tool_end on those turns.
- text_tools_allowed now uses is_image_model (covers both `-image`
/ `nano-banana` picker models AND text models that requested
`image_generation` via enabled_tools). Verified against the live
Gemini API which rejects both googleSearch and codeExecution
alongside responseModalities=["TEXT","IMAGE"] with explicit 400s
("Search as tool is not enabled for this model", "Code execution
is not enabled for this model").
- promptFeedback.blockReason is surfaced as a 400 content-filter
error chunk instead of returning an empty successful assistant
response. The streaming loop closes the response before exiting.
Route (routes/inference.py):
- _build_external_messages now propagates tool_calls (assistant),
tool_call_id, and name (tool result) through every code path
(string content, multimodal content, non-vision fallback). Without
this Gemini 3 function-call round trips lost their thoughtSignature
+ tool_call_id at the route boundary, and functionResponse.name
arrived empty on the second turn.
- Assistant messages with content=None and tool_calls populated are
preserved as a synthetic empty-string content turn so the
Gemini translator can rebuild the functionCall part.
Tests (test_gemini_provider.py): 42 -> 45, all green
- test_image_models_suppress_phantom_web_search_card
- test_image_generation_tool_drops_text_tools
- test_prompt_feedback_block_reason_surfaces_as_error
Verification:
- Backend pytest 1736 / 1736 (the two pre-existing unrelated fails
on main, test_help_output and Qwen3.5 flash-attn pin, are skipped).
- Frontend npx tsc -b clean.
- Live e2e 16/16 against generativelanguage.googleapis.com:
11 chat models single + multi turn, 3 image models returning
image bytes, web_search and code_execution both PASS.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: fix third-pass Gemini findings (PR #5720)
Round 3 review follow-ups:
Backend (studio/backend/core/inference/external_provider.py):
- Close response AND aiter_lines iterator in a finally so normal,
prompt-block, and cancellation exits all clean up (eliminates the
RuntimeWarning about aclose never being awaited).
- Pair the synthetic web_search tool_start with a tool_end on the
promptFeedback.blockReason path so the UI does not leave a stuck
"searching..." spinner after the error toast.
- Preserve native id and thoughtSignature on executableCode and
codeExecutionResult tool events under google.native_part, and pair
the tool_end on the code-exec id so multi-turn code-execution
replays do not lose Gemini-required history.
- Carry part-level thoughtSignature on text deltas via
delta.extra_content.google.thought_signature and on inline image
tool_end via google.thought_signature so Gemini 3 image editing
and tool turns round-trip the signature on the next request.
- Guess remote image_url MIME from the URL path so PNG / WebP / GIF
inputs are not silently relabeled as JPEG.
- Roll usageMetadata.toolUsePromptTokenCount into translated input
tokens and surface thoughtsTokenCount as
completion_tokens_details.reasoning_tokens in _build_usage_chunk.
- Only normalize the Google-hosted /v1beta/openai legacy base URL;
custom proxies whose paths happen to end in /openai are left
untouched.
- Forward ChatCompletionRequest.tools and tool_choice through
stream_chat_completion into _stream_gemini, translating to
tools[].functionDeclarations and toolConfig.functionCallingConfig.
Frontend:
- chat-adapter: when Gemini image-generation is enabled for the turn,
also disable Search and Code so the request, builder, and active
pills agree with what the backend actually sends (the backend
already strips text tools when image_generation is in enabled_tools).
- chat-adapter: consume OpenAI-shape delta.tool_calls chunks so
Gemini function-call deltas without text surface as tool-call parts.
- shared-composer: disable Search and Code pills while Gemini image
mode is active so the UI matches the request.
Tests (studio/backend/tests/test_gemini_provider.py): adds coverage
for proxy base-url gating, remote image MIME inference,
toolUsePromptTokenCount, reasoning_tokens propagation, prompt-block
web_search tool_end pairing, native code-exec id/thoughtSignature
metadata, inline image thoughtSignature, text-chunk extra_content,
OpenAI tools/tool_choice translation, and image-model tool drop.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: Gemini 3 thinkingLevel + image-model Search grounding (PR #5720)
Gemini 3.x migrated to a string `thinkingConfig.thinkingLevel`
(MINIMAL/LOW/MEDIUM/HIGH) and rejects `thinkingBudget`+`thinkingLevel`
in the same request. Gemini 3 also cannot turn thinking fully off, so
the lowest position is "minimal" (Flash) or "low" (Pro rejects
"minimal").
- external_provider._stream_gemini: split thinking translation by
family. Gemini 3.x (3 / 3.1 / 3.5 + gemini-pro-latest /
gemini-flash-latest / gemini-flash-lite-latest) emits
thinkingConfig.thinkingLevel; effort none/off coerces to "low" on
Pro and "minimal" on Flash. Gemini 2.5 stays on thinkingBudget.
- external_provider._stream_gemini: allow `tools: [{googleSearch: {}}]`
on the Gemini 3 image family (gemini-3-pro-image-preview,
gemini-3.1-flash-image-preview, nano-banana-pro). Google's docs
document Search grounding on these. codeExecution stays blocked
on image mode (still mutually exclusive with responseModalities).
- provider-capabilities.ts: mirror the Gemini 3 effort ladders in
resolveGeminiReasoningCapabilities (Pro: low/medium/high; Flash:
minimal/low/medium/high; 2.5 Flash keeps the off-position).
- provider-capabilities.ts: providerSupportsBuiltinWebSearch now
returns true on the documented Gemini 3 image models so the pill
is reachable; older image ids (gemini-2.5-flash-image) still hide.
Tests: splits the existing thinkingBudget cases by family (Gemini 3
checks thinkingLevel; Gemini 2.5 keeps thinkingBudget), adds positive
googleSearch coverage for Gemini 3 image models and negative
googleSearch coverage for legacy image models.
References:
- https://ai.google.dev/gemini-api/docs/thinking
- https://ai.google.dev/gemini-api/docs/gemini-3
- https://ai.google.dev/gemini-api/docs/models/gemini-3-pro-image-preview
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: attach Gemini code_execution inline images to the code card (PR #5720)
When a text Gemini turn wires codeExecution and the sandbox produces a
matplotlib plot, the inline image part ships right after the
codeExecutionResult. Previously this surfaced as a separate empty
image_generation card. Track the most recent code_execution
tool_call_id + result text and, when an inline image follows with
code_execution active, emit a second tool_end on the same id that
appends the image as a data: URI under the `__IMAGES__:` marker the
chat-adapter already understands.
Image-picker turns (`-image` / `nano-banana`) keep the standalone
image_generation envelope so Nano Banana outputs render the same way.
Tests: covers the merged code-execution card emission with no
standalone image_generation event when code_execution is the active
tool.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: fix fourth-pass Gemini findings (PR #5720)
Round 4 review follow-ups:
Backend:
- `_is_openai_compatible` + `_auth_headers` detect Gemini connections
pointed at a custom OpenAI-compatible proxy (non-Google host whose
path ends in `/openai`) and route them through the OpenAI-compat
surface with `Authorization: Bearer ...` instead of the native
`_stream_gemini` translator + `x-goog-api-key`. Google-hosted Gemini
keeps the native dispatch path it migrated to in this PR.
- `_stream_gemini` thinkingLevel handling for Gemini 3 Pro now coerces
both "minimal" and "medium" effort to "low" / "high" respectively
(Pro tier only accepts low/high per
https://ai.google.dev/gemini-api/docs/thinking).
- `providers.py` `default_models` restores the advertised
`gemini-3.5-pro` and the rolling `gemini-pro-latest` /
`gemini-flash-latest` / `gemini-flash-lite-latest` aliases that the
allowlist already admits.
Frontend:
- chat-adapter: lean on `providerSupportsBuiltinWebSearch` (which
already encodes the Gemini 3 image-model Search allowance) instead
of blanket-disabling Search whenever Gemini image mode is active.
Code execution stays blocked because Gemini image mode rejects it.
- shared-composer: mirror the same gate -- only the Code pill is
unconditionally disabled in Gemini image mode; the Search pill is
driven by `supportsBuiltinWebSearch`.
- provider-capabilities: Gemini 3 Pro reasoning levels now expose only
"low" and "high" (no Medium pill) to match the API.
Tests: covers the Gemini 3 Pro medium / minimal coercion, the custom
proxy OAI-compat dispatch + Authorization Bearer auth, and the
native-vs-proxy detection. Also closes the mocked httpx.AsyncClient
inside the test event loop so the Python 3.13 `aclose was never
awaited` warning no longer fires.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: fix fifth-pass Gemini findings (PR #5720)
Round 5 review follow-ups:
Backend:
- `_is_openai_compatible` + `_auth_headers` now treat ANY non-Google
Gemini base URL as OpenAI-compat (LiteLLM / custom OAI gateways /
OpenAI-compat vLLM routers), not just paths ending in `/openai`.
Pre-existing saved Gemini proxies on `/v1` keep working.
- Gemini 3 thinkingLevel coercion narrowed to the documented
inconsistencies: only "minimal" is coerced to "low" on Pro tier.
"medium" passes through (Gemini 3.1 Pro accepts it per
https://docs.cloud.google.com/vertex-ai/generative-ai/docs/models/gemini/3-1-pro).
- `_stream_gemini` only flips `responseModalities=[TEXT,IMAGE]` when
the selected model is image-capable. A stale
`enabled_tools=["image_generation"]` on a text model is silently
dropped instead of producing an invalid Gemini request.
- `_stream_gemini` validates the model id against
`[A-Za-z0-9._-]+` before URL interpolation so a model like
`../cachedContents/x` cannot redirect the request to an unintended
endpoint with the configured API key attached.
- Empty-text Gemini parts that still carry `thoughtSignature` emit a
content-free delta with `extra_content.google.thought_signature` so
Gemini 3 turns that end with a signature-only fragment do not lose
the replay state.
- ConnectError / ReadTimeout / generic HTTPError paths in
`_stream_gemini` now close the synthetic web_search tool_start
with a matching tool_end before the error chunk so the UI does not
leave a stuck "searching..." card on transport failure.
- `providers.py` default_models drop the non-existent
`gemini-3.5-pro` (Google launched only `gemini-3.5-flash` at
I/O 2026; Pro tier remains `gemini-3.1-pro-preview`).
- `routes/inference.py` only forwards `payload.top_k` when the caller
explicitly set it on the request (Pydantic `model_fields_set`).
Omitted top_k stays omitted, restoring the pre-PR behavior where
Gemini uses its server default.
- `ChatCompletionRequest.enable_prompt_caching` adds a `mode="before"`
validator that coerces the canonical string literals "true"/"false"
back to bool so historical opt-out callers keep working after the
field widened to `Union[bool, str]` for Gemini cache resource names.
Frontend:
- `providerSupportsBuiltinWebSearch` / Code / Image now accept the
saved connection `baseUrl` and return false for custom OAI-compat
Gemini proxies. Backend skips `_stream_gemini` for those bases, so
native tool envelopes never reach them; hiding the pills keeps the
request, builder, and UI consistent.
- `provider-capabilities.ts` Gemini 3 Pro effort ladder restores
`["low", "medium", "high"]` to match Google's documented levels.
- Call sites in `chat-page.tsx` and `chat-adapter.ts` pass through
`provider.baseUrl` so the proxy gate fires.
Tests: covers Gemini 3 Pro medium pass-through, custom proxy dispatch
on `/v1` and `/openai` bases, path-traversal model id rejection,
top_k omission when not explicit, text-model image_generation drop,
empty-text + thoughtSignature surfacing, and
enable_prompt_caching string coercion.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: fix sixth-pass Gemini findings (PR #5720)
Round 6 review follow-ups:
Frontend:
- chat-adapter `delta.tool_calls` accumulates fragments by `id` /
`index` instead of pushing a new tool-call card per chunk. The
standard OpenAI Chat Completions stream contract sends `id`/`name`
on the first chunk and partial `function.arguments` on subsequent
chunks; our previous handler parsed each fragment as a standalone
tool call. Local llama.cpp and OAI-compat providers that stream
fragments now reassemble into a single function-call part.
- chat-adapter also preserves `extra_content` on streamed tool-call
deltas so Gemini 3 `thoughtSignature` survives to the next turn.
- provider-capabilities Gemini 3 Pro restores "medium" in the
reasoning-effort ladder (Google's official Gemini API thinking
doc lists low/medium/high for Gemini 3.1 Pro; my earlier round 4
coercion was wrong).
- provider-capabilities orders `gemini-2.5-flash-lite` ahead of the
broader `gemini-2.5-flash` prefix so Flash-Lite falls into the
"no native thinking knob" branch as documented.
* Studio: round-trip Gemini tool_calls and tool results (PR #5720)
Recurring round 3-6 P1: the chat-adapter renders Gemini function-call
parts and code-execution events but `toOpenAIMessage` only serialized
text + image content, so the next turn lost the assistant
`tool_calls[]` (including Gemini 3's required
`extra_content.google.thought_signature`) and the matching
`role="tool"` result. Gemini 3 multi-turn function calling and code
execution failed validation on the second turn.
Frontend:
- types/api.ts widens OpenAIChatMessage to permit `role="tool"`,
`tool_calls`, `tool_call_id`, `name`, and `content: null`. Adds
OpenAIToolCallPart with `extra_content` for the Gemini round-trip.
- chat-adapter: new `toOpenAIMessages` expands an assistant turn with
tool-call parts into [assistant w/ tool_calls + extra_content,
role=tool result, ...]. tool result content is JSON-serialized so
the backend translator can rebuild Gemini's `functionResponse`
shape.
- chat-adapter outbound history now uses `flatMap(toOpenAIMessages)`
so each assistant tool-call round-trips through the standard OAI
shape the backend's `_stream_gemini` already understands.
* Studio: replay Gemini code_execution and image native parts on history (PR #5720)
Multi-turn Gemini history previously lost the native executableCode,
codeExecutionResult, and inlineData parts because the outbound
translator regenerated a generic functionCall for every assistant
tool_call. Stow the native dict on tool_end (frontend) and replay it
verbatim with thoughtSignature (backend) so follow-up turns preserve
the prior execution and image generation state. Skip role="tool"
fan-out for server-side builtin tools so Gemini does not 400 on a
functionResponse with no matching user-declared function.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: complete Gemini built-in tool replay round-trip (PR #5720)
Round 7 follow-up to the multi-turn native-part work. Three asymmetric
storage/consume gaps remained between the backend translator and the
chat adapter, so realistic Gemini follow-up turns degraded to generic
functionCalls instead of native history.
- Frontend collectAssistantToolCalls now drops web_search outright,
drops code_execution / image_generation when the native part is
missing, and promotes args.google to extra_content.google so the
backend native_part replay branch actually fires.
- Backend image_generation tool_end now emits google.native_part
with the inlineData (mimeType + base64) and thoughtSignature so the
follow-up image-edit turn can replay the prior image as a native
Gemini model part.
- Backend code-execution plot tool_end now stows google.native_part
with the inlineData so the merged code-exec card can round-trip
executableCode + codeExecutionResult + inlineData on the same id.
- Added regression tests for image-gen native-part replay and the
code-exec plot native_part stow.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: round 8 Gemini follow-ups (PR #5720)
- Text-part thoughtSignature: stow on the assistant message during
streaming and replay onto the last text part on the next turn so
Gemini 3 strict function-calling does not reject history.
- Function declarations: recursively strip Gemini-unsupported OpenAPI
keys (additionalProperties, $schema, $defs, strict, etc.) so OpenAI
strict tools stop 400ing as INVALID_ARGUMENT on Gemini.
- OpenAI-compat fallback: forward tools/tool_choice so custom Gemini
proxies (LiteLLM, gateways) keep function-calling.
- enable_prompt_caching: cover the Pydantic v1 legacy off/on/f/n/t/y
string set so explicit opt-outs stay opt-out (Gemini was sending
cachedContent: "off" otherwise).
- Frontend collectAssistantToolCalls / collectToolResultMessages: use
google.native_part + result presence to disambiguate provider
builtins from same-named user-declared functions.
- Added regression tests for text-signature replay and schema
sanitization.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: round 9 Gemini follow-ups (PR #5720)
Two round-9 convergent finds across the 12 reviewers:
- Server-side web_search was leaking onto the next turn as a fake
user functionCall/functionResponse. The previous heuristic (skip
builtin only when no native_part AND no result) let it through
because the synthetic tool card has a non-empty result string.
Always skip web_search by name on both serializers, accept that a
user-declared function literally named "web_search" must use a
different name.
- Assistant `extra_content` was dropped by ChatMessage validation
before _stream_gemini could replay text-part thought signatures.
Add the field to ChatMessage and forward it through
_build_external_messages so the multi-turn signature path actually
carries data.
Includes a regression test for the ChatMessage round-trip.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: round 10 Gemini follow-ups (PR #5720)
Three convergent round-10 reviewer findings closed:
- Tag synthetic provider-side builtins with `args._server_tool=True`
via a central helper that runs in every `_emit_tool_event` /
`_emit_synthetic_tool_event` path. The frontend filter now skips
on that marker instead of on the public tool name, so local
llama.cpp `web_search` and OpenAI function tools literally named
`web_search` / `code_execution` / `image_generation` round-trip
cleanly while Gemini grounding / hosted code-exec / hosted image
cards stay skipped.
- Gate Gemini image-mode (responseModalities=[TEXT,IMAGE]) on the
Images pill (enabled_tools containing `image_generation`).
Selecting an image-capable model with the pill off no longer forces
image output the UI says is disabled.
- Frontend missing-key guard now exempts custom Gemini OAI-compat
proxies (LiteLLM, gateways) the same way the backend already
does, so a saved Gemini connection on `http://localhost:4000/v1`
with no API key stops being blocked.
Existing tests updated to pass `enabled_tools=["image_generation"]`
on image-mode capture paths.
* Studio: round 11 Gemini follow-ups (PR #5720)
Four round-11 findings closed:
- Kimi _stream_kimi_web_search's local _synthetic_chunk helper now
runs through _stamp_server_tool_marker so Kimi search history is
not replayed as a fake user functionCall on the next turn (was an
asymmetric miss after the round-10 tagging work).
- OpenAI Responses path (/v1/responses for gpt-5.x) forwards
caller-supplied tools / tool_choice, translating the Chat
Completions function-tool shape into the Responses native shape.
Without this, standard OpenAI tools silently dropped on
Responses-routed traffic.
- Decoupled the Gemini image-tier model-id guards (text-tool /
thinking strip) from the Images pill flip
(responseModalities=[TEXT,IMAGE]). gemini-2.5-flash-image with
Search/Code on and the Images pill OFF no longer forwards
googleSearch + thinkingConfig (Gemini 400s on those for legacy
image ids).
- Gemini-only extra_content is now forwarded by
_build_external_messages only when provider_type=="gemini" so
Google's thought_signature does not leak into OpenAI / Mistral /
Kimi / OpenRouter request bodies as an unknown field.
Added a regression test for the image-tier strict-guard split and
extended the extra_content test to cover the non-Gemini suppression.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: round 12 Gemini follow-ups (PR #5720)
Three round-12 convergent findings closed:
- extra_content leak to custom Gemini OAI-compat proxies (8/12
reviewers). _build_external_messages now gates extra_content on
the native generativelanguage.googleapis.com host, not just
provider_type=="gemini", so LiteLLM / custom gateways routed
through /chat/completions do not get an unknown top-level field.
- OpenAI Responses function-tool round-trip (5/12 reviewers). I
added user `tools` forwarding in round 11 but did not parse the
matching response.output_item.done items of type=function_call.
The parser now translates them into Chat Completions
delta.tool_calls and the terminal chunk reports
finish_reason="tool_calls" when the model invoked a user
function.
- Image-tier model with Images pill OFF (2/12). Google's image
models default to text+image when responseModalities is omitted,
so the previous fix silently still billed image output. Force
responseModalities=["TEXT"] when the Images pill is off and the
selected model is image-capable.
Updated the two pre-existing tests that pinned the synthetic-tool
arguments shape to include the new `_server_tool: True` marker, and
added a regression test for the Responses function-call output
translation.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: round 13 Gemini/Responses follow-ups (PR #5720)
Three round-13 convergent findings closed:
- OpenAI Responses function_call indices: my round-12 translator
hardcoded every emitted tool_calls[*].index to 0, so parallel
function calls collapsed for index-keyed clients. Track and
increment function_call_index per emit (mirrors the Gemini
branch's distinct-index pattern). 10/12 reviewers flagged.
- _SERVER_SIDE_BUILTIN_TOOL_NAMES now includes web_fetch so
Anthropic-hosted web_fetch cards carry the _server_tool marker
and the frontend history serializer doesn't replay them as fake
user functions. 4 reviewers flagged.
- OpenAI Responses follow-up tool results now serialize as
Responses-shape function_call / function_call_output items keyed
by call_id, instead of Chat Completions role="tool" content.
Skips assistant tool_calls tagged with _server_tool so hosted
builtins don't round-trip as user functions. 2 reviewers flagged.
Updated the Anthropic code_execution and web_fetch test argument
pins to include the new _server_tool marker, and added two
regression tests (distinct indices on parallel function_call,
function_call_output round-trip).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: round 14 Gemini follow-ups (PR #5720)
Three round-14 findings closed:
- Remote `image_url` translation (5 reviewers convergent). Public
HTTPS image URLs can't be sent as `fileData.fileUri` -- Gemini
reserves that path for Files API URIs and YouTube. Fetch the
bytes server-side and inline them as base64 `inlineData`,
mirroring the pre-PR OpenAI-compat behaviour. YouTube URLs and
generativelanguage.googleapis.com/v1beta/files/* stay as
`fileData`.
- Nullable JSON Schema type arrays. OpenAI strict tools commonly
use `"type": ["string", "null"]`; the Gemini sanitizer now
flattens that to `"type": "string", "nullable": true` so strict
function tools stop 400ing.
- Parallel functionResponses now ride on one user content block
with multiple `functionResponse` parts, matching Google's
parallel tool docs. Consecutive `role="tool"` messages merge
into the previous user turn instead of splitting into separate
Gemini user turns.
Three regression tests added (remote URL fetch + inline, Files
API / YouTube fileData preservation, schema nullable flattening,
parallel-tool grouping).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: SSRF harden Gemini remote image fetch (PR #5720)
Round 15 convergent finding (12/12 reviewers). My round-14 fix to
download user-controlled image URLs for inlineData inlining was an
SSRF / data-exfiltration path: no scheme check, no private-host
guard, no size cap, no Content-Type validation, redirects could
bounce to internal services, and the full URL was logged.
Replace the inline fetch with `_safe_fetch_image_for_gemini`:
- Require https:// (reject http, file, data, ftp, etc).
- Resolve the hostname via socket.getaddrinfo and reject if ANY
resolved address is private / loopback / link-local / multicast /
reserved / unspecified (covers 127.0.0.0/8, 10/8, 172.16/12,
192.168/16, ::1, 169.254/16 metadata, RFC 6890).
- Block IP-literal URLs that resolve into those same ranges.
- Cap response body at 10 MB (Content-Length pre-check + streamed
byte counter).
- Require Content-Type to start with `image/`.
- Disable redirect following so a 302 to a private host can't slip
past the address check.
- Use a short 15s timeout and a tiny connection pool dedicated to
these fetches.
- Log only the host name + error class -- no full URL, no signed
querystring leak.
If the guard rejects, the image part is silently dropped (instead
of forwarding raw bytes or a fileData fallback). Files API URIs
and YouTube URLs still ride as `fileData.fileUri` unchanged.
Tests: replaced the live-fetch test with a `_safe_fetch_image_for_gemini`
monkeypatch, added four new SSRF-guard tests (non-https rejected,
loopback / private IP literals rejected, hostnames that resolve to
private IPs rejected).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: round 16 Gemini follow-ups (PR #5720)
- IP-pinned image fetch (`_safe_fetch_image_for_gemini`): reuse the
validated-once-then-pin pattern from `tools._fetch_page_text` via
`asyncio.to_thread`, so DNS rebinding between validation and the
HTTP connect cannot redirect us at a private/metadata address.
Catch malformed-bracketed IPv6 urlparse errors. Follow up to 4
redirect hops with per-hop SSRF re-validation.
- Replace contains-substring detection of Gemini Files API + YouTube
URLs with parsed scheme/host/path checks, so attacker URLs like
`https://evil.example/path/youtube.com/x.png` no longer skip the
safe-fetch path and serialize as `fileData.fileUri`.
- `_build_external_messages`: strip per-tool-call `extra_content`
for non-native-Gemini providers; the Gemini-only
`thought_signature` payload was leaking through `tool_calls[]`
into /chat/completions on OpenAI, Anthropic, and custom Gemini
OAI-compat gateways.
- `_server_tool` marker now gated on the function name being one of
the canonical builtin names (`web_search`, `web_fetch`,
`code_execution`, `image_generation`) AND the marker being set,
so a user function whose schema happens to define an
`_server_tool` field is no longer dropped. Frontend filter mirrors
the same gate, plus a backward-compat fallback for pre-PR
persisted server-tool cards (no marker) routed via name +
native_part / web-tool heuristic.
- Gemini schema sanitizer collapses `anyOf: [{X}, {"type":"null"}]`
to `{X, "nullable": true}` so Optional[X] tool args from
OpenAI/Pydantic schemas no longer 400 the Gemini request.
- Frontend tool-result serializer emits `{"result":""}` for empty
string outputs so the ChatMessage validator does not reject
`role="tool"` with empty content.
- Coerce `medium` thinkingLevel to `high` for legacy
`gemini-3-pro*` / `gemini-3-pro-preview*` (only low/high
documented; shut down 2026-03-09); 3.1+ Pro still passes through.
- Hide Gemini native thinking ladder on custom OAI-compat Gemini
gateways by routing `getExternalReasoningCapabilities` through
`isGeminiCustomOpenAICompatBase(baseUrl)`; thread baseUrl through
all four call sites.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: round 17 Gemini follow-ups (PR #5720)
- Frontend `collectAssistantToolCalls` and `collectToolResultMessages`
no longer drop unmarked `web_search` / `web_fetch` cards by name
alone: a user-defined function with one of those names must
round-trip. Pre-PR persisted `code_execution` / `image_generation`
cards still get filtered via a shape heuristic (kind/command/code/
prompt fields) instead of bare name.
- `_build_external_messages._filter_tool_calls` now drops marked
server-side builtin `tool_calls` entirely for non-native-Gemini
providers, not just their `extra_content`. An assistant turn whose
only payload was a marked builtin is dropped completely so the
receiving provider does not see an orphan tool_call.
- `_stream_anthropic` translates OpenAI top-level `tool_calls` into
Anthropic native `{type:"tool_use", id, name, input}` content
blocks, and translates `role="tool"` follow-ups into `role:"user"`
messages carrying a `tool_result` block. Anthropic's native
Messages API rejects the OpenAI shapes.
- `_safe_fetch_image_for_gemini_sync` factors URL validation through
`_safe_parse_https`, so malformed `port` access (e.g.
`https://host:bad/x.png`) and malformed redirect targets (e.g. a
302 to `https://[bad/x.png`) drop the image instead of raising mid-
request.
- `tool_choice="none"` now disables hosted builtins (Gemini
googleSearch / codeExecution and OpenAI Responses web_search /
shell / image_generation), not just user function declarations.
- Schema sanitizer handles multi-type `anyOf` with null
(`Union[str, int, None]`): keep the slim non-null anyOf and add
`nullable: true` so Gemini does not reject `{"type":"null"}`.
- Image fetch falls back to the caller-provided MIME (guessed from
URL extension) when the server omits Content-Type instead of
dropping the image as `non-image content-type=<none>`.
- Per-request aggregate caps on remote image inlining (8 images,
20MB total) so a single chat request cannot force unbounded
backend downloads.
- Frontend exposes the reasoning ladder for `gemini-2.5-flash-lite`
(`none/minimal/low/medium/high/max`) so the UI can drive the
thinkingBudget the backend already supports.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: round 18 Gemini follow-ups (PR #5720)
- `tool_choice="none"` now opts out of hosted builtin tools on every
provider path, not just Gemini and OpenAI Responses. Anthropic
web_search / web_fetch / code_execution, Kimi `$web_search` early
return, and OpenRouter `plugins:[{id:"web"}]` are all gated on
`tool_choice_disabled`. Passing `enabled_tools=[...]` with
`tool_choice="none"` no longer triggers provider-side search /
code execution for any provider.
- `_stream_anthropic` accepts `tool_choice` and threads it through;
the dispatcher in `stream_chat_completion` forwards it.
- Frontend `isServerSideBuiltinToolPart` simplified to drop only on
(marker) OR (canonical name + native_part). The previous shape
heuristic on `args.kind`/`args.command`/`args.code`/`args.prompt`
dropped real user-declared `code_execution` / `image_generation`
functions. Pre-PR persisted hosted cards lacking the marker now
leak to non-native providers on switch -- preferred to silently
deleting legitimate function-call history.
- Backend `_is_marked_server_builtin_tool_call` and the OpenAI
Responses translator's matching filter accept BOTH `_server_tool`
marker AND `args.google.native_part` as durable provider-side
signals so Gemini code_execution / image_generation cards are
still dropped on a provider switch.
- Per-request remote image count cap now counts ATTEMPTS, not just
successful inlines, so 100 failing/slow URLs cannot each consume
the 15s fetch timeout. Data: URL images now share the same count
and byte caps as fetched remote URLs.
- OpenAI Responses translator tracks skipped server-builtin
`function_call` ids and drops their matching `role="tool"`
follow-ups, preventing orphan `function_call_output` items in the
outbound body.
- Gemini schema sanitizer preserves multi-type unions with null:
`{"type":["string","integer","null"]}` becomes
`anyOf:[{string},{integer}] + nullable:true` instead of being
flattened to the first non-null type.
- Gemini model id validation moved to the top of `_stream_gemini`
so an invalid model id rejects the request before any remote
image fetch / message translation side effect.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: round 19 Gemini follow-ups (PR #5720)
- `_build_external_messages` now skips an empty assistant turn when
`_filter_tool_calls` drops every synthetic builtin tool_call (was
guarded only on the `content is None` branch; the string-content
and list-content branches still forwarded
`{"role":"assistant","content":""}` which several providers
reject). Also tracks the dropped server-builtin tool_call ids and
skips the matching `role="tool"` follow-ups so the receiving
provider does not see an orphan tool_result.
- OpenRouter `web_search_active` (the synthetic tool_start /
tool_end emitter) is now also gated on `tool_choice_disabled` so
a request with `tool_choice="none"` does not surface a fake
web_search card in the chat UI even though the plugin was
correctly stripped from the outbound body.
- `_stream_anthropic` translates an OpenAI role="tool" with list
content (`content=[{"type":"text","text":"..."}]`) into a native
`tool_result` block on a user message; previously only the
string-content shape was translated, so list-content tool results
were forwarded as invalid `role:"tool"` messages.
- Gemini `data:` URL image_url parts now require an `image/*` MIME
type; a `data:text/html;base64,...` is dropped instead of being
forwarded as `inlineData.mimeType="text/html"` (Gemini rejects
the malformed image part). Symmetric with the fetched-remote
image fetch path that already rejects non-image Content-Type.
- YouTube `fileData.fileUri` now declares `video/mp4` as the
mimeType instead of `image/jpeg` guessed from the URL path. The
YouTube/fileData input is the documented Gemini video path; the
guessed image MIME made valid YouTube inputs malformed.
- OpenAI Responses translator preserves `response.output` ordering
on assistant turns that emitted both text and a function_call:
assistant text is now serialized BEFORE the function_call item
so the subsequent function_call_output (the matching role=tool
follow-up) lands in the right position. Previously the order
was function_call -> assistant text -> function_call_output,
which can confuse multi-turn function-calling flows.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: round 20 Gemini follow-ups (PR #5720)
Convergent reviewer findings from round 20:
- tool_choice="none" no longer flips responseModalities=[TEXT,IMAGE]
on image-tier Gemini models. Forced-function tool_choice (e.g.
{type:function, function:{name:lookup}}) also drops hosted Search /
code execution from the Gemini body so the caller's pinned user
function is not silently joined by hosted builtins.
- Gemini code-execution thoughtSignature replay now uses an ordered
parts list (native_part.parts[]) so per-part signatures stay
attached to the exact part Gemini emitted. The previous merged
shape fanned one top-level thoughtSignature across executableCode
+ codeExecutionResult + inlineData and tripped Gemini 3 strict
validators. Backward-compat fallback keeps pre-round-21 persisted
history working: a legacy native_part with a single subpart still
replays the signature on that subpart; merged legacy objects pin
the signature to executableCode only.
- Remote-image fetch threads the remaining per-request byte budget
into _safe_fetch_image_for_gemini, so over-budget URLs are
refused via Content-Length pre-check / short read instead of
fully downloaded then discarded after the aggregate cap check.
- Gemini role=tool with OpenAI list-form content
([{type:text,text:result}]) now flattens text parts before
building functionResponse.response.result; previously the parts
arrived as the result value instead of the actual tool output.
- Frontend chat-adapter merges native_part by concatenating parts
lists (preserving per-part thoughtSignature). Wire types expose
enable_prompt_caching as boolean|string (Gemini cached-content
name) and OpenAIChatDelta now carries tool_calls and extra_content.
- Test test_openrouter_no_synthetic_web_search_event_on_tool_choice_none
reads _toolEvent from the top-level SSE payload so a backend
regression cannot mask the assertion.
Adds 7 regression tests covering image_generation gate, forced-function
gate, native_part list replay, legacy fallback, list-content
functionResponse flattening, fetch byte-budget threading, and wire
types.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Apply forced-function tool_choice gate to Anthropic, OpenRouter, Kimi
Previously only the Gemini path treated `tool_choice={"type":"function",
"function":{"name":...}}` as a hosted-tool opt-out. Anthropic,
OpenRouter, and Kimi still attached hosted web_search / web_fetch /
code_execution when the caller explicitly pinned a user function plus
`enabled_tools=[...]`. That contradicts the explicit function pin and
bills the caller for unwanted server-side calls.
Mirror the Gemini gate symmetrically:
- Anthropic web_search / web_fetch / code_execution
- OpenRouter `plugins:[{id:"web"}]` + the synthetic web_search SSE
event the same path emits at stream close
- Kimi `_stream_kimi_web_search` dispatch
Adds 4 regression tests:
- test_anthropic_forced_function_tool_choice_drops_hosted_tools
- test_openrouter_forced_function_tool_choice_drops_web_plugin
- test_kimi_forced_function_tool_choice_skips_web_search_helper
- test_openrouter_no_synthetic_web_search_event_on_forced_function_tool_choice
All 146 existing backend tests still pass.
* Strip Gemini-only synthetic tool history on local-GGUF dispatch
After a Gemini chat that ran code_execution / image_generation, switching
the same thread to a local GGUF model used to forward the synthetic
provider-side tool_calls (tagged with `args._server_tool` or carrying a
Gemini `args.google.native_part` payload) and the message-level
`extra_content` to llama-server. The receiving backend has no tool
declaration for those names and no use for Gemini thoughtSignature
metadata; in the worst case it can produce an orphan tool_call_id and a
confused continuation.
Add `_strip_provider_synthetic_tool_history()` and wire it through the
two local message builders:
- `_openai_messages_for_passthrough` (OAI-compat passthrough)
- `_openai_messages_for_gguf_chat` (standard GGUF chat path)
Real user-function `tool_calls` and their matching `role="tool"` replies
survive unchanged; only synthetic provider-side cards and Gemini-only
`extra_content` are stripped. If the synthetic call was the assistant
turn's only payload, the now-empty turn is dropped too so llama-server
does not reject the request.
Adds 2 regression tests:
- test_strip_provider_synthetic_tool_history_drops_synthetic_only
- test_strip_provider_synthetic_tool_history_drops_empty_assistant
142 existing backend tests still pass.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Disable Search/Code composer pills for Gemini image-tier models
For external Gemini image-tier models (gemini-2.5-flash-image,
gemini-3.x-image-preview, etc.), the backend unconditionally strips
code_execution and strips web_search on older image ids. Search is
still allowed on Gemini 3.x Pro/Flash image models, which
supportsBuiltinWebSearch already encodes per model.
Before this commit the composer pill gates were:
searchDisabled = !modelLoaded || !(supportsTools || supportsBuiltinWebSearch)
codeDisabled = !modelLoaded || !(supportsTools || supportsBuiltinCodeExecution) || imageModeDisablesCode
`supportsTools` here is a local-runtime fallback that becomes true when
any tool-capable local model has been loaded in the session. With a
local tool-capable runtime active, switching the chat to an external
Gemini image-tier model used to leave Search/Code clickable, even
though the backend will silently drop the tool on the wire.
Detect "external provider is Gemini AND the model is image-tier" (via
supportsBuiltinImageGeneration) and gate the two pills strictly on the
provider's own builtin support in that case. Non-Gemini paths and
non-image Gemini models keep the supportsTools fallback unchanged.
* Apply forced-function tool_choice gate to OpenAI Responses path
Round 22 added the gate for Gemini / Anthropic / OpenRouter / Kimi but
missed the OpenAI Responses translator. When a caller pinned a user
function via `tool_choice={"type":"function","function":{"name":...}}`
plus `enabled_tools=["web_search","code_execution","image_generation"]`,
the Responses body still attached `{"type":"web_search"}`,
`{"type":"shell"}`, and `{"type":"image_generation"}` server tools. The
function pin should suppress those for the same privacy + billing reason
the other provider paths now do.
Compute `_responses_tool_choice_forced_function` next to
`_responses_tool_choice_none` and gate each hosted-tool append on
`_responses_hosted_builtins_allowed = not none and not forced_function`.
The fix has to be applied in TWO places: the initial body builder and
`_build_body()` (called by the container-expiry retry path). User
function declarations still flow through so the pin has something to
target, and the Responses-shape `{type:"function", name:"..."}`
`tool_choice` is forwarded unchanged.
Adds regression test `test_openai_responses_forced_function_tool_choice_drops_hosted_tools`.
All 166 existing backend tests across Gemini + Responses + image-gen +
code-exec suites still pass.
* Round 24 P1s: SSRF shared-address gap + extra_content text-only leak + custom-Gemini model list
Three convergent P1s from round 24 review:
1. SSRF: the shared SSRF validator in `tools._validate_and_resolve_host`
used a denylist (is_private / loopback / link_local / multicast /
reserved / unspecified). Python classifies shared address space
(100.64.0.0/10 carrier-grade NAT, plus 240.0.0.0/4, benchmarking
ranges, etc.) with `is_private=False` AND `is_global=False`. The new
Gemini server-side image fetcher therefore accepts URLs whose
hostname resolves to 100.64.0.1 in cloud/VPC deployments. Add
`not ip.is_global` as the primary gate -- a single source of truth
that covers every current and future non-global range.
2. _strip_provider_synthetic_tool_history previously only stripped
message-level `extra_content` when the assistant turn had tool_calls.
A plain text Gemini reply carrying
`extra_content.google.thought_signature` flowed through to
llama-server when the thread was switched to a local GGUF backend.
Always strip message-level `extra_content` on assistant turns.
3. routes/providers.list_provider_models applied Gemini's native
`model_id_allowlist` regex to every Gemini provider, including
custom OAI-compatible bases (LiteLLM, deployment gateways). IDs like
`google/gemini-2.5-flash` and team-prefixed deployment aliases got
filtered out even though the chat-dispatch path now routes them via
the OpenAI-compatible client. Skip registry-level model-id filters
when the configured Gemini base_url host is not the canonical
`generativelanguage.googleapis.com`, mirroring the chat-dispatch
gate.
Three regression tests added:
- test_validate_and_resolve_host_blocks_shared_address_space
- test_strip_provider_synthetic_tool_history_drops_text_only_extra_content
- test_gemini_custom_oai_compat_base_skips_native_allowlist
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Round 25 P1s: skip synthetic server-tool replay + inline $ref/$defs into Gemini schema
Two convergent reviewer findings on the native Gemini path:
1. _stream_gemini's tool_calls replay loop falls through to a generic
functionCall emission whenever it sees an assistant tool_call. Marked
server-side builtin cards (web_search / web_fetch tagged with
_server_tool or args.google.native_part) hit that fallthrough with no
replayable native_part, which produces an outbound functionCall whose
name is not a declared user function. The Gemini turn 400s on the
undeclared name. Guard the loop to drop those entries instead, while
keeping the existing code_execution / image_generation native-part
replay branch intact.
2. _sanitize_gemini_schema uses a strict allowlist that drops local
$ref / $defs references. Pydantic-generated tool schemas hoist nested
object shapes into $defs and reference them via {"$ref": "#/$defs/X"},
so a property like address: {"$ref": "#/$defs/Address"} collapsed to
{} on the wire and the model lost the nested fields, types, and
required keys. Resolve local #/... pointers against the schema root
and inline the referenced subtree, with local siblings overriding
the reference (normal JSON Schema composition) and a seen-ref guard
for self-referential schemas.
Added regression coverage:
- test_gemini_native_skips_synthetic_server_builtin_replay
- test_function_declarations_inline_local_refs_into_gemini_schema
- test_function_declarations_inline_local_refs_in_anyof_and_items
- test_function_declarations_self_referential_schema_terminates
All 145 Gemini provider tests pass; touched provider regression set
(OpenAI Responses, code execution, image generation, Anthropic code
execution, Anthropic web_fetch) also 43/43 green.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Round 26 P1s: drop orphan Gemini functionResponse + Anthropic /messages synthetic-history strip
Reviewer round 26 surfaced two convergent asymmetric-fix bugs.
1. _stream_gemini drops a synthetic server-tool tool_call (web_search /
web_fetch tagged _server_tool) and also replays code_execution /
image_generation tool_calls as Gemini-native executableCode /
codeExecutionResult / inlineData parts. The matching role="tool"
follow-up was still falling through to the generic functionResponse
branch, producing either an orphan functionResponse (synthetic case)
or a duplicate response pointing at a name with no
functionDeclarations entry (native-part case). Both forms 400 the
next Gemini turn. Track skipped + native-replayed tool_call_ids in
_gemini_skip_tool_result_ids and short-circuit the role="tool"
branch on a match.
2. The Anthropic-compatible local /v1/messages route only called
_drop_empty_assistant_sentinels on the OpenAI-translated history,
while the sibling /v1/chat/completions and GGUF passthrough builders
chain that with _strip_provider_synthetic_tool_history. An Anthropic
caller replaying a prior provider-side tool_use therefore forwarded
fake builtin tool history straight into local llama-server. Apply
the same strip on the Anthropic route after the
anthropic_messages_to_openai conversion.
Regression coverage added:
- test_gemini_native_skips_orphan_function_response_for_dropped_builtin
- test_gemini_native_skips_orphan_function_response_for_native_part_replay
Gemini suite 147/147; touched provider regression set 43/43.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Round 27 P1s: native_part location fallback + Gemini image request budget for base64
Two convergent reviewer findings on the native Gemini path.
1. _stream_gemini's synthetic-builtin detector at lines 3519-3524
recognizes args.google.native_part as a server-tool marker, but
_native_part was only loaded from tc.extra_content.google.native_part.
A direct OpenAI-compatible API caller or imported third-party thread
round-trips the payload through function.arguments because
tool_calls[].extra_content is not in the OpenAI spec. The round-25
guard then saw a synthetic builtin with no _native_part and dropped
the entire assistant turn, so the next native Gemini request lost
the prior executableCode / inlineData / codeExecutionResult context.
Fall back to args.google.native_part when extra_content path is
missing, mirroring what the synthetic detector already accepts.
2. _GEMINI_REMOTE_IMAGE_MAX_TOTAL_BYTES capped DECODED bytes at 20MB.
Gemini receives images base64-encoded inside JSON, and base64
inflates payload size by ~4/3. With 20MB decoded the actual JSON
body is ~26.7MB plus prompt overhead, well over Gemini's ~20MB
request limit. Drop the decoded cap to 14MB so realistic multi-
image turns stay safely under 20MB encoded.
Added regression test test_gemini_native_part_falls_back_to_args_google
covering an OpenAI-compat-shaped image_generation tool_call whose
native_part lives only in function.arguments.
Gemini suite 148/148.
* Fix TS build errors from main merge: restore imageParts + refusal return [] + cast image-edit ref
Three errors in chat-adapter.ts surfaced by the frontend tsc step after merging
main into feat/gemini-provider:
1. The Anthropic refusal early-return used main's but
toOpenAIMessages returns SerializedMessage[]; flip to .
2. Restore -- the line
was lost when removing main's conflict block from the function body.
3. selectedImageEditReference splice was inserting OpenAIChatMessage
into a SerializedMessage[] array; the shapes differ on tool_calls.id
nullability. Cast the reference message through unknown -- it carries
no tool_calls, so the runtime payload is structurally compatible.
Reproduced locally with `tsc -b --pretty false` (now passes). Build
also failing in the in-repo `npm run build` step on PR CI; this commit
unblocks all 12 failing UI/API workflows.
* Tighten verbose comments in external_provider.py + chat-adapter.ts
Compress multi-line explanatory comments in the Gemini translator
and the chat adapter without changing any behaviour. All 148 Gemini
provider tests still pass; tsc --noEmit clean.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@users.noreply.github.com>
Daniel Han
