unsloth/studio/backend/tests/test_gemini_provider.py
Daniel Han ab48465135
Studio: add Gemini provider with web_search, code_execution, prompt caching, and Nano Banana image generation (#5720)
* Studio: add Gemini provider with web_search, code_execution, prompt caching, and Nano Banana image generation

Wires Google's native Gemini API into Studio's external-provider stack
so users can pick gemini-2.5-pro / gemini-2.5-flash / gemini-2.5-flash-image
(Nano Banana) alongside the existing OpenAI / Anthropic / OpenRouter
providers. Gemini does not speak OpenAI Chat Completions on its primary
endpoint; the new `_stream_gemini` async generator translates between
the two shapes the same way `_stream_anthropic` handles the Messages API.

Backend:
- New `_stream_gemini` translator in external_provider.py. Converts
  OpenAI messages -> Gemini `contents` + `systemInstruction`; maps
  generationConfig (temperature / topP / topK / maxOutputTokens);
  forwards `tools: [{googleSearch: {}}]` for web_search and
  `{codeExecution: {}}` for code_execution; passes `cachedContent`
  through for prompt caching; sets `responseModalities=[TEXT, IMAGE]`
  for Nano Banana image generation.
- Translates streamed `GenerateContentResponse` SSE frames back into
  OpenAI chat.completion.chunk frames (text deltas, function_call ->
  tool_calls deltas, inlineData -> image_b64 tool_end envelope, usage
  chunk before [DONE]).
- Registry entry switched to native base URL
  `https://generativelanguage.googleapis.com/v1beta` with
  `openai_compatible: False` and the `x-goog-api-key` auth header.
  Model lineup curated to current 2.5 / 2.0 family + Nano Banana.

Frontend:
- Provider-capability matrix: Gemini supports temperature, top_p, top_k,
  presence_penalty (matches generationConfig); min_p / repetition_penalty
  hidden because the API does not accept them.
- `providerSupportsBuiltinWebSearch` / `providerSupportsBuiltinCodeExecution`
  / `providerSupportsBuiltinImageGeneration` extended for Gemini.
- Prompt caching toggle now also lit on Gemini.

Tests:
- 21 new tests in `test_gemini_provider.py` using httpx.MockTransport.
  Cover request body shape conversion, URL/header wiring, web_search
  forwarded as googleSearch, function-call translation both directions,
  prompt caching passthrough, image generation emitting image_b64,
  grounded-search citations -> tool_end, finish_reason mapping, and
  vision data URL -> inlineData translation.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: forward presence_penalty to Gemini and recover function name from tool_call_id

Two follow-up fixes for the Gemini provider:

  * Thread presence_penalty into _stream_gemini and set
    generationConfig.presencePenalty when non-zero. The OpenAI-side
    capability matrix already exposes the slider for Gemini, so the
    value was being collected and silently dropped on the way out.

  * When an OpenAI role=tool message omits 'name' and only carries
    'tool_call_id', recover the function name from the matching
    functionCall on the prior assistant turn. Gemini 400s on an empty
    functionResponse name.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: surface Gemini code execution parts as code_execution tool events

The Gemini stream parser only handled text/functionCall/inlineData
parts, so when the user toggled the Code pill on a Gemini model the
sandbox output (executableCode + codeExecutionResult parts) was
dropped on the floor while adjacent text reached the UI. Reviewers
flagged this as the headline feature being silently broken.

Translate both parts into the existing code_execution tool envelope
that CodeExecutionToolUI already consumes for OpenAI / Anthropic:

  * executableCode  -> tool_start with kind=code_execution and the
    source code under arguments.code. We mint a tool_call_id and
    stash it so the matching result block can pair to it.
  * codeExecutionResult -> tool_end on that id with the stdout under
    result. Non-OK outcomes (OUTCOME_FAILED / OUTCOME_DEADLINE_EXCEEDED)
    are prefixed onto the text so the failure is visible.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: native Gemini model catalog, function-call ids, and honest cache claim

Three follow-ups to the Gemini provider PR after the codex pass:

  * list_models() now translates Gemini's native /v1beta/models
    payload ({models[{name, baseModelId, displayName,
    supportedGenerationMethods}]}) into the OpenAI-compatible shape
    Studio expects. Without this the picker stayed empty for Gemini
    and fell back to hardcoded defaults. Embedding-only models are
    filtered out.

  * Forward the OpenAI tool_call id into Gemini's functionCall.id
    and mirror it onto functionResponse.id. Two parallel calls to
    the same function name can now be paired unambiguously on the
    follow-up turn.

  * Drop Gemini from the prompt-caching capability set. The wire
    flow requires a separate cachedContents POST first and the
    boolean Studio emits today is a no-op; the toggle should not
    advertise a feature it cannot apply. Leaves a pointer to the
    docs for the eventual two-step orchestration.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: distinct tool_calls index per emitted Gemini function call

Codex flagged that the Gemini stream parser hardcoded
tool_calls[0].index to 0 on every emitted functionCall. OpenAI
reassemblers key tool_calls by index when joining deltas, so two
parallel function calls in one assistant turn collapsed onto a
single slot and the second call's arguments overwrote the first.

Track the running count via len(emitted_function_call_ids) - 1
and emit it as the per-call index. The dedupe guard above (skip
when fc_id already in the set) means the index is monotonic and
stable for the lifetime of the stream. Regression test asserts
[0, 1] across two parallel calls in one candidate parts list.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: surface Gemini 3.5/3.1/3 + Nano Banana 2/Pro and plumb thinking budget

`gemini-2.0-flash` / `gemini-2.0-flash-exp` were retired by Google in 2026
(`/v1beta/models/gemini-2.0-flash:streamGenerateContent` returns HTTP 404
"no longer available to new users"), and the picker had nothing past the
2.x family. Verified against the live ListModels catalog: drop the retired
ids from `default_models` + allowlist and surface the chat-capable
3.5 / 3.1 / 3 families plus the Nano Banana image trio.

Also plumb `enable_thinking` / `reasoning_effort` into Gemini's
`generationConfig.thinkingConfig`. Without this, Gemini 3.5 Flash,
gemini-pro-latest, and the 3.x previews silently spend the caller's
`max_tokens` budget on hidden "thoughts" before emitting any visible
answer -- the chat shows a truncated stub like "The capital of" and
streams stop. Mapping:
  - enable_thinking=False / reasoning_effort=none -> thinkingBudget=0
    (Flash tier; Pro tier coerces to a small positive budget because
    the API 400s on 0 with "This model only works in thinking mode")
  - minimal/low/medium/high -> 512/2048/8192/24576 budget tokens
  - max/xhigh -> -1 (dynamic)
  - default (neither knob set) -> thinkingConfig omitted, model decides

Frontend `getExternalReasoningCapabilities` now surfaces a
`reasoning_effort` picker for every Gemini chat id (Pro tier hides the
"none" option; image-tier ids stay knob-less). Adds 6 unit tests
covering Flash/Pro effort mapping, the off-toggle coercion on Pro,
default omission, and the nano-banana-pro-preview alias routing
through the image modalities path. 28 -> 34 tests in
`test_gemini_provider.py`, all green; full backend suite still passes
(1459/1460; the unrelated test_help_output flake is pre-existing and
not in any file this PR touches).

Live verification against generativelanguage.googleapis.com on
2026-05-24 with `_stream_gemini` directly:
  text   gemini-3.5-flash           single PASS  multi PASS
  text   gemini-3.1-pro-preview     single PASS  multi PASS
  text   gemini-3.1-flash-lite      single PASS  multi PASS
  text   gemini-3-pro-preview       single PASS  multi PASS
  text   gemini-3-flash-preview     single PASS  multi PASS
  text   gemini-2.5-pro             single PASS  multi PASS
  text   gemini-2.5-flash           single PASS  multi PASS
  text   gemini-2.5-flash-lite      single PASS  multi PASS
  text   gemini-flash-latest        single PASS  multi PASS
  text   gemini-flash-lite-latest   single PASS  multi PASS
  text   gemini-pro-latest          single PASS  multi PASS
  image  gemini-2.5-flash-image     PASS (1082 KB png returned)
  image  gemini-3.1-flash-image-preview  PASS (Nano Banana 2)
  image  gemini-3-pro-image-preview      PASS (Nano Banana Pro)
  tool   web_search                 PASS
  tool   code_execution             PASS
  -> 16/16 e2e through the actual ExternalProviderClient code path.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: tighten Gemini provider after review (PR #5720)

Fixes a batch of bugs surfaced by a second-pass review on top of the
3.5/3.1/3 + Nano Banana 2/Pro additions in c6724dbd.

Backend (external_provider.py):
- Constructor normalises legacy /v1beta/openai base URLs to /v1beta so
  Gemini providers saved before the native switch keep working without
  a manual re-config.
- Skip thinkingConfig, googleSearch, and codeExecution on image-tier
  models (-image / nano-banana). The image responseModalities path is
  mutually exclusive with text-tool wiring and stale UI state would
  otherwise 400 the turn.
- _PRO_THINKING_PREFIXES now includes gemini-3.5-pro and uses anchored
  prefix matching (exact id or "<prefix>-...") so the image-tier
  gemini-3-pro-image-preview cannot accidentally match the pro guard.
- Gemini 3 functionCall thoughtSignature is round-tripped through the
  tool_calls envelope via extra_content.google.thought_signature on
  emit, and replayed as a sibling of functionCall on the next request.
- finishReason swaps STOP -> tool_calls when any functionCall was
  emitted on the same turn so OAI clients trigger tool execution
  (matches the OpenAI Chat Completions contract).
- usageMetadata.thoughtsTokenCount is rolled into output_tokens and
  surfaced on output_tokens_details.reasoning_tokens so total_tokens
  reflects the full billable spend instead of dropping the hidden
  reasoning slice.

Registry (providers.py):
- Drop gemini-3-pro-preview from default_models. Google shut it down
  on 2026-03-09 and auto-redirects to gemini-3.1-pro-preview; we
  surface the canonical id only.
- Add model_id_deny_exact = ("gemini-3-pro-preview",) so the live
  ListModels fetch does not re-surface the redirect alias.

Route schema (models/inference.py):
- enable_prompt_caching widened to Optional[Union[bool, str]] so the
  /v1/chat/completions caller can pass a Gemini cachedContent resource
  name (e.g. cachedContents/abc123). Without this widening _stream_gemini
  s string cachedContent passthrough was unreachable from the public
  route (bool_parsing 422). stream_chat_completion signature mirrors.

Frontend (provider-capabilities.ts, chat-page.tsx, chat-adapter.ts):
- providerSupportsBuiltinImageGeneration now also recognises
  nano-banana ids (nano-banana-pro-preview was hidden from the image
  pill before).
- providerSupportsBuiltinWebSearch takes the model id so Gemini image
  models hide the Search pill (mirrors the backend skip).
- providerSupportsBuiltinCodeExecution uses the same isGeminiImageModel
  guard for nano-banana ids.
- GEMINI_THINKING_PRO_PREFIXES gains gemini-3.5-pro; gemini-3-pro
  tightened to gemini-3-pro-preview to avoid the image-id overlap.
- Updated 3 callers of providerSupportsBuiltinWebSearch to thread the
  selected model id through.

Tests (test_gemini_provider.py): 34 -> 42, all green
- test_image_models_skip_thinking_config
- test_image_models_drop_text_only_tools
- test_gemini_35_pro_recognized_as_pro_thinking
- test_legacy_openai_base_url_normalized
- test_finish_reason_swaps_to_tool_calls_when_function_call_emitted
- test_thought_signature_round_trips_into_gemini_function_call
- test_thought_signature_emitted_in_tool_call_delta
- test_usage_chunk_includes_thoughts_tokens

Verification:
- Backend pytest 1518/1519 passing (one unrelated Qwen3.5 flash-attn
  test fails on main as well; nothing in this PR touches that path).
- Frontend npx tsc -b clean.
- Live e2e 16/16 against generativelanguage.googleapis.com through the
  patched _stream_gemini code path (all 11 chat models single + multi
  turn, all 3 image models returned image bytes, web_search and
  code_execution tools both emit the expected envelope).
- Live /api/providers/models against the patched backend surfaces 16
  ids (gemini-3-pro-preview correctly filtered via deny_exact).

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: address second-pass review findings on Gemini (PR #5720)

Round-2 reviewer.py flagged a phantom web_search card on image
turns (12/12 reviewers), route-layer stripping of tool_calls /
tool_call_id / name, an over-narrow image-mode tool guard, and
silent safety blocks. This patch fixes all four.

Backend (external_provider.py):
- web_search_active is now derived from the outbound tools_array
  (whether googleSearch was actually forwarded), not the raw
  enabled_tools intent. Image-mode turns dropped the tool above so
  the inbound stream no longer emits a phantom "search complete"
  tool_start / tool_end on those turns.
- text_tools_allowed now uses is_image_model (covers both `-image`
  / `nano-banana` picker models AND text models that requested
  `image_generation` via enabled_tools). Verified against the live
  Gemini API which rejects both googleSearch and codeExecution
  alongside responseModalities=["TEXT","IMAGE"] with explicit 400s
  ("Search as tool is not enabled for this model", "Code execution
  is not enabled for this model").
- promptFeedback.blockReason is surfaced as a 400 content-filter
  error chunk instead of returning an empty successful assistant
  response. The streaming loop closes the response before exiting.

Route (routes/inference.py):
- _build_external_messages now propagates tool_calls (assistant),
  tool_call_id, and name (tool result) through every code path
  (string content, multimodal content, non-vision fallback). Without
  this Gemini 3 function-call round trips lost their thoughtSignature
  + tool_call_id at the route boundary, and functionResponse.name
  arrived empty on the second turn.
- Assistant messages with content=None and tool_calls populated are
  preserved as a synthetic empty-string content turn so the
  Gemini translator can rebuild the functionCall part.

Tests (test_gemini_provider.py): 42 -> 45, all green
- test_image_models_suppress_phantom_web_search_card
- test_image_generation_tool_drops_text_tools
- test_prompt_feedback_block_reason_surfaces_as_error

Verification:
- Backend pytest 1736 / 1736 (the two pre-existing unrelated fails
  on main, test_help_output and Qwen3.5 flash-attn pin, are skipped).
- Frontend npx tsc -b clean.
- Live e2e 16/16 against generativelanguage.googleapis.com:
  11 chat models single + multi turn, 3 image models returning
  image bytes, web_search and code_execution both PASS.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: fix third-pass Gemini findings (PR #5720)

Round 3 review follow-ups:

Backend (studio/backend/core/inference/external_provider.py):
- Close response AND aiter_lines iterator in a finally so normal,
  prompt-block, and cancellation exits all clean up (eliminates the
  RuntimeWarning about aclose never being awaited).
- Pair the synthetic web_search tool_start with a tool_end on the
  promptFeedback.blockReason path so the UI does not leave a stuck
  "searching..." spinner after the error toast.
- Preserve native id and thoughtSignature on executableCode and
  codeExecutionResult tool events under google.native_part, and pair
  the tool_end on the code-exec id so multi-turn code-execution
  replays do not lose Gemini-required history.
- Carry part-level thoughtSignature on text deltas via
  delta.extra_content.google.thought_signature and on inline image
  tool_end via google.thought_signature so Gemini 3 image editing
  and tool turns round-trip the signature on the next request.
- Guess remote image_url MIME from the URL path so PNG / WebP / GIF
  inputs are not silently relabeled as JPEG.
- Roll usageMetadata.toolUsePromptTokenCount into translated input
  tokens and surface thoughtsTokenCount as
  completion_tokens_details.reasoning_tokens in _build_usage_chunk.
- Only normalize the Google-hosted /v1beta/openai legacy base URL;
  custom proxies whose paths happen to end in /openai are left
  untouched.
- Forward ChatCompletionRequest.tools and tool_choice through
  stream_chat_completion into _stream_gemini, translating to
  tools[].functionDeclarations and toolConfig.functionCallingConfig.

Frontend:
- chat-adapter: when Gemini image-generation is enabled for the turn,
  also disable Search and Code so the request, builder, and active
  pills agree with what the backend actually sends (the backend
  already strips text tools when image_generation is in enabled_tools).
- chat-adapter: consume OpenAI-shape delta.tool_calls chunks so
  Gemini function-call deltas without text surface as tool-call parts.
- shared-composer: disable Search and Code pills while Gemini image
  mode is active so the UI matches the request.

Tests (studio/backend/tests/test_gemini_provider.py): adds coverage
for proxy base-url gating, remote image MIME inference,
toolUsePromptTokenCount, reasoning_tokens propagation, prompt-block
web_search tool_end pairing, native code-exec id/thoughtSignature
metadata, inline image thoughtSignature, text-chunk extra_content,
OpenAI tools/tool_choice translation, and image-model tool drop.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: Gemini 3 thinkingLevel + image-model Search grounding (PR #5720)

Gemini 3.x migrated to a string `thinkingConfig.thinkingLevel`
(MINIMAL/LOW/MEDIUM/HIGH) and rejects `thinkingBudget`+`thinkingLevel`
in the same request. Gemini 3 also cannot turn thinking fully off, so
the lowest position is "minimal" (Flash) or "low" (Pro rejects
"minimal").

- external_provider._stream_gemini: split thinking translation by
  family. Gemini 3.x (3 / 3.1 / 3.5 + gemini-pro-latest /
  gemini-flash-latest / gemini-flash-lite-latest) emits
  thinkingConfig.thinkingLevel; effort none/off coerces to "low" on
  Pro and "minimal" on Flash. Gemini 2.5 stays on thinkingBudget.
- external_provider._stream_gemini: allow `tools: [{googleSearch: {}}]`
  on the Gemini 3 image family (gemini-3-pro-image-preview,
  gemini-3.1-flash-image-preview, nano-banana-pro). Google's docs
  document Search grounding on these. codeExecution stays blocked
  on image mode (still mutually exclusive with responseModalities).
- provider-capabilities.ts: mirror the Gemini 3 effort ladders in
  resolveGeminiReasoningCapabilities (Pro: low/medium/high; Flash:
  minimal/low/medium/high; 2.5 Flash keeps the off-position).
- provider-capabilities.ts: providerSupportsBuiltinWebSearch now
  returns true on the documented Gemini 3 image models so the pill
  is reachable; older image ids (gemini-2.5-flash-image) still hide.

Tests: splits the existing thinkingBudget cases by family (Gemini 3
checks thinkingLevel; Gemini 2.5 keeps thinkingBudget), adds positive
googleSearch coverage for Gemini 3 image models and negative
googleSearch coverage for legacy image models.

References:
- https://ai.google.dev/gemini-api/docs/thinking
- https://ai.google.dev/gemini-api/docs/gemini-3
- https://ai.google.dev/gemini-api/docs/models/gemini-3-pro-image-preview

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: attach Gemini code_execution inline images to the code card (PR #5720)

When a text Gemini turn wires codeExecution and the sandbox produces a
matplotlib plot, the inline image part ships right after the
codeExecutionResult. Previously this surfaced as a separate empty
image_generation card. Track the most recent code_execution
tool_call_id + result text and, when an inline image follows with
code_execution active, emit a second tool_end on the same id that
appends the image as a data: URI under the `__IMAGES__:` marker the
chat-adapter already understands.

Image-picker turns (`-image` / `nano-banana`) keep the standalone
image_generation envelope so Nano Banana outputs render the same way.

Tests: covers the merged code-execution card emission with no
standalone image_generation event when code_execution is the active
tool.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: fix fourth-pass Gemini findings (PR #5720)

Round 4 review follow-ups:

Backend:
- `_is_openai_compatible` + `_auth_headers` detect Gemini connections
  pointed at a custom OpenAI-compatible proxy (non-Google host whose
  path ends in `/openai`) and route them through the OpenAI-compat
  surface with `Authorization: Bearer ...` instead of the native
  `_stream_gemini` translator + `x-goog-api-key`. Google-hosted Gemini
  keeps the native dispatch path it migrated to in this PR.
- `_stream_gemini` thinkingLevel handling for Gemini 3 Pro now coerces
  both "minimal" and "medium" effort to "low" / "high" respectively
  (Pro tier only accepts low/high per
  https://ai.google.dev/gemini-api/docs/thinking).
- `providers.py` `default_models` restores the advertised
  `gemini-3.5-pro` and the rolling `gemini-pro-latest` /
  `gemini-flash-latest` / `gemini-flash-lite-latest` aliases that the
  allowlist already admits.

Frontend:
- chat-adapter: lean on `providerSupportsBuiltinWebSearch` (which
  already encodes the Gemini 3 image-model Search allowance) instead
  of blanket-disabling Search whenever Gemini image mode is active.
  Code execution stays blocked because Gemini image mode rejects it.
- shared-composer: mirror the same gate -- only the Code pill is
  unconditionally disabled in Gemini image mode; the Search pill is
  driven by `supportsBuiltinWebSearch`.
- provider-capabilities: Gemini 3 Pro reasoning levels now expose only
  "low" and "high" (no Medium pill) to match the API.

Tests: covers the Gemini 3 Pro medium / minimal coercion, the custom
proxy OAI-compat dispatch + Authorization Bearer auth, and the
native-vs-proxy detection. Also closes the mocked httpx.AsyncClient
inside the test event loop so the Python 3.13 `aclose was never
awaited` warning no longer fires.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: fix fifth-pass Gemini findings (PR #5720)

Round 5 review follow-ups:

Backend:
- `_is_openai_compatible` + `_auth_headers` now treat ANY non-Google
  Gemini base URL as OpenAI-compat (LiteLLM / custom OAI gateways /
  OpenAI-compat vLLM routers), not just paths ending in `/openai`.
  Pre-existing saved Gemini proxies on `/v1` keep working.
- Gemini 3 thinkingLevel coercion narrowed to the documented
  inconsistencies: only "minimal" is coerced to "low" on Pro tier.
  "medium" passes through (Gemini 3.1 Pro accepts it per
  https://docs.cloud.google.com/vertex-ai/generative-ai/docs/models/gemini/3-1-pro).
- `_stream_gemini` only flips `responseModalities=[TEXT,IMAGE]` when
  the selected model is image-capable. A stale
  `enabled_tools=["image_generation"]` on a text model is silently
  dropped instead of producing an invalid Gemini request.
- `_stream_gemini` validates the model id against
  `[A-Za-z0-9._-]+` before URL interpolation so a model like
  `../cachedContents/x` cannot redirect the request to an unintended
  endpoint with the configured API key attached.
- Empty-text Gemini parts that still carry `thoughtSignature` emit a
  content-free delta with `extra_content.google.thought_signature` so
  Gemini 3 turns that end with a signature-only fragment do not lose
  the replay state.
- ConnectError / ReadTimeout / generic HTTPError paths in
  `_stream_gemini` now close the synthetic web_search tool_start
  with a matching tool_end before the error chunk so the UI does not
  leave a stuck "searching..." card on transport failure.
- `providers.py` default_models drop the non-existent
  `gemini-3.5-pro` (Google launched only `gemini-3.5-flash` at
  I/O 2026; Pro tier remains `gemini-3.1-pro-preview`).
- `routes/inference.py` only forwards `payload.top_k` when the caller
  explicitly set it on the request (Pydantic `model_fields_set`).
  Omitted top_k stays omitted, restoring the pre-PR behavior where
  Gemini uses its server default.
- `ChatCompletionRequest.enable_prompt_caching` adds a `mode="before"`
  validator that coerces the canonical string literals "true"/"false"
  back to bool so historical opt-out callers keep working after the
  field widened to `Union[bool, str]` for Gemini cache resource names.

Frontend:
- `providerSupportsBuiltinWebSearch` / Code / Image now accept the
  saved connection `baseUrl` and return false for custom OAI-compat
  Gemini proxies. Backend skips `_stream_gemini` for those bases, so
  native tool envelopes never reach them; hiding the pills keeps the
  request, builder, and UI consistent.
- `provider-capabilities.ts` Gemini 3 Pro effort ladder restores
  `["low", "medium", "high"]` to match Google's documented levels.
- Call sites in `chat-page.tsx` and `chat-adapter.ts` pass through
  `provider.baseUrl` so the proxy gate fires.

Tests: covers Gemini 3 Pro medium pass-through, custom proxy dispatch
on `/v1` and `/openai` bases, path-traversal model id rejection,
top_k omission when not explicit, text-model image_generation drop,
empty-text + thoughtSignature surfacing, and
enable_prompt_caching string coercion.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: fix sixth-pass Gemini findings (PR #5720)

Round 6 review follow-ups:

Frontend:
- chat-adapter `delta.tool_calls` accumulates fragments by `id` /
  `index` instead of pushing a new tool-call card per chunk. The
  standard OpenAI Chat Completions stream contract sends `id`/`name`
  on the first chunk and partial `function.arguments` on subsequent
  chunks; our previous handler parsed each fragment as a standalone
  tool call. Local llama.cpp and OAI-compat providers that stream
  fragments now reassemble into a single function-call part.
- chat-adapter also preserves `extra_content` on streamed tool-call
  deltas so Gemini 3 `thoughtSignature` survives to the next turn.
- provider-capabilities Gemini 3 Pro restores "medium" in the
  reasoning-effort ladder (Google's official Gemini API thinking
  doc lists low/medium/high for Gemini 3.1 Pro; my earlier round 4
  coercion was wrong).
- provider-capabilities orders `gemini-2.5-flash-lite` ahead of the
  broader `gemini-2.5-flash` prefix so Flash-Lite falls into the
  "no native thinking knob" branch as documented.

* Studio: round-trip Gemini tool_calls and tool results (PR #5720)

Recurring round 3-6 P1: the chat-adapter renders Gemini function-call
parts and code-execution events but `toOpenAIMessage` only serialized
text + image content, so the next turn lost the assistant
`tool_calls[]` (including Gemini 3's required
`extra_content.google.thought_signature`) and the matching
`role="tool"` result. Gemini 3 multi-turn function calling and code
execution failed validation on the second turn.

Frontend:
- types/api.ts widens OpenAIChatMessage to permit `role="tool"`,
  `tool_calls`, `tool_call_id`, `name`, and `content: null`. Adds
  OpenAIToolCallPart with `extra_content` for the Gemini round-trip.
- chat-adapter: new `toOpenAIMessages` expands an assistant turn with
  tool-call parts into [assistant w/ tool_calls + extra_content,
  role=tool result, ...]. tool result content is JSON-serialized so
  the backend translator can rebuild Gemini's `functionResponse`
  shape.
- chat-adapter outbound history now uses `flatMap(toOpenAIMessages)`
  so each assistant tool-call round-trips through the standard OAI
  shape the backend's `_stream_gemini` already understands.

* Studio: replay Gemini code_execution and image native parts on history (PR #5720)

Multi-turn Gemini history previously lost the native executableCode,
codeExecutionResult, and inlineData parts because the outbound
translator regenerated a generic functionCall for every assistant
tool_call. Stow the native dict on tool_end (frontend) and replay it
verbatim with thoughtSignature (backend) so follow-up turns preserve
the prior execution and image generation state. Skip role="tool"
fan-out for server-side builtin tools so Gemini does not 400 on a
functionResponse with no matching user-declared function.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: complete Gemini built-in tool replay round-trip (PR #5720)

Round 7 follow-up to the multi-turn native-part work. Three asymmetric
storage/consume gaps remained between the backend translator and the
chat adapter, so realistic Gemini follow-up turns degraded to generic
functionCalls instead of native history.

- Frontend collectAssistantToolCalls now drops web_search outright,
  drops code_execution / image_generation when the native part is
  missing, and promotes args.google to extra_content.google so the
  backend native_part replay branch actually fires.
- Backend image_generation tool_end now emits google.native_part
  with the inlineData (mimeType + base64) and thoughtSignature so the
  follow-up image-edit turn can replay the prior image as a native
  Gemini model part.
- Backend code-execution plot tool_end now stows google.native_part
  with the inlineData so the merged code-exec card can round-trip
  executableCode + codeExecutionResult + inlineData on the same id.
- Added regression tests for image-gen native-part replay and the
  code-exec plot native_part stow.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: round 8 Gemini follow-ups (PR #5720)

- Text-part thoughtSignature: stow on the assistant message during
  streaming and replay onto the last text part on the next turn so
  Gemini 3 strict function-calling does not reject history.
- Function declarations: recursively strip Gemini-unsupported OpenAPI
  keys (additionalProperties, $schema, $defs, strict, etc.) so OpenAI
  strict tools stop 400ing as INVALID_ARGUMENT on Gemini.
- OpenAI-compat fallback: forward tools/tool_choice so custom Gemini
  proxies (LiteLLM, gateways) keep function-calling.
- enable_prompt_caching: cover the Pydantic v1 legacy off/on/f/n/t/y
  string set so explicit opt-outs stay opt-out (Gemini was sending
  cachedContent: "off" otherwise).
- Frontend collectAssistantToolCalls / collectToolResultMessages: use
  google.native_part + result presence to disambiguate provider
  builtins from same-named user-declared functions.
- Added regression tests for text-signature replay and schema
  sanitization.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: round 9 Gemini follow-ups (PR #5720)

Two round-9 convergent finds across the 12 reviewers:

- Server-side web_search was leaking onto the next turn as a fake
  user functionCall/functionResponse. The previous heuristic (skip
  builtin only when no native_part AND no result) let it through
  because the synthetic tool card has a non-empty result string.
  Always skip web_search by name on both serializers, accept that a
  user-declared function literally named "web_search" must use a
  different name.
- Assistant `extra_content` was dropped by ChatMessage validation
  before _stream_gemini could replay text-part thought signatures.
  Add the field to ChatMessage and forward it through
  _build_external_messages so the multi-turn signature path actually
  carries data.

Includes a regression test for the ChatMessage round-trip.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: round 10 Gemini follow-ups (PR #5720)

Three convergent round-10 reviewer findings closed:

- Tag synthetic provider-side builtins with `args._server_tool=True`
  via a central helper that runs in every `_emit_tool_event` /
  `_emit_synthetic_tool_event` path. The frontend filter now skips
  on that marker instead of on the public tool name, so local
  llama.cpp `web_search` and OpenAI function tools literally named
  `web_search` / `code_execution` / `image_generation` round-trip
  cleanly while Gemini grounding / hosted code-exec / hosted image
  cards stay skipped.
- Gate Gemini image-mode (responseModalities=[TEXT,IMAGE]) on the
  Images pill (enabled_tools containing `image_generation`).
  Selecting an image-capable model with the pill off no longer forces
  image output the UI says is disabled.
- Frontend missing-key guard now exempts custom Gemini OAI-compat
  proxies (LiteLLM, gateways) the same way the backend already
  does, so a saved Gemini connection on `http://localhost:4000/v1`
  with no API key stops being blocked.

Existing tests updated to pass `enabled_tools=["image_generation"]`
on image-mode capture paths.

* Studio: round 11 Gemini follow-ups (PR #5720)

Four round-11 findings closed:

- Kimi _stream_kimi_web_search's local _synthetic_chunk helper now
  runs through _stamp_server_tool_marker so Kimi search history is
  not replayed as a fake user functionCall on the next turn (was an
  asymmetric miss after the round-10 tagging work).
- OpenAI Responses path (/v1/responses for gpt-5.x) forwards
  caller-supplied tools / tool_choice, translating the Chat
  Completions function-tool shape into the Responses native shape.
  Without this, standard OpenAI tools silently dropped on
  Responses-routed traffic.
- Decoupled the Gemini image-tier model-id guards (text-tool /
  thinking strip) from the Images pill flip
  (responseModalities=[TEXT,IMAGE]). gemini-2.5-flash-image with
  Search/Code on and the Images pill OFF no longer forwards
  googleSearch + thinkingConfig (Gemini 400s on those for legacy
  image ids).
- Gemini-only extra_content is now forwarded by
  _build_external_messages only when provider_type=="gemini" so
  Google's thought_signature does not leak into OpenAI / Mistral /
  Kimi / OpenRouter request bodies as an unknown field.

Added a regression test for the image-tier strict-guard split and
extended the extra_content test to cover the non-Gemini suppression.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: round 12 Gemini follow-ups (PR #5720)

Three round-12 convergent findings closed:

- extra_content leak to custom Gemini OAI-compat proxies (8/12
  reviewers). _build_external_messages now gates extra_content on
  the native generativelanguage.googleapis.com host, not just
  provider_type=="gemini", so LiteLLM / custom gateways routed
  through /chat/completions do not get an unknown top-level field.
- OpenAI Responses function-tool round-trip (5/12 reviewers). I
  added user `tools` forwarding in round 11 but did not parse the
  matching response.output_item.done items of type=function_call.
  The parser now translates them into Chat Completions
  delta.tool_calls and the terminal chunk reports
  finish_reason="tool_calls" when the model invoked a user
  function.
- Image-tier model with Images pill OFF (2/12). Google's image
  models default to text+image when responseModalities is omitted,
  so the previous fix silently still billed image output. Force
  responseModalities=["TEXT"] when the Images pill is off and the
  selected model is image-capable.

Updated the two pre-existing tests that pinned the synthetic-tool
arguments shape to include the new `_server_tool: True` marker, and
added a regression test for the Responses function-call output
translation.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: round 13 Gemini/Responses follow-ups (PR #5720)

Three round-13 convergent findings closed:

- OpenAI Responses function_call indices: my round-12 translator
  hardcoded every emitted tool_calls[*].index to 0, so parallel
  function calls collapsed for index-keyed clients. Track and
  increment function_call_index per emit (mirrors the Gemini
  branch's distinct-index pattern). 10/12 reviewers flagged.
- _SERVER_SIDE_BUILTIN_TOOL_NAMES now includes web_fetch so
  Anthropic-hosted web_fetch cards carry the _server_tool marker
  and the frontend history serializer doesn't replay them as fake
  user functions. 4 reviewers flagged.
- OpenAI Responses follow-up tool results now serialize as
  Responses-shape function_call / function_call_output items keyed
  by call_id, instead of Chat Completions role="tool" content.
  Skips assistant tool_calls tagged with _server_tool so hosted
  builtins don't round-trip as user functions. 2 reviewers flagged.

Updated the Anthropic code_execution and web_fetch test argument
pins to include the new _server_tool marker, and added two
regression tests (distinct indices on parallel function_call,
function_call_output round-trip).

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: round 14 Gemini follow-ups (PR #5720)

Three round-14 findings closed:

- Remote `image_url` translation (5 reviewers convergent). Public
  HTTPS image URLs can't be sent as `fileData.fileUri` -- Gemini
  reserves that path for Files API URIs and YouTube. Fetch the
  bytes server-side and inline them as base64 `inlineData`,
  mirroring the pre-PR OpenAI-compat behaviour. YouTube URLs and
  generativelanguage.googleapis.com/v1beta/files/* stay as
  `fileData`.
- Nullable JSON Schema type arrays. OpenAI strict tools commonly
  use `"type": ["string", "null"]`; the Gemini sanitizer now
  flattens that to `"type": "string", "nullable": true` so strict
  function tools stop 400ing.
- Parallel functionResponses now ride on one user content block
  with multiple `functionResponse` parts, matching Google's
  parallel tool docs. Consecutive `role="tool"` messages merge
  into the previous user turn instead of splitting into separate
  Gemini user turns.

Three regression tests added (remote URL fetch + inline, Files
API / YouTube fileData preservation, schema nullable flattening,
parallel-tool grouping).

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: SSRF harden Gemini remote image fetch (PR #5720)

Round 15 convergent finding (12/12 reviewers). My round-14 fix to
download user-controlled image URLs for inlineData inlining was an
SSRF / data-exfiltration path: no scheme check, no private-host
guard, no size cap, no Content-Type validation, redirects could
bounce to internal services, and the full URL was logged.

Replace the inline fetch with `_safe_fetch_image_for_gemini`:

- Require https:// (reject http, file, data, ftp, etc).
- Resolve the hostname via socket.getaddrinfo and reject if ANY
  resolved address is private / loopback / link-local / multicast /
  reserved / unspecified (covers 127.0.0.0/8, 10/8, 172.16/12,
  192.168/16, ::1, 169.254/16 metadata, RFC 6890).
- Block IP-literal URLs that resolve into those same ranges.
- Cap response body at 10 MB (Content-Length pre-check + streamed
  byte counter).
- Require Content-Type to start with `image/`.
- Disable redirect following so a 302 to a private host can't slip
  past the address check.
- Use a short 15s timeout and a tiny connection pool dedicated to
  these fetches.
- Log only the host name + error class -- no full URL, no signed
  querystring leak.

If the guard rejects, the image part is silently dropped (instead
of forwarding raw bytes or a fileData fallback). Files API URIs
and YouTube URLs still ride as `fileData.fileUri` unchanged.

Tests: replaced the live-fetch test with a `_safe_fetch_image_for_gemini`
monkeypatch, added four new SSRF-guard tests (non-https rejected,
loopback / private IP literals rejected, hostnames that resolve to
private IPs rejected).

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: round 16 Gemini follow-ups (PR #5720)

- IP-pinned image fetch (`_safe_fetch_image_for_gemini`): reuse the
  validated-once-then-pin pattern from `tools._fetch_page_text` via
  `asyncio.to_thread`, so DNS rebinding between validation and the
  HTTP connect cannot redirect us at a private/metadata address.
  Catch malformed-bracketed IPv6 urlparse errors. Follow up to 4
  redirect hops with per-hop SSRF re-validation.
- Replace contains-substring detection of Gemini Files API + YouTube
  URLs with parsed scheme/host/path checks, so attacker URLs like
  `https://evil.example/path/youtube.com/x.png` no longer skip the
  safe-fetch path and serialize as `fileData.fileUri`.
- `_build_external_messages`: strip per-tool-call `extra_content`
  for non-native-Gemini providers; the Gemini-only
  `thought_signature` payload was leaking through `tool_calls[]`
  into /chat/completions on OpenAI, Anthropic, and custom Gemini
  OAI-compat gateways.
- `_server_tool` marker now gated on the function name being one of
  the canonical builtin names (`web_search`, `web_fetch`,
  `code_execution`, `image_generation`) AND the marker being set,
  so a user function whose schema happens to define an
  `_server_tool` field is no longer dropped. Frontend filter mirrors
  the same gate, plus a backward-compat fallback for pre-PR
  persisted server-tool cards (no marker) routed via name +
  native_part / web-tool heuristic.
- Gemini schema sanitizer collapses `anyOf: [{X}, {"type":"null"}]`
  to `{X, "nullable": true}` so Optional[X] tool args from
  OpenAI/Pydantic schemas no longer 400 the Gemini request.
- Frontend tool-result serializer emits `{"result":""}` for empty
  string outputs so the ChatMessage validator does not reject
  `role="tool"` with empty content.
- Coerce `medium` thinkingLevel to `high` for legacy
  `gemini-3-pro*` / `gemini-3-pro-preview*` (only low/high
  documented; shut down 2026-03-09); 3.1+ Pro still passes through.
- Hide Gemini native thinking ladder on custom OAI-compat Gemini
  gateways by routing `getExternalReasoningCapabilities` through
  `isGeminiCustomOpenAICompatBase(baseUrl)`; thread baseUrl through
  all four call sites.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: round 17 Gemini follow-ups (PR #5720)

- Frontend `collectAssistantToolCalls` and `collectToolResultMessages`
  no longer drop unmarked `web_search` / `web_fetch` cards by name
  alone: a user-defined function with one of those names must
  round-trip. Pre-PR persisted `code_execution` / `image_generation`
  cards still get filtered via a shape heuristic (kind/command/code/
  prompt fields) instead of bare name.
- `_build_external_messages._filter_tool_calls` now drops marked
  server-side builtin `tool_calls` entirely for non-native-Gemini
  providers, not just their `extra_content`. An assistant turn whose
  only payload was a marked builtin is dropped completely so the
  receiving provider does not see an orphan tool_call.
- `_stream_anthropic` translates OpenAI top-level `tool_calls` into
  Anthropic native `{type:"tool_use", id, name, input}` content
  blocks, and translates `role="tool"` follow-ups into `role:"user"`
  messages carrying a `tool_result` block. Anthropic's native
  Messages API rejects the OpenAI shapes.
- `_safe_fetch_image_for_gemini_sync` factors URL validation through
  `_safe_parse_https`, so malformed `port` access (e.g.
  `https://host:bad/x.png`) and malformed redirect targets (e.g. a
  302 to `https://[bad/x.png`) drop the image instead of raising mid-
  request.
- `tool_choice="none"` now disables hosted builtins (Gemini
  googleSearch / codeExecution and OpenAI Responses web_search /
  shell / image_generation), not just user function declarations.
- Schema sanitizer handles multi-type `anyOf` with null
  (`Union[str, int, None]`): keep the slim non-null anyOf and add
  `nullable: true` so Gemini does not reject `{"type":"null"}`.
- Image fetch falls back to the caller-provided MIME (guessed from
  URL extension) when the server omits Content-Type instead of
  dropping the image as `non-image content-type=<none>`.
- Per-request aggregate caps on remote image inlining (8 images,
  20MB total) so a single chat request cannot force unbounded
  backend downloads.
- Frontend exposes the reasoning ladder for `gemini-2.5-flash-lite`
  (`none/minimal/low/medium/high/max`) so the UI can drive the
  thinkingBudget the backend already supports.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: round 18 Gemini follow-ups (PR #5720)

- `tool_choice="none"` now opts out of hosted builtin tools on every
  provider path, not just Gemini and OpenAI Responses. Anthropic
  web_search / web_fetch / code_execution, Kimi `$web_search` early
  return, and OpenRouter `plugins:[{id:"web"}]` are all gated on
  `tool_choice_disabled`. Passing `enabled_tools=[...]` with
  `tool_choice="none"` no longer triggers provider-side search /
  code execution for any provider.
- `_stream_anthropic` accepts `tool_choice` and threads it through;
  the dispatcher in `stream_chat_completion` forwards it.
- Frontend `isServerSideBuiltinToolPart` simplified to drop only on
  (marker) OR (canonical name + native_part). The previous shape
  heuristic on `args.kind`/`args.command`/`args.code`/`args.prompt`
  dropped real user-declared `code_execution` / `image_generation`
  functions. Pre-PR persisted hosted cards lacking the marker now
  leak to non-native providers on switch -- preferred to silently
  deleting legitimate function-call history.
- Backend `_is_marked_server_builtin_tool_call` and the OpenAI
  Responses translator's matching filter accept BOTH `_server_tool`
  marker AND `args.google.native_part` as durable provider-side
  signals so Gemini code_execution / image_generation cards are
  still dropped on a provider switch.
- Per-request remote image count cap now counts ATTEMPTS, not just
  successful inlines, so 100 failing/slow URLs cannot each consume
  the 15s fetch timeout. Data: URL images now share the same count
  and byte caps as fetched remote URLs.
- OpenAI Responses translator tracks skipped server-builtin
  `function_call` ids and drops their matching `role="tool"`
  follow-ups, preventing orphan `function_call_output` items in the
  outbound body.
- Gemini schema sanitizer preserves multi-type unions with null:
  `{"type":["string","integer","null"]}` becomes
  `anyOf:[{string},{integer}] + nullable:true` instead of being
  flattened to the first non-null type.
- Gemini model id validation moved to the top of `_stream_gemini`
  so an invalid model id rejects the request before any remote
  image fetch / message translation side effect.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: round 19 Gemini follow-ups (PR #5720)

- `_build_external_messages` now skips an empty assistant turn when
  `_filter_tool_calls` drops every synthetic builtin tool_call (was
  guarded only on the `content is None` branch; the string-content
  and list-content branches still forwarded
  `{"role":"assistant","content":""}` which several providers
  reject). Also tracks the dropped server-builtin tool_call ids and
  skips the matching `role="tool"` follow-ups so the receiving
  provider does not see an orphan tool_result.
- OpenRouter `web_search_active` (the synthetic tool_start /
  tool_end emitter) is now also gated on `tool_choice_disabled` so
  a request with `tool_choice="none"` does not surface a fake
  web_search card in the chat UI even though the plugin was
  correctly stripped from the outbound body.
- `_stream_anthropic` translates an OpenAI role="tool" with list
  content (`content=[{"type":"text","text":"..."}]`) into a native
  `tool_result` block on a user message; previously only the
  string-content shape was translated, so list-content tool results
  were forwarded as invalid `role:"tool"` messages.
- Gemini `data:` URL image_url parts now require an `image/*` MIME
  type; a `data:text/html;base64,...` is dropped instead of being
  forwarded as `inlineData.mimeType="text/html"` (Gemini rejects
  the malformed image part). Symmetric with the fetched-remote
  image fetch path that already rejects non-image Content-Type.
- YouTube `fileData.fileUri` now declares `video/mp4` as the
  mimeType instead of `image/jpeg` guessed from the URL path. The
  YouTube/fileData input is the documented Gemini video path; the
  guessed image MIME made valid YouTube inputs malformed.
- OpenAI Responses translator preserves `response.output` ordering
  on assistant turns that emitted both text and a function_call:
  assistant text is now serialized BEFORE the function_call item
  so the subsequent function_call_output (the matching role=tool
  follow-up) lands in the right position. Previously the order
  was function_call -> assistant text -> function_call_output,
  which can confuse multi-turn function-calling flows.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: round 20 Gemini follow-ups (PR #5720)

Convergent reviewer findings from round 20:

- tool_choice="none" no longer flips responseModalities=[TEXT,IMAGE]
  on image-tier Gemini models. Forced-function tool_choice (e.g.
  {type:function, function:{name:lookup}}) also drops hosted Search /
  code execution from the Gemini body so the caller's pinned user
  function is not silently joined by hosted builtins.

- Gemini code-execution thoughtSignature replay now uses an ordered
  parts list (native_part.parts[]) so per-part signatures stay
  attached to the exact part Gemini emitted. The previous merged
  shape fanned one top-level thoughtSignature across executableCode
  + codeExecutionResult + inlineData and tripped Gemini 3 strict
  validators. Backward-compat fallback keeps pre-round-21 persisted
  history working: a legacy native_part with a single subpart still
  replays the signature on that subpart; merged legacy objects pin
  the signature to executableCode only.

- Remote-image fetch threads the remaining per-request byte budget
  into _safe_fetch_image_for_gemini, so over-budget URLs are
  refused via Content-Length pre-check / short read instead of
  fully downloaded then discarded after the aggregate cap check.

- Gemini role=tool with OpenAI list-form content
  ([{type:text,text:result}]) now flattens text parts before
  building functionResponse.response.result; previously the parts
  arrived as the result value instead of the actual tool output.

- Frontend chat-adapter merges native_part by concatenating parts
  lists (preserving per-part thoughtSignature). Wire types expose
  enable_prompt_caching as boolean|string (Gemini cached-content
  name) and OpenAIChatDelta now carries tool_calls and extra_content.

- Test test_openrouter_no_synthetic_web_search_event_on_tool_choice_none
  reads _toolEvent from the top-level SSE payload so a backend
  regression cannot mask the assertion.

Adds 7 regression tests covering image_generation gate, forced-function
gate, native_part list replay, legacy fallback, list-content
functionResponse flattening, fetch byte-budget threading, and wire
types.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Apply forced-function tool_choice gate to Anthropic, OpenRouter, Kimi

Previously only the Gemini path treated `tool_choice={"type":"function",
"function":{"name":...}}` as a hosted-tool opt-out. Anthropic,
OpenRouter, and Kimi still attached hosted web_search / web_fetch /
code_execution when the caller explicitly pinned a user function plus
`enabled_tools=[...]`. That contradicts the explicit function pin and
bills the caller for unwanted server-side calls.

Mirror the Gemini gate symmetrically:
  - Anthropic web_search / web_fetch / code_execution
  - OpenRouter `plugins:[{id:"web"}]` + the synthetic web_search SSE
    event the same path emits at stream close
  - Kimi `_stream_kimi_web_search` dispatch

Adds 4 regression tests:
  - test_anthropic_forced_function_tool_choice_drops_hosted_tools
  - test_openrouter_forced_function_tool_choice_drops_web_plugin
  - test_kimi_forced_function_tool_choice_skips_web_search_helper
  - test_openrouter_no_synthetic_web_search_event_on_forced_function_tool_choice

All 146 existing backend tests still pass.

* Strip Gemini-only synthetic tool history on local-GGUF dispatch

After a Gemini chat that ran code_execution / image_generation, switching
the same thread to a local GGUF model used to forward the synthetic
provider-side tool_calls (tagged with `args._server_tool` or carrying a
Gemini `args.google.native_part` payload) and the message-level
`extra_content` to llama-server. The receiving backend has no tool
declaration for those names and no use for Gemini thoughtSignature
metadata; in the worst case it can produce an orphan tool_call_id and a
confused continuation.

Add `_strip_provider_synthetic_tool_history()` and wire it through the
two local message builders:
  - `_openai_messages_for_passthrough`  (OAI-compat passthrough)
  - `_openai_messages_for_gguf_chat`    (standard GGUF chat path)

Real user-function `tool_calls` and their matching `role="tool"` replies
survive unchanged; only synthetic provider-side cards and Gemini-only
`extra_content` are stripped. If the synthetic call was the assistant
turn's only payload, the now-empty turn is dropped too so llama-server
does not reject the request.

Adds 2 regression tests:
  - test_strip_provider_synthetic_tool_history_drops_synthetic_only
  - test_strip_provider_synthetic_tool_history_drops_empty_assistant

142 existing backend tests still pass.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Disable Search/Code composer pills for Gemini image-tier models

For external Gemini image-tier models (gemini-2.5-flash-image,
gemini-3.x-image-preview, etc.), the backend unconditionally strips
code_execution and strips web_search on older image ids. Search is
still allowed on Gemini 3.x Pro/Flash image models, which
supportsBuiltinWebSearch already encodes per model.

Before this commit the composer pill gates were:
  searchDisabled = !modelLoaded || !(supportsTools || supportsBuiltinWebSearch)
  codeDisabled   = !modelLoaded || !(supportsTools || supportsBuiltinCodeExecution) || imageModeDisablesCode

`supportsTools` here is a local-runtime fallback that becomes true when
any tool-capable local model has been loaded in the session. With a
local tool-capable runtime active, switching the chat to an external
Gemini image-tier model used to leave Search/Code clickable, even
though the backend will silently drop the tool on the wire.

Detect "external provider is Gemini AND the model is image-tier" (via
supportsBuiltinImageGeneration) and gate the two pills strictly on the
provider's own builtin support in that case. Non-Gemini paths and
non-image Gemini models keep the supportsTools fallback unchanged.

* Apply forced-function tool_choice gate to OpenAI Responses path

Round 22 added the gate for Gemini / Anthropic / OpenRouter / Kimi but
missed the OpenAI Responses translator. When a caller pinned a user
function via `tool_choice={"type":"function","function":{"name":...}}`
plus `enabled_tools=["web_search","code_execution","image_generation"]`,
the Responses body still attached `{"type":"web_search"}`,
`{"type":"shell"}`, and `{"type":"image_generation"}` server tools. The
function pin should suppress those for the same privacy + billing reason
the other provider paths now do.

Compute `_responses_tool_choice_forced_function` next to
`_responses_tool_choice_none` and gate each hosted-tool append on
`_responses_hosted_builtins_allowed = not none and not forced_function`.
The fix has to be applied in TWO places: the initial body builder and
`_build_body()` (called by the container-expiry retry path). User
function declarations still flow through so the pin has something to
target, and the Responses-shape `{type:"function", name:"..."}`
`tool_choice` is forwarded unchanged.

Adds regression test `test_openai_responses_forced_function_tool_choice_drops_hosted_tools`.
All 166 existing backend tests across Gemini + Responses + image-gen +
code-exec suites still pass.

* Round 24 P1s: SSRF shared-address gap + extra_content text-only leak + custom-Gemini model list

Three convergent P1s from round 24 review:

1. SSRF: the shared SSRF validator in `tools._validate_and_resolve_host`
   used a denylist (is_private / loopback / link_local / multicast /
   reserved / unspecified). Python classifies shared address space
   (100.64.0.0/10 carrier-grade NAT, plus 240.0.0.0/4, benchmarking
   ranges, etc.) with `is_private=False` AND `is_global=False`. The new
   Gemini server-side image fetcher therefore accepts URLs whose
   hostname resolves to 100.64.0.1 in cloud/VPC deployments. Add
   `not ip.is_global` as the primary gate -- a single source of truth
   that covers every current and future non-global range.

2. _strip_provider_synthetic_tool_history previously only stripped
   message-level `extra_content` when the assistant turn had tool_calls.
   A plain text Gemini reply carrying
   `extra_content.google.thought_signature` flowed through to
   llama-server when the thread was switched to a local GGUF backend.
   Always strip message-level `extra_content` on assistant turns.

3. routes/providers.list_provider_models applied Gemini's native
   `model_id_allowlist` regex to every Gemini provider, including
   custom OAI-compatible bases (LiteLLM, deployment gateways). IDs like
   `google/gemini-2.5-flash` and team-prefixed deployment aliases got
   filtered out even though the chat-dispatch path now routes them via
   the OpenAI-compatible client. Skip registry-level model-id filters
   when the configured Gemini base_url host is not the canonical
   `generativelanguage.googleapis.com`, mirroring the chat-dispatch
   gate.

Three regression tests added:
  - test_validate_and_resolve_host_blocks_shared_address_space
  - test_strip_provider_synthetic_tool_history_drops_text_only_extra_content
  - test_gemini_custom_oai_compat_base_skips_native_allowlist

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Round 25 P1s: skip synthetic server-tool replay + inline $ref/$defs into Gemini schema

Two convergent reviewer findings on the native Gemini path:

1. _stream_gemini's tool_calls replay loop falls through to a generic
   functionCall emission whenever it sees an assistant tool_call. Marked
   server-side builtin cards (web_search / web_fetch tagged with
   _server_tool or args.google.native_part) hit that fallthrough with no
   replayable native_part, which produces an outbound functionCall whose
   name is not a declared user function. The Gemini turn 400s on the
   undeclared name. Guard the loop to drop those entries instead, while
   keeping the existing code_execution / image_generation native-part
   replay branch intact.

2. _sanitize_gemini_schema uses a strict allowlist that drops local
   $ref / $defs references. Pydantic-generated tool schemas hoist nested
   object shapes into $defs and reference them via {"$ref": "#/$defs/X"},
   so a property like address: {"$ref": "#/$defs/Address"} collapsed to
   {} on the wire and the model lost the nested fields, types, and
   required keys. Resolve local #/... pointers against the schema root
   and inline the referenced subtree, with local siblings overriding
   the reference (normal JSON Schema composition) and a seen-ref guard
   for self-referential schemas.

Added regression coverage:
- test_gemini_native_skips_synthetic_server_builtin_replay
- test_function_declarations_inline_local_refs_into_gemini_schema
- test_function_declarations_inline_local_refs_in_anyof_and_items
- test_function_declarations_self_referential_schema_terminates

All 145 Gemini provider tests pass; touched provider regression set
(OpenAI Responses, code execution, image generation, Anthropic code
execution, Anthropic web_fetch) also 43/43 green.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Round 26 P1s: drop orphan Gemini functionResponse + Anthropic /messages synthetic-history strip

Reviewer round 26 surfaced two convergent asymmetric-fix bugs.

1. _stream_gemini drops a synthetic server-tool tool_call (web_search /
   web_fetch tagged _server_tool) and also replays code_execution /
   image_generation tool_calls as Gemini-native executableCode /
   codeExecutionResult / inlineData parts. The matching role="tool"
   follow-up was still falling through to the generic functionResponse
   branch, producing either an orphan functionResponse (synthetic case)
   or a duplicate response pointing at a name with no
   functionDeclarations entry (native-part case). Both forms 400 the
   next Gemini turn. Track skipped + native-replayed tool_call_ids in
   _gemini_skip_tool_result_ids and short-circuit the role="tool"
   branch on a match.

2. The Anthropic-compatible local /v1/messages route only called
   _drop_empty_assistant_sentinels on the OpenAI-translated history,
   while the sibling /v1/chat/completions and GGUF passthrough builders
   chain that with _strip_provider_synthetic_tool_history. An Anthropic
   caller replaying a prior provider-side tool_use therefore forwarded
   fake builtin tool history straight into local llama-server. Apply
   the same strip on the Anthropic route after the
   anthropic_messages_to_openai conversion.

Regression coverage added:
- test_gemini_native_skips_orphan_function_response_for_dropped_builtin
- test_gemini_native_skips_orphan_function_response_for_native_part_replay

Gemini suite 147/147; touched provider regression set 43/43.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Round 27 P1s: native_part location fallback + Gemini image request budget for base64

Two convergent reviewer findings on the native Gemini path.

1. _stream_gemini's synthetic-builtin detector at lines 3519-3524
   recognizes args.google.native_part as a server-tool marker, but
   _native_part was only loaded from tc.extra_content.google.native_part.
   A direct OpenAI-compatible API caller or imported third-party thread
   round-trips the payload through function.arguments because
   tool_calls[].extra_content is not in the OpenAI spec. The round-25
   guard then saw a synthetic builtin with no _native_part and dropped
   the entire assistant turn, so the next native Gemini request lost
   the prior executableCode / inlineData / codeExecutionResult context.
   Fall back to args.google.native_part when extra_content path is
   missing, mirroring what the synthetic detector already accepts.

2. _GEMINI_REMOTE_IMAGE_MAX_TOTAL_BYTES capped DECODED bytes at 20MB.
   Gemini receives images base64-encoded inside JSON, and base64
   inflates payload size by ~4/3. With 20MB decoded the actual JSON
   body is ~26.7MB plus prompt overhead, well over Gemini's ~20MB
   request limit. Drop the decoded cap to 14MB so realistic multi-
   image turns stay safely under 20MB encoded.

Added regression test test_gemini_native_part_falls_back_to_args_google
covering an OpenAI-compat-shaped image_generation tool_call whose
native_part lives only in function.arguments.

Gemini suite 148/148.

* Fix TS build errors from main merge: restore imageParts + refusal return [] + cast image-edit ref

Three errors in chat-adapter.ts surfaced by the frontend tsc step after merging
main into feat/gemini-provider:

1. The Anthropic refusal early-return used main's  but
   toOpenAIMessages returns SerializedMessage[]; flip to .
2. Restore  -- the line
   was lost when removing main's conflict block from the function body.
3. selectedImageEditReference splice was inserting OpenAIChatMessage
   into a SerializedMessage[] array; the shapes differ on tool_calls.id
   nullability. Cast the reference message through unknown -- it carries
   no tool_calls, so the runtime payload is structurally compatible.

Reproduced locally with `tsc -b --pretty false` (now passes). Build
also failing in the in-repo `npm run build` step on PR CI; this commit
unblocks all 12 failing UI/API workflows.

* Tighten verbose comments in external_provider.py + chat-adapter.ts

Compress multi-line explanatory comments in the Gemini translator
and the chat adapter without changing any behaviour. All 148 Gemini
provider tests still pass; tsc --noEmit clean.

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@users.noreply.github.com>
2026-05-27 06:01:24 -07:00

5501 lines
194 KiB
Python

# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
"""Unit tests for the native Gemini API translation layer.
Gemini does NOT speak OpenAI Chat Completions on its primary endpoint
(`streamGenerateContent`). `_stream_gemini` in
`core/inference/external_provider.py` translates between the two shapes:
Request:
OpenAI messages [{role, content}]
-> Gemini contents [{role, parts: [{text}|{inlineData}|{functionCall}|...]}]
+ systemInstruction.parts[].text for role=system messages
+ generationConfig.{temperature,topP,topK,maxOutputTokens}
+ tools[{googleSearch:{}}] for web_search
+ tools[{codeExecution:{}}] for code_execution
+ responseModalities=[TEXT,IMAGE] for Nano Banana (gemini-2.5-flash-image)
+ cachedContent for prompt caching
Response:
Gemini SSE chunks { candidates:[{content:{parts:[...]}, finishReason}],
usageMetadata:{promptTokenCount, candidatesTokenCount} }
-> OpenAI chat.completion.chunk frames
(delta.content for text, delta.tool_calls for functionCall,
_toolEvent for image_b64/web_search, usage block before [DONE])
These tests pin the outbound body shape AND the inbound translation
using httpx.MockTransport (no live network). Mirrors the structure of
test_anthropic_cache_ttl.py and test_openai_image_generation.py.
"""
import asyncio
import base64
import json
import httpx
import pytest
from core.inference import external_provider as ep_mod
from core.inference.external_provider import ExternalProviderClient
_active_mock_clients: list[httpx.AsyncClient] = []
def _drive(coro):
# Create a fresh loop per drive so tests don't share asyncio state.
# Close mocked clients + shutdown async-generators inside this loop
# so Python 3.13 doesn't emit the
# `Response.aiter_*.aclose was never awaited` warning on GC.
loop = asyncio.new_event_loop()
try:
result = loop.run_until_complete(coro)
while _active_mock_clients:
mc = _active_mock_clients.pop()
loop.run_until_complete(mc.aclose())
return result
finally:
try:
loop.run_until_complete(loop.shutdown_asyncgens())
finally:
loop.close()
def _make_gemini_client(
base_url: str = "https://generativelanguage.googleapis.com/v1beta",
) -> ExternalProviderClient:
return ExternalProviderClient(
provider_type = "gemini",
base_url = base_url,
api_key = "AIza-test-key",
)
def _mock_http(monkeypatch, handler):
mock_client = httpx.AsyncClient(transport = httpx.MockTransport(handler))
monkeypatch.setattr(ep_mod, "_http_client", mock_client)
# `_drive` will aclose this at the end of the run inside the same
# event loop so we do not leak an unawaited aclose() coroutine.
_active_mock_clients.append(mock_client)
def _gemini_sse(events: list[dict]) -> bytes:
"""Encode a list of dicts as Gemini-style SSE frames (`data:` lines)."""
chunks: list[str] = []
for event in events:
chunks.append(f"data: {json.dumps(event)}")
chunks.append("")
return ("\n".join(chunks) + "\n").encode("utf-8")
def _capture_body(monkeypatch, **kwargs) -> dict:
"""Drive a single stream and return the captured outbound request body."""
captured: dict = {}
def handler(request: httpx.Request) -> httpx.Response:
captured["body"] = json.loads(request.content.decode("utf-8"))
captured["headers"] = dict(request.headers)
captured["url"] = str(request.url)
captured["method"] = request.method
# Minimal valid Gemini stream so the helper can complete.
return httpx.Response(
200,
content = _gemini_sse(
[
{
"candidates": [
{
"content": {
"role": "model",
"parts": [{"text": "ok"}],
},
"finishReason": "STOP",
}
],
"usageMetadata": {
"promptTokenCount": 1,
"candidatesTokenCount": 1,
},
}
]
),
headers = {"content-type": "text/event-stream"},
)
_mock_http(monkeypatch, handler)
messages = kwargs.pop("messages", [{"role": "user", "content": "hi"}])
model = kwargs.pop("model", "gemini-2.5-flash")
temperature = kwargs.pop("temperature", 0.7)
top_p = kwargs.pop("top_p", 0.95)
max_tokens = kwargs.pop("max_tokens", 64)
async def run():
client = _make_gemini_client()
async for _ in client.stream_chat_completion(
messages = messages,
model = model,
temperature = temperature,
top_p = top_p,
max_tokens = max_tokens,
**kwargs,
):
pass
await client.close()
_drive(run())
return captured
def _collect(monkeypatch, sse_events, **kwargs) -> list[str]:
"""Drive a stream with a custom set of SSE events and return raw lines."""
def handler(request: httpx.Request) -> httpx.Response:
return httpx.Response(
200,
content = _gemini_sse(sse_events),
headers = {"content-type": "text/event-stream"},
)
_mock_http(monkeypatch, handler)
messages = kwargs.pop("messages", [{"role": "user", "content": "hi"}])
model = kwargs.pop("model", "gemini-2.5-flash")
temperature = kwargs.pop("temperature", 0.7)
top_p = kwargs.pop("top_p", 0.95)
max_tokens = kwargs.pop("max_tokens", 64)
out: list[str] = []
async def run():
client = _make_gemini_client()
async for line in client.stream_chat_completion(
messages = messages,
model = model,
temperature = temperature,
top_p = top_p,
max_tokens = max_tokens,
**kwargs,
):
out.append(line)
await client.close()
_drive(run())
return out
def _parse_chunks(lines: list[str]) -> list[dict]:
out: list[dict] = []
for raw in lines:
if not raw.startswith("data:"):
continue
payload = raw[len("data:") :].strip()
if not payload or payload == "[DONE]":
continue
try:
out.append(json.loads(payload))
except json.JSONDecodeError:
continue
return out
# ── request body translation ─────────────────────────────────────────
def test_request_body_uses_contents_and_parts_shape(monkeypatch):
"""OpenAI messages must be translated to Gemini's `contents` shape."""
captured = _capture_body(
monkeypatch,
messages = [
{"role": "system", "content": "Be brief."},
{"role": "user", "content": "Hello"},
{"role": "assistant", "content": "Hi there"},
{"role": "user", "content": "Follow up"},
],
)
body = captured["body"]
# system -> systemInstruction
assert body["systemInstruction"] == {"parts": [{"text": "Be brief."}]}, body
# user/assistant -> contents with role user/model
assert body["contents"] == [
{"role": "user", "parts": [{"text": "Hello"}]},
{"role": "model", "parts": [{"text": "Hi there"}]},
{"role": "user", "parts": [{"text": "Follow up"}]},
], body["contents"]
# generationConfig fields map across with Google's casing.
gc = body["generationConfig"]
assert gc["temperature"] == 0.7
assert gc["topP"] == 0.95
assert gc["maxOutputTokens"] == 64
def test_request_url_targets_stream_generate_content(monkeypatch):
"""Helper must POST to /v1beta/models/{model}:streamGenerateContent?alt=sse."""
captured = _capture_body(monkeypatch, model = "gemini-2.5-pro")
url = captured["url"]
assert ":streamGenerateContent" in url, url
assert "alt=sse" in url, url
assert "/v1beta/models/gemini-2.5-pro" in url, url
assert captured["method"] == "POST"
def test_request_auth_header_uses_x_goog_api_key(monkeypatch):
"""API key must be sent on `x-goog-api-key`, not Authorization."""
captured = _capture_body(monkeypatch)
hdrs = captured["headers"]
assert hdrs.get("x-goog-api-key") == "AIza-test-key", hdrs
assert "authorization" not in {k.lower() for k in hdrs}, hdrs
def test_top_k_forwarded_only_when_positive(monkeypatch):
"""top_k is opt-in; only positive integers reach the wire."""
captured = _capture_body(monkeypatch, top_k = 40)
assert captured["body"]["generationConfig"]["topK"] == 40
captured = _capture_body(monkeypatch, top_k = 0)
assert "topK" not in captured["body"]["generationConfig"]
def test_presence_penalty_forwarded_to_generation_config(monkeypatch):
"""A non-zero presence_penalty reaches generationConfig.presencePenalty."""
captured = _capture_body(monkeypatch, presence_penalty = 0.7)
assert captured["body"]["generationConfig"]["presencePenalty"] == 0.7
# And the default of zero is omitted, matching top_k semantics.
captured = _capture_body(monkeypatch, presence_penalty = 0.0)
assert "presencePenalty" not in captured["body"]["generationConfig"]
# ── thinkingConfig translation ────────────────────────────────────────
def test_gemini25_flash_thinking_disabled_sets_budget_zero(monkeypatch):
"""Gemini 2.5 Flash still uses thinkingBudget; 0 = off."""
captured = _capture_body(
monkeypatch,
model = "gemini-2.5-flash",
enable_thinking = False,
)
tc = captured["body"]["generationConfig"].get("thinkingConfig")
assert tc == {"thinkingBudget": 0}, tc
def test_gemini3_flash_thinking_disabled_uses_minimal_level(monkeypatch):
"""Gemini 3 Flash migrated to thinkingLevel; "off" maps to minimal
(Gemini 3 cannot turn thinking fully off)."""
captured = _capture_body(
monkeypatch,
model = "gemini-3.5-flash",
enable_thinking = False,
)
tc = captured["body"]["generationConfig"].get("thinkingConfig")
assert tc == {"thinkingLevel": "minimal"}, tc
def test_gemini25_pro_thinking_disabled_uses_small_budget(monkeypatch):
"""Gemini 2.5 Pro 400s on thinkingBudget=0 ("only works in thinking
mode"); coerce to a small positive budget."""
captured = _capture_body(
monkeypatch,
model = "gemini-2.5-pro",
enable_thinking = False,
)
tc = captured["body"]["generationConfig"].get("thinkingConfig")
assert tc is not None and tc.get("thinkingBudget", 0) > 0, tc
def test_gemini3_pro_thinking_disabled_uses_low_level(monkeypatch):
"""Gemini 3 Pro uses thinkingLevel and rejects 'minimal' (Pro tier),
so 'off' coerces to 'low' (lowest the API accepts)."""
for model in (
"gemini-3.1-pro-preview",
"gemini-3-pro-preview",
"gemini-3.5-pro",
"gemini-pro-latest",
):
captured = _capture_body(
monkeypatch,
model = model,
enable_thinking = False,
)
tc = captured["body"]["generationConfig"].get("thinkingConfig")
assert tc == {"thinkingLevel": "low"}, (model, tc)
def test_gemini25_flash_effort_levels_map_to_budgets(monkeypatch):
"""Gemini 2.5 Flash retains the integer thinkingBudget ladder."""
cases = {
"minimal": 512,
"low": 2048,
"medium": 8192,
"high": 24576,
"max": -1,
"xhigh": -1,
}
for effort, expected in cases.items():
captured = _capture_body(
monkeypatch,
model = "gemini-2.5-flash",
reasoning_effort = effort,
)
tc = captured["body"]["generationConfig"].get("thinkingConfig")
assert tc == {"thinkingBudget": expected}, (effort, tc)
def test_gemini3_flash_effort_levels_map_to_thinking_level(monkeypatch):
"""Gemini 3 Flash thinkingLevel ladder: minimal/low/medium/high."""
cases = {
"minimal": "minimal",
"low": "low",
"medium": "medium",
"high": "high",
"max": "high",
}
for effort, expected in cases.items():
captured = _capture_body(
monkeypatch,
model = "gemini-3.5-flash",
reasoning_effort = effort,
)
tc = captured["body"]["generationConfig"].get("thinkingConfig")
assert tc == {"thinkingLevel": expected}, (effort, tc)
def test_gemini3_pro_passes_medium_through(monkeypatch):
"""Gemini 3.1+ Pro accepts thinkingLevel="medium" per
https://docs.cloud.google.com/vertex-ai/generative-ai/docs/models/gemini/3-1-pro;
forward as-is (medium is the documented mid-tier on Gemini 3.1)."""
for model in (
"gemini-3.1-pro-preview",
"gemini-pro-latest",
):
captured = _capture_body(
monkeypatch,
model = model,
reasoning_effort = "medium",
)
tc = captured["body"]["generationConfig"].get("thinkingConfig")
assert tc == {"thinkingLevel": "medium"}, (model, tc)
def test_gemini3_pro_minimal_effort_coerces_to_low(monkeypatch):
"""Gemini 3 Pro rejects thinkingLevel="minimal"; coerce to "low"."""
captured = _capture_body(
monkeypatch,
model = "gemini-3.1-pro-preview",
reasoning_effort = "minimal",
)
tc = captured["body"]["generationConfig"].get("thinkingConfig")
assert tc == {"thinkingLevel": "low"}, tc
def test_gemini3_flash_effort_none_maps_to_minimal(monkeypatch):
"""reasoning_effort='none' on Gemini 3 Flash -> thinkingLevel=minimal."""
captured = _capture_body(
monkeypatch,
model = "gemini-3.5-flash",
reasoning_effort = "none",
)
tc = captured["body"]["generationConfig"].get("thinkingConfig")
assert tc == {"thinkingLevel": "minimal"}, tc
def test_thinking_default_omits_thinking_config(monkeypatch):
"""When neither knob is supplied, thinkingConfig is omitted entirely
(Google's server-side default applies)."""
captured = _capture_body(monkeypatch, model = "gemini-3.5-flash")
gc = captured["body"]["generationConfig"]
assert "thinkingConfig" not in gc, gc
def test_nano_banana_alias_routes_through_image_modalities(monkeypatch):
"""`nano-banana-pro-preview` is an alias for the Pro image model and
must set responseModalities=[TEXT,IMAGE] when the Images pill is on
(enabled_tools includes "image_generation")."""
captured = _capture_body(
monkeypatch,
model = "nano-banana-pro-preview",
enabled_tools = ["image_generation"],
)
gc = captured["body"]["generationConfig"]
assert gc.get("responseModalities") == ["TEXT", "IMAGE"], gc
def test_image_capable_model_without_image_pill_stays_text_only(monkeypatch):
"""When the Images pill is off (enabled_tools has no
image_generation), an image-capable model id (gemini-2.5-flash-image)
must force responseModalities=["TEXT"]. Google's image models
default to text+image when responseModalities is omitted, so
omitting it would silently bill image output the UI says is
disabled."""
captured = _capture_body(
monkeypatch,
model = "gemini-2.5-flash-image",
enabled_tools = [],
)
gc = captured["body"]["generationConfig"]
assert gc.get("responseModalities") == ["TEXT"], gc
def test_image_models_skip_thinking_config(monkeypatch):
"""Image-tier ids do not benefit from a visible thinking knob and
must NOT forward thinkingConfig even when stale UI state still
sends `reasoning_effort` or `enable_thinking=False`."""
for model in (
"gemini-2.5-flash-image",
"gemini-3.1-flash-image-preview",
"gemini-3-pro-image-preview",
"nano-banana-pro-preview",
):
captured = _capture_body(
monkeypatch,
model = model,
reasoning_effort = "high",
enable_thinking = False,
enabled_tools = ["image_generation"],
)
gc = captured["body"]["generationConfig"]
assert "thinkingConfig" not in gc, (model, gc)
def test_image_models_drop_code_execution(monkeypatch):
"""All image-tier ids reject `tools: [{codeExecution: {}}]`; drop
silently. (Gemini 3 image models DO accept googleSearch -- see
test_gemini3_image_models_allow_google_search; older image models
drop everything.)"""
for model in (
"gemini-2.5-flash-image",
"gemini-3.1-flash-image-preview",
"gemini-3-pro-image-preview",
"nano-banana-pro-preview",
):
captured = _capture_body(
monkeypatch,
model = model,
enabled_tools = ["image_generation", "code_execution"],
)
tools_arr = captured["body"].get("tools") or []
names = [list(t.keys())[0] for t in tools_arr]
assert "codeExecution" not in names, (model, tools_arr)
def test_gemini_35_pro_uses_thinking_level(monkeypatch):
"""`gemini-3.5-pro` is part of the Gemini 3 family and uses
thinkingLevel (not thinkingBudget). "Off" maps to "low" because Pro
tier rejects "minimal"."""
captured = _capture_body(
monkeypatch,
model = "gemini-3.5-pro",
enable_thinking = False,
)
tc = captured["body"]["generationConfig"].get("thinkingConfig")
assert tc == {"thinkingLevel": "low"}, tc
def test_gemini3_image_models_allow_google_search(monkeypatch):
"""Google documents Search grounding on the Gemini 3 image family
(gemini-3-pro-image-preview, gemini-3.1-flash-image-preview,
nano-banana-pro). codeExecution stays blocked on image mode."""
for model in (
"gemini-3-pro-image-preview",
"gemini-3.1-flash-image-preview",
"nano-banana-pro-preview",
):
captured = _capture_body(
monkeypatch,
model = model,
enabled_tools = ["image_generation", "web_search", "code_execution"],
)
tools_arr = captured["body"].get("tools") or []
names = [list(t.keys())[0] for t in tools_arr]
assert "googleSearch" in names, (model, tools_arr)
assert "codeExecution" not in names, (model, tools_arr)
def test_legacy_image_models_block_google_search(monkeypatch):
"""Older Gemini image ids (gemini-2.5-flash-image) still 400 on
`tools: [{googleSearch: {}}]`; backend keeps stripping it."""
captured = _capture_body(
monkeypatch,
model = "gemini-2.5-flash-image",
enabled_tools = ["image_generation", "web_search", "code_execution"],
)
assert "tools" not in captured["body"], captured["body"].get("tools")
def test_legacy_openai_base_url_normalized(monkeypatch):
"""Saved Gemini providers carrying the legacy `/v1beta/openai` base
(from the pre-PR OpenAI-compat plumbing) now point at the native
endpoint without the user re-saving the connection."""
client = ExternalProviderClient(
provider_type = "gemini",
base_url = "https://generativelanguage.googleapis.com/v1beta/openai",
api_key = "AIza-test-key",
)
assert client.base_url == "https://generativelanguage.googleapis.com/v1beta"
def test_finish_reason_swaps_to_tool_calls_when_function_call_emitted(monkeypatch):
"""Gemini emits finishReason="STOP" even for pure functionCall turns;
surface as `tool_calls` so OAI clients trigger tool execution."""
sse = [
{
"candidates": [
{
"content": {
"role": "model",
"parts": [
{"functionCall": {"name": "lookup", "args": {"k": "v"}}}
],
},
"finishReason": "STOP",
}
]
}
]
lines = _collect(monkeypatch, sse)
chunks = _parse_chunks(lines)
finish_chunks = [
c for c in chunks if c.get("choices", [{}])[0].get("finish_reason") is not None
]
assert finish_chunks, chunks
assert finish_chunks[-1]["choices"][0]["finish_reason"] == "tool_calls", chunks
def test_thought_signature_round_trips_into_gemini_function_call(monkeypatch):
"""An assistant tool_call carrying `extra_content.google.thought_signature`
must echo the value back as a sibling of the Gemini functionCall part."""
captured = _capture_body(
monkeypatch,
messages = [
{"role": "user", "content": "lookup x"},
{
"role": "assistant",
"content": "",
"tool_calls": [
{
"id": "call_0",
"type": "function",
"function": {"name": "lookup", "arguments": "{}"},
"extra_content": {"google": {"thought_signature": "SIG-ABC"}},
}
],
},
{
"role": "tool",
"tool_call_id": "call_0",
"name": "lookup",
"content": "{}",
},
],
)
contents = captured["body"]["contents"]
fc_turn = next((c for c in contents if c["role"] == "model"), None)
assert fc_turn is not None, contents
fc_part = next(
(p for p in fc_turn["parts"] if "functionCall" in p),
None,
)
assert fc_part is not None, fc_turn
assert fc_part.get("thoughtSignature") == "SIG-ABC", fc_part
def test_thought_signature_emitted_in_tool_call_delta(monkeypatch):
"""A Gemini functionCall part with `thoughtSignature` must surface
that signature on the outbound OpenAI tool_calls delta via
`extra_content.google.thought_signature`."""
sse = [
{
"candidates": [
{
"content": {
"role": "model",
"parts": [
{
"functionCall": {
"name": "lookup",
"args": {"k": "v"},
"id": "call_xyz",
},
"thoughtSignature": "SIG-FROM-GEMINI",
}
],
},
"finishReason": "STOP",
}
]
}
]
chunks = _parse_chunks(_collect(monkeypatch, sse))
deltas = [
tc
for c in chunks
for tc in (c.get("choices", [{}])[0].get("delta", {}) or {}).get(
"tool_calls", []
)
]
assert deltas, chunks
sig = deltas[0].get("extra_content", {}).get("google", {}).get("thought_signature")
assert sig == "SIG-FROM-GEMINI", deltas
def test_image_models_suppress_phantom_web_search_card(monkeypatch):
"""When the image guard filters googleSearch out of the outbound
request, the inbound stream must NOT emit web_search tool_start /
tool_end (otherwise the UI shows a misleading 'Search complete'
card on a turn where Gemini never actually searched)."""
sse = [
{
"candidates": [
{
"content": {"role": "model", "parts": [{"text": "drawn"}]},
"finishReason": "STOP",
}
]
}
]
lines = _collect(
monkeypatch,
sse,
model = "gemini-2.5-flash-image",
enabled_tools = ["image_generation", "web_search", "code_execution"],
)
chunks = _parse_chunks(lines)
tool_evs = [
ev
for c in chunks
for ev in [c.get("_toolEvent")]
if isinstance(ev, dict) and ev.get("tool_name") == "web_search"
]
assert tool_evs == [], tool_evs
def test_image_generation_tool_on_image_model_drops_text_tools(monkeypatch):
"""`enabled_tools=["image_generation", "web_search", "code_execution"]`
on a Gemini IMAGE model flips responseModalities to TEXT+IMAGE; in
that mode codeExecution must NOT be forwarded (Gemini rejects text
code tools alongside image responseModalities). Older image
families also drop googleSearch."""
captured = _capture_body(
monkeypatch,
model = "gemini-2.5-flash-image",
enabled_tools = [
"image_generation",
"web_search",
"code_execution",
],
)
assert "tools" not in captured["body"], captured["body"]
assert captured["body"]["generationConfig"].get("responseModalities") == [
"TEXT",
"IMAGE",
]
def test_prompt_feedback_block_reason_surfaces_as_error(monkeypatch):
"""`promptFeedback.blockReason` with zero candidates must produce
an error chunk, not a silent empty assistant reply."""
sse = [
{
"promptFeedback": {"blockReason": "SAFETY"},
}
]
chunks = _parse_chunks(_collect(monkeypatch, sse))
error_chunks = [c for c in chunks if "error" in c]
assert error_chunks, chunks
assert "SAFETY" in (
error_chunks[0].get("error", {}).get("message") or ""
), error_chunks
def test_usage_chunk_includes_thoughts_tokens(monkeypatch):
"""`thoughtsTokenCount` is the hidden-reasoning slice of output;
roll it into `output_tokens` AND surface it on
`output_tokens_details.reasoning_tokens` so total_tokens reflects
the full billable spend."""
sse = [
{
"candidates": [
{
"content": {"role": "model", "parts": [{"text": "ok"}]},
"finishReason": "STOP",
}
],
"usageMetadata": {
"promptTokenCount": 10,
"candidatesTokenCount": 5,
"thoughtsTokenCount": 20,
"totalTokenCount": 35,
},
}
]
chunks = _parse_chunks(_collect(monkeypatch, sse))
usage_chunk = next((c for c in chunks if isinstance(c.get("usage"), dict)), None)
assert usage_chunk is not None, chunks
usage = usage_chunk["usage"]
assert usage.get("prompt_tokens") == 10, usage
# candidates 5 + thoughts 20 = 25 output tokens; total = 35.
assert usage.get("completion_tokens") == 25, usage
assert usage.get("total_tokens") == 35, usage
# ── web_search forwarded as googleSearch tool ────────────────────────
def test_web_search_forwarded_as_google_search_tool(monkeypatch):
captured = _capture_body(
monkeypatch,
enabled_tools = ["web_search"],
)
tools = captured["body"].get("tools") or []
assert {"googleSearch": {}} in tools, tools
def test_code_execution_forwarded_as_code_execution_tool(monkeypatch):
captured = _capture_body(
monkeypatch,
enabled_tools = ["code_execution"],
)
tools = captured["body"].get("tools") or []
assert {"codeExecution": {}} in tools, tools
def test_omitted_tools_leaves_body_untouched(monkeypatch):
captured = _capture_body(monkeypatch, enabled_tools = [])
assert "tools" not in captured["body"], captured["body"]
# ── prompt caching passthrough ───────────────────────────────────────
def test_cached_content_pass_through(monkeypatch):
"""A string cache id on enable_prompt_caching is forwarded verbatim."""
cache_name = "cachedContents/abc123"
captured = _capture_body(
monkeypatch,
enable_prompt_caching = cache_name,
)
assert captured["body"].get("cachedContent") == cache_name
def test_boolean_caching_does_not_set_cached_content(monkeypatch):
"""Studio's existing True/False signals shouldn't fabricate a cache id."""
captured = _capture_body(monkeypatch, enable_prompt_caching = True)
assert "cachedContent" not in captured["body"]
# ── image generation: request modalities + response translation ──────
def test_image_model_sets_response_modalities(monkeypatch):
captured = _capture_body(
monkeypatch,
model = "gemini-2.5-flash-image",
enabled_tools = ["image_generation"],
)
assert captured["body"]["generationConfig"]["responseModalities"] == [
"TEXT",
"IMAGE",
]
def test_image_generation_tool_sets_response_modalities_on_image_model(monkeypatch):
"""`enabled_tools=["image_generation"]` flips responseModalities
only when the selected model is image-capable; otherwise the
request stays plain text (text-only models 400 on
responseModalities)."""
captured = _capture_body(
monkeypatch,
model = "gemini-2.5-flash-image",
enabled_tools = ["image_generation"],
)
assert captured["body"]["generationConfig"]["responseModalities"] == [
"TEXT",
"IMAGE",
]
def test_image_response_emits_image_b64_tool_event(monkeypatch):
"""`inlineData` parts become a tool_end with image_b64 + image_mime."""
fake_b64 = base64.b64encode(b"PNG-BYTES").decode()
sse = [
{
"candidates": [
{
"content": {
"role": "model",
"parts": [
{
"inlineData": {
"mimeType": "image/png",
"data": fake_b64,
}
}
],
},
"finishReason": "STOP",
}
],
"usageMetadata": {
"promptTokenCount": 5,
"candidatesTokenCount": 0,
},
}
]
lines = _collect(
monkeypatch,
sse,
model = "gemini-2.5-flash-image",
)
chunks = _parse_chunks(lines)
tool_events = [c["_toolEvent"] for c in chunks if "_toolEvent" in c]
starts = [e for e in tool_events if e.get("type") == "tool_start"]
ends = [e for e in tool_events if e.get("type") == "tool_end"]
image_starts = [e for e in starts if e.get("tool_name") == "image_generation"]
image_ends = [e for e in ends if e.get("image_b64")]
assert len(image_starts) == 1, tool_events
assert len(image_ends) == 1, tool_events
assert image_ends[0]["image_b64"] == fake_b64
assert image_ends[0]["image_mime"] == "image/png"
# ── function calling round-trips both directions ─────────────────────
def test_function_call_response_translates_to_tool_calls_delta(monkeypatch):
"""Gemini `functionCall` parts become OpenAI `tool_calls` delta chunks."""
sse = [
{
"candidates": [
{
"content": {
"role": "model",
"parts": [
{
"functionCall": {
"name": "get_weather",
"args": {"location": "Paris"},
}
}
],
},
"finishReason": "STOP",
}
],
"usageMetadata": {
"promptTokenCount": 12,
"candidatesTokenCount": 4,
},
}
]
lines = _collect(monkeypatch, sse)
chunks = _parse_chunks(lines)
tool_call_chunks = [
c
for c in chunks
if "_toolEvent" not in c
and any(
(isinstance(ch.get("delta"), dict) and "tool_calls" in ch["delta"])
for ch in c.get("choices", [])
)
]
assert len(tool_call_chunks) == 1, chunks
tc = tool_call_chunks[0]["choices"][0]["delta"]["tool_calls"][0]
assert tc["function"]["name"] == "get_weather"
args = json.loads(tc["function"]["arguments"])
assert args == {"location": "Paris"}
def test_tool_message_translates_to_function_response_part(monkeypatch):
"""role=tool follow-ups are rewritten to functionResponse parts."""
messages = [
{"role": "user", "content": "Weather?"},
{
"role": "assistant",
"content": "",
"tool_calls": [
{
"id": "call_1",
"type": "function",
"function": {
"name": "get_weather",
"arguments": json.dumps({"location": "Paris"}),
},
}
],
},
{
"role": "tool",
"name": "get_weather",
"content": json.dumps({"temp_c": 18, "summary": "Sunny"}),
},
]
captured = _capture_body(monkeypatch, messages = messages)
contents = captured["body"]["contents"]
# Last turn must be a functionResponse part (Gemini wraps it as a
# role=user turn carrying the result).
last = contents[-1]
assert last["role"] == "user", last
fr = last["parts"][0].get("functionResponse")
assert fr is not None, last
assert fr["name"] == "get_weather"
assert fr["response"] == {"temp_c": 18, "summary": "Sunny"}
# And the assistant turn carries the original functionCall so the
# model sees the round-trip context.
assistant_turn = [c for c in contents if c["role"] == "model"][0]
fc_part = next(
(p for p in assistant_turn["parts"] if "functionCall" in p),
None,
)
assert fc_part is not None, assistant_turn
assert fc_part["functionCall"]["name"] == "get_weather"
assert fc_part["functionCall"]["args"] == {"location": "Paris"}
def test_parallel_function_calls_get_distinct_tool_call_indices(monkeypatch):
"""Each emitted functionCall in one assistant turn needs its own
tool_calls[*].index. Hardcoding index=0 collapses parallel calls
onto a single slot in OpenAI-style reassemblers."""
sse = [
{
"candidates": [
{
"content": {
"role": "model",
"parts": [
{
"functionCall": {
"id": "call_alpha",
"name": "search",
"args": {"q": "alpha"},
}
},
{
"functionCall": {
"id": "call_beta",
"name": "search",
"args": {"q": "beta"},
}
},
],
},
"finishReason": "STOP",
}
],
"usageMetadata": {
"promptTokenCount": 8,
"candidatesTokenCount": 4,
},
}
]
lines = _collect(monkeypatch, sse)
chunks = _parse_chunks(lines)
tool_call_chunks = [
c
for c in chunks
if "_toolEvent" not in c
and any(
(isinstance(ch.get("delta"), dict) and "tool_calls" in ch["delta"])
for ch in c.get("choices", [])
)
]
assert len(tool_call_chunks) == 2, tool_call_chunks
indices = [
c["choices"][0]["delta"]["tool_calls"][0]["index"] for c in tool_call_chunks
]
assert indices == [0, 1], indices
def test_function_call_ids_forwarded_into_gemini_function_call_part(monkeypatch):
"""OpenAI tool_call id rides functionCall.id so parallel calls disambiguate."""
messages = [
{"role": "user", "content": "x"},
{
"role": "assistant",
"content": "",
"tool_calls": [
{
"id": "call_alpha",
"type": "function",
"function": {
"name": "search",
"arguments": json.dumps({"q": "a"}),
},
},
{
"id": "call_beta",
"type": "function",
"function": {
"name": "search",
"arguments": json.dumps({"q": "b"}),
},
},
],
},
{
"role": "tool",
"tool_call_id": "call_alpha",
"content": json.dumps({"hits": ["A"]}),
},
{
"role": "tool",
"tool_call_id": "call_beta",
"content": json.dumps({"hits": ["B"]}),
},
]
captured = _capture_body(monkeypatch, messages = messages)
contents = captured["body"]["contents"]
assistant_parts = next(c for c in contents if c["role"] == "model")["parts"]
call_ids = [p["functionCall"]["id"] for p in assistant_parts if "functionCall" in p]
assert call_ids == ["call_alpha", "call_beta"], assistant_parts
response_ids = [
p["functionResponse"]["id"]
for c in contents
for p in c["parts"]
if "functionResponse" in p
]
assert response_ids == ["call_alpha", "call_beta"], contents
def test_parse_gemini_models_translates_native_catalog():
"""Gemini's native /v1beta/models payload becomes OpenAI-shape entries."""
payload = {
"models": [
{
"name": "models/gemini-2.5-flash",
"baseModelId": "gemini-2.5-flash",
"displayName": "Gemini 2.5 Flash",
"supportedGenerationMethods": [
"generateContent",
"streamGenerateContent",
],
},
{
"name": "models/embedding-001",
"supportedGenerationMethods": ["embedContent"],
},
{
"name": "models/gemini-2.5-pro",
},
]
}
out = ExternalProviderClient._parse_gemini_models(payload)
ids = [m["id"] for m in out]
assert "gemini-2.5-flash" in ids
assert "gemini-2.5-pro" in ids
assert "embedding-001" not in ids
flash = next(m for m in out if m["id"] == "gemini-2.5-flash")
assert flash["display_name"] == "Gemini 2.5 Flash"
assert flash["owned_by"] == "google"
def test_code_execution_parts_translate_to_code_execution_tool_events(monkeypatch):
"""executableCode + codeExecutionResult parts emit code_execution events."""
sse = [
{
"candidates": [
{
"content": {
"role": "model",
"parts": [
{
"executableCode": {
"language": "PYTHON",
"code": "print(2+2)",
}
},
{
"codeExecutionResult": {
"outcome": "OUTCOME_OK",
"output": "4\n",
}
},
],
},
"finishReason": "STOP",
}
],
"usageMetadata": {
"promptTokenCount": 8,
"candidatesTokenCount": 4,
},
}
]
lines = _collect(monkeypatch, sse, enabled_tools = ["code_execution"])
chunks = _parse_chunks(lines)
tool_events = [c["_toolEvent"] for c in chunks if "_toolEvent" in c]
code_starts = [
e
for e in tool_events
if e.get("type") == "tool_start" and e.get("tool_name") == "code_execution"
]
code_ends = [
e
for e in tool_events
if e.get("type") == "tool_end" and "4" in str(e.get("result", ""))
]
assert len(code_starts) == 1, tool_events
assert code_starts[0]["arguments"]["code"] == "print(2+2)"
assert code_starts[0]["arguments"]["language"] == "python"
assert len(code_ends) == 1, tool_events
# tool_start and tool_end must share the same tool_call_id so the
# frontend pairs them onto a single CodeExecutionToolUI block.
assert code_starts[0]["tool_call_id"] == code_ends[0]["tool_call_id"]
def test_code_execution_failure_outcome_surfaces_in_result(monkeypatch):
"""OUTCOME_FAILED is prefixed onto the result text so the UI shows it."""
sse = [
{
"candidates": [
{
"content": {
"role": "model",
"parts": [
{
"executableCode": {
"language": "PYTHON",
"code": "1/0",
}
},
{
"codeExecutionResult": {
"outcome": "OUTCOME_FAILED",
"output": "ZeroDivisionError",
}
},
],
},
"finishReason": "STOP",
}
],
"usageMetadata": {
"promptTokenCount": 5,
"candidatesTokenCount": 2,
},
}
]
lines = _collect(monkeypatch, sse, enabled_tools = ["code_execution"])
chunks = _parse_chunks(lines)
tool_events = [c["_toolEvent"] for c in chunks if "_toolEvent" in c]
result_text = next(
(e["result"] for e in tool_events if e.get("type") == "tool_end"),
"",
)
assert "OUTCOME_FAILED" in result_text
assert "ZeroDivisionError" in result_text
def test_tool_message_recovers_name_from_tool_call_id(monkeypatch):
"""When name is omitted, recover it from the matching tool_call_id."""
messages = [
{"role": "user", "content": "Weather?"},
{
"role": "assistant",
"content": "",
"tool_calls": [
{
"id": "call_xyz",
"type": "function",
"function": {
"name": "get_weather",
"arguments": json.dumps({"location": "Paris"}),
},
}
],
},
{
"role": "tool",
"tool_call_id": "call_xyz",
"content": json.dumps({"temp_c": 18}),
},
]
captured = _capture_body(monkeypatch, messages = messages)
contents = captured["body"]["contents"]
last = contents[-1]
fr = last["parts"][0].get("functionResponse")
assert fr is not None, last
assert (
fr["name"] == "get_weather"
), "name should fall back to the prior tool_call's function name"
# ── usage chunk surfaces promptTokenCount / candidatesTokenCount ─────
def test_usage_chunk_translates_gemini_token_counts(monkeypatch):
sse = [
{
"candidates": [
{
"content": {
"role": "model",
"parts": [{"text": "ok"}],
},
"finishReason": "STOP",
}
],
"usageMetadata": {
"promptTokenCount": 1234,
"candidatesTokenCount": 56,
"cachedContentTokenCount": 1000,
},
}
]
lines = _collect(monkeypatch, sse)
chunks = _parse_chunks(lines)
usage_chunks = [c for c in chunks if c.get("choices") == [] and "usage" in c]
assert len(usage_chunks) == 1, chunks
usage = usage_chunks[0]["usage"]
assert usage["prompt_tokens"] == 1234
assert usage["completion_tokens"] == 56
assert usage["total_tokens"] == 1290
assert usage["prompt_tokens_details"]["cached_tokens"] == 1000
# ── multimodal: vision image -> inlineData ───────────────────────────
def test_vision_data_url_translates_to_inline_data(monkeypatch):
fake = base64.b64encode(b"JPGBYTES").decode()
messages = [
{
"role": "user",
"content": [
{"type": "text", "text": "What is this?"},
{
"type": "image_url",
"image_url": {
"url": f"data:image/jpeg;base64,{fake}",
},
},
],
}
]
captured = _capture_body(monkeypatch, messages = messages)
parts = captured["body"]["contents"][0]["parts"]
inline_parts = [p for p in parts if "inlineData" in p]
assert len(inline_parts) == 1, parts
assert inline_parts[0]["inlineData"] == {
"mimeType": "image/jpeg",
"data": fake,
}
# ── finish reason mapping ────────────────────────────────────────────
@pytest.mark.parametrize(
"gemini_reason, openai_reason",
[
("STOP", "stop"),
("MAX_TOKENS", "length"),
("SAFETY", "content_filter"),
("PROHIBITED_CONTENT", "content_filter"),
],
)
def test_finish_reason_translation(monkeypatch, gemini_reason, openai_reason):
sse = [
{
"candidates": [
{
"content": {
"role": "model",
"parts": [{"text": "x"}],
},
"finishReason": gemini_reason,
}
],
"usageMetadata": {
"promptTokenCount": 1,
"candidatesTokenCount": 1,
},
}
]
lines = _collect(monkeypatch, sse)
chunks = _parse_chunks(lines)
finish_chunks = [
c for c in chunks if any(ch.get("finish_reason") for ch in c.get("choices", []))
]
assert any(
ch["choices"][0]["finish_reason"] == openai_reason for ch in finish_chunks
), finish_chunks
# ── grounding citations surface as web_search tool_end ───────────────
def test_grounding_metadata_surfaces_as_tool_end_citations(monkeypatch):
"""`groundingMetadata.groundingChunks[].web` -> tool_end result block."""
sse = [
{
"candidates": [
{
"content": {
"role": "model",
"parts": [{"text": "Answer with sources."}],
},
"groundingMetadata": {
"groundingChunks": [
{
"web": {
"uri": "https://example.com/a",
"title": "Example A",
}
},
{
"web": {
"uri": "https://example.com/b",
"title": "Example B",
}
},
]
},
"finishReason": "STOP",
}
],
"usageMetadata": {
"promptTokenCount": 7,
"candidatesTokenCount": 3,
},
}
]
lines = _collect(
monkeypatch,
sse,
enabled_tools = ["web_search"],
)
chunks = _parse_chunks(lines)
tool_events = [c["_toolEvent"] for c in chunks if "_toolEvent" in c]
web_search_ends = [
e
for e in tool_events
if e.get("type") == "tool_end" and e.get("tool_call_id") == "gemini_web_search"
]
assert len(web_search_ends) == 1, tool_events
result = web_search_ends[0]["result"]
assert "https://example.com/a" in result
assert "https://example.com/b" in result
assert "Example A" in result
assert "Example B" in result
# ── round 3 review follow-ups ─────────────────────────────────────────
def test_custom_gemini_proxy_base_url_not_rewritten():
"""Only the Google-hosted /v1beta/openai base is normalized; a
custom gateway whose path ends in /openai must be left alone."""
client = ExternalProviderClient(
provider_type = "gemini",
base_url = "https://proxy.example.com/team/openai",
api_key = "AIza-test-key",
)
assert client.base_url == "https://proxy.example.com/team/openai"
def test_custom_gemini_proxy_uses_openai_dispatch():
"""Any non-Google Gemini base (LiteLLM, custom OpenAI-compat
routers) must route through the OpenAI-compatible forwarder, not
the native translator. Auth uses Authorization: Bearer ..., not
x-goog-api-key."""
for base in (
"https://proxy.example.com/team/openai",
"https://proxy.example.com/v1",
"https://litellm.internal.example/v1",
):
client = ExternalProviderClient(
provider_type = "gemini",
base_url = base,
api_key = "AIza-test-key",
)
assert client._is_openai_compatible() is True, base
headers = client._auth_headers()
assert "x-goog-api-key" not in {k.lower() for k in headers}, (
base,
headers,
)
assert headers["Authorization"] == "Bearer AIza-test-key", (
base,
headers,
)
def test_google_hosted_gemini_still_uses_native_dispatch():
"""Google-hosted Gemini keeps native dispatch + x-goog-api-key auth."""
client = ExternalProviderClient(
provider_type = "gemini",
base_url = "https://generativelanguage.googleapis.com/v1beta",
api_key = "AIza-test-key",
)
assert client._is_openai_compatible() is False
headers = client._auth_headers()
assert headers.get("x-goog-api-key") == "AIza-test-key", headers
def test_invalid_gemini_model_id_rejected_before_request(monkeypatch):
"""Path-traversal model ids must be rejected before the URL is
interpolated so the configured API key isn't sent to unintended
Gemini endpoints."""
captured: list[httpx.Request] = []
def handler(request: httpx.Request) -> httpx.Response:
captured.append(request)
return httpx.Response(
200,
content = _gemini_sse([]),
headers = {"content-type": "text/event-stream"},
)
_mock_http(monkeypatch, handler)
out: list[str] = []
async def run():
client = _make_gemini_client()
async for line in client.stream_chat_completion(
messages = [{"role": "user", "content": "hi"}],
model = "../cachedContents/leak",
temperature = 0.7,
top_p = 0.95,
max_tokens = 16,
):
out.append(line)
await client.close()
_drive(run())
# No outbound request should have been issued.
assert captured == [], captured
error_lines = [line for line in out if '"error"' in line]
assert error_lines, out
def test_top_k_omitted_when_not_explicit_default_for_gemini(monkeypatch):
"""top_k=None means "use provider default"; helper must not emit
`topK` in generationConfig when the caller didn't pass it."""
captured = _capture_body(monkeypatch, top_k = None)
assert "topK" not in captured["body"]["generationConfig"], captured["body"]
def test_text_model_image_generation_tool_silently_dropped(monkeypatch):
"""A stale `enabled_tools=["image_generation"]` on a text-only
Gemini model (e.g. gemini-2.5-flash) must NOT switch the request
into image mode -- Google's API 400s on responseModalities for
text models."""
captured = _capture_body(
monkeypatch,
model = "gemini-2.5-flash",
enabled_tools = ["image_generation"],
)
gc = captured["body"]["generationConfig"]
assert "responseModalities" not in gc, gc
def test_empty_text_part_with_thought_signature_emits_extra_content(
monkeypatch,
):
"""Gemini 3 can ship a content-free fragment whose only payload is
`thoughtSignature`. The translator must still surface that signature
on a delta.extra_content envelope so the next turn can replay it."""
sse = [
{
"candidates": [
{
"content": {
"role": "model",
"parts": [
{"text": "answer"},
{"thoughtSignature": "SIG-FINAL"},
],
},
"finishReason": "STOP",
}
],
"usageMetadata": {
"promptTokenCount": 2,
"candidatesTokenCount": 1,
},
}
]
lines = _collect(monkeypatch, sse)
chunks = _parse_chunks(lines)
extra_carriers = [
c
for c in chunks
if c.get("choices")
and c["choices"][0]["delta"].get("extra_content")
== {"google": {"thought_signature": "SIG-FINAL"}}
]
assert extra_carriers, chunks
def test_enable_prompt_caching_false_string_coerces_to_bool():
"""Pre-PR the field was Optional[bool]; widening to Union[bool,str]
must preserve historical coercion so callers sending `"false"`
still opt out of caching."""
from models.inference import ChatCompletionRequest
msg = {"role": "user", "content": "hi"}
req = ChatCompletionRequest.model_validate(
{
"model": "gemini-2.5-flash",
"messages": [msg],
"enable_prompt_caching": "false",
}
)
assert req.enable_prompt_caching is False, req.enable_prompt_caching
req = ChatCompletionRequest.model_validate(
{
"model": "gemini-2.5-flash",
"messages": [msg],
"enable_prompt_caching": "true",
}
)
assert req.enable_prompt_caching is True
# An actual cache resource name passes through untouched.
req = ChatCompletionRequest.model_validate(
{
"model": "gemini-2.5-flash",
"messages": [msg],
"enable_prompt_caching": "cachedContents/abc123",
}
)
assert req.enable_prompt_caching == "cachedContents/abc123"
def test_legacy_google_openai_base_url_is_rewritten():
"""The Google-hosted /v1beta/openai legacy base IS still rewritten."""
client = ExternalProviderClient(
provider_type = "gemini",
base_url = "https://generativelanguage.googleapis.com/v1beta/openai",
api_key = "AIza-test-key",
)
assert client.base_url == "https://generativelanguage.googleapis.com/v1beta"
def test_remote_image_url_downloads_and_inlines_as_base64(monkeypatch):
"""Round 14: arbitrary public HTTPS image URLs cannot be sent as
Gemini fileData (that path is reserved for Files API URIs and
YouTube). The translator must fetch the bytes server-side and
inline them as base64 inlineData."""
image_bytes = b"FAKEPNGBYTES"
async def fake_fetch(url, fallback_mime, max_bytes = None):
assert url == "https://cdn.example.com/diagram.png"
return ("image/png", base64.b64encode(image_bytes).decode("ascii"))
monkeypatch.setattr(ep_mod, "_safe_fetch_image_for_gemini", fake_fetch)
captured = _capture_body(
monkeypatch,
messages = [
{
"role": "user",
"content": [
{"type": "text", "text": "what is this?"},
{
"type": "image_url",
"image_url": {
"url": "https://cdn.example.com/diagram.png",
},
},
],
}
],
)
parts = captured["body"]["contents"][-1]["parts"]
inline = next((p for p in parts if "inlineData" in p), None)
assert inline is not None, parts
assert inline["inlineData"]["mimeType"] == "image/png"
assert inline["inlineData"]["data"] == base64.b64encode(image_bytes).decode()
assert not any("fileData" in p for p in parts), parts
def test_remote_image_url_dropped_when_fetch_returns_none(monkeypatch):
"""Round 15: if the SSRF guard rejects the URL (private host,
non-https, oversize, non-image), the helper returns None and the
image part is silently dropped instead of forwarding raw bytes
or a fileData fallback."""
async def fake_fetch_reject(url, fallback_mime, max_bytes = None):
return None
monkeypatch.setattr(ep_mod, "_safe_fetch_image_for_gemini", fake_fetch_reject)
captured = _capture_body(
monkeypatch,
messages = [
{
"role": "user",
"content": [
{"type": "text", "text": "what is this?"},
{
"type": "image_url",
"image_url": {"url": "http://10.0.0.5/private.png"},
},
],
}
],
)
parts = captured["body"]["contents"][-1]["parts"]
assert not any("inlineData" in p for p in parts), parts
assert not any("fileData" in p for p in parts), parts
def test_safe_fetch_image_rejects_non_https():
"""SSRF guard: only https URLs may be fetched."""
res = asyncio.new_event_loop().run_until_complete(
ep_mod._safe_fetch_image_for_gemini("http://cdn.example.com/x.png", "image/png")
)
assert res is None
def test_safe_fetch_image_rejects_loopback_ip_literal():
"""SSRF guard: refuse loopback / private IP literals before any
network call."""
for url in (
"https://127.0.0.1/x.png",
"https://[::1]/x.png",
"https://169.254.169.254/latest/meta-data",
"https://10.0.0.5/x.png",
"https://192.168.1.1/x.png",
):
res = asyncio.new_event_loop().run_until_complete(
ep_mod._safe_fetch_image_for_gemini(url, "image/png")
)
assert res is None, url
def test_safe_fetch_image_rejects_resolved_private_host(monkeypatch):
"""SSRF guard: if a hostname resolves to a private IP, refuse."""
import socket
def fake_getaddrinfo(host, *_args, **_kwargs):
return [(socket.AF_INET, None, None, "", ("10.0.0.5", 0))]
monkeypatch.setattr(socket, "getaddrinfo", fake_getaddrinfo)
res = asyncio.new_event_loop().run_until_complete(
ep_mod._safe_fetch_image_for_gemini(
"https://internal.example/x.png", "image/png"
)
)
assert res is None
def test_youtube_and_files_api_uris_stay_as_file_data(monkeypatch):
"""Round 14: YouTube URLs and generativelanguage.googleapis.com
Files API URIs are the documented `fileData.fileUri` paths and
must NOT be downloaded; arbitrary public URLs do get fetched."""
captured: dict = {}
def handler(request: httpx.Request) -> httpx.Response:
captured["body"] = json.loads(request.content.decode("utf-8"))
return httpx.Response(
200,
content = _gemini_sse(
[
{
"candidates": [
{
"content": {
"role": "model",
"parts": [{"text": "ok"}],
},
"finishReason": "STOP",
}
],
"usageMetadata": {
"promptTokenCount": 1,
"candidatesTokenCount": 1,
},
}
]
),
headers = {"content-type": "text/event-stream"},
)
_mock_http(monkeypatch, handler)
async def run():
client = _make_gemini_client()
async for _ in client.stream_chat_completion(
messages = [
{
"role": "user",
"content": [
{"type": "text", "text": "explain"},
{
"type": "image_url",
"image_url": {
"url": "https://www.youtube.com/watch?v=abc123",
},
},
{
"type": "image_url",
"image_url": {
"url": "https://generativelanguage.googleapis.com/v1beta/files/abc",
},
},
],
}
],
model = "gemini-2.5-flash",
temperature = 0.7,
top_p = 0.95,
max_tokens = 64,
):
pass
await client.close()
_drive(run())
parts = captured["body"]["contents"][-1]["parts"]
file_uris = [p["fileData"]["fileUri"] for p in parts if "fileData" in p]
assert "https://www.youtube.com/watch?v=abc123" in file_uris, parts
assert (
"https://generativelanguage.googleapis.com/v1beta/files/abc" in file_uris
), parts
def test_tool_use_prompt_tokens_added_to_input_tokens(monkeypatch):
"""`toolUsePromptTokenCount` must roll into the OpenAI prompt
total -- otherwise tool turns silently undercount input tokens."""
sse = [
{
"candidates": [
{
"content": {
"role": "model",
"parts": [{"text": "result"}],
},
"finishReason": "STOP",
}
],
"usageMetadata": {
"promptTokenCount": 10,
"toolUsePromptTokenCount": 100,
"candidatesTokenCount": 5,
"thoughtsTokenCount": 2,
},
}
]
lines = _collect(monkeypatch, sse)
chunks = _parse_chunks(lines)
usage_chunks = [c for c in chunks if c.get("usage")]
assert len(usage_chunks) == 1, chunks
usage = usage_chunks[0]["usage"]
assert usage["prompt_tokens"] == 110, usage
assert usage["completion_tokens"] == 7, usage
assert usage["total_tokens"] == 117, usage
assert usage["completion_tokens_details"]["reasoning_tokens"] == 2, usage
def test_usage_chunk_reasoning_tokens_surfaced(monkeypatch):
"""thoughtsTokenCount must surface as completion_tokens_details.
reasoning_tokens in the emitted OpenAI usage chunk."""
sse = [
{
"candidates": [
{
"content": {
"role": "model",
"parts": [{"text": "ok"}],
},
"finishReason": "STOP",
}
],
"usageMetadata": {
"promptTokenCount": 8,
"candidatesTokenCount": 5,
"thoughtsTokenCount": 20,
},
}
]
lines = _collect(monkeypatch, sse)
chunks = _parse_chunks(lines)
usage_chunks = [c for c in chunks if c.get("usage")]
assert len(usage_chunks) == 1, chunks
usage = usage_chunks[0]["usage"]
assert usage["completion_tokens"] == 25, usage
assert usage["completion_tokens_details"]["reasoning_tokens"] == 20, usage
def test_prompt_block_pairs_web_search_tool_end(monkeypatch):
"""When `promptFeedback.blockReason` triggers after the synthetic
web_search tool_start, the helper must emit a matching tool_end so
the UI does not leave a "searching..." spinner stuck on screen."""
sse = [
{"promptFeedback": {"blockReason": "SAFETY"}},
]
lines = _collect(
monkeypatch,
sse,
enabled_tools = ["web_search"],
)
chunks = _parse_chunks(lines)
tool_events = [c["_toolEvent"] for c in chunks if "_toolEvent" in c]
starts = [e for e in tool_events if e.get("type") == "tool_start"]
ends = [e for e in tool_events if e.get("type") == "tool_end"]
assert len(starts) == 1, tool_events
assert len(ends) == 1, tool_events
assert ends[0]["tool_call_id"] == "gemini_web_search"
assert "aborted" in ends[0]["result"]
error_chunks = [c for c in chunks if c.get("error")]
assert error_chunks, chunks
def test_code_execution_tool_events_stow_native_part(monkeypatch):
"""executableCode / codeExecutionResult must round-trip native ids
and thoughtSignature in google.native_part so follow-up turns can
replay Gemini's required history shape."""
sse = [
{
"candidates": [
{
"content": {
"role": "model",
"parts": [
{
"executableCode": {
"id": "code_a",
"language": "PYTHON",
"code": "print(1+1)",
},
"thoughtSignature": "SIG-CODE",
},
{
"codeExecutionResult": {
"id": "result_a",
"outcome": "OUTCOME_OK",
"output": "2\n",
},
},
],
},
"finishReason": "STOP",
}
],
"usageMetadata": {
"promptTokenCount": 5,
"candidatesTokenCount": 4,
},
}
]
lines = _collect(
monkeypatch,
sse,
enabled_tools = ["code_execution"],
)
chunks = _parse_chunks(lines)
tool_events = [c["_toolEvent"] for c in chunks if "_toolEvent" in c]
starts = [e for e in tool_events if e.get("type") == "tool_start"]
ends = [e for e in tool_events if e.get("type") == "tool_end"]
code_start = next(
(e for e in starts if e.get("tool_name") == "code_execution"),
None,
)
code_end = next(iter(ends), None)
assert code_start is not None, starts
assert code_start["tool_call_id"] == "code_a", code_start
native = code_start["arguments"]["google"]["native_part"]
# Round 21: native_part now uses an ordered `parts` list so per-part
# `thoughtSignature` survives a frontend merge of executableCode +
# codeExecutionResult into one tool-call card.
start_parts = native["parts"]
assert start_parts[0]["executableCode"]["id"] == "code_a"
assert start_parts[0]["thoughtSignature"] == "SIG-CODE"
assert code_end is not None, ends
assert code_end["tool_call_id"] == "code_a", code_end
native_end = code_end["google"]["native_part"]
end_parts = native_end["parts"]
assert end_parts[0]["codeExecutionResult"]["id"] == "result_a"
def test_inline_image_tool_end_carries_thought_signature(monkeypatch):
"""Inline image parts with thoughtSignature must persist it on the
emitted tool_end so Gemini 3 image editing can echo it back."""
sse = [
{
"candidates": [
{
"content": {
"role": "model",
"parts": [
{
"inlineData": {
"mimeType": "image/png",
"data": base64.b64encode(b"PNG").decode(),
},
"thoughtSignature": "SIG-IMG",
}
],
},
"finishReason": "STOP",
}
],
"usageMetadata": {
"promptTokenCount": 4,
"candidatesTokenCount": 1,
},
}
]
lines = _collect(
monkeypatch,
sse,
model = "gemini-2.5-flash-image",
)
chunks = _parse_chunks(lines)
tool_events = [c["_toolEvent"] for c in chunks if "_toolEvent" in c]
image_ends = [
e for e in tool_events if e.get("type") == "tool_end" and e.get("image_b64")
]
assert image_ends, tool_events
assert image_ends[0]["google"]["thought_signature"] == "SIG-IMG"
# Multi-turn image edit must replay the original inlineData part with
# its thoughtSignature; the outbound translator reads
# google.native_part.parts[].inlineData, so stow it on the tool_end
# too. Round 21 changed native_part to an ordered parts list so a
# per-part signature stays attached to inlineData only.
native = image_ends[0]["google"]["native_part"]
image_parts = native["parts"]
assert image_parts[0]["inlineData"]["mimeType"] == "image/png"
assert image_parts[0]["inlineData"]["data"] == base64.b64encode(b"PNG").decode()
assert image_parts[0]["thoughtSignature"] == "SIG-IMG"
def test_code_execution_plot_attaches_inline_image_native_part(monkeypatch):
"""A code_execution turn that returns a matplotlib plot must stow
the plot's inlineData on the secondary tool_end so the follow-up
turn can replay the image alongside executableCode and
codeExecutionResult."""
plot_data = base64.b64encode(b"PLOT").decode()
sse = [
{
"candidates": [
{
"content": {
"role": "model",
"parts": [
{
"executableCode": {
"id": "code_a",
"language": "PYTHON",
"code": "plt.plot([0,1])",
},
},
{
"codeExecutionResult": {
"id": "result_a",
"outcome": "OUTCOME_OK",
"output": "",
},
},
{
"inlineData": {
"mimeType": "image/png",
"data": plot_data,
},
},
],
},
"finishReason": "STOP",
}
],
"usageMetadata": {
"promptTokenCount": 5,
"candidatesTokenCount": 4,
},
}
]
lines = _collect(
monkeypatch,
sse,
enabled_tools = ["code_execution"],
)
chunks = _parse_chunks(lines)
tool_events = [c["_toolEvent"] for c in chunks if "_toolEvent" in c]
code_ends = [
e
for e in tool_events
if e.get("type") == "tool_end" and e.get("tool_call_id") == "code_a"
]
# Two tool_end events on the same id: one for codeExecutionResult,
# one merging in the inlineData plot. The plot one must carry the
# native inlineData under google.native_part so the frontend
# tool_end merge union joins it with the prior executableCode and
# codeExecutionResult parts on the same card.
assert len(code_ends) == 2, code_ends
image_end = next(
(e for e in code_ends if "__IMAGES__:" in (e.get("result") or "")),
None,
)
assert image_end is not None, code_ends
native = image_end["google"]["native_part"]
plot_parts = native["parts"]
assert plot_parts[0]["inlineData"]["mimeType"] == "image/png"
assert plot_parts[0]["inlineData"]["data"] == plot_data
def test_text_chunk_carries_thought_signature(monkeypatch):
"""Text parts with thoughtSignature surface it on delta.extra_content
so frontend persistence can replay it on the follow-up turn."""
sse = [
{
"candidates": [
{
"content": {
"role": "model",
"parts": [
{
"text": "hello",
"thoughtSignature": "SIG-TEXT",
}
],
},
"finishReason": "STOP",
}
],
"usageMetadata": {
"promptTokenCount": 2,
"candidatesTokenCount": 1,
},
}
]
lines = _collect(monkeypatch, sse)
chunks = _parse_chunks(lines)
text_chunks = [
c
for c in chunks
if c.get("choices") and c["choices"][0]["delta"].get("content") == "hello"
]
assert text_chunks, chunks
extra = text_chunks[0]["choices"][0]["delta"].get("extra_content")
assert extra == {"google": {"thought_signature": "SIG-TEXT"}}, text_chunks
def test_openai_tools_translated_into_function_declarations(monkeypatch):
"""Standard ChatCompletionRequest.tools must be forwarded into
Gemini's tools[].functionDeclarations envelope."""
captured = _capture_body(
monkeypatch,
tools = [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Look up the weather for a city.",
"parameters": {
"type": "object",
"properties": {
"city": {"type": "string"},
},
"required": ["city"],
},
},
}
],
tool_choice = {"type": "function", "function": {"name": "get_weather"}},
)
tools_arr = captured["body"].get("tools") or []
fn_decls = [t for t in tools_arr if "functionDeclarations" in t]
assert fn_decls, captured["body"]
decls = fn_decls[0]["functionDeclarations"]
assert decls[0]["name"] == "get_weather"
assert decls[0]["parameters"]["properties"]["city"]["type"] == "string"
tool_config = captured["body"].get("toolConfig")
assert tool_config is not None, captured["body"]
fcc = tool_config["functionCallingConfig"]
assert fcc["mode"] == "ANY"
assert fcc["allowedFunctionNames"] == ["get_weather"]
def test_tool_choice_auto_maps_to_function_calling_mode_auto(monkeypatch):
"""tool_choice="auto" maps to toolConfig.functionCallingConfig.mode."""
captured = _capture_body(
monkeypatch,
tools = [
{
"type": "function",
"function": {"name": "noop", "parameters": {"type": "object"}},
}
],
tool_choice = "auto",
)
fcc = captured["body"]["toolConfig"]["functionCallingConfig"]
assert fcc["mode"] == "AUTO"
assert "allowedFunctionNames" not in fcc
def test_code_exec_inline_image_attaches_to_code_execution_card(monkeypatch):
"""A codeExecution sandbox plot (matplotlib) ships as an inline
image part right after the codeExecutionResult. Instead of spawning
a separate empty image_generation card, attach to the same
code_execution tool_end via the `__IMAGES__:` marker the chat
adapter already understands."""
sse = [
{
"candidates": [
{
"content": {
"role": "model",
"parts": [
{
"executableCode": {
"id": "code_plot",
"language": "PYTHON",
"code": "import matplotlib.pyplot as plt; plt.plot([1,2,3]); plt.savefig('out.png')",
},
},
{
"codeExecutionResult": {
"outcome": "OUTCOME_OK",
"output": "saved",
},
},
{
"inlineData": {
"mimeType": "image/png",
"data": base64.b64encode(b"PNGDATA").decode(),
},
},
],
},
"finishReason": "STOP",
}
],
"usageMetadata": {
"promptTokenCount": 5,
"candidatesTokenCount": 4,
},
}
]
lines = _collect(
monkeypatch,
sse,
enabled_tools = ["code_execution"],
)
chunks = _parse_chunks(lines)
tool_events = [c["_toolEvent"] for c in chunks if "_toolEvent" in c]
# No standalone image_generation card should have been emitted.
image_starts = [
e
for e in tool_events
if e.get("type") == "tool_start" and e.get("tool_name") == "image_generation"
]
assert not image_starts, tool_events
# The code_execution tool_end should now carry the inline image
# via the `__IMAGES__:` marker.
code_ends = [
e
for e in tool_events
if e.get("type") == "tool_end" and e.get("tool_call_id") == "code_plot"
]
assert code_ends, tool_events
final_result = code_ends[-1]["result"]
assert "__IMAGES__:" in final_result, code_ends
assert "data:image/png;base64," in final_result, code_ends
def test_code_execution_tool_call_replays_native_executable_code(monkeypatch):
"""An assistant tool_call with toolName=code_execution and
extra_content.google.native_part containing the originally-emitted
`executableCode` + `codeExecutionResult` must round-trip as native
Gemini parts (not a generic functionCall) on the next turn."""
captured = _capture_body(
monkeypatch,
messages = [
{"role": "user", "content": "compute 2+2"},
{
"role": "assistant",
"content": None,
"tool_calls": [
{
"id": "code_a",
"type": "function",
"function": {
"name": "code_execution",
"arguments": "{}",
},
"extra_content": {
"google": {
"native_part": {
"executableCode": {
"id": "code_a",
"language": "PYTHON",
"code": "print(2+2)",
},
"codeExecutionResult": {
"outcome": "OUTCOME_OK",
"output": "4\n",
},
"thoughtSignature": "SIG-CODE",
},
},
},
},
],
},
{"role": "user", "content": "what was that result"},
],
)
assistant_turn = captured["body"]["contents"][1]
assert assistant_turn["role"] == "model"
parts = assistant_turn["parts"]
native_keys = [list(p.keys())[0] for p in parts if isinstance(p, dict)]
assert "executableCode" in native_keys, parts
assert "codeExecutionResult" in native_keys, parts
assert not any(
"functionCall" in p
and (p["functionCall"] or {}).get("name") == "code_execution"
for p in parts
), parts
exec_part = next(p for p in parts if "executableCode" in p)
assert exec_part.get("thoughtSignature") == "SIG-CODE", exec_part
def test_image_generation_tool_call_replays_native_inline_data(monkeypatch):
"""An assistant tool_call with toolName=image_generation and
extra_content.google.native_part.inlineData must replay the prior
image as a native Gemini inlineData part (not a generic
functionCall) so multi-turn image editing keeps the image
context."""
pixel = base64.b64encode(b"PNG").decode()
captured = _capture_body(
monkeypatch,
model = "gemini-2.5-flash-image",
messages = [
{"role": "user", "content": "make a circle"},
{
"role": "assistant",
"content": None,
"tool_calls": [
{
"id": "img_a",
"type": "function",
"function": {
"name": "image_generation",
"arguments": "{}",
},
"extra_content": {
"google": {
"native_part": {
"inlineData": {
"mimeType": "image/png",
"data": pixel,
},
"thoughtSignature": "SIG-IMG",
},
},
},
},
],
},
{"role": "user", "content": "now make it blue"},
],
)
assistant_turn = captured["body"]["contents"][1]
assert assistant_turn["role"] == "model"
parts = assistant_turn["parts"]
inline_parts = [p for p in parts if "inlineData" in p]
assert inline_parts, parts
assert inline_parts[0]["inlineData"]["mimeType"] == "image/png"
assert inline_parts[0]["inlineData"]["data"] == pixel
assert inline_parts[0].get("thoughtSignature") == "SIG-IMG", inline_parts
assert not any(
"functionCall" in p
and (p["functionCall"] or {}).get("name") == "image_generation"
for p in parts
), parts
def test_assistant_text_thought_signature_replays_on_outbound_text_part(monkeypatch):
"""Assistant text with extra_content.google.thought_signature must
attach `thoughtSignature` to the LAST text part of the replayed
Gemini history. Gemini 3 strict function-calling rejects history
that drops returned signatures, so the frontend stows the latest
signed-text signature and the backend pins it on the next turn."""
captured = _capture_body(
monkeypatch,
messages = [
{"role": "user", "content": "hi"},
{
"role": "assistant",
"content": [
{"type": "text", "text": "hello"},
],
"extra_content": {
"google": {"thought_signature": "SIG-TEXT"},
},
},
{"role": "user", "content": "again"},
],
)
assistant_turn = captured["body"]["contents"][1]
assert assistant_turn["role"] == "model"
parts = assistant_turn["parts"]
text_parts = [p for p in parts if "text" in p]
assert text_parts, parts
assert text_parts[-1].get("thoughtSignature") == "SIG-TEXT", text_parts
def test_function_declarations_strip_openai_only_schema_keys(monkeypatch):
"""OpenAI strict tools commonly include `additionalProperties`,
`$schema`, `$defs`, `strict`, etc. Gemini's Schema rejects those
with INVALID_ARGUMENT, so the translator must strip them while
keeping properties.<field>.type intact."""
captured = _capture_body(
monkeypatch,
tools = [
{
"type": "function",
"function": {
"name": "lookup",
"description": "Look up a value.",
"parameters": {
"type": "object",
"$schema": "http://json-schema.org/draft-07/schema#",
"additionalProperties": False,
"strict": True,
"properties": {
"key": {
"type": "string",
"additionalProperties": False,
},
},
"required": ["key"],
},
},
}
],
)
tools_arr = captured["body"].get("tools") or []
decls = next(
(
t.get("functionDeclarations")
for t in tools_arr
if "functionDeclarations" in t
),
None,
)
assert decls is not None, captured["body"]
params = decls[0]["parameters"]
assert "additionalProperties" not in params
assert "$schema" not in params
assert "strict" not in params
assert params["type"] == "object"
assert params["properties"]["key"]["type"] == "string"
assert "additionalProperties" not in params["properties"]["key"]
assert params["required"] == ["key"]
def test_function_declarations_inline_local_refs_into_gemini_schema(monkeypatch):
"""Round 25: Pydantic-generated tool schemas hoist nested object
shapes into `$defs` and reference them with `{"$ref": "#/$defs/..."}`.
Gemini's OpenAPI subset has no $ref, so a naive allowlist sanitizer
drops the reference and reduces the nested property to `{}`, losing
its type, fields, and required keys. The sanitizer must resolve
local `#/...` pointers and inline the referenced schema."""
captured = _capture_body(
monkeypatch,
tools = [
{
"type": "function",
"function": {
"name": "set_user",
"description": "Persist a user.",
"parameters": {
"type": "object",
"$defs": {
"Address": {
"type": "object",
"properties": {
"street": {"type": "string"},
"zip": {"type": "string"},
},
"required": ["street", "zip"],
},
},
"properties": {
"name": {"type": "string"},
"address": {"$ref": "#/$defs/Address"},
},
"required": ["name", "address"],
},
},
}
],
)
tools_arr = captured["body"].get("tools") or []
decls = next(
(
t.get("functionDeclarations")
for t in tools_arr
if "functionDeclarations" in t
),
None,
)
assert decls is not None, captured["body"]
params = decls[0]["parameters"]
assert "$defs" not in params
address = params["properties"]["address"]
assert address.get("type") == "object", address
assert address.get("properties", {}).get("street", {}).get("type") == "string"
assert address.get("properties", {}).get("zip", {}).get("type") == "string"
assert address.get("required") == ["street", "zip"]
def test_function_declarations_inline_local_refs_in_anyof_and_items(monkeypatch):
"""The recursive inliner must reach through `anyOf` branches and
`items` (array element schemas) as well, not just top-level
property refs."""
captured = _capture_body(
monkeypatch,
tools = [
{
"type": "function",
"function": {
"name": "bulk_set",
"parameters": {
"type": "object",
"$defs": {
"Address": {
"type": "object",
"properties": {"zip": {"type": "string"}},
"required": ["zip"],
},
},
"properties": {
"primary": {
"anyOf": [
{"$ref": "#/$defs/Address"},
{"type": "null"},
],
},
"extras": {
"type": "array",
"items": {"$ref": "#/$defs/Address"},
},
},
},
},
}
],
)
tools_arr = captured["body"].get("tools") or []
decls = next(
(
t.get("functionDeclarations")
for t in tools_arr
if "functionDeclarations" in t
),
None,
)
assert decls is not None
params = decls[0]["parameters"]
primary = params["properties"]["primary"]
# anyOf with single non-null branch + null collapses to inline +
# nullable: true, and the inlined branch must contain the resolved
# Address shape.
assert primary.get("nullable") is True
assert primary.get("type") == "object"
assert primary.get("properties", {}).get("zip", {}).get("type") == "string"
extras = params["properties"]["extras"]
assert extras.get("type") == "array"
assert extras.get("items", {}).get("type") == "object"
assert (
extras.get("items", {}).get("properties", {}).get("zip", {}).get("type")
== "string"
)
def test_function_declarations_self_referential_schema_terminates(monkeypatch):
"""Self-referential / cyclic JSON Schemas (a `Node` that contains
`children: [Node]`) must not infinite-loop. The inliner tracks the
set of refs in flight and short-circuits to `{}` on a cycle."""
captured = _capture_body(
monkeypatch,
tools = [
{
"type": "function",
"function": {
"name": "set_tree",
"parameters": {
"type": "object",
"$defs": {
"Node": {
"type": "object",
"properties": {
"value": {"type": "string"},
"children": {
"type": "array",
"items": {"$ref": "#/$defs/Node"},
},
},
},
},
"properties": {
"root": {"$ref": "#/$defs/Node"},
},
},
},
}
],
)
tools_arr = captured["body"].get("tools") or []
decls = next(
(
t.get("functionDeclarations")
for t in tools_arr
if "functionDeclarations" in t
),
None,
)
assert decls is not None
root = decls[0]["parameters"]["properties"]["root"]
assert root.get("type") == "object"
assert root.get("properties", {}).get("value", {}).get("type") == "string"
def test_gemini_native_skips_orphan_function_response_for_dropped_builtin(
monkeypatch,
):
"""Round 26: when the assistant-side synthetic web_search/web_fetch
tool_call is dropped from native Gemini history, the matching
role="tool" follow-up must also be dropped. Otherwise the outbound
body carries an orphan functionResponse with no preceding
functionCall, which 400s the Gemini turn."""
from models.inference import ChatCompletionRequest
from routes.inference import _build_external_messages
req = ChatCompletionRequest.model_validate(
{
"model": "gemini-2.5-flash",
"messages": [
{"role": "user", "content": "search please"},
{
"role": "assistant",
"content": "",
"tool_calls": [
{
"id": "call_s",
"type": "function",
"function": {
"name": "web_search",
"arguments": ('{"_server_tool": true, "query": "x"}'),
},
}
],
},
{
"role": "tool",
"tool_call_id": "call_s",
"content": "[search result]",
},
{"role": "user", "content": "again"},
],
"max_tokens": 64,
"stream": True,
}
)
built = _build_external_messages(
req.messages,
supports_vision = True,
provider_type = "gemini",
base_url = "https://generativelanguage.googleapis.com/v1beta",
)
captured = _capture_body(monkeypatch, messages = built)
contents = captured["body"].get("contents") or []
for entry in contents:
for part in entry.get("parts", []):
fr = part.get("functionResponse")
if isinstance(fr, dict):
assert fr.get("name") != "web_search", contents
def test_gemini_native_skips_orphan_function_response_for_native_part_replay(
monkeypatch,
):
"""Round 26: code_execution / image_generation tool_calls are
replayed as Gemini-native executableCode / codeExecutionResult /
inlineData parts. The matching role="tool" follow-up must NOT then
be emitted as a functionResponse named code_execution -- there is
no declared user function with that name, and Gemini's history
rules already attribute the result to the native parts above."""
captured = _capture_body(
monkeypatch,
messages = [
{"role": "user", "content": "plot something"},
{
"role": "assistant",
"content": "",
"tool_calls": [
{
"id": "call_a",
"type": "function",
"function": {
"name": "code_execution",
"arguments": "{}",
},
"extra_content": {
"google": {
"native_part": {
"parts": [
{
"executableCode": {
"language": "PYTHON",
"code": "print(2)",
}
},
{
"codeExecutionResult": {
"outcome": "OUTCOME_OK",
"output": "2\n",
}
},
]
}
}
},
}
],
},
{
"role": "tool",
"tool_call_id": "call_a",
"name": "code_execution",
"content": "2",
},
{"role": "user", "content": "next"},
],
)
contents = captured["body"].get("contents") or []
saw_native = False
for entry in contents:
for part in entry.get("parts", []):
if "executableCode" in part or "codeExecutionResult" in part:
saw_native = True
fr = part.get("functionResponse")
if isinstance(fr, dict):
assert fr.get("name") != "code_execution", contents
assert saw_native, contents
def test_gemini_native_part_falls_back_to_args_google(monkeypatch):
"""Round 27: a direct OpenAI-compat API caller (or imported third-
party thread) cannot use Studio's non-standard
`tool_calls[].extra_content` field, so the native_part payload
round-trips through `function.arguments` as
`{"google": {"native_part": {...}}}`. The synthetic-builtin
detector recognizes that location, but the replay branch was only
reading from `tc.extra_content.google.native_part`. Result: the
round-25 guard saw a synthetic builtin with no _native_part and
dropped the entire assistant turn, losing the prior code/image
context. The translator must fall back to args.google.native_part
and still emit the native executableCode / inlineData parts."""
import json as _json
captured = _capture_body(
monkeypatch,
messages = [
{"role": "user", "content": "draw a cat"},
{
"role": "assistant",
"content": "",
"tool_calls": [
{
"id": "call_img",
"type": "function",
"function": {
"name": "image_generation",
"arguments": _json.dumps(
{
"google": {
"native_part": {
"parts": [
{
"inlineData": {
"mimeType": "image/png",
"data": "AAAA",
}
}
]
}
}
}
),
},
}
],
},
{"role": "user", "content": "now make it a dog"},
],
)
contents = captured["body"].get("contents") or []
saw_inline = False
for entry in contents:
for part in entry.get("parts", []):
if "inlineData" in part:
saw_inline = True
assert saw_inline, contents
def test_gemini_native_skips_synthetic_server_builtin_replay(monkeypatch):
"""Round 25: Marked server-side builtin tool_calls (web_search /
web_fetch with `_server_tool` or `args.google.native_part`) must
not fall through to the generic Gemini `functionCall` replay path
when no replayable native part exists. Without this guard the
outbound body contains a fake `functionCall` whose name is not a
declared user function, and the Gemini turn 400s."""
from models.inference import ChatCompletionRequest
from routes.inference import _build_external_messages
req = ChatCompletionRequest.model_validate(
{
"model": "gemini-2.5-flash",
"messages": [
{"role": "user", "content": "search please"},
{
"role": "assistant",
"content": "",
"tool_calls": [
{
"id": "call_s",
"type": "function",
"function": {
"name": "web_search",
"arguments": ('{"_server_tool": true, "query": "x"}'),
},
}
],
},
{
"role": "tool",
"tool_call_id": "call_s",
"content": "[search result]",
},
{"role": "user", "content": "again"},
],
"max_tokens": 64,
"stream": True,
}
)
built = _build_external_messages(
req.messages,
supports_vision = True,
provider_type = "gemini",
base_url = "https://generativelanguage.googleapis.com/v1beta",
)
captured = _capture_body(monkeypatch, messages = built)
contents = captured["body"].get("contents") or []
for entry in contents:
for part in entry.get("parts", []):
fc = part.get("functionCall")
if isinstance(fc, dict):
assert fc.get("name") != "web_search", contents
def test_chat_message_extra_content_round_trips_through_validation():
"""Round 9: ChatMessage was missing `extra_content`, so Pydantic
discarded the field during request validation and the text-part
signature replay path read nothing. The field must survive
model_validate and pass through _build_external_messages."""
from models.inference import ChatCompletionRequest
from routes.inference import _build_external_messages
req = ChatCompletionRequest.model_validate(
{
"model": "gemini-2.5-flash",
"messages": [
{"role": "user", "content": "hi"},
{
"role": "assistant",
"content": [
{"type": "text", "text": "hello"},
],
"extra_content": {
"google": {"thought_signature": "SIG-TEXT"},
},
},
{"role": "user", "content": "again"},
],
"max_tokens": 64,
"stream": True,
}
)
assistant_msg = req.messages[1]
assert assistant_msg.extra_content == {
"google": {"thought_signature": "SIG-TEXT"},
}
built = _build_external_messages(
req.messages,
supports_vision = True,
provider_type = "gemini",
base_url = "https://generativelanguage.googleapis.com/v1beta",
)
assistant_out = built[1]
assert assistant_out["extra_content"] == {
"google": {"thought_signature": "SIG-TEXT"},
}
# Non-Gemini providers must NOT receive extra_content; Google's
# thought_signature field is unknown to OpenAI / Mistral / etc.
built_openai = _build_external_messages(
req.messages,
supports_vision = True,
provider_type = "openai",
)
assert "extra_content" not in built_openai[1], built_openai[1]
# Custom non-Google Gemini bases (LiteLLM / OAI-compat gateways)
# also must not receive Gemini-only extra_content because the
# backend dispatches them through /chat/completions.
built_custom = _build_external_messages(
req.messages,
supports_vision = True,
provider_type = "gemini",
base_url = "https://litellm.example/v1",
)
assert "extra_content" not in built_custom[1], built_custom[1]
def test_parallel_tool_results_group_into_one_user_block(monkeypatch):
"""Round 14: Gemini docs show parallel functionResponses grouped
in a single subsequent user content with multiple
functionResponse parts. Consecutive OpenAI role="tool" messages
must merge into one Gemini user block, not split into separate
user turns."""
captured = _capture_body(
monkeypatch,
messages = [
{"role": "user", "content": "compute"},
{
"role": "assistant",
"content": None,
"tool_calls": [
{
"id": "call_a",
"type": "function",
"function": {"name": "add", "arguments": '{"x":1}'},
},
{
"id": "call_b",
"type": "function",
"function": {"name": "mul", "arguments": '{"x":2}'},
},
],
},
{
"role": "tool",
"tool_call_id": "call_a",
"name": "add",
"content": "2",
},
{
"role": "tool",
"tool_call_id": "call_b",
"name": "mul",
"content": "4",
},
],
)
contents = captured["body"]["contents"]
# Initial user, model with two functionCalls, ONE user with two
# functionResponses.
tool_result_users = [
c
for c in contents
if c.get("role") == "user"
and all(
isinstance(p, dict) and "functionResponse" in p
for p in (c.get("parts") or [])
)
]
assert len(tool_result_users) == 1, contents
fr_parts = tool_result_users[0]["parts"]
assert len(fr_parts) == 2, fr_parts
names = [p["functionResponse"]["name"] for p in fr_parts]
assert names == ["add", "mul"], names
def test_function_schema_nullable_type_array_flattens(monkeypatch):
"""Round 14: OpenAI strict tools commonly use
`"type": ["string", "null"]` for optional fields. Gemini's
OpenAPI-style Schema rejects union types and expects
`"type": "string"` with `"nullable": true`. The sanitizer must
translate the union form."""
captured = _capture_body(
monkeypatch,
tools = [
{
"type": "function",
"function": {
"name": "lookup",
"parameters": {
"type": "object",
"properties": {
"city": {"type": ["string", "null"]},
"score": {"type": ["number", "null"]},
},
},
},
}
],
)
decls = next(
t["functionDeclarations"]
for t in captured["body"].get("tools") or []
if "functionDeclarations" in t
)
params = decls[0]["parameters"]["properties"]
assert params["city"]["type"] == "string"
assert params["city"]["nullable"] is True
assert params["score"]["type"] == "number"
assert params["score"]["nullable"] is True
def test_image_picker_model_with_search_off_pill_strips_text_tools(monkeypatch):
"""Round 11: image-tier model id rejects text-only tools and
thinkingConfig at the model level regardless of whether the Images
pill is on. Selecting gemini-2.5-flash-image + enabled_tools=
["web_search"] with no image_generation must NOT forward
googleSearch or thinkingConfig (Gemini 400s on text tools for
legacy image ids)."""
captured = _capture_body(
monkeypatch,
model = "gemini-2.5-flash-image",
enabled_tools = ["web_search"],
reasoning_effort = "high",
)
body = captured["body"]
assert "tools" not in body, body.get("tools")
assert "thinkingConfig" not in body.get("generationConfig", {}), body[
"generationConfig"
]
def test_image_models_drop_function_declarations(monkeypatch):
"""Image-mode requests cannot mix tools with responseModalities so
user-supplied function declarations must be dropped."""
captured = _capture_body(
monkeypatch,
model = "gemini-2.5-flash-image",
enabled_tools = ["image_generation"],
tools = [
{
"type": "function",
"function": {"name": "noop", "parameters": {"type": "object"}},
}
],
)
assert captured["body"].get("tools") is None
assert captured["body"]["generationConfig"]["responseModalities"] == [
"TEXT",
"IMAGE",
]
def test_safe_fetch_image_rejects_malformed_bracketed_url():
"""Round 17: bracketed IPv6 garbage like `https://[bad/x.png` makes
urlparse raise ValueError. The fetch helper must catch it and drop
the image rather than crashing the request mid-build."""
res = _drive(ep_mod._safe_fetch_image_for_gemini("https://[bad/x.png", "image/png"))
assert res is None
def test_safe_fetch_image_pins_validated_ip_no_hostname_in_request(
monkeypatch,
):
"""Round 17: the fetch helper must pin the validated IP into the
outgoing request URL (with a Host header carrying the original
hostname). A second hostname-style getaddrinfo after the validate
step would be a DNS-rebinding gap, so we assert the urllib opener
is called with an IP-rewritten URL."""
import socket
captured: dict = {"requests": []}
# Public IP during validate; record every getaddrinfo call.
original_getaddrinfo = socket.getaddrinfo
def fake_getaddrinfo(host, *args, **kwargs):
captured.setdefault("dns", []).append(host)
if host == "cdn.example.com":
return [
(
socket.AF_INET,
socket.SOCK_STREAM,
0,
"",
("8.8.8.8", 0),
)
]
return original_getaddrinfo(host, *args, **kwargs)
monkeypatch.setattr(socket, "getaddrinfo", fake_getaddrinfo)
class _StubResp:
status = 200
headers = {"content-type": "image/png", "content-length": "3"}
def __enter__(self):
return self
def __exit__(self, *a):
return False
def read(self, _n = None):
return b"PNG"
class _StubOpener:
def open(self, req, timeout = None):
captured["requests"].append(
{
"url": req.full_url,
"host_header": req.get_header("Host"),
}
)
return _StubResp()
monkeypatch.setattr(
"urllib.request.build_opener", lambda *_args, **_kw: _StubOpener()
)
res = _drive(
ep_mod._safe_fetch_image_for_gemini(
"https://cdn.example.com/x.png", "image/png"
)
)
assert res is not None
assert res[0] == "image/png"
# The outgoing URL must use the pinned IP literal, not the hostname.
assert any("8.8.8.8" in r["url"] for r in captured["requests"]), captured
assert all(
"cdn.example.com" not in r["url"] for r in captured["requests"]
), captured
# Host header still carries the original hostname for vhost/SNI.
assert captured["requests"][0]["host_header"] == "cdn.example.com"
def test_safe_fetch_image_redirect_to_private_host_rejected(monkeypatch):
"""Round 17: each redirect hop must re-validate the new host. A
public hop that redirects to an internal address must be dropped."""
import socket
import urllib.error
original_getaddrinfo = socket.getaddrinfo
def fake_getaddrinfo(host, *args, **kwargs):
if host == "cdn.example.com":
return [
(
socket.AF_INET,
socket.SOCK_STREAM,
0,
"",
("1.1.1.1", 0),
)
]
if host == "internal.bad":
return [
(
socket.AF_INET,
socket.SOCK_STREAM,
0,
"",
("10.0.0.5", 0),
)
]
return original_getaddrinfo(host, *args, **kwargs)
monkeypatch.setattr(socket, "getaddrinfo", fake_getaddrinfo)
class _StubOpener:
def open(self, req, timeout = None):
# Simulate a 302 to a private host.
raise urllib.error.HTTPError(
req.full_url,
302,
"Found",
{"Location": "https://internal.bad/secret.png"},
None,
)
monkeypatch.setattr(
"urllib.request.build_opener", lambda *_args, **_kw: _StubOpener()
)
res = _drive(
ep_mod._safe_fetch_image_for_gemini(
"https://cdn.example.com/x.png", "image/png"
)
)
assert res is None
def test_files_api_substring_url_not_misclassified_as_filedata(monkeypatch):
"""Round 17: a CDN URL whose path/query merely contains the Files
API substring must NOT be sent as `fileData.fileUri`; it must be
routed through the safe-fetch path. Previously the substring check
`"generativelanguage.googleapis.com/" in url.lower()` matched any
URL carrying that text anywhere."""
captured_outbound: dict = {}
fetch_calls: list[str] = []
async def fake_fetch(url, fallback_mime, max_bytes = None):
fetch_calls.append(url)
return "image/png", base64.b64encode(b"DATA").decode("ascii")
monkeypatch.setattr(ep_mod, "_safe_fetch_image_for_gemini", fake_fetch)
def handler(request: httpx.Request) -> httpx.Response:
captured_outbound["body"] = json.loads(request.content.decode("utf-8"))
return httpx.Response(
200,
content = _gemini_sse(
[
{
"candidates": [
{
"content": {
"role": "model",
"parts": [{"text": "ok"}],
},
"finishReason": "STOP",
}
],
"usageMetadata": {
"promptTokenCount": 1,
"candidatesTokenCount": 1,
},
}
]
),
headers = {"content-type": "text/event-stream"},
)
_mock_http(monkeypatch, handler)
async def run():
client = _make_gemini_client()
async for _ in client.stream_chat_completion(
messages = [
{
"role": "user",
"content": [
{"type": "text", "text": "describe"},
{
"type": "image_url",
"image_url": {
# Looks like a Files API URL in the path
# but the host is an attacker CDN.
"url": "https://evil.example/path/generativelanguage.googleapis.com/v1beta/files/abc.png",
},
},
{
"type": "image_url",
"image_url": {
# Looks YouTube-ish in the path.
"url": "https://cdn.example.com/youtube.com/cat.png",
},
},
],
}
],
model = "gemini-2.5-flash",
temperature = 0.7,
top_p = 0.95,
max_tokens = 64,
):
pass
await client.close()
_drive(run())
parts = captured_outbound["body"]["contents"][-1]["parts"]
assert not any("fileData" in p for p in parts), parts
inline_count = sum(1 for p in parts if "inlineData" in p)
assert inline_count == 2, parts
assert len(fetch_calls) == 2, fetch_calls
def test_function_schema_anyof_null_variant_flattens_to_nullable(monkeypatch):
"""Round 17: OpenAI/Pydantic emit `anyOf: [{X}, {"type":"null"}]`
for Optional[X]. Gemini's OpenAPI subset rejects `"type":"null"`
inside anyOf. The sanitizer must collapse a singleton-plus-null
union back to the non-null branch with `nullable: true`."""
captured = _capture_body(
monkeypatch,
tools = [
{
"type": "function",
"function": {
"name": "lookup",
"parameters": {
"type": "object",
"properties": {
"label": {
"anyOf": [
{"type": "string"},
{"type": "null"},
]
},
"count": {
"anyOf": [
{"type": "integer"},
{"type": "null"},
]
},
},
},
},
}
],
)
decls = next(
t["functionDeclarations"]
for t in captured["body"].get("tools") or []
if "functionDeclarations" in t
)
params = decls[0]["parameters"]["properties"]
assert params["label"]["type"] == "string"
assert params["label"]["nullable"] is True
assert "anyOf" not in params["label"]
assert params["count"]["type"] == "integer"
assert params["count"]["nullable"] is True
def test_legacy_gemini3_pro_medium_coerced_to_high(monkeypatch):
"""Round 17: legacy `gemini-3-pro*` (including `-preview`, shut down
2026-03-09) only accepted low/high. 3.1+ Pro added medium. The
backend must coerce medium → high for the legacy model so stale UI
state does not 400 the request."""
captured = _capture_body(
monkeypatch,
model = "gemini-3-pro-preview",
reasoning_effort = "medium",
)
assert captured["body"]["generationConfig"]["thinkingConfig"] == {
"thinkingLevel": "high",
}
def test_gemini_3_1_pro_medium_passes_through(monkeypatch):
"""Round 17 regression: 3.1+ Pro accepts medium; coercion must NOT
apply when the model id is gemini-3.1-pro*."""
captured = _capture_body(
monkeypatch,
model = "gemini-3.1-pro-preview",
reasoning_effort = "medium",
)
assert captured["body"]["generationConfig"]["thinkingConfig"] == {
"thinkingLevel": "medium",
}
def test_tool_calls_extra_content_stripped_for_non_native_gemini():
"""Round 17: per-tool-call `extra_content` (Gemini thoughtSignature
carrier) must not leak through `_build_external_messages` to
non-native-Gemini providers; OpenAI / Anthropic / custom Gemini
OAI-compat gateways would 400 on the unknown key."""
from models.inference import ChatCompletionRequest
from routes.inference import _build_external_messages
payload = {
"model": "gpt-5.5",
"messages": [
{
"role": "assistant",
"content": "",
"tool_calls": [
{
"id": "call_1",
"type": "function",
"function": {"name": "lookup", "arguments": "{}"},
"extra_content": {
"google": {"thought_signature": "SIG"},
},
}
],
}
],
"stream": True,
}
req = ChatCompletionRequest.model_validate(payload)
# Non-native providers (openai, custom Gemini OAI-compat proxy)
# must have extra_content stripped from the tool_call entry.
for provider_type, base_url in [
("openai", None),
("gemini", "https://litellm.example/v1"),
]:
result = _build_external_messages(
req.messages,
supports_vision = True,
provider_type = provider_type,
base_url = base_url,
)
assert len(result) == 1
tc = result[0]["tool_calls"][0]
assert "extra_content" not in tc, (provider_type, tc)
# Native Gemini still receives extra_content for the round-trip.
result_native = _build_external_messages(
req.messages,
supports_vision = True,
provider_type = "gemini",
base_url = "https://generativelanguage.googleapis.com/v1beta",
)
tc_native = result_native[0]["tool_calls"][0]
assert tc_native["extra_content"]["google"]["thought_signature"] == "SIG"
def test_user_function_named_with_server_tool_arg_not_dropped(monkeypatch):
"""Round 17: the OpenAI Responses translator must NOT drop a user
function whose JSON arguments happen to contain `_server_tool:
true` UNLESS the function name is also one of the canonical
builtin names. Otherwise a user schema with an `_server_tool` field
becomes invisible to the model."""
captured: dict = {"input_items": None}
def handler(request: httpx.Request) -> httpx.Response:
body = json.loads(request.content.decode("utf-8"))
captured["input_items"] = body.get("input")
return httpx.Response(
200,
content = b'data: {"type":"response.completed","response":{"output":[],"usage":{"input_tokens":1,"output_tokens":1}}}\n\n',
headers = {"content-type": "text/event-stream"},
)
_mock_http(monkeypatch, handler)
async def run():
client = ExternalProviderClient(
provider_type = "openai",
base_url = "https://api.openai.com/v1",
api_key = "sk-test",
)
async for _ in client.stream_chat_completion(
messages = [
{"role": "user", "content": "hi"},
{
"role": "assistant",
"content": "",
"tool_calls": [
{
"id": "call_user",
"type": "function",
"function": {
"name": "user_function",
"arguments": json.dumps(
{"_server_tool": True, "q": "x"}
),
},
}
],
},
{
"role": "tool",
"content": "result",
"tool_call_id": "call_user",
"name": "user_function",
},
{"role": "user", "content": "continue"},
],
model = "gpt-5.5",
temperature = 0.7,
top_p = 1.0,
max_tokens = 16,
):
pass
await client.close()
_drive(run())
items = captured["input_items"] or []
fn_calls = [i for i in items if i.get("type") == "function_call"]
fn_outs = [i for i in items if i.get("type") == "function_call_output"]
# User function call must survive (matching call + output).
assert any(c.get("name") == "user_function" for c in fn_calls), items
assert len(fn_outs) == 1, items
def test_builtin_named_with_server_tool_marker_dropped(monkeypatch):
"""Round 17 control: a builtin (web_search) tagged with
`_server_tool: true` continues to be filtered from outbound
history."""
captured: dict = {"input_items": None}
def handler(request: httpx.Request) -> httpx.Response:
body = json.loads(request.content.decode("utf-8"))
captured["input_items"] = body.get("input")
return httpx.Response(
200,
content = b'data: {"type":"response.completed","response":{"output":[],"usage":{"input_tokens":1,"output_tokens":1}}}\n\n',
headers = {"content-type": "text/event-stream"},
)
_mock_http(monkeypatch, handler)
async def run():
client = ExternalProviderClient(
provider_type = "openai",
base_url = "https://api.openai.com/v1",
api_key = "sk-test",
)
async for _ in client.stream_chat_completion(
messages = [
{"role": "user", "content": "search please"},
{
"role": "assistant",
"content": "",
"tool_calls": [
{
"id": "call_b",
"type": "function",
"function": {
"name": "web_search",
"arguments": json.dumps(
{"_server_tool": True, "query": "x"}
),
},
}
],
},
{"role": "user", "content": "continue"},
],
model = "gpt-5.5",
temperature = 0.7,
top_p = 1.0,
max_tokens = 16,
):
pass
await client.close()
_drive(run())
items = captured["input_items"] or []
fn_calls = [i for i in items if i.get("type") == "function_call"]
# Builtin server-side tool call must be filtered out.
assert all(c.get("name") != "web_search" for c in fn_calls), items
def test_gemini_tool_choice_none_disables_hosted_builtins(monkeypatch):
"""Round 18: `tool_choice="none"` must drop hosted Google Search /
code execution from the outbound Gemini body, not just user
function declarations. Otherwise an API client that opted out of
tool use still triggers grounded search (privacy + billing)."""
captured = _capture_body(
monkeypatch,
enabled_tools = ["web_search", "code_execution"],
tool_choice = "none",
)
assert captured["body"].get("tools") is None, captured["body"]
def test_gemini_tool_choice_none_disables_function_declarations(monkeypatch):
"""Round 18: `tool_choice="none"` must drop user function
declarations as well as hosted builtins from the Gemini body."""
captured = _capture_body(
monkeypatch,
tool_choice = "none",
tools = [
{
"type": "function",
"function": {"name": "lookup", "parameters": {"type": "object"}},
}
],
)
assert captured["body"].get("tools") is None, captured["body"]
def test_schema_anyof_multitype_with_null_keeps_anyof_and_nullable(
monkeypatch,
):
"""Round 18: multi-branch unions with null (e.g.
`Union[str, int, None]`) must keep the slim anyOf without the null
branch and add `nullable: true`; Gemini rejects
`{"type":"null"}` inside anyOf."""
captured = _capture_body(
monkeypatch,
tools = [
{
"type": "function",
"function": {
"name": "lookup",
"parameters": {
"type": "object",
"properties": {
"either": {
"anyOf": [
{"type": "string"},
{"type": "integer"},
{"type": "null"},
]
},
},
},
},
}
],
)
decls = next(
t["functionDeclarations"]
for t in captured["body"].get("tools") or []
if "functionDeclarations" in t
)
either = decls[0]["parameters"]["properties"]["either"]
assert either.get("nullable") is True
inner = either.get("anyOf")
assert isinstance(inner, list) and len(inner) == 2, either
assert all(
not (isinstance(b, dict) and b.get("type") == "null") for b in inner
), inner
def test_safe_fetch_image_redirect_malformed_url_no_crash(monkeypatch):
"""Round 18: when the upstream 302 Location is a malformed
bracketed-IPv6 URL, the helper must return None instead of letting
a urlparse ValueError abort the chat stream."""
import socket
import urllib.error
original_getaddrinfo = socket.getaddrinfo
def fake_getaddrinfo(host, *args, **kwargs):
if host == "cdn.example.com":
return [
(
socket.AF_INET,
socket.SOCK_STREAM,
0,
"",
("1.1.1.1", 0),
)
]
return original_getaddrinfo(host, *args, **kwargs)
monkeypatch.setattr(socket, "getaddrinfo", fake_getaddrinfo)
class _StubOpener:
def open(self, req, timeout = None):
raise urllib.error.HTTPError(
req.full_url,
302,
"Found",
{"Location": "https://[bad/x.png"},
None,
)
monkeypatch.setattr(
"urllib.request.build_opener", lambda *_args, **_kw: _StubOpener()
)
res = _drive(
ep_mod._safe_fetch_image_for_gemini(
"https://cdn.example.com/x.png", "image/png"
)
)
assert res is None
def test_safe_fetch_image_malformed_port_no_crash():
"""Round 18: a URL with a non-numeric port (`https://h:bad/x.png`)
must not raise; urlparse's port property lazily ValueErrors."""
res = _drive(
ep_mod._safe_fetch_image_for_gemini(
"https://example.com:bad/x.png", "image/png"
)
)
assert res is None
def test_safe_fetch_image_missing_content_type_uses_fallback(monkeypatch):
"""Round 18: when the server returns image bytes but no
Content-Type header, the helper must use the caller-provided
fallback MIME (guessed from URL extension) instead of dropping the
image as `non-image content-type=<none>`."""
import socket
original_getaddrinfo = socket.getaddrinfo
def fake_getaddrinfo(host, *args, **kwargs):
if host == "cdn.example.com":
return [
(
socket.AF_INET,
socket.SOCK_STREAM,
0,
"",
("1.1.1.1", 0),
)
]
return original_getaddrinfo(host, *args, **kwargs)
monkeypatch.setattr(socket, "getaddrinfo", fake_getaddrinfo)
class _StubResp:
status = 200
headers = {"content-length": "3"}
def __enter__(self):
return self
def __exit__(self, *a):
return False
def read(self, _n = None):
return b"PNG"
class _StubOpener:
def open(self, req, timeout = None):
return _StubResp()
monkeypatch.setattr(
"urllib.request.build_opener", lambda *_args, **_kw: _StubOpener()
)
res = _drive(
ep_mod._safe_fetch_image_for_gemini(
"https://cdn.example.com/cat.png", "image/png"
)
)
assert res is not None
assert res[0] == "image/png"
def test_anthropic_translates_openai_tool_calls_into_tool_use_blocks(monkeypatch):
"""Round 18: an assistant turn with OpenAI-style top-level
`tool_calls` must be translated into Anthropic native
`{type:"tool_use", id, name, input}` content blocks before being
forwarded. The OpenAI `role="tool"` follow-up must become a
`role:"user"` message with a `tool_result` content block."""
captured: dict = {"messages": None}
def handler(request: httpx.Request) -> httpx.Response:
body = json.loads(request.content.decode("utf-8"))
captured["messages"] = body.get("messages")
return httpx.Response(
200,
content = b'event: message_stop\ndata: {"type":"message_stop"}\n\n',
headers = {"content-type": "text/event-stream"},
)
_mock_http(monkeypatch, handler)
async def run():
client = ExternalProviderClient(
provider_type = "anthropic",
base_url = "https://api.anthropic.com",
api_key = "sk-ant-test",
)
async for _ in client.stream_chat_completion(
messages = [
{"role": "user", "content": "look up X"},
{
"role": "assistant",
"content": "let me check",
"tool_calls": [
{
"id": "call_a",
"type": "function",
"function": {
"name": "lookup",
"arguments": '{"q":"x"}',
},
}
],
},
{
"role": "tool",
"content": "result_text",
"tool_call_id": "call_a",
"name": "lookup",
},
{"role": "user", "content": "summarise"},
],
model = "claude-sonnet-4-5",
temperature = 0.7,
top_p = 0.95,
max_tokens = 64,
):
pass
await client.close()
_drive(run())
msgs = captured["messages"] or []
# No top-level tool_calls should remain.
assert all("tool_calls" not in m for m in msgs), msgs
# The assistant turn must now have content blocks including a
# tool_use block.
asst = [m for m in msgs if m.get("role") == "assistant"]
assert asst and isinstance(asst[0]["content"], list), asst
tool_uses = [b for b in asst[0]["content"] if b.get("type") == "tool_use"]
assert len(tool_uses) == 1, asst[0]
assert tool_uses[0]["name"] == "lookup"
assert tool_uses[0]["input"] == {"q": "x"}
# The role="tool" message must become a user/tool_result message.
tool_results: list[dict] = []
for m in msgs:
if m.get("role") == "user" and isinstance(m.get("content"), list):
tool_results.extend(
b for b in m["content"] if b.get("type") == "tool_result"
)
assert any(
tr.get("tool_use_id") == "call_a" and tr.get("content") == "result_text"
for tr in tool_results
), msgs
def test_unmarked_user_web_search_function_survives_serialization():
"""Round 18: a user-defined function literally named `web_search`
with NO `_server_tool` marker must survive `_build_external_messages`
when forwarded to a non-native provider; only marked synthetic
builtin cards may be dropped."""
from models.inference import ChatCompletionRequest
from routes.inference import _build_external_messages
payload = {
"model": "gpt-5.5",
"messages": [
{
"role": "assistant",
"content": "",
"tool_calls": [
{
"id": "call_user",
"type": "function",
"function": {
"name": "web_search",
"arguments": '{"query": "x"}',
},
}
],
}
],
"stream": True,
}
req = ChatCompletionRequest.model_validate(payload)
result = _build_external_messages(
req.messages,
supports_vision = True,
provider_type = "openai",
base_url = None,
)
assert len(result) == 1, result
tcs = result[0].get("tool_calls") or []
assert len(tcs) == 1, result
assert tcs[0]["function"]["name"] == "web_search"
def test_marked_server_builtin_dropped_from_build_external_messages():
"""Round 18: when a Gemini-native turn carrying a marked
`image_generation` server-tool card is forwarded to OpenAI / a
custom Gemini OAI-compat proxy, the tool_call must be dropped, not
just have its extra_content stripped. Forwarding an orphan
`image_generation` tool_call would 400 the receiving API."""
from models.inference import ChatCompletionRequest
from routes.inference import _build_external_messages
marked_args = json.dumps({"_server_tool": True, "kind": "image"})
payload = {
"model": "gpt-5.5",
"messages": [
{
"role": "assistant",
"content": "",
"tool_calls": [
{
"id": "call_b",
"type": "function",
"function": {
"name": "image_generation",
"arguments": marked_args,
},
}
],
}
],
"stream": True,
}
req = ChatCompletionRequest.model_validate(payload)
# Non-native providers: marked builtin tool_call must be dropped
# AND if it was the only payload, the whole message disappears.
for provider_type, base_url in [
("openai", None),
("gemini", "https://litellm.example/v1"),
]:
result = _build_external_messages(
req.messages,
supports_vision = True,
provider_type = provider_type,
base_url = base_url,
)
# Empty assistant turn with only synthetic tool_call dropped.
assert result == [] or all(not (m.get("tool_calls") or []) for m in result), (
provider_type,
result,
)
# Native Gemini preserves it (round-trips via extra_content).
result_native = _build_external_messages(
req.messages,
supports_vision = True,
provider_type = "gemini",
base_url = "https://generativelanguage.googleapis.com/v1beta",
)
assert len(result_native) == 1
assert result_native[0]["tool_calls"][0]["function"]["name"] == "image_generation"
def test_openai_responses_tool_choice_none_drops_hosted_tools(monkeypatch):
"""Round 18: `tool_choice="none"` must also drop hosted OpenAI
Responses builtins (web_search, code execution shell, image
generation), not just user function tools."""
captured: dict = {"body": None}
def handler(request: httpx.Request) -> httpx.Response:
captured["body"] = json.loads(request.content.decode("utf-8"))
return httpx.Response(
200,
content = b'data: {"type":"response.completed","response":{"output":[],"usage":{"input_tokens":1,"output_tokens":1}}}\n\n',
headers = {"content-type": "text/event-stream"},
)
_mock_http(monkeypatch, handler)
async def run():
client = ExternalProviderClient(
provider_type = "openai",
base_url = "https://api.openai.com/v1",
api_key = "sk-test",
)
async for _ in client.stream_chat_completion(
messages = [{"role": "user", "content": "hi"}],
model = "gpt-5.5",
temperature = 0.7,
top_p = 1.0,
max_tokens = 16,
enabled_tools = ["web_search", "code_execution", "image_generation"],
tool_choice = "none",
):
pass
await client.close()
_drive(run())
body = captured["body"] or {}
assert body.get("tools") in (None, []), body
def test_anthropic_tool_choice_none_drops_hosted_tools(monkeypatch):
"""Round 19: tool_choice="none" must opt out of Anthropic hosted
builtins (web_search, web_fetch, code_execution) just like it does
for Gemini and OpenAI Responses."""
captured: dict = {"body": None}
def handler(request: httpx.Request) -> httpx.Response:
captured["body"] = json.loads(request.content.decode("utf-8"))
return httpx.Response(
200,
content = b'event: message_stop\ndata: {"type":"message_stop"}\n\n',
headers = {"content-type": "text/event-stream"},
)
_mock_http(monkeypatch, handler)
async def run():
client = ExternalProviderClient(
provider_type = "anthropic",
base_url = "https://api.anthropic.com",
api_key = "sk-ant-test",
)
async for _ in client.stream_chat_completion(
messages = [{"role": "user", "content": "hi"}],
model = "claude-sonnet-4-5",
temperature = 0.7,
top_p = 0.95,
max_tokens = 16,
enabled_tools = ["web_search", "web_fetch", "code_execution"],
tool_choice = "none",
):
pass
await client.close()
_drive(run())
body = captured["body"] or {}
assert body.get("tools") in (None, []), body
def test_openrouter_tool_choice_none_drops_web_plugin(monkeypatch):
"""Round 19: tool_choice="none" must drop the OpenRouter web
plugin so a request that opted out of tool use does not still
trigger hosted web search."""
captured: dict = {"body": None}
def handler(request: httpx.Request) -> httpx.Response:
captured["body"] = json.loads(request.content.decode("utf-8"))
return httpx.Response(
200,
content = b"data: [DONE]\n\n",
headers = {"content-type": "text/event-stream"},
)
_mock_http(monkeypatch, handler)
async def run():
client = ExternalProviderClient(
provider_type = "openrouter",
base_url = "https://openrouter.ai/api/v1",
api_key = "sk-or-test",
)
async for _ in client.stream_chat_completion(
messages = [{"role": "user", "content": "hi"}],
model = "openai/gpt-5.5",
temperature = 0.7,
top_p = 0.95,
max_tokens = 16,
enabled_tools = ["web_search"],
tool_choice = "none",
):
pass
await client.close()
_drive(run())
body = captured["body"] or {}
assert body.get("plugins") in (None, []), body
def test_kimi_tool_choice_none_skips_web_search_helper(monkeypatch):
"""Round 19: when tool_choice="none" plus enabled_tools=
["web_search"] on Kimi, the dispatcher must NOT route into
`_stream_kimi_web_search`. Falling through to the generic OAI-
compat path is the expected behavior."""
routed_to_helper = {"called": False}
real_helper = ExternalProviderClient._stream_kimi_web_search
async def fake_helper(self, *args, **kwargs): # noqa: ARG001
routed_to_helper["called"] = True
if False:
yield "" # pragma: no cover
monkeypatch.setattr(
ExternalProviderClient,
"_stream_kimi_web_search",
fake_helper,
)
def handler(request: httpx.Request) -> httpx.Response:
return httpx.Response(
200,
content = b"data: [DONE]\n\n",
headers = {"content-type": "text/event-stream"},
)
_mock_http(monkeypatch, handler)
async def run():
client = ExternalProviderClient(
provider_type = "kimi",
base_url = "https://api.moonshot.ai/v1",
api_key = "sk-kimi-test",
)
async for _ in client.stream_chat_completion(
messages = [{"role": "user", "content": "hi"}],
model = "kimi-k2.6",
temperature = 0.7,
top_p = 0.95,
max_tokens = 16,
enabled_tools = ["web_search"],
tool_choice = "none",
):
pass
await client.close()
_drive(run())
assert routed_to_helper["called"] is False
monkeypatch.setattr(
ExternalProviderClient,
"_stream_kimi_web_search",
real_helper,
)
def test_user_code_execution_function_not_dropped():
"""Round 19: a user-declared function literally named
`code_execution` with normal `code` arguments must survive
`_build_external_messages` -- round 17's shape heuristic dropped
it, which broke function-calling round-trips."""
from models.inference import ChatCompletionRequest
from routes.inference import _build_external_messages
payload = {
"model": "gpt-5.5",
"messages": [
{
"role": "assistant",
"content": "",
"tool_calls": [
{
"id": "call_user",
"type": "function",
"function": {
"name": "code_execution",
"arguments": '{"code": "print(1)"}',
},
}
],
}
],
"stream": True,
}
req = ChatCompletionRequest.model_validate(payload)
result = _build_external_messages(
req.messages,
supports_vision = True,
provider_type = "openai",
base_url = None,
)
assert len(result) == 1, result
tcs = result[0].get("tool_calls") or []
assert len(tcs) == 1, result
assert tcs[0]["function"]["name"] == "code_execution"
def test_native_part_code_execution_treated_as_server_side():
"""Round 19: a Gemini `code_execution` card persists its replay
payload at `args.google.native_part` (no `_server_tool` marker on
pre-PR cards). The backend filter must still drop it for non-native
providers because it is a synthetic card, not a real user function."""
from models.inference import ChatCompletionRequest
from routes.inference import _build_external_messages
args_with_native_part = json.dumps(
{
"google": {
"native_part": {
"executableCode": {
"language": "PYTHON",
"code": "print(1)",
}
}
}
}
)
payload = {
"model": "gpt-5.5",
"messages": [
{
"role": "assistant",
"content": "",
"tool_calls": [
{
"id": "call_x",
"type": "function",
"function": {
"name": "code_execution",
"arguments": args_with_native_part,
},
}
],
}
],
"stream": True,
}
req = ChatCompletionRequest.model_validate(payload)
result = _build_external_messages(
req.messages,
supports_vision = True,
provider_type = "openai",
base_url = None,
)
assert result == [] or all(not (m.get("tool_calls") or []) for m in result), result
def test_remote_image_fetch_attempt_cap_includes_failures(monkeypatch):
"""Round 19: the per-request image fetch count cap must count
ATTEMPTS, not just successes. Otherwise a request with 100
failing/slow URLs runs 100 fetches each up to the 15s timeout."""
fetch_calls: list[str] = []
async def fake_fetch(url, fallback_mime, max_bytes = None):
fetch_calls.append(url)
return None
monkeypatch.setattr(ep_mod, "_safe_fetch_image_for_gemini", fake_fetch)
def handler(request: httpx.Request) -> httpx.Response:
return httpx.Response(
200,
content = _gemini_sse(
[
{
"candidates": [
{
"content": {
"role": "model",
"parts": [{"text": "ok"}],
},
"finishReason": "STOP",
}
],
"usageMetadata": {
"promptTokenCount": 1,
"candidatesTokenCount": 1,
},
}
]
),
headers = {"content-type": "text/event-stream"},
)
_mock_http(monkeypatch, handler)
async def run():
client = _make_gemini_client()
image_parts = [
{
"type": "image_url",
"image_url": {"url": f"https://cdn.example.com/img{idx}.png"},
}
for idx in range(20)
]
async for _ in client.stream_chat_completion(
messages = [
{
"role": "user",
"content": [
{"type": "text", "text": "describe"},
*image_parts,
],
}
],
model = "gemini-2.5-flash",
temperature = 0.7,
top_p = 0.95,
max_tokens = 64,
):
pass
await client.close()
_drive(run())
assert len(fetch_calls) <= 8, len(fetch_calls)
def test_orphan_function_call_output_dropped_when_call_skipped(monkeypatch):
"""Round 19: when a marked server-side builtin `function_call` is
dropped from OpenAI Responses input items, the matching role=tool
follow-up must also be dropped to avoid an orphan
`function_call_output`."""
captured: dict = {"input_items": None}
def handler(request: httpx.Request) -> httpx.Response:
body = json.loads(request.content.decode("utf-8"))
captured["input_items"] = body.get("input")
return httpx.Response(
200,
content = b'data: {"type":"response.completed","response":{"output":[],"usage":{"input_tokens":1,"output_tokens":1}}}\n\n',
headers = {"content-type": "text/event-stream"},
)
_mock_http(monkeypatch, handler)
async def run():
client = ExternalProviderClient(
provider_type = "openai",
base_url = "https://api.openai.com/v1",
api_key = "sk-test",
)
async for _ in client.stream_chat_completion(
messages = [
{"role": "user", "content": "search please"},
{
"role": "assistant",
"content": "",
"tool_calls": [
{
"id": "call_b",
"type": "function",
"function": {
"name": "web_search",
"arguments": json.dumps(
{"_server_tool": True, "query": "x"}
),
},
}
],
},
{
"role": "tool",
"content": "result_text",
"tool_call_id": "call_b",
"name": "web_search",
},
{"role": "user", "content": "continue"},
],
model = "gpt-5.5",
temperature = 0.7,
top_p = 1.0,
max_tokens = 16,
):
pass
await client.close()
_drive(run())
items = captured["input_items"] or []
fn_calls = [i for i in items if i.get("type") == "function_call"]
fn_outs = [i for i in items if i.get("type") == "function_call_output"]
assert all(c.get("call_id") != "call_b" for c in fn_calls), items
assert all(o.get("call_id") != "call_b" for o in fn_outs), items
def test_schema_multitype_union_with_null_preserves_anyof(monkeypatch):
"""Round 19: a JSON Schema `"type": ["string","integer","null"]`
must be sanitized to anyOf:[{string},{integer}] + nullable:true.
Flattening to just `{"type":"string"}` silently drops the integer
branch and changes the function contract."""
captured = _capture_body(
monkeypatch,
tools = [
{
"type": "function",
"function": {
"name": "lookup",
"parameters": {
"type": "object",
"properties": {
"either": {"type": ["string", "integer", "null"]},
},
},
},
}
],
)
decls = next(
t["functionDeclarations"]
for t in captured["body"].get("tools") or []
if "functionDeclarations" in t
)
either = decls[0]["parameters"]["properties"]["either"]
assert either.get("nullable") is True
inner = either.get("anyOf")
assert isinstance(inner, list) and len(inner) == 2, either
types = sorted(
b.get("type") for b in inner if isinstance(b, dict) and b.get("type")
)
assert types == ["integer", "string"], inner
def test_invalid_gemini_model_rejected_before_image_fetch(monkeypatch):
"""Round 19: invalid Gemini model IDs are rejected at the top of
`_stream_gemini`, BEFORE any user-controlled remote image fetch
runs."""
fetch_calls: list[str] = []
async def fake_fetch(url, fallback_mime, max_bytes = None):
fetch_calls.append(url)
return None
monkeypatch.setattr(ep_mod, "_safe_fetch_image_for_gemini", fake_fetch)
def handler(request: httpx.Request) -> httpx.Response:
return httpx.Response(
200,
content = b"",
headers = {"content-type": "text/event-stream"},
)
_mock_http(monkeypatch, handler)
async def run():
client = _make_gemini_client()
async for _ in client.stream_chat_completion(
messages = [
{
"role": "user",
"content": [
{"type": "text", "text": "hi"},
{
"type": "image_url",
"image_url": {"url": "https://cdn.example.com/x.png"},
},
],
}
],
model = "../cachedContents/leak",
temperature = 0.7,
top_p = 0.95,
max_tokens = 64,
):
pass
await client.close()
_drive(run())
assert fetch_calls == [], fetch_calls
def test_empty_assistant_turn_skipped_after_synthetic_tool_calls_dropped():
"""Round 20: when `_filter_tool_calls` drops every synthetic
server-builtin tool_call on an empty-content assistant turn, the
whole message must be skipped. Forwarding
`{"role":"assistant","content":""}` is rejected by several
providers as an empty assistant turn."""
from models.inference import ChatCompletionRequest
from routes.inference import _build_external_messages
marked_args = json.dumps({"_server_tool": True, "kind": "image"})
payload = {
"model": "gpt-5.5",
"messages": [
{
"role": "assistant",
"content": "",
"tool_calls": [
{
"id": "call_b",
"type": "function",
"function": {
"name": "image_generation",
"arguments": marked_args,
},
}
],
}
],
"stream": True,
}
req = ChatCompletionRequest.model_validate(payload)
for provider_type, base_url in [
("openai", None),
("gemini", "https://litellm.example/v1"),
]:
result = _build_external_messages(
req.messages,
supports_vision = True,
provider_type = provider_type,
base_url = base_url,
)
# The empty assistant turn (only a synthetic builtin) must
# NOT appear in the output at all.
assert result == [], (provider_type, result)
def test_role_tool_dropped_when_matching_synthetic_call_filtered():
"""Round 20: `_build_external_messages` drops the matching role=
tool follow-up when its tool_call was a synthetic builtin that
`_filter_tool_calls` removed. Otherwise the receiving provider
sees an orphan tool_result with no tool_call."""
from models.inference import ChatCompletionRequest
from routes.inference import _build_external_messages
marked_args = json.dumps({"_server_tool": True, "query": "x"})
payload = {
"model": "gpt-5.5",
"messages": [
{
"role": "assistant",
"content": "",
"tool_calls": [
{
"id": "call_b",
"type": "function",
"function": {
"name": "web_search",
"arguments": marked_args,
},
}
],
},
{
"role": "tool",
"content": "result_text",
"tool_call_id": "call_b",
"name": "web_search",
},
{"role": "user", "content": "continue"},
],
"stream": True,
}
req = ChatCompletionRequest.model_validate(payload)
result = _build_external_messages(
req.messages,
supports_vision = True,
provider_type = "openai",
base_url = None,
)
# Only the user "continue" message survives.
roles = [m.get("role") for m in result]
assert roles == ["user"], result
def test_openrouter_no_synthetic_web_search_event_on_tool_choice_none(
monkeypatch,
):
"""Round 20: OpenRouter dispatcher must not emit synthetic
web_search tool_start / tool_end events when tool_choice="none";
otherwise the chat UI shows a search card for a search that
never happened."""
captured_events: list[dict] = []
def handler(request: httpx.Request) -> httpx.Response:
return httpx.Response(
200,
content = b"data: [DONE]\n\n",
headers = {"content-type": "text/event-stream"},
)
_mock_http(monkeypatch, handler)
async def run():
client = ExternalProviderClient(
provider_type = "openrouter",
base_url = "https://openrouter.ai/api/v1",
api_key = "sk-or-test",
)
async for line in client.stream_chat_completion(
messages = [{"role": "user", "content": "hi"}],
model = "openai/gpt-5.5",
temperature = 0.7,
top_p = 0.95,
max_tokens = 16,
enabled_tools = ["web_search"],
tool_choice = "none",
):
if not line.startswith("data: "):
continue
payload = line[len("data: ") :].strip()
if not payload or payload == "[DONE]":
continue
try:
obj = json.loads(payload)
except Exception:
continue
# Backend emits synthetic tool events as a top-level
# `_toolEvent` on the SSE payload (not nested inside
# `delta`). Read both shapes so a future format change
# cannot mask this regression.
evt = obj.get("_toolEvent")
if isinstance(evt, dict):
captured_events.append(evt)
for ch in obj.get("choices") or []:
delta = ch.get("delta") or {}
nested = delta.get("_toolEvent") if isinstance(delta, dict) else None
if isinstance(nested, dict):
captured_events.append(nested)
await client.close()
_drive(run())
# No synthetic web_search tool_start / tool_end emitted.
assert all(
e.get("tool_name") != "web_search" for e in captured_events
), captured_events
def test_anthropic_role_tool_list_content_translates_to_tool_result(
monkeypatch,
):
"""Round 20: an OpenAI-shape role=tool message with list content
(`content=[{"type":"text","text":"result"}]`) must be translated
into Anthropic's native tool_result block, not forwarded as an
invalid role=tool message."""
captured: dict = {"messages": None}
def handler(request: httpx.Request) -> httpx.Response:
body = json.loads(request.content.decode("utf-8"))
captured["messages"] = body.get("messages")
return httpx.Response(
200,
content = b'event: message_stop\ndata: {"type":"message_stop"}\n\n',
headers = {"content-type": "text/event-stream"},
)
_mock_http(monkeypatch, handler)
async def run():
client = ExternalProviderClient(
provider_type = "anthropic",
base_url = "https://api.anthropic.com",
api_key = "sk-ant-test",
)
async for _ in client.stream_chat_completion(
messages = [
{"role": "user", "content": "look up X"},
{
"role": "assistant",
"content": "let me check",
"tool_calls": [
{
"id": "call_a",
"type": "function",
"function": {
"name": "lookup",
"arguments": '{"q":"x"}',
},
}
],
},
{
"role": "tool",
"content": [{"type": "text", "text": "result_text"}],
"tool_call_id": "call_a",
"name": "lookup",
},
{"role": "user", "content": "summarise"},
],
model = "claude-sonnet-4-5",
temperature = 0.7,
top_p = 0.95,
max_tokens = 64,
):
pass
await client.close()
_drive(run())
msgs = captured["messages"] or []
assert all(m.get("role") != "tool" for m in msgs), msgs
tool_results: list[dict] = []
for m in msgs:
if m.get("role") == "user" and isinstance(m.get("content"), list):
tool_results.extend(
b for b in m["content"] if b.get("type") == "tool_result"
)
assert any(
tr.get("tool_use_id") == "call_a" and tr.get("content") == "result_text"
for tr in tool_results
), msgs
def test_data_url_non_image_mime_dropped(monkeypatch):
"""Round 20: a `data:text/html;base64,...` image_url must be
dropped from the outbound Gemini body, not forwarded as
`inlineData.mimeType="text/html"` which Gemini rejects."""
captured = _capture_body(
monkeypatch,
messages = [
{
"role": "user",
"content": [
{"type": "text", "text": "look"},
{
"type": "image_url",
"image_url": {
"url": "data:text/html;base64,PGgxPmhpPC9oMT4=",
},
},
],
}
],
)
parts = captured["body"]["contents"][-1]["parts"]
assert not any("inlineData" in p for p in parts), parts
def test_youtube_filedata_uses_video_mime(monkeypatch):
"""Round 20: YouTube `fileData.fileUri` must declare a video
mimeType, not `image/jpeg` guessed from the URL path."""
captured = _capture_body(
monkeypatch,
messages = [
{
"role": "user",
"content": [
{"type": "text", "text": "summarise"},
{
"type": "image_url",
"image_url": {
"url": "https://www.youtube.com/watch?v=abc",
},
},
],
}
],
)
parts = captured["body"]["contents"][-1]["parts"]
yt = next((p for p in parts if "fileData" in p), None)
assert yt is not None, parts
assert yt["fileData"]["mimeType"].startswith("video/"), yt
def test_openai_responses_assistant_text_serialized_before_function_call(
monkeypatch,
):
"""Round 20: in OpenAI Responses history, the assistant's
visible text for a turn that ALSO emitted a function_call must
serialize BEFORE the function_call item, matching the prior
response.output sequence. Otherwise function_call_output (the
role=tool follow-up) appears to follow an unrelated assistant
message."""
captured: dict = {"input_items": None}
def handler(request: httpx.Request) -> httpx.Response:
body = json.loads(request.content.decode("utf-8"))
captured["input_items"] = body.get("input")
return httpx.Response(
200,
content = b'data: {"type":"response.completed","response":{"output":[],"usage":{"input_tokens":1,"output_tokens":1}}}\n\n',
headers = {"content-type": "text/event-stream"},
)
_mock_http(monkeypatch, handler)
async def run():
client = ExternalProviderClient(
provider_type = "openai",
base_url = "https://api.openai.com/v1",
api_key = "sk-test",
)
async for _ in client.stream_chat_completion(
messages = [
{"role": "user", "content": "weather?"},
{
"role": "assistant",
"content": "Let me check that.",
"tool_calls": [
{
"id": "call_w",
"type": "function",
"function": {
"name": "get_weather",
"arguments": "{}",
},
}
],
},
{
"role": "tool",
"content": "sunny",
"tool_call_id": "call_w",
"name": "get_weather",
},
{"role": "user", "content": "thanks"},
],
model = "gpt-5.5",
temperature = 0.7,
top_p = 1.0,
max_tokens = 16,
):
pass
await client.close()
_drive(run())
items = captured["input_items"] or []
types = [i.get("type") or i.get("role") for i in items]
# Expected order:
# user ("weather?")
# assistant ("Let me check that.")
# function_call (get_weather)
# function_call_output (sunny)
# user ("thanks")
assert types == [
"user",
"assistant",
"function_call",
"function_call_output",
"user",
], items
def test_gemini_tool_choice_none_disables_image_generation(monkeypatch):
"""Round 21: `tool_choice="none"` must also flip the implicit
image-generation hosted tool off on image-tier models. Otherwise
`responseModalities=["TEXT","IMAGE"]` still rides on the outbound
body and the provider can generate (and bill for) image output
despite the explicit OpenAI tool opt-out."""
captured = _capture_body(
monkeypatch,
model = "gemini-2.5-flash-image",
enabled_tools = ["image_generation"],
tool_choice = "none",
)
body = captured["body"]
assert body["generationConfig"].get("responseModalities") == ["TEXT"], body
def test_gemini_forced_function_tool_choice_drops_hosted_builtins(monkeypatch):
"""Round 21: forced-function `tool_choice` (e.g.
`{"type":"function","function":{"name":"lookup"}}`) must suppress
hosted Google Search / code execution. Gemini's toolConfig only
constrains function declarations, not hosted tools, so leaving
`googleSearch`/`codeExecution` in `tools[]` lets them fire despite
the caller pinning a specific user function."""
captured = _capture_body(
monkeypatch,
enabled_tools = ["web_search", "code_execution"],
tools = [
{
"type": "function",
"function": {"name": "lookup", "parameters": {"type": "object"}},
}
],
tool_choice = {
"type": "function",
"function": {"name": "lookup"},
},
)
body = captured["body"]
tool_kinds = [list(t.keys())[0] for t in (body.get("tools") or [])]
assert "googleSearch" not in tool_kinds, body
assert "codeExecution" not in tool_kinds, body
# User function declaration still survives.
assert "functionDeclarations" in tool_kinds, body
def test_gemini_forced_function_tool_choice_drops_image_generation(monkeypatch):
"""Round 21: forced-function `tool_choice` must also flip the
implicit image-generation hosted tool off on image-tier models."""
captured = _capture_body(
monkeypatch,
model = "gemini-2.5-flash-image",
enabled_tools = ["image_generation"],
tool_choice = {
"type": "function",
"function": {"name": "lookup"},
},
tools = [
{
"type": "function",
"function": {"name": "lookup", "parameters": {"type": "object"}},
}
],
)
body = captured["body"]
assert body["generationConfig"].get("responseModalities") == ["TEXT"], body
def test_gemini_code_execution_native_part_list_replays_per_part_signatures(
monkeypatch,
):
"""Round 21: merged code-execution history must replay per-part
`thoughtSignature`s, not fan one top-level signature across every
native subpart. Gemini 3 strict validators reject a signature
placed on the wrong part."""
history = [
{"role": "user", "content": "plot 1+1"},
{
"role": "assistant",
"content": None,
"tool_calls": [
{
"id": "call_a",
"type": "function",
"function": {
"name": "code_execution",
"arguments": "{}",
},
"extra_content": {
"google": {
"native_part": {
"parts": [
{
"executableCode": {
"id": "code_a",
"language": "PYTHON",
"code": "print(1+1)",
},
"thoughtSignature": "SIG-EXEC",
},
{
"codeExecutionResult": {
"id": "res_a",
"outcome": "OUTCOME_OK",
"output": "2\n",
},
},
],
},
},
},
}
],
},
{
"role": "tool",
"tool_call_id": "call_a",
"name": "code_execution",
"content": "2",
},
{"role": "user", "content": "next"},
]
captured = _capture_body(monkeypatch, messages = history)
contents = captured["body"]["contents"]
# Locate the assistant turn replayed as native code-exec parts.
assistant_turn = next(c for c in contents if c["role"] == "model")
parts = assistant_turn["parts"]
exec_parts = [p for p in parts if "executableCode" in p]
result_parts = [p for p in parts if "codeExecutionResult" in p]
assert exec_parts and result_parts, parts
assert exec_parts[0].get("thoughtSignature") == "SIG-EXEC", exec_parts[0]
# codeExecutionResult had no signature -- must NOT inherit one.
assert "thoughtSignature" not in result_parts[0], result_parts[0]
def test_gemini_code_execution_legacy_merged_signature_only_on_executable(
monkeypatch,
):
"""Round 21: backward compatibility for pre-round-21 persisted
history that stored merged `native_part` as a single object plus a
top-level `thoughtSignature`. The replay branch must attach that
signature only to `executableCode` (where Gemini 3 emits it), not
fan it across `codeExecutionResult` / `inlineData`."""
history = [
{"role": "user", "content": "plot 1+1"},
{
"role": "assistant",
"content": None,
"tool_calls": [
{
"id": "call_b",
"type": "function",
"function": {
"name": "code_execution",
"arguments": "{}",
},
"extra_content": {
"google": {
"native_part": {
"executableCode": {
"id": "code_b",
"language": "PYTHON",
"code": "print(1+1)",
},
"codeExecutionResult": {
"id": "res_b",
"outcome": "OUTCOME_OK",
"output": "2\n",
},
"thoughtSignature": "LEGACY-SIG",
},
},
},
}
],
},
{
"role": "tool",
"tool_call_id": "call_b",
"name": "code_execution",
"content": "2",
},
{"role": "user", "content": "next"},
]
captured = _capture_body(monkeypatch, messages = history)
contents = captured["body"]["contents"]
assistant_turn = next(c for c in contents if c["role"] == "model")
exec_parts = [p for p in assistant_turn["parts"] if "executableCode" in p]
result_parts = [p for p in assistant_turn["parts"] if "codeExecutionResult" in p]
assert exec_parts[0].get("thoughtSignature") == "LEGACY-SIG", exec_parts[0]
assert "thoughtSignature" not in result_parts[0], result_parts[0]
def test_gemini_role_tool_list_content_flattens_to_result_text(monkeypatch):
"""Round 21: OpenAI-shape role=tool messages may carry list content
like `[{"type":"text","text":"result"}]`. Forwarding those parts
verbatim into `functionResponse.response.result` yields a list of
content-part objects instead of the actual tool output text."""
history = [
{"role": "user", "content": "look up"},
{
"role": "assistant",
"content": "",
"tool_calls": [
{
"id": "call_1",
"type": "function",
"function": {
"name": "lookup",
"arguments": json.dumps({"q": "x"}),
},
}
],
},
{
"role": "tool",
"tool_call_id": "call_1",
"name": "lookup",
"content": [{"type": "text", "text": "answer-text"}],
},
{"role": "user", "content": "next"},
]
captured = _capture_body(monkeypatch, messages = history)
contents = captured["body"]["contents"]
fn_response = None
for c in contents:
for p in c.get("parts") or []:
if isinstance(p, dict) and "functionResponse" in p:
fn_response = p["functionResponse"]
break
if fn_response:
break
assert fn_response is not None, contents
assert fn_response["response"] == {"result": "answer-text"}, fn_response
def test_safe_fetch_image_threads_per_request_byte_budget(monkeypatch):
"""Round 21: the aggregate per-request byte cap must be passed into
`_safe_fetch_image_for_gemini` so an oversize URL is refused via
Content-Length (short-circuit) rather than fully downloaded then
discarded after the fact."""
import socket
captured: dict = {"reads": 0, "content_length_seen": None}
original_getaddrinfo = socket.getaddrinfo
def fake_getaddrinfo(host, *args, **kwargs):
if host == "cdn.example.com":
return [
(
socket.AF_INET,
socket.SOCK_STREAM,
0,
"",
("8.8.8.8", 0),
)
]
return original_getaddrinfo(host, *args, **kwargs)
monkeypatch.setattr(socket, "getaddrinfo", fake_getaddrinfo)
class _StubResp:
status = 200
# Declared 5 MiB, but caller passes a 1 MiB remaining budget.
headers = {
"content-type": "image/png",
"content-length": str(5 * 1024 * 1024),
}
def __enter__(self):
return self
def __exit__(self, *a):
return False
def read(self, _n = None):
captured["reads"] += 1
return b"\x00" * (5 * 1024 * 1024)
class _StubOpener:
def open(self, req, timeout = None):
return _StubResp()
monkeypatch.setattr(
"urllib.request.build_opener", lambda *_args, **_kw: _StubOpener()
)
res = _drive(
ep_mod._safe_fetch_image_for_gemini(
"https://cdn.example.com/big.png",
"image/png",
max_bytes = 1 * 1024 * 1024,
)
)
assert res is None
# Refused via Content-Length pre-check, never read.
assert captured["reads"] == 0
def test_openai_chat_delta_type_includes_tool_calls_and_extra_content():
"""Round 21: the frontend `OpenAIChatDelta` interface must expose
`tool_calls` and `extra_content` so TypeScript callers can consume
the Gemini-native stream fields without `any` casts. This test is
a static-string assertion against the .ts source; mirrors how other
frontend wire-contract tests are pinned from the backend suite."""
import os
here = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
types_path = os.path.join(
here, "frontend", "src", "features", "chat", "types", "api.ts"
)
with open(types_path, "r", encoding = "utf-8") as f:
src = f.read()
assert "tool_calls?: OpenAIToolCallPart[]" in src, src[:200]
assert "extra_content?: Record<string, unknown>" in src, src[:200]
assert "boolean | string | null" in src, src[:200]
def test_anthropic_forced_function_tool_choice_drops_hosted_tools(monkeypatch):
"""Round 22: forced-function tool_choice must suppress Anthropic
hosted builtins the same way it does for Gemini. Pinning a user
function (`tool_choice={"type":"function","function":{"name":...}}`)
while also passing `enabled_tools=["web_search","web_fetch",
"code_execution"]` should not still fire those server-side."""
captured: dict = {"body": None}
def handler(request: httpx.Request) -> httpx.Response:
captured["body"] = json.loads(request.content.decode("utf-8"))
return httpx.Response(
200,
content = b'event: message_stop\ndata: {"type":"message_stop"}\n\n',
headers = {"content-type": "text/event-stream"},
)
_mock_http(monkeypatch, handler)
async def run():
client = ExternalProviderClient(
provider_type = "anthropic",
base_url = "https://api.anthropic.com",
api_key = "sk-ant-test",
)
async for _ in client.stream_chat_completion(
messages = [{"role": "user", "content": "hi"}],
model = "claude-sonnet-4-5",
temperature = 0.7,
top_p = 0.95,
max_tokens = 16,
enabled_tools = ["web_search", "web_fetch", "code_execution"],
tool_choice = {
"type": "function",
"function": {"name": "lookup_record"},
},
):
pass
await client.close()
_drive(run())
body = captured["body"] or {}
# No hosted tools should be in the body — only the caller's user-
# function declarations (which this test doesn't pass any of).
tools = body.get("tools") or []
hosted_tool_names = {"web_search", "web_fetch", "code_execution"}
for tool in tools:
assert tool.get("name") not in hosted_tool_names, body
def test_openrouter_forced_function_tool_choice_drops_web_plugin(monkeypatch):
"""Round 22: forced-function tool_choice must drop the OpenRouter
web plugin too — caller pinned a user function, OpenRouter must not
still attach the hosted web-search plugin."""
captured: dict = {"body": None}
def handler(request: httpx.Request) -> httpx.Response:
captured["body"] = json.loads(request.content.decode("utf-8"))
return httpx.Response(
200,
content = b"data: [DONE]\n\n",
headers = {"content-type": "text/event-stream"},
)
_mock_http(monkeypatch, handler)
async def run():
client = ExternalProviderClient(
provider_type = "openrouter",
base_url = "https://openrouter.ai/api/v1",
api_key = "sk-or-test",
)
async for _ in client.stream_chat_completion(
messages = [{"role": "user", "content": "hi"}],
model = "openai/gpt-5.5",
temperature = 0.7,
top_p = 0.95,
max_tokens = 16,
enabled_tools = ["web_search"],
tool_choice = {
"type": "function",
"function": {"name": "lookup_record"},
},
):
pass
await client.close()
_drive(run())
body = captured["body"] or {}
assert body.get("plugins") in (None, []), body
def test_kimi_forced_function_tool_choice_skips_web_search_helper(monkeypatch):
"""Round 22: forced-function tool_choice plus enabled_tools=
["web_search"] on Kimi must NOT route into `_stream_kimi_web_search`.
Caller pinned a user function; hosted $web_search should be
suppressed for the same privacy/billing reason."""
routed_to_helper = {"called": False}
async def fake_helper(self, *args, **kwargs): # noqa: ARG001
routed_to_helper["called"] = True
if False:
yield "" # pragma: no cover
monkeypatch.setattr(
ExternalProviderClient,
"_stream_kimi_web_search",
fake_helper,
)
def handler(request: httpx.Request) -> httpx.Response:
return httpx.Response(
200,
content = b"data: [DONE]\n\n",
headers = {"content-type": "text/event-stream"},
)
_mock_http(monkeypatch, handler)
async def run():
client = ExternalProviderClient(
provider_type = "kimi",
base_url = "https://api.moonshot.ai/v1",
api_key = "sk-kimi-test",
)
async for _ in client.stream_chat_completion(
messages = [{"role": "user", "content": "hi"}],
model = "kimi-k2.6",
temperature = 0.7,
top_p = 0.95,
max_tokens = 16,
enabled_tools = ["web_search"],
tool_choice = {
"type": "function",
"function": {"name": "lookup_record"},
},
):
pass
await client.close()
_drive(run())
assert not routed_to_helper["called"]
def test_openai_responses_forced_function_tool_choice_drops_hosted_tools(monkeypatch):
"""Round 23: forced-function tool_choice on the OpenAI Responses
path must suppress hosted builtins (web_search, shell,
image_generation) the same way it does for Gemini / Anthropic /
OpenRouter / Kimi. User-defined function tools still flow through
so the pinned function can resolve."""
captured: dict = {"body": None}
def handler(request: httpx.Request) -> httpx.Response:
captured["body"] = json.loads(request.content.decode("utf-8"))
return httpx.Response(
200,
content = b"event: response.completed\ndata: {}\n\n",
headers = {"content-type": "text/event-stream"},
)
_mock_http(monkeypatch, handler)
async def run():
client = ExternalProviderClient(
provider_type = "openai",
base_url = "https://api.openai.com/v1",
api_key = "sk-openai-test",
)
async for _ in client.stream_chat_completion(
messages = [{"role": "user", "content": "hi"}],
model = "gpt-5",
temperature = 0.7,
top_p = 0.95,
max_tokens = 16,
enabled_tools = ["web_search", "code_execution", "image_generation"],
tools = [
{
"type": "function",
"function": {
"name": "lookup_record",
"parameters": {"type": "object", "properties": {}},
},
},
],
tool_choice = {
"type": "function",
"function": {"name": "lookup_record"},
},
):
pass
await client.close()
_drive(run())
body = captured["body"] or {}
tools = body.get("tools") or []
hosted_types = {"web_search", "shell", "image_generation"}
hosted_seen = {t.get("type") for t in tools if isinstance(t, dict)}
assert not (hosted_seen & hosted_types), body
# The user function declaration must still be present so the pin
# has something to target.
user_function_seen = any(
isinstance(t, dict) and t.get("type") == "function" for t in tools
)
assert user_function_seen, body
# And the forced-function tool_choice must be forwarded in Responses
# shape: `{type:"function", name:"..."}`.
tc = body.get("tool_choice")
assert isinstance(tc, dict) and tc.get("type") == "function", body
assert tc.get("name") == "lookup_record", body
def test_strip_provider_synthetic_tool_history_drops_text_only_extra_content():
"""Round 24: a plain text Gemini reply (no tool_calls) carrying
`extra_content.google.thought_signature` must still have that
metadata stripped before being forwarded to a local llama-server
backend. Without this, switching a Gemini thread mid-stream to a
local GGUF model leaks Gemini-only fields to llama-server."""
from routes.inference import _strip_provider_synthetic_tool_history
messages = [
{"role": "user", "content": "hi"},
{
"role": "assistant",
"content": "Hello!",
"extra_content": {"google": {"thought_signature": "SIG_ABC"}},
},
{"role": "user", "content": "now in pirate voice"},
]
out = _strip_provider_synthetic_tool_history(messages)
# Same three turns, but the assistant's `extra_content` is gone.
assert [m["role"] for m in out] == ["user", "assistant", "user"]
assistant = out[1]
assert "extra_content" not in assistant, assistant
assert assistant["content"] == "Hello!"
def test_validate_and_resolve_host_blocks_shared_address_space():
"""Round 24 SSRF P1: 100.64.0.0/10 carrier-grade NAT addresses are
`is_private=False` AND `is_global=False` per Python's ipaddress
docs. The previous denylist (is_private/loopback/link_local/etc.)
missed them. Adding `not ip.is_global` as the primary gate covers
all non-public ranges, current and future."""
import socket as _socket
from core.inference import tools as _tools
orig_getaddrinfo = _socket.getaddrinfo
def fake_getaddrinfo(hostname, port, *args, **kwargs):
if hostname == "shared.example":
return [
(
_socket.AF_INET,
_socket.SOCK_STREAM,
0,
"",
("100.64.0.1", port),
),
]
return orig_getaddrinfo(hostname, port, *args, **kwargs)
_socket.getaddrinfo = fake_getaddrinfo
try:
ok, reason, _ip = _tools._validate_and_resolve_host("shared.example", 443)
finally:
_socket.getaddrinfo = orig_getaddrinfo
assert ok is False, (ok, reason)
assert "non-public" in reason.lower() or "100.64.0.1" in reason
def test_gemini_custom_oai_compat_base_skips_native_allowlist():
"""Round 24: a custom Gemini OAI-compatible base (LiteLLM/proxy)
must NOT have its model list filtered through the native Gemini
allowlist regex. A LiteLLM gateway returning
`["google/gemini-2.5-flash", "my-team/gemini", "gemini-2.5-flash"]`
should be passed through; the native filter would strip the
prefixed IDs even though the chat dispatch routes them via the
OpenAI-compatible client."""
import asyncio as _asyncio
from routes import providers as _providers
from routes.providers import (
ProviderModelsRequest,
list_provider_models,
)
captured: dict = {"base": None}
class _FakeClient:
def __init__(self, *, base_url, **kwargs):
captured["base"] = base_url
async def list_models(self):
return [
{"id": "google/gemini-2.5-flash"},
{"id": "my-team/gemini"},
{"id": "gemini-2.5-flash"},
]
async def close(self):
return None
orig = _providers.ExternalProviderClient
_providers.ExternalProviderClient = _FakeClient
try:
req = ProviderModelsRequest(
provider_type = "gemini",
base_url = "https://litellm.example/v1",
)
result = _asyncio.run(list_provider_models(req, current_subject = "unsloth"))
finally:
_providers.ExternalProviderClient = orig
ids = {m.id for m in result}
# All three IDs survive — the native allowlist was bypassed.
assert "google/gemini-2.5-flash" in ids, ids
assert "my-team/gemini" in ids, ids
assert "gemini-2.5-flash" in ids, ids
def test_strip_provider_synthetic_tool_history_drops_synthetic_only():
"""Round 22: switching a thread from native Gemini (code_execution
/ image_generation tool_cards in history) to a local GGUF backend
must strip the synthetic tool_calls + matching role=tool replies
before llama-server sees them. Real user-function tool_calls and
their matching tool replies must survive."""
from routes.inference import _strip_provider_synthetic_tool_history
messages = [
{"role": "user", "content": "hi"},
{
"role": "assistant",
"content": "let me run it",
"tool_calls": [
{
"id": "synth_ce_1",
"type": "function",
"function": {
"name": "code_execution",
"arguments": json.dumps(
{
"_server_tool": True,
"google": {"native_part": {"parts": []}},
}
),
},
"extra_content": {"google": {"thought_signature": "abc"}},
},
{
"id": "real_lookup",
"type": "function",
"function": {
"name": "lookup_user",
"arguments": json.dumps({"id": 42}),
},
},
],
"extra_content": {"google": {"thought_signature": "msglevel"}},
},
{
"role": "tool",
"tool_call_id": "synth_ce_1",
"content": "Gemini-only result text",
},
{
"role": "tool",
"tool_call_id": "real_lookup",
"content": '{"name": "alice"}',
},
]
out = _strip_provider_synthetic_tool_history(messages)
assistant = next(m for m in out if m.get("role") == "assistant")
tcs = assistant["tool_calls"]
assert len(tcs) == 1, tcs
assert tcs[0]["id"] == "real_lookup"
assert "extra_content" not in tcs[0]
assert "extra_content" not in assistant
tool_msgs = [m for m in out if m.get("role") == "tool"]
assert len(tool_msgs) == 1
assert tool_msgs[0]["tool_call_id"] == "real_lookup"
def test_strip_provider_synthetic_tool_history_drops_empty_assistant():
"""If every tool_call was synthetic and the assistant turn had no
content, the entire turn must be dropped (llama-server rejects
empty assistant messages with no tool_calls)."""
from routes.inference import _strip_provider_synthetic_tool_history
messages = [
{"role": "user", "content": "draw a sloth"},
{
"role": "assistant",
"content": "",
"tool_calls": [
{
"id": "synth_imggen",
"type": "function",
"function": {
"name": "image_generation",
"arguments": json.dumps(
{
"google": {
"native_part": {
"parts": [
{
"inlineData": {
"mimeType": "image/png",
"data": "Zm9v",
}
}
]
}
}
}
),
},
}
],
},
{"role": "tool", "tool_call_id": "synth_imggen", "content": "(image)"},
{"role": "user", "content": "now try in pirate voice"},
]
out = _strip_provider_synthetic_tool_history(messages)
roles = [m.get("role") for m in out]
# The synthetic assistant + its tool reply are both gone; only the
# two user turns survive.
assert roles == ["user", "user"], out
def test_openrouter_no_synthetic_web_search_event_on_forced_function_tool_choice(
monkeypatch,
):
"""Round 22 sibling of the round-20 `tool_choice='none'` test: when
the caller forces a specific function via `tool_choice={"type":
"function", ...}` AND passes `enabled_tools=["web_search"]`, the
OpenRouter path must NOT synthesize a fake `web_search` tool card.
The plugin was not attached upstream so the UI must not see a
server-tool card."""
captured_events: list[dict] = []
def handler(request: httpx.Request) -> httpx.Response:
return httpx.Response(
200,
content = (
b'data: {"choices":[{"delta":{"content":"ok"}}]}\n\n'
b"data: [DONE]\n\n"
),
headers = {"content-type": "text/event-stream"},
)
_mock_http(monkeypatch, handler)
async def run():
client = ExternalProviderClient(
provider_type = "openrouter",
base_url = "https://openrouter.ai/api/v1",
api_key = "sk-or-test",
)
async for line in client.stream_chat_completion(
messages = [{"role": "user", "content": "hi"}],
model = "openai/gpt-5.5",
temperature = 0.7,
top_p = 0.95,
max_tokens = 16,
enabled_tools = ["web_search"],
tool_choice = {
"type": "function",
"function": {"name": "lookup_record"},
},
):
payload = line.strip().removeprefix("data: ")
if payload and payload != "[DONE]":
try:
captured_events.append(json.loads(payload))
except Exception:
pass
await client.close()
_drive(run())
for evt in captured_events:
for choice in evt.get("choices") or []:
delta = choice.get("delta") or {}
extra = delta.get("extra_content") or {}
tool_event = extra.get("toolEvent") if isinstance(extra, dict) else None
if isinstance(tool_event, dict):
assert tool_event.get("tool_name") != "web_search", evt