ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Per-provider stop cap on Kimi web-search bypass and frontend sheet

Browse source 
Round 5 review flagged two asymmetries:

1. Kimi web-search bypass hard-capped stops at 4 while the default OAI-compat path honours provider_info["stop_max"]. Apply the same provider-aware logic in _stream_kimi_web_search so kimi-with-search and kimi-without-search match. Also add Kimi's documented 5-stop max (https://platform.kimi.ai/docs/api/chat) to the provider registry so the cap actually fires.

2. chat-settings-sheet.tsx caps every non-Anthropic external provider at 4 stops. Replace with a per-provider getProviderStopMax helper in provider-capabilities.ts so DeepSeek, Mistral, and local backends are not artificially restricted while OpenAI Chat still hits its 4-entry hard limit and Kimi hits its documented 5-entry cap.

Tests pin the Kimi 5-cap on both Kimi paths.
This commit is contained in:
Daniel Han 2026-05-24 14:49:57 +00:00
commit 95e143545f
5 changed files with 107 additions and 26 deletions
Download patch file Download diff file Expand all files Collapse all files

27
studio/backend/core/inference/external_provider.py
View file

@ -895,41 +895,40 @@ class ExternalProviderClient:
body["max_tokens"] = max_tokens
# The default OAI-compat body construction is skipped because
# this helper returns early. Forward the optional sampling
# extensions here so kimi-with-search behaves the same as
# this helper returns early. Apply the same provider-aware
# sampling / stop logic here so kimi-with-search matches
# kimi-without-search.
from core.inference.providers import get_provider_info
provider_info = get_provider_info(self.provider_type) or {}
if presence_penalty is not None:
body["presence_penalty"] = presence_penalty
if frequency_penalty is not None:
body["frequency_penalty"] = frequency_penalty
if seed is not None:
body["seed"] = seed
seed_field = provider_info.get("seed_field", "seed")
body[seed_field] = seed
if stop:
# Match the default OAI-compat path: forward a single string
# verbatim, dedupe + cap lists to 4 (OpenAI hard limit).
stop_max = int(provider_info.get("stop_max", 16))
if isinstance(stop, str):
body["stop"] = stop
elif isinstance(stop, list):
sequences = list(
dict.fromkeys(s for s in stop if isinstance(s, str) and s)
)
if len(sequences) > 4:
if len(sequences) > stop_max:
logger.warning(
"stop sequences truncated to 4 entries "
"(received %d, OpenAI's hard cap is 4)",
"stop sequences truncated to %d entries (received %d)",
stop_max,
len(sequences),
)
body["stop"] = sequences[:4]
body["stop"] = sequences[:stop_max]
elif sequences:
body["stop"] = sequences
if parallel_tool_calls is not None:
body["parallel_tool_calls"] = parallel_tool_calls
# Strip body fields the Kimi registry declares unusable
# (temperature/top_p — see body_omit in providers.py).
from core.inference.providers import get_provider_info
provider_info = get_provider_info(self.provider_type) or {}
# Drop body fields the provider's registry entry locks down.
for field in provider_info.get("body_omit", ()):
body.pop(field, None)

3
studio/backend/core/inference/providers.py
View file

@ -162,6 +162,9 @@ PROVIDER_REGISTRY: dict[str, dict[str, Any]] = {
# (and the same shape for top_p). Strip both fields from the
# outbound body so the server falls back to its required defaults.
"body_omit": ("temperature", "top_p"),
# Kimi accepts at most 5 stop strings (each <= 32 bytes) per
# https://platform.kimi.ai/docs/api/chat
"stop_max": 5,
},
"qwen": {
"display_name": "Qwen",