563fcf8952* Studio: detect reasoning_effort and preserve_thinking in chat templates
Previously Studio's chat template sniffer only recognized Qwen's
enable_thinking and DeepSeek's thinking markers. For gpt-oss (Harmony
templates) and newer Qwen3.6 templates, the Think toggle was hidden or
could only be flipped on/off.
This change adds two new detections and corresponding UI controls:
1. reasoning_effort style (gpt-oss). When the chat template contains
reasoning_effort, the Think button becomes a Low / Medium / High
dropdown and the backend forwards {"reasoning_effort": <level>} in
chat_template_kwargs. Load-time --chat-template-kwargs flag is also
switched to the new style.
2. preserve_thinking kwarg (Qwen3.6). Independent of the reasoning
toggle. When the template mentions preserve_thinking, a new
Preserve Thinking on/off pill is shown next to Think. Off by
default, persisted via localStorage. When on, the backend adds
{"preserve_thinking": true} to chat_template_kwargs so past-turn
<think> blocks are kept in the prompt instead of being stripped.
Backend helper _request_reasoning_kwargs now merges all applicable
kwargs into a single chat_template_kwargs dict based on the model's
detected style and template capabilities. Inputs are validated with
Literal types in the Pydantic request model.
Tested end to end against cached GGUFs for unsloth/gpt-oss-20b-GGUF and
unsloth/Qwen3.6-35B-A3B-GGUF. Confirmed the llama-server startup
--chat-template-kwargs flag and per-request JSON body carry the
expected keys for all combinations.
* Studio: review pass and CI format fixes for reasoning-styles PR
Addresses review feedback and pre-commit CI:
- Preserve Thinking pill in shared-composer now gates on modelLoaded
only, matching the thread.tsx toggle. Previously the inline version
disabled whenever supports_reasoning was false.
- The non-GGUF already_loaded LoadResponse now emits reasoning_style
(and supports_preserve_thinking=False) so a reconnecting frontend
sees the correct style for an already-running gpt-oss safetensors
model.
- use-chat-model-runtime reconnect path now always clears
reasoningEnabled for models without reasoning support instead of
inheriting the previous model's state.
- _reasoning_default is now reset alongside the other reasoning flags
in both backend reset blocks.
- supports_reasoning description updated to mention reasoning_effort
alongside enable_thinking.
- Ran scripts/run_ruff_format.py on the touched Python files to
satisfy pre-commit.ci.
* Studio: detect reasoning flags on safetensors load + share Qwen param helper
Addresses bot review feedback:
- Extract the chat-template substring sniffer out of _read_gguf_metadata
into a module-level detect_reasoning_flags(template, model_id) helper.
Also runs on the safetensors / transformers load paths:
- POST /api/inference/load non-GGUF LoadResponse
- already_loaded non-GGUF early return
- GET /api/inference/status non-GGUF branch
The gpt-oss fallback via backend._is_gpt_oss_model() is preserved so
safetensors gpt-oss still surfaces reasoning controls even when no
chat_template is stored on the model record.
- Deduplicate the Qwen3 / Qwen3.5 / Qwen3.6 Think-toggle parameter
adjustment into a single features/chat/utils/qwen-params.ts. Both
the assistant-ui Think toggle (thread.tsx) and the shared composer
(shared-composer.tsx) now import the same helper. The superset that
applies presence_penalty=1.5 for Qwen3.5 and Qwen3.6 is now used by
both sites (thread.tsx previously did not apply it).
* Studio: fill missing reasoning flags on safetensors status + add always_on reset
Round 4 review fixes:
- routes/inference.py safetensors status response now populates
reasoning_always_on and supports_tools from detect_reasoning_flags.
Previously Pydantic defaulted both to False, so safetensors models
with always-on <think> templates or tool-calling templates were
silently losing those flags on /api/inference/status reconnect.
- routes/inference.py already_loaded safetensors branch now falls back
to backend._is_gpt_oss_model() when the chat template is missing,
matching the status-endpoint behaviour.
- Safetensors status endpoint log_source set to "Safetensors status"
so the emitted template-detection log lines are attributable.
- chat-runtime-store clearCheckpoint now also resets reasoningAlwaysOn
so switching from an always-on reasoning model to a non-always-on
one does not leave the Think button permanently locked on.
* Studio: skip reasoning kwargs when always-on; narrow non-GGUF advertisement
Round 5 addresses reviewer feedback:
- _request_reasoning_kwargs and the load-time --chat-template-kwargs
emission now skip when _reasoning_always_on is true. Templates with
hardcoded <think> tags do not consume enable_thinking / reasoning_effort
so sending them was noise.
- Non-GGUF (Unsloth / transformers) LoadResponse and InferenceStatusResponse
paths no longer advertise template-derived supports_reasoning /
reasoning_style / supports_preserve_thinking / supports_tools. The
transformers generation path does not yet forward chat_template_kwargs
to tokenizer.apply_chat_template, so exposing the UI controls on those
models was misleading. Only the gpt-oss Harmony case is kept
(reasoning_style = reasoning_effort) because it is handled via the
HarmonyTextStreamer at the tokenizer level. A follow-up PR can thread
chat_template_kwargs through the transformers path and re-enable the
broader detection.
- GGUF / llama-server paths keep the full detect_reasoning_flags output.
|
||
|---|---|---|
| .. | ||
| backend | ||
| frontend | ||
| src-tauri | ||
| __init__.py | ||
| install_llama_prebuilt.py | ||
| install_python_stack.py | ||
| LICENSE.AGPL-3.0 | ||
| setup.bat | ||
| setup.ps1 | ||
| setup.sh | ||
| Unsloth_Studio_Colab.ipynb | ||