studio: fix stale GGUF metadata when switching models, update default helper model

Reset _supports_reasoning, _supports_tools, _chat_template, and
_context_length at the top of _read_gguf_metadata so flags from a
previously loaded model do not carry over. Without this, loading a
reasoning model (eg Qwen3.5-4B) then switching to a non-reasoning
model (eg Qwen3-4B-Instruct-2507) would keep supports_reasoning=True
from the first model, causing the UI to show "Thought for 0 seconds"
and passing --chat-template-kwargs enable_thinking to a model whose
chat template does not support it.

Also update the default helper GGUF from Qwen3-4B-Instruct-2507-GGUF
to Qwen3.5-4B-GGUF to match the frontend fallback auto-load model.
This commit is contained in:
Daniel Han 2026-03-17 07:46:21 +00:00
commit 852755cc92
2 changed files with 8 additions and 1 deletions

View file

@ -446,6 +446,13 @@ class LlamaCppBackend:
Parses only the KV pairs we need (~30ms even for multi-GB files).
For split GGUFs, metadata is always in shard 1.
"""
# Reset metadata from any previously loaded model so stale flags
# (eg _supports_reasoning) do not carry over when switching models.
self._context_length = None
self._chat_template = None
self._supports_reasoning = False
self._supports_tools = False
try:
WANTED = {"general.architecture", "tokenizer.chat_template"}
arch = None

View file

@ -26,7 +26,7 @@ from loggers import get_logger
logger = get_logger(__name__)
DEFAULT_HELPER_MODEL_REPO = "unsloth/Qwen3-4B-Instruct-2507-GGUF"
DEFAULT_HELPER_MODEL_REPO = "unsloth/Qwen3.5-4B-GGUF"
DEFAULT_HELPER_MODEL_VARIANT = "UD-Q4_K_XL"
README_MAX_CHARS = 1500