studio: fix stale GGUF metadata when switching models, update default helper model
Reset _supports_reasoning, _supports_tools, _chat_template, and _context_length at the top of _read_gguf_metadata so flags from a previously loaded model do not carry over. Without this, loading a reasoning model (eg Qwen3.5-4B) then switching to a non-reasoning model (eg Qwen3-4B-Instruct-2507) would keep supports_reasoning=True from the first model, causing the UI to show "Thought for 0 seconds" and passing --chat-template-kwargs enable_thinking to a model whose chat template does not support it. Also update the default helper GGUF from Qwen3-4B-Instruct-2507-GGUF to Qwen3.5-4B-GGUF to match the frontend fallback auto-load model.
This commit is contained in:
parent
eeffa4c065
commit
852755cc92
2 changed files with 8 additions and 1 deletions
|
|
@ -446,6 +446,13 @@ class LlamaCppBackend:
|
|||
Parses only the KV pairs we need (~30ms even for multi-GB files).
|
||||
For split GGUFs, metadata is always in shard 1.
|
||||
"""
|
||||
# Reset metadata from any previously loaded model so stale flags
|
||||
# (eg _supports_reasoning) do not carry over when switching models.
|
||||
self._context_length = None
|
||||
self._chat_template = None
|
||||
self._supports_reasoning = False
|
||||
self._supports_tools = False
|
||||
|
||||
try:
|
||||
WANTED = {"general.architecture", "tokenizer.chat_template"}
|
||||
arch = None
|
||||
|
|
|
|||
|
|
@ -26,7 +26,7 @@ from loggers import get_logger
|
|||
|
||||
logger = get_logger(__name__)
|
||||
|
||||
DEFAULT_HELPER_MODEL_REPO = "unsloth/Qwen3-4B-Instruct-2507-GGUF"
|
||||
DEFAULT_HELPER_MODEL_REPO = "unsloth/Qwen3.5-4B-GGUF"
|
||||
DEFAULT_HELPER_MODEL_VARIANT = "UD-Q4_K_XL"
|
||||
|
||||
README_MAX_CHARS = 1500
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue