e4d1499230* fix(studio): prevent small models from stalling on tool-calling tasks
Small GGUF models (< 9B params) in "Think, Search, Code" mode would
often describe what they planned to do ("Let me create this dashboard")
and then stop generating without ever calling a tool.
Three changes:
1. Simplify web_tips for small models: remove the "fetch its full content
by calling web_search with the url parameter" guidance for models < 9B.
This multi-step instruction causes small models to plan elaborate
search-then-fetch-then-code sequences they cannot reliably execute.
2. Add "always call tools directly" imperative to the system prompt nudge
so models act immediately instead of narrating their intentions.
3. Add plan-without-action re-prompt in the agentic loop: when the model
emits planning text (matching patterns like "let me", "I'll", etc.)
without calling any tool, inject a nudge asking it to call the tool
and continue the loop. Capped at 2 re-prompts per request.
Benchmarked with Qwen3.5-4B-GGUF (N=5 trials per variant):
- Baseline: 40% of requests had any tool call
- Combined fix: 100% of requests had at least one tool call
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Daniel Han <danielhanchen@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
46 lines
1.1 KiB
Python
46 lines
1.1 KiB
Python
# SPDX-License-Identifier: AGPL-3.0-only
|
|
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
|
|
|
"""
|
|
Model and LoRA configuration handling
|
|
"""
|
|
|
|
from .model_config import (
|
|
ModelConfig,
|
|
GgufVariantInfo,
|
|
is_vision_model,
|
|
is_embedding_model,
|
|
detect_audio_type,
|
|
is_audio_input_type,
|
|
VALID_AUDIO_TYPES,
|
|
scan_trained_loras,
|
|
scan_exported_models,
|
|
load_model_defaults,
|
|
get_base_model_from_lora,
|
|
load_model_config,
|
|
list_gguf_variants,
|
|
extract_model_size_b,
|
|
MODEL_NAME_MAPPING,
|
|
UI_STATUS_INDICATORS,
|
|
)
|
|
from .checkpoints import scan_checkpoints
|
|
|
|
__all__ = [
|
|
"ModelConfig",
|
|
"GgufVariantInfo",
|
|
"is_vision_model",
|
|
"is_embedding_model",
|
|
"detect_audio_type",
|
|
"is_audio_input_type",
|
|
"VALID_AUDIO_TYPES",
|
|
"scan_trained_loras",
|
|
"scan_exported_models",
|
|
"load_model_defaults",
|
|
"get_base_model_from_lora",
|
|
"load_model_config",
|
|
"list_gguf_variants",
|
|
"extract_model_size_b",
|
|
"MODEL_NAME_MAPPING",
|
|
"UI_STATUS_INDICATORS",
|
|
"scan_checkpoints",
|
|
]
|