44dcf30b9b* studio: extract param count from model name as fallback When HuggingFace API doesn't return totalParams for a model, extract the param count from the model name (e.g. "Qwen3-0.6B" -> "0.6B", "Llama-3.2-1B-Instruct" -> "1B"). Applied to both the recommended list and HF search results. * studio: read GGUF context_length via fast header parser, set max tokens - Fast GGUF metadata reader (~30-55ms) parses only KV header, skips tensor data and large arrays (tokenizer vocab etc) - Extracts context_length and chat_template from GGUF metadata - Returns context_length in LoadResponse for frontend to use - Frontend sets maxTokens to actual context_length for GGUFs (e.g. 262144 for Qwen3.5-9B, 131072 for Qwen2.5-7B) - Max Tokens slider shows "Max" and is locked for GGUFs - Auto-load path also uses actual context_length from load response - Toast auto-dismiss (5s) and close button for auto-load toast * studio: GGUF TTS audio support (from PR #4318) Add GGUF TTS audio generation via llama-server. When a GGUF model loads, the backend probes its vocabulary to detect audio codecs (SNAC/BiCodec/DAC/CSM/Whisper). If detected, the codec is pre-loaded and the model is reported as audio to the frontend. During chat, TTS models route to the audio generation path which sends a per-codec prompt to llama-server's /completion endpoint, extracts generated tokens/text, and decodes to WAV using AudioCodecManager. Also strips base64 audio data from prior assistant messages to prevent context overflow. Co-authored-by: Manan Shah <mananshah511@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remove package-lock.json from tracking * studio: per-model inference defaults, GGUF max tokens fix, reasoning toggle - Add inference_defaults.json with per-model-family sampling parameters for ~50 families (Qwen3.5, Qwen3, Gemma-3, Llama-3, DeepSeek, etc.). Values sourced from unslothai/docs and Ollama params blobs. - Family-based lookup in inference_config.py: extracts model family from identifier, matches against patterns (longest match first), merges with priority: model-specific YAML > family JSON > default.yaml. - Fix GGUF Max Tokens slider locked at "Max": store ggufContextLength separately from maxTokens so the slider is adjustable (step=64). - Fix Ministral YAML: top_p was literal string "default", now 0.95. - Add reasoning toggle for thinking models (Qwen3.5, Qwen3, DeepSeek-R1, DeepSeek-V3.1, etc.): detect enable_thinking support from GGUF chat template metadata, pass --jinja to llama-server, send chat_template_kwargs per-request. Frontend shows "Reasoning is ON/OFF" pill button next to attachment button in composer. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * studio: remove default system prompt injection Backend was injecting "You are a helpful AI assistant." when no system prompt was provided. Neither unslothai/docs nor Ollama specify a default system prompt for most models. Now defaults to empty string, letting the model's own chat template handle system behavior. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * studio: use lightbulb icons and "Think" label for reasoning toggle Lightbulb on when thinking enabled, lightbulb-off when disabled. Label is just "Think" in both states; grayed out styling when off. * studio: fix HTML file upload breaking chat Replace SimpleTextAttachmentAdapter with custom TextAttachmentAdapter (excludes text/html) and HtmlAttachmentAdapter that strips tags via DOMParser, removing scripts/styles and extracting readable text content instead of dumping raw HTML markup into the conversation. * studio: show chat template in Configuration panel Display the model's Jinja2 chat template in a new "Chat Template" section under Settings (now open by default). For GGUFs, reads from GGUF metadata; for safetensors, reads from tokenizer.chat_template. Template is editable with a "Restore default chat template" button that appears when modified. Section only shows when a model with a chat template is loaded. * studio: editable chat template with Apply & Reload Chat template section now functional: - Editing the template shows "Apply & Reload" (reloads model with custom template) and "Revert changes" buttons - For GGUFs: writes template to temp .jinja file, passes --chat-template-file to llama-server on reload - For non-GGUF: passes chat_template_override in load request - Settings section now open by default - selectModel supports forceReload to reload same model * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * studio: fix DeepSeek reasoning detection and auto-load metadata - Set _model_identifier before _read_gguf_metadata so DeepSeek "thinking" template detection works (was always None before) - Populate ggufContextLength, supportsReasoning, reasoningEnabled, defaultChatTemplate in autoLoadSmallestModel GGUF path * studio: add spacing before BETA badge in navbar Add gap-1.5 on the logo Link container to space the BETA label from the wordmark. Co-authored-by: Imagineer99 <Imagineer99@users.noreply.github.com> * studio: vertically center BETA badge with logo --------- Co-authored-by: Manan Shah <mananshah511@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Imagineer99 <Imagineer99@users.noreply.github.com>
381 lines
8 KiB
JSON
381 lines
8 KiB
JSON
{
|
|
"_comment": "Per-model-family inference parameter defaults. Sources: (1) Ollama params blobs, (2) Existing Unsloth Studio YAML configs. Patterns ordered longest-match-first.",
|
|
"families": {
|
|
"qwen3.5": {
|
|
"temperature": 1.0,
|
|
"top_p": 0.95,
|
|
"top_k": 20,
|
|
"min_p": 0.0,
|
|
"repetition_penalty": 1.0
|
|
},
|
|
"qwen3-coder": {
|
|
"temperature": 0.7,
|
|
"top_p": 0.8,
|
|
"top_k": 20,
|
|
"min_p": 0.0,
|
|
"repetition_penalty": 1.0
|
|
},
|
|
"qwen3-next": {
|
|
"temperature": 0.7,
|
|
"top_p": 0.8,
|
|
"top_k": 20,
|
|
"min_p": 0.0,
|
|
"repetition_penalty": 1.0
|
|
},
|
|
"qwen3-vl": {
|
|
"temperature": 0.7,
|
|
"top_p": 0.8,
|
|
"top_k": 20,
|
|
"min_p": 0.0,
|
|
"repetition_penalty": 1.0
|
|
},
|
|
"qwen3": {
|
|
"temperature": 0.6,
|
|
"top_p": 0.95,
|
|
"top_k": 20,
|
|
"min_p": 0.0,
|
|
"repetition_penalty": 1.0
|
|
},
|
|
"qwen2.5-coder": {
|
|
"temperature": 1.5,
|
|
"top_p": 0.95,
|
|
"top_k": -1,
|
|
"min_p": 0.1,
|
|
"repetition_penalty": 1.0
|
|
},
|
|
"qwen2.5-vl": {
|
|
"temperature": 1.5,
|
|
"top_p": 0.95,
|
|
"top_k": -1,
|
|
"min_p": 0.1,
|
|
"repetition_penalty": 1.0
|
|
},
|
|
"qwen2.5-omni": {
|
|
"temperature": 0.7,
|
|
"top_p": 0.8,
|
|
"top_k": 20,
|
|
"min_p": 0.0,
|
|
"repetition_penalty": 1.0
|
|
},
|
|
"qwen2.5-math": {
|
|
"temperature": 0.7,
|
|
"top_p": 0.8,
|
|
"top_k": 20,
|
|
"min_p": 0.0,
|
|
"repetition_penalty": 1.0
|
|
},
|
|
"qwen2.5": {
|
|
"temperature": 0.7,
|
|
"top_p": 0.8,
|
|
"top_k": 20,
|
|
"min_p": 0.0,
|
|
"repetition_penalty": 1.0
|
|
},
|
|
"qwen2-vl": {
|
|
"temperature": 1.5,
|
|
"top_p": 0.95,
|
|
"top_k": -1,
|
|
"min_p": 0.1,
|
|
"repetition_penalty": 1.0
|
|
},
|
|
"qwen2": {
|
|
"temperature": 0.7,
|
|
"top_p": 0.8,
|
|
"top_k": 20,
|
|
"min_p": 0.0,
|
|
"repetition_penalty": 1.0
|
|
},
|
|
"qwq": {
|
|
"temperature": 0.6,
|
|
"top_p": 0.95,
|
|
"top_k": 40,
|
|
"min_p": 0.0,
|
|
"repetition_penalty": 1.0
|
|
},
|
|
"gemma-3n": {
|
|
"temperature": 1.0,
|
|
"top_p": 0.95,
|
|
"top_k": 64,
|
|
"min_p": 0.0,
|
|
"repetition_penalty": 1.0
|
|
},
|
|
"gemma-3": {
|
|
"temperature": 1.0,
|
|
"top_p": 0.95,
|
|
"top_k": 64,
|
|
"min_p": 0.0,
|
|
"repetition_penalty": 1.0
|
|
},
|
|
"medgemma": {
|
|
"temperature": 1.0,
|
|
"top_p": 0.95,
|
|
"top_k": 64,
|
|
"min_p": 0.0,
|
|
"repetition_penalty": 1.0
|
|
},
|
|
"gemma-2": {
|
|
"temperature": 1.0,
|
|
"top_p": 0.95,
|
|
"top_k": 64,
|
|
"min_p": 0.0,
|
|
"repetition_penalty": 1.0
|
|
},
|
|
"llama-4": {
|
|
"temperature": 1.0,
|
|
"top_p": 0.9,
|
|
"top_k": -1,
|
|
"min_p": 0.01,
|
|
"repetition_penalty": 1.0
|
|
},
|
|
"llama-3.3": {
|
|
"temperature": 1.5,
|
|
"top_p": 0.95,
|
|
"top_k": -1,
|
|
"min_p": 0.1,
|
|
"repetition_penalty": 1.0
|
|
},
|
|
"llama-3.2": {
|
|
"temperature": 1.5,
|
|
"top_p": 0.95,
|
|
"top_k": -1,
|
|
"min_p": 0.1,
|
|
"repetition_penalty": 1.0
|
|
},
|
|
"llama-3.1": {
|
|
"temperature": 1.5,
|
|
"top_p": 0.95,
|
|
"top_k": -1,
|
|
"min_p": 0.1,
|
|
"repetition_penalty": 1.0
|
|
},
|
|
"llama-3": {
|
|
"temperature": 1.5,
|
|
"top_p": 0.95,
|
|
"top_k": -1,
|
|
"min_p": 0.1,
|
|
"repetition_penalty": 1.0
|
|
},
|
|
"phi-4": {
|
|
"temperature": 0.8,
|
|
"top_p": 0.95,
|
|
"top_k": -1,
|
|
"min_p": 0.0,
|
|
"repetition_penalty": 1.0
|
|
},
|
|
"phi-3": {
|
|
"temperature": 0.7,
|
|
"top_p": 0.9,
|
|
"top_k": -1,
|
|
"min_p": 0.01,
|
|
"repetition_penalty": 1.0
|
|
},
|
|
"mistral-nemo": {
|
|
"temperature": 0.7,
|
|
"top_p": 0.95,
|
|
"top_k": -1,
|
|
"min_p": 0.01,
|
|
"repetition_penalty": 1.0
|
|
},
|
|
"mistral-small": {
|
|
"temperature": 0.15,
|
|
"top_p": 0.95,
|
|
"top_k": -1,
|
|
"min_p": 0.01,
|
|
"repetition_penalty": 1.0
|
|
},
|
|
"mistral-large": {
|
|
"temperature": 0.7,
|
|
"top_p": 0.95,
|
|
"top_k": -1,
|
|
"min_p": 0.01,
|
|
"repetition_penalty": 1.0
|
|
},
|
|
"magistral": {
|
|
"temperature": 0.7,
|
|
"top_p": 0.95,
|
|
"top_k": -1,
|
|
"min_p": 0.01,
|
|
"repetition_penalty": 1.0
|
|
},
|
|
"ministral": {
|
|
"temperature": 0.15,
|
|
"top_p": 0.95,
|
|
"top_k": -1,
|
|
"min_p": 0.01,
|
|
"repetition_penalty": 1.0
|
|
},
|
|
"devstral": {
|
|
"temperature": 0.7,
|
|
"top_p": 0.95,
|
|
"top_k": -1,
|
|
"min_p": 0.01,
|
|
"repetition_penalty": 1.0
|
|
},
|
|
"pixtral": {
|
|
"temperature": 1.5,
|
|
"top_p": 0.95,
|
|
"top_k": -1,
|
|
"min_p": 0.1,
|
|
"repetition_penalty": 1.0
|
|
},
|
|
"deepseek-r1": {
|
|
"temperature": 0.6,
|
|
"top_p": 0.95,
|
|
"top_k": -1,
|
|
"min_p": 0.01,
|
|
"repetition_penalty": 1.0
|
|
},
|
|
"deepseek-v3": {
|
|
"temperature": 0.6,
|
|
"top_p": 0.95,
|
|
"top_k": -1,
|
|
"min_p": 0.01,
|
|
"repetition_penalty": 1.0
|
|
},
|
|
"deepseek-ocr": {
|
|
"temperature": 0.0,
|
|
"top_p": 0.95,
|
|
"top_k": -1,
|
|
"min_p": 0.01,
|
|
"repetition_penalty": 1.0
|
|
},
|
|
"glm-5": {
|
|
"temperature": 1.0,
|
|
"top_p": 0.95,
|
|
"top_k": -1,
|
|
"min_p": 0.01,
|
|
"repetition_penalty": 1.0
|
|
},
|
|
"glm-4": {
|
|
"temperature": 1.0,
|
|
"top_p": 0.95,
|
|
"top_k": -1,
|
|
"min_p": 0.01,
|
|
"repetition_penalty": 1.0
|
|
},
|
|
"nemotron": {
|
|
"temperature": 1.0,
|
|
"top_p": 1.0,
|
|
"top_k": -1,
|
|
"min_p": 0.01,
|
|
"repetition_penalty": 1.0
|
|
},
|
|
"minimax-m2.5": {
|
|
"temperature": 1.0,
|
|
"top_p": 0.95,
|
|
"top_k": 40,
|
|
"min_p": 0.01,
|
|
"repetition_penalty": 1.0
|
|
},
|
|
"minimax": {
|
|
"temperature": 1.0,
|
|
"top_p": 0.95,
|
|
"top_k": 40,
|
|
"min_p": 0.01,
|
|
"repetition_penalty": 1.0
|
|
},
|
|
"gpt-oss": {
|
|
"temperature": 1.0,
|
|
"top_p": 1.0,
|
|
"top_k": 0,
|
|
"min_p": 0.01,
|
|
"repetition_penalty": 1.0
|
|
},
|
|
"granite-4": {
|
|
"temperature": 0.0,
|
|
"top_p": 1.0,
|
|
"top_k": 0,
|
|
"min_p": 0.01,
|
|
"repetition_penalty": 1.0
|
|
},
|
|
"kimi-k2": {
|
|
"temperature": 0.6,
|
|
"top_p": 0.95,
|
|
"top_k": -1,
|
|
"min_p": 0.01,
|
|
"repetition_penalty": 1.0
|
|
},
|
|
"kimi": {
|
|
"temperature": 0.6,
|
|
"top_p": 0.95,
|
|
"top_k": -1,
|
|
"min_p": 0.01,
|
|
"repetition_penalty": 1.0
|
|
},
|
|
"lfm2": {
|
|
"temperature": 0.1,
|
|
"top_p": 0.1,
|
|
"top_k": 50,
|
|
"min_p": 0.15,
|
|
"repetition_penalty": 1.05
|
|
},
|
|
"smollm": {
|
|
"temperature": 0.7,
|
|
"top_p": 0.95,
|
|
"top_k": -1,
|
|
"min_p": 0.01,
|
|
"repetition_penalty": 1.0
|
|
},
|
|
"olmo": {
|
|
"temperature": 0.7,
|
|
"top_p": 0.95,
|
|
"top_k": -1,
|
|
"min_p": 0.01,
|
|
"repetition_penalty": 1.0
|
|
},
|
|
"falcon": {
|
|
"temperature": 0.7,
|
|
"top_p": 0.95,
|
|
"top_k": -1,
|
|
"min_p": 0.01,
|
|
"repetition_penalty": 1.0
|
|
},
|
|
"ernie": {
|
|
"temperature": 0.7,
|
|
"top_p": 0.95,
|
|
"top_k": -1,
|
|
"min_p": 0.01,
|
|
"repetition_penalty": 1.0
|
|
},
|
|
"seed": {
|
|
"temperature": 0.7,
|
|
"top_p": 0.95,
|
|
"top_k": -1,
|
|
"min_p": 0.01,
|
|
"repetition_penalty": 1.0
|
|
},
|
|
"grok": {
|
|
"temperature": 1.0,
|
|
"top_p": 0.95,
|
|
"top_k": -1,
|
|
"min_p": 0.01,
|
|
"repetition_penalty": 1.0
|
|
},
|
|
"mimo": {
|
|
"temperature": 0.7,
|
|
"top_p": 0.95,
|
|
"top_k": -1,
|
|
"min_p": 0.01,
|
|
"repetition_penalty": 1.0
|
|
}
|
|
},
|
|
"patterns": [
|
|
"qwen3.5",
|
|
"qwen3-coder", "qwen3-next", "qwen3-vl", "qwen3",
|
|
"qwen2.5-coder", "qwen2.5-vl", "qwen2.5-omni", "qwen2.5-math", "qwen2.5",
|
|
"qwen2-vl", "qwen2",
|
|
"qwq",
|
|
"gemma-3n", "gemma-3", "medgemma", "gemma-2",
|
|
"llama-4", "llama-3.3", "llama-3.2", "llama-3.1", "llama-3",
|
|
"phi-4", "phi-3",
|
|
"mistral-nemo", "mistral-small", "mistral-large", "magistral", "ministral",
|
|
"devstral", "pixtral",
|
|
"deepseek-r1", "deepseek-v3", "deepseek-ocr",
|
|
"glm-5", "glm-4",
|
|
"nemotron",
|
|
"minimax-m2.5", "minimax",
|
|
"gpt-oss", "granite-4",
|
|
"kimi-k2", "kimi",
|
|
"lfm2", "smollm", "olmo", "falcon", "ernie", "seed", "grok", "mimo"
|
|
]
|
|
}
|