44dcf30b9b* studio: extract param count from model name as fallback When HuggingFace API doesn't return totalParams for a model, extract the param count from the model name (e.g. "Qwen3-0.6B" -> "0.6B", "Llama-3.2-1B-Instruct" -> "1B"). Applied to both the recommended list and HF search results. * studio: read GGUF context_length via fast header parser, set max tokens - Fast GGUF metadata reader (~30-55ms) parses only KV header, skips tensor data and large arrays (tokenizer vocab etc) - Extracts context_length and chat_template from GGUF metadata - Returns context_length in LoadResponse for frontend to use - Frontend sets maxTokens to actual context_length for GGUFs (e.g. 262144 for Qwen3.5-9B, 131072 for Qwen2.5-7B) - Max Tokens slider shows "Max" and is locked for GGUFs - Auto-load path also uses actual context_length from load response - Toast auto-dismiss (5s) and close button for auto-load toast * studio: GGUF TTS audio support (from PR #4318) Add GGUF TTS audio generation via llama-server. When a GGUF model loads, the backend probes its vocabulary to detect audio codecs (SNAC/BiCodec/DAC/CSM/Whisper). If detected, the codec is pre-loaded and the model is reported as audio to the frontend. During chat, TTS models route to the audio generation path which sends a per-codec prompt to llama-server's /completion endpoint, extracts generated tokens/text, and decodes to WAV using AudioCodecManager. Also strips base64 audio data from prior assistant messages to prevent context overflow. Co-authored-by: Manan Shah <mananshah511@gmail.com> * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Remove package-lock.json from tracking * studio: per-model inference defaults, GGUF max tokens fix, reasoning toggle - Add inference_defaults.json with per-model-family sampling parameters for ~50 families (Qwen3.5, Qwen3, Gemma-3, Llama-3, DeepSeek, etc.). Values sourced from unslothai/docs and Ollama params blobs. - Family-based lookup in inference_config.py: extracts model family from identifier, matches against patterns (longest match first), merges with priority: model-specific YAML > family JSON > default.yaml. - Fix GGUF Max Tokens slider locked at "Max": store ggufContextLength separately from maxTokens so the slider is adjustable (step=64). - Fix Ministral YAML: top_p was literal string "default", now 0.95. - Add reasoning toggle for thinking models (Qwen3.5, Qwen3, DeepSeek-R1, DeepSeek-V3.1, etc.): detect enable_thinking support from GGUF chat template metadata, pass --jinja to llama-server, send chat_template_kwargs per-request. Frontend shows "Reasoning is ON/OFF" pill button next to attachment button in composer. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * studio: remove default system prompt injection Backend was injecting "You are a helpful AI assistant." when no system prompt was provided. Neither unslothai/docs nor Ollama specify a default system prompt for most models. Now defaults to empty string, letting the model's own chat template handle system behavior. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * studio: use lightbulb icons and "Think" label for reasoning toggle Lightbulb on when thinking enabled, lightbulb-off when disabled. Label is just "Think" in both states; grayed out styling when off. * studio: fix HTML file upload breaking chat Replace SimpleTextAttachmentAdapter with custom TextAttachmentAdapter (excludes text/html) and HtmlAttachmentAdapter that strips tags via DOMParser, removing scripts/styles and extracting readable text content instead of dumping raw HTML markup into the conversation. * studio: show chat template in Configuration panel Display the model's Jinja2 chat template in a new "Chat Template" section under Settings (now open by default). For GGUFs, reads from GGUF metadata; for safetensors, reads from tokenizer.chat_template. Template is editable with a "Restore default chat template" button that appears when modified. Section only shows when a model with a chat template is loaded. * studio: editable chat template with Apply & Reload Chat template section now functional: - Editing the template shows "Apply & Reload" (reloads model with custom template) and "Revert changes" buttons - For GGUFs: writes template to temp .jinja file, passes --chat-template-file to llama-server on reload - For non-GGUF: passes chat_template_override in load request - Settings section now open by default - selectModel supports forceReload to reload same model * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * studio: fix DeepSeek reasoning detection and auto-load metadata - Set _model_identifier before _read_gguf_metadata so DeepSeek "thinking" template detection works (was always None before) - Populate ggufContextLength, supportsReasoning, reasoningEnabled, defaultChatTemplate in autoLoadSmallestModel GGUF path * studio: add spacing before BETA badge in navbar Add gap-1.5 on the logo Link container to space the BETA label from the wordmark. Co-authored-by: Imagineer99 <Imagineer99@users.noreply.github.com> * studio: vertically center BETA badge with logo --------- Co-authored-by: Manan Shah <mananshah511@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Imagineer99 <Imagineer99@users.noreply.github.com>
55 lines
1.1 KiB
YAML
55 lines
1.1 KiB
YAML
# Model defaults for unsloth/Ministral-3-3B-Instruct-2512
|
|
# Based on Ministral_3_VL_(3B)_Vision.ipynb
|
|
# Also applies to: unsloth/Ministral-3-3B-Instruct-2512
|
|
# added inference parameters from unsloth guides
|
|
|
|
training:
|
|
trust_remote_code: false
|
|
max_seq_length: 2048
|
|
# num_epochs: 4
|
|
num_epochs: 0
|
|
learning_rate: 2e-4
|
|
batch_size: 4
|
|
gradient_accumulation_steps: 2
|
|
warmup_steps: 5
|
|
max_steps: 30
|
|
save_steps: 30
|
|
weight_decay: 0.001
|
|
random_seed: 3407
|
|
packing: false
|
|
train_on_completions: true
|
|
gradient_checkpointing: "unsloth"
|
|
optim: "adamw_8bit"
|
|
lr_scheduler_type: "linear"
|
|
|
|
lora:
|
|
lora_r: 32
|
|
lora_alpha: 32
|
|
lora_dropout: 0.0
|
|
target_modules:
|
|
- "q_proj"
|
|
- "k_proj"
|
|
- "v_proj"
|
|
- "o_proj"
|
|
- "gate_proj"
|
|
- "up_proj"
|
|
- "down_proj"
|
|
use_rslora: false
|
|
use_loftq: false
|
|
finetune_vision_layers: true
|
|
finetune_language_layers: true
|
|
finetune_attention_modules: true
|
|
finetune_mlp_modules: true
|
|
|
|
logging:
|
|
enable_wandb: false
|
|
wandb_project: "llm-finetuning"
|
|
enable_tensorboard: false
|
|
tensorboard_dir: "runs"
|
|
log_frequency: 10
|
|
|
|
inference:
|
|
trust_remote_code: false
|
|
temperature: 0.15
|
|
top_p: 0.95
|
|
|