Merge branch 'main' into pip
This commit is contained in:
Daniel Han
commit
dddc9eac92
16 changed files with 563 additions and 157 deletions
3
.github/ISSUE_TEMPLATE/bug---issue.md
vendored
3
.github/ISSUE_TEMPLATE/bug---issue.md
vendored
|
|
@ -6,7 +6,7 @@ labels: bug
|
|||
assignees: ''
|
||||
|
||||
---
|
||||
|
||||
Note: Please do not remove the questions. Answer beside them.
|
||||
1. Did you update? `pip install --upgrade unsloth unsloth_zoo`
|
||||
2. `Colab` or `Kaggle` or local / cloud
|
||||
3. Number GPUs used, use `nvidia-smi`
|
||||
|
|
@ -16,6 +16,7 @@ assignees: ''
|
|||
|
||||
```python
|
||||
Put Minimal code to reproduce error here ###Remove Hugging Face token###
|
||||
###Please make sure to check formatting properly, edit if needed.###
|
||||
```
|
||||
|
||||
🦥 You can also ask via our Reddit page: https://reddit.com/r/unsloth/
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
repos:
|
||||
- repo: https://github.com/astral-sh/ruff-pre-commit
|
||||
rev: v0.15.6
|
||||
rev: v0.15.7
|
||||
hooks:
|
||||
- id: ruff
|
||||
args:
|
||||
|
|
|
|||
17
README.md
17
README.md
|
|
@ -14,7 +14,7 @@ Run and train AI models with a unified local interface.
|
|||
<a href="#-quickstart">Quickstart</a> •
|
||||
<a href="#-free-notebooks">Notebooks</a> •
|
||||
<a href="https://unsloth.ai/docs">Documentation</a> •
|
||||
<a href="https://discord.com/invite/unsloth">Discord</a>
|
||||
<a href="https://www.reddit.com/r/unsloth/">Reddit</a>
|
||||
</p>
|
||||
<a href="https://unsloth.ai/docs/new/studio">
|
||||
<img alt="unsloth studio ui homepage" src="https://raw.githubusercontent.com/unslothai/unsloth/main/studio/frontend/public/studio%20github%20landscape%20colab%20display.png" style="max-width: 100%; margin-bottom: 0;"></a>
|
||||
|
|
@ -32,12 +32,12 @@ Unsloth provides several key features for both inference and training:
|
|||
* We work directly with teams behind [gpt-oss](https://docs.unsloth.ai/new/gpt-oss-how-to-run-and-fine-tune#unsloth-fixes-for-gpt-oss), [Qwen3](https://www.reddit.com/r/LocalLLaMA/comments/1kaodxu/qwen3_unsloth_dynamic_ggufs_128k_context_bug_fixes/), [Llama 4](https://github.com/ggml-org/llama.cpp/pull/12889), [Mistral](models/tutorials/devstral-how-to-run-and-fine-tune.md), [Gemma 1-3](https://news.ycombinator.com/item?id=39671146), and [Phi-4](https://unsloth.ai/blog/phi4), where we’ve fixed bugs that improve model accuracy.
|
||||
* Upload images, audio, PDFs, code, DOCX and more file types to chat with.
|
||||
### Training
|
||||
* Train **500+ models** up to **2x faster** with up to **70% less VRAM**, with no accuracy loss.
|
||||
* Train and RL **500+ models** up to **2x faster** with up to **70% less VRAM**, with no accuracy loss.
|
||||
* Custom Triton and mathematical **kernels**. See some collabs we did with [PyTorch](https://unsloth.ai/docs/get-started/reinforcement-learning-rl-guide/fp8-reinforcement-learning) and [Hugging Face](https://unsloth.ai/docs/new/faster-moe).
|
||||
* **Data Recipes**: [Auto-create datasets](https://unsloth.ai/docs/new/studio/data-recipe) from **PDF, CSV, DOCX** etc. Edit data in a visual-node workflow.
|
||||
* Supports full fine-tuning, pretraining, 4-bit, 16-bit and, FP8 training.
|
||||
* **[Reinforcement Learning](https://unsloth.ai/docs/get-started/reinforcement-learning-rl-guide)** (RL): The most efficient [RL](https://unsloth.ai/docs/get-started/reinforcement-learning-rl-guide) library, using **80% less VRAM** for GRPO, [FP8](https://unsloth.ai/docs/get-started/reinforcement-learning-rl-guide/fp8-reinforcement-learning) etc.
|
||||
* Supports full fine-tuning, RL, pretraining, 4-bit, 16-bit and, FP8 training.
|
||||
* **Observability**: Monitor training live, track loss and GPU usage and customize graphs.
|
||||
* **Reinforcement Learning**: The most efficient [RL](https://unsloth.ai/docs/get-started/reinforcement-learning-rl-guide) library, using **80% less VRAM** for GRPO, [FP8](https://unsloth.ai/docs/get-started/reinforcement-learning-rl-guide/fp8-reinforcement-learning) etc.
|
||||
* [Multi-GPU](https://unsloth.ai/docs/basics/multi-gpu-training-with-unsloth) training is supported, with major improvements coming soon.
|
||||
|
||||
## ⚡ Quickstart
|
||||
|
|
@ -49,7 +49,7 @@ Unsloth Studio (Beta) works on **Windows, Linux, WSL** and **macOS**.
|
|||
* **CPU:** Supported for Chat and Data Recipes currently
|
||||
* **NVIDIA:** Training works on RTX 30/40/50, Blackwell, DGX Spark, Station and more
|
||||
* **macOS:** Currently supports chat and Data Recipes. **MLX training** is coming very soon
|
||||
* **AMD:** Chat works. Train with [Unsloth Core](#unsloth-core-code-based). Studio support is coming soon.
|
||||
* **AMD:** Chat + Data works. Train with [Unsloth Core](#unsloth-core-code-based). Studio support is out soon.
|
||||
* **Coming soon:** Training support for Apple MLX, AMD, and Intel.
|
||||
* **Multi-GPU:** Available now, with a major upgrade on the way
|
||||
|
||||
|
|
@ -172,8 +172,9 @@ Train for free with our notebooks. Read our [guide](https://unsloth.ai/docs/get-
|
|||
|-----------|---------|--------|----------|
|
||||
| **Qwen3.5 (4B)** | [▶️ Start for free](https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Qwen3_5_(4B)_Vision.ipynb) | 1.5x faster | 60% less |
|
||||
| **gpt-oss (20B)** | [▶️ Start for free](https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/gpt-oss-(20B)-Fine-tuning.ipynb) | 2x faster | 70% less |
|
||||
| **Qwen3.5 GSPO** | [▶️ Start for free](https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Qwen3_5_(4B)_Vision_GRPO.ipynb) | 2x faster | 70% less |
|
||||
| **gpt-oss (20B): GRPO** | [▶️ Start for free](https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/gpt-oss-(20B)-GRPO.ipynb) | 2x faster | 80% less |
|
||||
| **Qwen3: Advanced GRPO** | [▶️ Start for free](https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Qwen3_(4B)-GRPO.ipynb) | 2x faster | 50% less |
|
||||
| **Qwen3: Advanced GRPO** | [▶️ Start for free](https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Qwen3_(4B)-GRPO.ipynb) | 2x faster | 70% less |
|
||||
| **Gemma 3 (4B) Vision** | [▶️ Start for free](https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Gemma3_(4B)-Vision.ipynb) | 1.7x faster | 60% less |
|
||||
| **embeddinggemma (300M)** | [▶️ Start for free](https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/EmbeddingGemma_(300M).ipynb) | 2x faster | 20% less |
|
||||
| **Mistral Ministral 3 (3B)** | [▶️ Start for free](https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Ministral_3_VL_(3B)_Vision.ipynb) | 1.5x faster | 60% less |
|
||||
|
|
@ -196,13 +197,13 @@ Train for free with our notebooks. Read our [guide](https://unsloth.ai/docs/get-
|
|||
- **FP8 & Vision RL**: You can now do FP8 & VLM GRPO on consumer GPUs. [FP8 Blog](https://unsloth.ai/docs/get-started/reinforcement-learning-rl-guide/fp8-reinforcement-learning) • [Vision RL](https://unsloth.ai/docs/get-started/reinforcement-learning-rl-guide/vision-reinforcement-learning-vlm-rl)
|
||||
- **gpt-oss** by OpenAI: Read our [RL blog](https://unsloth.ai/docs/models/gpt-oss-how-to-run-and-fine-tune/gpt-oss-reinforcement-learning), [Flex Attention](https://unsloth.ai/docs/models/gpt-oss-how-to-run-and-fine-tune/long-context-gpt-oss-training) blog and [Guide](https://unsloth.ai/docs/models/gpt-oss-how-to-run-and-fine-tune).
|
||||
|
||||
## 🔗 Links and Resources
|
||||
## 💚 Community and Links
|
||||
| Type | Links |
|
||||
| ----------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------ |
|
||||
| <img width="16" src="https://cdn.prod.website-files.com/6257adef93867e50d84d30e2/66e3d80db9971f10a9757c99_Symbol.svg" /> **Discord** | [Join Discord server](https://discord.com/invite/unsloth) |
|
||||
| <img width="15" src="https://redditinc.com/hs-fs/hubfs/Reddit%20Inc/Brand/Reddit_Logo.png" /> **r/unsloth Reddit** | [Join Reddit community](https://reddit.com/r/unsloth) |
|
||||
| 📚 **Documentation & Wiki** | [Read Our Docs](https://unsloth.ai/docs) |
|
||||
| <img width="13" src="https://upload.wikimedia.org/wikipedia/commons/0/09/X_(formerly_Twitter)_logo_late_2025.svg" /> **Twitter (aka X)** | [Follow us on X](https://twitter.com/unslothai) |
|
||||
| 💾 **Installation** | [Pip & Docker Install](https://unsloth.ai/docs/get-started/install) |
|
||||
| 🔮 **Our Models** | [Unsloth Catalog](https://unsloth.ai/docs/get-started/unsloth-model-catalog) |
|
||||
| ✍️ **Blog** | [Read our Blogs](https://unsloth.ai/blog) |
|
||||
|
||||
|
|
|
|||
|
|
@ -927,6 +927,12 @@ class InferenceBackend:
|
|||
logger.warning(f"Could not apply get_chat_template: {e}")
|
||||
|
||||
# Step 2: Format with tokenizer.apply_chat_template()
|
||||
if system_prompt:
|
||||
template_messages = [
|
||||
{"role": "system", "content": system_prompt}
|
||||
] + messages
|
||||
else:
|
||||
template_messages = messages
|
||||
try:
|
||||
if not (hasattr(tokenizer, "chat_template") and tokenizer.chat_template):
|
||||
raise ValueError(
|
||||
|
|
@ -937,7 +943,7 @@ class InferenceBackend:
|
|||
f"one via tokenizer.chat_template before inference."
|
||||
)
|
||||
formatted_prompt = tokenizer.apply_chat_template(
|
||||
messages, tokenize = False, add_generation_prompt = True
|
||||
template_messages, tokenize = False, add_generation_prompt = True
|
||||
)
|
||||
logger.debug(f"Formatted prompt: {formatted_prompt[:200]}...")
|
||||
except Exception as e:
|
||||
|
|
@ -992,19 +998,40 @@ class InferenceBackend:
|
|||
|
||||
# Prepare vision messages
|
||||
if image:
|
||||
vision_messages = [
|
||||
{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{"type": "image"},
|
||||
{"type": "text", "text": user_message},
|
||||
],
|
||||
}
|
||||
]
|
||||
user_msg = {
|
||||
"role": "user",
|
||||
"content": [
|
||||
{"type": "image"},
|
||||
{"type": "text", "text": user_message},
|
||||
],
|
||||
}
|
||||
if system_prompt:
|
||||
vision_messages = [
|
||||
{
|
||||
"role": "system",
|
||||
"content": [{"type": "text", "text": system_prompt}],
|
||||
},
|
||||
user_msg,
|
||||
]
|
||||
else:
|
||||
vision_messages = [user_msg]
|
||||
|
||||
input_text = processor.apply_chat_template(
|
||||
vision_messages, add_generation_prompt = True, tokenize = False
|
||||
)
|
||||
try:
|
||||
input_text = processor.apply_chat_template(
|
||||
vision_messages, add_generation_prompt = True, tokenize = False
|
||||
)
|
||||
except Exception as e:
|
||||
if system_prompt:
|
||||
logger.warning(
|
||||
f"Vision processor for '{self.active_model_name}' may not support "
|
||||
f"system messages; retrying without. Original error: {e}"
|
||||
)
|
||||
vision_messages = [user_msg]
|
||||
input_text = processor.apply_chat_template(
|
||||
vision_messages, add_generation_prompt = True, tokenize = False
|
||||
)
|
||||
else:
|
||||
raise
|
||||
inputs = processor(
|
||||
image,
|
||||
input_text,
|
||||
|
|
|
|||
|
|
@ -73,18 +73,79 @@ def _resolve_external_ip() -> str:
|
|||
return "0.0.0.0"
|
||||
|
||||
|
||||
def _get_pid_on_port(port: int) -> "tuple[int, str] | None":
|
||||
"""Return (pid, process_name) of the process listening on *port*, or None.
|
||||
|
||||
Uses psutil when available. Falls back gracefully to None so callers
|
||||
can still report the port conflict without process details.
|
||||
|
||||
Works on Windows, macOS, and Linux wherever psutil is installed.
|
||||
"""
|
||||
try:
|
||||
import psutil
|
||||
except ImportError:
|
||||
return None
|
||||
try:
|
||||
for conn in psutil.net_connections(kind = "tcp"):
|
||||
if conn.status == "LISTEN" and conn.laddr.port == port:
|
||||
if conn.pid is None:
|
||||
return None
|
||||
try:
|
||||
proc = psutil.Process(conn.pid)
|
||||
return (conn.pid, proc.name())
|
||||
except (psutil.NoSuchProcess, psutil.AccessDenied):
|
||||
return (conn.pid, "<unknown>")
|
||||
except (psutil.AccessDenied, OSError) as e:
|
||||
# psutil.net_connections() needs elevated privileges on some platforms
|
||||
logger.debug("Failed to scan network connections for port %s: %s", port, e)
|
||||
return None
|
||||
|
||||
|
||||
def _is_port_free(host: str, port: int) -> bool:
|
||||
"""Check if a port is available for binding."""
|
||||
"""Check if a port is available for binding.
|
||||
|
||||
When *host* is ``0.0.0.0`` (wildcard), we also check whether anything
|
||||
is already listening on ``127.0.0.1`` (and ``::1`` when IPv6 is
|
||||
available). An SSH tunnel or similar process may hold the loopback
|
||||
address while our wildcard bind still succeeds, making Unsloth Studio
|
||||
unreachable via ``localhost``.
|
||||
|
||||
Works on Windows, macOS, and Linux.
|
||||
"""
|
||||
import socket
|
||||
|
||||
# 1. Can we bind to the requested address?
|
||||
# Use getaddrinfo so both IPv4 ("0.0.0.0") and IPv6 ("::") hosts
|
||||
# resolve to the correct address family automatically.
|
||||
try:
|
||||
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
|
||||
addr_info = socket.getaddrinfo(host, port, socket.AF_UNSPEC, socket.SOCK_STREAM)
|
||||
family, socktype, proto, _, sockaddr = addr_info[0]
|
||||
with socket.socket(family, socktype, proto) as s:
|
||||
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
|
||||
s.bind((host, port))
|
||||
return True
|
||||
s.bind(sockaddr)
|
||||
except OSError:
|
||||
return False
|
||||
|
||||
# 2. When binding to all interfaces, verify that localhost is not
|
||||
# already claimed by another process (e.g. an SSH -L tunnel).
|
||||
# We attempt a TCP connect -- if it succeeds something is listening.
|
||||
if host in ("0.0.0.0", "::"):
|
||||
for loopback, family in [
|
||||
("127.0.0.1", socket.AF_INET),
|
||||
("::1", socket.AF_INET6),
|
||||
]:
|
||||
try:
|
||||
with socket.socket(family, socket.SOCK_STREAM) as s:
|
||||
s.settimeout(1)
|
||||
if s.connect_ex((loopback, port)) == 0:
|
||||
# Connection succeeded -- port is taken on loopback
|
||||
return False
|
||||
except OSError:
|
||||
# IPv6 disabled or other OS-level restriction -- skip
|
||||
continue
|
||||
|
||||
return True
|
||||
|
||||
|
||||
def _find_free_port(host: str, start: int, max_attempts: int = 20) -> int:
|
||||
"""Find a free port starting from `start`, trying up to max_attempts ports."""
|
||||
|
|
@ -149,11 +210,11 @@ def _graceful_shutdown(server = None):
|
|||
logger.info("All subprocesses cleaned up")
|
||||
|
||||
|
||||
# The uvicorn server instance — set by run_server(), used by callers
|
||||
# The uvicorn server instance -- set by run_server(), used by callers
|
||||
# that need to tell the server to exit (e.g. signal handlers).
|
||||
_server = None
|
||||
|
||||
# Shutdown event — used to wake the main loop on signal
|
||||
# Shutdown event -- used to wake the main loop on signal
|
||||
_shutdown_event = None
|
||||
|
||||
|
||||
|
|
@ -205,9 +266,22 @@ def run_server(
|
|||
# Auto-find free port if requested port is in use
|
||||
if not _is_port_free(host, port):
|
||||
original_port = port
|
||||
port = _find_free_port(host, port)
|
||||
blocker = _get_pid_on_port(port)
|
||||
port = _find_free_port(host, port + 1)
|
||||
if not silent:
|
||||
print(f"Port {original_port} is in use, using port {port} instead")
|
||||
print("")
|
||||
print("=" * 50)
|
||||
if blocker:
|
||||
pid, name = blocker
|
||||
print(
|
||||
f"Port {original_port} is already in use by " f"{name} (PID {pid})."
|
||||
)
|
||||
else:
|
||||
print(f"Port {original_port} is already in use.")
|
||||
print(f"Unsloth Studio will use port {port} instead.")
|
||||
print(f"Open http://localhost:{port} in your browser.")
|
||||
print("=" * 50)
|
||||
print("")
|
||||
|
||||
# Setup frontend if path provided
|
||||
if frontend_path:
|
||||
|
|
@ -297,7 +371,7 @@ if __name__ == "__main__":
|
|||
sys.stderr.flush()
|
||||
sys.exit(1)
|
||||
|
||||
# ── Signal handler — ensures subprocess cleanup on Ctrl+C ────
|
||||
# Signal handler -- ensures subprocess cleanup on Ctrl+C
|
||||
def _signal_handler(signum, frame):
|
||||
_graceful_shutdown(_server)
|
||||
_shutdown_event.set()
|
||||
|
|
|
|||
|
|
@ -151,7 +151,7 @@ function ReasoningTrigger({
|
|||
<CollapsibleTrigger
|
||||
data-slot="reasoning-trigger"
|
||||
className={cn(
|
||||
"aui-reasoning-trigger group/trigger flex max-w-[75%] items-center gap-2 py-1 text-muted-foreground text-sm transition-colors hover:text-foreground",
|
||||
"aui-reasoning-trigger group/trigger flex min-w-0 flex-1 items-center gap-2 py-1 text-muted-foreground text-sm transition-colors hover:text-foreground",
|
||||
className,
|
||||
)}
|
||||
{...props}
|
||||
|
|
@ -219,22 +219,34 @@ function ReasoningText({
|
|||
...props
|
||||
}: ComponentProps<"div"> & { streaming?: boolean }) {
|
||||
const scrollRef = useRef<HTMLDivElement>(null);
|
||||
const shouldAutoScrollRef = useRef(true);
|
||||
|
||||
useEffect(() => {
|
||||
if (!(streaming && scrollRef.current)) {
|
||||
return;
|
||||
}
|
||||
const el = scrollRef.current;
|
||||
const updateAutoScroll = () => {
|
||||
const distanceFromBottom = el.scrollHeight - el.scrollTop - el.clientHeight;
|
||||
shouldAutoScrollRef.current = distanceFromBottom <= 24;
|
||||
};
|
||||
const observer = new MutationObserver(() => {
|
||||
el.scrollTop = el.scrollHeight;
|
||||
if (shouldAutoScrollRef.current) {
|
||||
el.scrollTop = el.scrollHeight;
|
||||
}
|
||||
});
|
||||
el.addEventListener("scroll", updateAutoScroll);
|
||||
observer.observe(el, {
|
||||
childList: true,
|
||||
subtree: true,
|
||||
characterData: true,
|
||||
});
|
||||
shouldAutoScrollRef.current = true;
|
||||
el.scrollTop = el.scrollHeight;
|
||||
return () => observer.disconnect();
|
||||
return () => {
|
||||
observer.disconnect();
|
||||
el.removeEventListener("scroll", updateAutoScroll);
|
||||
};
|
||||
}, [streaming]);
|
||||
|
||||
return (
|
||||
|
|
@ -330,6 +342,7 @@ const ReasoningGroupImpl: ReasoningGroupComponent = ({
|
|||
});
|
||||
|
||||
const [manualOpen, setManualOpen] = useState(false);
|
||||
const [dismissedWhileStreaming, setDismissedWhileStreaming] = useState(false);
|
||||
const [duration, setDuration] = useState<number>(0);
|
||||
const startTimeRef = useRef<number | null>(null);
|
||||
|
||||
|
|
@ -345,17 +358,23 @@ const ReasoningGroupImpl: ReasoningGroupComponent = ({
|
|||
}
|
||||
}, [isReasoningStreaming]);
|
||||
|
||||
const isOpen = isReasoningStreaming || manualOpen;
|
||||
// Reset dismissed flag when a new stream starts
|
||||
useEffect(() => {
|
||||
if (isReasoningStreaming) {
|
||||
setDismissedWhileStreaming(false);
|
||||
}
|
||||
}, [isReasoningStreaming]);
|
||||
|
||||
const variant = isReasoningStreaming
|
||||
? "outline"
|
||||
: manualOpen
|
||||
? "outline"
|
||||
: "ghost";
|
||||
// Derived: open during streaming (unless dismissed), or if user manually opened after
|
||||
const isOpen = (isReasoningStreaming && !dismissedWhileStreaming) || manualOpen;
|
||||
const variant = isOpen ? "outline" : "ghost";
|
||||
|
||||
// Allow closing during streaming (matches ChatGPT)
|
||||
const handleOpenChange = useCallback(
|
||||
(open: boolean) => {
|
||||
if (!isReasoningStreaming) {
|
||||
if (isReasoningStreaming) {
|
||||
setDismissedWhileStreaming(!open);
|
||||
} else {
|
||||
setManualOpen(open);
|
||||
}
|
||||
},
|
||||
|
|
@ -368,14 +387,17 @@ const ReasoningGroupImpl: ReasoningGroupComponent = ({
|
|||
onOpenChange={handleOpenChange}
|
||||
variant={variant}
|
||||
>
|
||||
<div className="flex items-center justify-between">
|
||||
<div className="flex items-center gap-2">
|
||||
<ReasoningTrigger
|
||||
className="min-w-0 flex-1"
|
||||
active={isReasoningStreaming}
|
||||
duration={duration || persistedDuration}
|
||||
/>
|
||||
{isOpen && !isReasoningStreaming && (
|
||||
<ReasoningCopyButton startIndex={startIndex} endIndex={endIndex} />
|
||||
)}
|
||||
<div className="flex w-16 shrink-0 justify-end">
|
||||
{isOpen && !isReasoningStreaming && (
|
||||
<ReasoningCopyButton startIndex={startIndex} endIndex={endIndex} />
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
<ReasoningContent
|
||||
aria-busy={isReasoningStreaming}
|
||||
|
|
|
|||
|
|
@ -132,13 +132,7 @@ const SuggestionItem: FC = () => {
|
|||
const prompt = useAuiState(({ suggestion }) => suggestion.prompt);
|
||||
const isDisabled = useAuiState(({ thread }) => thread.isDisabled);
|
||||
const isRunning = useAuiState(({ thread }) => thread.isRunning);
|
||||
const supportsTools = useChatRuntimeStore((s) => s.supportsTools);
|
||||
const supportsReasoning = useChatRuntimeStore((s) => s.supportsReasoning);
|
||||
const allTools = SUGGESTION_TOOLS[prompt] ?? [];
|
||||
const tools = allTools.filter((tool) => {
|
||||
if (tool === "thinking") return supportsReasoning;
|
||||
return supportsTools;
|
||||
});
|
||||
const tools = SUGGESTION_TOOLS[prompt] ?? [];
|
||||
|
||||
return (
|
||||
<button
|
||||
|
|
@ -335,9 +329,6 @@ function applyQwenThinkingParams(thinkingOn: boolean): void {
|
|||
const store = useChatRuntimeStore.getState();
|
||||
const checkpoint = store.params.checkpoint?.toLowerCase() ?? "";
|
||||
if (!checkpoint.includes("qwen3")) return;
|
||||
// Qwen3 & Qwen3.5 share the same recommended settings:
|
||||
// Thinking ON (general): temp=1.0, top_p=0.95, top_k=20
|
||||
// Thinking OFF (general): temp=0.7, top_p=0.8, top_k=20
|
||||
const params = thinkingOn
|
||||
? { temperature: 0.6, topP: 0.95, topK: 20, minP: 0.0 }
|
||||
: { temperature: 0.7, topP: 0.8, topK: 20, minP: 0.0 };
|
||||
|
|
@ -345,15 +336,18 @@ function applyQwenThinkingParams(thinkingOn: boolean): void {
|
|||
}
|
||||
|
||||
const ReasoningToggle: FC = () => {
|
||||
const modelLoaded = useChatRuntimeStore(
|
||||
(s) => !!s.params.checkpoint && !s.modelLoading,
|
||||
);
|
||||
const supportsReasoning = useChatRuntimeStore((s) => s.supportsReasoning);
|
||||
const reasoningEnabled = useChatRuntimeStore((s) => s.reasoningEnabled);
|
||||
const setReasoningEnabled = useChatRuntimeStore((s) => s.setReasoningEnabled);
|
||||
|
||||
if (!supportsReasoning) return null;
|
||||
const disabled = !modelLoaded || !supportsReasoning;
|
||||
|
||||
return (
|
||||
<button
|
||||
type="button"
|
||||
disabled={disabled}
|
||||
onClick={() => {
|
||||
const next = !reasoningEnabled;
|
||||
setReasoningEnabled(next);
|
||||
|
|
@ -361,13 +355,15 @@ const ReasoningToggle: FC = () => {
|
|||
}}
|
||||
className={cn(
|
||||
"flex items-center gap-1.5 rounded-full px-2.5 py-1 text-xs font-medium transition-colors",
|
||||
reasoningEnabled
|
||||
? "bg-primary/10 text-primary hover:bg-primary/20"
|
||||
: "bg-muted text-muted-foreground hover:bg-muted-foreground/15",
|
||||
disabled
|
||||
? "cursor-not-allowed opacity-40"
|
||||
: reasoningEnabled
|
||||
? "bg-primary/10 text-primary hover:bg-primary/20"
|
||||
: "bg-muted text-muted-foreground hover:bg-muted-foreground/15",
|
||||
)}
|
||||
aria-label={reasoningEnabled ? "Disable thinking" : "Enable thinking"}
|
||||
>
|
||||
{reasoningEnabled ? (
|
||||
{reasoningEnabled && !disabled ? (
|
||||
<LightbulbIcon className="size-3.5" />
|
||||
) : (
|
||||
<LightbulbOffIcon className="size-3.5" />
|
||||
|
|
@ -378,21 +374,26 @@ const ReasoningToggle: FC = () => {
|
|||
};
|
||||
|
||||
const WebSearchToggle: FC = () => {
|
||||
const modelLoaded = useChatRuntimeStore(
|
||||
(s) => !!s.params.checkpoint && !s.modelLoading,
|
||||
);
|
||||
const supportsTools = useChatRuntimeStore((s) => s.supportsTools);
|
||||
const toolsEnabled = useChatRuntimeStore((s) => s.toolsEnabled);
|
||||
const setToolsEnabled = useChatRuntimeStore((s) => s.setToolsEnabled);
|
||||
|
||||
if (!supportsTools) return null;
|
||||
const disabled = !modelLoaded || !supportsTools;
|
||||
|
||||
return (
|
||||
<button
|
||||
type="button"
|
||||
disabled={disabled}
|
||||
onClick={() => setToolsEnabled(!toolsEnabled)}
|
||||
className={cn(
|
||||
"flex items-center gap-1.5 rounded-full px-2.5 py-1 text-xs font-medium transition-colors",
|
||||
toolsEnabled
|
||||
? "bg-primary/10 text-primary hover:bg-primary/20"
|
||||
: "bg-muted text-muted-foreground hover:bg-muted-foreground/15",
|
||||
disabled
|
||||
? "cursor-not-allowed opacity-40"
|
||||
: toolsEnabled
|
||||
? "bg-primary/10 text-primary hover:bg-primary/20"
|
||||
: "bg-muted text-muted-foreground hover:bg-muted-foreground/15",
|
||||
)}
|
||||
aria-label={toolsEnabled ? "Disable web search" : "Enable web search"}
|
||||
>
|
||||
|
|
@ -403,23 +404,28 @@ const WebSearchToggle: FC = () => {
|
|||
};
|
||||
|
||||
const CodeToolsToggle: FC = () => {
|
||||
const modelLoaded = useChatRuntimeStore(
|
||||
(s) => !!s.params.checkpoint && !s.modelLoading,
|
||||
);
|
||||
const supportsTools = useChatRuntimeStore((s) => s.supportsTools);
|
||||
const codeToolsEnabled = useChatRuntimeStore((s) => s.codeToolsEnabled);
|
||||
const setCodeToolsEnabled = useChatRuntimeStore(
|
||||
(s) => s.setCodeToolsEnabled,
|
||||
);
|
||||
|
||||
if (!supportsTools) return null;
|
||||
const disabled = !modelLoaded || !supportsTools;
|
||||
|
||||
return (
|
||||
<button
|
||||
type="button"
|
||||
disabled={disabled}
|
||||
onClick={() => setCodeToolsEnabled(!codeToolsEnabled)}
|
||||
className={cn(
|
||||
"flex items-center gap-1.5 rounded-full px-2.5 py-1 text-xs font-medium transition-colors",
|
||||
codeToolsEnabled
|
||||
? "bg-primary/10 text-primary hover:bg-primary/20"
|
||||
: "bg-muted text-muted-foreground hover:bg-muted-foreground/15",
|
||||
disabled
|
||||
? "cursor-not-allowed opacity-40"
|
||||
: codeToolsEnabled
|
||||
? "bg-primary/10 text-primary hover:bg-primary/20"
|
||||
: "bg-muted text-muted-foreground hover:bg-muted-foreground/15",
|
||||
)}
|
||||
aria-label={codeToolsEnabled ? "Disable code execution" : "Enable code execution"}
|
||||
>
|
||||
|
|
|
|||
|
|
@ -9,6 +9,7 @@ import { Eye, EyeOff } from "lucide-react";
|
|||
import { useEffect, useState } from "react";
|
||||
import type { ReactElement } from "react";
|
||||
import type { SyntheticEvent } from "react";
|
||||
import { usePlatformStore } from "@/config/env";
|
||||
import { refreshSession } from "../api";
|
||||
|
||||
// Bootstrap credentials injected into index.html by the backend
|
||||
|
|
@ -278,7 +279,14 @@ export function AuthForm({ mode }: AuthFormProps): ReactElement | null {
|
|||
);
|
||||
navigate({ to: getPostAuthRoute() });
|
||||
} catch (err: unknown) {
|
||||
setError(err instanceof Error ? err.message : "Auth failed.");
|
||||
let msg = err instanceof Error ? err.message : "Auth failed.";
|
||||
if (msg.includes("unsloth studio reset-password") && usePlatformStore.getState().deviceType === "windows") {
|
||||
msg = msg.replace(
|
||||
"unsloth studio reset-password",
|
||||
".\\unsloth_studio\\Scripts\\unsloth.exe studio reset-password",
|
||||
);
|
||||
}
|
||||
setError(msg);
|
||||
} finally {
|
||||
setLoading(false);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -442,10 +442,12 @@ export function createOpenAIStreamAdapter(): ChatModelAdapter {
|
|||
Boolean(message),
|
||||
);
|
||||
|
||||
if (params.systemPrompt.trim()) {
|
||||
const safeSystemPrompt =
|
||||
typeof params.systemPrompt === "string" ? params.systemPrompt : "";
|
||||
if (safeSystemPrompt.trim()) {
|
||||
outboundMessages.unshift({
|
||||
role: "system",
|
||||
content: params.systemPrompt.trim(),
|
||||
content: safeSystemPrompt.trim(),
|
||||
});
|
||||
}
|
||||
const imageBase64 = findLatestUserImageBase64(messages);
|
||||
|
|
|
|||
|
|
@ -10,6 +10,16 @@ import {
|
|||
} from "@/components/ui/select";
|
||||
import { Slider } from "@/components/ui/slider";
|
||||
import { Textarea } from "@/components/ui/textarea";
|
||||
import { Input } from "@/components/ui/input";
|
||||
import { Button } from "@/components/ui/button";
|
||||
import {
|
||||
Dialog,
|
||||
DialogContent,
|
||||
DialogDescription,
|
||||
DialogFooter,
|
||||
DialogHeader,
|
||||
DialogTitle,
|
||||
} from "@/components/ui/dialog";
|
||||
import {
|
||||
ArrowDown01Icon,
|
||||
CodeIcon,
|
||||
|
|
@ -30,7 +40,7 @@ import {
|
|||
} from "@/components/ui/sheet";
|
||||
import { useIsMobile } from "@/hooks/use-mobile";
|
||||
import type { ReactNode } from "react";
|
||||
import { useState } from "react";
|
||||
import { useEffect, useMemo, useState } from "react";
|
||||
import {
|
||||
DEFAULT_INFERENCE_PARAMS,
|
||||
type InferenceParams,
|
||||
|
|
@ -72,6 +82,52 @@ const BUILTIN_PRESETS: Preset[] = [
|
|||
},
|
||||
];
|
||||
|
||||
const CHAT_PRESETS_KEY = "unsloth_chat_custom_presets";
|
||||
const CHAT_ACTIVE_PRESET_KEY = "unsloth_chat_active_preset";
|
||||
|
||||
function canUseStorage(): boolean {
|
||||
return typeof window !== "undefined";
|
||||
}
|
||||
|
||||
function loadSavedCustomPresets(): Preset[] {
|
||||
if (!canUseStorage()) return [];
|
||||
try {
|
||||
const raw = localStorage.getItem(CHAT_PRESETS_KEY);
|
||||
if (!raw) return [];
|
||||
const parsed = JSON.parse(raw) as unknown;
|
||||
if (!Array.isArray(parsed)) return [];
|
||||
return parsed
|
||||
.filter((item): item is Preset => {
|
||||
if (!item || typeof item !== "object") return false;
|
||||
const maybe = item as Partial<Preset>;
|
||||
return typeof maybe.name === "string" && !!maybe.params;
|
||||
})
|
||||
.map((preset) => ({
|
||||
name: preset.name.trim(),
|
||||
params: {
|
||||
...defaultInferenceParams,
|
||||
...preset.params,
|
||||
},
|
||||
}))
|
||||
.filter(
|
||||
(preset) =>
|
||||
preset.name.length > 0 &&
|
||||
!BUILTIN_PRESETS.some((builtin) => builtin.name === preset.name),
|
||||
);
|
||||
} catch {
|
||||
return [];
|
||||
}
|
||||
}
|
||||
|
||||
function loadSavedActivePreset(): string {
|
||||
if (!canUseStorage()) return "Default";
|
||||
try {
|
||||
return localStorage.getItem(CHAT_ACTIVE_PRESET_KEY) ?? "Default";
|
||||
} catch {
|
||||
return "Default";
|
||||
}
|
||||
}
|
||||
|
||||
function ParamSlider({
|
||||
label,
|
||||
value,
|
||||
|
|
@ -181,8 +237,16 @@ export function ChatSettingsPanel({
|
|||
const ggufContextLength = useChatRuntimeStore((s) => s.ggufContextLength);
|
||||
const kvCacheDtype = useChatRuntimeStore((s) => s.kvCacheDtype);
|
||||
const setKvCacheDtype = useChatRuntimeStore((s) => s.setKvCacheDtype);
|
||||
const [presets, setPresets] = useState<Preset[]>(BUILTIN_PRESETS);
|
||||
const [activePreset, setActivePreset] = useState("Default");
|
||||
const [customPresets, setCustomPresets] = useState<Preset[]>(() =>
|
||||
loadSavedCustomPresets(),
|
||||
);
|
||||
const [activePreset, setActivePreset] = useState(() => loadSavedActivePreset());
|
||||
const [savePresetOpen, setSavePresetOpen] = useState(false);
|
||||
const [presetNameDraft, setPresetNameDraft] = useState("");
|
||||
const presets = useMemo(
|
||||
() => [...BUILTIN_PRESETS, ...customPresets],
|
||||
[customPresets],
|
||||
);
|
||||
const isBuiltinPreset = BUILTIN_PRESETS.some((p) => p.name === activePreset);
|
||||
|
||||
function set<K extends keyof InferenceParams>(key: K) {
|
||||
|
|
@ -199,32 +263,93 @@ export function ChatSettingsPanel({
|
|||
trustRemoteCode: params.trustRemoteCode,
|
||||
});
|
||||
setActivePreset(name);
|
||||
if (canUseStorage()) {
|
||||
try {
|
||||
localStorage.setItem(CHAT_ACTIVE_PRESET_KEY, name);
|
||||
} catch {
|
||||
// ignore
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function savePreset() {
|
||||
const name = prompt("Preset name:");
|
||||
if (!name?.trim()) {
|
||||
function openSavePresetDialog() {
|
||||
setPresetNameDraft(activePreset === "Default" ? "" : activePreset);
|
||||
setSavePresetOpen(true);
|
||||
}
|
||||
|
||||
function savePresetWithName(rawName: string) {
|
||||
const trimmed = rawName.trim();
|
||||
if (!trimmed) {
|
||||
return;
|
||||
}
|
||||
const trimmed = name.trim();
|
||||
setPresets((prev) => [
|
||||
...prev.filter((p) => p.name !== trimmed),
|
||||
{ name: trimmed, params: { ...params } },
|
||||
]);
|
||||
if (BUILTIN_PRESETS.some((preset) => preset.name === trimmed)) {
|
||||
return;
|
||||
}
|
||||
setCustomPresets((prev) => {
|
||||
const next = [
|
||||
...prev.filter((preset) => preset.name !== trimmed),
|
||||
{ name: trimmed, params: { ...params } },
|
||||
];
|
||||
if (canUseStorage()) {
|
||||
try {
|
||||
localStorage.setItem(CHAT_PRESETS_KEY, JSON.stringify(next));
|
||||
} catch {
|
||||
// ignore
|
||||
}
|
||||
}
|
||||
return next;
|
||||
});
|
||||
if (canUseStorage()) {
|
||||
try {
|
||||
localStorage.setItem(CHAT_ACTIVE_PRESET_KEY, trimmed);
|
||||
} catch {
|
||||
// ignore
|
||||
}
|
||||
}
|
||||
setActivePreset(trimmed);
|
||||
setSavePresetOpen(false);
|
||||
}
|
||||
|
||||
function deletePreset(name: string) {
|
||||
if (BUILTIN_PRESETS.some((p) => p.name === name)) {
|
||||
return;
|
||||
}
|
||||
setPresets((prev) => prev.filter((p) => p.name !== name));
|
||||
setCustomPresets((prev) => {
|
||||
const next = prev.filter((preset) => preset.name !== name);
|
||||
if (canUseStorage()) {
|
||||
try {
|
||||
localStorage.setItem(CHAT_PRESETS_KEY, JSON.stringify(next));
|
||||
} catch {
|
||||
// ignore
|
||||
}
|
||||
}
|
||||
return next;
|
||||
});
|
||||
if (activePreset === name) {
|
||||
setActivePreset("Default");
|
||||
if (canUseStorage()) {
|
||||
try {
|
||||
localStorage.setItem(CHAT_ACTIVE_PRESET_KEY, "Default");
|
||||
} catch {
|
||||
// ignore
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
useEffect(() => {
|
||||
if (presets.some((preset) => preset.name === activePreset)) return;
|
||||
setActivePreset("Default");
|
||||
if (canUseStorage()) {
|
||||
try {
|
||||
localStorage.setItem(CHAT_ACTIVE_PRESET_KEY, "Default");
|
||||
} catch {
|
||||
// ignore
|
||||
}
|
||||
}
|
||||
}, [activePreset, presets]);
|
||||
|
||||
const settingsContent = (
|
||||
<>
|
||||
<div className="flex items-center gap-2 px-4 py-3">
|
||||
|
|
@ -255,7 +380,7 @@ export function ChatSettingsPanel({
|
|||
</Select>
|
||||
<button
|
||||
type="button"
|
||||
onClick={savePreset}
|
||||
onClick={openSavePresetDialog}
|
||||
className="flex h-8 items-center gap-1.5 rounded-md border px-2.5 text-xs text-muted-foreground transition-colors hover:bg-accent"
|
||||
title="Save preset"
|
||||
>
|
||||
|
|
@ -442,6 +567,51 @@ export function ChatSettingsPanel({
|
|||
|
||||
<ChatTemplateSection onReloadModel={onReloadModel} />
|
||||
</div>
|
||||
<Dialog
|
||||
open={savePresetOpen}
|
||||
onOpenChange={(nextOpen) => {
|
||||
setSavePresetOpen(nextOpen);
|
||||
if (!nextOpen) {
|
||||
setPresetNameDraft("");
|
||||
}
|
||||
}}
|
||||
>
|
||||
<DialogContent className="corner-squircle sm:max-w-sm">
|
||||
<DialogHeader>
|
||||
<DialogTitle>Save Preset</DialogTitle>
|
||||
<DialogDescription>
|
||||
Enter a name for this inference preset.
|
||||
</DialogDescription>
|
||||
</DialogHeader>
|
||||
<form
|
||||
onSubmit={(event) => {
|
||||
event.preventDefault();
|
||||
savePresetWithName(presetNameDraft);
|
||||
}}
|
||||
className="space-y-4"
|
||||
>
|
||||
<Input
|
||||
autoFocus={true}
|
||||
value={presetNameDraft}
|
||||
onChange={(event) => setPresetNameDraft(event.target.value)}
|
||||
placeholder="Preset name"
|
||||
maxLength={80}
|
||||
/>
|
||||
<DialogFooter>
|
||||
<Button
|
||||
type="button"
|
||||
variant="outline"
|
||||
onClick={() => setSavePresetOpen(false)}
|
||||
>
|
||||
Cancel
|
||||
</Button>
|
||||
<Button type="submit" disabled={presetNameDraft.trim().length === 0}>
|
||||
Save
|
||||
</Button>
|
||||
</DialogFooter>
|
||||
</form>
|
||||
</DialogContent>
|
||||
</Dialog>
|
||||
</>
|
||||
);
|
||||
|
||||
|
|
|
|||
|
|
@ -237,6 +237,9 @@ export function SharedComposer({
|
|||
const checkpoint = s.params.checkpoint;
|
||||
return s.models.find((m) => m.id === checkpoint);
|
||||
});
|
||||
const modelLoaded = useChatRuntimeStore(
|
||||
(s) => !!s.params.checkpoint && !s.modelLoading,
|
||||
);
|
||||
const supportsReasoning = useChatRuntimeStore((s) => s.supportsReasoning);
|
||||
const reasoningEnabled = useChatRuntimeStore((s) => s.reasoningEnabled);
|
||||
const setReasoningEnabled = useChatRuntimeStore((s) => s.setReasoningEnabled);
|
||||
|
|
@ -245,6 +248,8 @@ export function SharedComposer({
|
|||
const setToolsEnabled = useChatRuntimeStore((s) => s.setToolsEnabled);
|
||||
const codeToolsEnabled = useChatRuntimeStore((s) => s.codeToolsEnabled);
|
||||
const setCodeToolsEnabled = useChatRuntimeStore((s) => s.setCodeToolsEnabled);
|
||||
const reasoningDisabled = !modelLoaded || !supportsReasoning;
|
||||
const toolsDisabled = !modelLoaded || !supportsTools;
|
||||
const setPendingAudioStore = useChatRuntimeStore((s) => s.setPendingAudio);
|
||||
const clearPendingAudioStore = useChatRuntimeStore((s) => s.clearPendingAudio);
|
||||
|
||||
|
|
@ -519,70 +524,73 @@ export function SharedComposer({
|
|||
</TooltipIconButton>
|
||||
</>
|
||||
)}
|
||||
{supportsReasoning && (
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => {
|
||||
const next = !reasoningEnabled;
|
||||
setReasoningEnabled(next);
|
||||
// Qwen3/3.5: adjust params for thinking on/off
|
||||
const store = useChatRuntimeStore.getState();
|
||||
const cp = store.params.checkpoint?.toLowerCase() ?? "";
|
||||
if (cp.includes("qwen3")) {
|
||||
const p = next
|
||||
? { temperature: 0.6, topP: 0.95, topK: 20, minP: 0.0 }
|
||||
: { temperature: 0.7, topP: 0.8, topK: 20, minP: 0.0 };
|
||||
store.setParams({ ...store.params, ...p });
|
||||
}
|
||||
}}
|
||||
className={cn(
|
||||
"flex items-center gap-0.5 rounded-full px-2 py-0.5 text-xs font-medium transition-colors",
|
||||
reasoningEnabled
|
||||
<button
|
||||
type="button"
|
||||
disabled={reasoningDisabled}
|
||||
onClick={() => {
|
||||
const next = !reasoningEnabled;
|
||||
setReasoningEnabled(next);
|
||||
// Qwen3/3.5: adjust params for thinking on/off
|
||||
const store = useChatRuntimeStore.getState();
|
||||
const cp = store.params.checkpoint?.toLowerCase() ?? "";
|
||||
if (cp.includes("qwen3")) {
|
||||
const p = next
|
||||
? { temperature: 0.6, topP: 0.95, topK: 20, minP: 0.0 }
|
||||
: { temperature: 0.7, topP: 0.8, topK: 20, minP: 0.0 };
|
||||
store.setParams({ ...store.params, ...p });
|
||||
}
|
||||
}}
|
||||
className={cn(
|
||||
"flex items-center gap-0.5 rounded-full px-2 py-0.5 text-xs font-medium transition-colors",
|
||||
reasoningDisabled
|
||||
? "cursor-not-allowed opacity-40"
|
||||
: reasoningEnabled
|
||||
? "bg-primary/10 text-primary hover:bg-primary/20"
|
||||
: "bg-muted text-muted-foreground hover:bg-muted-foreground/15",
|
||||
)}
|
||||
aria-label={reasoningEnabled ? "Disable thinking" : "Enable thinking"}
|
||||
>
|
||||
{reasoningEnabled ? (
|
||||
<LightbulbIcon className="size-3" />
|
||||
) : (
|
||||
<LightbulbOffIcon className="size-3" />
|
||||
)}
|
||||
<span>Think</span>
|
||||
</button>
|
||||
)}
|
||||
{supportsTools && (
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => setToolsEnabled(!toolsEnabled)}
|
||||
className={cn(
|
||||
"flex items-center gap-1.5 rounded-full px-2.5 py-1 text-xs font-medium transition-colors",
|
||||
toolsEnabled
|
||||
)}
|
||||
aria-label={reasoningEnabled ? "Disable thinking" : "Enable thinking"}
|
||||
>
|
||||
{reasoningEnabled && !reasoningDisabled ? (
|
||||
<LightbulbIcon className="size-3" />
|
||||
) : (
|
||||
<LightbulbOffIcon className="size-3" />
|
||||
)}
|
||||
<span>Think</span>
|
||||
</button>
|
||||
<button
|
||||
type="button"
|
||||
disabled={toolsDisabled}
|
||||
onClick={() => setToolsEnabled(!toolsEnabled)}
|
||||
className={cn(
|
||||
"flex items-center gap-1.5 rounded-full px-2.5 py-1 text-xs font-medium transition-colors",
|
||||
toolsDisabled
|
||||
? "cursor-not-allowed opacity-40"
|
||||
: toolsEnabled
|
||||
? "bg-primary/10 text-primary hover:bg-primary/20"
|
||||
: "bg-muted text-muted-foreground hover:bg-muted-foreground/15",
|
||||
)}
|
||||
aria-label={toolsEnabled ? "Disable web search" : "Enable web search"}
|
||||
>
|
||||
<GlobeIcon className="size-3.5" />
|
||||
<span>Search</span>
|
||||
</button>
|
||||
)}
|
||||
{supportsTools && (
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => setCodeToolsEnabled(!codeToolsEnabled)}
|
||||
className={cn(
|
||||
"flex items-center gap-1.5 rounded-full px-2.5 py-1 text-xs font-medium transition-colors",
|
||||
codeToolsEnabled
|
||||
)}
|
||||
aria-label={toolsEnabled ? "Disable web search" : "Enable web search"}
|
||||
>
|
||||
<GlobeIcon className="size-3.5" />
|
||||
<span>Search</span>
|
||||
</button>
|
||||
<button
|
||||
type="button"
|
||||
disabled={toolsDisabled}
|
||||
onClick={() => setCodeToolsEnabled(!codeToolsEnabled)}
|
||||
className={cn(
|
||||
"flex items-center gap-1.5 rounded-full px-2.5 py-1 text-xs font-medium transition-colors",
|
||||
toolsDisabled
|
||||
? "cursor-not-allowed opacity-40"
|
||||
: codeToolsEnabled
|
||||
? "bg-primary/10 text-primary hover:bg-primary/20"
|
||||
: "bg-muted text-muted-foreground hover:bg-muted-foreground/15",
|
||||
)}
|
||||
aria-label={codeToolsEnabled ? "Disable code execution" : "Enable code execution"}
|
||||
>
|
||||
<TerminalIcon className="size-3.5" />
|
||||
<span>Code</span>
|
||||
</button>
|
||||
)}
|
||||
)}
|
||||
aria-label={codeToolsEnabled ? "Disable code execution" : "Enable code execution"}
|
||||
>
|
||||
<TerminalIcon className="size-3.5" />
|
||||
<span>Code</span>
|
||||
</button>
|
||||
</div>
|
||||
<div className="flex items-center gap-1">
|
||||
{dictationSupported && (
|
||||
|
|
|
|||
|
|
@ -2,6 +2,7 @@
|
|||
// Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
import { create } from "zustand";
|
||||
import { toast } from "sonner";
|
||||
import {
|
||||
DEFAULT_INFERENCE_PARAMS,
|
||||
type ChatLoraSummary,
|
||||
|
|
@ -13,6 +14,8 @@ const AUTO_TITLE_KEY = "unsloth_chat_auto_title";
|
|||
const AUTO_HEAL_TOOL_CALLS_KEY = "unsloth_auto_heal_tool_calls";
|
||||
const MAX_TOOL_CALLS_KEY = "unsloth_max_tool_calls_per_message";
|
||||
const TOOL_CALL_TIMEOUT_KEY = "unsloth_tool_call_timeout";
|
||||
const INFERENCE_PARAMS_KEY = "unsloth_chat_inference_params";
|
||||
let hasShownInferencePersistenceWarning = false;
|
||||
|
||||
function canUseStorage(): boolean {
|
||||
return typeof window !== "undefined";
|
||||
|
|
@ -59,6 +62,65 @@ function saveInt(key: string, value: number): void {
|
|||
}
|
||||
}
|
||||
|
||||
function asFiniteNumber(value: unknown, fallback: number): number {
|
||||
return typeof value === "number" && Number.isFinite(value) ? value : fallback;
|
||||
}
|
||||
|
||||
function asString(value: unknown, fallback: string): string {
|
||||
return typeof value === "string" ? value : fallback;
|
||||
}
|
||||
|
||||
function asBoolean(value: unknown, fallback: boolean): boolean {
|
||||
return typeof value === "boolean" ? value : fallback;
|
||||
}
|
||||
|
||||
function loadInferenceParams(): InferenceParams {
|
||||
if (!canUseStorage()) return DEFAULT_INFERENCE_PARAMS;
|
||||
try {
|
||||
const raw = localStorage.getItem(INFERENCE_PARAMS_KEY);
|
||||
if (!raw) return DEFAULT_INFERENCE_PARAMS;
|
||||
const parsed = JSON.parse(raw) as Partial<InferenceParams>;
|
||||
return {
|
||||
temperature: asFiniteNumber(parsed.temperature, DEFAULT_INFERENCE_PARAMS.temperature),
|
||||
topP: asFiniteNumber(parsed.topP, DEFAULT_INFERENCE_PARAMS.topP),
|
||||
topK: asFiniteNumber(parsed.topK, DEFAULT_INFERENCE_PARAMS.topK),
|
||||
minP: asFiniteNumber(parsed.minP, DEFAULT_INFERENCE_PARAMS.minP),
|
||||
repetitionPenalty: asFiniteNumber(
|
||||
parsed.repetitionPenalty,
|
||||
DEFAULT_INFERENCE_PARAMS.repetitionPenalty,
|
||||
),
|
||||
presencePenalty: asFiniteNumber(
|
||||
parsed.presencePenalty,
|
||||
DEFAULT_INFERENCE_PARAMS.presencePenalty,
|
||||
),
|
||||
maxSeqLength: asFiniteNumber(
|
||||
parsed.maxSeqLength,
|
||||
DEFAULT_INFERENCE_PARAMS.maxSeqLength,
|
||||
),
|
||||
maxTokens: asFiniteNumber(parsed.maxTokens, DEFAULT_INFERENCE_PARAMS.maxTokens),
|
||||
systemPrompt: asString(parsed.systemPrompt, DEFAULT_INFERENCE_PARAMS.systemPrompt),
|
||||
checkpoint: DEFAULT_INFERENCE_PARAMS.checkpoint,
|
||||
trustRemoteCode: asBoolean(
|
||||
parsed.trustRemoteCode,
|
||||
DEFAULT_INFERENCE_PARAMS.trustRemoteCode ?? false,
|
||||
),
|
||||
};
|
||||
} catch {
|
||||
return DEFAULT_INFERENCE_PARAMS;
|
||||
}
|
||||
}
|
||||
|
||||
function saveInferenceParams(params: InferenceParams): boolean {
|
||||
if (!canUseStorage()) return false;
|
||||
try {
|
||||
const { checkpoint: _, ...rest } = params;
|
||||
localStorage.setItem(INFERENCE_PARAMS_KEY, JSON.stringify(rest));
|
||||
return true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
type ChatRuntimeStore = {
|
||||
params: InferenceParams;
|
||||
models: ChatModelSummary[];
|
||||
|
|
@ -117,7 +179,7 @@ type ChatRuntimeStore = {
|
|||
};
|
||||
|
||||
export const useChatRuntimeStore = create<ChatRuntimeStore>((set) => ({
|
||||
params: DEFAULT_INFERENCE_PARAMS,
|
||||
params: loadInferenceParams(),
|
||||
models: [],
|
||||
loras: [],
|
||||
runningByThreadId: {},
|
||||
|
|
@ -144,7 +206,18 @@ export const useChatRuntimeStore = create<ChatRuntimeStore>((set) => ({
|
|||
contextUsage: null,
|
||||
modelLoading: false,
|
||||
setModelLoading: (loading) => set({ modelLoading: loading }),
|
||||
setParams: (params) => set({ params }),
|
||||
setParams: (params) =>
|
||||
set(() => {
|
||||
const persisted = saveInferenceParams(params);
|
||||
if (!persisted && !hasShownInferencePersistenceWarning) {
|
||||
hasShownInferencePersistenceWarning = true;
|
||||
toast.warning("Chat settings could not be persisted", {
|
||||
description:
|
||||
"Your changes apply now, but may reset after refresh.",
|
||||
});
|
||||
}
|
||||
return { params };
|
||||
}),
|
||||
setModels: (models) => set({ models }),
|
||||
setLoras: (loras) => set({ loras }),
|
||||
setThreadRunning: (threadId, running) =>
|
||||
|
|
|
|||
|
|
@ -12,7 +12,7 @@
|
|||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
__version__ = "2026.3.10"
|
||||
__version__ = "2026.3.11"
|
||||
|
||||
__all__ = [
|
||||
"SUPPORTS_BFLOAT16",
|
||||
|
|
|
|||
|
|
@ -153,11 +153,18 @@ def sft_trainer_prepare_dataset(function_name, function):
|
|||
"if 'tokenizer' not in locals(): tokenizer = processing_class\n"
|
||||
"if 'formatting_func' not in locals(): raise RuntimeError('Unsloth: Please file a bug report - `formatting_func` does not exist!')\n"
|
||||
"if 'dataset_text_field' not in locals() and 'args' in locals(): dataset_text_field = args.dataset_text_field\n"
|
||||
"if 'dataset_text_field' not in locals(): raise RuntimeError('Unsloth: Please file a bug report - `dataset_text_field` does not exist!')\n"
|
||||
"test_text = dataset[0][dataset_text_field] if (formatting_func is None and dataset_text_field is not None) else formatting_func(dataset[0])[0]\n"
|
||||
"if 'dataset_text_field' not in locals(): dataset_text_field = None\n"
|
||||
"if formatting_func is None and dataset_text_field is None and 'prompt' in dataset[0] and 'completion' in dataset[0]:\n"
|
||||
" test_text = (dataset[0]['prompt'] + dataset[0]['completion']) if (isinstance(dataset[0]['prompt'], str) and isinstance(dataset[0]['completion'], str)) else None\n"
|
||||
"elif formatting_func is None and dataset_text_field is not None:\n"
|
||||
" test_text = dataset[0][dataset_text_field]\n"
|
||||
"elif formatting_func is not None:\n"
|
||||
" test_text = formatting_func(dataset[0])[0]\n"
|
||||
"else:\n"
|
||||
" test_text = None\n"
|
||||
"chat_template = getattr(tokenizer, 'chat_template', None)\n"
|
||||
"chat_template = '' if chat_template is None else chat_template\n"
|
||||
"has_bos_token_already = (test_text.startswith(tokenizer.bos_token) or tokenizer.bos_token in chat_template) "
|
||||
"has_bos_token_already = ((test_text is not None and test_text.startswith(tokenizer.bos_token)) or tokenizer.bos_token in chat_template) "
|
||||
"if getattr(tokenizer, 'bos_token', None) is not None else False\n"
|
||||
"if 'add_special_tokens' not in locals() and has_bos_token_already:\n"
|
||||
" from functools import partial\n"
|
||||
|
|
|
|||
|
|
@ -974,11 +974,18 @@ def patch_sft_trainer_tokenizer():
|
|||
"if 'tokenizer' not in locals(): tokenizer = processing_class\n"
|
||||
"if 'formatting_func' not in locals(): raise RuntimeError('Unsloth: Please file a bug report - `formatting_func` does not exist!')\n"
|
||||
"if 'dataset_text_field' not in locals() and 'args' in locals(): dataset_text_field = args.dataset_text_field\n"
|
||||
"if 'dataset_text_field' not in locals(): raise RuntimeError('Unsloth: Please file a bug report - `dataset_text_field` does not exist!')\n"
|
||||
"test_text = dataset[0][dataset_text_field] if (formatting_func is None and dataset_text_field is not None) else formatting_func(dataset[0])[0]\n"
|
||||
"if 'dataset_text_field' not in locals(): dataset_text_field = None\n"
|
||||
"if formatting_func is None and dataset_text_field is None and 'prompt' in dataset[0] and 'completion' in dataset[0]:\n"
|
||||
" test_text = (dataset[0]['prompt'] + dataset[0]['completion']) if (isinstance(dataset[0]['prompt'], str) and isinstance(dataset[0]['completion'], str)) else None\n"
|
||||
"elif formatting_func is None and dataset_text_field is not None:\n"
|
||||
" test_text = dataset[0][dataset_text_field]\n"
|
||||
"elif formatting_func is not None:\n"
|
||||
" test_text = formatting_func(dataset[0])[0]\n"
|
||||
"else:\n"
|
||||
" test_text = None\n"
|
||||
"chat_template = getattr(tokenizer, 'chat_template', None)\n"
|
||||
"chat_template = '' if chat_template is None else chat_template\n"
|
||||
"has_bos_token_already = (test_text.startswith(tokenizer.bos_token) or tokenizer.bos_token in chat_template) "
|
||||
"has_bos_token_already = ((test_text is not None and test_text.startswith(tokenizer.bos_token)) or tokenizer.bos_token in chat_template) "
|
||||
"if getattr(tokenizer, 'bos_token', None) is not None else False\n"
|
||||
"if 'add_special_tokens' not in locals() and has_bos_token_already:\n"
|
||||
" from functools import partial\n"
|
||||
|
|
|
|||
|
|
@ -132,8 +132,8 @@ except:
|
|||
|
||||
class UnslothTrainingArguments(TrainingArguments):
|
||||
def __init__(self, embedding_learning_rate: float = None, *args, **kwargs):
|
||||
embedding_learning_rate = embedding_learning_rate
|
||||
super().__init__(*args, **kwargs)
|
||||
self.embedding_learning_rate = embedding_learning_rate
|
||||
|
||||
|
||||
def _create_unsloth_optimizer(
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue