resolved merge conflicts
This commit is contained in:
Manan17
commit
8203637d89
32 changed files with 2178 additions and 148 deletions
118
README.md
118
README.md
|
|
@ -30,28 +30,119 @@
|
|||
|
||||
## Quick Start
|
||||
|
||||
### One-command setup
|
||||
### Prerequisites
|
||||
|
||||
| Requirement | Linux / WSL | Windows |
|
||||
|---|---|---|
|
||||
| **GPU** | NVIDIA GPU with working driver | NVIDIA GPU with working driver |
|
||||
| **Python** | 3.11 – 3.13 | 3.11 – 3.13 |
|
||||
| **Git** | Pre-installed on most distros | Auto-installed by setup script (via `winget`) |
|
||||
| **CMake** | Pre-installed or `sudo apt install cmake` | Auto-installed by setup script (via `winget`) |
|
||||
| **C++ compiler** | `build-essential` (auto-detected) | Visual Studio Build Tools 2022 (auto-installed by setup script) |
|
||||
| **CUDA Toolkit** | Optional — setup auto-detects `nvcc` | Auto-installed by setup script (version matched to driver) |
|
||||
|
||||
> [!NOTE]
|
||||
> On **WSL**, the setup script will also run `sudo apt-get install build-essential cmake curl git libcurl4-openssl-dev` so that GGUF export works in non-interactive subprocesses. You may be prompted for your password during setup.
|
||||
|
||||
---
|
||||
|
||||
### Linux / Windows WSL
|
||||
|
||||
```bash
|
||||
# 1. Clone the repo
|
||||
git clone https://github.com/unslothai/unsloth-studio.git
|
||||
cd unsloth-studio
|
||||
|
||||
# 2. Run setup (installs Node, builds frontend, creates .venv, builds llama.cpp)
|
||||
bash setup.sh
|
||||
|
||||
# 3. Open a new terminal (or source your shell rc), then launch:
|
||||
unsloth-studio -H 0.0.0.0 -p 8000
|
||||
```
|
||||
|
||||
This script will:
|
||||
1. Install **Node.js ≥ 20** via nvm (if needed)
|
||||
2. Build the frontend to `studio/frontend/dist`
|
||||
3. Create a Python virtual environment and install all dependencies (including `unsloth`)
|
||||
4. Register a convenient `unsloth-ui` shell alias
|
||||
<details>
|
||||
<summary><b>What does <code>setup.sh</code> do?</b></summary>
|
||||
|
||||
### Launch the studio
|
||||
1. Installs **Node.js ≥ 20** via nvm (if needed)
|
||||
2. Runs `npm install && npm run build` for the React frontend
|
||||
3. Detects the best **Python 3.11 – 3.13** on your system and creates a `.venv`
|
||||
4. Installs all Python dependencies (unsloth, PyTorch with CUDA, triton kernels, etc.)
|
||||
5. On **WSL**: pre-installs build dependencies via `apt-get`
|
||||
6. Clones and builds **llama.cpp** at `~/.unsloth/llama.cpp` (GPU-accelerated if CUDA is found)
|
||||
7. Registers `unsloth-studio` and `unsloth-ui` shell aliases in your shell rc (bash, zsh, fish, or ksh)
|
||||
|
||||
</details>
|
||||
|
||||
---
|
||||
|
||||
### Windows (Native)
|
||||
|
||||
> [!IMPORTANT]
|
||||
> Requires an **NVIDIA GPU** — CPU-only machines are not supported on Windows.
|
||||
|
||||
```powershell
|
||||
# 1. Clone the repo
|
||||
git clone https://github.com/unslothai/unsloth-studio.git
|
||||
cd unsloth-studio
|
||||
|
||||
# 2. Run setup (Right-click → "Run with PowerShell", or from a terminal):
|
||||
.\setup.bat
|
||||
# Or directly:
|
||||
powershell -ExecutionPolicy Bypass -File setup.ps1
|
||||
```
|
||||
|
||||
After setup completes, **open a new terminal** and run:
|
||||
|
||||
```powershell
|
||||
# PowerShell
|
||||
unsloth-studio -H 0.0.0.0 -p 8000
|
||||
|
||||
# Or cmd.exe
|
||||
unsloth-studio -H 0.0.0.0 -p 8000
|
||||
```
|
||||
|
||||
<details>
|
||||
<summary><b>What does <code>setup.ps1</code> do?</b></summary>
|
||||
|
||||
1. Enables **Windows Long Paths** (required for deep dependency trees — prompts for UAC)
|
||||
2. Auto-installs missing system tools via `winget`: **Git**, **CMake**, **Visual Studio Build Tools 2022**, **CUDA Toolkit** (version-matched to your driver), **Node.js LTS**, **Python 3.12**, **OpenSSL dev**
|
||||
3. Builds the React frontend (`npm install && npm run build`)
|
||||
4. Creates a `.venv` and installs all Python dependencies (including CUDA-enabled PyTorch from the official index)
|
||||
5. Sets `TORCHINDUCTOR_CACHE_DIR=C:\tc` to avoid Windows MAX_PATH issues with Triton
|
||||
6. Clones and builds **llama.cpp** at `%USERPROFILE%\.unsloth\llama.cpp` with CUDA + Visual Studio
|
||||
7. Registers `unsloth-studio` and `unsloth-ui` commands in both PowerShell profile and `cmd.exe` (via batch files on PATH)
|
||||
|
||||
</details>
|
||||
|
||||
---
|
||||
|
||||
### Google Colab
|
||||
|
||||
The setup script auto-detects Colab and installs everything into the existing system Python (no venv):
|
||||
|
||||
```python
|
||||
!bash setup.sh
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Launching the Studio
|
||||
|
||||
After setup on any platform, the command is the same:
|
||||
|
||||
```bash
|
||||
# After setup, open a new terminal (or source ~/.bashrc), then inside your working directory:
|
||||
unsloth-ui -H 0.0.0.0 -p 8000
|
||||
unsloth-studio -H 0.0.0.0 -p 8000
|
||||
```
|
||||
|
||||
On **first launch**, a one-time setup token is printed to the console. Use it in the browser to create your admin account.
|
||||
| Flag | Description |
|
||||
|---|---|
|
||||
| `-H` / `--host` | Bind address (`0.0.0.0` for all interfaces, `127.0.0.1` for local only) |
|
||||
| `-p` / `--port` | Port number (default: `8000`) |
|
||||
|
||||
As this repo is in continuous development, please make sure to run the setup.sh file everytime you pull new changes from the repo.
|
||||
On **first launch**, a one-time setup token is printed to the console. Open the URL shown in your browser and use this token to create your admin account.
|
||||
|
||||
> [!TIP]
|
||||
> This repo is in active development. After pulling new changes, **always re-run the setup script** (`bash setup.sh` or `.\setup.bat`) to pick up dependency and build updates.
|
||||
|
||||
## API Reference
|
||||
|
||||
|
|
@ -105,7 +196,10 @@ new-ui-prototype/
|
|||
│ ├── export.py
|
||||
│ ├── ui.py
|
||||
│ └── studio.py
|
||||
├── setup.sh # One-command bootstrap script
|
||||
├── setup.sh # Bootstrap script (Linux / WSL / Colab)
|
||||
├── setup.ps1 # Bootstrap script (Windows native)
|
||||
├── setup.bat # Wrapper to launch setup.ps1 via double-click
|
||||
├── install_python_stack.py # Cross-platform Python dependency installer
|
||||
└── studio/
|
||||
├── backend/
|
||||
│ ├── main.py # FastAPI app & middleware
|
||||
|
|
|
|||
245
install_python_stack.py
Normal file
245
install_python_stack.py
Normal file
|
|
@ -0,0 +1,245 @@
|
|||
#!/usr/bin/env python3
|
||||
"""Cross-platform Python dependency installer for Unsloth Studio.
|
||||
|
||||
Called by both setup.sh (Linux / WSL) and setup.ps1 (Windows) after the
|
||||
virtual environment is already activated. Expects `pip` and `python` on
|
||||
PATH to point at the venv.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
|
||||
IS_WINDOWS = sys.platform == "win32"
|
||||
|
||||
# ── Paths ──────────────────────────────────────────────────────────────
|
||||
SCRIPT_DIR = Path(__file__).resolve().parent
|
||||
REQ_ROOT = SCRIPT_DIR / "studio" / "backend" / "requirements"
|
||||
SINGLE_ENV = REQ_ROOT / "single-env"
|
||||
CONSTRAINTS = SINGLE_ENV / "constraints.txt"
|
||||
|
||||
# ── Color support ──────────────────────────────────────────────────────
|
||||
|
||||
def _enable_colors() -> bool:
|
||||
"""Try to enable ANSI color support. Returns True if available."""
|
||||
if not hasattr(sys.stdout, "fileno"):
|
||||
return False
|
||||
try:
|
||||
if not os.isatty(sys.stdout.fileno()):
|
||||
return False
|
||||
except Exception:
|
||||
return False
|
||||
if IS_WINDOWS:
|
||||
try:
|
||||
import ctypes
|
||||
kernel32 = ctypes.windll.kernel32
|
||||
# Enable ENABLE_VIRTUAL_TERMINAL_PROCESSING (0x0004) on stdout
|
||||
handle = kernel32.GetStdHandle(-11) # STD_OUTPUT_HANDLE
|
||||
mode = ctypes.c_ulong()
|
||||
kernel32.GetConsoleMode(handle, ctypes.byref(mode))
|
||||
kernel32.SetConsoleMode(handle, mode.value | 0x0004)
|
||||
return True
|
||||
except Exception:
|
||||
return False
|
||||
return True # Unix terminals support ANSI by default
|
||||
|
||||
_HAS_COLOR = _enable_colors()
|
||||
|
||||
def _green(msg: str) -> str:
|
||||
return f"\033[92m{msg}\033[0m" if _HAS_COLOR else msg
|
||||
|
||||
def _cyan(msg: str) -> str:
|
||||
return f"\033[96m{msg}\033[0m" if _HAS_COLOR else msg
|
||||
|
||||
def _red(msg: str) -> str:
|
||||
return f"\033[91m{msg}\033[0m" if _HAS_COLOR else msg
|
||||
|
||||
|
||||
def run(label: str, cmd: list[str], *, quiet: bool = True) -> None:
|
||||
"""Run a command; on failure print output and exit."""
|
||||
print(_cyan(f" {label}..."))
|
||||
result = subprocess.run(
|
||||
cmd,
|
||||
stdout=subprocess.PIPE if quiet else None,
|
||||
stderr=subprocess.STDOUT if quiet else None,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
print(_red(f"❌ {label} failed (exit code {result.returncode}):"))
|
||||
if result.stdout:
|
||||
print(result.stdout.decode(errors="replace"))
|
||||
sys.exit(result.returncode)
|
||||
|
||||
|
||||
# Packages to skip on Windows (require special build steps)
|
||||
WINDOWS_SKIP_PACKAGES = {"open_spiel"}
|
||||
|
||||
|
||||
def _filter_requirements(req: Path, skip: set[str]) -> Path:
|
||||
"""Return a temp copy of a requirements file with certain packages removed."""
|
||||
lines = req.read_text(encoding="utf-8").splitlines(keepends=True)
|
||||
filtered = [
|
||||
line for line in lines
|
||||
if not any(line.strip().lower().startswith(pkg) for pkg in skip)
|
||||
]
|
||||
tmp = tempfile.NamedTemporaryFile(
|
||||
mode="w", suffix=".txt", delete=False, encoding="utf-8",
|
||||
)
|
||||
tmp.writelines(filtered)
|
||||
tmp.close()
|
||||
return Path(tmp.name)
|
||||
|
||||
|
||||
def pip_install(
|
||||
label: str,
|
||||
*args: str,
|
||||
req: Path | None = None,
|
||||
constrain: bool = True,
|
||||
) -> None:
|
||||
"""Build and run a pip install command."""
|
||||
cmd = [sys.executable, "-m", "pip", "install"]
|
||||
cmd.extend(args)
|
||||
if constrain and CONSTRAINTS.is_file():
|
||||
cmd.extend(["-c", str(CONSTRAINTS)])
|
||||
actual_req = req
|
||||
if req is not None and IS_WINDOWS and WINDOWS_SKIP_PACKAGES:
|
||||
actual_req = _filter_requirements(req, WINDOWS_SKIP_PACKAGES)
|
||||
if actual_req is not None:
|
||||
cmd.extend(["-r", str(actual_req)])
|
||||
try:
|
||||
run(label, cmd)
|
||||
finally:
|
||||
# Clean up temp file if we created one
|
||||
if actual_req is not None and actual_req != req:
|
||||
actual_req.unlink(missing_ok=True)
|
||||
|
||||
|
||||
|
||||
def download_file(url: str, dest: Path) -> None:
|
||||
"""Download a file using urllib (no curl dependency)."""
|
||||
urllib.request.urlretrieve(url, dest)
|
||||
|
||||
|
||||
def patch_package_file(package_name: str, relative_path: str, url: str) -> None:
|
||||
"""Download a file from url and overwrite a file inside an installed package."""
|
||||
result = subprocess.run(
|
||||
[sys.executable, "-m", "pip", "show", package_name],
|
||||
capture_output=True, text=True,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
print(_red(f" ⚠️ Could not find package {package_name}, skipping patch"))
|
||||
return
|
||||
|
||||
location = None
|
||||
for line in result.stdout.splitlines():
|
||||
if line.lower().startswith("location:"):
|
||||
location = line.split(":", 1)[1].strip()
|
||||
break
|
||||
|
||||
if not location:
|
||||
print(_red(f" ⚠️ Could not determine location of {package_name}"))
|
||||
return
|
||||
|
||||
dest = Path(location) / relative_path
|
||||
print(_cyan(f" Patching {dest.name} in {package_name}..."))
|
||||
download_file(url, dest)
|
||||
|
||||
|
||||
# ── Main install sequence ─────────────────────────────────────────────
|
||||
|
||||
def install_python_stack() -> int:
|
||||
print(_cyan("── Installing Python stack ──"))
|
||||
|
||||
# 1. Upgrade pip
|
||||
run("Upgrading pip", [sys.executable, "-m", "pip", "install", "--upgrade", "pip"])
|
||||
|
||||
# 2. Core packages: unsloth-zoo + unsloth
|
||||
pip_install(
|
||||
"Installing unsloth-zoo + unsloth",
|
||||
"--no-cache-dir",
|
||||
req=REQ_ROOT / "base.txt",
|
||||
)
|
||||
|
||||
# 3. Extra dependencies
|
||||
pip_install(
|
||||
"Installing additional unsloth dependencies",
|
||||
"--no-cache-dir",
|
||||
req=REQ_ROOT / "extras.txt",
|
||||
)
|
||||
|
||||
# 4. Overrides (torchao, transformers) — force-reinstall
|
||||
pip_install(
|
||||
"Installing torchao + transformers overrides",
|
||||
"--force-reinstall", "--no-cache-dir",
|
||||
req=REQ_ROOT / "overrides.txt",
|
||||
)
|
||||
|
||||
# 5. Triton kernels (no-deps, from source)
|
||||
pip_install(
|
||||
"Installing triton kernels",
|
||||
"--no-deps", "--no-cache-dir",
|
||||
req=REQ_ROOT / "triton-kernels.txt",
|
||||
constrain=False,
|
||||
)
|
||||
|
||||
# 6. Patch: override llama_cpp.py with fix from unsloth-zoo feature/llama-cpp-windows-support branch
|
||||
patch_package_file(
|
||||
"unsloth-zoo",
|
||||
os.path.join("unsloth_zoo", "llama_cpp.py"),
|
||||
"https://raw.githubusercontent.com/unslothai/unsloth-zoo/refs/heads/main/unsloth_zoo/llama_cpp.py",
|
||||
)
|
||||
|
||||
# 7a. Patch: override vision.py with fix from unsloth PR #4091
|
||||
patch_package_file(
|
||||
"unsloth",
|
||||
os.path.join("unsloth", "models", "vision.py"),
|
||||
"https://raw.githubusercontent.com/unslothai/unsloth/80e0108a684c882965a02a8ed851e3473c1145ab/unsloth/models/vision.py",
|
||||
)
|
||||
|
||||
# 7b. Patch : override save.py with fix from feature/llama-cpp-windows-support
|
||||
patch_package_file(
|
||||
"unsloth",
|
||||
os.path.join("unsloth", "save.py"),
|
||||
"https://raw.githubusercontent.com/unslothai/unsloth/refs/heads/main/unsloth/save.py",
|
||||
)
|
||||
|
||||
# 8. Studio dependencies
|
||||
pip_install(
|
||||
"Installing studio dependencies",
|
||||
"--no-cache-dir",
|
||||
req=REQ_ROOT / "studio.txt",
|
||||
)
|
||||
|
||||
# 9. Data-designer dependencies
|
||||
pip_install(
|
||||
"Installing data-designer dependencies",
|
||||
"--no-cache-dir",
|
||||
req=SINGLE_ENV / "data-designer-deps.txt",
|
||||
)
|
||||
|
||||
# 10. Data-designer packages (no-deps to avoid conflicts)
|
||||
pip_install(
|
||||
"Installing data-designer",
|
||||
"--no-cache-dir", "--no-deps",
|
||||
req=SINGLE_ENV / "data-designer.txt",
|
||||
)
|
||||
|
||||
# 11. Patch metadata for single-env compatibility
|
||||
run(
|
||||
"Patching single-env metadata",
|
||||
[sys.executable, str(SINGLE_ENV / "patch_metadata.py")],
|
||||
)
|
||||
|
||||
# 12. Final check
|
||||
run("Running pip check", [sys.executable, "-m", "pip", "check"], quiet=False)
|
||||
|
||||
print(_green("✅ Python dependencies installed"))
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(install_python_stack())
|
||||
2
setup.bat
Normal file
2
setup.bat
Normal file
|
|
@ -0,0 +1,2 @@
|
|||
@echo off
|
||||
powershell -ExecutionPolicy Bypass -File "%~dp0setup.ps1" %*
|
||||
964
setup.ps1
Normal file
964
setup.ps1
Normal file
|
|
@ -0,0 +1,964 @@
|
|||
#Requires -Version 5.1
|
||||
<#
|
||||
.SYNOPSIS
|
||||
Full environment setup for Unsloth Studio on Windows (bundled version).
|
||||
.DESCRIPTION
|
||||
Always installs Node.js if needed. When running from pip install:
|
||||
skips frontend build (already bundled). When running from git repo:
|
||||
full setup including frontend build.
|
||||
Requires an NVIDIA GPU -- CPU-only machines are not supported.
|
||||
.NOTES
|
||||
Usage: powershell -ExecutionPolicy Bypass -File setup.ps1
|
||||
#>
|
||||
|
||||
$ErrorActionPreference = "Stop"
|
||||
$ScriptDir = Split-Path -Parent $MyInvocation.MyCommand.Path
|
||||
$PackageDir = Split-Path -Parent $ScriptDir
|
||||
|
||||
# Detect if running from pip install (no studio/frontend/ dir in repo)
|
||||
$FrontendDir = Join-Path $ScriptDir "studio\frontend"
|
||||
$IsPipInstall = -not (Test-Path $FrontendDir)
|
||||
|
||||
# ─────────────────────────────────────────────
|
||||
# Helper functions
|
||||
# ─────────────────────────────────────────────
|
||||
|
||||
# Reload ALL environment variables from registry.
|
||||
# Picks up changes made by installers (winget, msi, etc.) including
|
||||
# Path, CUDA_PATH, CUDA_PATH_V*, and any other vars they set.
|
||||
function Refresh-Environment {
|
||||
foreach ($level in @('Machine', 'User')) {
|
||||
$vars = [System.Environment]::GetEnvironmentVariables($level)
|
||||
foreach ($key in $vars.Keys) {
|
||||
if ($key -eq 'Path') { continue }
|
||||
Set-Item -Path "Env:$key" -Value $vars[$key] -ErrorAction SilentlyContinue
|
||||
}
|
||||
}
|
||||
$machinePath = [System.Environment]::GetEnvironmentVariable('Path', 'Machine')
|
||||
$userPath = [System.Environment]::GetEnvironmentVariable('Path', 'User')
|
||||
$env:Path = "$machinePath;$userPath"
|
||||
}
|
||||
|
||||
# Find nvcc on PATH, CUDA_PATH, or standard toolkit dirs.
|
||||
# Returns the path to nvcc.exe, or $null if not found.
|
||||
function Find-Nvcc {
|
||||
param([string]$MaxVersion = "")
|
||||
|
||||
# If MaxVersion is set, we need to find a toolkit <= that version.
|
||||
# CUDA toolkits install side-by-side under C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\vX.Y\
|
||||
|
||||
$toolkitBase = 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA'
|
||||
|
||||
if ($MaxVersion -and (Test-Path $toolkitBase)) {
|
||||
$drMajor = [int]$MaxVersion.Split('.')[0]
|
||||
$drMinor = [int]$MaxVersion.Split('.')[1]
|
||||
|
||||
# Get all installed CUDA dirs, sorted descending (highest first)
|
||||
$cudaDirs = Get-ChildItem -Directory $toolkitBase | Where-Object {
|
||||
$_.Name -match '^v(\d+)\.(\d+)'
|
||||
} | Sort-Object { [version]($_.Name -replace '^v','') } -Descending
|
||||
|
||||
foreach ($dir in $cudaDirs) {
|
||||
if ($dir.Name -match '^v(\d+)\.(\d+)') {
|
||||
$tkMajor = [int]$Matches[1]; $tkMinor = [int]$Matches[2]
|
||||
$compatible = ($tkMajor -lt $drMajor) -or ($tkMajor -eq $drMajor -and $tkMinor -le $drMinor)
|
||||
if ($compatible) {
|
||||
$nvcc = Join-Path $dir.FullName 'bin\nvcc.exe'
|
||||
if (Test-Path $nvcc) {
|
||||
return $nvcc
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
# No compatible side-by-side version found
|
||||
return $null
|
||||
}
|
||||
|
||||
# Fallback: no version constraint — pick latest or whatever is available
|
||||
|
||||
# 1. Check nvcc on PATH
|
||||
$cmd = Get-Command nvcc -ErrorAction SilentlyContinue
|
||||
if ($cmd) { return $cmd.Source }
|
||||
|
||||
# 2. Check CUDA_PATH env var
|
||||
$cudaRoot = [Environment]::GetEnvironmentVariable('CUDA_PATH', 'Process')
|
||||
if (-not $cudaRoot) { $cudaRoot = [Environment]::GetEnvironmentVariable('CUDA_PATH', 'Machine') }
|
||||
if (-not $cudaRoot) { $cudaRoot = [Environment]::GetEnvironmentVariable('CUDA_PATH', 'User') }
|
||||
if ($cudaRoot -and (Test-Path (Join-Path $cudaRoot 'bin\nvcc.exe'))) {
|
||||
return (Join-Path $cudaRoot 'bin\nvcc.exe')
|
||||
}
|
||||
|
||||
# 3. Scan standard toolkit directory
|
||||
if (Test-Path $toolkitBase) {
|
||||
$latest = Get-ChildItem -Directory $toolkitBase | Sort-Object Name | Select-Object -Last 1
|
||||
if ($latest -and (Test-Path (Join-Path $latest.FullName 'bin\nvcc.exe'))) {
|
||||
return (Join-Path $latest.FullName 'bin\nvcc.exe')
|
||||
}
|
||||
}
|
||||
|
||||
return $null
|
||||
}
|
||||
|
||||
# Detect CUDA Compute Capability via nvidia-smi.
|
||||
# Returns e.g. "80" for A100 (8.0), "89" for RTX 4090 (8.9), etc.
|
||||
# Returns $null if detection fails.
|
||||
function Get-CudaComputeCapability {
|
||||
$nvSmi = Get-Command nvidia-smi -ErrorAction SilentlyContinue
|
||||
if (-not $nvSmi) { return $null }
|
||||
|
||||
try {
|
||||
$raw = & nvidia-smi --query-gpu=compute_cap --format=csv,noheader 2>$null
|
||||
if ($LASTEXITCODE -ne 0 -or -not $raw) { return $null }
|
||||
|
||||
# nvidia-smi may return multiple GPUs; take the first one
|
||||
$cap = ($raw -split "`n")[0].Trim()
|
||||
if ($cap -match '^(\d+)\.(\d+)$') {
|
||||
$major = $Matches[1]
|
||||
$minor = $Matches[2]
|
||||
return "$major$minor"
|
||||
}
|
||||
} catch { }
|
||||
|
||||
return $null
|
||||
}
|
||||
|
||||
# Detect driver's max CUDA version from nvidia-smi and return the highest
|
||||
# compatible PyTorch CUDA index tag (e.g. "cu128").
|
||||
# PyTorch on Windows ships CPU-only by default from PyPI; CUDA wheels live at
|
||||
# https://download.pytorch.org/whl/<tag>. The tag must not exceed the driver's
|
||||
# capability: e.g. driver "CUDA Version: 12.9" → cu128 (not cu130).
|
||||
function Get-PytorchCudaTag {
|
||||
$nvSmi = Get-Command nvidia-smi -ErrorAction SilentlyContinue
|
||||
if (-not $nvSmi) { return "cu124" }
|
||||
|
||||
try {
|
||||
# 2>&1 | Out-String merges stderr into stdout then converts to a single
|
||||
# string. Plain 2>$null doesn't fully suppress stderr in PS 5.1 —
|
||||
# ErrorRecord objects leak into $output and break the -match.
|
||||
$output = & nvidia-smi 2>&1 | Out-String
|
||||
if ($output -match 'CUDA Version:\s+(\d+)\.(\d+)') {
|
||||
$major = [int]$Matches[1]
|
||||
$minor = [int]$Matches[2]
|
||||
# PyTorch 2.10 offers: cu124, cu126, cu128, cu130
|
||||
if ($major -ge 13) { return "cu130" }
|
||||
if ($major -eq 12 -and $minor -ge 8) { return "cu128" }
|
||||
if ($major -eq 12 -and $minor -ge 6) { return "cu126" }
|
||||
return "cu124"
|
||||
}
|
||||
} catch { }
|
||||
|
||||
return "cu124"
|
||||
}
|
||||
|
||||
# Find Visual Studio Build Tools for cmake -G flag.
|
||||
# Strategy: (1) vswhere, (2) scan filesystem (handles broken vswhere registration).
|
||||
# Returns @{ Generator = "Visual Studio 17 2022"; InstallPath = "C:\..."; Source = "..." } or $null.
|
||||
function Find-VsBuildTools {
|
||||
$map = @{ '2022' = '17'; '2019' = '16'; '2017' = '15' }
|
||||
|
||||
# --- Try vswhere first (works when VS is properly registered) ---
|
||||
$vsw = "${env:ProgramFiles(x86)}\Microsoft Visual Studio\Installer\vswhere.exe"
|
||||
if (Test-Path $vsw) {
|
||||
$info = & $vsw -latest -requires Microsoft.VisualStudio.Component.VC.Tools.x86.x64 -property catalog_productLineVersion 2>$null
|
||||
$path = & $vsw -latest -requires Microsoft.VisualStudio.Component.VC.Tools.x86.x64 -property installationPath 2>$null
|
||||
if ($info -and $path) {
|
||||
$y = $info.Trim()
|
||||
$n = $map[$y]
|
||||
if ($n) {
|
||||
return @{ Generator = "Visual Studio $n $y"; InstallPath = $path.Trim(); Source = 'vswhere' }
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
# --- Scan filesystem (handles broken vswhere registration after winget cycles) ---
|
||||
$roots = @($env:ProgramFiles, ${env:ProgramFiles(x86)})
|
||||
$editions = @('BuildTools', 'Community', 'Professional', 'Enterprise')
|
||||
$years = @('2022', '2019', '2017')
|
||||
|
||||
foreach ($y in $years) {
|
||||
foreach ($r in $roots) {
|
||||
foreach ($ed in $editions) {
|
||||
$candidate = Join-Path $r "Microsoft Visual Studio\$y\$ed"
|
||||
if (Test-Path $candidate) {
|
||||
$vcDir = Join-Path $candidate "VC\Tools\MSVC"
|
||||
if (Test-Path $vcDir) {
|
||||
$cl = Get-ChildItem -Path $vcDir -Filter "cl.exe" -Recurse -ErrorAction SilentlyContinue | Select-Object -First 1
|
||||
if ($cl) {
|
||||
$n = $map[$y]
|
||||
if ($n) {
|
||||
return @{ Generator = "Visual Studio $n $y"; InstallPath = $candidate; Source = "filesystem ($ed)"; ClExe = $cl.FullName }
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return $null
|
||||
}
|
||||
|
||||
# ─────────────────────────────────────────────
|
||||
# Banner
|
||||
# ─────────────────────────────────────────────
|
||||
Write-Host "+==============================================+" -ForegroundColor Green
|
||||
Write-Host "| Unsloth Studio Setup (Windows) |" -ForegroundColor Green
|
||||
Write-Host "+==============================================+" -ForegroundColor Green
|
||||
|
||||
# ==========================================================================
|
||||
# PHASE 1: System-level prerequisites (winget installs, env vars)
|
||||
# All heavy system tool installs happen here BEFORE touching Python.
|
||||
# ==========================================================================
|
||||
|
||||
# ============================================
|
||||
# 1a. GPU requirement check
|
||||
# ============================================
|
||||
$HasNvidiaSmi = $null -ne (Get-Command nvidia-smi -ErrorAction SilentlyContinue)
|
||||
if (-not $HasNvidiaSmi) {
|
||||
Write-Host ""
|
||||
Write-Host "[ERROR] Unsloth Studio requires an NVIDIA GPU." -ForegroundColor Red
|
||||
Write-Host " CPU-only machines are not supported." -ForegroundColor Red
|
||||
Write-Host ""
|
||||
Write-Host " If you have an NVIDIA GPU, ensure the driver is installed:" -ForegroundColor Yellow
|
||||
Write-Host " https://www.nvidia.com/Download/index.aspx" -ForegroundColor Yellow
|
||||
exit 1
|
||||
}
|
||||
Write-Host "[OK] NVIDIA GPU detected" -ForegroundColor Green
|
||||
|
||||
# ============================================
|
||||
# 1a.5. Windows Long Paths (required for deep node_modules / Python paths)
|
||||
# ============================================
|
||||
$LongPathsEnabled = $false
|
||||
try {
|
||||
$regVal = Get-ItemProperty -Path "HKLM:\SYSTEM\CurrentControlSet\Control\FileSystem" -Name "LongPathsEnabled" -ErrorAction SilentlyContinue
|
||||
if ($regVal -and $regVal.LongPathsEnabled -eq 1) {
|
||||
$LongPathsEnabled = $true
|
||||
}
|
||||
} catch {}
|
||||
|
||||
if ($LongPathsEnabled) {
|
||||
Write-Host "[OK] Windows Long Paths enabled" -ForegroundColor Green
|
||||
} else {
|
||||
Write-Host "Windows Long Paths not enabled (required for Triton compilation and deep dependency paths)." -ForegroundColor Yellow
|
||||
Write-Host " Requesting admin access to fix..." -ForegroundColor Yellow
|
||||
try {
|
||||
# Spawn an elevated process to set the registry key (triggers UAC prompt)
|
||||
$proc = Start-Process -FilePath "reg.exe" `
|
||||
-ArgumentList 'add "HKLM\SYSTEM\CurrentControlSet\Control\FileSystem" /v LongPathsEnabled /t REG_DWORD /d 1 /f' `
|
||||
-Verb RunAs -Wait -PassThru -ErrorAction Stop
|
||||
if ($proc.ExitCode -eq 0) {
|
||||
$LongPathsEnabled = $true
|
||||
Write-Host "[OK] Windows Long Paths enabled (via UAC)" -ForegroundColor Green
|
||||
} else {
|
||||
Write-Host "[WARN] Failed to enable Long Paths (exit code: $($proc.ExitCode))" -ForegroundColor Yellow
|
||||
}
|
||||
} catch {
|
||||
Write-Host "[WARN] Could not enable Long Paths (UAC was declined or not available)" -ForegroundColor Yellow
|
||||
Write-Host " Run this manually in an Admin terminal:" -ForegroundColor Yellow
|
||||
Write-Host ' reg add "HKLM\SYSTEM\CurrentControlSet\Control\FileSystem" /v LongPathsEnabled /t REG_DWORD /d 1 /f' -ForegroundColor Cyan
|
||||
}
|
||||
}
|
||||
|
||||
# ============================================
|
||||
# 1b. Git (required by pip for git+https:// deps and by npm)
|
||||
# ============================================
|
||||
$HasGit = $null -ne (Get-Command git -ErrorAction SilentlyContinue)
|
||||
if (-not $HasGit) {
|
||||
Write-Host "Git not found -- installing via winget..." -ForegroundColor Yellow
|
||||
$HasWinget = $null -ne (Get-Command winget -ErrorAction SilentlyContinue)
|
||||
if ($HasWinget) {
|
||||
try {
|
||||
winget install Git.Git --source winget --accept-package-agreements --accept-source-agreements 2>&1 | Out-Null
|
||||
Refresh-Environment
|
||||
$HasGit = $null -ne (Get-Command git -ErrorAction SilentlyContinue)
|
||||
} catch { }
|
||||
}
|
||||
if (-not $HasGit) {
|
||||
Write-Host "[ERROR] Git is required but could not be installed automatically." -ForegroundColor Red
|
||||
Write-Host " Install Git from https://git-scm.com/download/win and re-run." -ForegroundColor Red
|
||||
exit 1
|
||||
}
|
||||
Write-Host "[OK] Git installed: $(git --version)" -ForegroundColor Green
|
||||
} else {
|
||||
Write-Host "[OK] Git found: $(git --version)" -ForegroundColor Green
|
||||
}
|
||||
|
||||
# ============================================
|
||||
# 1c. CMake (required for llama.cpp build)
|
||||
# ============================================
|
||||
$HasCmake = $null -ne (Get-Command cmake -ErrorAction SilentlyContinue)
|
||||
if (-not $HasCmake) {
|
||||
Write-Host "CMake not found -- installing via winget..." -ForegroundColor Yellow
|
||||
$HasWinget = $null -ne (Get-Command winget -ErrorAction SilentlyContinue)
|
||||
if ($HasWinget) {
|
||||
try {
|
||||
winget install Kitware.CMake --source winget --accept-package-agreements --accept-source-agreements 2>&1 | Out-Null
|
||||
Refresh-Environment
|
||||
$HasCmake = $null -ne (Get-Command cmake -ErrorAction SilentlyContinue)
|
||||
} catch { }
|
||||
}
|
||||
if ($HasCmake) {
|
||||
Write-Host "[OK] CMake installed" -ForegroundColor Green
|
||||
} else {
|
||||
Write-Host "[ERROR] CMake is required but could not be installed." -ForegroundColor Red
|
||||
Write-Host " Install CMake from https://cmake.org/download/ and re-run." -ForegroundColor Red
|
||||
exit 1
|
||||
}
|
||||
} else {
|
||||
Write-Host "[OK] CMake found: $(cmake --version | Select-Object -First 1)" -ForegroundColor Green
|
||||
}
|
||||
|
||||
# ============================================
|
||||
# 1d. Visual Studio Build Tools (C++ compiler for llama.cpp)
|
||||
# ============================================
|
||||
$CmakeGenerator = $null
|
||||
$VsInstallPath = $null
|
||||
$vsResult = Find-VsBuildTools
|
||||
|
||||
if (-not $vsResult) {
|
||||
Write-Host "Visual Studio Build Tools not found -- installing via winget..." -ForegroundColor Yellow
|
||||
Write-Host " (This is a one-time install, may take several minutes)" -ForegroundColor Gray
|
||||
$HasWinget = $null -ne (Get-Command winget -ErrorAction SilentlyContinue)
|
||||
if ($HasWinget) {
|
||||
$prevEAPTemp = $ErrorActionPreference
|
||||
$ErrorActionPreference = "Continue"
|
||||
winget install Microsoft.VisualStudio.2022.BuildTools --source winget --accept-package-agreements --accept-source-agreements --override "--add Microsoft.VisualStudio.Workload.VCTools --includeRecommended --passive --wait"
|
||||
$ErrorActionPreference = $prevEAPTemp
|
||||
# Re-scan after install (don't trust vswhere catalog)
|
||||
$vsResult = Find-VsBuildTools
|
||||
}
|
||||
}
|
||||
|
||||
if ($vsResult) {
|
||||
$CmakeGenerator = $vsResult.Generator
|
||||
$VsInstallPath = $vsResult.InstallPath
|
||||
Write-Host "[OK] $CmakeGenerator detected via $($vsResult.Source)" -ForegroundColor Green
|
||||
if ($vsResult.ClExe) { Write-Host " cl.exe: $($vsResult.ClExe)" -ForegroundColor Gray }
|
||||
} else {
|
||||
Write-Host "[ERROR] Visual Studio Build Tools could not be found or installed." -ForegroundColor Red
|
||||
Write-Host " Manual install:" -ForegroundColor Red
|
||||
Write-Host ' 1. winget install Microsoft.VisualStudio.2022.BuildTools --source winget' -ForegroundColor Yellow
|
||||
Write-Host ' 2. Open Visual Studio Installer -> Modify -> check "Desktop development with C++"' -ForegroundColor Yellow
|
||||
exit 1
|
||||
}
|
||||
|
||||
# ============================================
|
||||
# 1e. CUDA Toolkit (nvcc for llama.cpp build + env vars)
|
||||
# ============================================
|
||||
# IMPORTANT: The CUDA Toolkit version must be <= the max CUDA version the
|
||||
# NVIDIA driver supports. nvidia-smi reports this as "CUDA Version: X.Y".
|
||||
# If we install a toolkit newer than the driver supports, llama-server will
|
||||
# fail at runtime with "ggml_cuda_init: failed to initialize CUDA: (null)".
|
||||
|
||||
# -- Detect max CUDA version the driver supports --
|
||||
$DriverMaxCuda = $null
|
||||
try {
|
||||
$smiOut = nvidia-smi 2>&1 | Out-String
|
||||
if ($smiOut -match "CUDA Version:\s+([\d]+)\.([\d]+)") {
|
||||
$DriverMaxCuda = "$($Matches[1]).$($Matches[2])"
|
||||
Write-Host " Driver supports up to CUDA $DriverMaxCuda" -ForegroundColor Gray
|
||||
}
|
||||
} catch {}
|
||||
|
||||
# -- Find a toolkit that's compatible with the driver --
|
||||
$IncompatibleToolkit = $null
|
||||
if ($DriverMaxCuda) {
|
||||
$NvccPath = Find-Nvcc -MaxVersion $DriverMaxCuda
|
||||
if ($NvccPath) {
|
||||
Write-Host " [OK] Found compatible CUDA Toolkit (nvcc: $NvccPath)" -ForegroundColor Green
|
||||
} else {
|
||||
# Check if there's an incompatible (too new) toolkit installed
|
||||
$AnyNvcc = Find-Nvcc
|
||||
if ($AnyNvcc) {
|
||||
$NvccOut = & $AnyNvcc --version 2>&1 | Out-String
|
||||
if ($NvccOut -match "release\s+([\d]+\.[\d]+)") {
|
||||
$IncompatibleToolkit = $Matches[1]
|
||||
}
|
||||
}
|
||||
}
|
||||
} else {
|
||||
$NvccPath = Find-Nvcc
|
||||
}
|
||||
|
||||
# -- If incompatible toolkit is blocking, tell user to uninstall it --
|
||||
if (-not $NvccPath -and $IncompatibleToolkit) {
|
||||
Write-Host "" -ForegroundColor Red
|
||||
Write-Host "========================================================================" -ForegroundColor Red
|
||||
Write-Host "[ERROR] CUDA Toolkit $IncompatibleToolkit is installed but INCOMPATIBLE" -ForegroundColor Red
|
||||
Write-Host " with your NVIDIA driver (which supports up to CUDA $DriverMaxCuda)." -ForegroundColor Red
|
||||
Write-Host "" -ForegroundColor Red
|
||||
Write-Host " This will cause 'failed to initialize CUDA' errors at runtime." -ForegroundColor Red
|
||||
Write-Host "" -ForegroundColor Red
|
||||
Write-Host " To fix:" -ForegroundColor Yellow
|
||||
Write-Host " 1. Open Control Panel -> Programs -> Uninstall a program" -ForegroundColor Yellow
|
||||
Write-Host " 2. Uninstall 'NVIDIA CUDA Toolkit $IncompatibleToolkit'" -ForegroundColor Yellow
|
||||
Write-Host " 3. Re-run setup.bat (it will install CUDA $DriverMaxCuda automatically)" -ForegroundColor Yellow
|
||||
Write-Host "" -ForegroundColor Yellow
|
||||
Write-Host " Alternatively, update your NVIDIA driver to one that supports CUDA $IncompatibleToolkit." -ForegroundColor Gray
|
||||
Write-Host "========================================================================" -ForegroundColor Red
|
||||
exit 1
|
||||
}
|
||||
|
||||
# -- No toolkit at all: install via winget --
|
||||
if (-not $NvccPath) {
|
||||
Write-Host "CUDA toolkit (nvcc) not found -- installing via winget..." -ForegroundColor Yellow
|
||||
$HasWinget = $null -ne (Get-Command winget -ErrorAction SilentlyContinue)
|
||||
if ($HasWinget) {
|
||||
if ($DriverMaxCuda) {
|
||||
# Try descending compatible versions
|
||||
$drMajor = [int]$DriverMaxCuda.Split('.')[0]
|
||||
$drMinor = [int]$DriverMaxCuda.Split('.')[1]
|
||||
for ($m = $drMinor; $m -ge 0; $m--) {
|
||||
$ver = "$drMajor.$m"
|
||||
Write-Host " Trying CUDA Toolkit $ver via winget..." -ForegroundColor Cyan
|
||||
$prevEAPCuda = $ErrorActionPreference
|
||||
$ErrorActionPreference = "Continue"
|
||||
winget install --id=Nvidia.CUDA --version=$ver -e --source winget --accept-package-agreements --accept-source-agreements 2>&1 | Out-Null
|
||||
$ErrorActionPreference = $prevEAPCuda
|
||||
Refresh-Environment
|
||||
$NvccPath = Find-Nvcc -MaxVersion $DriverMaxCuda
|
||||
if ($NvccPath) {
|
||||
Write-Host " [OK] CUDA Toolkit $ver installed (nvcc: $NvccPath)" -ForegroundColor Green
|
||||
break
|
||||
}
|
||||
}
|
||||
} else {
|
||||
Write-Host " Installing CUDA Toolkit (latest) via winget..." -ForegroundColor Cyan
|
||||
winget install --id=Nvidia.CUDA -e --source winget --accept-package-agreements --accept-source-agreements
|
||||
Refresh-Environment
|
||||
$NvccPath = Find-Nvcc
|
||||
if ($NvccPath) {
|
||||
Write-Host " [OK] CUDA Toolkit installed (nvcc: $NvccPath)" -ForegroundColor Green
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if (-not $NvccPath) {
|
||||
Write-Host "[ERROR] CUDA Toolkit (nvcc) is required but could not be found or installed." -ForegroundColor Red
|
||||
if ($DriverMaxCuda) {
|
||||
Write-Host " Install CUDA Toolkit $DriverMaxCuda from https://developer.nvidia.com/cuda-toolkit-archive" -ForegroundColor Yellow
|
||||
} else {
|
||||
Write-Host " Install CUDA Toolkit from https://developer.nvidia.com/cuda-downloads" -ForegroundColor Yellow
|
||||
}
|
||||
exit 1
|
||||
}
|
||||
|
||||
# -- Set CUDA env vars so cmake AND MSBuild can find the toolkit --
|
||||
$CudaToolkitRoot = Split-Path (Split-Path $NvccPath -Parent) -Parent
|
||||
# CUDA_PATH: used by cmake's find_package(CUDAToolkit)
|
||||
[Environment]::SetEnvironmentVariable('CUDA_PATH', $CudaToolkitRoot, 'Process')
|
||||
# CudaToolkitDir: the MSBuild property that CUDA .targets checks directly
|
||||
# Trailing backslash required -- the .targets file appends subpaths to it
|
||||
[Environment]::SetEnvironmentVariable('CudaToolkitDir', "$CudaToolkitRoot\", 'Process')
|
||||
# Always persist CUDA_PATH to User registry so the compatible toolkit is used
|
||||
# in future sessions (overwrites any existing value pointing to a newer, incompatible version)
|
||||
[Environment]::SetEnvironmentVariable('CUDA_PATH', $CudaToolkitRoot, 'User')
|
||||
Write-Host " Persisted CUDA_PATH=$CudaToolkitRoot to user environment" -ForegroundColor Gray
|
||||
# Ensure nvcc's bin dir is on PATH for this process
|
||||
$nvccBinDir = Split-Path $NvccPath -Parent
|
||||
if ($env:PATH -notlike "*$nvccBinDir*") {
|
||||
[Environment]::SetEnvironmentVariable('PATH', "$nvccBinDir;$env:PATH", 'Process')
|
||||
}
|
||||
# Persist nvcc bin dir to User PATH so it works in new terminals
|
||||
$userPath = [Environment]::GetEnvironmentVariable('Path', 'User')
|
||||
if (-not $userPath -or $userPath -notlike "*$nvccBinDir*") {
|
||||
if ($userPath) {
|
||||
[Environment]::SetEnvironmentVariable('Path', "$nvccBinDir;$userPath", 'User')
|
||||
} else {
|
||||
[Environment]::SetEnvironmentVariable('Path', "$nvccBinDir", 'User')
|
||||
}
|
||||
Write-Host " Persisted CUDA bin dir to user PATH" -ForegroundColor Gray
|
||||
}
|
||||
|
||||
Write-Host "[OK] CUDA Toolkit: $NvccPath" -ForegroundColor Green
|
||||
Write-Host " CUDA_PATH = $CudaToolkitRoot" -ForegroundColor Gray
|
||||
Write-Host " CudaToolkitDir = $CudaToolkitRoot\" -ForegroundColor Gray
|
||||
|
||||
# Detect compute capability (used later for llama.cpp cmake)
|
||||
$CudaArch = Get-CudaComputeCapability
|
||||
if ($CudaArch) {
|
||||
Write-Host " Compute Capability = $($CudaArch.Insert($CudaArch.Length-1, '.')) (sm_$CudaArch)" -ForegroundColor Gray
|
||||
} else {
|
||||
Write-Host " [WARN] Could not detect compute capability -- cmake will use defaults" -ForegroundColor Yellow
|
||||
}
|
||||
|
||||
# ============================================
|
||||
# 1f. Node.js / npm (always -- needed regardless of install method)
|
||||
# ============================================
|
||||
# setup.sh installs Node LTS (v22) via nvm. We enforce the same range here:
|
||||
# Node >= 20, npm >= 11.
|
||||
$NeedNode = $true
|
||||
try {
|
||||
$NodeVersion = (node -v 2>$null)
|
||||
$NpmVersion = (npm -v 2>$null)
|
||||
if ($NodeVersion -and $NpmVersion) {
|
||||
$NodeMajor = [int]($NodeVersion -replace 'v','').Split('.')[0]
|
||||
$NpmMajor = [int]$NpmVersion.Split('.')[0]
|
||||
|
||||
if ($NodeMajor -ge 20 -and $NpmMajor -ge 11) {
|
||||
Write-Host "[OK] Node $NodeVersion and npm $NpmVersion already meet requirements." -ForegroundColor Green
|
||||
$NeedNode = $false
|
||||
} else {
|
||||
Write-Host "[WARN] Node $NodeVersion / npm $NpmVersion too old." -ForegroundColor Yellow
|
||||
}
|
||||
}
|
||||
} catch {
|
||||
Write-Host "[WARN] Node/npm not found." -ForegroundColor Yellow
|
||||
}
|
||||
|
||||
if ($NeedNode) {
|
||||
Write-Host "Installing Node.js LTS via winget..." -ForegroundColor Cyan
|
||||
try {
|
||||
winget install OpenJS.NodeJS.LTS --source winget --accept-package-agreements --accept-source-agreements
|
||||
Refresh-Environment
|
||||
} catch {
|
||||
Write-Host "[ERROR] Could not install Node.js automatically." -ForegroundColor Red
|
||||
Write-Host "Please install Node.js >= 20 from https://nodejs.org/" -ForegroundColor Red
|
||||
exit 1
|
||||
}
|
||||
}
|
||||
|
||||
Write-Host "[OK] Node $(node -v) | npm $(npm -v)" -ForegroundColor Green
|
||||
|
||||
# ============================================
|
||||
# 1g. Python (>= 3.11 and < 3.14, matching setup.sh)
|
||||
# ============================================
|
||||
$HasPython = $null -ne (Get-Command python -ErrorAction SilentlyContinue)
|
||||
$PythonOk = $false
|
||||
|
||||
if ($HasPython) {
|
||||
$PyVer = python --version 2>&1
|
||||
if ($PyVer -match "(\d+)\.(\d+)") {
|
||||
$PyMajor = [int]$Matches[1]; $PyMinor = [int]$Matches[2]
|
||||
if ($PyMajor -eq 3 -and $PyMinor -ge 11 -and $PyMinor -lt 14) {
|
||||
Write-Host "[OK] Python $PyVer" -ForegroundColor Green
|
||||
$PythonOk = $true
|
||||
} else {
|
||||
Write-Host "[ERROR] Python $PyVer is outside supported range (need >= 3.11 and < 3.14)." -ForegroundColor Red
|
||||
Write-Host " Install Python 3.12 from https://python.org/downloads/" -ForegroundColor Yellow
|
||||
exit 1
|
||||
}
|
||||
}
|
||||
} else {
|
||||
# No Python at all -- install 3.12
|
||||
Write-Host "Python not found -- installing Python 3.12 via winget..." -ForegroundColor Yellow
|
||||
$HasWinget = $null -ne (Get-Command winget -ErrorAction SilentlyContinue)
|
||||
if ($HasWinget) {
|
||||
winget install -e --id Python.Python.3.12 --source winget --accept-package-agreements --accept-source-agreements
|
||||
Refresh-Environment
|
||||
}
|
||||
$HasPython = $null -ne (Get-Command python -ErrorAction SilentlyContinue)
|
||||
if (-not $HasPython) {
|
||||
Write-Host "[ERROR] Python could not be installed automatically." -ForegroundColor Red
|
||||
Write-Host " Install Python 3.12 from https://python.org/downloads/" -ForegroundColor Yellow
|
||||
exit 1
|
||||
}
|
||||
Write-Host "[OK] Python $(python --version)" -ForegroundColor Green
|
||||
$PythonOk = $true
|
||||
}
|
||||
|
||||
Write-Host ""
|
||||
Write-Host "--- System prerequisites ready ---" -ForegroundColor Green
|
||||
Write-Host ""
|
||||
|
||||
# ==========================================================================
|
||||
# PHASE 2: Frontend build (skip if pip-installed -- already bundled)
|
||||
# ==========================================================================
|
||||
if ($IsPipInstall) {
|
||||
Write-Host "[OK] Running from pip install - frontend already bundled, skipping build" -ForegroundColor Green
|
||||
} else {
|
||||
Write-Host ""
|
||||
Write-Host "Building frontend..." -ForegroundColor Cyan
|
||||
# npm writes warnings to stderr; lower ErrorActionPreference so PS doesn't
|
||||
# treat them as terminating errors (same pattern as the pip section below).
|
||||
$prevEAP_npm = $ErrorActionPreference
|
||||
$ErrorActionPreference = "Continue"
|
||||
Push-Location $FrontendDir
|
||||
# Remove stale node_modules and package-lock.json to avoid version conflicts
|
||||
if (Test-Path "node_modules") { Remove-Item -Recurse -Force "node_modules" }
|
||||
if (Test-Path "package-lock.json") { Remove-Item -Force "package-lock.json" }
|
||||
npm install 2>&1 | Out-Null
|
||||
if ($LASTEXITCODE -ne 0) {
|
||||
Pop-Location
|
||||
$ErrorActionPreference = $prevEAP_npm
|
||||
Write-Host "[ERROR] npm install failed (exit code $LASTEXITCODE)" -ForegroundColor Red
|
||||
Write-Host " Try running 'npm install' manually in studio/frontend/ to see errors" -ForegroundColor Yellow
|
||||
exit 1
|
||||
}
|
||||
npm run build 2>&1 | Out-Null
|
||||
if ($LASTEXITCODE -ne 0) {
|
||||
Pop-Location
|
||||
$ErrorActionPreference = $prevEAP_npm
|
||||
Write-Host "[ERROR] npm run build failed (exit code $LASTEXITCODE)" -ForegroundColor Red
|
||||
exit 1
|
||||
}
|
||||
Pop-Location
|
||||
$ErrorActionPreference = $prevEAP_npm
|
||||
Write-Host "[OK] Frontend built to studio/frontend/dist" -ForegroundColor Green
|
||||
}
|
||||
|
||||
# ==========================================================================
|
||||
# PHASE 3: Python environment + dependencies
|
||||
# ==========================================================================
|
||||
Write-Host ""
|
||||
Write-Host "Setting up Python environment..." -ForegroundColor Cyan
|
||||
|
||||
# Find Python
|
||||
$PythonCmd = $null
|
||||
foreach ($candidate in @("python3.12", "python3.11", "python3.10", "python3.9", "python3", "python")) {
|
||||
try {
|
||||
$ver = & $candidate --version 2>&1
|
||||
if ($ver -match 'Python 3\.(\d+)') {
|
||||
$minor = [int]$Matches[1]
|
||||
if ($minor -le 12) {
|
||||
$PythonCmd = $candidate
|
||||
break
|
||||
}
|
||||
}
|
||||
} catch { }
|
||||
}
|
||||
|
||||
if (-not $PythonCmd) {
|
||||
Write-Host "[ERROR] No Python <= 3.12 found." -ForegroundColor Red
|
||||
exit 1
|
||||
}
|
||||
|
||||
Write-Host "[OK] Using $PythonCmd ($(& $PythonCmd --version 2>&1))" -ForegroundColor Green
|
||||
|
||||
# Always create a .venv for isolation -- even for pip installs.
|
||||
# Created in the current working directory (where user ran the command).
|
||||
$VenvDir = Join-Path (Get-Location) ".venv"
|
||||
if (-not (Test-Path $VenvDir)) {
|
||||
Write-Host " Creating virtual environment at $VenvDir..." -ForegroundColor Cyan
|
||||
& $PythonCmd -m venv $VenvDir
|
||||
} else {
|
||||
Write-Host " Reusing existing virtual environment at $VenvDir" -ForegroundColor Green
|
||||
}
|
||||
|
||||
# pip and python write to stderr even on success (progress bars, warnings).
|
||||
# With $ErrorActionPreference = "Stop" (set at top of script), PS 5.1
|
||||
# converts stderr lines into terminating ErrorRecords, breaking output.
|
||||
# Lower to "Continue" for the pip/python section.
|
||||
$prevEAP = $ErrorActionPreference
|
||||
$ErrorActionPreference = "Continue"
|
||||
|
||||
$ActivateScript = Join-Path $VenvDir "Scripts\Activate.ps1"
|
||||
. $ActivateScript
|
||||
pip install --upgrade pip 2>&1 | Out-Null
|
||||
|
||||
# if (-not $IsPipInstall) {
|
||||
# # Running from repo: copy requirements and do editable install
|
||||
# $RepoRoot = (Resolve-Path (Join-Path $ScriptDir "..\..")).Path
|
||||
# $ReqsSrc = Join-Path $RepoRoot "backend\requirements"
|
||||
# $ReqsDst = Join-Path $PackageDir "requirements"
|
||||
# if (-not (Test-Path $ReqsDst)) { New-Item -ItemType Directory -Path $ReqsDst | Out-Null }
|
||||
# Copy-Item (Join-Path $ReqsSrc "*.txt") $ReqsDst -Force
|
||||
|
||||
# Write-Host " Installing CLI entry point..." -ForegroundColor Cyan
|
||||
# pip install -e $RepoRoot 2>&1 | Out-Null
|
||||
# } else {
|
||||
# # Running from pip install: the package is in system Python but not in
|
||||
# # the fresh .venv. Install it so run_install() can find its modules
|
||||
# # and bundled requirements files.
|
||||
# Write-Host " Installing package into venv..." -ForegroundColor Cyan
|
||||
# pip install unsloth-roland-test 2>&1 | Out-Null
|
||||
# }
|
||||
|
||||
# Pre-install PyTorch with CUDA support.
|
||||
# On Windows, the default PyPI torch wheel is CPU-only.
|
||||
# We need PyTorch's CUDA index to get GPU-enabled wheels.
|
||||
# PyTorch bundles its own CUDA runtime, so this works regardless
|
||||
# of whether the CUDA Toolkit is installed yet.
|
||||
# The CUDA tag is chosen based on the driver's max supported CUDA version.
|
||||
|
||||
# Windows MAX_PATH (260 chars) causes Triton kernel compilation to fail because
|
||||
# the auto-generated filenames are extremely long. Use a short cache directory.
|
||||
$TorchCacheDir = "C:\tc"
|
||||
if (-not (Test-Path $TorchCacheDir)) { New-Item -ItemType Directory -Path $TorchCacheDir -Force | Out-Null }
|
||||
$env:TORCHINDUCTOR_CACHE_DIR = $TorchCacheDir
|
||||
[Environment]::SetEnvironmentVariable('TORCHINDUCTOR_CACHE_DIR', $TorchCacheDir, 'User')
|
||||
Write-Host "[OK] TORCHINDUCTOR_CACHE_DIR set to $TorchCacheDir (avoids MAX_PATH issues)" -ForegroundColor Green
|
||||
|
||||
$CuTag = Get-PytorchCudaTag
|
||||
Write-Host " Installing PyTorch with CUDA support ($CuTag)..." -ForegroundColor Cyan
|
||||
pip install torch torchvision torchaudio --index-url "https://download.pytorch.org/whl/$CuTag" 2>&1 | Out-Null
|
||||
|
||||
# Ordered heavy dependency installation — shared cross-platform script
|
||||
Write-Host " Running ordered dependency installation..." -ForegroundColor Cyan
|
||||
python "$PSScriptRoot\install_python_stack.py"
|
||||
# Restore ErrorActionPreference after pip/python work
|
||||
$ErrorActionPreference = $prevEAP
|
||||
|
||||
# ==========================================================================
|
||||
# PHASE 3.5: Install OpenSSL dev (for HTTPS support in llama-server)
|
||||
# ==========================================================================
|
||||
# llama-server needs OpenSSL to download models from HuggingFace via -hf.
|
||||
# ShiningLight.OpenSSL.Dev includes headers + libs that cmake can find.
|
||||
$OpenSslAvailable = $false
|
||||
|
||||
# Check if OpenSSL dev is already installed (look for include dir)
|
||||
$OpenSslRoots = @(
|
||||
'C:\Program Files\OpenSSL-Win64',
|
||||
'C:\Program Files\OpenSSL',
|
||||
'C:\OpenSSL-Win64'
|
||||
)
|
||||
$OpenSslRoot = $null
|
||||
foreach ($root in $OpenSslRoots) {
|
||||
if (Test-Path (Join-Path $root 'include\openssl\ssl.h')) {
|
||||
$OpenSslRoot = $root
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
if ($OpenSslRoot) {
|
||||
$OpenSslAvailable = $true
|
||||
Write-Host "[OK] OpenSSL dev found at $OpenSslRoot" -ForegroundColor Green
|
||||
} else {
|
||||
Write-Host ""
|
||||
Write-Host "Installing OpenSSL dev (for HTTPS in llama-server)..." -ForegroundColor Cyan
|
||||
$HasWinget = $null -ne (Get-Command winget -ErrorAction SilentlyContinue)
|
||||
if ($HasWinget) {
|
||||
winget install -e --id ShiningLight.OpenSSL.Dev --accept-package-agreements --accept-source-agreements
|
||||
# Re-check after install
|
||||
foreach ($root in $OpenSslRoots) {
|
||||
if (Test-Path (Join-Path $root 'include\openssl\ssl.h')) {
|
||||
$OpenSslRoot = $root
|
||||
$OpenSslAvailable = $true
|
||||
Write-Host "[OK] OpenSSL dev installed at $OpenSslRoot" -ForegroundColor Green
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
if (-not $OpenSslAvailable) {
|
||||
Write-Host "[WARN] OpenSSL dev not available -- llama-server will be built without HTTPS" -ForegroundColor Yellow
|
||||
}
|
||||
}
|
||||
|
||||
# ==========================================================================
|
||||
# PHASE 4: Build llama.cpp with CUDA for GGUF inference + export
|
||||
# ==========================================================================
|
||||
# Builds at ~/.unsloth/llama.cpp — a single shared location under the user's
|
||||
# home directory. This is used by both the inference server and the GGUF
|
||||
# export pipeline (unsloth-zoo).
|
||||
# We build:
|
||||
# - llama-server: for GGUF model inference (with HTTPS if OpenSSL available)
|
||||
# - llama-quantize: for GGUF export quantization
|
||||
# Prerequisites (git, cmake, VS Build Tools, CUDA Toolkit) already installed in Phase 1.
|
||||
$UnslothHome = Join-Path $env:USERPROFILE ".unsloth"
|
||||
if (-not (Test-Path $UnslothHome)) { New-Item -ItemType Directory -Force $UnslothHome | Out-Null }
|
||||
$LlamaCppDir = Join-Path $UnslothHome "llama.cpp"
|
||||
$BuildDir = Join-Path $LlamaCppDir "build"
|
||||
$LlamaServerBin = Join-Path $BuildDir "bin\Release\llama-server.exe"
|
||||
|
||||
if (Test-Path $LlamaServerBin) {
|
||||
Write-Host ""
|
||||
Write-Host "[OK] llama-server already exists at $LlamaServerBin" -ForegroundColor Green
|
||||
} else {
|
||||
Write-Host ""
|
||||
Write-Host "Building llama.cpp with CUDA support..." -ForegroundColor Cyan
|
||||
Write-Host " This typically takes 5-10 minutes on first build." -ForegroundColor Gray
|
||||
Write-Host ""
|
||||
|
||||
# Start total build timer
|
||||
$totalSw = [System.Diagnostics.Stopwatch]::StartNew()
|
||||
|
||||
# Native commands (git, cmake) write to stderr even on success.
|
||||
# With $ErrorActionPreference = "Stop" (set at top of script), PS 5.1
|
||||
# converts stderr lines into terminating ErrorRecords, breaking output.
|
||||
# Lower to "Continue" for the build section.
|
||||
$prevEAP = $ErrorActionPreference
|
||||
$ErrorActionPreference = "Continue"
|
||||
|
||||
$BuildOk = $true
|
||||
$FailedStep = ""
|
||||
|
||||
# -- Step A: Clone or pull llama.cpp --
|
||||
|
||||
if (Test-Path (Join-Path $LlamaCppDir ".git")) {
|
||||
Write-Host " llama.cpp repo already cloned, pulling latest..." -ForegroundColor Gray
|
||||
git -C $LlamaCppDir pull
|
||||
if ($LASTEXITCODE -ne 0) {
|
||||
Write-Host " [WARN] git pull failed -- using existing source" -ForegroundColor Yellow
|
||||
}
|
||||
} else {
|
||||
Write-Host " Cloning llama.cpp..." -ForegroundColor Gray
|
||||
if (Test-Path $LlamaCppDir) { Remove-Item -Recurse -Force $LlamaCppDir }
|
||||
git clone --depth 1 https://github.com/ggml-org/llama.cpp.git $LlamaCppDir
|
||||
if ($LASTEXITCODE -ne 0) {
|
||||
$BuildOk = $false
|
||||
$FailedStep = "git clone"
|
||||
}
|
||||
}
|
||||
|
||||
# -- Step B: cmake configure (CUDA + Unsloth flags) --
|
||||
if ($BuildOk) {
|
||||
Write-Host ""
|
||||
Write-Host "--- cmake configure ---" -ForegroundColor Cyan
|
||||
|
||||
$CmakeArgs = @(
|
||||
'-S', $LlamaCppDir,
|
||||
'-B', $BuildDir,
|
||||
'-G', $CmakeGenerator,
|
||||
'-Wno-dev'
|
||||
)
|
||||
# Tell cmake exactly where VS is (bypasses registry lookup)
|
||||
if ($VsInstallPath) {
|
||||
$CmakeArgs += "-DCMAKE_GENERATOR_INSTANCE=$VsInstallPath"
|
||||
}
|
||||
# Common flags
|
||||
$CmakeArgs += '-DBUILD_SHARED_LIBS=OFF'
|
||||
# HTTPS support via OpenSSL
|
||||
if ($OpenSslAvailable -and $OpenSslRoot) {
|
||||
$CmakeArgs += "-DOPENSSL_ROOT_DIR=$OpenSslRoot"
|
||||
$CmakeArgs += '-DLLAMA_OPENSSL=ON'
|
||||
} else {
|
||||
$CmakeArgs += '-DLLAMA_CURL=OFF'
|
||||
}
|
||||
$CmakeArgs += '-DCMAKE_EXE_LINKER_FLAGS=/NODEFAULTLIB:LIBCMT'
|
||||
# CUDA flags (Unsloth-aligned)
|
||||
$CmakeArgs += '-DGGML_CUDA=ON'
|
||||
$CmakeArgs += "-DCUDAToolkit_ROOT=$CudaToolkitRoot"
|
||||
$CmakeArgs += "-DCMAKE_CUDA_COMPILER=$NvccPath"
|
||||
$CmakeArgs += '-DGGML_CUDA_FA_ALL_QUANTS=ON'
|
||||
$CmakeArgs += '-DGGML_CUDA_F16=OFF'
|
||||
$CmakeArgs += '-DGGML_CUDA_GRAPHS=OFF'
|
||||
$CmakeArgs += '-DGGML_CUDA_FORCE_CUBLAS=OFF'
|
||||
$CmakeArgs += '-DGGML_CUDA_PEER_MAX_BATCH_SIZE=8192'
|
||||
if ($CudaArch) {
|
||||
$CmakeArgs += "-DCMAKE_CUDA_ARCHITECTURES=$CudaArch"
|
||||
}
|
||||
|
||||
Write-Host " cmake args:" -ForegroundColor Gray
|
||||
foreach ($arg in $CmakeArgs) {
|
||||
Write-Host " $arg" -ForegroundColor Gray
|
||||
}
|
||||
Write-Host ""
|
||||
|
||||
cmake @CmakeArgs
|
||||
if ($LASTEXITCODE -ne 0) {
|
||||
$BuildOk = $false
|
||||
$FailedStep = "cmake configure"
|
||||
}
|
||||
}
|
||||
|
||||
# -- Step C: Build llama-server --
|
||||
$NumCpu = [Environment]::ProcessorCount
|
||||
if ($NumCpu -lt 1) { $NumCpu = 4 }
|
||||
|
||||
if ($BuildOk) {
|
||||
Write-Host ""
|
||||
Write-Host "--- cmake build (llama-server) ---" -ForegroundColor Cyan
|
||||
Write-Host " Parallel jobs: $NumCpu" -ForegroundColor Gray
|
||||
Write-Host ""
|
||||
|
||||
cmake --build $BuildDir --config Release --target llama-server -j $NumCpu
|
||||
if ($LASTEXITCODE -ne 0) {
|
||||
$BuildOk = $false
|
||||
$FailedStep = "cmake build (llama-server)"
|
||||
}
|
||||
}
|
||||
|
||||
# -- Step D: Build llama-quantize (optional, best-effort) --
|
||||
if ($BuildOk) {
|
||||
Write-Host ""
|
||||
Write-Host "--- cmake build (llama-quantize) ---" -ForegroundColor Cyan
|
||||
cmake --build $BuildDir --config Release --target llama-quantize -j $NumCpu
|
||||
if ($LASTEXITCODE -ne 0) {
|
||||
Write-Host " [WARN] llama-quantize build failed (GGUF export may be unavailable)" -ForegroundColor Yellow
|
||||
}
|
||||
}
|
||||
|
||||
# Restore ErrorActionPreference
|
||||
$ErrorActionPreference = $prevEAP
|
||||
|
||||
# Stop timer
|
||||
$totalSw.Stop()
|
||||
$totalMin = [math]::Floor($totalSw.Elapsed.TotalMinutes)
|
||||
$totalSec = [math]::Round($totalSw.Elapsed.TotalSeconds % 60, 1)
|
||||
|
||||
# -- Summary --
|
||||
Write-Host ""
|
||||
if ($BuildOk -and (Test-Path $LlamaServerBin)) {
|
||||
Write-Host "[OK] llama-server built at $LlamaServerBin" -ForegroundColor Green
|
||||
$QuantizeBin = Join-Path $BuildDir "bin\Release\llama-quantize.exe"
|
||||
if (Test-Path $QuantizeBin) {
|
||||
Write-Host "[OK] llama-quantize available for GGUF export" -ForegroundColor Green
|
||||
}
|
||||
Write-Host " Build time: ${totalMin}m ${totalSec}s" -ForegroundColor Cyan
|
||||
} else {
|
||||
# Check alternate paths (some cmake generators don't use Release subdir)
|
||||
$altBin = Join-Path $BuildDir "bin\llama-server.exe"
|
||||
if ($BuildOk -and (Test-Path $altBin)) {
|
||||
Write-Host "[OK] llama-server built at $altBin" -ForegroundColor Green
|
||||
Write-Host " Build time: ${totalMin}m ${totalSec}s" -ForegroundColor Cyan
|
||||
} else {
|
||||
Write-Host "[FAILED] llama.cpp build failed at step: $FailedStep (${totalMin}m ${totalSec}s)" -ForegroundColor Red
|
||||
Write-Host " To retry: delete $LlamaCppDir and re-run setup." -ForegroundColor Yellow
|
||||
exit 1
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
# ============================================
|
||||
# Add shell aliases (PowerShell profile + cmd batch files)
|
||||
# ============================================
|
||||
Write-Host ""
|
||||
$RepoDir = $PSScriptRoot
|
||||
$VenvPython = Join-Path $RepoDir ".venv\Scripts\python.exe"
|
||||
$CliScript = Join-Path $RepoDir "cli.py"
|
||||
$FrontendDist = Join-Path $RepoDir "studio\frontend\dist"
|
||||
$AliasAdded = $false
|
||||
|
||||
# --- PowerShell profile: add functions ---
|
||||
$ProfileDir = Split-Path $PROFILE -Parent
|
||||
if (-not (Test-Path $ProfileDir)) { New-Item -ItemType Directory -Path $ProfileDir -Force | Out-Null }
|
||||
if (-not (Test-Path $PROFILE)) { New-Item -ItemType File -Path $PROFILE -Force | Out-Null }
|
||||
|
||||
if (-not (Select-String -Path $PROFILE -Pattern "unsloth-studio" -Quiet -ErrorAction SilentlyContinue)) {
|
||||
$block = @"
|
||||
|
||||
# Unsloth Studio launcher
|
||||
function unsloth-studio { & "$VenvPython" "$CliScript" studio -f "$FrontendDist" @args }
|
||||
function unsloth-ui { & "$VenvPython" "$CliScript" studio -f "$FrontendDist" @args }
|
||||
"@
|
||||
Add-Content -Path $PROFILE -Value $block
|
||||
Write-Host "[OK] Aliases 'unsloth-studio' and 'unsloth-ui' added to $PROFILE" -ForegroundColor Green
|
||||
$AliasAdded = $true
|
||||
} else {
|
||||
Write-Host "[OK] Aliases 'unsloth-studio' and 'unsloth-ui' already exist in $PROFILE" -ForegroundColor Green
|
||||
}
|
||||
|
||||
# --- cmd.exe: create batch files and ensure they're on PATH ---
|
||||
$BatDir = Join-Path $RepoDir ".venv\Scripts"
|
||||
foreach ($name in @("unsloth-studio", "unsloth-ui")) {
|
||||
$batPath = Join-Path $BatDir "$name.bat"
|
||||
if (-not (Test-Path $batPath)) {
|
||||
Set-Content -Path $batPath -Value "@echo off`r`n`"$VenvPython`" `"$CliScript`" studio -f `"$FrontendDist`" %*"
|
||||
}
|
||||
}
|
||||
# Persist .venv\Scripts to User PATH so commands work in new cmd.exe terminals without activation
|
||||
$userPath = [Environment]::GetEnvironmentVariable('Path', 'User')
|
||||
if (-not $userPath -or $userPath -notlike "*$BatDir*") {
|
||||
if ($userPath) {
|
||||
[Environment]::SetEnvironmentVariable('Path', "$BatDir;$userPath", 'User')
|
||||
} else {
|
||||
[Environment]::SetEnvironmentVariable('Path', "$BatDir", 'User')
|
||||
}
|
||||
Write-Host " Persisted $BatDir to User PATH" -ForegroundColor Gray
|
||||
}
|
||||
Write-Host "[OK] Batch launchers created (works from any new cmd.exe or PowerShell)" -ForegroundColor Green
|
||||
|
||||
# ============================================
|
||||
# Done
|
||||
# ============================================
|
||||
Write-Host ""
|
||||
Write-Host "+===============================================+" -ForegroundColor Green
|
||||
Write-Host "| Setup Complete! |" -ForegroundColor Green
|
||||
Write-Host "| |" -ForegroundColor Green
|
||||
Write-Host "| IMPORTANT: Open a NEW terminal, then run: |" -ForegroundColor Yellow
|
||||
Write-Host "| |" -ForegroundColor Green
|
||||
Write-Host "| unsloth-studio -H 0.0.0.0 -p 8000 |" -ForegroundColor Green
|
||||
Write-Host "| |" -ForegroundColor Green
|
||||
Write-Host "+===============================================+" -ForegroundColor Green
|
||||
34
setup.sh
34
setup.sh
|
|
@ -170,30 +170,7 @@ SINGLE_ENV_DATA_DESIGNER_DEPS="$REQ_ROOT/single-env/data-designer-deps.txt"
|
|||
SINGLE_ENV_PATCH="$REQ_ROOT/single-env/patch_metadata.py"
|
||||
|
||||
install_python_stack() {
|
||||
run_quiet "pip upgrade" pip install --upgrade pip
|
||||
echo " Installing unsloth-zoo + unsloth..."
|
||||
run_quiet "pip install unsloth" pip install --no-cache-dir -c "$SINGLE_ENV_CONSTRAINTS" -r "$REQ_ROOT/base.txt"
|
||||
echo " Installing additional unsloth dependencies..."
|
||||
run_quiet "pip install extras" pip install --no-cache-dir -c "$SINGLE_ENV_CONSTRAINTS" -r "$REQ_ROOT/extras.txt"
|
||||
run_quiet "pip install torchao+transformers" pip install --force-reinstall --no-cache-dir -c "$SINGLE_ENV_CONSTRAINTS" -r "$REQ_ROOT/overrides.txt"
|
||||
run_quiet "pip install triton_kernels" pip install --no-deps --no-cache-dir -r "$REQ_ROOT/triton-kernels.txt"
|
||||
# Patch: override llama_cpp.py with fix from unsloth-zoo branch
|
||||
LLAMA_CPP_DST="$(pip show unsloth-zoo | grep -i '^Location:' | awk '{print $2}')/unsloth_zoo/llama_cpp.py"
|
||||
curl -sSL "https://raw.githubusercontent.com/unslothai/unsloth-zoo/refs/heads/main/unsloth_zoo/llama_cpp.py" \
|
||||
-o "$LLAMA_CPP_DST"
|
||||
# Patch: override vision.py with fix from unsloth PR: https://github.com/unslothai/unsloth/pull/4091 until next pypi release
|
||||
VISION_DST="$(pip show unsloth | grep -i '^Location:' | awk '{print $2}')/unsloth/models/vision.py"
|
||||
curl -sSL "https://raw.githubusercontent.com/unslothai/unsloth/80e0108a684c882965a02a8ed851e3473c1145ab/unsloth/models/vision.py" \
|
||||
-o "$VISION_DST"
|
||||
echo " Installing studio dependencies..."
|
||||
run_quiet "pip install studio" pip install --no-cache-dir -c "$SINGLE_ENV_CONSTRAINTS" -r "$REQ_ROOT/studio.txt"
|
||||
echo " Installing data-designer dependencies..."
|
||||
run_quiet "pip install data-designer deps" pip install --no-cache-dir -c "$SINGLE_ENV_CONSTRAINTS" -r "$SINGLE_ENV_DATA_DESIGNER_DEPS"
|
||||
echo " Installing data-designer..."
|
||||
run_quiet "pip install data-designer" pip install --no-cache-dir --no-deps -c "$SINGLE_ENV_CONSTRAINTS" -r "$SINGLE_ENV_DATA_DESIGNER"
|
||||
run_quiet "patch single-env metadata" python "$SINGLE_ENV_PATCH"
|
||||
run_quiet "pip check" pip check
|
||||
echo "✅ Python dependencies installed"
|
||||
python "$SCRIPT_DIR/install_python_stack.py"
|
||||
}
|
||||
|
||||
if [ "$IS_COLAB" = true ]; then
|
||||
|
|
@ -220,11 +197,14 @@ else
|
|||
fi
|
||||
|
||||
# ── 8. Build llama.cpp binaries for GGUF inference + export ──
|
||||
# Builds in-tree at $REPO/llama.cpp/. This directory is shared with
|
||||
# unsloth-zoo's GGUF export pipeline. We build:
|
||||
# Builds at ~/.unsloth/llama.cpp — a single shared location under the user's
|
||||
# home directory. This is used by both the inference server and the GGUF
|
||||
# export pipeline (unsloth-zoo).
|
||||
# - llama-server: for GGUF model inference
|
||||
# - llama-quantize: for GGUF export quantization (symlinked to root for check_llama_cpp())
|
||||
LLAMA_CPP_DIR="$SCRIPT_DIR/llama.cpp"
|
||||
UNSLOTH_HOME="$HOME/.unsloth"
|
||||
mkdir -p "$UNSLOTH_HOME"
|
||||
LLAMA_CPP_DIR="$UNSLOTH_HOME/llama.cpp"
|
||||
LLAMA_SERVER_BIN="$LLAMA_CPP_DIR/build/bin/llama-server"
|
||||
rm -rf "$LLAMA_CPP_DIR"
|
||||
{
|
||||
|
|
|
|||
|
|
@ -418,9 +418,8 @@ class ExportBackend:
|
|||
pre_existing_ggufs = set(glob.glob(os.path.join(cwd, "*.gguf")))
|
||||
|
||||
# Pass absolute path — no os.chdir needed.
|
||||
# unsloth saves intermediate HF model files into model_save_path,
|
||||
# while check_llama_cpp("llama.cpp") resolves against cwd (repo root)
|
||||
# where setup.sh already built llama.cpp with quantizer.
|
||||
# unsloth saves intermediate HF model files into model_save_path.
|
||||
# unsloth-zoo's check_llama_cpp() uses ~/.unsloth/llama.cpp by default.
|
||||
model_save_path = os.path.join(abs_save_dir, "model")
|
||||
self.current_model.save_pretrained_gguf(
|
||||
model_save_path,
|
||||
|
|
|
|||
|
|
@ -1593,6 +1593,7 @@ class InferenceBackend:
|
|||
return False
|
||||
|
||||
|
||||
|
||||
# Global inference backend instance
|
||||
inference_backend = InferenceBackend()
|
||||
|
||||
|
|
|
|||
|
|
@ -76,33 +76,83 @@ class LlamaCppBackend:
|
|||
Locate the llama-server binary.
|
||||
|
||||
Search order:
|
||||
1. LLAMA_SERVER_PATH environment variable
|
||||
2. ./llama.cpp/build/bin/llama-server (built by setup.sh in-tree)
|
||||
3. llama-server on PATH (system install)
|
||||
4. ./bin/llama-server (legacy: extracted binary)
|
||||
1. LLAMA_SERVER_PATH environment variable (direct path to binary)
|
||||
1b. UNSLOTH_LLAMA_CPP_PATH env var (custom llama.cpp install dir)
|
||||
2. ~/.unsloth/llama.cpp/llama-server (make build, root dir)
|
||||
3. ~/.unsloth/llama.cpp/build/bin/llama-server (cmake build, Linux)
|
||||
4. ~/.unsloth/llama.cpp/build/bin/Release/llama-server.exe (cmake build, Windows)
|
||||
5. ./llama.cpp/llama-server (legacy: make build, root dir)
|
||||
6. ./llama.cpp/build/bin/llama-server (legacy: cmake in-tree build)
|
||||
7. llama-server on PATH (system install)
|
||||
8. ./bin/llama-server (legacy: extracted binary)
|
||||
"""
|
||||
import os
|
||||
import sys
|
||||
|
||||
# 1. Env var
|
||||
binary_name = "llama-server.exe" if sys.platform == "win32" else "llama-server"
|
||||
|
||||
# 1. Env var — direct path to binary
|
||||
env_path = os.environ.get("LLAMA_SERVER_PATH")
|
||||
if env_path and Path(env_path).is_file():
|
||||
return env_path
|
||||
|
||||
# Project root: llama_cpp.py → inference/ → core/ → backend/ → studio/ → root
|
||||
project_root = Path(__file__).resolve().parents[4]
|
||||
# 1b. UNSLOTH_LLAMA_CPP_PATH — custom llama.cpp install directory
|
||||
custom_llama_cpp = os.environ.get("UNSLOTH_LLAMA_CPP_PATH")
|
||||
if custom_llama_cpp:
|
||||
custom_dir = Path(custom_llama_cpp)
|
||||
# Root dir (make builds)
|
||||
root_bin = custom_dir / binary_name
|
||||
if root_bin.is_file():
|
||||
return str(root_bin)
|
||||
# build/bin/ (cmake builds on Linux)
|
||||
cmake_bin = custom_dir / "build" / "bin" / binary_name
|
||||
if cmake_bin.is_file():
|
||||
return str(cmake_bin)
|
||||
# build/bin/Release/ (cmake builds on Windows)
|
||||
if sys.platform == "win32":
|
||||
win_bin = custom_dir / "build" / "bin" / "Release" / binary_name
|
||||
if win_bin.is_file():
|
||||
return str(win_bin)
|
||||
|
||||
# 2. In-tree llama.cpp build (setup.sh builds here)
|
||||
build_path = project_root / "llama.cpp" / "build" / "bin" / "llama-server"
|
||||
# 2–4. ~/.unsloth/llama.cpp (primary — setup.sh / setup.ps1 build here)
|
||||
unsloth_home = Path.home() / ".unsloth" / "llama.cpp"
|
||||
# Root dir (make builds copy binaries here)
|
||||
home_root = unsloth_home / binary_name
|
||||
if home_root.is_file():
|
||||
return str(home_root)
|
||||
# build/bin/ (cmake builds on Linux)
|
||||
home_linux = unsloth_home / "build" / "bin" / binary_name
|
||||
if home_linux.is_file():
|
||||
return str(home_linux)
|
||||
|
||||
# 3. Windows MSVC build has Release subdir
|
||||
if sys.platform == "win32":
|
||||
home_win = unsloth_home / "build" / "bin" / "Release" / binary_name
|
||||
if home_win.is_file():
|
||||
return str(home_win)
|
||||
|
||||
# 5–6. Legacy: in-tree build (older setup.sh / setup.ps1 versions)
|
||||
project_root = Path(__file__).resolve().parents[4]
|
||||
# Root dir (make builds)
|
||||
root_path = project_root / "llama.cpp" / binary_name
|
||||
if root_path.is_file():
|
||||
return str(root_path)
|
||||
# build/bin/ (cmake builds)
|
||||
build_path = project_root / "llama.cpp" / "build" / "bin" / binary_name
|
||||
if build_path.is_file():
|
||||
return str(build_path)
|
||||
if sys.platform == "win32":
|
||||
win_path = project_root / "llama.cpp" / "build" / "bin" / "Release" / binary_name
|
||||
if win_path.is_file():
|
||||
return str(win_path)
|
||||
|
||||
# 3. System PATH
|
||||
# 7. System PATH
|
||||
system_path = shutil.which("llama-server")
|
||||
if system_path:
|
||||
return system_path
|
||||
|
||||
# 4. Legacy: extracted to bin/
|
||||
bin_path = project_root / "bin" / "llama-server"
|
||||
# 8. Legacy: extracted to bin/
|
||||
bin_path = project_root / "bin" / binary_name
|
||||
if bin_path.is_file():
|
||||
return str(bin_path)
|
||||
|
||||
|
|
@ -184,16 +234,58 @@ class LlamaCppBackend:
|
|||
|
||||
# Build command based on mode
|
||||
if hf_repo:
|
||||
hf_spec = f"{hf_repo}:{hf_variant}" if hf_variant else hf_repo
|
||||
# Download the GGUF file ourselves using huggingface_hub
|
||||
# (llama-server's -hf flag requires HTTPS/curl which may not
|
||||
# be available, e.g. Windows builds with -DLLAMA_CURL=OFF)
|
||||
try:
|
||||
from huggingface_hub import hf_hub_download
|
||||
except ImportError:
|
||||
raise RuntimeError(
|
||||
"huggingface_hub is required for HF model loading. "
|
||||
"Install it with: pip install huggingface_hub"
|
||||
)
|
||||
|
||||
# Determine the filename from the variant (e.g., "Q4_K_M" -> find matching file)
|
||||
gguf_filename = None
|
||||
if hf_variant:
|
||||
# Try common naming patterns
|
||||
try:
|
||||
from huggingface_hub import list_repo_files
|
||||
files = list_repo_files(hf_repo, token=hf_token)
|
||||
variant_lower = hf_variant.lower()
|
||||
for f in files:
|
||||
if f.endswith(".gguf") and variant_lower in f.lower():
|
||||
gguf_filename = f
|
||||
break
|
||||
except Exception as e:
|
||||
logger.warning(f"Could not list repo files: {e}")
|
||||
|
||||
if not gguf_filename:
|
||||
# Fallback: construct common filename pattern
|
||||
# e.g., "unsloth/gemma-3-4b-it-GGUF" + "Q4_K_M" -> try model name
|
||||
repo_name = hf_repo.split("/")[-1].replace("-GGUF", "")
|
||||
gguf_filename = f"{repo_name}-{hf_variant}.gguf"
|
||||
|
||||
logger.info(f"Downloading GGUF: {hf_repo}/{gguf_filename}")
|
||||
try:
|
||||
local_path = hf_hub_download(
|
||||
repo_id=hf_repo,
|
||||
filename=gguf_filename,
|
||||
token=hf_token,
|
||||
)
|
||||
except Exception as e:
|
||||
raise RuntimeError(
|
||||
f"Failed to download GGUF file '{gguf_filename}' from {hf_repo}: {e}"
|
||||
)
|
||||
|
||||
logger.info(f"GGUF downloaded to: {local_path}")
|
||||
cmd = [
|
||||
binary,
|
||||
"-hf", hf_spec,
|
||||
"-m", local_path,
|
||||
"--port", str(self._port),
|
||||
"-c", str(n_ctx),
|
||||
"-ngl", str(n_gpu_layers),
|
||||
]
|
||||
if hf_token:
|
||||
cmd.extend(["--hf-token", hf_token])
|
||||
elif gguf_path:
|
||||
if not Path(gguf_path).is_file():
|
||||
raise FileNotFoundError(f"GGUF file not found: {gguf_path}")
|
||||
|
|
@ -220,13 +312,31 @@ class LlamaCppBackend:
|
|||
|
||||
logger.info(f"Starting llama-server: {' '.join(cmd)}")
|
||||
|
||||
# Set LD_LIBRARY_PATH so llama-server can find its shared libs
|
||||
# (libmtmd.so, libllama.so, etc.) which live next to the binary
|
||||
# Set library paths so llama-server can find its shared libs and CUDA DLLs
|
||||
import os
|
||||
import sys
|
||||
env = os.environ.copy()
|
||||
binary_dir = str(Path(binary).parent)
|
||||
existing_ld = env.get("LD_LIBRARY_PATH", "")
|
||||
env["LD_LIBRARY_PATH"] = f"{binary_dir}:{existing_ld}" if existing_ld else binary_dir
|
||||
|
||||
if sys.platform == "win32":
|
||||
# On Windows, CUDA DLLs (cublas64_12.dll, cudart64_12.dll, etc.)
|
||||
# must be on PATH. Add CUDA_PATH\bin if available.
|
||||
path_dirs = [binary_dir]
|
||||
cuda_path = os.environ.get("CUDA_PATH", "")
|
||||
if cuda_path:
|
||||
cuda_bin = os.path.join(cuda_path, "bin")
|
||||
if os.path.isdir(cuda_bin):
|
||||
path_dirs.append(cuda_bin)
|
||||
# Some CUDA installs put DLLs in bin\x64
|
||||
cuda_bin_x64 = os.path.join(cuda_path, "bin", "x64")
|
||||
if os.path.isdir(cuda_bin_x64):
|
||||
path_dirs.append(cuda_bin_x64)
|
||||
existing_path = env.get("PATH", "")
|
||||
env["PATH"] = ";".join(path_dirs) + ";" + existing_path
|
||||
else:
|
||||
# Linux: set LD_LIBRARY_PATH for shared libs next to the binary
|
||||
existing_ld = env.get("LD_LIBRARY_PATH", "")
|
||||
env["LD_LIBRARY_PATH"] = f"{binary_dir}:{existing_ld}" if existing_ld else binary_dir
|
||||
|
||||
self._stdout_lines = []
|
||||
self._process = subprocess.Popen(
|
||||
|
|
@ -249,9 +359,8 @@ class LlamaCppBackend:
|
|||
self._is_vision = is_vision
|
||||
self._model_identifier = model_identifier
|
||||
|
||||
# HF mode: llama-server downloads before becoming healthy — need longer timeout
|
||||
timeout = 600.0 if hf_repo else 120.0
|
||||
if not self._wait_for_health(timeout=timeout):
|
||||
# Wait for llama-server to become healthy
|
||||
if not self._wait_for_health(timeout=120.0):
|
||||
self._kill_process()
|
||||
raise RuntimeError(
|
||||
"llama-server failed to start. "
|
||||
|
|
|
|||
|
|
@ -1716,7 +1716,9 @@ class UnslothTrainer:
|
|||
subset: str = None,
|
||||
train_split: str = "train",
|
||||
eval_split: str = None,
|
||||
eval_steps: float = 0.00) -> Optional[tuple]:
|
||||
eval_steps: float = 0.00,
|
||||
dataset_slice_start: int = None,
|
||||
dataset_slice_end: int = None) -> Optional[tuple]:
|
||||
"""
|
||||
Load and prepare dataset for training.
|
||||
|
||||
|
|
@ -1808,6 +1810,18 @@ class UnslothTrainer:
|
|||
if dataset is None:
|
||||
raise ValueError("No dataset provided")
|
||||
|
||||
# Apply index range slicing if requested (inclusive on both ends)
|
||||
if dataset_slice_start is not None or dataset_slice_end is not None:
|
||||
total_rows = len(dataset)
|
||||
start = dataset_slice_start if dataset_slice_start is not None else 0
|
||||
end = dataset_slice_end if dataset_slice_end is not None else total_rows - 1
|
||||
# Clamp to valid range
|
||||
start = max(0, min(start, total_rows - 1))
|
||||
end = max(start, min(end, total_rows - 1))
|
||||
dataset = dataset.select(range(start, end + 1))
|
||||
print(f"Sliced dataset to rows [{start}, {end}]: {len(dataset)} of {total_rows} rows\n")
|
||||
self._update_progress(status_message=f"Sliced dataset to {len(dataset)} rows (indices {start}-{end})")
|
||||
|
||||
# Check if stopped before applying template
|
||||
if self.should_stop:
|
||||
print("Stopped before applying chat template\n")
|
||||
|
|
@ -1851,6 +1865,7 @@ class UnslothTrainer:
|
|||
format_type=format_type,
|
||||
dataset_name=dataset_source,
|
||||
custom_format_mapping=custom_format_mapping,
|
||||
progress_callback=self._update_progress,
|
||||
)
|
||||
|
||||
# Check if stopped during formatting
|
||||
|
|
@ -1858,6 +1873,14 @@ class UnslothTrainer:
|
|||
print("Stopped during dataset formatting\n")
|
||||
return None
|
||||
|
||||
# Abort if dataset formatting/conversion failed
|
||||
if not dataset_info.get("success", True):
|
||||
errors = dataset_info.get("errors", [])
|
||||
error_msg = "; ".join(errors) if errors else "Dataset formatting failed"
|
||||
logger.error(f"Dataset conversion failed: {error_msg}")
|
||||
self._update_progress(error=error_msg)
|
||||
return None
|
||||
|
||||
self._update_progress(status_message=f"Dataset formatted and ready for training")
|
||||
print(f"Dataset formatted successfully\n")
|
||||
|
||||
|
|
|
|||
|
|
@ -117,7 +117,9 @@ class TrainingBackend:
|
|||
eval_split: str = None,
|
||||
eval_steps: float = 0.00,
|
||||
is_dataset_image: bool = False,
|
||||
is_dataset_audio: bool = False) -> bool:
|
||||
is_dataset_audio: bool = False,
|
||||
dataset_slice_start: int = None,
|
||||
dataset_slice_end: int = None) -> bool:
|
||||
"""
|
||||
Start training.
|
||||
|
||||
|
|
@ -228,6 +230,8 @@ class TrainingBackend:
|
|||
train_split=train_split,
|
||||
eval_split=eval_split,
|
||||
eval_steps=eval_steps,
|
||||
dataset_slice_start=dataset_slice_start,
|
||||
dataset_slice_end=dataset_slice_end,
|
||||
)
|
||||
|
||||
# Unpack: load_and_format_dataset returns (dataset, eval_dataset)
|
||||
|
|
|
|||
|
|
@ -37,3 +37,4 @@ class CheckFormatResponse(BaseModel):
|
|||
detected_speaker_column: Optional[str] = None
|
||||
preview_samples: Optional[List[Dict]] = None
|
||||
total_rows: Optional[int] = None
|
||||
warning: Optional[str] = None
|
||||
|
|
|
|||
|
|
@ -22,6 +22,8 @@ class TrainingStartRequest(BaseModel):
|
|||
train_split: Optional[str] = Field("train", description="Training split name")
|
||||
eval_split: Optional[str] = Field(None, description="Eval split name. None = auto-detect")
|
||||
eval_steps: float = Field(0.00, description="Fraction of total steps between evals (0-1)")
|
||||
dataset_slice_start: Optional[int] = Field(None, description="Inclusive start row index for dataset slicing")
|
||||
dataset_slice_end: Optional[int] = Field(None, description="Inclusive end row index for dataset slicing")
|
||||
|
||||
@model_validator(mode="before")
|
||||
@classmethod
|
||||
|
|
|
|||
|
|
@ -5,7 +5,7 @@ import base64
|
|||
import io
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from fastapi import APIRouter, HTTPException
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
import logging
|
||||
|
||||
# Add backend directory to path
|
||||
|
|
@ -15,6 +15,7 @@ if str(backend_path) not in sys.path:
|
|||
|
||||
# Import dataset utilities
|
||||
from utils.datasets import check_dataset_format
|
||||
from auth.authentication import get_current_subject
|
||||
|
||||
router = APIRouter()
|
||||
logger = logging.getLogger(__name__)
|
||||
|
|
@ -84,7 +85,10 @@ DATA_EXTS = (
|
|||
|
||||
|
||||
@router.post("/check-format", response_model=CheckFormatResponse)
|
||||
def check_format(request: CheckFormatRequest):
|
||||
def check_format(
|
||||
request: CheckFormatRequest,
|
||||
current_subject: str = Depends(get_current_subject),
|
||||
):
|
||||
"""
|
||||
Check if a dataset requires manual column mapping.
|
||||
|
||||
|
|
@ -207,6 +211,21 @@ def check_format(request: CheckFormatRequest):
|
|||
else:
|
||||
preview_samples = _serialize_preview_rows(preview_slice)
|
||||
|
||||
# Lightweight URL-based image detection for VLM datasets
|
||||
warning = None
|
||||
image_col = result.get("detected_image_column")
|
||||
if image_col and image_col in (result.get("columns") or []):
|
||||
try:
|
||||
sample_val = preview_slice[0][image_col]
|
||||
if isinstance(sample_val, str) and sample_val.startswith(("http://", "https://")):
|
||||
warning = (
|
||||
"This dataset contains image URLs instead of embedded images. "
|
||||
"Images will be downloaded during training, which may be slow for large datasets."
|
||||
)
|
||||
logger.info(f"URL-based image column detected: {image_col}")
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
return CheckFormatResponse(
|
||||
requires_manual_mapping=result["requires_manual_mapping"],
|
||||
detected_format=result["detected_format"],
|
||||
|
|
@ -221,6 +240,7 @@ def check_format(request: CheckFormatRequest):
|
|||
detected_speaker_column=result.get("detected_speaker_column"),
|
||||
preview_samples=preview_samples,
|
||||
total_rows=total_rows,
|
||||
warning=warning,
|
||||
)
|
||||
|
||||
except HTTPException:
|
||||
|
|
|
|||
|
|
@ -149,6 +149,8 @@ async def start_training(
|
|||
"train_split": request.train_split,
|
||||
"eval_split": request.eval_split,
|
||||
"eval_steps": request.eval_steps,
|
||||
"dataset_slice_start": request.dataset_slice_start,
|
||||
"dataset_slice_end": request.dataset_slice_end,
|
||||
"custom_format_mapping": request.custom_format_mapping,
|
||||
"num_epochs": request.num_epochs,
|
||||
"learning_rate": request.learning_rate,
|
||||
|
|
|
|||
|
|
@ -621,6 +621,7 @@ def format_and_template_dataset(
|
|||
aliases_for_assistant=["gpt", "assistant", "output",],
|
||||
batch_size=1000,
|
||||
num_proc=None,
|
||||
progress_callback=None,
|
||||
):
|
||||
"""
|
||||
Convenience function that combines format_dataset and apply_chat_template_to_dataset.
|
||||
|
|
@ -666,6 +667,7 @@ def format_and_template_dataset(
|
|||
text_column=user_vlm_text_column,
|
||||
image_column=user_vlm_image_column,
|
||||
dataset_name=dataset_name,
|
||||
progress_callback=progress_callback,
|
||||
)
|
||||
warnings.append(f"Applied user VLM mapping: image='{user_vlm_image_column}', text='{user_vlm_text_column}'")
|
||||
|
||||
|
|
@ -762,6 +764,7 @@ def format_and_template_dataset(
|
|||
text_column=vlm_text_column,
|
||||
image_column=vlm_image_column,
|
||||
dataset_name=dataset_name,
|
||||
progress_callback=progress_callback,
|
||||
)
|
||||
|
||||
if vlm_instruction:
|
||||
|
|
|
|||
|
|
@ -238,24 +238,51 @@ def convert_alpaca_to_chatml(dataset, batch_size=1000, num_proc=None):
|
|||
return dataset.map(_convert, **dataset_map_kwargs)
|
||||
|
||||
|
||||
def _format_eta(seconds):
|
||||
"""Format seconds into a human-readable ETA string."""
|
||||
if seconds < 60:
|
||||
return f"{seconds:.0f}s"
|
||||
elif seconds < 3600:
|
||||
m, s = divmod(int(seconds), 60)
|
||||
return f"{m}m {s}s"
|
||||
else:
|
||||
h, remainder = divmod(int(seconds), 3600)
|
||||
m, _ = divmod(remainder, 60)
|
||||
return f"{h}h {m}m"
|
||||
|
||||
|
||||
def convert_to_vlm_format(
|
||||
dataset,
|
||||
instruction=None,
|
||||
text_column="text",
|
||||
image_column="image",
|
||||
dataset_name=None,
|
||||
progress_callback=None,
|
||||
):
|
||||
"""
|
||||
Converts simple {image, text} format to VLM messages format.
|
||||
|
||||
Returns a LIST, not a HuggingFace Dataset (to preserve PIL Images).
|
||||
|
||||
For URL-based image datasets, runs a 200-sample parallel probe first to
|
||||
estimate download speed and failure rate, then reports time estimate or
|
||||
warning through progress_callback before proceeding with the full conversion.
|
||||
|
||||
Args:
|
||||
progress_callback: Optional callable(status_message=str) to report
|
||||
progress to the training overlay.
|
||||
|
||||
Returns:
|
||||
list: List of dicts with 'messages' field
|
||||
"""
|
||||
from PIL import Image
|
||||
from .vlm_processing import generate_smart_vlm_instruction
|
||||
|
||||
def _notify(msg):
|
||||
"""Send status update to the training overlay if callback is available."""
|
||||
if progress_callback:
|
||||
progress_callback(status_message=msg)
|
||||
|
||||
# Generate smart instruction if not provided
|
||||
if instruction is None:
|
||||
instruction_info = generate_smart_vlm_instruction(
|
||||
|
|
@ -281,12 +308,17 @@ def convert_to_vlm_format(
|
|||
|
||||
def _convert_single_sample(sample):
|
||||
"""Convert a single sample to VLM format."""
|
||||
# Get image (might be PIL Image or path)
|
||||
# Get image (might be PIL Image, local path, or URL)
|
||||
image_data = sample[image_column]
|
||||
|
||||
# Handle image paths
|
||||
if isinstance(image_data, str):
|
||||
image_data = Image.open(image_data).convert("RGB")
|
||||
if image_data.startswith(("http://", "https://")):
|
||||
import fsspec
|
||||
from io import BytesIO
|
||||
with fsspec.open(image_data, "rb", expand=True) as f:
|
||||
image_data = Image.open(BytesIO(f.read())).convert("RGB")
|
||||
else:
|
||||
image_data = Image.open(image_data).convert("RGB")
|
||||
|
||||
# Get text
|
||||
text_data = sample[text_column]
|
||||
|
|
@ -317,11 +349,143 @@ def convert_to_vlm_format(
|
|||
# Return dict with messages
|
||||
return {"messages": messages}
|
||||
|
||||
# Use list comprehension and return the LIST directly
|
||||
print(f"🔄 Converting {len(dataset)} samples to VLM format...")
|
||||
converted_list = [_convert_single_sample(sample) for sample in dataset]
|
||||
total = len(dataset)
|
||||
first_image = next(iter(dataset))[image_column]
|
||||
has_urls = isinstance(first_image, str) and first_image.startswith(("http://", "https://"))
|
||||
|
||||
print(f"✅ Converted {len(converted_list)} samples")
|
||||
# ── URL probe: 200 samples with parallel workers to estimate speed + failure rate ──
|
||||
PROBE_SIZE = 200
|
||||
MAX_FAIL_RATE = 0.3
|
||||
|
||||
if has_urls and total > PROBE_SIZE:
|
||||
import time
|
||||
from concurrent.futures import ThreadPoolExecutor, as_completed
|
||||
from utils.hardware import safe_num_proc
|
||||
|
||||
num_workers = safe_num_proc()
|
||||
_notify(f"Probing {PROBE_SIZE} image URLs with {num_workers} workers...")
|
||||
print(f"🔍 Probing {PROBE_SIZE}/{total} image URLs with {num_workers} workers...")
|
||||
|
||||
probe_samples = [dataset[i] for i in range(PROBE_SIZE)]
|
||||
probe_ok = 0
|
||||
probe_fail = 0
|
||||
probe_start = time.time()
|
||||
|
||||
with ThreadPoolExecutor(max_workers=num_workers) as executor:
|
||||
futures = {executor.submit(_convert_single_sample, s): s for s in probe_samples}
|
||||
for future in as_completed(futures):
|
||||
try:
|
||||
future.result()
|
||||
probe_ok += 1
|
||||
except Exception:
|
||||
probe_fail += 1
|
||||
|
||||
probe_elapsed = time.time() - probe_start
|
||||
probe_total = probe_ok + probe_fail
|
||||
fail_rate = probe_fail / probe_total if probe_total > 0 else 0
|
||||
throughput = probe_total / probe_elapsed if probe_elapsed > 0 else 0
|
||||
|
||||
if fail_rate >= MAX_FAIL_RATE:
|
||||
msg = (
|
||||
f"⚠️ {fail_rate:.0%} of the first {PROBE_SIZE} images failed to download "
|
||||
f"({probe_fail}/{probe_total}). "
|
||||
"This dataset has too many broken or unreachable image URLs. "
|
||||
"Consider using a dataset with embedded images instead."
|
||||
)
|
||||
print(msg)
|
||||
_notify(msg)
|
||||
raise ValueError(msg)
|
||||
|
||||
# Estimate total time for remaining samples
|
||||
remaining = total - PROBE_SIZE
|
||||
estimated_seconds = remaining / throughput if throughput > 0 else 0
|
||||
eta_str = _format_eta(estimated_seconds)
|
||||
|
||||
info_msg = (
|
||||
f"Downloading {total:,} images ({num_workers} workers, ~{throughput:.1f} img/s). "
|
||||
f"Estimated time: ~{eta_str}"
|
||||
)
|
||||
if probe_fail > 0:
|
||||
info_msg += f" | {fail_rate:.0%} broken URLs will be skipped"
|
||||
|
||||
print(f"✅ Probe passed: {probe_ok}/{probe_total} ok, {probe_fail} failed ({fail_rate:.0%}), {throughput:.1f} img/s")
|
||||
print(f"⏱️ Estimated time for {total:,} samples: ~{eta_str}")
|
||||
_notify(info_msg)
|
||||
|
||||
# ── Full conversion with progress ──
|
||||
from tqdm import tqdm
|
||||
|
||||
print(f"🔄 Converting {total} samples to VLM format...")
|
||||
converted_list = []
|
||||
failed_count = 0
|
||||
|
||||
if has_urls:
|
||||
# Parallel conversion for URL-based datasets
|
||||
import time
|
||||
from concurrent.futures import ThreadPoolExecutor, as_completed
|
||||
from utils.hardware import safe_num_proc
|
||||
|
||||
num_workers = safe_num_proc()
|
||||
batch_size = 500
|
||||
start_time = time.time()
|
||||
|
||||
for batch_start in range(0, total, batch_size):
|
||||
batch_end = min(batch_start + batch_size, total)
|
||||
batch_samples = [dataset[i] for i in range(batch_start, batch_end)]
|
||||
|
||||
with ThreadPoolExecutor(max_workers=num_workers) as executor:
|
||||
futures = {executor.submit(_convert_single_sample, s): i for i, s in enumerate(batch_samples)}
|
||||
batch_results = [None] * len(batch_samples)
|
||||
for future in as_completed(futures):
|
||||
idx = futures[future]
|
||||
try:
|
||||
batch_results[idx] = future.result()
|
||||
except Exception:
|
||||
failed_count += 1
|
||||
|
||||
converted_list.extend(r for r in batch_results if r is not None)
|
||||
|
||||
# Progress update every batch
|
||||
elapsed = time.time() - start_time
|
||||
done = batch_end
|
||||
rate = done / elapsed if elapsed > 0 else 0
|
||||
remaining_time = (total - done) / rate if rate > 0 else 0
|
||||
eta_str = _format_eta(remaining_time)
|
||||
progress_msg = f"Downloading images: {done:,}/{total:,} ({done*100//total}%) | ~{eta_str} remaining | {failed_count} skipped"
|
||||
print(f" [{done}/{total}] {rate:.1f} img/s, {failed_count} failed, ETA {eta_str}")
|
||||
_notify(progress_msg)
|
||||
else:
|
||||
# Sequential conversion for local/embedded images (fast, no I/O bottleneck)
|
||||
pbar = tqdm(dataset, total=total, desc="Converting VLM samples", unit="sample")
|
||||
for sample in pbar:
|
||||
try:
|
||||
converted_list.append(_convert_single_sample(sample))
|
||||
except Exception:
|
||||
failed_count += 1
|
||||
pbar.set_postfix(ok=len(converted_list), failed=failed_count, refresh=False)
|
||||
pbar.close()
|
||||
|
||||
if failed_count > 0:
|
||||
fail_rate = failed_count / total
|
||||
print(f"⚠️ Skipped {failed_count}/{total} ({fail_rate:.0%}) samples with broken/unreachable images")
|
||||
# For datasets that skipped the probe (small URL datasets), check fail rate now
|
||||
if has_urls and fail_rate >= MAX_FAIL_RATE:
|
||||
msg = (
|
||||
f"⚠️ {fail_rate:.0%} of images failed to download ({failed_count}/{total}). "
|
||||
"This dataset has too many broken or unreachable image URLs. "
|
||||
"Consider using a dataset with embedded images instead."
|
||||
)
|
||||
_notify(msg)
|
||||
raise ValueError(msg)
|
||||
|
||||
if len(converted_list) == 0:
|
||||
raise ValueError(
|
||||
f"All {total} samples failed during VLM conversion — no usable images found. "
|
||||
"This dataset may contain only image URLs that are no longer accessible."
|
||||
)
|
||||
|
||||
print(f"✅ Converted {len(converted_list)}/{total} samples")
|
||||
_notify(f"Converted {len(converted_list):,}/{total:,} images successfully")
|
||||
|
||||
# Return list, NOT Dataset
|
||||
return converted_list
|
||||
|
|
|
|||
|
|
@ -423,6 +423,11 @@ def safe_num_proc(desired: Optional[int] = None) -> int:
|
|||
"""
|
||||
Return a safe ``num_proc`` for ``dataset.map()`` calls.
|
||||
|
||||
On Windows, always returns 1 because Python uses ``spawn`` instead of
|
||||
``fork`` for multiprocessing — the overhead of re-importing torch,
|
||||
transformers, unsloth etc. per worker is typically slower than
|
||||
single-process for normal dataset sizes.
|
||||
|
||||
On multi-GPU machines the NVIDIA driver spawns extra background threads,
|
||||
making ``os.fork()`` prone to deadlocks when many workers are created.
|
||||
This helper caps ``num_proc`` to 4 on such machines.
|
||||
|
|
@ -438,6 +443,12 @@ def safe_num_proc(desired: Optional[int] = None) -> int:
|
|||
A safe integer ≥ 1.
|
||||
"""
|
||||
import os
|
||||
import sys
|
||||
|
||||
# Windows uses 'spawn' for multiprocessing — the overhead of re-importing
|
||||
# torch/transformers/unsloth per worker is typically slower than single-process.
|
||||
if sys.platform == "win32":
|
||||
return 1
|
||||
|
||||
if desired is None or not isinstance(desired, int):
|
||||
desired = max(1, os.cpu_count() // 3)
|
||||
|
|
|
|||
|
|
@ -4,8 +4,8 @@ import { Select as SelectPrimitive } from "radix-ui";
|
|||
import type * as React from "react";
|
||||
import { createContext, useContext, useState } from "react";
|
||||
|
||||
import { cn } from "@/lib/utils";
|
||||
import { useDialogPortalContainer } from "@/components/ui/dialog";
|
||||
import { cn } from "@/lib/utils";
|
||||
import { useDialogPortalContainer } from "@/components/ui/dialog";
|
||||
import {
|
||||
ArrowDown01Icon,
|
||||
ArrowUp01Icon,
|
||||
|
|
@ -92,22 +92,22 @@ function SelectTrigger({
|
|||
);
|
||||
}
|
||||
|
||||
function SelectContent({
|
||||
className,
|
||||
children,
|
||||
position = "item-aligned",
|
||||
align = "center",
|
||||
container,
|
||||
...props
|
||||
}: React.ComponentProps<typeof SelectPrimitive.Content> & {
|
||||
container?: HTMLElement | null;
|
||||
}) {
|
||||
const dialogContainer = useDialogPortalContainer();
|
||||
return (
|
||||
<SelectPrimitive.Portal container={container ?? dialogContainer ?? undefined}>
|
||||
<SelectPrimitive.Content
|
||||
data-slot="select-content"
|
||||
data-align-trigger={position === "item-aligned"}
|
||||
function SelectContent({
|
||||
className,
|
||||
children,
|
||||
position = "item-aligned",
|
||||
align = "center",
|
||||
container,
|
||||
...props
|
||||
}: React.ComponentProps<typeof SelectPrimitive.Content> & {
|
||||
container?: HTMLElement | null;
|
||||
}) {
|
||||
const dialogContainer = useDialogPortalContainer();
|
||||
return (
|
||||
<SelectPrimitive.Portal container={container ?? dialogContainer ?? undefined}>
|
||||
<SelectPrimitive.Content
|
||||
data-slot="select-content"
|
||||
data-align-trigger={position === "item-aligned"}
|
||||
className={cn(
|
||||
"bg-popover text-popover-foreground data-open:animate-in data-closed:animate-out data-closed:fade-out-0 data-open:fade-in-0 data-closed:zoom-out-95 data-open:zoom-in-95 data-[side=bottom]:slide-in-from-top-2 data-[side=left]:slide-in-from-right-2 data-[side=right]:slide-in-from-left-2 data-[side=top]:slide-in-from-bottom-2 shadow-border ring-1 ring-border min-w-36 rounded-xl p-1 corner-squircle duration-100 relative z-50 max-h-(--radix-select-content-available-height) origin-(--radix-select-content-transform-origin) overflow-x-hidden overflow-y-auto ",
|
||||
position === "popper" &&
|
||||
|
|
|
|||
|
|
@ -341,6 +341,13 @@ export function DatasetPreviewDialog({
|
|||
/>
|
||||
</div>
|
||||
|
||||
{data.warning && (
|
||||
<div className="rounded-lg border border-amber-200 bg-amber-50 px-4 py-3 text-xs text-amber-700 dark:border-amber-800 dark:bg-amber-950 dark:text-amber-400 mb-4 flex items-start gap-2.5">
|
||||
<HugeiconsIcon icon={AlertCircleIcon} className="size-4 shrink-0 mt-0.5" />
|
||||
<span>{data.warning}</span>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{mappingEnabled && (
|
||||
<DatasetMappingCard
|
||||
mapping={manualMapping}
|
||||
|
|
|
|||
|
|
@ -13,6 +13,7 @@ import {
|
|||
ComboboxItem,
|
||||
ComboboxList,
|
||||
} from "@/components/ui/combobox";
|
||||
import { Input } from "@/components/ui/input";
|
||||
import { InputGroupAddon } from "@/components/ui/input-group";
|
||||
import {
|
||||
Select,
|
||||
|
|
@ -75,6 +76,10 @@ export function DatasetSection() {
|
|||
setDatasetEvalSplit,
|
||||
hfToken,
|
||||
modelType,
|
||||
datasetSliceStart,
|
||||
setDatasetSliceStart,
|
||||
datasetSliceEnd,
|
||||
setDatasetSliceEnd,
|
||||
} = useTrainingConfigStore(
|
||||
useShallow((s) => ({
|
||||
dataset: s.dataset,
|
||||
|
|
@ -89,6 +94,10 @@ export function DatasetSection() {
|
|||
setDatasetEvalSplit: s.setDatasetEvalSplit,
|
||||
hfToken: s.hfToken,
|
||||
modelType: s.modelType,
|
||||
datasetSliceStart: s.datasetSliceStart,
|
||||
setDatasetSliceStart: s.setDatasetSliceStart,
|
||||
datasetSliceEnd: s.datasetSliceEnd,
|
||||
setDatasetSliceEnd: s.setDatasetSliceEnd,
|
||||
})),
|
||||
);
|
||||
|
||||
|
|
@ -293,51 +302,118 @@ export function DatasetSection() {
|
|||
Advanced
|
||||
</CollapsibleTrigger>
|
||||
<CollapsibleContent className="mt-3">
|
||||
<div className="flex flex-col gap-2">
|
||||
<span className="flex items-center gap-1.5 text-xs font-medium text-muted-foreground">
|
||||
Target Format
|
||||
<Tooltip>
|
||||
<TooltipTrigger asChild={true}>
|
||||
<button
|
||||
type="button"
|
||||
className="text-foreground/70 hover:text-foreground"
|
||||
>
|
||||
<HugeiconsIcon
|
||||
icon={InformationCircleIcon}
|
||||
className="size-3"
|
||||
/>
|
||||
</button>
|
||||
</TooltipTrigger>
|
||||
<TooltipContent>
|
||||
Format of your training data. Auto-detect works for most
|
||||
datasets.{" "}
|
||||
<a
|
||||
href="https://unsloth.ai/docs/get-started/fine-tuning-llms-guide/datasets-guide"
|
||||
target="_blank"
|
||||
rel="noopener noreferrer"
|
||||
className="text-primary underline"
|
||||
>
|
||||
Read more
|
||||
</a>
|
||||
</TooltipContent>
|
||||
</Tooltip>
|
||||
</span>
|
||||
<Select
|
||||
value={datasetFormat}
|
||||
onValueChange={(v) =>
|
||||
setDatasetFormat(v as typeof datasetFormat)
|
||||
}
|
||||
>
|
||||
<SelectTrigger className="w-full">
|
||||
<SelectValue />
|
||||
</SelectTrigger>
|
||||
<SelectContent>
|
||||
<SelectItem value="auto">Auto</SelectItem>
|
||||
<SelectItem value="alpaca">Alpaca</SelectItem>
|
||||
<SelectItem value="chatml">ChatML</SelectItem>
|
||||
<SelectItem value="sharegpt">ShareGPT</SelectItem>
|
||||
</SelectContent>
|
||||
</Select>
|
||||
<div className="flex flex-col gap-4">
|
||||
<div className="flex flex-col gap-2">
|
||||
<span className="flex items-center gap-1.5 text-xs font-medium text-muted-foreground">
|
||||
Target Format
|
||||
<Tooltip>
|
||||
<TooltipTrigger asChild={true}>
|
||||
<button
|
||||
type="button"
|
||||
className="text-foreground/70 hover:text-foreground"
|
||||
>
|
||||
<HugeiconsIcon
|
||||
icon={InformationCircleIcon}
|
||||
className="size-3"
|
||||
/>
|
||||
</button>
|
||||
</TooltipTrigger>
|
||||
<TooltipContent>
|
||||
Format of your training data. Auto-detect works for most
|
||||
datasets.{" "}
|
||||
<a
|
||||
href="https://unsloth.ai/docs/get-started/fine-tuning-llms-guide/datasets-guide"
|
||||
target="_blank"
|
||||
rel="noopener noreferrer"
|
||||
className="text-primary underline"
|
||||
>
|
||||
Read more
|
||||
</a>
|
||||
</TooltipContent>
|
||||
</Tooltip>
|
||||
</span>
|
||||
<Select
|
||||
value={datasetFormat}
|
||||
onValueChange={(v) =>
|
||||
setDatasetFormat(v as typeof datasetFormat)
|
||||
}
|
||||
>
|
||||
<SelectTrigger className="w-full">
|
||||
<SelectValue />
|
||||
</SelectTrigger>
|
||||
<SelectContent>
|
||||
<SelectItem value="auto">Auto</SelectItem>
|
||||
<SelectItem value="alpaca">Alpaca</SelectItem>
|
||||
<SelectItem value="chatml">ChatML</SelectItem>
|
||||
<SelectItem value="sharegpt">ShareGPT</SelectItem>
|
||||
</SelectContent>
|
||||
</Select>
|
||||
</div>
|
||||
<div className="grid grid-cols-2 gap-3">
|
||||
<div className="flex flex-col gap-1.5">
|
||||
<span className="flex items-center gap-1.5 text-xs font-medium text-muted-foreground">
|
||||
Train Split Start
|
||||
<Tooltip>
|
||||
<TooltipTrigger asChild={true}>
|
||||
<button
|
||||
type="button"
|
||||
className="text-foreground/70 hover:text-foreground"
|
||||
>
|
||||
<HugeiconsIcon
|
||||
icon={InformationCircleIcon}
|
||||
className="size-3"
|
||||
/>
|
||||
</button>
|
||||
</TooltipTrigger>
|
||||
<TooltipContent>
|
||||
Only train on a subset of your training split by
|
||||
specifying a start row index (inclusive, 0-based).
|
||||
Leave empty to start from the first row.
|
||||
</TooltipContent>
|
||||
</Tooltip>
|
||||
</span>
|
||||
<Input
|
||||
inputMode="numeric"
|
||||
placeholder="0"
|
||||
value={datasetSliceStart ?? ""}
|
||||
onChange={(e) =>
|
||||
setDatasetSliceStart(e.target.value || null)
|
||||
}
|
||||
/>
|
||||
</div>
|
||||
<div className="flex flex-col gap-1.5">
|
||||
<span className="flex items-center gap-1.5 text-xs font-medium text-muted-foreground">
|
||||
Train Split End
|
||||
<Tooltip>
|
||||
<TooltipTrigger asChild={true}>
|
||||
<button
|
||||
type="button"
|
||||
className="text-foreground/70 hover:text-foreground"
|
||||
>
|
||||
<HugeiconsIcon
|
||||
icon={InformationCircleIcon}
|
||||
className="size-3"
|
||||
/>
|
||||
</button>
|
||||
</TooltipTrigger>
|
||||
<TooltipContent>
|
||||
Last row index to include from the training split
|
||||
(inclusive, 0-based). For example, set Start to 0 and
|
||||
End to 99 to train on the first 100 rows. Leave empty
|
||||
to use all remaining rows.
|
||||
</TooltipContent>
|
||||
</Tooltip>
|
||||
</span>
|
||||
<Input
|
||||
inputMode="numeric"
|
||||
placeholder="End"
|
||||
value={datasetSliceEnd ?? ""}
|
||||
onChange={(e) =>
|
||||
setDatasetSliceEnd(e.target.value || null)
|
||||
}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</CollapsibleContent>
|
||||
</Collapsible>
|
||||
|
|
|
|||
|
|
@ -1,3 +1,4 @@
|
|||
import { authFetch } from "@/features/auth";
|
||||
import type { CheckFormatResponse } from "../types/datasets";
|
||||
|
||||
type CheckDatasetFormatArgs = {
|
||||
|
|
@ -15,7 +16,7 @@ export async function checkDatasetFormat({
|
|||
split,
|
||||
isVlm,
|
||||
}: CheckDatasetFormatArgs): Promise<CheckFormatResponse> {
|
||||
const res = await fetch("/api/datasets/check-format", {
|
||||
const res = await authFetch("/api/datasets/check-format", {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({
|
||||
|
|
|
|||
|
|
@ -4,6 +4,15 @@ import type { TrainingStartRequest } from "../types/api";
|
|||
const BACKEND_LORA_TYPE = "LoRA/QLoRA";
|
||||
const BACKEND_FULL_TYPE = "Full Finetuning";
|
||||
|
||||
function parseSliceValue(value: string | null): number | null {
|
||||
if (value == null) return null;
|
||||
const trimmed = value.trim();
|
||||
if (!trimmed) return null;
|
||||
const num = Number(trimmed);
|
||||
if (!Number.isFinite(num) || !Number.isInteger(num)) return null;
|
||||
return num;
|
||||
}
|
||||
|
||||
export function toBackendTrainingType(trainingMethod: string): string {
|
||||
return trainingMethod === "full" ? BACKEND_FULL_TYPE : BACKEND_LORA_TYPE;
|
||||
}
|
||||
|
|
@ -27,6 +36,8 @@ export function buildTrainingStartPayload(
|
|||
subset: hfDataset ? config.datasetSubset : null,
|
||||
train_split: hfDataset ? config.datasetSplit : null,
|
||||
eval_split: hfDataset ? config.datasetEvalSplit : null,
|
||||
dataset_slice_start: parseSliceValue(config.datasetSliceStart),
|
||||
dataset_slice_end: parseSliceValue(config.datasetSliceEnd),
|
||||
local_datasets: [],
|
||||
format_type: config.datasetFormat,
|
||||
custom_format_mapping: customFormatMapping,
|
||||
|
|
|
|||
|
|
@ -106,7 +106,7 @@ export function HfDatasetSubsetSplitSelectors({
|
|||
: "rounded-lg border border-amber-200 bg-amber-50 px-3.5 py-2.5 text-xs text-amber-700 dark:border-amber-800 dark:bg-amber-950 dark:text-amber-400"
|
||||
}
|
||||
>
|
||||
Could not fetch dataset splits: {error}
|
||||
{error}
|
||||
</div>
|
||||
)}
|
||||
|
||||
|
|
|
|||
|
|
@ -16,6 +16,19 @@ const ROLE_REMAP: Record<string, Record<string, string>> = {
|
|||
sharegpt: { user: "human", assistant: "gpt", system: "system" },
|
||||
};
|
||||
|
||||
function normalizeTrainingStartError(message: string): string {
|
||||
const normalized = message.toLowerCase();
|
||||
const isLegacyDatasetScriptError =
|
||||
normalized.includes("failed to check dataset format") &&
|
||||
normalized.includes("dataset scripts are no longer supported");
|
||||
|
||||
if (isLegacyDatasetScriptError) {
|
||||
return "This Hub dataset relies on a legacy custom script and isn’t supported in this training flow.";
|
||||
}
|
||||
|
||||
return message;
|
||||
}
|
||||
|
||||
export function useTrainingActions() {
|
||||
const isStarting = useTrainingRuntimeStore((state) => state.isStarting);
|
||||
const startError = useTrainingRuntimeStore((state) => state.startError);
|
||||
|
|
@ -94,7 +107,9 @@ export function useTrainingActions() {
|
|||
const response = await startTraining(payload);
|
||||
|
||||
if (response.status === "error") {
|
||||
runtimeStore.setStartError(response.error || response.message);
|
||||
const rawMessage = response.error || response.message;
|
||||
const safeMessage = normalizeTrainingStartError(rawMessage);
|
||||
runtimeStore.setStartError(safeMessage);
|
||||
runtimeStore.setStarting(false);
|
||||
return false;
|
||||
}
|
||||
|
|
@ -103,9 +118,10 @@ export function useTrainingActions() {
|
|||
await syncTrainingRuntimeFromBackend();
|
||||
return true;
|
||||
} catch (error) {
|
||||
const message =
|
||||
const rawMessage =
|
||||
error instanceof Error ? error.message : "Failed to start training";
|
||||
runtimeStore.setStartError(message);
|
||||
const safeMessage = normalizeTrainingStartError(rawMessage);
|
||||
runtimeStore.setStartError(safeMessage);
|
||||
runtimeStore.setStarting(false);
|
||||
return false;
|
||||
}
|
||||
|
|
|
|||
|
|
@ -28,6 +28,8 @@ const initialState: TrainingConfigState = {
|
|||
datasetSplit: null,
|
||||
datasetEvalSplit: null,
|
||||
datasetManualMapping: emptyManualMapping(),
|
||||
datasetSliceStart: null,
|
||||
datasetSliceEnd: null,
|
||||
uploadedFile: null,
|
||||
isCheckingVision: false,
|
||||
isVisionModel: false,
|
||||
|
|
@ -261,6 +263,8 @@ export const useTrainingConfigStore = create<TrainingConfigStore>()(
|
|||
datasetSplit: null,
|
||||
datasetEvalSplit: null,
|
||||
datasetManualMapping: emptyManualMapping(),
|
||||
datasetSliceStart: null,
|
||||
datasetSliceEnd: null,
|
||||
isDatasetImage: null,
|
||||
isCheckingDataset: false,
|
||||
});
|
||||
|
|
@ -317,7 +321,10 @@ export const useTrainingConfigStore = create<TrainingConfigStore>()(
|
|||
},
|
||||
setDatasetManualMapping: (datasetManualMapping) =>
|
||||
set({ datasetManualMapping }),
|
||||
setUploadedFile: (uploadedFile) => set({ uploadedFile }),
|
||||
setDatasetSliceStart: (datasetSliceStart) => set({ datasetSliceStart }),
|
||||
setDatasetSliceEnd: (datasetSliceEnd) => set({ datasetSliceEnd }),
|
||||
setUploadedFile: (uploadedFile) =>
|
||||
set({ uploadedFile, datasetSliceStart: null, datasetSliceEnd: null }),
|
||||
setEpochs: (epochs) => set({ epochs }),
|
||||
setContextLength: (contextLength) => set({ contextLength }),
|
||||
setLearningRate: (learningRate) => set({ learningRate }),
|
||||
|
|
@ -374,7 +381,7 @@ export const useTrainingConfigStore = create<TrainingConfigStore>()(
|
|||
},
|
||||
{
|
||||
name: "unsloth_training_config_v1",
|
||||
version: 6,
|
||||
version: 7,
|
||||
migrate: (persisted, version) => {
|
||||
const s = persisted as Record<string, unknown>;
|
||||
if (version < 2 && s.datasetSubset == null && s.datasetConfig != null) {
|
||||
|
|
@ -393,6 +400,10 @@ export const useTrainingConfigStore = create<TrainingConfigStore>()(
|
|||
if (version < 6 && s.datasetEvalSplit == null) {
|
||||
s.datasetEvalSplit = null;
|
||||
}
|
||||
if (version < 7) {
|
||||
s.datasetSliceStart ??= null;
|
||||
s.datasetSliceEnd ??= null;
|
||||
}
|
||||
return s as unknown as TrainingConfigStore;
|
||||
},
|
||||
partialize: partializePersistedState,
|
||||
|
|
|
|||
|
|
@ -187,7 +187,6 @@ export const useTrainingRuntimeStore = create<TrainingRuntimeStore>()((set) => (
|
|||
evalEnabled: payload.eval_enabled ?? state.evalEnabled,
|
||||
message: payload.message,
|
||||
error: payload.error,
|
||||
startError: null,
|
||||
currentStep:
|
||||
typeof detailStep === "number" ? Math.max(detailStep, 0) : state.currentStep,
|
||||
totalSteps:
|
||||
|
|
|
|||
|
|
@ -8,6 +8,8 @@ export interface TrainingStartRequest {
|
|||
subset: string | null;
|
||||
train_split: string | null;
|
||||
eval_split: string | null;
|
||||
dataset_slice_start: number | null;
|
||||
dataset_slice_end: number | null;
|
||||
local_datasets: string[];
|
||||
format_type: string;
|
||||
custom_format_mapping?: Record<string, string> | null;
|
||||
|
|
|
|||
|
|
@ -26,6 +26,8 @@ export interface TrainingConfigState {
|
|||
datasetSplit: string | null;
|
||||
datasetEvalSplit: string | null;
|
||||
datasetManualMapping: DatasetManualMapping;
|
||||
datasetSliceStart: string | null;
|
||||
datasetSliceEnd: string | null;
|
||||
uploadedFile: string | null;
|
||||
epochs: number;
|
||||
contextLength: number;
|
||||
|
|
@ -85,6 +87,8 @@ export interface TrainingConfigActions {
|
|||
setDatasetSplit: (split: string | null) => void;
|
||||
setDatasetEvalSplit: (split: string | null) => void;
|
||||
setDatasetManualMapping: (mapping: DatasetManualMapping) => void;
|
||||
setDatasetSliceStart: (value: string | null) => void;
|
||||
setDatasetSliceEnd: (value: string | null) => void;
|
||||
setUploadedFile: (file: string | null) => void;
|
||||
setEpochs: (epochs: number) => void;
|
||||
setContextLength: (length: number) => void;
|
||||
|
|
|
|||
|
|
@ -12,5 +12,6 @@ export type CheckFormatResponse = {
|
|||
is_image?: boolean;
|
||||
is_audio?: boolean;
|
||||
multimodal_columns?: string[] | null;
|
||||
warning?: string | null;
|
||||
};
|
||||
|
||||
|
|
|
|||
|
|
@ -35,6 +35,36 @@ export interface HfDatasetSplitsResult {
|
|||
|
||||
const HF_SPLITS_API = "https://datasets-server.huggingface.co/splits";
|
||||
|
||||
function normalizeDatasetSplitsError(message: string): string {
|
||||
const normalized = message.toLowerCase();
|
||||
|
||||
// datasets-server returns technical script/runtime details for legacy datasets.
|
||||
if (
|
||||
normalized.includes("dataset scripts are no longer supported") ||
|
||||
normalized.includes("runs arbitrary python code")
|
||||
) {
|
||||
return "We can’t load subset/split options for this Hub dataset because it relies on a legacy custom script.";
|
||||
}
|
||||
|
||||
if (
|
||||
normalized.includes("unauthorized") ||
|
||||
normalized.includes("forbidden") ||
|
||||
normalized.includes("access token") ||
|
||||
normalized.includes("private") ||
|
||||
normalized.includes("gated") ||
|
||||
normalized.includes("401") ||
|
||||
normalized.includes("403")
|
||||
) {
|
||||
return "Unable to load dataset splits. This dataset may be private or gated. Add a Hugging Face token with access and try again.";
|
||||
}
|
||||
|
||||
if (normalized.includes("not found") || normalized.includes("404")) {
|
||||
return "Dataset not found. Check the dataset name and try again.";
|
||||
}
|
||||
|
||||
return "Unable to load dataset split options for this dataset.";
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Hook
|
||||
// ---------------------------------------------------------------------------
|
||||
|
|
@ -101,7 +131,18 @@ export function useHfDatasetSplits(
|
|||
})
|
||||
.catch((err) => {
|
||||
if (!controller.signal.aborted) {
|
||||
setError(err.message || "Failed to fetch dataset splits");
|
||||
const rawErrorMessage =
|
||||
err instanceof Error
|
||||
? err.message
|
||||
: typeof err === "string"
|
||||
? err
|
||||
: "Failed to fetch dataset splits";
|
||||
console.warn("[useHfDatasetSplits] Failed to fetch dataset splits", {
|
||||
datasetName,
|
||||
message: rawErrorMessage,
|
||||
error: err,
|
||||
});
|
||||
setError(normalizeDatasetSplitsError(rawErrorMessage));
|
||||
setEntries([]);
|
||||
}
|
||||
})
|
||||
|
|
|
|||
|
|
@ -270,6 +270,9 @@
|
|||
@apply font-sans;
|
||||
scrollbar-gutter: stable;
|
||||
}
|
||||
body[data-scroll-locked] {
|
||||
margin-right: 0 !important;
|
||||
}
|
||||
h1,
|
||||
h2,
|
||||
h3,
|
||||
|
|
|
|||
234
test_llama_cpp.ps1
Normal file
234
test_llama_cpp.ps1
Normal file
|
|
@ -0,0 +1,234 @@
|
|||
<#
|
||||
.SYNOPSIS
|
||||
Test script for llama.cpp compilation and binary validation on Windows.
|
||||
Verifies that llama-server was built with CUDA support and can start.
|
||||
|
||||
.USAGE
|
||||
.\test_llama_cpp.ps1
|
||||
.\test_llama_cpp.ps1 -BinaryPath "C:\path\to\llama-server.exe"
|
||||
#>
|
||||
param(
|
||||
[string]$BinaryPath = ""
|
||||
)
|
||||
|
||||
$ErrorActionPreference = "Continue"
|
||||
|
||||
Write-Host ""
|
||||
Write-Host "============================================" -ForegroundColor Cyan
|
||||
Write-Host " llama.cpp Windows Build Test" -ForegroundColor Cyan
|
||||
Write-Host "============================================" -ForegroundColor Cyan
|
||||
Write-Host ""
|
||||
|
||||
# -- Step 1: Locate the binary ------------------------------------------
|
||||
Write-Host "1. Locating llama-server binary..." -ForegroundColor Yellow
|
||||
|
||||
$SearchPaths = @()
|
||||
|
||||
if ($BinaryPath) {
|
||||
$SearchPaths += $BinaryPath
|
||||
}
|
||||
|
||||
# Add all known locations
|
||||
$RepoRoot = $PSScriptRoot
|
||||
$SearchPaths += Join-Path $RepoRoot "llama.cpp\build\bin\Release\llama-server.exe"
|
||||
$SearchPaths += Join-Path $RepoRoot "llama.cpp\build\bin\llama-server.exe"
|
||||
# Legacy: older setup.ps1 built under ~/.unsloth
|
||||
$SearchPaths += Join-Path $env:USERPROFILE ".unsloth\llama.cpp\build\bin\Release\llama-server.exe"
|
||||
|
||||
# Check LLAMA_SERVER_PATH env var
|
||||
$envPath = $env:LLAMA_SERVER_PATH
|
||||
if ($envPath) {
|
||||
$SearchPaths = @($envPath) + $SearchPaths
|
||||
}
|
||||
|
||||
# Also check system PATH
|
||||
$systemPath = (Get-Command llama-server -ErrorAction SilentlyContinue)
|
||||
if ($systemPath) {
|
||||
$SearchPaths += $systemPath.Source
|
||||
}
|
||||
|
||||
$FoundBinary = $null
|
||||
foreach ($p in $SearchPaths) {
|
||||
if (Test-Path $p) {
|
||||
$FoundBinary = $p
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
if (-not $FoundBinary) {
|
||||
Write-Host " [FAIL] llama-server.exe not found!" -ForegroundColor Red
|
||||
Write-Host ""
|
||||
Write-Host " Searched locations:" -ForegroundColor Gray
|
||||
foreach ($p in $SearchPaths) {
|
||||
Write-Host " - $p" -ForegroundColor Gray
|
||||
}
|
||||
Write-Host ""
|
||||
Write-Host " To fix: Run setup.bat to build llama.cpp, or set:" -ForegroundColor Yellow
|
||||
Write-Host ' $env:LLAMA_SERVER_PATH = "C:\path\to\llama-server.exe"' -ForegroundColor Yellow
|
||||
exit 1
|
||||
}
|
||||
|
||||
Write-Host " [OK] Found: $FoundBinary" -ForegroundColor Green
|
||||
|
||||
# -- Step 2: Check file info --------------------------------------------
|
||||
Write-Host ""
|
||||
Write-Host "2. Binary info..." -ForegroundColor Yellow
|
||||
|
||||
$fileInfo = Get-Item $FoundBinary
|
||||
$sizeMB = [math]::Round($fileInfo.Length / 1MB, 1)
|
||||
Write-Host " Size: $sizeMB MB" -ForegroundColor Gray
|
||||
Write-Host " Modified: $($fileInfo.LastWriteTime)" -ForegroundColor Gray
|
||||
|
||||
# -- Step 3: Check for CUDA symbols ------------------------------------
|
||||
Write-Host ""
|
||||
Write-Host "3. Checking for CUDA support..." -ForegroundColor Yellow
|
||||
|
||||
# Run with --help or -v and capture output to check for CUDA indicators
|
||||
$helpOutput = & $FoundBinary --version 2>&1 | Out-String
|
||||
if (-not $helpOutput) {
|
||||
$helpOutput = ""
|
||||
}
|
||||
|
||||
# Check binary dependencies for CUDA DLLs using dumpbin if available
|
||||
$dumpbin = (Get-Command dumpbin -ErrorAction SilentlyContinue)
|
||||
$hasCudaDlls = $false
|
||||
|
||||
if ($dumpbin) {
|
||||
$deps = & dumpbin /dependents $FoundBinary 2>&1 | Out-String
|
||||
if ($deps -match "cudart|cublas|cublasLt|nvcuda") {
|
||||
$hasCudaDlls = $true
|
||||
Write-Host " [OK] CUDA DLLs found in dependencies (dumpbin)" -ForegroundColor Green
|
||||
# Extract CUDA DLL names
|
||||
$cudaDlls = ($deps -split "`n") | Where-Object { $_ -match "cuda|cublas|nvcuda" } | ForEach-Object { $_.Trim() }
|
||||
foreach ($dll in $cudaDlls) {
|
||||
if ($dll) { Write-Host " - $dll" -ForegroundColor Gray }
|
||||
}
|
||||
} else {
|
||||
Write-Host " [WARN] No CUDA DLLs found in dependencies!" -ForegroundColor Red
|
||||
Write-Host " This binary was likely compiled WITHOUT -DGGML_CUDA=ON" -ForegroundColor Red
|
||||
}
|
||||
} else {
|
||||
# Fallback: check file size (CUDA builds are typically > 50MB)
|
||||
if ($sizeMB -gt 40) {
|
||||
Write-Host " [LIKELY OK] Binary is $sizeMB MB (CUDA builds are typically > 50MB)" -ForegroundColor Green
|
||||
} else {
|
||||
Write-Host " [WARN] Binary is only $sizeMB MB (CPU-only builds are typically < 30MB)" -ForegroundColor Yellow
|
||||
Write-Host " dumpbin not available for detailed check. Install VS Build Tools." -ForegroundColor Gray
|
||||
}
|
||||
}
|
||||
|
||||
# -- Step 4: Quick startup test -----------------------------------------
|
||||
Write-Host ""
|
||||
Write-Host "4. Running startup test (will start and immediately stop)..." -ForegroundColor Yellow
|
||||
|
||||
# Start llama-server on a random port with no model -- just check it initializes
|
||||
$testPort = Get-Random -Minimum 49152 -Maximum 65535
|
||||
$proc = $null
|
||||
|
||||
try {
|
||||
$proc = Start-Process -FilePath $FoundBinary `
|
||||
-ArgumentList "--port", $testPort, "--host", "127.0.0.1" `
|
||||
-PassThru -NoNewWindow -RedirectStandardError "$env:TEMP\llama_test_stderr.txt" `
|
||||
-RedirectStandardOutput "$env:TEMP\llama_test_stdout.txt"
|
||||
|
||||
# Give it 3 seconds to start
|
||||
Start-Sleep -Seconds 3
|
||||
|
||||
# Check if it crashed
|
||||
if ($proc.HasExited) {
|
||||
$exitCode = $proc.ExitCode
|
||||
$stderr = ""
|
||||
if (Test-Path "$env:TEMP\llama_test_stderr.txt") {
|
||||
$stderr = Get-Content "$env:TEMP\llama_test_stderr.txt" -Raw
|
||||
}
|
||||
$stdout = ""
|
||||
if (Test-Path "$env:TEMP\llama_test_stdout.txt") {
|
||||
$stdout = Get-Content "$env:TEMP\llama_test_stdout.txt" -Raw
|
||||
}
|
||||
|
||||
$allOutput = "$stdout`n$stderr"
|
||||
|
||||
if ($allOutput -match "failed to initialize CUDA") {
|
||||
Write-Host " [FAIL] CUDA initialization failed!" -ForegroundColor Red
|
||||
Write-Host " The binary was compiled without CUDA support or CUDA drivers are missing." -ForegroundColor Red
|
||||
Write-Host ""
|
||||
Write-Host " Rebuild with: cmake -DGGML_CUDA=ON ..." -ForegroundColor Yellow
|
||||
} elseif ($allOutput -match "HTTPS is not supported") {
|
||||
# This is expected when LLAMA_CURL=OFF -- not a real failure
|
||||
Write-Host " [OK] Binary started (HTTPS warning is expected -- we use local files)" -ForegroundColor Green
|
||||
} else {
|
||||
Write-Host " [WARN] Process exited with code $exitCode" -ForegroundColor Yellow
|
||||
}
|
||||
|
||||
if ($allOutput.Trim()) {
|
||||
Write-Host ""
|
||||
Write-Host " --- Output ---" -ForegroundColor Gray
|
||||
$allOutput.Trim().Split("`n") | ForEach-Object { Write-Host " $_" -ForegroundColor Gray }
|
||||
}
|
||||
} else {
|
||||
Write-Host " [OK] llama-server started successfully on port $testPort" -ForegroundColor Green
|
||||
|
||||
# Check CUDA detection from startup output
|
||||
Start-Sleep -Seconds 1
|
||||
$stderr = ""
|
||||
if (Test-Path "$env:TEMP\llama_test_stderr.txt") {
|
||||
$stderr = Get-Content "$env:TEMP\llama_test_stderr.txt" -Raw
|
||||
}
|
||||
|
||||
if ($stderr -match "CUDA") {
|
||||
if ($stderr -match "failed to initialize CUDA") {
|
||||
Write-Host " [FAIL] CUDA init failed at runtime!" -ForegroundColor Red
|
||||
} else {
|
||||
Write-Host " [OK] CUDA detected at runtime" -ForegroundColor Green
|
||||
}
|
||||
}
|
||||
|
||||
# Kill it
|
||||
Stop-Process -Id $proc.Id -Force -ErrorAction SilentlyContinue
|
||||
Write-Host " Stopped test server." -ForegroundColor Gray
|
||||
}
|
||||
} catch {
|
||||
Write-Host " [ERROR] Could not start llama-server: $_" -ForegroundColor Red
|
||||
} finally {
|
||||
if ($proc -and -not $proc.HasExited) {
|
||||
Stop-Process -Id $proc.Id -Force -ErrorAction SilentlyContinue
|
||||
}
|
||||
Remove-Item "$env:TEMP\llama_test_stderr.txt" -ErrorAction SilentlyContinue
|
||||
Remove-Item "$env:TEMP\llama_test_stdout.txt" -ErrorAction SilentlyContinue
|
||||
}
|
||||
|
||||
# -- Step 5: Check llama-quantize ---------------------------------------
|
||||
Write-Host ""
|
||||
Write-Host "5. Checking llama-quantize..." -ForegroundColor Yellow
|
||||
|
||||
$quantizePath = Join-Path (Split-Path $FoundBinary) "llama-quantize.exe"
|
||||
if (Test-Path $quantizePath) {
|
||||
$qSize = [math]::Round((Get-Item $quantizePath).Length / 1MB, 1)
|
||||
Write-Host " [OK] Found: $quantizePath ($qSize MB)" -ForegroundColor Green
|
||||
} else {
|
||||
Write-Host " [WARN] llama-quantize.exe not found alongside llama-server" -ForegroundColor Yellow
|
||||
Write-Host " GGUF export/quantization won't work without it" -ForegroundColor Yellow
|
||||
}
|
||||
|
||||
# -- Summary ------------------------------------------------------------
|
||||
Write-Host ""
|
||||
Write-Host "============================================" -ForegroundColor Cyan
|
||||
Write-Host " Summary" -ForegroundColor Cyan
|
||||
Write-Host "============================================" -ForegroundColor Cyan
|
||||
Write-Host " Binary: $FoundBinary" -ForegroundColor Gray
|
||||
Write-Host " Size: $sizeMB MB" -ForegroundColor Gray
|
||||
if ($hasCudaDlls) {
|
||||
Write-Host " CUDA: YES (confirmed via DLL deps)" -ForegroundColor Green
|
||||
} elseif ($sizeMB -gt 40) {
|
||||
Write-Host " CUDA: LIKELY (large binary size)" -ForegroundColor Yellow
|
||||
} else {
|
||||
Write-Host " CUDA: NO (rebuild with -DGGML_CUDA=ON)" -ForegroundColor Red
|
||||
}
|
||||
Write-Host ""
|
||||
|
||||
if (-not $hasCudaDlls -and $sizeMB -le 40) {
|
||||
Write-Host "To rebuild with CUDA:" -ForegroundColor Yellow
|
||||
Write-Host ' 1. Delete the build dir: Remove-Item -Recurse -Force "$env:USERPROFILE\.unsloth\llama.cpp\build"' -ForegroundColor Gray
|
||||
Write-Host ' 2. Re-run: .\setup.bat' -ForegroundColor Gray
|
||||
Write-Host ""
|
||||
}
|
||||
Loading…
Add table
Add a link
Reference in a new issue