Merge nightly into feature/transformers-v5-support

.gitignore

@@ -21,10 +21,17 @@ unsloth_compiled_cache/
 # ML artifacts (large files)
 outputs/
 exports/
+/datasets/
 unsloth_training_checkpoints/
 *.gguf
 *.safetensors
 
+# llama.cpp build (built by setup.sh, shared with unsloth-zoo export)
+llama.cpp/
+
+# Built binaries (llama-server etc.)
+bin/
+
 # IDE / Editors
 .vscode/
 .idea/
@@ -47,6 +54,16 @@ resources/
 tmp/
 auth.db
 studio/frontend/package-lock.json
+
+# Local working docs
+**/CLAUDE.md
+**/claude.md
+**/AGENT.md
+**/agent.md
+docs/canvas-lab-architecture.md
+studio/frontend/test/
+studio/tests/
+studio/backend/tests/
 log_rtx.txt
 log.txt
 setup_leo.sh

README.md

@@ -30,28 +30,119 @@
 
 ## Quick Start
 
-### One-command setup
+### Prerequisites
+
+| Requirement | Linux / WSL | Windows |
+|---|---|---|
+| **GPU** | NVIDIA GPU with working driver | NVIDIA GPU with working driver |
+| **Python** | 3.11 – 3.13 | 3.11 – 3.13 |
+| **Git** | Pre-installed on most distros | Auto-installed by setup script (via `winget`) |
+| **CMake** | Pre-installed or `sudo apt install cmake` | Auto-installed by setup script (via `winget`) |
+| **C++ compiler** | `build-essential` (auto-detected) | Visual Studio Build Tools 2022 (auto-installed by setup script) |
+| **CUDA Toolkit** | Optional — setup auto-detects `nvcc` | Auto-installed by setup script (version matched to driver) |
+
+> [!NOTE]
+> On **WSL**, the setup script will also run `sudo apt-get install build-essential cmake curl git libcurl4-openssl-dev` so that GGUF export works in non-interactive subprocesses. You may be prompted for your password during setup.
+
+---
+
+### Linux / Windows WSL
 
 ```bash
+# 1. Clone the repo
+git clone https://github.com/unslothai/unsloth-studio.git
+cd unsloth-studio
+
+# 2. Run setup (installs Node, builds frontend, creates .venv, builds llama.cpp)
 bash setup.sh
+
+# 3. Open a new terminal (or source your shell rc), then launch:
+unsloth-studio -H 0.0.0.0 -p 8000
 ```
 
-This script will:
-1. Install **Node.js ≥ 20** via nvm (if needed)
-2. Build the frontend to `studio/frontend/dist`
-3. Create a Python virtual environment and install all dependencies (including `unsloth`)
-4. Register a convenient `unsloth-ui` shell alias
+<details>
+<summary><b>What does <code>setup.sh</code> do?</b></summary>
 
-### Launch the studio
+1. Installs **Node.js ≥ 20** via nvm (if needed)
+2. Runs `npm install && npm run build` for the React frontend
+3. Detects the best **Python 3.11 – 3.13** on your system and creates a `.venv`
+4. Installs all Python dependencies (unsloth, PyTorch with CUDA, triton kernels, etc.)
+5. On **WSL**: pre-installs build dependencies via `apt-get`
+6. Clones and builds **llama.cpp** at `~/.unsloth/llama.cpp` (GPU-accelerated if CUDA is found)
+7. Registers `unsloth-studio` and `unsloth-ui` shell aliases in your shell rc (bash, zsh, fish, or ksh)
+
+</details>
+
+---
+
+### Windows (Native)
+
+> [!IMPORTANT]
+> Requires an **NVIDIA GPU** — CPU-only machines are not supported on Windows.
+
+```powershell
+# 1. Clone the repo
+git clone https://github.com/unslothai/unsloth-studio.git
+cd unsloth-studio
+
+# 2. Run setup (Right-click → "Run with PowerShell", or from a terminal):
+.\setup.bat
+# Or directly:
+powershell -ExecutionPolicy Bypass -File setup.ps1
+```
+
+After setup completes, **open a new terminal** and run:
+
+```powershell
+# PowerShell
+unsloth-studio -H 0.0.0.0 -p 8000
+
+# Or cmd.exe
+unsloth-studio -H 0.0.0.0 -p 8000
+```
+
+<details>
+<summary><b>What does <code>setup.ps1</code> do?</b></summary>
+
+1. Enables **Windows Long Paths** (required for deep dependency trees — prompts for UAC)
+2. Auto-installs missing system tools via `winget`: **Git**, **CMake**, **Visual Studio Build Tools 2022**, **CUDA Toolkit** (version-matched to your driver), **Node.js LTS**, **Python 3.12**, **OpenSSL dev**
+3. Builds the React frontend (`npm install && npm run build`)
+4. Creates a `.venv` and installs all Python dependencies (including CUDA-enabled PyTorch from the official index)
+5. Sets `TORCHINDUCTOR_CACHE_DIR=C:\tc` to avoid Windows MAX_PATH issues with Triton
+6. Clones and builds **llama.cpp** at `%USERPROFILE%\.unsloth\llama.cpp` with CUDA + Visual Studio
+7. Registers `unsloth-studio` and `unsloth-ui` commands in both PowerShell profile and `cmd.exe` (via batch files on PATH)
+
+</details>
+
+---
+
+### Google Colab
+
+The setup script auto-detects Colab and installs everything into the existing system Python (no venv):
+
+```python
+!bash setup.sh
+```
+
+---
+
+### Launching the Studio
+
+After setup on any platform, the command is the same:
 
 ```bash
-# After setup, open a new terminal (or source ~/.bashrc), then inside your working directory:
-unsloth-ui -H 0.0.0.0 -p 8000
+unsloth-studio -H 0.0.0.0 -p 8000
 ```
 
-On **first launch**, a one-time setup token is printed to the console. Use it in the browser to create your admin account.
+| Flag | Description |
+|---|---|
+| `-H` / `--host` | Bind address (`0.0.0.0` for all interfaces, `127.0.0.1` for local only) |
+| `-p` / `--port` | Port number (default: `8000`) |
 
-As this repo is in continuous development, please make sure to run the setup.sh file everytime you pull new changes from the repo.
+On **first launch**, a one-time setup token is printed to the console. Open the URL shown in your browser and use this token to create your admin account.
+
+> [!TIP]
+> This repo is in active development. After pulling new changes, **always re-run the setup script** (`bash setup.sh` or `.\setup.bat`) to pick up dependency and build updates.
 
 ## API Reference
 
@@ -105,7 +196,10 @@ new-ui-prototype/
 │       ├── export.py
 │       ├── ui.py
 │       └── studio.py
-├── setup.sh                   # One-command bootstrap script
+├── setup.sh                   # Bootstrap script (Linux / WSL / Colab)
+├── setup.ps1                  # Bootstrap script (Windows native)
+├── setup.bat                  # Wrapper to launch setup.ps1 via double-click
+├── install_python_stack.py    # Cross-platform Python dependency installer
 └── studio/
     ├── backend/
     │   ├── main.py            # FastAPI app & middleware

install_python_stack.py

@@ -0,0 +1,245 @@
+#!/usr/bin/env python3
+"""Cross-platform Python dependency installer for Unsloth Studio.
+
+Called by both setup.sh (Linux / WSL) and setup.ps1 (Windows) after the
+virtual environment is already activated.  Expects `pip` and `python` on
+PATH to point at the venv.
+"""
+
+from __future__ import annotations
+
+import os
+import subprocess
+import sys
+import tempfile
+import urllib.request
+from pathlib import Path
+
+IS_WINDOWS = sys.platform == "win32"
+
+# ── Paths ──────────────────────────────────────────────────────────────
+SCRIPT_DIR = Path(__file__).resolve().parent
+REQ_ROOT = SCRIPT_DIR / "studio" / "backend" / "requirements"
+SINGLE_ENV = REQ_ROOT / "single-env"
+CONSTRAINTS = SINGLE_ENV / "constraints.txt"
+
+# ── Color support ──────────────────────────────────────────────────────
+
+def _enable_colors() -> bool:
+    """Try to enable ANSI color support. Returns True if available."""
+    if not hasattr(sys.stdout, "fileno"):
+        return False
+    try:
+        if not os.isatty(sys.stdout.fileno()):
+            return False
+    except Exception:
+        return False
+    if IS_WINDOWS:
+        try:
+            import ctypes
+            kernel32 = ctypes.windll.kernel32
+            # Enable ENABLE_VIRTUAL_TERMINAL_PROCESSING (0x0004) on stdout
+            handle = kernel32.GetStdHandle(-11)  # STD_OUTPUT_HANDLE
+            mode = ctypes.c_ulong()
+            kernel32.GetConsoleMode(handle, ctypes.byref(mode))
+            kernel32.SetConsoleMode(handle, mode.value | 0x0004)
+            return True
+        except Exception:
+            return False
+    return True  # Unix terminals support ANSI by default
+
+_HAS_COLOR = _enable_colors()
+
+def _green(msg: str) -> str:
+    return f"\033[92m{msg}\033[0m" if _HAS_COLOR else msg
+
+def _cyan(msg: str) -> str:
+    return f"\033[96m{msg}\033[0m" if _HAS_COLOR else msg
+
+def _red(msg: str) -> str:
+    return f"\033[91m{msg}\033[0m" if _HAS_COLOR else msg
+
+
+def run(label: str, cmd: list[str], *, quiet: bool = True) -> None:
+    """Run a command; on failure print output and exit."""
+    print(_cyan(f"   {label}..."))
+    result = subprocess.run(
+        cmd,
+        stdout=subprocess.PIPE if quiet else None,
+        stderr=subprocess.STDOUT if quiet else None,
+    )
+    if result.returncode != 0:
+        print(_red(f"❌ {label} failed (exit code {result.returncode}):"))
+        if result.stdout:
+            print(result.stdout.decode(errors="replace"))
+        sys.exit(result.returncode)
+
+
+# Packages to skip on Windows (require special build steps)
+WINDOWS_SKIP_PACKAGES = {"open_spiel"}
+
+
+def _filter_requirements(req: Path, skip: set[str]) -> Path:
+    """Return a temp copy of a requirements file with certain packages removed."""
+    lines = req.read_text(encoding="utf-8").splitlines(keepends=True)
+    filtered = [
+        line for line in lines
+        if not any(line.strip().lower().startswith(pkg) for pkg in skip)
+    ]
+    tmp = tempfile.NamedTemporaryFile(
+        mode="w", suffix=".txt", delete=False, encoding="utf-8",
+    )
+    tmp.writelines(filtered)
+    tmp.close()
+    return Path(tmp.name)
+
+
+def pip_install(
+    label: str,
+    *args: str,
+    req: Path | None = None,
+    constrain: bool = True,
+) -> None:
+    """Build and run a pip install command."""
+    cmd = [sys.executable, "-m", "pip", "install"]
+    cmd.extend(args)
+    if constrain and CONSTRAINTS.is_file():
+        cmd.extend(["-c", str(CONSTRAINTS)])
+    actual_req = req
+    if req is not None and IS_WINDOWS and WINDOWS_SKIP_PACKAGES:
+        actual_req = _filter_requirements(req, WINDOWS_SKIP_PACKAGES)
+    if actual_req is not None:
+        cmd.extend(["-r", str(actual_req)])
+    try:
+        run(label, cmd)
+    finally:
+        # Clean up temp file if we created one
+        if actual_req is not None and actual_req != req:
+            actual_req.unlink(missing_ok=True)
+
+
+
+def download_file(url: str, dest: Path) -> None:
+    """Download a file using urllib (no curl dependency)."""
+    urllib.request.urlretrieve(url, dest)
+
+
+def patch_package_file(package_name: str, relative_path: str, url: str) -> None:
+    """Download a file from url and overwrite a file inside an installed package."""
+    result = subprocess.run(
+        [sys.executable, "-m", "pip", "show", package_name],
+        capture_output=True, text=True,
+    )
+    if result.returncode != 0:
+        print(_red(f"   ⚠️  Could not find package {package_name}, skipping patch"))
+        return
+
+    location = None
+    for line in result.stdout.splitlines():
+        if line.lower().startswith("location:"):
+            location = line.split(":", 1)[1].strip()
+            break
+
+    if not location:
+        print(_red(f"   ⚠️  Could not determine location of {package_name}"))
+        return
+
+    dest = Path(location) / relative_path
+    print(_cyan(f"   Patching {dest.name} in {package_name}..."))
+    download_file(url, dest)
+
+
+# ── Main install sequence ─────────────────────────────────────────────
+
+def install_python_stack() -> int:
+    print(_cyan("── Installing Python stack ──"))
+
+    # 1. Upgrade pip
+    run("Upgrading pip", [sys.executable, "-m", "pip", "install", "--upgrade", "pip"])
+
+    # 2. Core packages: unsloth-zoo + unsloth
+    pip_install(
+        "Installing unsloth-zoo + unsloth",
+        "--no-cache-dir",
+        req=REQ_ROOT / "base.txt",
+    )
+
+    # 3. Extra dependencies
+    pip_install(
+        "Installing additional unsloth dependencies",
+        "--no-cache-dir",
+        req=REQ_ROOT / "extras.txt",
+    )
+
+    # 4. Overrides (torchao, transformers) — force-reinstall
+    pip_install(
+        "Installing torchao + transformers overrides",
+        "--force-reinstall", "--no-cache-dir",
+        req=REQ_ROOT / "overrides.txt",
+    )
+
+    # 5. Triton kernels (no-deps, from source)
+    pip_install(
+        "Installing triton kernels",
+        "--no-deps", "--no-cache-dir",
+        req=REQ_ROOT / "triton-kernels.txt",
+        constrain=False,
+    )
+
+    # 6. Patch: override llama_cpp.py with fix from unsloth-zoo  feature/llama-cpp-windows-support branch
+    patch_package_file(
+        "unsloth-zoo",
+        os.path.join("unsloth_zoo", "llama_cpp.py"),
+        "https://raw.githubusercontent.com/unslothai/unsloth-zoo/refs/heads/main/unsloth_zoo/llama_cpp.py",
+    )
+
+    # 7a. Patch: override vision.py with fix from unsloth PR #4091
+    patch_package_file(
+        "unsloth",
+        os.path.join("unsloth", "models", "vision.py"),
+        "https://raw.githubusercontent.com/unslothai/unsloth/80e0108a684c882965a02a8ed851e3473c1145ab/unsloth/models/vision.py",
+    )
+
+    # 7b. Patch : override save.py with fix from feature/llama-cpp-windows-support
+    patch_package_file(
+        "unsloth",
+        os.path.join("unsloth", "save.py"),
+        "https://raw.githubusercontent.com/unslothai/unsloth/refs/heads/main/unsloth/save.py",
+    )
+
+    # 8. Studio dependencies
+    pip_install(
+        "Installing studio dependencies",
+        "--no-cache-dir",
+        req=REQ_ROOT / "studio.txt",
+    )
+
+    # 9. Data-designer dependencies
+    pip_install(
+        "Installing data-designer dependencies",
+        "--no-cache-dir",
+        req=SINGLE_ENV / "data-designer-deps.txt",
+    )
+
+    # 10. Data-designer packages (no-deps to avoid conflicts)
+    pip_install(
+        "Installing data-designer",
+        "--no-cache-dir", "--no-deps",
+        req=SINGLE_ENV / "data-designer.txt",
+    )
+
+    # 11. Patch metadata for single-env compatibility
+    run(
+        "Patching single-env metadata",
+        [sys.executable, str(SINGLE_ENV / "patch_metadata.py")],
+    )
+
+    # 12. Final check
+    run("Running pip check", [sys.executable, "-m", "pip", "check"], quiet=False)
+
+    print(_green("✅ Python dependencies installed"))
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(install_python_stack())

setup.bat

@@ -0,0 +1,2 @@
+@echo off
+powershell -ExecutionPolicy Bypass -File "%~dp0setup.ps1" %*

setup.ps1

@@ -0,0 +1,964 @@
+#Requires -Version 5.1
+<#
+.SYNOPSIS
+    Full environment setup for Unsloth Studio on Windows (bundled version).
+.DESCRIPTION
+    Always installs Node.js if needed. When running from pip install:
+    skips frontend build (already bundled). When running from git repo:
+    full setup including frontend build.
+    Requires an NVIDIA GPU -- CPU-only machines are not supported.
+.NOTES
+    Usage: powershell -ExecutionPolicy Bypass -File setup.ps1
+#>
+
+$ErrorActionPreference = "Stop"
+$ScriptDir = Split-Path -Parent $MyInvocation.MyCommand.Path
+$PackageDir = Split-Path -Parent $ScriptDir
+
+# Detect if running from pip install (no studio/frontend/ dir in repo)
+$FrontendDir = Join-Path $ScriptDir "studio\frontend"
+$IsPipInstall = -not (Test-Path $FrontendDir)
+
+# ─────────────────────────────────────────────
+# Helper functions
+# ─────────────────────────────────────────────
+
+# Reload ALL environment variables from registry.
+# Picks up changes made by installers (winget, msi, etc.) including
+# Path, CUDA_PATH, CUDA_PATH_V*, and any other vars they set.
+function Refresh-Environment {
+    foreach ($level in @('Machine', 'User')) {
+        $vars = [System.Environment]::GetEnvironmentVariables($level)
+        foreach ($key in $vars.Keys) {
+            if ($key -eq 'Path') { continue }
+            Set-Item -Path "Env:$key" -Value $vars[$key] -ErrorAction SilentlyContinue
+        }
+    }
+    $machinePath = [System.Environment]::GetEnvironmentVariable('Path', 'Machine')
+    $userPath = [System.Environment]::GetEnvironmentVariable('Path', 'User')
+    $env:Path = "$machinePath;$userPath"
+}
+
+# Find nvcc on PATH, CUDA_PATH, or standard toolkit dirs.
+# Returns the path to nvcc.exe, or $null if not found.
+function Find-Nvcc {
+    param([string]$MaxVersion = "")
+
+    # If MaxVersion is set, we need to find a toolkit <= that version.
+    # CUDA toolkits install side-by-side under C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\vX.Y\
+
+    $toolkitBase = 'C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA'
+
+    if ($MaxVersion -and (Test-Path $toolkitBase)) {
+        $drMajor = [int]$MaxVersion.Split('.')[0]
+        $drMinor = [int]$MaxVersion.Split('.')[1]
+
+        # Get all installed CUDA dirs, sorted descending (highest first)
+        $cudaDirs = Get-ChildItem -Directory $toolkitBase | Where-Object {
+            $_.Name -match '^v(\d+)\.(\d+)'
+        } | Sort-Object { [version]($_.Name -replace '^v','') } -Descending
+
+        foreach ($dir in $cudaDirs) {
+            if ($dir.Name -match '^v(\d+)\.(\d+)') {
+                $tkMajor = [int]$Matches[1]; $tkMinor = [int]$Matches[2]
+                $compatible = ($tkMajor -lt $drMajor) -or ($tkMajor -eq $drMajor -and $tkMinor -le $drMinor)
+                if ($compatible) {
+                    $nvcc = Join-Path $dir.FullName 'bin\nvcc.exe'
+                    if (Test-Path $nvcc) {
+                        return $nvcc
+                    }
+                }
+            }
+        }
+
+        # No compatible side-by-side version found
+        return $null
+    }
+
+    # Fallback: no version constraint — pick latest or whatever is available
+
+    # 1. Check nvcc on PATH
+    $cmd = Get-Command nvcc -ErrorAction SilentlyContinue
+    if ($cmd) { return $cmd.Source }
+
+    # 2. Check CUDA_PATH env var
+    $cudaRoot = [Environment]::GetEnvironmentVariable('CUDA_PATH', 'Process')
+    if (-not $cudaRoot) { $cudaRoot = [Environment]::GetEnvironmentVariable('CUDA_PATH', 'Machine') }
+    if (-not $cudaRoot) { $cudaRoot = [Environment]::GetEnvironmentVariable('CUDA_PATH', 'User') }
+    if ($cudaRoot -and (Test-Path (Join-Path $cudaRoot 'bin\nvcc.exe'))) {
+        return (Join-Path $cudaRoot 'bin\nvcc.exe')
+    }
+
+    # 3. Scan standard toolkit directory
+    if (Test-Path $toolkitBase) {
+        $latest = Get-ChildItem -Directory $toolkitBase | Sort-Object Name | Select-Object -Last 1
+        if ($latest -and (Test-Path (Join-Path $latest.FullName 'bin\nvcc.exe'))) {
+            return (Join-Path $latest.FullName 'bin\nvcc.exe')
+        }
+    }
+
+    return $null
+}
+
+# Detect CUDA Compute Capability via nvidia-smi.
+# Returns e.g. "80" for A100 (8.0), "89" for RTX 4090 (8.9), etc.
+# Returns $null if detection fails.
+function Get-CudaComputeCapability {
+    $nvSmi = Get-Command nvidia-smi -ErrorAction SilentlyContinue
+    if (-not $nvSmi) { return $null }
+
+    try {
+        $raw = & nvidia-smi --query-gpu=compute_cap --format=csv,noheader 2>$null
+        if ($LASTEXITCODE -ne 0 -or -not $raw) { return $null }
+
+        # nvidia-smi may return multiple GPUs; take the first one
+        $cap = ($raw -split "`n")[0].Trim()
+        if ($cap -match '^(\d+)\.(\d+)$') {
+            $major = $Matches[1]
+            $minor = $Matches[2]
+            return "$major$minor"
+        }
+    } catch { }
+
+    return $null
+}
+
+# Detect driver's max CUDA version from nvidia-smi and return the highest
+# compatible PyTorch CUDA index tag (e.g. "cu128").
+# PyTorch on Windows ships CPU-only by default from PyPI; CUDA wheels live at
+# https://download.pytorch.org/whl/<tag>. The tag must not exceed the driver's
+# capability: e.g. driver "CUDA Version: 12.9" → cu128 (not cu130).
+function Get-PytorchCudaTag {
+    $nvSmi = Get-Command nvidia-smi -ErrorAction SilentlyContinue
+    if (-not $nvSmi) { return "cu124" }
+
+    try {
+        # 2>&1 | Out-String merges stderr into stdout then converts to a single
+        # string.  Plain 2>$null doesn't fully suppress stderr in PS 5.1 —
+        # ErrorRecord objects leak into $output and break the -match.
+        $output = & nvidia-smi 2>&1 | Out-String
+        if ($output -match 'CUDA Version:\s+(\d+)\.(\d+)') {
+            $major = [int]$Matches[1]
+            $minor = [int]$Matches[2]
+            # PyTorch 2.10 offers: cu124, cu126, cu128, cu130
+            if ($major -ge 13) { return "cu130" }
+            if ($major -eq 12 -and $minor -ge 8) { return "cu128" }
+            if ($major -eq 12 -and $minor -ge 6) { return "cu126" }
+            return "cu124"
+        }
+    } catch { }
+
+    return "cu124"
+}
+
+# Find Visual Studio Build Tools for cmake -G flag.
+# Strategy: (1) vswhere, (2) scan filesystem (handles broken vswhere registration).
+# Returns @{ Generator = "Visual Studio 17 2022"; InstallPath = "C:\..."; Source = "..." } or $null.
+function Find-VsBuildTools {
+    $map = @{ '2022' = '17'; '2019' = '16'; '2017' = '15' }
+
+    # --- Try vswhere first (works when VS is properly registered) ---
+    $vsw = "${env:ProgramFiles(x86)}\Microsoft Visual Studio\Installer\vswhere.exe"
+    if (Test-Path $vsw) {
+        $info = & $vsw -latest -requires Microsoft.VisualStudio.Component.VC.Tools.x86.x64 -property catalog_productLineVersion 2>$null
+        $path = & $vsw -latest -requires Microsoft.VisualStudio.Component.VC.Tools.x86.x64 -property installationPath 2>$null
+        if ($info -and $path) {
+            $y = $info.Trim()
+            $n = $map[$y]
+            if ($n) {
+                return @{ Generator = "Visual Studio $n $y"; InstallPath = $path.Trim(); Source = 'vswhere' }
+            }
+        }
+    }
+
+    # --- Scan filesystem (handles broken vswhere registration after winget cycles) ---
+    $roots = @($env:ProgramFiles, ${env:ProgramFiles(x86)})
+    $editions = @('BuildTools', 'Community', 'Professional', 'Enterprise')
+    $years = @('2022', '2019', '2017')
+
+    foreach ($y in $years) {
+        foreach ($r in $roots) {
+            foreach ($ed in $editions) {
+                $candidate = Join-Path $r "Microsoft Visual Studio\$y\$ed"
+                if (Test-Path $candidate) {
+                    $vcDir = Join-Path $candidate "VC\Tools\MSVC"
+                    if (Test-Path $vcDir) {
+                        $cl = Get-ChildItem -Path $vcDir -Filter "cl.exe" -Recurse -ErrorAction SilentlyContinue | Select-Object -First 1
+                        if ($cl) {
+                            $n = $map[$y]
+                            if ($n) {
+                                return @{ Generator = "Visual Studio $n $y"; InstallPath = $candidate; Source = "filesystem ($ed)"; ClExe = $cl.FullName }
+                            }
+                        }
+                    }
+                }
+            }
+        }
+    }
+
+    return $null
+}
+
+# ─────────────────────────────────────────────
+# Banner
+# ─────────────────────────────────────────────
+Write-Host "+==============================================+" -ForegroundColor Green
+Write-Host "|       Unsloth Studio Setup (Windows)         |" -ForegroundColor Green
+Write-Host "+==============================================+" -ForegroundColor Green
+
+# ==========================================================================
+#  PHASE 1: System-level prerequisites (winget installs, env vars)
+#  All heavy system tool installs happen here BEFORE touching Python.
+# ==========================================================================
+
+# ============================================
+# 1a. GPU requirement check
+# ============================================
+$HasNvidiaSmi = $null -ne (Get-Command nvidia-smi -ErrorAction SilentlyContinue)
+if (-not $HasNvidiaSmi) {
+    Write-Host ""
+    Write-Host "[ERROR] Unsloth Studio requires an NVIDIA GPU." -ForegroundColor Red
+    Write-Host "        CPU-only machines are not supported." -ForegroundColor Red
+    Write-Host ""
+    Write-Host "        If you have an NVIDIA GPU, ensure the driver is installed:" -ForegroundColor Yellow
+    Write-Host "        https://www.nvidia.com/Download/index.aspx" -ForegroundColor Yellow
+    exit 1
+}
+Write-Host "[OK] NVIDIA GPU detected" -ForegroundColor Green
+
+# ============================================
+# 1a.5. Windows Long Paths (required for deep node_modules / Python paths)
+# ============================================
+$LongPathsEnabled = $false
+try {
+    $regVal = Get-ItemProperty -Path "HKLM:\SYSTEM\CurrentControlSet\Control\FileSystem" -Name "LongPathsEnabled" -ErrorAction SilentlyContinue
+    if ($regVal -and $regVal.LongPathsEnabled -eq 1) {
+        $LongPathsEnabled = $true
+    }
+} catch {}
+
+if ($LongPathsEnabled) {
+    Write-Host "[OK] Windows Long Paths enabled" -ForegroundColor Green
+} else {
+    Write-Host "Windows Long Paths not enabled (required for Triton compilation and deep dependency paths)." -ForegroundColor Yellow
+    Write-Host "   Requesting admin access to fix..." -ForegroundColor Yellow
+    try {
+        # Spawn an elevated process to set the registry key (triggers UAC prompt)
+        $proc = Start-Process -FilePath "reg.exe" `
+            -ArgumentList 'add "HKLM\SYSTEM\CurrentControlSet\Control\FileSystem" /v LongPathsEnabled /t REG_DWORD /d 1 /f' `
+            -Verb RunAs -Wait -PassThru -ErrorAction Stop
+        if ($proc.ExitCode -eq 0) {
+            $LongPathsEnabled = $true
+            Write-Host "[OK] Windows Long Paths enabled (via UAC)" -ForegroundColor Green
+        } else {
+            Write-Host "[WARN] Failed to enable Long Paths (exit code: $($proc.ExitCode))" -ForegroundColor Yellow
+        }
+    } catch {
+        Write-Host "[WARN] Could not enable Long Paths (UAC was declined or not available)" -ForegroundColor Yellow
+        Write-Host "       Run this manually in an Admin terminal:" -ForegroundColor Yellow
+        Write-Host '       reg add "HKLM\SYSTEM\CurrentControlSet\Control\FileSystem" /v LongPathsEnabled /t REG_DWORD /d 1 /f' -ForegroundColor Cyan
+    }
+}
+
+# ============================================
+# 1b. Git (required by pip for git+https:// deps and by npm)
+# ============================================
+$HasGit = $null -ne (Get-Command git -ErrorAction SilentlyContinue)
+if (-not $HasGit) {
+    Write-Host "Git not found -- installing via winget..." -ForegroundColor Yellow
+    $HasWinget = $null -ne (Get-Command winget -ErrorAction SilentlyContinue)
+    if ($HasWinget) {
+        try {
+            winget install Git.Git --source winget --accept-package-agreements --accept-source-agreements 2>&1 | Out-Null
+            Refresh-Environment
+            $HasGit = $null -ne (Get-Command git -ErrorAction SilentlyContinue)
+        } catch { }
+    }
+    if (-not $HasGit) {
+        Write-Host "[ERROR] Git is required but could not be installed automatically." -ForegroundColor Red
+        Write-Host "        Install Git from https://git-scm.com/download/win and re-run." -ForegroundColor Red
+        exit 1
+    }
+    Write-Host "[OK] Git installed: $(git --version)" -ForegroundColor Green
+} else {
+    Write-Host "[OK] Git found: $(git --version)" -ForegroundColor Green
+}
+
+# ============================================
+# 1c. CMake (required for llama.cpp build)
+# ============================================
+$HasCmake = $null -ne (Get-Command cmake -ErrorAction SilentlyContinue)
+if (-not $HasCmake) {
+    Write-Host "CMake not found -- installing via winget..." -ForegroundColor Yellow
+    $HasWinget = $null -ne (Get-Command winget -ErrorAction SilentlyContinue)
+    if ($HasWinget) {
+        try {
+            winget install Kitware.CMake --source winget --accept-package-agreements --accept-source-agreements 2>&1 | Out-Null
+            Refresh-Environment
+            $HasCmake = $null -ne (Get-Command cmake -ErrorAction SilentlyContinue)
+        } catch { }
+    }
+    if ($HasCmake) {
+        Write-Host "[OK] CMake installed" -ForegroundColor Green
+    } else {
+        Write-Host "[ERROR] CMake is required but could not be installed." -ForegroundColor Red
+        Write-Host "        Install CMake from https://cmake.org/download/ and re-run." -ForegroundColor Red
+        exit 1
+    }
+} else {
+    Write-Host "[OK] CMake found: $(cmake --version | Select-Object -First 1)" -ForegroundColor Green
+}
+
+# ============================================
+# 1d. Visual Studio Build Tools (C++ compiler for llama.cpp)
+# ============================================
+$CmakeGenerator = $null
+$VsInstallPath = $null
+$vsResult = Find-VsBuildTools
+
+if (-not $vsResult) {
+    Write-Host "Visual Studio Build Tools not found -- installing via winget..." -ForegroundColor Yellow
+    Write-Host "   (This is a one-time install, may take several minutes)" -ForegroundColor Gray
+    $HasWinget = $null -ne (Get-Command winget -ErrorAction SilentlyContinue)
+    if ($HasWinget) {
+        $prevEAPTemp = $ErrorActionPreference
+        $ErrorActionPreference = "Continue"
+        winget install Microsoft.VisualStudio.2022.BuildTools --source winget --accept-package-agreements --accept-source-agreements --override "--add Microsoft.VisualStudio.Workload.VCTools --includeRecommended --passive --wait"
+        $ErrorActionPreference = $prevEAPTemp
+        # Re-scan after install (don't trust vswhere catalog)
+        $vsResult = Find-VsBuildTools
+    }
+}
+
+if ($vsResult) {
+    $CmakeGenerator = $vsResult.Generator
+    $VsInstallPath = $vsResult.InstallPath
+    Write-Host "[OK] $CmakeGenerator detected via $($vsResult.Source)" -ForegroundColor Green
+    if ($vsResult.ClExe) { Write-Host "   cl.exe: $($vsResult.ClExe)" -ForegroundColor Gray }
+} else {
+    Write-Host "[ERROR] Visual Studio Build Tools could not be found or installed." -ForegroundColor Red
+    Write-Host "        Manual install:" -ForegroundColor Red
+    Write-Host '        1. winget install Microsoft.VisualStudio.2022.BuildTools --source winget' -ForegroundColor Yellow
+    Write-Host '        2. Open Visual Studio Installer -> Modify -> check "Desktop development with C++"' -ForegroundColor Yellow
+    exit 1
+}
+
+# ============================================
+# 1e. CUDA Toolkit (nvcc for llama.cpp build + env vars)
+# ============================================
+# IMPORTANT: The CUDA Toolkit version must be <= the max CUDA version the
+# NVIDIA driver supports.  nvidia-smi reports this as "CUDA Version: X.Y".
+# If we install a toolkit newer than the driver supports, llama-server will
+# fail at runtime with "ggml_cuda_init: failed to initialize CUDA: (null)".
+
+# -- Detect max CUDA version the driver supports --
+$DriverMaxCuda = $null
+try {
+    $smiOut = nvidia-smi 2>&1 | Out-String
+    if ($smiOut -match "CUDA Version:\s+([\d]+)\.([\d]+)") {
+        $DriverMaxCuda = "$($Matches[1]).$($Matches[2])"
+        Write-Host "   Driver supports up to CUDA $DriverMaxCuda" -ForegroundColor Gray
+    }
+} catch {}
+
+# -- Find a toolkit that's compatible with the driver --
+$IncompatibleToolkit = $null
+if ($DriverMaxCuda) {
+    $NvccPath = Find-Nvcc -MaxVersion $DriverMaxCuda
+    if ($NvccPath) {
+        Write-Host "   [OK] Found compatible CUDA Toolkit (nvcc: $NvccPath)" -ForegroundColor Green
+    } else {
+        # Check if there's an incompatible (too new) toolkit installed
+        $AnyNvcc = Find-Nvcc
+        if ($AnyNvcc) {
+            $NvccOut = & $AnyNvcc --version 2>&1 | Out-String
+            if ($NvccOut -match "release\s+([\d]+\.[\d]+)") {
+                $IncompatibleToolkit = $Matches[1]
+            }
+        }
+    }
+} else {
+    $NvccPath = Find-Nvcc
+}
+
+# -- If incompatible toolkit is blocking, tell user to uninstall it --
+if (-not $NvccPath -and $IncompatibleToolkit) {
+    Write-Host "" -ForegroundColor Red
+    Write-Host "========================================================================" -ForegroundColor Red
+    Write-Host "[ERROR] CUDA Toolkit $IncompatibleToolkit is installed but INCOMPATIBLE" -ForegroundColor Red
+    Write-Host "        with your NVIDIA driver (which supports up to CUDA $DriverMaxCuda)." -ForegroundColor Red
+    Write-Host "" -ForegroundColor Red
+    Write-Host "  This will cause 'failed to initialize CUDA' errors at runtime." -ForegroundColor Red
+    Write-Host "" -ForegroundColor Red
+    Write-Host "  To fix:" -ForegroundColor Yellow
+    Write-Host "    1. Open Control Panel -> Programs -> Uninstall a program" -ForegroundColor Yellow
+    Write-Host "    2. Uninstall 'NVIDIA CUDA Toolkit $IncompatibleToolkit'" -ForegroundColor Yellow
+    Write-Host "    3. Re-run setup.bat (it will install CUDA $DriverMaxCuda automatically)" -ForegroundColor Yellow
+    Write-Host "" -ForegroundColor Yellow
+    Write-Host "  Alternatively, update your NVIDIA driver to one that supports CUDA $IncompatibleToolkit." -ForegroundColor Gray
+    Write-Host "========================================================================" -ForegroundColor Red
+    exit 1
+}
+
+# -- No toolkit at all: install via winget --
+if (-not $NvccPath) {
+    Write-Host "CUDA toolkit (nvcc) not found -- installing via winget..." -ForegroundColor Yellow
+    $HasWinget = $null -ne (Get-Command winget -ErrorAction SilentlyContinue)
+    if ($HasWinget) {
+        if ($DriverMaxCuda) {
+            # Try descending compatible versions
+            $drMajor = [int]$DriverMaxCuda.Split('.')[0]
+            $drMinor = [int]$DriverMaxCuda.Split('.')[1]
+            for ($m = $drMinor; $m -ge 0; $m--) {
+                $ver = "$drMajor.$m"
+                Write-Host "   Trying CUDA Toolkit $ver via winget..." -ForegroundColor Cyan
+                $prevEAPCuda = $ErrorActionPreference
+                $ErrorActionPreference = "Continue"
+                winget install --id=Nvidia.CUDA --version=$ver -e --source winget --accept-package-agreements --accept-source-agreements 2>&1 | Out-Null
+                $ErrorActionPreference = $prevEAPCuda
+                Refresh-Environment
+                $NvccPath = Find-Nvcc -MaxVersion $DriverMaxCuda
+                if ($NvccPath) {
+                    Write-Host "   [OK] CUDA Toolkit $ver installed (nvcc: $NvccPath)" -ForegroundColor Green
+                    break
+                }
+            }
+        } else {
+            Write-Host "   Installing CUDA Toolkit (latest) via winget..." -ForegroundColor Cyan
+            winget install --id=Nvidia.CUDA -e --source winget --accept-package-agreements --accept-source-agreements
+            Refresh-Environment
+            $NvccPath = Find-Nvcc
+            if ($NvccPath) {
+                Write-Host "   [OK] CUDA Toolkit installed (nvcc: $NvccPath)" -ForegroundColor Green
+            }
+        }
+    }
+}
+
+if (-not $NvccPath) {
+    Write-Host "[ERROR] CUDA Toolkit (nvcc) is required but could not be found or installed." -ForegroundColor Red
+    if ($DriverMaxCuda) {
+        Write-Host "        Install CUDA Toolkit $DriverMaxCuda from https://developer.nvidia.com/cuda-toolkit-archive" -ForegroundColor Yellow
+    } else {
+        Write-Host "        Install CUDA Toolkit from https://developer.nvidia.com/cuda-downloads" -ForegroundColor Yellow
+    }
+    exit 1
+}
+
+# -- Set CUDA env vars so cmake AND MSBuild can find the toolkit --
+$CudaToolkitRoot = Split-Path (Split-Path $NvccPath -Parent) -Parent
+# CUDA_PATH: used by cmake's find_package(CUDAToolkit)
+[Environment]::SetEnvironmentVariable('CUDA_PATH', $CudaToolkitRoot, 'Process')
+# CudaToolkitDir: the MSBuild property that CUDA .targets checks directly
+# Trailing backslash required -- the .targets file appends subpaths to it
+[Environment]::SetEnvironmentVariable('CudaToolkitDir', "$CudaToolkitRoot\", 'Process')
+# Always persist CUDA_PATH to User registry so the compatible toolkit is used
+# in future sessions (overwrites any existing value pointing to a newer, incompatible version)
+[Environment]::SetEnvironmentVariable('CUDA_PATH', $CudaToolkitRoot, 'User')
+Write-Host "   Persisted CUDA_PATH=$CudaToolkitRoot to user environment" -ForegroundColor Gray
+# Ensure nvcc's bin dir is on PATH for this process
+$nvccBinDir = Split-Path $NvccPath -Parent
+if ($env:PATH -notlike "*$nvccBinDir*") {
+    [Environment]::SetEnvironmentVariable('PATH', "$nvccBinDir;$env:PATH", 'Process')
+}
+# Persist nvcc bin dir to User PATH so it works in new terminals
+$userPath = [Environment]::GetEnvironmentVariable('Path', 'User')
+if (-not $userPath -or $userPath -notlike "*$nvccBinDir*") {
+    if ($userPath) {
+        [Environment]::SetEnvironmentVariable('Path', "$nvccBinDir;$userPath", 'User')
+    } else {
+        [Environment]::SetEnvironmentVariable('Path', "$nvccBinDir", 'User')
+    }
+    Write-Host "   Persisted CUDA bin dir to user PATH" -ForegroundColor Gray
+}
+
+Write-Host "[OK] CUDA Toolkit: $NvccPath" -ForegroundColor Green
+Write-Host "   CUDA_PATH      = $CudaToolkitRoot" -ForegroundColor Gray
+Write-Host "   CudaToolkitDir = $CudaToolkitRoot\" -ForegroundColor Gray
+
+# Detect compute capability (used later for llama.cpp cmake)
+$CudaArch = Get-CudaComputeCapability
+if ($CudaArch) {
+    Write-Host "   Compute Capability = $($CudaArch.Insert($CudaArch.Length-1, '.')) (sm_$CudaArch)" -ForegroundColor Gray
+} else {
+    Write-Host "   [WARN] Could not detect compute capability -- cmake will use defaults" -ForegroundColor Yellow
+}
+
+# ============================================
+# 1f. Node.js / npm (always -- needed regardless of install method)
+# ============================================
+# setup.sh installs Node LTS (v22) via nvm. We enforce the same range here:
+# Node >= 20, npm >= 11.
+$NeedNode = $true
+try {
+    $NodeVersion = (node -v 2>$null)
+    $NpmVersion = (npm -v 2>$null)
+    if ($NodeVersion -and $NpmVersion) {
+        $NodeMajor = [int]($NodeVersion -replace 'v','').Split('.')[0]
+        $NpmMajor = [int]$NpmVersion.Split('.')[0]
+
+        if ($NodeMajor -ge 20 -and $NpmMajor -ge 11) {
+            Write-Host "[OK] Node $NodeVersion and npm $NpmVersion already meet requirements." -ForegroundColor Green
+            $NeedNode = $false
+        } else {
+            Write-Host "[WARN] Node $NodeVersion / npm $NpmVersion too old." -ForegroundColor Yellow
+        }
+    }
+} catch {
+    Write-Host "[WARN] Node/npm not found." -ForegroundColor Yellow
+}
+
+if ($NeedNode) {
+    Write-Host "Installing Node.js LTS via winget..." -ForegroundColor Cyan
+    try {
+        winget install OpenJS.NodeJS.LTS --source winget --accept-package-agreements --accept-source-agreements
+        Refresh-Environment
+    } catch {
+        Write-Host "[ERROR] Could not install Node.js automatically." -ForegroundColor Red
+        Write-Host "Please install Node.js >= 20 from https://nodejs.org/" -ForegroundColor Red
+        exit 1
+    }
+}
+
+Write-Host "[OK] Node $(node -v) | npm $(npm -v)" -ForegroundColor Green
+
+# ============================================
+# 1g. Python (>= 3.11 and < 3.14, matching setup.sh)
+# ============================================
+$HasPython = $null -ne (Get-Command python -ErrorAction SilentlyContinue)
+$PythonOk = $false
+
+if ($HasPython) {
+    $PyVer = python --version 2>&1
+    if ($PyVer -match "(\d+)\.(\d+)") {
+        $PyMajor = [int]$Matches[1]; $PyMinor = [int]$Matches[2]
+        if ($PyMajor -eq 3 -and $PyMinor -ge 11 -and $PyMinor -lt 14) {
+            Write-Host "[OK] Python $PyVer" -ForegroundColor Green
+            $PythonOk = $true
+        } else {
+            Write-Host "[ERROR] Python $PyVer is outside supported range (need >= 3.11 and < 3.14)." -ForegroundColor Red
+            Write-Host "        Install Python 3.12 from https://python.org/downloads/" -ForegroundColor Yellow
+            exit 1
+        }
+    }
+} else {
+    # No Python at all -- install 3.12
+    Write-Host "Python not found -- installing Python 3.12 via winget..." -ForegroundColor Yellow
+    $HasWinget = $null -ne (Get-Command winget -ErrorAction SilentlyContinue)
+    if ($HasWinget) {
+        winget install -e --id Python.Python.3.12 --source winget --accept-package-agreements --accept-source-agreements
+        Refresh-Environment
+    }
+    $HasPython = $null -ne (Get-Command python -ErrorAction SilentlyContinue)
+    if (-not $HasPython) {
+        Write-Host "[ERROR] Python could not be installed automatically." -ForegroundColor Red
+        Write-Host "        Install Python 3.12 from https://python.org/downloads/" -ForegroundColor Yellow
+        exit 1
+    }
+    Write-Host "[OK] Python $(python --version)" -ForegroundColor Green
+    $PythonOk = $true
+}
+
+Write-Host ""
+Write-Host "--- System prerequisites ready ---" -ForegroundColor Green
+Write-Host ""
+
+# ==========================================================================
+#  PHASE 2: Frontend build (skip if pip-installed -- already bundled)
+# ==========================================================================
+if ($IsPipInstall) {
+    Write-Host "[OK] Running from pip install - frontend already bundled, skipping build" -ForegroundColor Green
+} else {
+    Write-Host ""
+    Write-Host "Building frontend..." -ForegroundColor Cyan
+    # npm writes warnings to stderr; lower ErrorActionPreference so PS doesn't
+    # treat them as terminating errors (same pattern as the pip section below).
+    $prevEAP_npm = $ErrorActionPreference
+    $ErrorActionPreference = "Continue"
+    Push-Location $FrontendDir
+    # Remove stale node_modules and package-lock.json to avoid version conflicts
+    if (Test-Path "node_modules") { Remove-Item -Recurse -Force "node_modules" }
+    if (Test-Path "package-lock.json") { Remove-Item -Force "package-lock.json" }
+    npm install 2>&1 | Out-Null
+    if ($LASTEXITCODE -ne 0) {
+        Pop-Location
+        $ErrorActionPreference = $prevEAP_npm
+        Write-Host "[ERROR] npm install failed (exit code $LASTEXITCODE)" -ForegroundColor Red
+        Write-Host "   Try running 'npm install' manually in studio/frontend/ to see errors" -ForegroundColor Yellow
+        exit 1
+    }
+    npm run build 2>&1 | Out-Null
+    if ($LASTEXITCODE -ne 0) {
+        Pop-Location
+        $ErrorActionPreference = $prevEAP_npm
+        Write-Host "[ERROR] npm run build failed (exit code $LASTEXITCODE)" -ForegroundColor Red
+        exit 1
+    }
+    Pop-Location
+    $ErrorActionPreference = $prevEAP_npm
+    Write-Host "[OK] Frontend built to studio/frontend/dist" -ForegroundColor Green
+}
+
+# ==========================================================================
+#  PHASE 3: Python environment + dependencies
+# ==========================================================================
+Write-Host ""
+Write-Host "Setting up Python environment..." -ForegroundColor Cyan
+
+# Find Python
+$PythonCmd = $null
+foreach ($candidate in @("python3.12", "python3.11", "python3.10", "python3.9", "python3", "python")) {
+    try {
+        $ver = & $candidate --version 2>&1
+        if ($ver -match 'Python 3\.(\d+)') {
+            $minor = [int]$Matches[1]
+            if ($minor -le 12) {
+                $PythonCmd = $candidate
+                break
+            }
+        }
+    } catch { }
+}
+
+if (-not $PythonCmd) {
+    Write-Host "[ERROR] No Python <= 3.12 found." -ForegroundColor Red
+    exit 1
+}
+
+Write-Host "[OK] Using $PythonCmd ($(& $PythonCmd --version 2>&1))" -ForegroundColor Green
+
+# Always create a .venv for isolation -- even for pip installs.
+# Created in the current working directory (where user ran the command).
+$VenvDir = Join-Path (Get-Location) ".venv"
+if (-not (Test-Path $VenvDir)) {
+    Write-Host "   Creating virtual environment at $VenvDir..." -ForegroundColor Cyan
+    & $PythonCmd -m venv $VenvDir
+} else {
+    Write-Host "   Reusing existing virtual environment at $VenvDir" -ForegroundColor Green
+}
+
+# pip and python write to stderr even on success (progress bars, warnings).
+# With $ErrorActionPreference = "Stop" (set at top of script), PS 5.1
+# converts stderr lines into terminating ErrorRecords, breaking output.
+# Lower to "Continue" for the pip/python section.
+$prevEAP = $ErrorActionPreference
+$ErrorActionPreference = "Continue"
+
+$ActivateScript = Join-Path $VenvDir "Scripts\Activate.ps1"
+. $ActivateScript
+pip install --upgrade pip 2>&1 | Out-Null
+
+# if (-not $IsPipInstall) {
+#     # Running from repo: copy requirements and do editable install
+#     $RepoRoot = (Resolve-Path (Join-Path $ScriptDir "..\..")).Path
+#     $ReqsSrc = Join-Path $RepoRoot "backend\requirements"
+#     $ReqsDst = Join-Path $PackageDir "requirements"
+#     if (-not (Test-Path $ReqsDst)) { New-Item -ItemType Directory -Path $ReqsDst | Out-Null }
+#     Copy-Item (Join-Path $ReqsSrc "*.txt") $ReqsDst -Force
+
+#     Write-Host "   Installing CLI entry point..." -ForegroundColor Cyan
+#     pip install -e $RepoRoot 2>&1 | Out-Null
+# } else {
+#     # Running from pip install: the package is in system Python but not in
+#     # the fresh .venv. Install it so run_install() can find its modules
+#     # and bundled requirements files.
+#     Write-Host "   Installing package into venv..." -ForegroundColor Cyan
+#     pip install unsloth-roland-test 2>&1 | Out-Null
+# }
+
+# Pre-install PyTorch with CUDA support.
+# On Windows, the default PyPI torch wheel is CPU-only.
+# We need PyTorch's CUDA index to get GPU-enabled wheels.
+# PyTorch bundles its own CUDA runtime, so this works regardless
+# of whether the CUDA Toolkit is installed yet.
+# The CUDA tag is chosen based on the driver's max supported CUDA version.
+
+# Windows MAX_PATH (260 chars) causes Triton kernel compilation to fail because
+# the auto-generated filenames are extremely long. Use a short cache directory.
+$TorchCacheDir = "C:\tc"
+if (-not (Test-Path $TorchCacheDir)) { New-Item -ItemType Directory -Path $TorchCacheDir -Force | Out-Null }
+$env:TORCHINDUCTOR_CACHE_DIR = $TorchCacheDir
+[Environment]::SetEnvironmentVariable('TORCHINDUCTOR_CACHE_DIR', $TorchCacheDir, 'User')
+Write-Host "[OK] TORCHINDUCTOR_CACHE_DIR set to $TorchCacheDir (avoids MAX_PATH issues)" -ForegroundColor Green
+
+$CuTag = Get-PytorchCudaTag
+Write-Host "   Installing PyTorch with CUDA support ($CuTag)..." -ForegroundColor Cyan
+pip install torch torchvision torchaudio --index-url "https://download.pytorch.org/whl/$CuTag" 2>&1 | Out-Null
+
+# Ordered heavy dependency installation — shared cross-platform script
+Write-Host "   Running ordered dependency installation..." -ForegroundColor Cyan
+python "$PSScriptRoot\install_python_stack.py"
+# Restore ErrorActionPreference after pip/python work
+$ErrorActionPreference = $prevEAP
+
+# ==========================================================================
+#  PHASE 3.5: Install OpenSSL dev (for HTTPS support in llama-server)
+# ==========================================================================
+# llama-server needs OpenSSL to download models from HuggingFace via -hf.
+# ShiningLight.OpenSSL.Dev includes headers + libs that cmake can find.
+$OpenSslAvailable = $false
+
+# Check if OpenSSL dev is already installed (look for include dir)
+$OpenSslRoots = @(
+    'C:\Program Files\OpenSSL-Win64',
+    'C:\Program Files\OpenSSL',
+    'C:\OpenSSL-Win64'
+)
+$OpenSslRoot = $null
+foreach ($root in $OpenSslRoots) {
+    if (Test-Path (Join-Path $root 'include\openssl\ssl.h')) {
+        $OpenSslRoot = $root
+        break
+    }
+}
+
+if ($OpenSslRoot) {
+    $OpenSslAvailable = $true
+    Write-Host "[OK] OpenSSL dev found at $OpenSslRoot" -ForegroundColor Green
+} else {
+    Write-Host "" 
+    Write-Host "Installing OpenSSL dev (for HTTPS in llama-server)..." -ForegroundColor Cyan
+    $HasWinget = $null -ne (Get-Command winget -ErrorAction SilentlyContinue)
+    if ($HasWinget) {
+        winget install -e --id ShiningLight.OpenSSL.Dev --accept-package-agreements --accept-source-agreements
+        # Re-check after install
+        foreach ($root in $OpenSslRoots) {
+            if (Test-Path (Join-Path $root 'include\openssl\ssl.h')) {
+                $OpenSslRoot = $root
+                $OpenSslAvailable = $true
+                Write-Host "[OK] OpenSSL dev installed at $OpenSslRoot" -ForegroundColor Green
+                break
+            }
+        }
+    }
+    if (-not $OpenSslAvailable) {
+        Write-Host "[WARN] OpenSSL dev not available -- llama-server will be built without HTTPS" -ForegroundColor Yellow
+    }
+}
+
+# ==========================================================================
+#  PHASE 4: Build llama.cpp with CUDA for GGUF inference + export
+# ==========================================================================
+# Builds at ~/.unsloth/llama.cpp — a single shared location under the user's
+# home directory. This is used by both the inference server and the GGUF
+# export pipeline (unsloth-zoo).
+# We build:
+#   - llama-server:   for GGUF model inference (with HTTPS if OpenSSL available)
+#   - llama-quantize: for GGUF export quantization
+# Prerequisites (git, cmake, VS Build Tools, CUDA Toolkit) already installed in Phase 1.
+$UnslothHome = Join-Path $env:USERPROFILE ".unsloth"
+if (-not (Test-Path $UnslothHome)) { New-Item -ItemType Directory -Force $UnslothHome | Out-Null }
+$LlamaCppDir = Join-Path $UnslothHome "llama.cpp"
+$BuildDir = Join-Path $LlamaCppDir "build"
+$LlamaServerBin = Join-Path $BuildDir "bin\Release\llama-server.exe"
+
+if (Test-Path $LlamaServerBin) {
+    Write-Host ""
+    Write-Host "[OK] llama-server already exists at $LlamaServerBin" -ForegroundColor Green
+} else {
+    Write-Host ""
+    Write-Host "Building llama.cpp with CUDA support..." -ForegroundColor Cyan
+    Write-Host "   This typically takes 5-10 minutes on first build." -ForegroundColor Gray
+    Write-Host ""
+
+    # Start total build timer
+    $totalSw = [System.Diagnostics.Stopwatch]::StartNew()
+
+    # Native commands (git, cmake) write to stderr even on success.
+    # With $ErrorActionPreference = "Stop" (set at top of script), PS 5.1
+    # converts stderr lines into terminating ErrorRecords, breaking output.
+    # Lower to "Continue" for the build section.
+    $prevEAP = $ErrorActionPreference
+    $ErrorActionPreference = "Continue"
+
+    $BuildOk = $true
+    $FailedStep = ""
+
+    # -- Step A: Clone or pull llama.cpp --
+
+    if (Test-Path (Join-Path $LlamaCppDir ".git")) {
+        Write-Host "   llama.cpp repo already cloned, pulling latest..." -ForegroundColor Gray
+        git -C $LlamaCppDir pull
+        if ($LASTEXITCODE -ne 0) {
+            Write-Host "   [WARN] git pull failed -- using existing source" -ForegroundColor Yellow
+        }
+    } else {
+        Write-Host "   Cloning llama.cpp..." -ForegroundColor Gray
+        if (Test-Path $LlamaCppDir) { Remove-Item -Recurse -Force $LlamaCppDir }
+        git clone --depth 1 https://github.com/ggml-org/llama.cpp.git $LlamaCppDir
+        if ($LASTEXITCODE -ne 0) {
+            $BuildOk = $false
+            $FailedStep = "git clone"
+        }
+    }
+
+    # -- Step B: cmake configure (CUDA + Unsloth flags) --
+    if ($BuildOk) {
+        Write-Host ""
+        Write-Host "--- cmake configure ---" -ForegroundColor Cyan
+
+        $CmakeArgs = @(
+            '-S', $LlamaCppDir,
+            '-B', $BuildDir,
+            '-G', $CmakeGenerator,
+            '-Wno-dev'
+        )
+        # Tell cmake exactly where VS is (bypasses registry lookup)
+        if ($VsInstallPath) {
+            $CmakeArgs += "-DCMAKE_GENERATOR_INSTANCE=$VsInstallPath"
+        }
+        # Common flags
+        $CmakeArgs += '-DBUILD_SHARED_LIBS=OFF'
+        # HTTPS support via OpenSSL
+        if ($OpenSslAvailable -and $OpenSslRoot) {
+            $CmakeArgs += "-DOPENSSL_ROOT_DIR=$OpenSslRoot"
+            $CmakeArgs += '-DLLAMA_OPENSSL=ON'
+        } else {
+            $CmakeArgs += '-DLLAMA_CURL=OFF'
+        }
+        $CmakeArgs += '-DCMAKE_EXE_LINKER_FLAGS=/NODEFAULTLIB:LIBCMT'
+        # CUDA flags (Unsloth-aligned)
+        $CmakeArgs += '-DGGML_CUDA=ON'
+        $CmakeArgs += "-DCUDAToolkit_ROOT=$CudaToolkitRoot"
+        $CmakeArgs += "-DCMAKE_CUDA_COMPILER=$NvccPath"
+        $CmakeArgs += '-DGGML_CUDA_FA_ALL_QUANTS=ON'
+        $CmakeArgs += '-DGGML_CUDA_F16=OFF'
+        $CmakeArgs += '-DGGML_CUDA_GRAPHS=OFF'
+        $CmakeArgs += '-DGGML_CUDA_FORCE_CUBLAS=OFF'
+        $CmakeArgs += '-DGGML_CUDA_PEER_MAX_BATCH_SIZE=8192'
+        if ($CudaArch) {
+            $CmakeArgs += "-DCMAKE_CUDA_ARCHITECTURES=$CudaArch"
+        }
+
+        Write-Host "   cmake args:" -ForegroundColor Gray
+        foreach ($arg in $CmakeArgs) {
+            Write-Host "     $arg" -ForegroundColor Gray
+        }
+        Write-Host ""
+
+        cmake @CmakeArgs
+        if ($LASTEXITCODE -ne 0) {
+            $BuildOk = $false
+            $FailedStep = "cmake configure"
+        }
+    }
+
+    # -- Step C: Build llama-server --
+    $NumCpu = [Environment]::ProcessorCount
+    if ($NumCpu -lt 1) { $NumCpu = 4 }
+
+    if ($BuildOk) {
+        Write-Host ""
+        Write-Host "--- cmake build (llama-server) ---" -ForegroundColor Cyan
+        Write-Host "   Parallel jobs: $NumCpu" -ForegroundColor Gray
+        Write-Host ""
+
+        cmake --build $BuildDir --config Release --target llama-server -j $NumCpu
+        if ($LASTEXITCODE -ne 0) {
+            $BuildOk = $false
+            $FailedStep = "cmake build (llama-server)"
+        }
+    }
+
+    # -- Step D: Build llama-quantize (optional, best-effort) --
+    if ($BuildOk) {
+        Write-Host ""
+        Write-Host "--- cmake build (llama-quantize) ---" -ForegroundColor Cyan
+        cmake --build $BuildDir --config Release --target llama-quantize -j $NumCpu
+        if ($LASTEXITCODE -ne 0) {
+            Write-Host "   [WARN] llama-quantize build failed (GGUF export may be unavailable)" -ForegroundColor Yellow
+        }
+    }
+
+    # Restore ErrorActionPreference
+    $ErrorActionPreference = $prevEAP
+
+    # Stop timer
+    $totalSw.Stop()
+    $totalMin = [math]::Floor($totalSw.Elapsed.TotalMinutes)
+    $totalSec = [math]::Round($totalSw.Elapsed.TotalSeconds % 60, 1)
+
+    # -- Summary --
+    Write-Host ""
+    if ($BuildOk -and (Test-Path $LlamaServerBin)) {
+        Write-Host "[OK] llama-server built at $LlamaServerBin" -ForegroundColor Green
+        $QuantizeBin = Join-Path $BuildDir "bin\Release\llama-quantize.exe"
+        if (Test-Path $QuantizeBin) {
+            Write-Host "[OK] llama-quantize available for GGUF export" -ForegroundColor Green
+        }
+        Write-Host "   Build time: ${totalMin}m ${totalSec}s" -ForegroundColor Cyan
+    } else {
+        # Check alternate paths (some cmake generators don't use Release subdir)
+        $altBin = Join-Path $BuildDir "bin\llama-server.exe"
+        if ($BuildOk -and (Test-Path $altBin)) {
+            Write-Host "[OK] llama-server built at $altBin" -ForegroundColor Green
+            Write-Host "   Build time: ${totalMin}m ${totalSec}s" -ForegroundColor Cyan
+        } else {
+            Write-Host "[FAILED] llama.cpp build failed at step: $FailedStep (${totalMin}m ${totalSec}s)" -ForegroundColor Red
+            Write-Host "         To retry: delete $LlamaCppDir and re-run setup." -ForegroundColor Yellow
+            exit 1
+        }
+    }
+}
+
+# ============================================
+# Add shell aliases (PowerShell profile + cmd batch files)
+# ============================================
+Write-Host ""
+$RepoDir = $PSScriptRoot
+$VenvPython = Join-Path $RepoDir ".venv\Scripts\python.exe"
+$CliScript = Join-Path $RepoDir "cli.py"
+$FrontendDist = Join-Path $RepoDir "studio\frontend\dist"
+$AliasAdded = $false
+
+# --- PowerShell profile: add functions ---
+$ProfileDir = Split-Path $PROFILE -Parent
+if (-not (Test-Path $ProfileDir)) { New-Item -ItemType Directory -Path $ProfileDir -Force | Out-Null }
+if (-not (Test-Path $PROFILE)) { New-Item -ItemType File -Path $PROFILE -Force | Out-Null }
+
+if (-not (Select-String -Path $PROFILE -Pattern "unsloth-studio" -Quiet -ErrorAction SilentlyContinue)) {
+    $block = @"
+
+# Unsloth Studio launcher
+function unsloth-studio { & "$VenvPython" "$CliScript" studio -f "$FrontendDist" @args }
+function unsloth-ui     { & "$VenvPython" "$CliScript" studio -f "$FrontendDist" @args }
+"@
+    Add-Content -Path $PROFILE -Value $block
+    Write-Host "[OK] Aliases 'unsloth-studio' and 'unsloth-ui' added to $PROFILE" -ForegroundColor Green
+    $AliasAdded = $true
+} else {
+    Write-Host "[OK] Aliases 'unsloth-studio' and 'unsloth-ui' already exist in $PROFILE" -ForegroundColor Green
+}
+
+# --- cmd.exe: create batch files and ensure they're on PATH ---
+$BatDir = Join-Path $RepoDir ".venv\Scripts"
+foreach ($name in @("unsloth-studio", "unsloth-ui")) {
+    $batPath = Join-Path $BatDir "$name.bat"
+    if (-not (Test-Path $batPath)) {
+        Set-Content -Path $batPath -Value "@echo off`r`n`"$VenvPython`" `"$CliScript`" studio -f `"$FrontendDist`" %*"
+    }
+}
+# Persist .venv\Scripts to User PATH so commands work in new cmd.exe terminals without activation
+$userPath = [Environment]::GetEnvironmentVariable('Path', 'User')
+if (-not $userPath -or $userPath -notlike "*$BatDir*") {
+    if ($userPath) {
+        [Environment]::SetEnvironmentVariable('Path', "$BatDir;$userPath", 'User')
+    } else {
+        [Environment]::SetEnvironmentVariable('Path', "$BatDir", 'User')
+    }
+    Write-Host "   Persisted $BatDir to User PATH" -ForegroundColor Gray
+}
+Write-Host "[OK] Batch launchers created (works from any new cmd.exe or PowerShell)" -ForegroundColor Green
+
+# ============================================
+# Done
+# ============================================
+Write-Host ""
+Write-Host "+===============================================+" -ForegroundColor Green
+Write-Host "|           Setup Complete!                     |" -ForegroundColor Green
+Write-Host "|                                               |" -ForegroundColor Green
+Write-Host "|  IMPORTANT: Open a NEW terminal, then run:    |" -ForegroundColor Yellow
+Write-Host "|                                               |" -ForegroundColor Green
+Write-Host "|    unsloth-studio -H 0.0.0.0 -p 8000         |" -ForegroundColor Green
+Write-Host "|                                               |" -ForegroundColor Green
+Write-Host "+===============================================+" -ForegroundColor Green

setup.sh

@@ -69,13 +69,14 @@ if [ "$NEED_NODE" = true ]; then
 
     # Load nvm (source ~/.bashrc won't work inside a script)
     export NVM_DIR="$HOME/.nvm"
+    set +u
     [ -s "$NVM_DIR/nvm.sh" ] && \. "$NVM_DIR/nvm.sh"
 
     # ── 3. Install Node LTS ──
     echo "Installing Node LTS..."
     run_quiet "nvm install" nvm install --lts
     nvm use --lts > /dev/null 2>&1
-
+    set -u
     # ── 4. Verify versions ──
     NODE_MAJOR=$(node -v | sed 's/v//' | cut -d. -f1)
     NPM_MAJOR=$(npm -v | cut -d. -f1)
@@ -105,7 +106,9 @@ echo "✅ Frontend built to studio/frontend/dist"
 echo ""
 echo "Setting up Python environment..."
 
-# ── 6a. Discover best Python <= 3.12.x ──
+# ── 6a. Discover best Python >= 3.11 and < 3.14 (i.e. 3.11.x, 3.12.x, or 3.13.x) ──
+MIN_PY_MINOR=11   # minimum minor version (>= 3.11)
+MAX_PY_MINOR=13   # maximum minor version (< 3.14)
 BEST_PY=""
 BEST_MAJOR=0
 BEST_MINOR=0
@@ -115,7 +118,7 @@ for candidate in $(compgen -c python3 2>/dev/null | grep -E '^python3(\.[0-9]+)?
     if ! command -v "$candidate" &>/dev/null; then
         continue
     fi
-    # Get version string, e.g. "Python 3.11.5"
+    # Get version string, e.g. "Python 3.12.5"
     ver_str=$("$candidate" --version 2>&1 | awk '{print $2}')
     py_major=$(echo "$ver_str" | cut -d. -f1)
     py_minor=$(echo "$ver_str" | cut -d. -f2)
@@ -125,8 +128,13 @@ for candidate in $(compgen -c python3 2>/dev/null | grep -E '^python3(\.[0-9]+)?
         continue
     fi
 
-    # Skip versions above 3.12
-    if [ "$py_minor" -gt 12 ] 2>/dev/null; then
+    # Skip versions below 3.12 (require > 3.11)
+    if [ "$py_minor" -lt "$MIN_PY_MINOR" ] 2>/dev/null; then
+        continue
+    fi
+
+    # Skip versions above 3.13 (require < 3.14)
+    if [ "$py_minor" -gt "$MAX_PY_MINOR" ] 2>/dev/null; then
         continue
     fi
 
@@ -139,7 +147,7 @@ for candidate in $(compgen -c python3 2>/dev/null | grep -E '^python3(\.[0-9]+)?
 done
 
 if [ -z "$BEST_PY" ]; then
-    echo "❌ ERROR: No Python version <= 3.12.x found on this system."
+    echo "❌ ERROR: No Python version between 3.${MIN_PY_MINOR} and 3.${MAX_PY_MINOR} found on this system."
     echo "   Detected Python 3 installations:"
     for candidate in $(compgen -c python3 2>/dev/null | grep -E '^python3(\.[0-9]+)?$' | sort -u); do
         if command -v "$candidate" &>/dev/null; then
@@ -147,52 +155,34 @@ if [ -z "$BEST_PY" ]; then
         fi
     done
     echo ""
-    echo "   Please install Python <= 3.12.x for maximum compatibility."
+    echo "   Please install Python 3.${MIN_PY_MINOR} or 3.${MAX_PY_MINOR}."
     echo "   For example:  sudo apt install python3.12 python3.12-venv"
     exit 1
 fi
 
 BEST_VER=$("$BEST_PY" --version 2>&1 | awk '{print $2}')
-echo "✅ Using $BEST_PY ($BEST_VER) — compatible (≤ 3.12.x)"
+echo "✅ Using $BEST_PY ($BEST_VER) — compatible (3.${MIN_PY_MINOR}.x – 3.${MAX_PY_MINOR}.x)"
+
+REQ_ROOT="$SCRIPT_DIR/studio/backend/requirements"
+SINGLE_ENV_CONSTRAINTS="$REQ_ROOT/single-env/constraints.txt"
+SINGLE_ENV_DATA_DESIGNER="$REQ_ROOT/single-env/data-designer.txt"
+SINGLE_ENV_DATA_DESIGNER_DEPS="$REQ_ROOT/single-env/data-designer-deps.txt"
+SINGLE_ENV_PATCH="$REQ_ROOT/single-env/patch_metadata.py"
+
+install_python_stack() {
+    python "$SCRIPT_DIR/install_python_stack.py"
+}
 
 if [ "$IS_COLAB" = true ]; then
     # Colab: install packages directly without venv
-    run_quiet "pip upgrade" pip install --upgrade pip
-    echo "   Installing unsloth-zoo + unsloth..."
-    run_quiet "pip install unsloth" pip install -r "$SCRIPT_DIR/studio/backend/requirements/base.txt"
-    echo "   Installing additional unsloth dependencies..."
-    run_quiet "pip install extras" pip install --no-cache-dir -r "$SCRIPT_DIR/studio/backend/requirements/extras.txt"
-    run_quiet "pip install extras" pip install --no-deps --no-cache-dir -r "$SCRIPT_DIR/studio/backend/requirements/extras-no-deps.txt"
-    run_quiet "pip install torchao+transformers" pip install --force-reinstall --no-cache-dir -r "$SCRIPT_DIR/studio/backend/requirements/overrides.txt"
-    run_quiet "pip install triton_kernels" pip install --no-deps -r "$SCRIPT_DIR/studio/backend/requirements/triton-kernels.txt"
-    # Patch: override llama_cpp.py with fix from unsloth-zoo branch
-    LLAMA_CPP_DST="$(pip show unsloth-zoo | grep -i '^Location:' | awk '{print $2}')/unsloth_zoo/llama_cpp.py"
-    curl -sSL "https://raw.githubusercontent.com/unslothai/unsloth-zoo/refs/heads/main/unsloth_zoo/llama_cpp.py" \
-        -o "$LLAMA_CPP_DST"
-    echo "   Installing studio dependencies..."
-    run_quiet "pip install studio" pip install -r "$SCRIPT_DIR/studio/backend/requirements/studio.txt"
-    echo "✅ Python dependencies installed"
+    install_python_stack
 else
     # Local: create venv (always start fresh to preserve correct install order)
     rm -rf .venv
     rm -rf .venv_overlay  # Clean up stale transformers version overlay
     "$BEST_PY" -m venv .venv
     source .venv/bin/activate
-    run_quiet "pip upgrade" pip install --upgrade pip
-    echo "   Installing unsloth-zoo + unsloth..."
-    run_quiet "pip install unsloth" pip install -r "$SCRIPT_DIR/studio/backend/requirements/base.txt"
-    echo "   Installing additional unsloth dependencies..."
-    run_quiet "pip install extras" pip install --no-cache-dir -r "$SCRIPT_DIR/studio/backend/requirements/extras.txt"
-    run_quiet "pip install extras" pip install --no-deps --no-cache-dir -r "$SCRIPT_DIR/studio/backend/requirements/extras-no-deps.txt"
-    run_quiet "pip install torchao+transformers" pip install --force-reinstall --no-cache-dir -r "$SCRIPT_DIR/studio/backend/requirements/overrides.txt"
-    run_quiet "pip install triton_kernels" pip install --no-deps -r "$SCRIPT_DIR/studio/backend/requirements/triton-kernels.txt"
-    # Patch: override llama_cpp.py with fix from unsloth-zoo branch
-    LLAMA_CPP_DST="$(pip show unsloth-zoo | grep -i '^Location:' | awk '{print $2}')/unsloth_zoo/llama_cpp.py"
-    curl -sSL "https://raw.githubusercontent.com/unslothai/unsloth-zoo/refs/heads/main/unsloth_zoo/llama_cpp.py" \
-        -o "$LLAMA_CPP_DST"
-    echo "   Installing studio dependencies..."
-    run_quiet "pip install studio" pip install -r "$SCRIPT_DIR/studio/backend/requirements/studio.txt"
-    echo "✅ Python dependencies installed"
+    install_python_stack
     
     # ── 7. WSL: pre-install GGUF build dependencies ──
     # On WSL, sudo requires a password and can't be entered during GGUF export
@@ -207,7 +197,93 @@ else
     fi
 fi
 
-# ── 8. Add shell alias (skip in Colab) ──
+# ── 8. Build llama.cpp binaries for GGUF inference + export ──
+# Builds at ~/.unsloth/llama.cpp — a single shared location under the user's
+# home directory. This is used by both the inference server and the GGUF
+# export pipeline (unsloth-zoo).
+#   - llama-server: for GGUF model inference
+#   - llama-quantize: for GGUF export quantization (symlinked to root for check_llama_cpp())
+UNSLOTH_HOME="$HOME/.unsloth"
+mkdir -p "$UNSLOTH_HOME"
+LLAMA_CPP_DIR="$UNSLOTH_HOME/llama.cpp"
+LLAMA_SERVER_BIN="$LLAMA_CPP_DIR/build/bin/llama-server"
+rm -rf "$LLAMA_CPP_DIR"
+{
+    # Check prerequisites
+    if ! command -v cmake &>/dev/null; then
+        echo ""
+        echo "⚠️  cmake not found — skipping llama-server build (GGUF inference won't be available)"
+        echo "   Install cmake and re-run setup.sh to enable GGUF inference."
+    elif ! command -v git &>/dev/null; then
+        echo ""
+        echo "⚠️  git not found — skipping llama-server build (GGUF inference won't be available)"
+    else
+        echo ""
+        echo "Building llama-server for GGUF inference..."
+
+        BUILD_OK=true
+        run_quiet "clone llama.cpp" git clone --depth 1 https://github.com/ggml-org/llama.cpp.git "$LLAMA_CPP_DIR" || BUILD_OK=false
+
+        if [ "$BUILD_OK" = true ]; then
+            CMAKE_ARGS=""
+            # Detect CUDA: check nvcc on PATH, then common install locations
+            NVCC_PATH=""
+            if command -v nvcc &>/dev/null; then
+                NVCC_PATH="$(command -v nvcc)"
+            elif [ -x /usr/local/cuda/bin/nvcc ]; then
+                NVCC_PATH="/usr/local/cuda/bin/nvcc"
+                export PATH="/usr/local/cuda/bin:$PATH"
+            elif ls /usr/local/cuda-*/bin/nvcc &>/dev/null 2>&1; then
+                # Pick the newest cuda-XX.X directory
+                NVCC_PATH="$(ls -d /usr/local/cuda-*/bin/nvcc 2>/dev/null | sort -V | tail -1)"
+                export PATH="$(dirname "$NVCC_PATH"):$PATH"
+            fi
+
+            if [ -n "$NVCC_PATH" ]; then
+                echo "   Building with CUDA support (nvcc: $NVCC_PATH)..."
+                CMAKE_ARGS="-DGGML_CUDA=ON"
+            elif [ -d /usr/local/cuda ] || nvidia-smi &>/dev/null; then
+                echo "   CUDA driver detected but nvcc not found — building CPU-only"
+                echo "   To enable GPU: install cuda-toolkit or add nvcc to PATH"
+            else
+                echo "   Building CPU-only (no CUDA detected)..."
+            fi
+
+            NCPU=$(nproc 2>/dev/null || sysctl -n hw.ncpu 2>/dev/null || echo 4)
+
+            run_quiet "cmake llama.cpp" cmake -S "$LLAMA_CPP_DIR" -B "$LLAMA_CPP_DIR/build" $CMAKE_ARGS || BUILD_OK=false
+        fi
+
+        if [ "$BUILD_OK" = true ]; then
+            run_quiet "build llama-server" cmake --build "$LLAMA_CPP_DIR/build" --config Release --target llama-server -j"$NCPU" || BUILD_OK=false
+        fi
+
+        # Also build llama-quantize (needed by unsloth-zoo's GGUF export pipeline)
+        if [ "$BUILD_OK" = true ]; then
+            run_quiet "build llama-quantize" cmake --build "$LLAMA_CPP_DIR/build" --config Release --target llama-quantize -j"$NCPU" || true
+            # Symlink to llama.cpp root — check_llama_cpp() looks for the binary there
+            QUANTIZE_BIN="$LLAMA_CPP_DIR/build/bin/llama-quantize"
+            if [ -f "$QUANTIZE_BIN" ]; then
+                ln -sf build/bin/llama-quantize "$LLAMA_CPP_DIR/llama-quantize"
+            fi
+        fi
+
+        if [ "$BUILD_OK" = true ]; then
+            if [ -f "$LLAMA_SERVER_BIN" ]; then
+                echo "✅ llama-server built at $LLAMA_SERVER_BIN"
+            else
+                echo "⚠️  llama-server binary not found after build — GGUF inference won't be available"
+            fi
+            if [ -f "$LLAMA_CPP_DIR/llama-quantize" ]; then
+                echo "✅ llama-quantize available for GGUF export"
+            fi
+        else
+            echo "⚠️  llama-server build failed — GGUF inference won't be available, but everything else works"
+        fi
+    fi
+}
+
+# ── 9. Add shell alias (skip in Colab) ──
 # Note: venv activation does NOT persist across terminal sessions.
 # This alias hardcodes the venv python path so users don't need to activate.
 if [ "$IS_COLAB" = false ]; then

studio/backend/auth/authentication.py

@@ -1,5 +1,5 @@
 import secrets
-from datetime import UTC, datetime, timedelta
+from datetime import datetime, timedelta, timezone
 from typing import Optional
 
 from fastapi import Depends, HTTPException, status
@@ -34,7 +34,7 @@ def create_access_token(
     Tokens are valid across restarts because SECRET_KEY is stored in SQLite.
     """
     to_encode = {"sub": subject}
-    expire = datetime.now(UTC) + (
+    expire = datetime.now(timezone.utc) + (
         expires_delta or timedelta(minutes=ACCESS_TOKEN_EXPIRE_MINUTES)
     )
     to_encode.update({"exp": expire})
@@ -48,7 +48,7 @@ def create_refresh_token(subject: str) -> str:
     Refresh tokens are opaque (not JWTs) and expire after REFRESH_TOKEN_EXPIRE_DAYS.
     """
     token = secrets.token_urlsafe(48)
-    expires_at = datetime.now(UTC) + timedelta(days=REFRESH_TOKEN_EXPIRE_DAYS)
+    expires_at = datetime.now(timezone.utc) + timedelta(days=REFRESH_TOKEN_EXPIRE_DAYS)
     save_refresh_token(token, subject, expires_at.isoformat())
     return token
 

studio/backend/auth/storage.py

@@ -3,7 +3,7 @@ SQLite storage for authentication data (user credentials + JWT secret).
 """
 import hashlib
 import sqlite3
-from datetime import UTC, datetime
+from datetime import datetime, timezone
 from pathlib import Path
 from typing import Optional, Tuple
 
@@ -218,7 +218,7 @@ def verify_refresh_token(token: str) -> Optional[str]:
         # Clean up any expired tokens while we're here
         conn.execute(
             "DELETE FROM refresh_tokens WHERE expires_at < ?",
-            (datetime.now(UTC).isoformat(),),
+            (datetime.now(timezone.utc).isoformat(),),
         )
         conn.commit()
 
@@ -235,7 +235,7 @@ def verify_refresh_token(token: str) -> Optional[str]:
 
         # Check expiry
         expires_at = datetime.fromisoformat(row["expires_at"])
-        if datetime.now(UTC) > expires_at:
+        if datetime.now(timezone.utc) > expires_at:
             conn.execute("DELETE FROM refresh_tokens WHERE id = ?", (row["id"],))
             conn.commit()
             return None

studio/backend/core/data_recipe/__init__.py

@@ -0,0 +1,7 @@
+"""
+Data Recipe core (DataDesigner wrapper + job runner).
+"""
+
+from .jobs import JobManager, get_job_manager
+
+__all__ = ["JobManager", "get_job_manager"]

studio/backend/core/data_recipe/jobs/__init__.py

@@ -0,0 +1,4 @@
+from .manager import JobManager, get_job_manager
+
+__all__ = ["JobManager", "get_job_manager"]
+

studio/backend/core/data_recipe/jobs/constants.py

@@ -0,0 +1,30 @@
+from __future__ import annotations
+
+# stages parsed from data-designer logs
+STAGE_CREATE = "create"
+STAGE_PREVIEW = "preview"
+STAGE_DAG = "dag"
+STAGE_HEALTHCHECK = "healthcheck"
+STAGE_SAMPLING = "sampling"
+STAGE_COLUMN_CONFIG = "column_config"
+STAGE_GENERATING = "generating"
+STAGE_BATCH = "batch"
+STAGE_PROFILING = "profiling"
+
+USAGE_RESET_STAGES = {
+    STAGE_CREATE,
+    STAGE_PREVIEW,
+    STAGE_DAG,
+    STAGE_HEALTHCHECK,
+    STAGE_SAMPLING,
+    STAGE_GENERATING,
+    STAGE_PROFILING,
+}
+
+# job event types emitted by worker/manager
+EVENT_JOB_ENQUEUED = "job.enqueued"
+EVENT_JOB_STARTED = "job.started"
+EVENT_JOB_CANCELLING = "job.cancelling"
+EVENT_JOB_CANCELLED = "job.cancelled"
+EVENT_JOB_COMPLETED = "job.completed"
+EVENT_JOB_ERROR = "job.error"

studio/backend/core/data_recipe/jobs/manager.py

@@ -0,0 +1,452 @@
+from __future__ import annotations
+
+import asyncio
+import json
+import queue
+import threading
+import time
+import uuid
+from pathlib import Path
+from collections import deque
+from dataclasses import dataclass
+from typing import Any
+
+import multiprocessing as mp
+
+from ..jsonable import to_jsonable
+from .constants import (
+    EVENT_JOB_CANCELLING,
+    EVENT_JOB_CANCELLED,
+    EVENT_JOB_COMPLETED,
+    EVENT_JOB_ENQUEUED,
+    EVENT_JOB_ERROR,
+    EVENT_JOB_STARTED,
+)
+from .parse import apply_update, coerce_event, parse_log_message
+from .types import Job
+from .worker import run_job_process
+
+
+_CTX = mp.get_context("spawn")
+
+
+@dataclass
+class Subscription:
+    replay: list[dict]
+    _q: queue.Queue
+    _next_id: int = 0
+
+    async def next_event(self, *, timeout_sec: float) -> dict | None:
+        """Wait for next event (SSE), w/ timeout so we can check disconnects."""
+        try:
+            return await asyncio.to_thread(self._q.get, True, timeout_sec)
+        except queue.Empty:
+            return None
+
+    def format_sse(self, event: dict) -> bytes:
+        """Turn event dict into SSE bytes (id/event/data)."""
+        event_id = event.get("seq")
+        if event_id is None:
+            self._next_id += 1
+            event_id = self._next_id
+        body = json.dumps(event, separators=(",", ":"), ensure_ascii=False)
+        event_type = event.get("type") or "message"
+        return (
+            f"id: {event_id}\n"
+            f"event: {event_type}\n"
+            f"data: {body}\n\n"
+        ).encode("utf-8")
+
+
+class JobManager:
+    def __init__(self) -> None:
+        """Single-job runner (in-mem). Simple on purpose, not a whole platform."""
+        self._lock = threading.Lock()
+        self._job: Job | None = None
+        self._proc: mp.Process | None = None
+        self._mp_q: Any | None = None
+        self._events: deque[dict] = deque(maxlen=5000)
+        self._subs: list[queue.Queue] = []
+        self._pump_thread: threading.Thread | None = None
+        self._seq: int = 0
+
+    def start(self, *, recipe: dict, run: dict) -> str:
+        """Spawn the job subprocess (one at a time, no cap)."""
+        llm_columns = recipe.get("columns") or []
+        llm_column_count = 0
+        if isinstance(llm_columns, list):
+            for column in llm_columns:
+                if not isinstance(column, dict):
+                    continue
+                column_type = str(column.get("column_type") or "").strip().lower()
+                if column_type.startswith("llm"):
+                    llm_column_count += 1
+        if llm_column_count <= 0:
+            llm_column_count = 1
+
+        with self._lock:
+            if self._proc is not None and self._proc.is_alive():
+                raise RuntimeError("job already running")
+
+            job_id = uuid.uuid4().hex
+            self._job = Job(job_id=job_id, status="pending", started_at=time.time())
+            self._job.progress_columns_total = llm_column_count
+            self._events.clear()
+            self._seq = 0
+
+            run_payload = dict(run)
+            run_payload["_job_id"] = job_id
+            mp_q = _CTX.Queue()
+            proc = _CTX.Process(
+                target=run_job_process,
+                kwargs={"event_queue": mp_q, "recipe": recipe, "run": run_payload},
+                daemon=True,
+            )
+            proc.start()
+
+            self._mp_q = mp_q
+            self._proc = proc
+            self._pump_thread = threading.Thread(target=self._pump_loop, daemon=True)
+            self._pump_thread.start()
+
+            self._emit({"type": EVENT_JOB_ENQUEUED, "ts": time.time(), "job_id": job_id})
+            return job_id
+
+    def cancel(self, job_id: str) -> bool:
+        """Hard stop. We terminate the subprocess. Quick + reliable."""
+        with self._lock:
+            if self._job is None or self._job.job_id != job_id:
+                return False
+            if self._proc is None or not self._proc.is_alive():
+                return True
+            self._job.status = "cancelling"
+            self._emit({"type": EVENT_JOB_CANCELLING, "ts": time.time(), "job_id": job_id})
+            try:
+                self._proc.terminate()
+            except (AttributeError, OSError):
+                pass
+            return True
+
+    def get_status(self, job_id: str) -> dict | None:
+        """UI friendly snapshot that we need. Alternative to sse kinda of and structured"""
+        with self._lock:
+            if self._job is None or self._job.job_id != job_id:
+                return None
+            job = self._job
+            return {
+                "job_id": job.job_id,
+                "status": job.status,
+                "stage": job.stage,
+                "current_column": job.current_column,
+                "batch": {"idx": job.batch.idx, "total": job.batch.total},
+                "progress": {
+                    "done": job.progress.done,
+                    "total": job.progress.total,
+                    "percent": job.progress.percent,
+                    "eta_sec": job.progress.eta_sec,
+                    "rate": job.progress.rate,
+                    "ok": job.progress.ok,
+                    "failed": job.progress.failed,
+                },
+                "column_progress": {
+                    "done": job.column_progress.done,
+                    "total": job.column_progress.total,
+                    "percent": job.column_progress.percent,
+                    "eta_sec": job.column_progress.eta_sec,
+                    "rate": job.column_progress.rate,
+                    "ok": job.column_progress.ok,
+                    "failed": job.column_progress.failed,
+                },
+                "model_usage": {
+                    name: {
+                        "model": usage.model,
+                        "tokens": {
+                            "input": usage.input_tokens,
+                            "output": usage.output_tokens,
+                            "total": usage.total_tokens,
+                            "tps": usage.tps,
+                        },
+                        "requests": {
+                            "success": usage.requests_success,
+                            "failed": usage.requests_failed,
+                            "total": usage.requests_total,
+                            "rpm": usage.rpm,
+                        },
+                    }
+                    for name, usage in job.model_usage.items()
+                },
+                "rows": job.rows,
+                "cols": job.cols,
+                "error": job.error,
+                "has_analysis": job.analysis is not None,
+                "dataset_rows": None if job.dataset is None else len(job.dataset),
+                "artifact_path": job.artifact_path,
+                "started_at": job.started_at,
+                "finished_at": job.finished_at,
+            }
+
+    def get_current_status(self) -> dict | None:
+        """Single-job convenience (last/current)."""
+        job_id = self.get_current_job_id()
+        if job_id is None:
+            return None
+        return self.get_status(job_id)
+
+    def get_current_job_id(self) -> str | None:
+        """Return current job_id (or None)."""
+        with self._lock:
+            return None if self._job is None else self._job.job_id
+
+    def get_analysis(self, job_id: str) -> dict | None:
+        """Final profiling output (only after job completes)."""
+        with self._lock:
+            if self._job is None or self._job.job_id != job_id:
+                return None
+            return self._job.analysis
+
+    def get_dataset(
+        self,
+        job_id: str,
+        *,
+        limit: int,
+        offset: int = 0,
+    ) -> dict[str, Any] | None:
+        """Load dataset page (offset + limit) and include total rows."""
+        with self._lock:
+            if self._job is None or self._job.job_id != job_id:
+                return None
+            in_memory_dataset = self._job.dataset
+            artifact_path = self._job.artifact_path
+            job_status = self._job.status
+
+        if in_memory_dataset is not None:
+            total = len(in_memory_dataset)
+            rows = in_memory_dataset[offset:offset + limit]
+            return {"dataset": rows, "total": total}
+        if not artifact_path:
+            if job_status in {"completed", "error", "cancelled"}:
+                return {"error": "artifact path missing"}
+            return None
+
+        try:
+            base_dataset_path = Path(artifact_path)
+            parquet_dir = base_dataset_path / "parquet-files"
+            if not parquet_dir.exists():
+                return {"error": f"dataset path missing: {parquet_dir}"}
+
+            return self._load_dataset_page(parquet_dir=parquet_dir, limit=limit, offset=offset)
+        except Exception as exc:
+            return {"error": f"dataset load failed: {exc}"}
+
+    @staticmethod
+    def _load_dataset_page(
+        *,
+        parquet_dir: Path,
+        limit: int,
+        offset: int,
+    ) -> dict[str, Any]:
+        dataset_page = JobManager._load_dataset_page_with_duckdb(
+            parquet_dir=parquet_dir,
+            limit=limit,
+            offset=offset,
+        )
+        if dataset_page is not None:
+            return dataset_page
+        return JobManager._load_dataset_page_with_data_designer(
+            parquet_dir=parquet_dir,
+            limit=limit,
+            offset=offset,
+        )
+
+    @staticmethod
+    def _load_dataset_page_with_duckdb(
+        *,
+        parquet_dir: Path,
+        limit: int,
+        offset: int,
+    ) -> dict[str, Any] | None:
+        parquet_glob = str((parquet_dir / "*.parquet").resolve())
+        try:
+            import duckdb  # type: ignore
+        except Exception:
+            return None
+
+        try:
+            conn = duckdb.connect(":memory:")
+            try:
+                total_row = conn.execute(
+                    "SELECT COUNT(*) FROM read_parquet(?)",
+                    [parquet_glob],
+                ).fetchone()
+                total = int(total_row[0] if total_row else 0)
+                dataframe = conn.execute(
+                    (
+                        "SELECT *, row_number() OVER (PARTITION BY filename) AS __row_num__ "
+                        "FROM read_parquet(?, filename=true) "
+                        "ORDER BY filename, __row_num__ "
+                        "LIMIT ? OFFSET ?"
+                    ),
+                    [parquet_glob, int(limit), int(offset)],
+                ).fetchdf()
+            finally:
+                conn.close()
+        except (RuntimeError, ValueError, duckdb.Error):
+            return None
+
+        for helper_col in ("filename", "__row_num__"):
+            if helper_col in dataframe.columns:
+                dataframe = dataframe.drop(columns=[helper_col])
+
+        rows = dataframe.to_dict(orient="records")
+        return {"dataset": to_jsonable(rows), "total": total}
+
+    @staticmethod
+    def _load_dataset_page_with_data_designer(
+        *,
+        parquet_dir: Path,
+        limit: int,
+        offset: int,
+    ) -> dict[str, Any]:
+        from data_designer.config.utils.io_helpers import read_parquet_dataset
+
+        dataframe = read_parquet_dataset(parquet_dir)
+        total = int(len(dataframe.index))
+        rows = dataframe.iloc[offset:offset + limit].to_dict(orient="records")
+        return {"dataset": to_jsonable(rows), "total": total}
+
+    def subscribe(self, job_id: str, *, after_seq: int | None = None) -> Subscription | None:
+        """SSE subscribe: get replay buffer + live events stream."""
+        with self._lock:
+            if self._job is None or self._job.job_id != job_id:
+                return None
+            q: queue.Queue = queue.Queue(maxsize=2000)
+            self._subs.append(q)
+            if after_seq is None:
+                replay = list(self._events)
+            else:
+                replay = [e for e in self._events if int(e.get("seq") or 0) > after_seq]
+            return Subscription(replay=replay, _q=q)
+
+    def unsubscribe(self, sub: Subscription) -> None:
+        """Drop SSE subscriber (client disconnected)."""
+        with self._lock:
+            self._subs = [q for q in self._subs if q is not sub._q]
+
+    def _emit(self, event: dict) -> None:
+        """Broadcast event to replay buffer + all subscribers."""
+        self._seq += 1
+        event["seq"] = self._seq
+        self._events.append(event)
+        stale: list[queue.Queue] = []
+        for q in self._subs:
+            try:
+                q.put_nowait(event)
+            except queue.Full:
+                stale.append(q)
+        if stale:
+            self._subs = [q for q in self._subs if q not in stale]
+
+    def _snapshot(self) -> tuple[Job, mp.Process, Any] | None:
+        """Grab pointers for the pump loop (avoid holding lock too long)."""
+        with self._lock:
+            if self._job is None or self._proc is None or self._mp_q is None:
+                return None
+            return self._job, self._proc, self._mp_q
+
+    @staticmethod
+    def _read_queue_with_timeout(q: Any, *, timeout_sec: float) -> dict | None:
+        """Try read 1 event from mp queue. Timeout = pump stays responsive."""
+        try:
+            return coerce_event(q.get(timeout=timeout_sec))
+        except queue.Empty:
+            return None
+        except (EOFError, OSError, ValueError):
+            return None
+
+    @staticmethod
+    def _drain_queue(q: Any) -> list[dict]:
+        """Drain mp queue fast (used on process exit)."""
+        events: list[dict] = []
+        while True:
+            try:
+                events.append(coerce_event(q.get_nowait()))
+            except queue.Empty:
+                return events
+            except (EOFError, OSError, ValueError):
+                return events
+
+    def _pump_loop(self) -> None:
+        """Background thread: consumes worker events + updates job snapshot."""
+        while True:
+            snap = self._snapshot()
+            if snap is None:
+                return
+            job, proc, mp_q = snap
+
+            event = self._read_queue_with_timeout(mp_q, timeout_sec=0.25)
+            if event is not None:
+                self._handle_event(job, event)
+                continue
+
+            if proc.is_alive():
+                continue
+
+            for e in self._drain_queue(mp_q):
+                self._handle_event(job, e)
+
+            with self._lock:
+                if self._job and self._job.status in {"pending", "active", "cancelling"}:
+                    if self._job.status == "cancelling":
+                        self._job.status = "cancelled"
+                    else:
+                        self._job.status = "error"
+                        self._job.error = self._job.error or "process exited"
+                    self._job.finished_at = time.time()
+                    event_type = (
+                        EVENT_JOB_CANCELLED if self._job.status == "cancelled" else EVENT_JOB_ERROR
+                    )
+                    self._emit({"type": event_type, "ts": time.time(), "job_id": self._job.job_id})
+            return
+
+    def _handle_event(self, job: Job, event: dict) -> None:
+        """Apply event -> job state + forward to SSE."""
+        et = event.get("type")
+        msg = event.get("message") if et == "log" else None
+
+        with self._lock:
+            if self._job is None or self._job.job_id != job.job_id:
+                return
+            if et == EVENT_JOB_STARTED:
+                self._job.status = "active"
+            if et == EVENT_JOB_COMPLETED:
+                self._job.status = "completed"
+                self._job.finished_at = time.time()
+                self._job.analysis = event.get("analysis")
+                self._job.artifact_path = event.get("artifact_path")
+                self._job.dataset = event.get("dataset")
+                self._job.processor_artifacts = event.get("processor_artifacts")
+                if self._job.progress.total and self._job.progress.total > 0:
+                    self._job.progress.done = self._job.progress.total
+                    self._job.progress.percent = 100.0
+            if et == EVENT_JOB_ERROR:
+                self._job.status = "error"
+                self._job.finished_at = time.time()
+                self._job.error = event.get("error") or "error"
+
+            if msg:
+                upd = parse_log_message(msg)
+                if upd:
+                    apply_update(self._job, upd)
+
+        self._emit(event)
+
+
+_JOB_MANAGER: JobManager | None = None
+
+
+def get_job_manager() -> JobManager:
+    """Singleton JobManager (we only run 1 job anyway)."""
+    global _JOB_MANAGER
+    if _JOB_MANAGER is None:
+        _JOB_MANAGER = JobManager()
+    return _JOB_MANAGER

studio/backend/core/data_recipe/jobs/parse.py

@@ -0,0 +1,244 @@
+from __future__ import annotations
+
+import re
+from dataclasses import dataclass
+from typing import Any
+
+from .constants import (
+    STAGE_BATCH,
+    STAGE_COLUMN_CONFIG,
+    STAGE_CREATE,
+    STAGE_DAG,
+    STAGE_GENERATING,
+    STAGE_HEALTHCHECK,
+    STAGE_PREVIEW,
+    STAGE_PROFILING,
+    STAGE_SAMPLING,
+    USAGE_RESET_STAGES,
+)
+from .types import Job, ModelUsage, Progress
+
+
+@dataclass(frozen=True)
+class ParsedUpdate:
+    stage: str | None = None
+    current_column: str | None = None
+    progress: Progress | None = None
+    rows: int | None = None
+    cols: int | None = None
+    batch_idx: int | None = None
+    batch_total: int | None = None
+    usage_model: str | None = None
+    usage_input_tokens: int | None = None
+    usage_output_tokens: int | None = None
+    usage_total_tokens: int | None = None
+    usage_tps: float | None = None
+    usage_requests_success: int | None = None
+    usage_requests_failed: int | None = None
+    usage_requests_total: int | None = None
+    usage_rpm: float | None = None
+    usage_section_start: bool | None = None
+
+# kinda of a bummber but currently only option, Best effort parser from data-designer logs -> structured status for UI.
+_RE_SAMPLERS = re.compile(
+    r"Preparing samplers to generate (?P<rows>\d+) records across (?P<cols>\d+) columns"
+)
+_RE_COLCFG = re.compile(r"model config for column '(?P<col>[^']+)'")
+_RE_PROCESSING_COL = re.compile(r"Processing .* column '(?P<col>[^']+)'")
+_RE_PROGRESS = re.compile(
+    r"progress: (?P<done>\d+)/(?P<total>\d+) \((?P<pct>\d+)%\) complete, "
+    r"(?P<ok>\d+) ok, (?P<failed>\d+) failed, (?P<rate>[0-9.]+) rec/s, eta (?P<eta>[0-9.]+)s"
+)
+_RE_BATCH = re.compile(r"Processing batch (?P<idx>\d+) of (?P<total>\d+)")
+_RE_USAGE_MODEL = re.compile(r"model:\s*(?P<model>.+)$")
+_RE_USAGE_TOKENS = re.compile(
+    r"tokens:\s*input=(?P<input>\d+),\s*output=(?P<output>\d+),\s*total=(?P<total>\d+),\s*tps=(?P<tps>[0-9.]+)"
+)
+_RE_USAGE_REQUESTS = re.compile(
+    r"requests:\s*success=(?P<success>\d+),\s*failed=(?P<failed>\d+),\s*total=(?P<total>\d+),\s*rpm=(?P<rpm>[0-9.]+)"
+)
+
+
+def parse_log_message(msg: str) -> ParsedUpdate | None:
+    m = _RE_SAMPLERS.search(msg)
+    if m:
+        return ParsedUpdate(
+            stage=STAGE_SAMPLING,
+            rows=int(m.group("rows")),
+            cols=int(m.group("cols")),
+        )
+
+    if "Sorting column configs into a Directed Acyclic Graph" in msg:
+        return ParsedUpdate(stage=STAGE_DAG)
+    if "Running health checks for models" in msg:
+        return ParsedUpdate(stage=STAGE_HEALTHCHECK)
+    if "Preview generation in progress" in msg:
+        return ParsedUpdate(stage=STAGE_PREVIEW)
+    if "Creating Data Designer dataset" in msg:
+        return ParsedUpdate(stage=STAGE_CREATE)
+    if "Measuring dataset column statistics" in msg:
+        return ParsedUpdate(stage=STAGE_PROFILING)
+
+    m = _RE_COLCFG.search(msg)
+    if m:
+        col = m.group("col")
+        return ParsedUpdate(stage=STAGE_COLUMN_CONFIG, current_column=col)
+
+    m = _RE_PROCESSING_COL.search(msg)
+    if m:
+        col = m.group("col")
+        return ParsedUpdate(stage=STAGE_GENERATING, current_column=col)
+
+    m = _RE_PROGRESS.search(msg)
+    if m:
+        p = Progress(
+            done=int(m.group("done")),
+            total=int(m.group("total")),
+            percent=float(m.group("pct")),
+            ok=int(m.group("ok")),
+            failed=int(m.group("failed")),
+            rate=float(m.group("rate")),
+            eta_sec=float(m.group("eta")),
+        )
+        return ParsedUpdate(stage=STAGE_GENERATING, progress=p)
+
+    m = _RE_BATCH.search(msg)
+    if m:
+        return ParsedUpdate(
+            stage=STAGE_BATCH,
+            batch_idx=int(m.group("idx")),
+            batch_total=int(m.group("total")),
+        )
+
+    if "Model usage summary" in msg:
+        return ParsedUpdate(usage_section_start=True)
+
+    m = _RE_USAGE_MODEL.search(msg)
+    if m and "|-- model:" in msg:
+        return ParsedUpdate(usage_model=str(m.group("model")).strip())
+
+    m = _RE_USAGE_TOKENS.search(msg)
+    if m:
+        return ParsedUpdate(
+            usage_input_tokens=int(m.group("input")),
+            usage_output_tokens=int(m.group("output")),
+            usage_total_tokens=int(m.group("total")),
+            usage_tps=float(m.group("tps")),
+        )
+
+    m = _RE_USAGE_REQUESTS.search(msg)
+    if m:
+        return ParsedUpdate(
+            usage_requests_success=int(m.group("success")),
+            usage_requests_failed=int(m.group("failed")),
+            usage_requests_total=int(m.group("total")),
+            usage_rpm=float(m.group("rpm")),
+        )
+
+    return None
+
+
+def apply_update(job: Job, update: ParsedUpdate) -> None:
+    if update.stage is not None:
+        job.stage = update.stage
+    if update.current_column is not None:
+        job.current_column = update.current_column
+        if update.stage == STAGE_GENERATING and update.current_column not in job._seen_generation_columns:
+            job._seen_generation_columns.append(update.current_column)
+    if update.rows is not None:
+        job.rows = update.rows
+    if update.cols is not None:
+        job.cols = update.cols
+    if update.progress is not None:
+        job.column_progress = update.progress
+        job.progress = _compute_overall_progress(job, update.progress)
+    if update.batch_idx is not None:
+        job.batch.idx = update.batch_idx
+    if update.batch_total is not None:
+        job.batch.total = update.batch_total
+
+    if update.stage in USAGE_RESET_STAGES:
+        # usage summary is a short block so we reset once we move into the next stage.
+        job._in_usage_summary = False
+
+    if update.usage_section_start is not None:
+        job._in_usage_summary = update.usage_section_start
+        if update.usage_section_start:
+            job._current_usage_model = None
+
+    if not job._in_usage_summary:
+        return
+
+    if update.usage_model is not None:
+        name = update.usage_model.strip().strip("'").strip('"')
+        job._current_usage_model = name
+        if name not in job.model_usage:
+            job.model_usage[name] = ModelUsage(model=name)
+
+    if job._current_usage_model is None:
+        return
+
+    usage = job.model_usage.get(job._current_usage_model)
+    if usage is None:
+        return
+
+    if update.usage_input_tokens is not None:
+        usage.input_tokens = update.usage_input_tokens
+    if update.usage_output_tokens is not None:
+        usage.output_tokens = update.usage_output_tokens
+    if update.usage_total_tokens is not None:
+        usage.total_tokens = update.usage_total_tokens
+    if update.usage_tps is not None:
+        usage.tps = update.usage_tps
+    if update.usage_requests_success is not None:
+        usage.requests_success = update.usage_requests_success
+    if update.usage_requests_failed is not None:
+        usage.requests_failed = update.usage_requests_failed
+    if update.usage_requests_total is not None:
+        usage.requests_total = update.usage_requests_total
+    if update.usage_rpm is not None:
+        usage.rpm = update.usage_rpm
+
+
+def _compute_overall_progress(job: Job, column_progress: Progress) -> Progress:
+    if not job.rows:
+        return column_progress
+
+    total_rows = max(1, int(job.rows))
+    current_done = 0 if column_progress.done is None else int(column_progress.done)
+    current_done = max(0, min(current_done, total_rows))
+    total_columns = max(1, int(job.progress_columns_total or 1))
+
+    if job.current_column:
+        job._column_done[job.current_column] = current_done
+
+    if len(job._column_done) == 0:
+        done = current_done
+    else:
+        sum_done = sum(max(0, min(value, total_rows)) for value in job._column_done.values())
+        done = int(sum_done / total_columns)
+
+    prev_done = int(job.progress.done or 0)
+    if done < prev_done:
+        done = prev_done
+    if done > total_rows:
+        done = total_rows
+    percent = (done / total_rows) * 100 if total_rows > 0 else 100.0
+    prev_percent = float(job.progress.percent or 0.0)
+    if percent < prev_percent:
+        percent = prev_percent
+
+    return Progress(
+        done=done,
+        total=total_rows,
+        percent=percent,
+        eta_sec=column_progress.eta_sec,
+        rate=column_progress.rate,
+        ok=column_progress.ok,
+        failed=column_progress.failed,
+    )
+
+
+def coerce_event(obj: Any) -> dict:
+    # worker sends dict already
+    return obj if isinstance(obj, dict) else {"type": "log", "message": str(obj)}

studio/backend/core/data_recipe/jobs/types.py

@@ -0,0 +1,72 @@
+from __future__ import annotations
+
+from dataclasses import dataclass, field
+from typing import Any, Literal
+
+
+JobStatus = Literal[
+    "created",
+    "pending",
+    "active",
+    "cancelling",
+    "cancelled",
+    "error",
+    "completed",
+]
+
+
+@dataclass
+class Progress:
+    done: int | None = None
+    total: int | None = None
+    percent: float | None = None
+    eta_sec: float | None = None
+    rate: float | None = None
+    ok: int | None = None
+    failed: int | None = None
+
+
+@dataclass
+class BatchProgress:
+    idx: int | None = None
+    total: int | None = None
+
+
+@dataclass
+class ModelUsage:
+    model: str
+    input_tokens: int | None = None
+    output_tokens: int | None = None
+    total_tokens: int | None = None
+    tps: float | None = None
+    requests_success: int | None = None
+    requests_failed: int | None = None
+    requests_total: int | None = None
+    rpm: float | None = None
+
+
+@dataclass
+class Job:
+    job_id: str
+    status: JobStatus = "created"
+    stage: str | None = None
+    current_column: str | None = None
+    progress: Progress = field(default_factory=Progress)
+    column_progress: Progress = field(default_factory=Progress)
+    batch: BatchProgress = field(default_factory=BatchProgress)
+    rows: int | None = None
+    cols: int | None = None
+    error: str | None = None
+    started_at: float | None = None
+    finished_at: float | None = None
+
+    analysis: dict[str, Any] | None = None
+    artifact_path: str | None = None
+    dataset: list[dict[str, Any]] | None = None
+    processor_artifacts: dict[str, Any] | None = None
+    model_usage: dict[str, ModelUsage] = field(default_factory=dict)
+    progress_columns_total: int | None = None
+    _current_usage_model: str | None = None
+    _in_usage_summary: bool = False
+    _seen_generation_columns: list[str] = field(default_factory=list)
+    _column_done: dict[str, int] = field(default_factory=dict)

studio/backend/core/data_recipe/jobs/worker.py

@@ -0,0 +1,145 @@
+from __future__ import annotations
+
+import logging
+import shutil
+import time
+import traceback
+from pathlib import Path
+from typing import Any
+
+from ..jsonable import to_jsonable
+from .constants import EVENT_JOB_COMPLETED, EVENT_JOB_ERROR, EVENT_JOB_STARTED
+from ..service import build_config_builder, create_data_designer
+
+_PROJECT_ROOT = Path(__file__).resolve().parents[5]
+_ARTIFACT_ROOT = _PROJECT_ROOT / "studio" / "backend" / "assets" / "datasets"
+
+
+class _QueueLogHandler(logging.Handler):
+    def __init__(self, event_queue):
+        super().__init__()
+        self._q = event_queue
+
+    def emit(self, record: logging.LogRecord) -> None:
+        try:
+            event = {
+                "type": "log",
+                "ts": record.created,
+                "level": record.levelname,
+                "logger": record.name,
+                "message": record.getMessage(),
+            }
+            self._q.put(event)
+        except (OSError, RuntimeError, ValueError):
+            pass
+
+
+def run_job_process(
+    *,
+    event_queue,
+    recipe: dict[str, Any],
+    run: dict[str, Any],
+) -> None:
+    """
+    Subprocess entrypoint.
+    Sends events to `event_queue`.
+    """
+    event_queue.put({"type": EVENT_JOB_STARTED, "ts": time.time()})
+
+    try:
+        from data_designer.config.run_config import RunConfig
+
+        rows = int(run.get("rows") or 1000)
+        job_id = str(run.get("_job_id") or "").strip()
+        if not job_id:
+            job_id = f"{int(time.time())}"
+        dataset_name = f"recipe_{job_id}"
+        merge_batches = bool(run.get("merge_batches"))
+        _ARTIFACT_ROOT.mkdir(parents=True, exist_ok=True)
+        run_config_raw = run.get("run_config") or {}
+
+        builder = build_config_builder(recipe)
+        designer = create_data_designer(recipe, artifact_path=str(_ARTIFACT_ROOT))
+
+        # DataDesigner configures root logging in DataDesigner.__init__.
+        # Attach queue logger directly to `data_designer` so parser events survive root resets.
+        handler = _QueueLogHandler(event_queue)
+        handler.setLevel(logging.INFO)
+        data_designer_logger = logging.getLogger("data_designer")
+        data_designer_logger.addHandler(handler)
+        data_designer_logger.setLevel(logging.INFO)
+        data_designer_logger.propagate = True
+
+        if run_config_raw:
+            designer.set_run_config(RunConfig.model_validate(run_config_raw))
+
+        execution_type = str(run.get("execution_type") or "full").strip().lower()
+        if execution_type == "preview":
+            results = designer.preview(builder, num_records=rows)
+            analysis = (
+                None
+                if results.analysis is None
+                else to_jsonable(results.analysis.model_dump(mode="json"))
+            )
+            dataset = (
+                []
+                if results.dataset is None
+                else to_jsonable(results.dataset.to_dict(orient="records"))
+            )
+            processor_artifacts = (
+                None
+                if results.processor_artifacts is None
+                else to_jsonable(results.processor_artifacts)
+            )
+            event_queue.put(
+                {
+                    "type": EVENT_JOB_COMPLETED,
+                    "ts": time.time(),
+                    "analysis": analysis,
+                    "dataset": dataset,
+                    "processor_artifacts": processor_artifacts,
+                    "artifact_path": None,
+                    "execution_type": execution_type,
+                }
+            )
+        else:
+            results = designer.create(builder, num_records=rows, dataset_name=dataset_name)
+            analysis = to_jsonable(results.load_analysis().model_dump(mode="json"))
+            if merge_batches:
+                _merge_batches_to_single_parquet(results.artifact_storage.base_dataset_path)
+            artifact_path = str(results.artifact_storage.base_dataset_path)
+            event_queue.put(
+                {
+                    "type": EVENT_JOB_COMPLETED,
+                    "ts": time.time(),
+                    "analysis": analysis,
+                    "artifact_path": artifact_path,
+                    "execution_type": execution_type,
+                }
+            )
+    except Exception as exc:
+        event_queue.put(
+            {
+                "type": EVENT_JOB_ERROR,
+                "ts": time.time(),
+                "error": str(exc),
+                "stack": traceback.format_exc(limit=20),
+            }
+        )
+
+
+def _merge_batches_to_single_parquet(base_dataset_path: Path) -> None:
+    parquet_dir = base_dataset_path / "parquet-files"
+    parquet_files = sorted(parquet_dir.glob("*.parquet"))
+    if len(parquet_files) <= 1:
+        return
+
+    try:
+        from data_designer.config.utils.io_helpers import read_parquet_dataset
+    except ImportError:
+        return
+
+    dataframe = read_parquet_dataset(parquet_dir)
+    shutil.rmtree(parquet_dir)
+    parquet_dir.mkdir(parents=True, exist_ok=True)
+    dataframe.to_parquet(parquet_dir / "batch_00000.parquet", index=False)

studio/backend/core/data_recipe/jsonable.py

@@ -0,0 +1,30 @@
+from __future__ import annotations
+
+from typing import Any
+
+
+def to_jsonable(value: Any) -> Any:
+    """Convert numpy/pandas-ish values into plain JSON-safe values."""
+    try:
+        import numpy as np  # type: ignore
+    except ImportError:  # pragma: no cover
+        np = None  # type: ignore
+
+    if np is not None:
+        if isinstance(value, np.ndarray):
+            return value.tolist()
+        if isinstance(value, np.generic):
+            return value.item()
+
+    if isinstance(value, dict):
+        return {str(k): to_jsonable(v) for k, v in value.items()}
+    if isinstance(value, (list, tuple, set)):
+        return [to_jsonable(v) for v in value]
+
+    if hasattr(value, "isoformat") and callable(value.isoformat):
+        try:
+            return value.isoformat()
+        except (TypeError, ValueError):
+            return value
+
+    return value

studio/backend/core/data_recipe/service.py

@@ -0,0 +1,149 @@
+from __future__ import annotations
+
+import os
+from typing import Any
+
+from .jsonable import to_jsonable
+
+
+def build_model_providers(recipe: dict[str, Any]):
+    from data_designer.config.default_model_settings import get_default_providers
+    from data_designer.config.models import ModelProvider
+
+    providers: list[ModelProvider] = []
+    for provider in recipe.get("model_providers", []):
+        api_key = provider.get("api_key")
+        api_key_env = provider.get("api_key_env")
+        if not api_key and api_key_env:
+            api_key = os.getenv(api_key_env)
+        providers.append(
+            ModelProvider(
+                name=provider["name"],
+                endpoint=provider["endpoint"],
+                provider_type=provider.get("provider_type", "openai"),
+                api_key=api_key,
+                extra_headers=provider.get("extra_headers"),
+                extra_body=provider.get("extra_body"),
+            )
+        )
+
+    # DataDesigner currently expects at least one provider even if they only use static samplers,
+    # but it's fine it gives a warning only.
+    return providers or get_default_providers()
+
+
+def build_mcp_providers(
+    recipe: dict[str, Any],
+) -> list:
+    from data_designer.config.mcp import LocalStdioMCPProvider, MCPProvider
+
+    providers: list[MCPProvider | LocalStdioMCPProvider] = []
+    for provider in recipe.get("mcp_providers", []):
+        if not isinstance(provider, dict):
+            continue
+        provider_type = provider.get("provider_type")
+        if provider_type == "stdio":
+            env = provider.get("env")
+            if not isinstance(env, dict):
+                env = {}
+            args = provider.get("args")
+            if not isinstance(args, list):
+                args = []
+            providers.append(
+                LocalStdioMCPProvider(
+                    name=str(provider.get("name", "")),
+                    command=str(provider.get("command", "")),
+                    args=[str(value) for value in args],
+                    env={str(key): str(value) for key, value in env.items()},
+                )
+            )
+            continue
+
+        if provider_type in {"sse", "streamable_http"}:
+            api_key = provider.get("api_key")
+            api_key_env = provider.get("api_key_env")
+            if not api_key and api_key_env:
+                api_key = os.getenv(str(api_key_env))
+            providers.append(
+                MCPProvider(
+                    name=str(provider.get("name", "")),
+                    endpoint=str(provider.get("endpoint", "")),
+                    api_key=str(api_key) if api_key else None,
+                )
+            )
+    return providers
+
+
+def build_config_builder(recipe: dict[str, Any]):
+    from data_designer.config import DataDesignerConfigBuilder
+    from data_designer.config.processors import ProcessorType
+
+    recipe_core = {
+        key: value
+        for key, value in recipe.items()
+        if key not in {"model_providers", "mcp_providers"}
+    }
+    builder = DataDesignerConfigBuilder.from_config({"data_designer": recipe_core})
+
+    # DataDesignerConfigBuilder.from_config currently skips processors.
+    # Re-attach explicitly so drop_columns/schema_transform survive API payload.
+    for processor in recipe_core.get("processors") or []:
+        if not isinstance(processor, dict):
+            continue
+        processor_type_raw = processor.get("processor_type")
+        if not isinstance(processor_type_raw, str):
+            continue
+        kwargs = {k: v for k, v in processor.items() if k != "processor_type"}
+        builder.add_processor(
+            processor_type=ProcessorType(processor_type_raw),
+            **kwargs,
+        )
+
+    return builder
+
+
+def create_data_designer(
+    recipe: dict[str, Any],
+    *,
+    artifact_path: str | None = None,
+):
+    from data_designer.interface.data_designer import DataDesigner
+
+    return DataDesigner(
+        artifact_path=artifact_path,
+        model_providers=build_model_providers(recipe),
+        mcp_providers=build_mcp_providers(recipe),
+    )
+
+
+def validate_recipe(recipe: dict[str, Any]) -> None:
+    builder = build_config_builder(recipe)
+    designer = create_data_designer(recipe)
+    designer.validate(builder)
+
+
+def preview_recipe(
+    recipe: dict[str, Any],
+    num_records: int,
+) -> tuple[list[dict[str, Any]], dict[str, Any] | None, dict[str, Any] | None]:
+    builder = build_config_builder(recipe)
+    designer = create_data_designer(recipe)
+    results = designer.preview(builder, num_records=num_records)
+
+    dataset: list[dict[str, Any]] = []
+    if results.dataset is not None:
+        raw_rows = results.dataset.to_dict(orient="records")
+        dataset = [to_jsonable(row) for row in raw_rows]
+
+    artifacts = (
+        None
+        if results.processor_artifacts is None
+        else to_jsonable(results.processor_artifacts)
+    )
+    analysis = (
+        None
+        if results.analysis is None
+        else to_jsonable(results.analysis.model_dump(mode="json"))
+    )
+
+    return dataset, artifacts, analysis

studio/backend/core/export/export.py

@@ -2,8 +2,11 @@
 """
 Export backend - handles model exporting in various formats
 """
+import glob
+import json
 import logging
 import os
+import shutil
 from pathlib import Path
 from typing import Optional, Tuple, List
 from peft import PeftModel, PeftModelForCausalLM
@@ -200,6 +203,18 @@ class ExportBackend:
             logger.error(traceback.format_exc())
             return False, f"Failed to load checkpoint: {str(e)}"
 
+    def _write_export_metadata(self, save_directory: str):
+        """Write export_metadata.json with base model info for Chat page discovery."""
+        try:
+            base_model = get_base_model_from_lora(self.current_checkpoint) if self.current_checkpoint else None
+            metadata = {"base_model": base_model}
+            metadata_path = os.path.join(save_directory, "export_metadata.json")
+            with open(metadata_path, "w") as f:
+                json.dump(metadata, f, indent=2)
+            logger.info(f"Wrote export metadata to {metadata_path}")
+        except Exception as e:
+            logger.warning(f"Could not write export metadata: {e}")
+
     def export_merged_model(self,
                            save_directory: str,
                            format_type: str = "16-bit (FP16)",
@@ -244,6 +259,9 @@ class ExportBackend:
                     self.current_tokenizer,
                     save_method=save_method
                 )
+
+                # Write export metadata so the Chat page can identify the base model
+                self._write_export_metadata(save_directory)
                 logger.info(f"Model saved successfully to {save_directory}")
 
             # Push to hub if requested
@@ -297,6 +315,9 @@ class ExportBackend:
 
                 self.current_model.save_pretrained(save_directory)
                 self.current_tokenizer.save_pretrained(save_directory)
+
+                # Write export metadata so the Chat page can identify the base model
+                self._write_export_metadata(save_directory)
                 logger.info(f"Model saved successfully to {save_directory}")
 
             # Push to hub if requested
@@ -378,53 +399,61 @@ class ExportBackend:
 
             # Save locally if requested
             if save_directory:
-                logger.info(f"Saving GGUF model locally to: {save_directory}")
+                # Resolve to absolute path so unsloth's relative-path internals
+                # (check_llama_cpp, use_local_gguf, _download_convert_hf_to_gguf)
+                # all resolve against the repo root cwd, NOT the export directory.
+                abs_save_dir = os.path.abspath(save_directory)
+                logger.info(f"Saving GGUF model locally to: {abs_save_dir}")
 
                 # Create the directory if it doesn't exist
-                os.makedirs(save_directory, exist_ok=True)
+                os.makedirs(abs_save_dir, exist_ok=True)
 
-                # Get the base filename for the GGUF file
-                import shutil
-                original_dir = os.getcwd()
+                # On WSL, patch out sudo check before llama.cpp build
+                _apply_wsl_sudo_patch()
 
-                try:
-                    # Change to target directory
-                    os.chdir(save_directory)
-                    logger.info(f"Changed directory to: {save_directory}")
+                # Snapshot existing .gguf files in cwd before conversion.
+                # unsloth's convert_to_gguf writes output files relative to
+                # cwd (repo root), so we diff afterwards and relocate them.
+                cwd = os.getcwd()
+                pre_existing_ggufs = set(glob.glob(os.path.join(cwd, "*.gguf")))
 
-                    # On WSL, patch out sudo check before llama.cpp build
-                    _apply_wsl_sudo_patch()
+                # Pass absolute path — no os.chdir needed.
+                # unsloth saves intermediate HF model files into model_save_path.
+                # unsloth-zoo's check_llama_cpp() uses ~/.unsloth/llama.cpp by default.
+                model_save_path = os.path.join(abs_save_dir, "model")
+                self.current_model.save_pretrained_gguf(
+                    model_save_path,
+                    self.current_tokenizer,
+                    quantization_method=quant_method
+                )
 
-                    # Now save (will save in current directory)
-                    self.current_model.save_pretrained_gguf(
-                        "model",  # Base filename
-                        self.current_tokenizer,
-                        quantization_method=quant_method
-                    )
+                # Relocate GGUF artifacts into the export directory.
+                # convert_to_gguf writes .gguf files to cwd (repo root)
+                # because --outfile is a relative path like "model.Q4_K_M.gguf".
+                new_ggufs = set(glob.glob(os.path.join(cwd, "*.gguf"))) - pre_existing_ggufs
+                for src in sorted(new_ggufs):
+                    dest = os.path.join(abs_save_dir, os.path.basename(src))
+                    shutil.move(src, dest)
+                    logger.info(f"Relocated GGUF: {os.path.basename(src)} → {abs_save_dir}/")
 
-                    logger.info(f"GGUF model saved successfully in {save_directory}")
+                # Flatten any .gguf files from subdirectories into abs_save_dir.
+                # save_pretrained_gguf may create subdirs (e.g. model_gguf/)
+                # with a name different from model_save_path.
+                for sub in list(Path(abs_save_dir).iterdir()):
+                    if not sub.is_dir():
+                        continue
+                    for src in sub.glob("*.gguf"):
+                        dest = os.path.join(abs_save_dir, src.name)
+                        shutil.move(str(src), dest)
+                        logger.info(f"Relocated GGUF: {src.name} → {abs_save_dir}/")
+                    # Clean up the subdirectory (intermediate HF files, etc.)
+                    shutil.rmtree(str(sub), ignore_errors=True)
+                    logger.info(f"Cleaned up subdirectory: {sub.name}")
 
-                    # Check if llama.cpp directory was created here
-                    llama_cpp_in_target = os.path.join(save_directory, "llama.cpp")
-                    llama_cpp_in_original = os.path.join(original_dir, "llama.cpp")
+                # Write export metadata so the Chat page can identify the base model
+                self._write_export_metadata(abs_save_dir)
 
-                    if os.path.exists(llama_cpp_in_target):
-                        logger.info(f"Found llama.cpp directory in {save_directory}")
-
-                        # Remove llama.cpp from original directory if it exists
-                        if os.path.exists(llama_cpp_in_original):
-                            logger.info(f"Removing existing llama.cpp in {original_dir}")
-                            shutil.rmtree(llama_cpp_in_original)
-
-                        # Move llama.cpp back to original directory
-                        logger.info(f"Moving llama.cpp to {original_dir}")
-                        shutil.move(llama_cpp_in_target, llama_cpp_in_original)
-                        logger.info(f"Successfully moved llama.cpp back to original directory")
-
-                finally:
-                    # Always change back to original directory
-                    os.chdir(original_dir)
-                    logger.info(f"Changed back to original directory: {original_dir}")
+                logger.info(f"GGUF model saved successfully in {abs_save_dir}")
 
             # Push to hub if requested
             if push_to_hub:

studio/backend/core/inference/__init__.py

@@ -2,8 +2,10 @@
 Inference submodule - Inference backend for model loading and generation
 """
 from .inference import InferenceBackend, get_inference_backend
+from .llama_cpp import LlamaCppBackend
 
 __all__ = [
     'InferenceBackend',
     'get_inference_backend',
+    'LlamaCppBackend',
 ]

studio/backend/core/inference/inference.py

@@ -6,8 +6,10 @@ from unsloth.chat_templates import get_chat_template
 from transformers import TextStreamer
 from peft import PeftModel, PeftModelForCausalLM
 
+import json
 import sys
 import torch
+from pathlib import Path
 from typing import Optional, Union, Generator, Tuple
 from utils.models import ModelConfig, get_base_model_from_lora
 from utils.paths import is_model_cached
@@ -112,7 +114,18 @@ class InferenceBackend:
                 # In that case, load the real processor from the base model.
                 from transformers import ProcessorMixin
                 if not (isinstance(processor, ProcessorMixin) or hasattr(processor, "image_processor")):
+                    # For LoRA adapters, use the base model. For local merged exports,
+                    # read export_metadata.json to find the original base model.
                     processor_source = config.base_model if config.is_lora else config.identifier
+                    if not config.is_lora and config.is_local:
+                        _meta_path = Path(config.path) / "export_metadata.json"
+                        try:
+                            if _meta_path.exists():
+                                _meta = json.loads(_meta_path.read_text())
+                                if _meta.get("base_model"):
+                                    processor_source = _meta["base_model"]
+                        except Exception:
+                            pass
                     logger.warning(
                         f"FastVisionModel returned {type(processor).__name__} (no image_processor) "
                         f"for '{model_name}' — loading proper processor from '{processor_source}'"
@@ -255,47 +268,6 @@ class InferenceBackend:
             logger.error(traceback.format_exc())
             return False, None
 
-    def load_adapter(self, base_model_name: str, adapter_path: str, adapter_name: str = None) -> bool:
-        """
-        Load a LoRA adapter onto the base model if it's not already registered.
-        This method is idempotent.
-        """
-        if base_model_name not in self.models:
-            logger.error(f"Base model {base_model_name} not loaded")
-            return False
-
-        model = self.models[base_model_name].get("model")
-        if model is None:
-            logger.error(f"Model object for {base_model_name} is None.")
-            return False
-
-        if adapter_name is None:
-            adapter_name = adapter_path.split("/")[-1].replace(".", "_")
-
-        # If we've loaded this adapter before, we don't need to do anything.
-        if adapter_name in self.models[base_model_name].get("loaded_adapters", {}):
-            logger.info(f"Adapter '{adapter_name}' is already registered. Skipping.")
-            return True
-
-        try:
-            logger.info(f"Loading new adapter '{adapter_name}' from '{adapter_path}' onto {base_model_name}")
-
-            # Unsloth modifies the model in-place and returns None. Do NOT re-assign.
-            model.load_adapter(adapter_path, adapter_name=adapter_name)
-
-            # Update our internal registry so we don't load it again.
-            self.models[base_model_name]["loaded_adapters"][adapter_name] = adapter_path
-
-            total_adapters = len(getattr(model, 'peft_config', {}))
-            logger.info(f"Adapter '{adapter_name}' loaded successfully. (Total adapters on model: {total_adapters})")
-            return True
-        except Exception as e:
-            logger.error(f"Failed to load adapter '{adapter_name}': {e}")
-            import traceback
-            logger.error(traceback.format_exc())
-            return False
-    pass
-
     def enable_adapter(self, base_model_name: str, adapter_name: str) -> bool:
         """Enable specific adapter (for generation)"""
         if base_model_name not in self.models:
@@ -328,55 +300,6 @@ class InferenceBackend:
             logger.error(f"Failed to disable adapters: {e}")
             return False
 
-    # In backend/inference.py
-
-    def load_for_eval(self, lora_path: str, max_seq_length: int = 2048,
-                     dtype = None, load_in_4bit: bool = True,
-                     hf_token: Optional[str] = None) -> Tuple[bool, Optional[str], Optional[str]]:
-        """
-        Prepare for eval: ensure base model and the specified adapter are loaded.
-        """
-        try:
-            from utils.models import ModelConfig
-            lora_config = ModelConfig.from_lora_path(lora_path, hf_token)
-            if not lora_config:
-                return False, None, None
-
-            base_model_name = lora_config.base_model
-
-            # 1. Load the base model if it's not already in memory (this logic is correct)
-            if base_model_name not in self.models or not self.models[base_model_name].get("model"):
-                logger.info(f"Base model '{base_model_name}' not loaded, loading now.")
-                base_config = ModelConfig.from_ui_selection(base_model_name, None, is_lora=False)
-                if not self.load_model(base_config, max_seq_length, dtype, load_in_4bit, hf_token):
-                    return False, None, None
-            else:
-                logger.info(f"Base model '{base_model_name}' is already in memory.")
-
-            self.active_model_name = base_model_name
-
-            # 2. Delegate to our now-idempotent load_adapter function.
-            # It will handle all cases: first adapter, or subsequent adapters.
-            adapter_name = lora_path.split("/")[-1].replace(".", "_")
-            adapter_success = self.load_adapter(
-                base_model_name=base_model_name,
-                adapter_path=lora_path,
-                adapter_name=adapter_name
-            )
-
-            if not adapter_success:
-                return False, base_model_name, None
-
-            return True, base_model_name, adapter_name
-
-        except Exception as e:
-            logger.error(f"Error during load_for_eval: {e}")
-            import traceback
-            logger.error(traceback.format_exc())
-            return False, None, None
-    pass
-
-
     def load_for_eval(self, lora_path: str, max_seq_length: int = 2048,
                      dtype = None, load_in_4bit: bool = True,
                      hf_token: Optional[str] = None) -> Tuple[bool, Optional[str], Optional[str]]:
@@ -1259,47 +1182,6 @@ class InferenceBackend:
         """Get name of currently loading model"""
         return next(iter(self.loading_models)) if self.loading_models else None
 
-    def load_model_simple(self,
-                         model_path: str,
-                         hf_token: Optional[str] = None,
-                         max_seq_length: int = 2048,
-                         load_in_4bit: bool = True) -> bool:
-        """
-        Simple model loading wrapper for chat interface.
-        Accepts model path as string and handles ModelConfig creation internally.
-
-        Args:
-            model_path: Model name or path (e.g., "unsloth/llama-3-8b")
-            hf_token: HuggingFace token for gated models
-            max_seq_length: Maximum sequence length
-            load_in_4bit: Whether to use 4-bit quantization
-
-        Returns:
-            bool: True if successful, False otherwise
-        """
-        try:
-            # Create config from string path
-            config = ModelConfig.from_ui_selection(
-                model_path,
-                lora_path=None,  # No LoRA for chat
-                is_lora=False
-            )
-
-            # Call existing load_model with config
-            return self.load_model(
-                config=config,
-                max_seq_length=max_seq_length,
-                dtype=None,  # Auto-detect
-                load_in_4bit=load_in_4bit,
-                hf_token=hf_token
-            )
-
-        except Exception as e:
-            logger.error(f"Error in load_model_simple: {e}")
-            return False
-
-
-
     def load_model_simple(self,
                      model_path: str,
                      hf_token: Optional[str] = None,

studio/backend/core/inference/llama_cpp.py

@@ -0,0 +1,573 @@
+"""
+llama-server inference backend for GGUF models.
+
+Manages a llama-server subprocess and proxies chat completions
+through its OpenAI-compatible /v1/chat/completions endpoint.
+"""
+import atexit
+import json
+import logging
+import shutil
+import signal
+import socket
+import subprocess
+import threading
+import time
+from pathlib import Path
+from typing import Generator, Optional
+
+import httpx
+
+logger = logging.getLogger(__name__)
+
+
+class LlamaCppBackend:
+    """
+    Manages a llama-server subprocess for GGUF model inference.
+
+    Lifecycle:
+        1. load_model()  — starts llama-server with the GGUF file
+        2. generate_chat_completion() — proxies to /v1/chat/completions, streams back
+        3. unload_model() — terminates llama-server subprocess
+    """
+
+    def __init__(self):
+        self._process: Optional[subprocess.Popen] = None
+        self._port: Optional[int] = None
+        self._model_identifier: Optional[str] = None
+        self._gguf_path: Optional[str] = None
+        self._hf_repo: Optional[str] = None
+        self._hf_variant: Optional[str] = None
+        self._is_vision: bool = False
+        self._healthy = False
+        self._lock = threading.Lock()
+        self._stdout_lines: list[str] = []
+        self._stdout_thread: Optional[threading.Thread] = None
+
+        atexit.register(self._cleanup)
+
+    # ── Properties ────────────────────────────────────────────────
+
+    @property
+    def is_loaded(self) -> bool:
+        return self._process is not None and self._healthy
+
+    @property
+    def base_url(self) -> str:
+        return f"http://127.0.0.1:{self._port}"
+
+    @property
+    def model_identifier(self) -> Optional[str]:
+        return self._model_identifier
+
+    @property
+    def is_vision(self) -> bool:
+        return self._is_vision
+
+    @property
+    def hf_variant(self) -> Optional[str]:
+        return self._hf_variant
+
+    # ── Binary discovery ──────────────────────────────────────────
+
+    @staticmethod
+    def _find_llama_server_binary() -> Optional[str]:
+        """
+        Locate the llama-server binary.
+
+        Search order:
+        1.  LLAMA_SERVER_PATH environment variable (direct path to binary)
+        1b. UNSLOTH_LLAMA_CPP_PATH env var (custom llama.cpp install dir)
+        2.  ~/.unsloth/llama.cpp/llama-server        (make build, root dir)
+        3.  ~/.unsloth/llama.cpp/build/bin/llama-server  (cmake build, Linux)
+        4.  ~/.unsloth/llama.cpp/build/bin/Release/llama-server.exe  (cmake build, Windows)
+        5.  ./llama.cpp/llama-server                 (legacy: make build, root dir)
+        6.  ./llama.cpp/build/bin/llama-server        (legacy: cmake in-tree build)
+        7.  llama-server on PATH                     (system install)
+        8.  ./bin/llama-server                       (legacy: extracted binary)
+        """
+        import os
+        import sys
+
+        binary_name = "llama-server.exe" if sys.platform == "win32" else "llama-server"
+
+        # 1. Env var — direct path to binary
+        env_path = os.environ.get("LLAMA_SERVER_PATH")
+        if env_path and Path(env_path).is_file():
+            return env_path
+
+        # 1b. UNSLOTH_LLAMA_CPP_PATH — custom llama.cpp install directory
+        custom_llama_cpp = os.environ.get("UNSLOTH_LLAMA_CPP_PATH")
+        if custom_llama_cpp:
+            custom_dir = Path(custom_llama_cpp)
+            # Root dir (make builds)
+            root_bin = custom_dir / binary_name
+            if root_bin.is_file():
+                return str(root_bin)
+            # build/bin/ (cmake builds on Linux)
+            cmake_bin = custom_dir / "build" / "bin" / binary_name
+            if cmake_bin.is_file():
+                return str(cmake_bin)
+            # build/bin/Release/ (cmake builds on Windows)
+            if sys.platform == "win32":
+                win_bin = custom_dir / "build" / "bin" / "Release" / binary_name
+                if win_bin.is_file():
+                    return str(win_bin)
+
+        # 2–4. ~/.unsloth/llama.cpp (primary — setup.sh / setup.ps1 build here)
+        unsloth_home = Path.home() / ".unsloth" / "llama.cpp"
+        # Root dir (make builds copy binaries here)
+        home_root = unsloth_home / binary_name
+        if home_root.is_file():
+            return str(home_root)
+        # build/bin/ (cmake builds on Linux)
+        home_linux = unsloth_home / "build" / "bin" / binary_name
+        if home_linux.is_file():
+            return str(home_linux)
+
+        # 3. Windows MSVC build has Release subdir
+        if sys.platform == "win32":
+            home_win = unsloth_home / "build" / "bin" / "Release" / binary_name
+            if home_win.is_file():
+                return str(home_win)
+
+        # 5–6. Legacy: in-tree build (older setup.sh / setup.ps1 versions)
+        project_root = Path(__file__).resolve().parents[4]
+        # Root dir (make builds)
+        root_path = project_root / "llama.cpp" / binary_name
+        if root_path.is_file():
+            return str(root_path)
+        # build/bin/ (cmake builds)
+        build_path = project_root / "llama.cpp" / "build" / "bin" / binary_name
+        if build_path.is_file():
+            return str(build_path)
+        if sys.platform == "win32":
+            win_path = project_root / "llama.cpp" / "build" / "bin" / "Release" / binary_name
+            if win_path.is_file():
+                return str(win_path)
+
+        # 7. System PATH
+        system_path = shutil.which("llama-server")
+        if system_path:
+            return system_path
+
+        # 8. Legacy: extracted to bin/
+        bin_path = project_root / "bin" / binary_name
+        if bin_path.is_file():
+            return str(bin_path)
+
+        return None
+
+    # ── Port allocation ───────────────────────────────────────────
+
+    @staticmethod
+    def _find_free_port() -> int:
+        """Find an available TCP port."""
+        with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
+            s.bind(("", 0))
+            return s.getsockname()[1]
+
+    # ── Stdout drain (prevents pipe deadlock on Windows) ─────────
+
+    def _drain_stdout(self):
+        """
+        Read lines from the subprocess stdout in a background thread.
+
+        This prevents a pipe-buffer deadlock on Windows where the default
+        pipe buffer is only ~4 KB.  Without draining, llama-server blocks
+        on writes and never becomes healthy.
+        """
+        try:
+            for line in self._process.stdout:
+                line = line.rstrip()
+                if line:
+                    self._stdout_lines.append(line)
+                    logger.info(f"[llama-server] {line}")
+        except (ValueError, OSError):
+            # Pipe closed — process is terminating
+            pass
+
+    # ── Lifecycle ─────────────────────────────────────────────────
+
+    def load_model(
+        self,
+        *,
+        # Local mode: pass a path to a .gguf file
+        gguf_path: Optional[str] = None,
+        # Vision projection (mmproj) for local vision models
+        mmproj_path: Optional[str] = None,
+        # HF mode: let llama-server download via -hf "repo:quant"
+        hf_repo: Optional[str] = None,
+        hf_variant: Optional[str] = None,
+        hf_token: Optional[str] = None,
+        # Common
+        model_identifier: str,
+        is_vision: bool = False,
+        n_ctx: int = 4096,
+        n_gpu_layers: int = -1,
+        n_threads: Optional[int] = None,
+    ) -> bool:
+        """
+        Start llama-server with a GGUF model.
+
+        Two modes:
+        - Local: ``gguf_path="/path/to/model.gguf"`` → uses ``-m``
+        - HF:    ``hf_repo="unsloth/gemma-3-4b-it-GGUF", hf_variant="Q4_K_M"`` → uses ``-hf``
+
+        In HF mode, llama-server handles downloading, caching, and
+        auto-loading mmproj files for vision models.
+
+        Returns True if server started and health check passed.
+        """
+        with self._lock:
+            self._kill_process()
+
+            binary = self._find_llama_server_binary()
+            if not binary:
+                raise RuntimeError(
+                    "llama-server binary not found. "
+                    "Run setup.sh to build it, install llama.cpp, "
+                    "or set LLAMA_SERVER_PATH environment variable."
+                )
+
+            self._port = self._find_free_port()
+
+            # Build command based on mode
+            if hf_repo:
+                # Download the GGUF file ourselves using huggingface_hub
+                # (llama-server's -hf flag requires HTTPS/curl which may not
+                #  be available, e.g. Windows builds with -DLLAMA_CURL=OFF)
+                try:
+                    from huggingface_hub import hf_hub_download
+                except ImportError:
+                    raise RuntimeError(
+                        "huggingface_hub is required for HF model loading. "
+                        "Install it with: pip install huggingface_hub"
+                    )
+
+                # Determine the filename from the variant (e.g., "Q4_K_M" -> find matching file)
+                gguf_filename = None
+                if hf_variant:
+                    # Try common naming patterns
+                    try:
+                        from huggingface_hub import list_repo_files
+                        files = list_repo_files(hf_repo, token=hf_token)
+                        variant_lower = hf_variant.lower()
+                        for f in files:
+                            if f.endswith(".gguf") and variant_lower in f.lower():
+                                gguf_filename = f
+                                break
+                    except Exception as e:
+                        logger.warning(f"Could not list repo files: {e}")
+
+                    if not gguf_filename:
+                        # Fallback: construct common filename pattern
+                        # e.g., "unsloth/gemma-3-4b-it-GGUF" + "Q4_K_M" -> try model name
+                        repo_name = hf_repo.split("/")[-1].replace("-GGUF", "")
+                        gguf_filename = f"{repo_name}-{hf_variant}.gguf"
+
+                logger.info(f"Downloading GGUF: {hf_repo}/{gguf_filename}")
+                try:
+                    local_path = hf_hub_download(
+                        repo_id=hf_repo,
+                        filename=gguf_filename,
+                        token=hf_token,
+                    )
+                except Exception as e:
+                    raise RuntimeError(
+                        f"Failed to download GGUF file '{gguf_filename}' from {hf_repo}: {e}"
+                    )
+
+                logger.info(f"GGUF downloaded to: {local_path}")
+                cmd = [
+                    binary,
+                    "-m", local_path,
+                    "--port", str(self._port),
+                    "-c", str(n_ctx),
+                    "-ngl", str(n_gpu_layers),
+                ]
+            elif gguf_path:
+                if not Path(gguf_path).is_file():
+                    raise FileNotFoundError(f"GGUF file not found: {gguf_path}")
+                cmd = [
+                    binary,
+                    "-m", gguf_path,
+                    "--port", str(self._port),
+                    "-c", str(n_ctx),
+                    "-ngl", str(n_gpu_layers),
+                ]
+            else:
+                raise ValueError("Either gguf_path or hf_repo must be provided")
+
+            if n_threads is not None:
+                cmd.extend(["--threads", str(n_threads)])
+
+            # Append mmproj for local vision models
+            if mmproj_path:
+                if not Path(mmproj_path).is_file():
+                    logger.warning(f"mmproj file not found: {mmproj_path}")
+                else:
+                    cmd.extend(["--mmproj", mmproj_path])
+                    logger.info(f"Using mmproj for vision: {mmproj_path}")
+
+            logger.info(f"Starting llama-server: {' '.join(cmd)}")
+
+            # Set library paths so llama-server can find its shared libs and CUDA DLLs
+            import os
+            import sys
+            env = os.environ.copy()
+            binary_dir = str(Path(binary).parent)
+
+            if sys.platform == "win32":
+                # On Windows, CUDA DLLs (cublas64_12.dll, cudart64_12.dll, etc.)
+                # must be on PATH. Add CUDA_PATH\bin if available.
+                path_dirs = [binary_dir]
+                cuda_path = os.environ.get("CUDA_PATH", "")
+                if cuda_path:
+                    cuda_bin = os.path.join(cuda_path, "bin")
+                    if os.path.isdir(cuda_bin):
+                        path_dirs.append(cuda_bin)
+                    # Some CUDA installs put DLLs in bin\x64
+                    cuda_bin_x64 = os.path.join(cuda_path, "bin", "x64")
+                    if os.path.isdir(cuda_bin_x64):
+                        path_dirs.append(cuda_bin_x64)
+                existing_path = env.get("PATH", "")
+                env["PATH"] = ";".join(path_dirs) + ";" + existing_path
+            else:
+                # Linux: set LD_LIBRARY_PATH for shared libs next to the binary
+                existing_ld = env.get("LD_LIBRARY_PATH", "")
+                env["LD_LIBRARY_PATH"] = f"{binary_dir}:{existing_ld}" if existing_ld else binary_dir
+
+            self._stdout_lines = []
+            self._process = subprocess.Popen(
+                cmd,
+                stdout=subprocess.PIPE,
+                stderr=subprocess.STDOUT,
+                text=True,
+                env=env,
+            )
+
+            # Start background thread to drain stdout and prevent pipe deadlock
+            self._stdout_thread = threading.Thread(
+                target=self._drain_stdout, daemon=True, name="llama-stdout"
+            )
+            self._stdout_thread.start()
+
+            self._gguf_path = gguf_path
+            self._hf_repo = hf_repo
+            self._hf_variant = hf_variant
+            self._is_vision = is_vision
+            self._model_identifier = model_identifier
+
+            # Wait for llama-server to become healthy
+            if not self._wait_for_health(timeout=120.0):
+                self._kill_process()
+                raise RuntimeError(
+                    "llama-server failed to start. "
+                    "Check that the GGUF file is valid and you have enough memory."
+                )
+
+            self._healthy = True
+
+            logger.info(
+                f"llama-server ready on port {self._port} "
+                f"for model '{model_identifier}'"
+            )
+            return True
+
+    def unload_model(self) -> bool:
+        """Terminate the llama-server subprocess and clean up state."""
+        with self._lock:
+            self._kill_process()
+            logger.info(f"Unloaded GGUF model: {self._model_identifier}")
+            self._model_identifier = None
+            self._gguf_path = None
+            self._hf_repo = None
+            self._hf_variant = None
+            self._is_vision = False
+            self._port = None
+            self._healthy = False
+            return True
+
+    def _kill_process(self):
+        """Terminate the subprocess if running."""
+        if self._process is None:
+            return
+        try:
+            self._process.terminate()
+            self._process.wait(timeout=5)
+        except subprocess.TimeoutExpired:
+            logger.warning("llama-server did not exit on SIGTERM, sending SIGKILL")
+            self._process.kill()
+            self._process.wait(timeout=5)
+        except Exception as e:
+            logger.warning(f"Error killing llama-server process: {e}")
+        finally:
+            self._process = None
+            if self._stdout_thread is not None:
+                self._stdout_thread.join(timeout=2)
+                self._stdout_thread = None
+
+    def _cleanup(self):
+        """atexit handler to ensure llama-server is terminated."""
+        self._kill_process()
+
+    def _wait_for_health(self, timeout: float = 120.0, interval: float = 0.5) -> bool:
+        """
+        Poll llama-server's /health endpoint until it responds 200.
+
+        Also monitors subprocess for early exit/crash.
+        """
+        deadline = time.monotonic() + timeout
+        url = f"http://127.0.0.1:{self._port}/health"
+
+        while time.monotonic() < deadline:
+            # Check if process crashed
+            if self._process.poll() is not None:
+                # Give the drain thread a moment to collect final output
+                if self._stdout_thread is not None:
+                    self._stdout_thread.join(timeout=2)
+                output = "\n".join(self._stdout_lines[-50:])
+                logger.error(
+                    f"llama-server exited with code {self._process.returncode}. "
+                    f"Output: {output[:2000]}"
+                )
+                return False
+
+            try:
+                resp = httpx.get(url, timeout=2.0)
+                if resp.status_code == 200:
+                    return True
+            except (httpx.ConnectError, httpx.TimeoutException):
+                pass
+
+            time.sleep(interval)
+
+        logger.error(f"llama-server health check timed out after {timeout}s")
+        return False
+
+    # ── Message building (OpenAI format) ──────────────────────────
+
+    @staticmethod
+    def _build_openai_messages(
+        messages: list[dict],
+        image_b64: Optional[str] = None,
+    ) -> list[dict]:
+        """
+        Build OpenAI-format messages, optionally injecting an image_url
+        content part into the last user message for vision models.
+
+        If no image is provided, returns messages as-is.
+        """
+        if not image_b64:
+            return messages
+
+        # Find the last user message and convert to multimodal content parts
+        result = [msg.copy() for msg in messages]
+        last_user_idx = None
+        for i, msg in enumerate(result):
+            if msg["role"] == "user":
+                last_user_idx = i
+
+        if last_user_idx is not None:
+            text_content = result[last_user_idx].get("content", "")
+            result[last_user_idx]["content"] = [
+                {"type": "text", "text": text_content},
+                {
+                    "type": "image_url",
+                    "image_url": {
+                        "url": f"data:image/png;base64,{image_b64}",
+                    },
+                },
+            ]
+
+        return result
+
+    # ── Generation (proxy to llama-server) ────────────────────────
+
+    def generate_chat_completion(
+        self,
+        messages: list[dict],
+        image_b64: Optional[str] = None,
+        temperature: float = 0.7,
+        top_p: float = 0.9,
+        top_k: int = 40,
+        min_p: float = 0.0,
+        max_tokens: int = 512,
+        repetition_penalty: float = 1.1,
+        stop: Optional[list[str]] = None,
+        cancel_event: Optional[threading.Event] = None,
+    ) -> Generator[str, None, None]:
+        """
+        Send a chat completion request to llama-server and stream tokens back.
+
+        Uses /v1/chat/completions — llama-server handles chat template
+        application and vision (multimodal image_url parts) natively.
+
+        Yields cumulative text (matching InferenceBackend's convention).
+        """
+        if not self.is_loaded:
+            raise RuntimeError("llama-server is not loaded")
+
+        openai_messages = self._build_openai_messages(messages, image_b64)
+
+        payload = {
+            "messages": openai_messages,
+            "stream": True,
+            "temperature": temperature,
+            "top_p": top_p,
+            "top_k": top_k if top_k >= 0 else 0,
+            "min_p": min_p,
+            "max_tokens": max_tokens,
+            "repeat_penalty": repetition_penalty,
+        }
+        if stop:
+            payload["stop"] = stop
+
+        url = f"{self.base_url}/v1/chat/completions"
+        cumulative = ""
+
+        try:
+            with httpx.Client(timeout=None) as client:
+                with client.stream("POST", url, json=payload) as response:
+                    if response.status_code != 200:
+                        error_body = response.read().decode()
+                        raise RuntimeError(
+                            f"llama-server returned {response.status_code}: {error_body}"
+                        )
+
+                    buffer = ""
+                    for raw_chunk in response.iter_text():
+                        if cancel_event is not None and cancel_event.is_set():
+                            break
+
+                        buffer += raw_chunk
+                        while "\n" in buffer:
+                            line, buffer = buffer.split("\n", 1)
+                            line = line.strip()
+
+                            if not line:
+                                continue
+                            if line == "data: [DONE]":
+                                return
+                            if not line.startswith("data: "):
+                                continue
+
+                            try:
+                                data = json.loads(line[6:])
+                                choices = data.get("choices", [])
+                                if choices:
+                                    delta = choices[0].get("delta", {})
+                                    token = delta.get("content", "")
+                                    if token:
+                                        cumulative += token
+                                        yield cumulative
+                            except json.JSONDecodeError:
+                                logger.debug(f"Skipping malformed SSE line: {line[:100]}")
+
+        except httpx.ConnectError:
+            raise RuntimeError("Lost connection to llama-server")
+        except Exception as e:
+            if cancel_event is not None and cancel_event.is_set():
+                return
+            raise

studio/backend/core/training/trainer.py

@@ -7,7 +7,7 @@ import os
 os.environ["TOKENIZERS_PARALLELISM"] = "false"
 
 import torch
-from utils.hardware import clear_gpu_cache
+from utils.hardware import clear_gpu_cache, safe_num_proc
 torch._dynamo.config.recompile_limit = 64
 from unsloth import FastLanguageModel, FastVisionModel, is_bfloat16_supported
 from unsloth.chat_templates import get_chat_template
@@ -175,6 +175,15 @@ class UnslothTrainer:
                     token=hf_token,
                 )
                 logger.info("Loaded vision model")
+
+                # Diagnostic: check if FastVisionModel returned a real Processor or a raw tokenizer
+                from transformers import ProcessorMixin
+                tok = self.tokenizer
+                has_image_proc = isinstance(tok, ProcessorMixin) or hasattr(tok, "image_processor")
+                print(f"\n[VLM Diagnostic] FastVisionModel returned: {type(tok).__name__}")
+                print(f"[VLM Diagnostic] Is ProcessorMixin: {isinstance(tok, ProcessorMixin)}")
+                print(f"[VLM Diagnostic] Has image_processor: {hasattr(tok, 'image_processor')}")
+                print(f"[VLM Diagnostic] Usable as vision processor: {has_image_proc}\n")
             else:
                 # Load text model - returns (model, tokenizer)
                 self.model, self.tokenizer = FastLanguageModel.from_pretrained(
@@ -345,7 +354,10 @@ class UnslothTrainer:
                      custom_format_mapping: dict = None,
                      subset: str = None,
                      train_split: str = "train",
-                     eval_split: str = None) -> Optional[tuple]:
+                     eval_split: str = None,
+                     eval_steps: float = 0.00,
+                     dataset_slice_start: int = None,
+                     dataset_slice_end: int = None) -> Optional[tuple]:
         """
         Load and prepare dataset for training.
 
@@ -360,6 +372,7 @@ class UnslothTrainer:
             dataset = None
             eval_dataset = None
             has_separate_eval_source = False  # True if eval comes from a separate HF split
+            eval_enabled = eval_steps is not None and eval_steps > 0
 
             if local_datasets:
                 # Load local datasets
@@ -412,27 +425,42 @@ class UnslothTrainer:
                 print(f"Loaded dataset from Hugging Face: {dataset_source}\n")
 
                 # Resolve eval split from a separate HF split (explicit or auto-detected)
-                if eval_split:
-                    # Explicit eval split provided - load it directly
-                    print(f"Loading explicit eval split: '{eval_split}'\n")
-                    eval_load_kwargs = {"path": dataset_source, "split": eval_split}
-                    if subset:
-                        eval_load_kwargs["name"] = subset
-                    eval_dataset = load_dataset(**eval_load_kwargs)
-                    has_separate_eval_source = True
-                    print(f"Loaded eval split '{eval_split}' with {len(eval_dataset)} rows\n")
-                else:
-                    # Auto-detect eval split from HF (returns a separate dataset, or None)
-                    eval_dataset = self._auto_detect_eval_split_from_hf(
-                        dataset_source=dataset_source,
-                        subset=subset,
-                    )
-                    if eval_dataset is not None:
+                if eval_enabled:
+                    if eval_split:
+                        # Explicit eval split provided - load it directly
+                        print(f"Loading explicit eval split: '{eval_split}'\n")
+                        eval_load_kwargs = {"path": dataset_source, "split": eval_split}
+                        if subset:
+                            eval_load_kwargs["name"] = subset
+                        eval_dataset = load_dataset(**eval_load_kwargs)
                         has_separate_eval_source = True
+                        print(f"Loaded eval split '{eval_split}' with {len(eval_dataset)} rows\n")
+                    else:
+                        # Auto-detect eval split from HF (returns a separate dataset, or None)
+                        eval_dataset = self._auto_detect_eval_split_from_hf(
+                            dataset_source=dataset_source,
+                            subset=subset,
+                        )
+                        if eval_dataset is not None:
+                            has_separate_eval_source = True
+                else:
+                    print("Eval disabled (eval_steps <= 0), skipping eval split detection\n")
 
             if dataset is None:
                 raise ValueError("No dataset provided")
 
+            # Apply index range slicing if requested (inclusive on both ends)
+            if dataset_slice_start is not None or dataset_slice_end is not None:
+                total_rows = len(dataset)
+                start = dataset_slice_start if dataset_slice_start is not None else 0
+                end = dataset_slice_end if dataset_slice_end is not None else total_rows - 1
+                # Clamp to valid range
+                start = max(0, min(start, total_rows - 1))
+                end = max(start, min(end, total_rows - 1))
+                dataset = dataset.select(range(start, end + 1))
+                print(f"Sliced dataset to rows [{start}, {end}]: {len(dataset)} of {total_rows} rows\n")
+                self._update_progress(status_message=f"Sliced dataset to {len(dataset)} rows (indices {start}-{end})")
+
             # Check if stopped before applying template
             if self.should_stop:
                 print("Stopped before applying chat template\n")
@@ -449,6 +477,7 @@ class UnslothTrainer:
                 format_type=format_type,
                 dataset_name=dataset_source,
                 custom_format_mapping=custom_format_mapping,
+                progress_callback=self._update_progress,
             )
 
             # Check if stopped during formatting
@@ -456,6 +485,14 @@ class UnslothTrainer:
                 print("Stopped during dataset formatting\n")
                 return None
 
+            # Abort if dataset formatting/conversion failed
+            if not dataset_info.get("success", True):
+                errors = dataset_info.get("errors", [])
+                error_msg = "; ".join(errors) if errors else "Dataset formatting failed"
+                logger.error(f"Dataset conversion failed: {error_msg}")
+                self._update_progress(error=error_msg)
+                return None
+
             self._update_progress(status_message=f"Dataset formatted and ready for training")
             print(f"Dataset formatted successfully\n")
 
@@ -474,7 +511,7 @@ class UnslothTrainer:
                 )
                 eval_dataset = eval_info["dataset"]
                 print(f"Eval dataset formatted successfully\n")
-            elif not has_separate_eval_source:
+            elif eval_enabled and not has_separate_eval_source:
                 # No separate eval source — split the already-formatted dataset
                 formatted_dataset = dataset_info["dataset"]
                 split_result = self._resolve_eval_split_from_dataset(formatted_dataset)
@@ -545,7 +582,7 @@ class UnslothTrainer:
     def start_training(self,
                        dataset: Dataset,
                        eval_dataset: Dataset = None,
-                       eval_steps: float = 0.01,
+                       eval_steps: float = 0.00,
                        output_dir: str = "./outputs",
                        num_epochs: int = 3,
                        learning_rate: float = 5e-5,
@@ -713,7 +750,7 @@ class UnslothTrainer:
                 "output_dir": output_dir,
                 "report_to": ["wandb"] if training_args.get('enable_wandb', False) else "none",
                 "include_num_input_tokens_seen": True,  # Enable token counting
-                "dataset_num_proc": max(1, os.cpu_count() // 4),
+                "dataset_num_proc": safe_num_proc(max(1, os.cpu_count() // 4)),
             }
             
             # Add warmup parameter - use warmup_ratio if provided, otherwise warmup_steps
@@ -745,12 +782,16 @@ class UnslothTrainer:
 
             # ========== EVAL CONFIGURATION ==========
             eval_dataset = training_args.get('eval_dataset', None)
-            eval_steps_val = training_args.get('eval_steps', 0.01)
+            eval_steps_val = training_args.get('eval_steps', 0.00)
             if eval_dataset is not None:
-                config_args["eval_strategy"] = "steps"
-                config_args["eval_steps"] = eval_steps_val
-                print(f"Evaluation enabled: eval_steps={eval_steps_val} (fraction of total steps)\n")
-                print(f"Eval dataset: {len(eval_dataset)} rows\n")
+                if eval_steps_val > 0:
+                    config_args["eval_strategy"] = "steps"
+                    config_args["eval_steps"] = eval_steps_val
+                    print(f"✅ Evaluation enabled: eval_steps={eval_steps_val} (fraction of total steps)\n")
+                    print(f"Eval dataset: {len(eval_dataset)} rows\n")
+                else:
+                    print(f"⚠️  Eval dataset provided but eval_steps={eval_steps_val} (disabled)\n")
+                    print("To enable evaluation, set eval_steps > 0.0\n")
             else:
                 print("No eval dataset — evaluation disabled\n")
 
@@ -873,7 +914,7 @@ class UnslothTrainer:
                         self.trainer,
                         instruction_part=instruction_part,
                         response_part=response_part,
-                        num_proc=config_args.get("dataset_num_proc", max(1, os.cpu_count() // 4)),
+                        num_proc=config_args.get("dataset_num_proc", safe_num_proc(max(1, os.cpu_count() // 4))),
                     )
                     print("Train on responses only configured successfully\n")
 

studio/backend/core/training/training.py

@@ -115,8 +115,10 @@ class TrainingBackend:
                        subset: str = None,
                        train_split: str = "train",
                        eval_split: str = None,
-                       eval_steps: float = 0.01,
-                       is_dataset_multimodal: bool = False) -> bool:
+                       eval_steps: float = 0.00,
+                       is_dataset_multimodal: bool = False,
+                       dataset_slice_start: int = None,
+                       dataset_slice_end: int = None) -> bool:
         """
         Start training.
 
@@ -223,6 +225,9 @@ class TrainingBackend:
                 subset=subset,
                 train_split=train_split,
                 eval_split=eval_split,
+                eval_steps=eval_steps,
+                dataset_slice_start=dataset_slice_start,
+                dataset_slice_end=dataset_slice_end,
             )
 
             # Unpack: load_and_format_dataset returns (dataset, eval_dataset)
@@ -232,10 +237,6 @@ class TrainingBackend:
                 dataset = dataset_result
                 eval_dataset = None
 
-            # If user set eval_steps to 0, disable evaluation entirely
-            if eval_steps is not None and float(eval_steps) <= 0:
-                eval_dataset = None
-
             # Track whether eval is enabled for status reporting
             self.eval_enabled = eval_dataset is not None
 

studio/backend/main.py

@@ -9,12 +9,20 @@ from contextlib import asynccontextmanager
 from fastapi import FastAPI
 from fastapi.middleware.cors import CORSMiddleware
 from fastapi.staticfiles import StaticFiles
-from fastapi.responses import FileResponse
+from fastapi.responses import FileResponse, HTMLResponse, Response
 from pathlib import Path
 from datetime import datetime
 
 # Import routers
-from routes import training_router, models_router, inference_router, datasets_router, auth_router, export_router
+from routes import (
+    auth_router,
+    data_recipe_router,
+    datasets_router,
+    export_router,
+    inference_router,
+    models_router,
+    training_router,
+)
 from auth import storage
 from utils.hardware import detect_hardware, get_device, DeviceType
 import utils.hardware.hardware as _hw_module
@@ -89,6 +97,7 @@ app.include_router(training_router, prefix="/api/train", tags=["training"])
 app.include_router(models_router, prefix="/api/models", tags=["models"])
 app.include_router(inference_router, prefix="/api/inference", tags=["inference"])
 app.include_router(datasets_router, prefix="/api/datasets", tags=["datasets"])
+app.include_router(data_recipe_router, prefix="/api/data-recipe", tags=["data-recipe"])
 app.include_router(export_router, prefix="/api/export", tags=["export"])
 
 
@@ -154,27 +163,44 @@ async def get_hardware_info():
 
 def setup_frontend(app: FastAPI, build_path: Path):
     """Mount frontend static files (optional)"""
-    if build_path.exists():
-        # Mount assets
-        assets_dir = build_path / "assets"
-        if assets_dir.exists():
-            app.mount("/assets", StaticFiles(directory=assets_dir), name="assets")
+    if not build_path.exists():
+        return False
 
-        @app.get("/")
-        async def serve_root():
-            return FileResponse(build_path / "index.html", headers={"Cache-Control": "no-cache, no-store, must-revalidate"})
+    # Mount assets
+    assets_dir = build_path / "assets"
+    if assets_dir.exists():
+        app.mount("/assets", StaticFiles(directory=assets_dir), name="assets")
 
-        @app.get("/{full_path:path}")
-        async def serve_frontend(full_path: str):
-            if full_path.startswith("api"):
-                return {"error": "API endpoint not found"}
+    @app.get("/")
+    async def serve_root():
+        content = (build_path / "index.html").read_bytes()
+        return Response(
+            content=content,
+            media_type="text/html",
+            headers={"Cache-Control": "no-cache, no-store, must-revalidate"},
+        )
 
-            file_path = build_path / full_path
-            if file_path.is_file():
-                return FileResponse(file_path)
+    @app.get("/{full_path:path}")
+    async def serve_frontend(full_path: str):
+        if full_path.startswith("api"):
+            return {"error": "API endpoint not found"}
 
-            return FileResponse(build_path / "index.html", headers={"Cache-Control": "no-cache, no-store, must-revalidate"})
+        file_path = (build_path / full_path).resolve()
 
-        return True
-    return False
+        # Block path traversal — ensure resolved path stays inside build_path
+        if not str(file_path).startswith(str(build_path.resolve())):
+            return Response(status_code=403)
+
+        if file_path.is_file():
+            return FileResponse(file_path)
+
+        # Serve index.html as bytes — avoids Content-Length mismatch
+        content = (build_path / "index.html").read_bytes()
+        return Response(
+            content=content,
+            media_type="text/html",
+            headers={"Cache-Control": "no-cache, no-store, must-revalidate"},
+        )
+
+    return True
 

studio/backend/models/__init__.py

@@ -52,6 +52,13 @@ from .responses import (
     LoRABaseModelResponse,
     VisionCheckResponse,
 )
+from .data_recipe import (
+    RecipePayload,
+    PreviewResponse,
+    ValidateError,
+    ValidateResponse,
+    JobCreateResponse,
+)
 
 __all__ = [
     # Training schemas
@@ -98,4 +105,10 @@ __all__ = [
     "TrainingMetricsResponse",
     "LoRABaseModelResponse",
     "VisionCheckResponse",
+    # Data recipe
+    "RecipePayload",
+    "PreviewResponse",
+    "ValidateError",
+    "ValidateResponse",
+    "JobCreateResponse",
 ]

studio/backend/models/data_recipe.py

@@ -0,0 +1,60 @@
+"""
+Pydantic schemas for Data Recipe (DataDesigner) API.
+"""
+
+from __future__ import annotations
+
+from typing import Any
+
+from pydantic import BaseModel, Field
+
+
+class RecipePayload(BaseModel):
+    recipe: dict[str, Any] = Field(default_factory=dict)
+    run: dict[str, Any] | None = None
+    ui: dict[str, Any] | None = None
+
+
+class PreviewResponse(BaseModel):
+    dataset: list[dict[str, Any]] = Field(default_factory=list)
+    processor_artifacts: dict[str, Any] | None = None
+    analysis: dict[str, Any] | None = None
+
+
+class ValidateError(BaseModel):
+    message: str
+    path: str | None = None
+    code: str | None = None
+
+
+class ValidateResponse(BaseModel):
+    valid: bool
+    errors: list[ValidateError] = Field(default_factory=list)
+    raw_detail: str | None = None
+
+
+class JobCreateResponse(BaseModel):
+    job_id: str
+
+
+class SeedInspectRequest(BaseModel):
+    dataset_name: str = Field(min_length=1)
+    hf_token: str | None = None
+    subset: str | None = None
+    split: str | None = "train"
+    preview_size: int = Field(default=10, ge=1, le=50)
+
+
+class SeedInspectUploadRequest(BaseModel):
+    filename: str = Field(min_length=1)
+    content_base64: str = Field(min_length=1)
+    preview_size: int = Field(default=10, ge=1, le=50)
+
+
+class SeedInspectResponse(BaseModel):
+    dataset_name: str
+    resolved_path: str
+    columns: list[str] = Field(default_factory=list)
+    preview_rows: list[dict[str, Any]] = Field(default_factory=list)
+    split: str | None = None
+    subset: str | None = None

studio/backend/models/datasets.py

@@ -34,3 +34,4 @@ class CheckFormatResponse(BaseModel):
     detected_text_column: Optional[str] = None
     preview_samples: Optional[List[Dict]] = None
     total_rows: Optional[int] = None
+    warning: Optional[str] = None

studio/backend/models/inference.py

@@ -17,6 +17,7 @@ class LoadRequest(BaseModel):
     max_seq_length: int = Field(2048, ge=128, le=32768, description="Maximum sequence length")
     load_in_4bit: bool = Field(True, description="Load model in 4-bit quantization")
     is_lora: bool = Field(False, description="Whether this is a LoRA adapter")
+    gguf_variant: Optional[str] = Field(None, description="GGUF quantization variant (e.g. 'Q4_K_M')")
 
 
 class UnloadRequest(BaseModel):
@@ -43,6 +44,7 @@ class LoadResponse(BaseModel):
     display_name: str = Field(..., description="Display name of the model")
     is_vision: bool = Field(False, description="Whether model is a vision model")
     is_lora: bool = Field(False, description="Whether model is a LoRA adapter")
+    is_gguf: bool = Field(False, description="Whether model is a GGUF model (llama.cpp)")
     inference: dict = Field(..., description="Inference parameters (temperature, top_p, top_k, min_p)")
 
 
@@ -56,6 +58,8 @@ class InferenceStatusResponse(BaseModel):
     """Current inference backend status"""
     active_model: Optional[str] = Field(None, description="Currently active model identifier")
     is_vision: bool = Field(False, description="Whether the active model is a vision model")
+    is_gguf: bool = Field(False, description="Whether the active model is a GGUF model (llama.cpp)")
+    gguf_variant: Optional[str] = Field(None, description="GGUF quantization variant (e.g. Q4_K_M)")
     loading: List[str] = Field(default_factory=list, description="Models currently being loaded")
     loaded: List[str] = Field(default_factory=list, description="Models currently loaded")
 

studio/backend/models/models.py

@@ -53,14 +53,17 @@ class ModelDetails(BaseModel):
     config: Optional[Dict[str, Any]] = Field(None, description="Model configuration dictionary")
     is_vision: bool = Field(False, description="Whether model is a vision model")
     is_lora: bool = Field(False, description="Whether model is a LoRA adapter")
+    is_gguf: bool = Field(False, description="Whether model is a GGUF model (llama.cpp format)")
     base_model: Optional[str] = Field(None, description="Base model if this is a LoRA adapter")
 
 
 class LoRAInfo(BaseModel):
-    """LoRA adapter information"""
+    """LoRA adapter or exported model information"""
     display_name: str = Field(..., description="Display name for the LoRA")
-    adapter_path: str = Field(..., description="Path to the LoRA adapter")
+    adapter_path: str = Field(..., description="Path to the LoRA adapter or exported model")
     base_model: Optional[str] = Field(None, description="Base model identifier")
+    source: Optional[str] = Field(None, description="'training' or 'exported'")
+    export_type: Optional[str] = Field(None, description="'lora', 'merged', or 'gguf' (for exports)")
 
 
 class LoRAScanResponse(BaseModel):
@@ -75,6 +78,21 @@ class ModelListResponse(BaseModel):
     default_models: List[str] = Field(default_factory=list, description="List of default model IDs")
 
 
+class GgufVariantDetail(BaseModel):
+    """A single GGUF quantization variant in a HuggingFace repo."""
+    filename: str = Field(..., description="GGUF filename (e.g., 'gemma-3-4b-it-Q4_K_M.gguf')")
+    quant: str = Field(..., description="Quantization label (e.g., 'Q4_K_M')")
+    size_bytes: int = Field(0, description="File size in bytes")
+
+
+class GgufVariantsResponse(BaseModel):
+    """Response for listing GGUF quantization variants in a HuggingFace repo."""
+    repo_id: str = Field(..., description="HuggingFace repo ID")
+    variants: List[GgufVariantDetail] = Field(default_factory=list, description="Available GGUF variants")
+    has_vision: bool = Field(False, description="Whether the model has vision support (mmproj files)")
+    default_variant: Optional[str] = Field(None, description="Recommended default quantization variant")
+
+
 class LocalModelInfo(BaseModel):
     """Discovered local model candidate."""
     id: str = Field(..., description="Identifier to use for loading/training")

studio/backend/models/training.py

@@ -21,7 +21,9 @@ class TrainingStartRequest(BaseModel):
     subset: Optional[str] = None
     train_split: Optional[str] = Field("train", description="Training split name")
     eval_split: Optional[str] = Field(None, description="Eval split name. None = auto-detect")
-    eval_steps: float = Field(0.01, description="Fraction of total steps between evals (0-1)")
+    eval_steps: float = Field(0.00, description="Fraction of total steps between evals (0-1)")
+    dataset_slice_start: Optional[int] = Field(None, description="Inclusive start row index for dataset slicing")
+    dataset_slice_end: Optional[int] = Field(None, description="Inclusive end row index for dataset slicing")
 
     @model_validator(mode="before")
     @classmethod

studio/backend/requirements/extras.txt

@@ -51,6 +51,6 @@ addict
 easydict
 einops
 tabulate
-fastmcp>=2.0.0
+fastmcp>=3.0.2
 openai>=2.7.2
-websockets>=13.0,<14
+websockets>=15.0.1

studio/backend/requirements/single-env/constraints.txt

@@ -0,0 +1,16 @@
+# Single-env pins for unsloth + studio + data-designer
+# Keep compatible with unsloth transformers bounds.
+transformers==4.57.1
+trl==0.23.1
+huggingface-hub==0.36.2
+
+# Studio stack
+datasets==4.3.0
+pyarrow==23.0.1
+
+# FastMCP/OpenEnv compat
+fastmcp>=3.0.2
+mcp>=1.24,<2
+websockets>=15.0.1
+
+pandas==2.3.3

studio/backend/requirements/single-env/data-designer-deps.txt

@@ -0,0 +1,18 @@
+# Data Designer runtime deps installed explicitly (single-env mode).
+anyascii<1,>=0.3.3
+duckdb<2,>=1.1.3
+faker<21,>=20.1.0
+httpx<1,>=0.27.2
+httpx-retries<1,>=0.4.2
+json-repair<1,>=0.48.0
+jsonpath-rust-bindings<2,>=1.0
+jsonschema<5,>=4.0.0
+litellm<1.80.12,>=1.73.6
+lxml<7,>=6.0.2
+marko<3,>=2.1.2
+networkx<4,>=3.0
+python-json-logger<4,>=3
+ruff<1,>=0.14.10
+scipy<2,>=1.11.0
+sqlfluff<4,>=3.2.0
+tiktoken<1,>=0.8.0

studio/backend/requirements/single-env/data-designer.txt

@@ -0,0 +1,5 @@
+# Install Data Designer in same env as Unsloth.
+data-designer==0.5.1
+data-designer-config==0.5.1
+data-designer-engine==0.5.1
+prompt-toolkit>=3,<4

studio/backend/requirements/single-env/patch_metadata.py

@@ -0,0 +1,74 @@
+#!/usr/bin/env python3
+"""Relax strict metadata pins so pip check matches known working single-env stack.
+
+Why:
+- data-designer pins huggingface-hub>=1.0.1 and pyarrow<20.
+- unsloth/transformers pins huggingface-hub<1.
+- studio datasets pins pyarrow>=21.
+
+Runtime works in this app with hub 0.36.x + pyarrow 23.x, but metadata conflicts.
+"""
+
+from __future__ import annotations
+
+import importlib.metadata as im
+import re
+from pathlib import Path
+
+TARGETS = (
+    "data-designer",
+    "data-designer-engine",
+    "data-designer-config",
+)
+
+PATCHES: tuple[tuple[re.Pattern[str], str], ...] = (
+    (
+        re.compile(r"^Requires-Dist: huggingface-hub<2,>=1\.0\.1$", re.MULTILINE),
+        "Requires-Dist: huggingface-hub<2,>=0.34.0",
+    ),
+    (
+        re.compile(r"^Requires-Dist: pyarrow<20,>=19\.0\.1$", re.MULTILINE),
+        "Requires-Dist: pyarrow>=21.0.0",
+    ),
+)
+
+
+def metadata_path(dist_name: str) -> Path | None:
+    try:
+        dist = im.distribution(dist_name)
+    except im.PackageNotFoundError:
+        return None
+    for f in dist.files or []:
+        sf = str(f)
+        if sf.endswith(".dist-info/METADATA"):
+            return Path(dist.locate_file(f))
+    return None
+
+
+def patch_file(path: Path) -> bool:
+    original = path.read_text(encoding="utf-8")
+    updated = original
+    for pattern, repl in PATCHES:
+        updated = pattern.sub(repl, updated)
+    if updated == original:
+        return False
+    path.write_text(updated, encoding="utf-8")
+    return True
+
+
+def main() -> int:
+    changed = 0
+    checked = 0
+    for name in TARGETS:
+        p = metadata_path(name)
+        if p is None:
+            continue
+        checked += 1
+        if patch_file(p):
+            changed += 1
+    print(f"single-env metadata patch: checked={checked}, changed={changed}")
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

studio/backend/requirements/studio.txt

@@ -11,4 +11,4 @@ pyjwt
 easydict
 addict
 gradio>=4.0.0
-huggingface-hub==0.36.0
+huggingface-hub==0.36.2

studio/backend/routes/__init__.py

@@ -7,6 +7,7 @@ from routes.models import router as models_router
 from routes.inference import router as inference_router
 from routes.datasets import router as datasets_router
 from routes.auth import router as auth_router
+from routes.data_recipe import router as data_recipe_router
 from routes.export import router as export_router
 
 __all__ = [
@@ -15,5 +16,6 @@ __all__ = [
     "inference_router",
     "datasets_router",
     "auth_router",
+    "data_recipe_router",
     "export_router",
-]
+]

studio/backend/routes/data_recipe/__init__.py

@@ -0,0 +1,23 @@
+"""Data Recipe route package."""
+
+from __future__ import annotations
+
+import sys
+from pathlib import Path
+
+from fastapi import APIRouter
+
+backend_path = Path(__file__).parent.parent.parent
+if str(backend_path) not in sys.path:
+    sys.path.insert(0, str(backend_path))
+
+from .jobs import router as jobs_router
+from .seed import router as seed_router
+from .validate import router as validate_router
+
+router = APIRouter()
+router.include_router(seed_router)
+router.include_router(validate_router)
+router.include_router(jobs_router)
+
+__all__ = ["router"]

studio/backend/routes/data_recipe/jobs.py

@@ -0,0 +1,143 @@
+"""Job lifecycle endpoints for data recipe."""
+
+from __future__ import annotations
+
+from typing import Any
+
+from fastapi import APIRouter, HTTPException, Query, Request
+from fastapi.responses import JSONResponse, StreamingResponse
+from pydantic import ValidationError
+
+from core.data_recipe.jobs import get_job_manager
+from models.data_recipe import JobCreateResponse, RecipePayload
+
+router = APIRouter()
+
+
+@router.post("/jobs", response_class=JSONResponse, response_model=JobCreateResponse)
+def create_job(payload: RecipePayload):
+    recipe = payload.recipe
+    if not recipe.get("columns"):
+        raise HTTPException(status_code=400, detail="Recipe must include columns.")
+
+    run: dict[str, Any] = payload.run or {}
+    run.pop("artifact_path", None)
+    run.pop("dataset_name", None)
+    execution_type = str(run.get("execution_type") or "full").strip().lower()
+    if execution_type not in {"preview", "full"}:
+        raise HTTPException(status_code=400, detail="invalid execution_type: must be 'preview' or 'full'")
+    run["execution_type"] = execution_type
+    run_config_raw = run.get("run_config")
+    if run_config_raw is not None:
+        try:
+            from data_designer.config.run_config import RunConfig
+
+            RunConfig.model_validate(run_config_raw)
+        except (ImportError, ValidationError, TypeError, ValueError) as exc:
+            raise HTTPException(status_code=400, detail=f"invalid run_config: {exc}") from exc
+
+    mgr = get_job_manager()
+    try:
+        job_id = mgr.start(recipe=recipe, run=run)
+    except RuntimeError as exc:
+        raise HTTPException(status_code=409, detail=str(exc)) from exc
+    except ValueError as exc:
+        raise HTTPException(status_code=400, detail=str(exc)) from exc
+
+    return {"job_id": job_id}
+
+
+@router.get("/jobs/{job_id}/status")
+def job_status(job_id: str):
+    mgr = get_job_manager()
+    state = mgr.get_status(job_id)
+    if state is None:
+        raise HTTPException(status_code=404, detail="job not found")
+    return state
+
+
+@router.get("/jobs/current")
+def current_job():
+    mgr = get_job_manager()
+    state = mgr.get_current_status()
+    if state is None:
+        raise HTTPException(status_code=404, detail="no job")
+    return state
+
+
+@router.post("/jobs/{job_id}/cancel")
+def cancel_job(job_id: str):
+    mgr = get_job_manager()
+    ok = mgr.cancel(job_id)
+    if not ok:
+        raise HTTPException(status_code=404, detail="job not found")
+    return mgr.get_status(job_id)
+
+
+@router.get("/jobs/{job_id}/analysis")
+def job_analysis(job_id: str):
+    mgr = get_job_manager()
+    analysis = mgr.get_analysis(job_id)
+    if analysis is None:
+        raise HTTPException(status_code=404, detail="analysis not ready")
+    return analysis
+
+
+@router.get("/jobs/{job_id}/dataset")
+def job_dataset(
+    job_id: str,
+    limit: int = Query(default=20, ge=1, le=500),
+    offset: int = Query(default=0, ge=0),
+):
+    mgr = get_job_manager()
+    result = mgr.get_dataset(job_id, limit=limit, offset=offset)
+    if result is None:
+        raise HTTPException(status_code=404, detail="dataset not ready")
+    if "error" in result:
+        raise HTTPException(status_code=422, detail=result["error"])
+    return {
+        "dataset": result["dataset"],
+        "total": result["total"],
+        "limit": limit,
+        "offset": offset,
+    }
+
+
+@router.get("/jobs/{job_id}/events")
+async def job_events(request: Request, job_id: str):
+    mgr = get_job_manager()
+    last_id = request.headers.get("last-event-id")
+    after_seq: int | None = None
+    if last_id:
+        try:
+            after_seq = int(str(last_id).strip())
+        except (TypeError, ValueError):
+            after_seq = None
+
+    after_q = request.query_params.get("after")
+    if after_q:
+        try:
+            after_seq = int(str(after_q).strip())
+        except (TypeError, ValueError):
+            pass
+
+    sub = mgr.subscribe(job_id, after_seq=after_seq)
+    if sub is None:
+        raise HTTPException(status_code=404, detail="job not found")
+
+    async def gen():
+        try:
+            for event in sub.replay:
+                yield sub.format_sse(event)
+
+            while True:
+                if await request.is_disconnected():
+                    break
+                event = await sub.next_event(timeout_sec=1.0)
+                if event is None:
+                    continue
+                yield sub.format_sse(event)
+        finally:
+            mgr.unsubscribe(sub)
+
+    return StreamingResponse(gen(), media_type="text/event-stream")

studio/backend/routes/data_recipe/seed.py

@@ -0,0 +1,298 @@
+"""Seed inspect endpoints for data recipe."""
+
+from __future__ import annotations
+
+import base64
+import binascii
+from itertools import islice
+from pathlib import Path
+from typing import Any
+from uuid import uuid4
+
+from fastapi import APIRouter, HTTPException
+
+from models.data_recipe import (
+    SeedInspectRequest,
+    SeedInspectResponse,
+    SeedInspectUploadRequest,
+)
+
+router = APIRouter()
+
+DATA_EXTS = (".parquet", ".jsonl", ".json", ".csv")
+DEFAULT_SPLIT = "train"
+LOCAL_UPLOAD_EXTS = {".csv", ".json", ".jsonl"}
+SEED_UPLOAD_DIR = Path.home() / ".cache" / "unsloth" / "data-recipe" / "seed-uploads"
+
+
+def _serialize_preview_value(value: Any) -> Any:
+    if value is None or isinstance(value, (str, int, float, bool)):
+        return value
+    if isinstance(value, dict):
+        return {str(key): _serialize_preview_value(item) for key, item in value.items()}
+    if isinstance(value, (list, tuple)):
+        return [_serialize_preview_value(item) for item in value]
+    return str(value)
+
+
+def _serialize_preview_rows(rows: list[dict[str, Any]]) -> list[dict[str, Any]]:
+    return [
+        {str(key): _serialize_preview_value(value) for key, value in row.items()}
+        for row in rows
+    ]
+
+
+def _normalize_optional_text(value: str | None) -> str | None:
+    if value is None:
+        return None
+    trimmed = value.strip()
+    return trimmed if trimmed else None
+
+
+def _list_hf_data_files(*, dataset_name: str, token: str | None) -> list[str]:
+    try:
+        from huggingface_hub import HfApi
+        from huggingface_hub.utils import HfHubHTTPError
+    except ImportError:
+        return []
+    try:
+        api = HfApi()
+        repo_files = api.list_repo_files(dataset_name, repo_type="dataset", token=token)
+        return [file for file in repo_files if file.lower().endswith(DATA_EXTS)]
+    except (HfHubHTTPError, OSError, ValueError):
+        return []
+
+
+def _select_best_file(data_files: list[str]) -> str | None:
+    if not data_files:
+        return None
+    split_lower = DEFAULT_SPLIT
+
+    def score(path: str) -> tuple[int, int]:
+        name = path.lower()
+        if f"/{split_lower}/" in name:
+            return (0, len(path))
+        if (
+            f"_{split_lower}." in name
+            or f"-{split_lower}." in name
+            or f"/{split_lower}." in name
+            or f"/{split_lower}_" in name
+            or f"/{split_lower}-" in name
+        ):
+            return (1, len(path))
+        return (2, len(path))
+
+    return sorted(data_files, key=score)[0]
+
+
+def _resolve_seed_hf_path(dataset_name: str, data_files: list[str]) -> str | None:
+    selected = _select_best_file(data_files)
+    if not selected:
+        return None
+
+    ext = Path(selected).suffix.lower()
+    if ext not in DATA_EXTS:
+        return f"datasets/{dataset_name}/{selected}"
+
+    parent = Path(selected).parent.as_posix()
+    if not parent or parent == ".":
+        return f"datasets/{dataset_name}/**/*{ext}"
+    return f"datasets/{dataset_name}/{parent}/**/*{ext}"
+
+
+def _build_stream_load_kwargs(
+    *,
+    dataset_name: str,
+    split: str,
+    subset: str | None,
+    token: str | None,
+    data_file: str | None = None,
+) -> dict[str, Any]:
+    kwargs: dict[str, Any] = {
+        "path": dataset_name,
+        "split": split,
+        "streaming": True,
+    }
+    if data_file:
+        kwargs["data_files"] = [data_file]
+    if subset:
+        kwargs["name"] = subset
+    if token:
+        kwargs["token"] = token
+    return kwargs
+
+
+def _load_preview_rows(
+    *,
+    load_dataset_fn,
+    load_kwargs: dict[str, Any],
+    preview_size: int,
+) -> list[dict[str, Any]]:
+    streamed_ds = load_dataset_fn(**load_kwargs)
+    return [row for row in islice(streamed_ds, preview_size)]
+
+
+def _extract_columns(rows: list[dict[str, Any]]) -> list[str]:
+    columns_seen: dict[str, None] = {}
+    for row in rows:
+        for key in row.keys():
+            columns_seen[str(key)] = None
+    return list(columns_seen.keys())
+
+
+def _sanitize_filename(filename: str) -> str:
+    name = Path(filename).name.strip().replace("\x00", "")
+    if not name:
+        return "seed_upload"
+    return name
+
+
+def _decode_base64_payload(content_base64: str) -> bytes:
+    raw = content_base64.strip()
+    if "," in raw and raw.lower().startswith("data:"):
+        raw = raw.split(",", 1)[1]
+    try:
+        return base64.b64decode(raw, validate=True)
+    except binascii.Error as exc:
+        raise HTTPException(status_code=400, detail="invalid base64 payload") from exc
+
+
+def _read_preview_rows_from_local_file(path: Path, preview_size: int) -> list[dict[str, Any]]:
+    try:
+        import pandas as pd
+    except ImportError as exc:
+        raise HTTPException(status_code=500, detail=f"seed inspect dependencies unavailable: {exc}") from exc
+
+    ext = path.suffix.lower()
+    try:
+        if ext == ".csv":
+            df = pd.read_csv(path, nrows=preview_size)
+        elif ext == ".jsonl":
+            df = pd.read_json(path, lines=True).head(preview_size)
+        elif ext == ".json":
+            try:
+                df = pd.read_json(path, lines=True).head(preview_size)
+            except ValueError:
+                df = pd.read_json(path).head(preview_size)
+        else:
+            raise HTTPException(status_code=422, detail=f"unsupported file type: {ext}")
+    except HTTPException:
+        raise
+    except (ValueError, OSError) as exc:
+        raise HTTPException(status_code=422, detail=f"seed inspect failed: {exc}") from exc
+
+    rows = df.to_dict(orient="records")
+    return _serialize_preview_rows(rows)
+
+
+@router.post("/seed/inspect", response_model=SeedInspectResponse)
+def inspect_seed_dataset(payload: SeedInspectRequest) -> SeedInspectResponse:
+    dataset_name = payload.dataset_name.strip()
+    if not dataset_name or dataset_name.count("/") < 1:
+        raise HTTPException(status_code=400, detail="dataset_name must be a Hugging Face repo id like org/repo")
+
+    try:
+        from datasets import load_dataset
+    except ImportError as exc:
+        raise HTTPException(status_code=500, detail=f"seed inspect dependencies unavailable: {exc}") from exc
+
+    split = DEFAULT_SPLIT
+    subset = _normalize_optional_text(payload.subset)
+    token = _normalize_optional_text(payload.hf_token)
+    preview_size = int(payload.preview_size)
+
+    preview_rows: list[dict[str, Any]] = []
+    data_files = _list_hf_data_files(dataset_name=dataset_name, token=token)
+
+    selected_file = _select_best_file(data_files)
+    if selected_file:
+        try:
+            single_file_kwargs = _build_stream_load_kwargs(
+                dataset_name=dataset_name,
+                split=DEFAULT_SPLIT,
+                subset=subset,
+                token=token,
+                data_file=selected_file,
+            )
+            preview_rows = _load_preview_rows(
+                load_dataset_fn=load_dataset,
+                load_kwargs=single_file_kwargs,
+                preview_size=preview_size,
+            )
+        except (ValueError, OSError, RuntimeError):
+            preview_rows = []
+
+    if not preview_rows:
+        try:
+            split_kwargs = _build_stream_load_kwargs(
+                dataset_name=dataset_name,
+                split=split,
+                subset=subset,
+                token=token,
+            )
+            preview_rows = _load_preview_rows(
+                load_dataset_fn=load_dataset,
+                load_kwargs=split_kwargs,
+                preview_size=preview_size,
+            )
+        except (ValueError, OSError, RuntimeError) as exc:
+            raise HTTPException(status_code=422, detail=f"seed inspect failed: {exc}") from exc
+
+    if not preview_rows:
+        raise HTTPException(status_code=422, detail="dataset appears empty or unreadable")
+    preview_rows = _serialize_preview_rows(preview_rows)
+    columns = _extract_columns(preview_rows)
+
+    if not data_files:
+        resolved_path = f"datasets/{dataset_name}/**/*.parquet"
+    else:
+        resolved_path = _resolve_seed_hf_path(dataset_name, data_files)
+        if not resolved_path:
+            raise HTTPException(status_code=422, detail="unable to resolve seed dataset path")
+
+    return SeedInspectResponse(
+        dataset_name=dataset_name,
+        resolved_path=resolved_path,
+        columns=columns,
+        preview_rows=preview_rows,
+        split=None,
+        subset=subset,
+    )
+
+
+@router.post("/seed/inspect-upload", response_model=SeedInspectResponse)
+def inspect_seed_upload(payload: SeedInspectUploadRequest) -> SeedInspectResponse:
+    filename = _sanitize_filename(payload.filename)
+    ext = Path(filename).suffix.lower()
+    if ext not in LOCAL_UPLOAD_EXTS:
+        allowed = ", ".join(sorted(LOCAL_UPLOAD_EXTS))
+        raise HTTPException(status_code=400, detail=f"unsupported file type: {ext}. allowed: {allowed}")
+
+    file_bytes = _decode_base64_payload(payload.content_base64)
+    if not file_bytes:
+        raise HTTPException(status_code=400, detail="empty upload payload")
+    max_size_bytes = 50 * 1024 * 1024
+    if len(file_bytes) > max_size_bytes:
+        raise HTTPException(status_code=413, detail="file too large (max 50MB)")
+
+    SEED_UPLOAD_DIR.mkdir(parents=True, exist_ok=True)
+    stored_name = f"{uuid4().hex}_{filename}"
+    stored_path = SEED_UPLOAD_DIR / stored_name
+    stored_path.write_bytes(file_bytes)
+
+    preview_rows = _read_preview_rows_from_local_file(
+        stored_path,
+        int(payload.preview_size),
+    )
+    if not preview_rows:
+        raise HTTPException(status_code=422, detail="dataset appears empty or unreadable")
+    columns = _extract_columns(preview_rows)
+
+    return SeedInspectResponse(
+        dataset_name=filename,
+        resolved_path=str(stored_path),
+        columns=columns,
+        preview_rows=preview_rows,
+        split=None,
+        subset=None,
+    )

studio/backend/routes/data_recipe/validate.py

@@ -0,0 +1,90 @@
+"""Validation endpoints for data recipe."""
+
+from __future__ import annotations
+
+from typing import Any
+
+from fastapi import APIRouter, HTTPException
+
+from core.data_recipe.service import (
+    build_config_builder,
+    create_data_designer,
+    validate_recipe,
+)
+from models.data_recipe import RecipePayload, ValidateError, ValidateResponse
+
+router = APIRouter()
+
+
+def _collect_validation_errors(recipe: dict[str, Any]) -> list[ValidateError]:
+    try:
+        from data_designer.engine.compiler import (
+            _add_internal_row_id_column_if_needed,
+            _get_allowed_references,
+            _resolve_and_add_seed_columns,
+        )
+        from data_designer.engine.validation import (
+            ViolationLevel,
+            validate_data_designer_config,
+        )
+    except ImportError:
+        return []
+
+    try:
+        builder = build_config_builder(recipe)
+        designer = create_data_designer(recipe)
+        resource_provider = designer._create_resource_provider(  # type: ignore[attr-defined]
+            "validate-configuration",
+            builder,
+        )
+        config = builder.build()
+        _resolve_and_add_seed_columns(config, resource_provider.seed_reader)
+        _add_internal_row_id_column_if_needed(config)
+        violations = validate_data_designer_config(
+            columns=config.columns,
+            processor_configs=config.processors or [],
+            allowed_references=_get_allowed_references(config),
+        )
+    except (TypeError, ValueError, AttributeError):
+        return []
+
+    errors: list[ValidateError] = []
+    for violation in violations:
+        if violation.level != ViolationLevel.ERROR:
+            continue
+        code = getattr(violation.type, "value", None)
+        path = violation.column if violation.column else None
+        message = str(violation.message).strip() or "Validation failed."
+        errors.append(
+            ValidateError(
+                message=message,
+                path=path,
+                code=code,
+            )
+        )
+    return errors
+
+
+@router.post("/validate", response_model=ValidateResponse)
+def validate(payload: RecipePayload) -> ValidateResponse:
+    recipe = payload.recipe
+    if not recipe.get("columns"):
+        return ValidateResponse(
+            valid=False,
+            errors=[ValidateError(message="Recipe must include columns.")],
+        )
+
+    try:
+        validate_recipe(recipe)
+    except RuntimeError as exc:
+        raise HTTPException(status_code=503, detail=str(exc)) from exc
+    except Exception as exc:
+        detail = str(exc).strip() or "Validation failed."
+        parsed_errors = _collect_validation_errors(recipe)
+        return ValidateResponse(
+            valid=False,
+            errors=parsed_errors or [ValidateError(message=detail)],
+            raw_detail=detail,
+        )
+
+    return ValidateResponse(valid=True)

studio/backend/routes/datasets.py

@@ -5,7 +5,7 @@ import base64
 import io
 import sys
 from pathlib import Path
-from fastapi import APIRouter, HTTPException
+from fastapi import APIRouter, Depends, HTTPException
 import logging
 
 # Add backend directory to path
@@ -15,6 +15,7 @@ if str(backend_path) not in sys.path:
 
 # Import dataset utilities
 from utils.datasets import check_dataset_format
+from auth.authentication import get_current_subject
 
 router = APIRouter()
 logger = logging.getLogger(__name__)
@@ -84,7 +85,10 @@ DATA_EXTS = (
 
 
 @router.post("/check-format", response_model=CheckFormatResponse)
-def check_format(request: CheckFormatRequest):
+def check_format(
+    request: CheckFormatRequest,
+    current_subject: str = Depends(get_current_subject),
+):
     """
     Check if a dataset requires manual column mapping.
 
@@ -188,21 +192,40 @@ def check_format(request: CheckFormatRequest):
         # Generate preview samples
         preview_samples = None
         if not result["requires_manual_mapping"]:
-            try:
-                format_result = format_dataset(
-                    preview_slice,
-                    format_type="auto",
-                    custom_format_mapping=result.get("suggested_mapping"),
-                    num_proc=1,  # Only 10 preview rows — no need for multiprocessing
-                )
-                processed = format_result["dataset"]
-                preview_samples = _serialize_preview_rows(processed)
-            except Exception as e:
-                logger.warning(f"Processed preview generation failed (non-fatal): {e}")
+            if result.get("suggested_mapping"):
+                # Heuristic-detected: show raw data so columns match the API response.
+                # Processing (column stripping) happens at training time, not preview.
                 preview_samples = _serialize_preview_rows(preview_slice)
+            else:
+                try:
+                    format_result = format_dataset(
+                        preview_slice,
+                        format_type="auto",
+                        num_proc=1,  # Only 10 preview rows — no need for multiprocessing
+                    )
+                    processed = format_result["dataset"]
+                    preview_samples = _serialize_preview_rows(processed)
+                except Exception as e:
+                    logger.warning(f"Processed preview generation failed (non-fatal): {e}")
+                    preview_samples = _serialize_preview_rows(preview_slice)
         else:
             preview_samples = _serialize_preview_rows(preview_slice)
 
+        # Lightweight URL-based image detection for VLM datasets
+        warning = None
+        image_col = result.get("detected_image_column")
+        if image_col and image_col in (result.get("columns") or []):
+            try:
+                sample_val = preview_slice[0][image_col]
+                if isinstance(sample_val, str) and sample_val.startswith(("http://", "https://")):
+                    warning = (
+                        "This dataset contains image URLs instead of embedded images. "
+                        "Images will be downloaded during training, which may be slow for large datasets."
+                    )
+                    logger.info(f"URL-based image column detected: {image_col}")
+            except Exception:
+                pass
+
         return CheckFormatResponse(
             requires_manual_mapping=result["requires_manual_mapping"],
             detected_format=result["detected_format"],
@@ -214,6 +237,7 @@ def check_format(request: CheckFormatRequest):
             detected_text_column=result.get("detected_text_column"),
             preview_samples=preview_samples,
             total_rows=total_rows,
+            warning=warning,
         )
 
     except HTTPException:

studio/backend/routes/inference.py

@@ -5,7 +5,7 @@ import sys
 import time
 import uuid
 from pathlib import Path
-from fastapi import APIRouter, HTTPException, Request
+from fastapi import APIRouter, Depends, HTTPException, Request
 from fastapi.responses import StreamingResponse, JSONResponse
 from typing import Optional
 import json
@@ -23,6 +23,7 @@ if str(backend_path) not in sys.path:
 # Import backend functions
 try:
     from core.inference import get_inference_backend
+    from core.inference.llama_cpp import LlamaCppBackend
     from utils.models import ModelConfig
     from utils.inference import load_inference_config
 except ImportError:
@@ -30,6 +31,7 @@ except ImportError:
     if str(parent_backend) not in sys.path:
         sys.path.insert(0, str(parent_backend))
     from core.inference import get_inference_backend
+    from core.inference.llama_cpp import LlamaCppBackend
     from utils.models import ModelConfig
     from utils.inference import load_inference_config
 
@@ -48,6 +50,7 @@ from models.inference import (
     CompletionChoice,
     CompletionMessage,
 )
+from auth.authentication import get_current_subject
 
 router = APIRouter()
 logger = logging.getLogger(__name__)
@@ -61,36 +64,106 @@ if not logger.handlers:
     logger.addHandler(handler)
     logger.setLevel(logging.INFO)
 
+# GGUF inference backend (llama-server)
+_llama_cpp_backend = LlamaCppBackend()
+
+def get_llama_cpp_backend() -> LlamaCppBackend:
+    return _llama_cpp_backend
+
 
 @router.post("/load", response_model=LoadResponse)
-async def load_model(request: LoadRequest):
+async def load_model(
+    request: LoadRequest,
+    current_subject: str = Depends(get_current_subject),
+):
     """
     Load a model for inference.
-    
+
     The model_path should be a clean identifier from GET /models/list.
     Returns inference configuration parameters (temperature, top_p, top_k, min_p)
     from the model's YAML config, falling back to default.yaml for missing values.
+
+    GGUF models are loaded via llama-server (llama.cpp) instead of Unsloth.
     """
     try:
         # Ensure correct transformers version for this model architecture
         from utils.transformers_version import ensure_transformers_version
         ensure_transformers_version(request.model_path)
 
-        backend = get_inference_backend()
-        
         # Create config using clean factory method
         # is_lora is auto-detected from adapter_config.json on disk/HF
         config = ModelConfig.from_identifier(
             model_id=request.model_path,
             hf_token=request.hf_token,
+            gguf_variant=request.gguf_variant,
         )
-        
+
         if not config:
             raise HTTPException(
                 status_code=400,
                 detail=f"Invalid model identifier: {request.model_path}"
             )
-        
+
+        # ── GGUF path: load via llama-server ──────────────────────
+        if config.is_gguf:
+            llama_backend = get_llama_cpp_backend()
+            unsloth_backend = get_inference_backend()
+
+            # Unload any active Unsloth model first to free VRAM
+            if unsloth_backend.active_model_name:
+                logger.info(f"Unloading Unsloth model '{unsloth_backend.active_model_name}' before loading GGUF")
+                unsloth_backend.unload_model(unsloth_backend.active_model_name)
+
+            # Route to HF mode or local mode based on config
+            if config.gguf_hf_repo:
+                # HF mode: llama-server downloads via -hf "repo:quant"
+                success = llama_backend.load_model(
+                    hf_repo=config.gguf_hf_repo,
+                    hf_variant=config.gguf_variant,
+                    hf_token=request.hf_token,
+                    model_identifier=config.identifier,
+                    is_vision=config.is_vision,
+                    n_ctx=request.max_seq_length,
+                )
+            else:
+                # Local mode: llama-server loads via -m <path>
+                success = llama_backend.load_model(
+                    gguf_path=config.gguf_file,
+                    mmproj_path=config.gguf_mmproj_file,
+                    model_identifier=config.identifier,
+                    is_vision=config.is_vision,
+                    n_ctx=request.max_seq_length,
+                )
+
+            if not success:
+                raise HTTPException(
+                    status_code=500,
+                    detail=f"Failed to load GGUF model: {config.display_name}"
+                )
+
+            logger.info(f"Loaded GGUF model via llama-server: {config.identifier}")
+
+            inference_config = load_inference_config(config.identifier)
+
+            return LoadResponse(
+                status="loaded",
+                model=config.identifier,
+                display_name=config.display_name,
+                is_vision=config.is_vision,
+                is_lora=False,
+                is_gguf=True,
+                inference=inference_config,
+            )
+
+        # ── Standard path: load via Unsloth/transformers ──────────
+        backend = get_inference_backend()
+
+        # Unload any active GGUF model first
+        llama_backend = get_llama_cpp_backend()
+        if llama_backend.is_loaded:
+            logger.info("Unloading GGUF model before loading Unsloth model")
+            llama_backend.unload_model()
+
         # Auto-detect quantization for LoRA adapters from adapter_config.json
         # The training pipeline patches this file with "unsloth_training_method"
         # which is 'qlora' or 'lora'. Only LoRA (16-bit) needs load_in_4bit=False.
@@ -129,7 +202,7 @@ async def load_model(request: LoadRequest):
                             load_in_4bit = False
                 except Exception as e:
                     logger.warning(f"Could not read adapter_config.json: {e}")
-        
+
         # Load the model
         success = backend.load_model(
             config=config,
@@ -137,27 +210,28 @@ async def load_model(request: LoadRequest):
             load_in_4bit=load_in_4bit,
             hf_token=request.hf_token,
         )
-        
+
         if not success:
             raise HTTPException(
                 status_code=500,
                 detail=f"Failed to load model: {config.display_name}"
             )
-        
+
         logger.info(f"Loaded model: {config.identifier}")
-        
+
         # Load inference configuration parameters
         inference_config = load_inference_config(config.identifier)
-        
+
         return LoadResponse(
             status="loaded",
             model=config.identifier,
             display_name=config.display_name,
             is_vision=config.is_vision,
             is_lora=config.is_lora,
+            is_gguf=False,
             inference=inference_config,
         )
-        
+
     except HTTPException:
         raise
     except Exception as e:
@@ -169,16 +243,28 @@ async def load_model(request: LoadRequest):
 
 
 @router.post("/unload", response_model=UnloadResponse)
-async def unload_model(request: UnloadRequest):
+async def unload_model(
+    request: UnloadRequest,
+    current_subject: str = Depends(get_current_subject),
+):
     """
     Unload a model from memory.
+    Routes to the correct backend (llama-server for GGUF, Unsloth otherwise).
     """
     try:
+        # Check if the GGUF backend has this model loaded
+        llama_backend = get_llama_cpp_backend()
+        if llama_backend.is_loaded and llama_backend.model_identifier == request.model_path:
+            llama_backend.unload_model()
+            logger.info(f"Unloaded GGUF model: {request.model_path}")
+            return UnloadResponse(status="unloaded", model=request.model_path)
+
+        # Otherwise, unload from Unsloth backend
         backend = get_inference_backend()
         backend.unload_model(request.model_path)
         logger.info(f"Unloaded model: {request.model_path}")
         return UnloadResponse(status="unloaded", model=request.model_path)
-        
+
     except Exception as e:
         logger.error(f"Error unloading model: {e}", exc_info=True)
         raise HTTPException(
@@ -188,7 +274,10 @@ async def unload_model(request: UnloadRequest):
 
 
 @router.post("/generate/stream")
-async def generate_stream(request: GenerateRequest):
+async def generate_stream(
+    request: GenerateRequest,
+    current_subject: str = Depends(get_current_subject),
+):
     """
     Generate a chat response with Server-Sent Events (SSE) streaming.
     
@@ -261,25 +350,43 @@ async def generate_stream(request: GenerateRequest):
 
 
 @router.get("/status", response_model=InferenceStatusResponse)
-async def get_status():
+async def get_status(
+    current_subject: str = Depends(get_current_subject),
+):
     """
     Get current inference backend status.
+    Reports whichever backend (Unsloth or llama-server) is currently active.
     """
     try:
+        llama_backend = get_llama_cpp_backend()
+
+        # If a GGUF model is loaded via llama-server, report that
+        if llama_backend.is_loaded:
+            return InferenceStatusResponse(
+                active_model=llama_backend.model_identifier,
+                is_vision=llama_backend.is_vision,
+                is_gguf=True,
+                gguf_variant=llama_backend.hf_variant,
+                loading=[],
+                loaded=[llama_backend.model_identifier],
+            )
+
+        # Otherwise, report Unsloth backend status
         backend = get_inference_backend()
-        
+
         is_vision = False
         if backend.active_model_name:
             model_info = backend.models.get(backend.active_model_name, {})
             is_vision = model_info.get("is_vision", False)
-        
+
         return InferenceStatusResponse(
             active_model=backend.active_model_name,
             is_vision=is_vision,
+            is_gguf=False,
             loading=list(getattr(backend, 'loading_models', set())),
             loaded=list(backend.models.keys()),
         )
-        
+
     except Exception as e:
         logger.error(f"Error getting status: {e}", exc_info=True)
         raise HTTPException(
@@ -349,7 +456,11 @@ def _extract_content_parts(
 
 
 @router.post("/chat/completions")
-async def openai_chat_completions(payload: ChatCompletionRequest, request: Request):
+async def openai_chat_completions(
+    payload: ChatCompletionRequest,
+    request: Request,
+    current_subject: str = Depends(get_current_subject),
+):
     """
     OpenAI-compatible chat completions endpoint.
 
@@ -358,29 +469,163 @@ async def openai_chat_completions(payload: ChatCompletionRequest, request: Reque
 
     Streaming (default):  returns SSE chunks matching OpenAI's format.
     Non-streaming:        returns a single ChatCompletion JSON object.
-    """
-    backend = get_inference_backend()
 
-    if not backend.active_model_name:
-        raise HTTPException(
-            status_code=400,
-            detail="No model loaded. Call POST /inference/load first.",
-        )
+    Automatically routes to the correct backend:
+    - GGUF models → llama-server via LlamaCppBackend
+    - Other models → Unsloth/transformers via InferenceBackend
+    """
+    llama_backend = get_llama_cpp_backend()
+    using_gguf = llama_backend.is_loaded
+
+    # ── Determine which backend is active ─────────────────────
+    if using_gguf:
+        model_name = llama_backend.model_identifier or payload.model
+    else:
+        backend = get_inference_backend()
+        if not backend.active_model_name:
+            raise HTTPException(
+                status_code=400,
+                detail="No model loaded. Call POST /inference/load first.",
+            )
+        model_name = backend.active_model_name or payload.model
 
     # ── Parse messages (handles multimodal content parts) ─────
     system_prompt, chat_messages, extracted_image_b64 = _extract_content_parts(
         payload.messages
     )
 
-    # If no non-system messages were provided, error out
     if not chat_messages:
         raise HTTPException(
             status_code=400,
             detail="At least one non-system message is required.",
         )
 
-    # ── Decode image (from content parts OR legacy field) ─────
-    # Content-part images take priority; fall back to legacy field
+    # ── GGUF path: proxy to llama-server /v1/chat/completions ──
+    if using_gguf:
+        # Reject images if this GGUF model doesn't support vision
+        image_b64 = extracted_image_b64 or payload.image_base64
+        if image_b64 and not llama_backend.is_vision:
+            raise HTTPException(
+                status_code=400,
+                detail="Image provided but current GGUF model does not support vision.",
+            )
+
+        # Build message list with system prompt prepended
+        gguf_messages = []
+        if system_prompt:
+            gguf_messages.append({"role": "system", "content": system_prompt})
+        gguf_messages.extend(chat_messages)
+
+        cancel_event = threading.Event()
+
+        completion_id = f"chatcmpl-{uuid.uuid4().hex[:12]}"
+        created = int(time.time())
+
+        def gguf_generate():
+            return llama_backend.generate_chat_completion(
+                messages=gguf_messages,
+                image_b64=image_b64,
+                temperature=payload.temperature,
+                top_p=payload.top_p,
+                top_k=payload.top_k,
+                min_p=payload.min_p,
+                max_tokens=payload.max_tokens or 512,
+                repetition_penalty=payload.repetition_penalty,
+                cancel_event=cancel_event,
+            )
+
+        if payload.stream:
+            async def gguf_stream_chunks():
+                try:
+                    # First chunk: role
+                    first_chunk = ChatCompletionChunk(
+                        id=completion_id,
+                        created=created,
+                        model=model_name,
+                        choices=[ChunkChoice(
+                            delta=ChoiceDelta(role="assistant"),
+                            finish_reason=None,
+                        )],
+                    )
+                    yield f"data: {first_chunk.model_dump_json(exclude_none=True)}\n\n"
+
+                    # Content chunks — llama backend yields cumulative text
+                    prev_text = ""
+                    for cumulative in gguf_generate():
+                        if await request.is_disconnected():
+                            cancel_event.set()
+                            return
+                        new_text = cumulative[len(prev_text):]
+                        prev_text = cumulative
+                        if not new_text:
+                            continue
+                        chunk = ChatCompletionChunk(
+                            id=completion_id,
+                            created=created,
+                            model=model_name,
+                            choices=[ChunkChoice(
+                                delta=ChoiceDelta(content=new_text),
+                                finish_reason=None,
+                            )],
+                        )
+                        yield f"data: {chunk.model_dump_json(exclude_none=True)}\n\n"
+
+                    # Final chunk
+                    final_chunk = ChatCompletionChunk(
+                        id=completion_id,
+                        created=created,
+                        model=model_name,
+                        choices=[ChunkChoice(
+                            delta=ChoiceDelta(),
+                            finish_reason="stop",
+                        )],
+                    )
+                    yield f"data: {final_chunk.model_dump_json(exclude_none=True)}\n\n"
+                    yield "data: [DONE]\n\n"
+
+                except asyncio.CancelledError:
+                    cancel_event.set()
+                    raise
+                except Exception as e:
+                    logger.error(f"Error during GGUF streaming: {e}", exc_info=True)
+                    error_chunk = {
+                        "error": {"message": str(e), "type": "server_error"},
+                    }
+                    yield f"data: {json.dumps(error_chunk)}\n\n"
+
+            return StreamingResponse(
+                gguf_stream_chunks(),
+                media_type="text/event-stream",
+                headers={
+                    "Cache-Control": "no-cache",
+                    "Connection": "keep-alive",
+                    "X-Accel-Buffering": "no",
+                },
+            )
+        else:
+            try:
+                full_text = ""
+                for token in gguf_generate():
+                    full_text = token
+
+                response = ChatCompletion(
+                    id=completion_id,
+                    created=created,
+                    model=model_name,
+                    choices=[CompletionChoice(
+                        message=CompletionMessage(content=full_text),
+                        finish_reason="stop",
+                    )],
+                )
+                return JSONResponse(content=response.model_dump())
+
+            except Exception as e:
+                logger.error(f"Error during GGUF completion: {e}", exc_info=True)
+                raise HTTPException(status_code=500, detail=str(e))
+
+    # ── Standard Unsloth path ─────────────────────────────────
+
+    # Decode image (from content parts OR legacy field)
     image_b64 = extracted_image_b64 or payload.image_base64
     image = None
 
@@ -406,7 +651,7 @@ async def openai_chat_completions(payload: ChatCompletionRequest, request: Reque
         except Exception as e:
             raise HTTPException(status_code=400, detail=f"Failed to decode image: {e}")
 
-    # ── Shared generation kwargs ──────────────────────────────
+    # Shared generation kwargs
     gen_kwargs = dict(
         messages=chat_messages,
         system_prompt=system_prompt,
@@ -419,11 +664,10 @@ async def openai_chat_completions(payload: ChatCompletionRequest, request: Reque
         repetition_penalty=payload.repetition_penalty,
     )
 
-    # ── Choose generation path (adapter-controlled or standard) ──
+    # Choose generation path (adapter-controlled or standard)
     cancel_event = threading.Event()
 
     if payload.use_adapter is not None:
-        # Compare mode: toggle adapter state atomically with generation
         def generate():
             return backend.generate_with_adapter_control(
                 use_adapter=payload.use_adapter,
@@ -431,11 +675,9 @@ async def openai_chat_completions(payload: ChatCompletionRequest, request: Reque
                 **gen_kwargs,
             )
     else:
-        # Standard path: no adapter toggling
         def generate():
             return backend.generate_chat_response(cancel_event=cancel_event, **gen_kwargs)
 
-    model_name = backend.active_model_name or payload.model
     completion_id = f"chatcmpl-{uuid.uuid4().hex[:12]}"
     created = int(time.time())
 
@@ -443,7 +685,6 @@ async def openai_chat_completions(payload: ChatCompletionRequest, request: Reque
     if payload.stream:
         async def stream_chunks():
             try:
-                # First chunk: send the role
                 first_chunk = ChatCompletionChunk(
                     id=completion_id,
                     created=created,
@@ -455,8 +696,6 @@ async def openai_chat_completions(payload: ChatCompletionRequest, request: Reque
                 )
                 yield f"data: {first_chunk.model_dump_json(exclude_none=True)}\n\n"
 
-                # Content chunks — generate_chat_response yields cumulative
-                # text, so we diff to get incremental deltas.
                 prev_text = ""
                 for cumulative in generate():
                     if await request.is_disconnected():
@@ -478,7 +717,6 @@ async def openai_chat_completions(payload: ChatCompletionRequest, request: Reque
                     )
                     yield f"data: {chunk.model_dump_json(exclude_none=True)}\n\n"
 
-                # Final chunk: finish_reason = stop
                 final_chunk = ChatCompletionChunk(
                     id=completion_id,
                     created=created,
@@ -518,7 +756,7 @@ async def openai_chat_completions(payload: ChatCompletionRequest, request: Reque
         try:
             full_text = ""
             for token in generate():
-                full_text = token  # generate_stream yields cumulative text
+                full_text = token
 
             response = ChatCompletion(
                 id=completion_id,

studio/backend/routes/models.py

@@ -18,12 +18,15 @@ from auth.authentication import get_current_subject
 try:
     from utils.models import (
         scan_trained_loras,
+        scan_exported_models,
         load_model_defaults,
         get_base_model_from_lora,
         is_vision_model,
         scan_checkpoints,
+        list_gguf_variants,
         ModelConfig,
     )
+    from utils.models.model_config import _pick_best_gguf, _extract_quant_label
     from core.inference import get_inference_backend
 except ImportError:
     # Fallback: try to import from parent directory
@@ -32,12 +35,15 @@ except ImportError:
         sys.path.insert(0, str(parent_backend))
     from utils.models import (
         scan_trained_loras,
+        scan_exported_models,
         load_model_defaults,
         get_base_model_from_lora,
         is_vision_model,
         scan_checkpoints,
+        list_gguf_variants,
         ModelConfig,
     )
+    from utils.models.model_config import _pick_best_gguf, _extract_quant_label
     from core.inference import get_inference_backend
 
 from models import (
@@ -51,6 +57,7 @@ from models import (
     LoRAInfo,
     ModelListResponse,
 )
+from models.models import GgufVariantDetail, GgufVariantsResponse
 from models.responses import LoRABaseModelResponse, VisionCheckResponse
 
 router = APIRouter()
@@ -88,6 +95,7 @@ def _scan_models_dir(models_dir: Path) -> List[LocalModelInfo]:
             or (child / "adapter_config.json").exists()
             or any(child.glob("*.safetensors"))
             or any(child.glob("*.bin"))
+            or any(child.glob("*.gguf"))
         )
         if not has_model_files:
             continue
@@ -104,6 +112,23 @@ def _scan_models_dir(models_dir: Path) -> List[LocalModelInfo]:
                 updated_at=updated_at,
             ),
         )
+    # Also scan for standalone .gguf files directly in the models directory
+    for gguf_file in models_dir.glob("*.gguf"):
+        if gguf_file.is_file():
+            try:
+                updated_at = gguf_file.stat().st_mtime
+            except OSError:
+                updated_at = None
+            found.append(
+                LocalModelInfo(
+                    id=str(gguf_file),
+                    display_name=gguf_file.stem,
+                    path=str(gguf_file),
+                    source="models_dir",
+                    updated_at=updated_at,
+                ),
+            )
+
     return found
 
 
@@ -294,35 +319,45 @@ async def get_model_config(
 @router.get("/loras")
 async def scan_loras(
     outputs_dir: str = Query(default="./outputs", description="Directory to scan for LoRA adapters"),
+    exports_dir: str = Query(default="./exports", description="Directory to scan for exported models"),
     current_subject: str = Depends(get_current_subject),
 ):
     """
-    Scan for trained LoRA adapters in the outputs directory.
-    
-    This endpoint wraps the backend scan_trained_loras function.
+    Scan for trained LoRA adapters and exported models.
+
+    Returns both training outputs (from outputs_dir) and exported models
+    (from exports_dir) in a single list, distinguished by source field.
     """
     try:
-        # Call backend scan function
-        trained_loras = scan_trained_loras(outputs_dir=outputs_dir)
-        
-        # Convert to LoRAInfo objects
         lora_list = []
+
+        # Scan training outputs
+        trained_loras = scan_trained_loras(outputs_dir=outputs_dir)
         for display_name, adapter_path in trained_loras:
-            # Get base model if available
             base_model = get_base_model_from_lora(adapter_path)
-            
-            lora_info = LoRAInfo(
+            lora_list.append(LoRAInfo(
                 display_name=display_name,
                 adapter_path=adapter_path,
-                base_model=base_model
-            )
-            lora_list.append(lora_info)
-        
+                base_model=base_model,
+                source="training",
+            ))
+
+        # Scan exported models (merged, LoRA, base — skips GGUF)
+        exported = scan_exported_models(exports_dir=exports_dir)
+        for display_name, model_path, export_type, base_model in exported:
+            lora_list.append(LoRAInfo(
+                display_name=display_name,
+                adapter_path=model_path,
+                base_model=base_model,
+                source="exported",
+                export_type=export_type,
+            ))
+
         return LoRAScanResponse(
             loras=lora_list,
             outputs_dir=outputs_dir
         )
-        
+
     except Exception as e:
         logger.error(f"Error scanning LoRAs: {e}", exc_info=True)
         raise HTTPException(
@@ -398,6 +433,49 @@ async def check_vision_model(
             detail=f"Failed to check vision model: {str(e)}"
         )
 
+@router.get("/gguf-variants", response_model=GgufVariantsResponse)
+async def get_gguf_variants(
+    repo_id: str = Query(..., description="HuggingFace repo ID (e.g. 'unsloth/gemma-3-4b-it-GGUF')"),
+    hf_token: Optional[str] = Query(None, description="HuggingFace token for private repos"),
+    current_subject: str = Depends(get_current_subject),
+):
+    """
+    List available GGUF quantization variants for a HuggingFace repo.
+
+    Returns all available quantization variants (Q4_K_M, Q8_0, BF16, etc.)
+    with file sizes, whether the model supports vision, and the recommended
+    default variant.
+    """
+    try:
+        variants, has_vision = list_gguf_variants(repo_id, hf_token=hf_token)
+
+        # Determine default variant
+        filenames = [v.filename for v in variants]
+        best = _pick_best_gguf(filenames)
+        default_variant = _extract_quant_label(best) if best else None
+
+        return GgufVariantsResponse(
+            repo_id=repo_id,
+            variants=[
+                GgufVariantDetail(
+                    filename=v.filename,
+                    quant=v.quant,
+                    size_bytes=v.size_bytes,
+                )
+                for v in variants
+            ],
+            has_vision=has_vision,
+            default_variant=default_variant,
+        )
+
+    except Exception as e:
+        logger.error(f"Error listing GGUF variants for '{repo_id}': {e}", exc_info=True)
+        raise HTTPException(
+            status_code=500,
+            detail=f"Failed to list GGUF variants: {str(e)}",
+        )
+
+
 @router.get("/checkpoints", response_model=CheckpointListResponse)
 async def list_checkpoints(
     outputs_dir: str = Query(

studio/backend/routes/training.py

@@ -154,6 +154,8 @@ async def start_training(
             "train_split": request.train_split,
             "eval_split": request.eval_split,
             "eval_steps": request.eval_steps,
+            "dataset_slice_start": request.dataset_slice_start,
+            "dataset_slice_end": request.dataset_slice_end,
             "custom_format_mapping": request.custom_format_mapping,
             "num_epochs": request.num_epochs,
             "learning_rate": request.learning_rate,

studio/backend/utils/datasets/chat_templates.py

@@ -283,9 +283,11 @@ def apply_chat_template_to_dataset(
             }
 
             if not isinstance(dataset, IterableDataset):
-                from multiprocessing import cpu_count
+                from utils.hardware import safe_num_proc
                 if num_proc is None or type(num_proc) is not int:
-                    num_proc = max(1, cpu_count() // 3)
+                    num_proc = safe_num_proc()
+                else:
+                    num_proc = safe_num_proc(num_proc)
                 dataset_map_kwargs['num_proc'] = num_proc
                 dataset_map_kwargs['desc'] = "Applying template to Alpaca format"
 
@@ -347,9 +349,11 @@ def apply_chat_template_to_dataset(
             }
 
             if not isinstance(dataset, IterableDataset):
-                from multiprocessing import cpu_count
+                from utils.hardware import safe_num_proc
                 if num_proc is None or type(num_proc) is not int:
-                    num_proc = max(1, cpu_count() // 3)
+                    num_proc = safe_num_proc()
+                else:
+                    num_proc = safe_num_proc(num_proc)
                 dataset_map_kwargs['num_proc'] = num_proc
                 dataset_map_kwargs['desc'] = f"Applying chat template to {final_format}"
 

studio/backend/utils/datasets/dataset_utils.py

@@ -126,38 +126,76 @@ def check_dataset_format(dataset, is_vlm: bool = False) -> dict:
             "multimodal_columns": None,
         }
 
+# Normalise any format-specific role to canonical chatml (user/assistant/system)
+_TO_CHATML = {
+    "user": "user", "human": "user", "instruction": "user",
+    "assistant": "assistant", "gpt": "assistant", "output": "assistant",
+    "system": "system", "input": "system",
+}
+_CHATML_ROLE_ORDER = ("system", "user", "assistant")
+_CHATML_TO_ALPACA = {"user": "instruction", "system": "input", "assistant": "output"}
+
+
 def _apply_user_mapping(dataset, mapping: dict, batch_size: int = 1000):
     """
     Apply user-provided column mapping to convert dataset to conversations format.
-    
-    Args:
-        dataset: HuggingFace dataset
-        mapping: Dict like {"question": "user", "answer": "assistant", "context": "system"}
-        batch_size: Batch size for processing
-    
+
+    Accepts chatml (user/assistant/system), sharegpt (human/gpt/system), and
+    alpaca (instruction/input/output) role names — all normalised to chatml output.
+
     Returns:
-        Dataset with single 'conversations' column (no extra columns preserved)
+        Dataset with single 'conversations' column
     """
+    # Pre-compute: group columns by canonical chatml role
+    role_groups: dict[str, list[str]] = {r: [] for r in _CHATML_ROLE_ORDER}
+    for col_name, role in mapping.items():
+        canonical = _TO_CHATML.get(role)
+        if canonical:
+            role_groups[canonical].append(col_name)
+
     def _convert(examples):
-        num_examples = len(examples[list(examples.keys())[0]])
+        num = len(next(iter(examples.values())))
         conversations = []
-        
-        for i in range(num_examples):
+        for i in range(num):
             convo = []
-            role_order = ['system', 'user', 'assistant']
-            
-            for target_role in role_order:
-                for col_name, role in mapping.items():
-                    if role == target_role and col_name in examples:
-                        content = examples[col_name][i]
-                        # User explicitly mapped - always include even if empty
-                        convo.append({"role": role, "content": str(content) if content else ""})
-            
+            for chatml_role in _CHATML_ROLE_ORDER:
+                for col in role_groups[chatml_role]:
+                    if col in examples:
+                        content = examples[col][i]
+                        convo.append({"role": chatml_role, "content": str(content) if content else ""})
             conversations.append(convo)
-        
-        # ONLY return conversations - no extra columns
         return {"conversations": conversations}
-    
+
+    return dataset.map(_convert, batched=True, batch_size=batch_size, remove_columns=dataset.column_names)
+
+
+def _apply_user_mapping_alpaca(dataset, mapping: dict, batch_size: int = 1000):
+    """
+    Apply user-provided column mapping to convert dataset to Alpaca format.
+
+    Accepts any format's role names — normalises via _TO_CHATML, then maps
+    user → instruction, system → input, assistant → output.
+
+    Returns:
+        Dataset with instruction/input/output columns
+    """
+    col_for: dict[str, str | None] = {"instruction": None, "input": None, "output": None}
+    for col_name, role in mapping.items():
+        canonical = _TO_CHATML.get(role)
+        alpaca_field = _CHATML_TO_ALPACA.get(canonical) if canonical else None
+        if alpaca_field:
+            col_for[alpaca_field] = col_name
+
+    def _convert(examples):
+        num = len(next(iter(examples.values())))
+        instructions, inputs, outputs = [], [], []
+        for i in range(num):
+            for field, dest in (("instruction", instructions), ("input", inputs), ("output", outputs)):
+                col = col_for[field]
+                val = str(examples[col][i]) if col and col in examples and examples[col][i] else ""
+                dest.append(val)
+        return {"instruction": instructions, "input": inputs, "output": outputs}
+
     return dataset.map(_convert, batched=True, batch_size=batch_size, remove_columns=dataset.column_names)
 
 
@@ -191,20 +229,30 @@ def format_dataset(
     # Detect multimodal first (needed for all flows)
     multimodal_info = detect_multimodal_dataset(dataset)
 
-    # NEW: If user provided explicit mapping, skip detection and apply directly
+    # If user provided explicit mapping, skip detection and apply in the requested format
     if custom_format_mapping:
         try:
-            mapped_dataset = _apply_user_mapping(dataset, custom_format_mapping, batch_size)
+            if format_type == "alpaca":
+                mapped_dataset = _apply_user_mapping_alpaca(dataset, custom_format_mapping, batch_size)
+                final_format = "alpaca"
+                chat_column = None
+            else:
+                # auto / chatml / sharegpt / conversational — all produce chatml conversations
+                # (sharegpt is always standardized to role/content internally)
+                mapped_dataset = _apply_user_mapping(dataset, custom_format_mapping, batch_size)
+                final_format = "chatml_conversations"
+                chat_column = "conversations"
+
             return {
                 "dataset": mapped_dataset,
                 "detected_format": "user_mapped",
-                "final_format": "chatml_conversations",
-                "chat_column": "conversations",
+                "final_format": final_format,
+                "chat_column": chat_column,
                 "is_standardized": True,
                 "requires_manual_mapping": False,
                 "is_multimodal": multimodal_info["is_multimodal"],
                 "multimodal_info": multimodal_info,
-                "warnings": [f"Applied user-provided column mapping: {custom_format_mapping}"]
+                "warnings": [f"Applied user-provided column mapping ({format_type}): {custom_format_mapping}"]
             }
         except Exception as e:
             return {
@@ -224,7 +272,7 @@ def format_dataset(
     detected = detect_dataset_format(dataset)
     warnings = []
 
-     # Add multimodal warning if detected
+    # Add multimodal warning if detected
     if multimodal_info["is_multimodal"]:
         warnings.append(
             f"Multimodal dataset detected. Found columns: {multimodal_info['multimodal_columns']}"
@@ -309,48 +357,25 @@ def format_dataset(
                         conversations = []
                         num_examples = len(examples[list(examples.keys())[0]])
 
-                        # NEW: Check if this is user-provided or auto-detected
-                        is_user_provided = custom_format_mapping is not None  # Passed explicitly
-
-                        # Preserve non-mapped columns ONLY if auto-detected
-                        preserved_columns = {}
-                        if not is_user_provided:  # Only preserve for auto-detection
-                            all_columns = set(examples.keys())
-                            mapped_columns = set(custom_mapping.keys())
-                            non_mapped_columns = all_columns - mapped_columns
-
-                            for col in non_mapped_columns:
-                                preserved_columns[col] = examples[col]
+                        # Preserve non-mapped columns
+                        all_columns = set(examples.keys())
+                        mapped_columns = set(custom_mapping.keys())
+                        preserved_columns = {
+                            col: examples[col]
+                            for col in all_columns - mapped_columns
+                        }
 
                         for i in range(num_examples):
                             convo = []
-
-                            # Enforce standard role order
-                            role_order = ['system', 'user', 'assistant']
-
-                            for target_role in role_order:
+                            for target_role in ['system', 'user', 'assistant']:
                                 for col_name, role in custom_mapping.items():
                                     if role == target_role and col_name in examples:
                                         content = examples[col_name][i]
-
-                                        # NEW: Different behavior based on mapping source
-                                        if is_user_provided:
-                                            # User explicitly mapped this - always include even if empty
-                                            convo.append({"role": role, "content": str(content) if content else ""})
-                                        else:
-                                            # Auto-detected - skip empty (original behavior)
-                                            if content and str(content).strip():
-                                                convo.append({"role": role, "content": str(content)})
-
+                                        if content and str(content).strip():
+                                            convo.append({"role": role, "content": str(content)})
                             conversations.append(convo)
 
-                        result = {"conversations": conversations}
-
-                        # Only add preserved columns if auto-detected
-                        if not is_user_provided:
-                            result.update(preserved_columns)
-
-                        return result
+                        return {"conversations": conversations, **preserved_columns}
 
 
                     try:
@@ -459,7 +484,7 @@ def format_dataset(
             }
 
     # CHATML MODE: Convert to ChatML
-    elif format_type in ["chatml", "conversational"]:
+    elif format_type in ["chatml", "conversational", "sharegpt"]:
 
         if detected["format"] == "alpaca":
             converted = convert_alpaca_to_chatml(dataset, batch_size, num_proc)
@@ -508,36 +533,38 @@ def format_dataset(
 
         else:
             warnings.append(f"Unknown format, attempting standardization")
-            try:
-                standardized = standardize_chat_format(
-                    dataset, tokenizer, aliases_for_system,
-                    aliases_for_user, aliases_for_assistant,
-                    batch_size, num_proc
-                )
-                return {
-                    "dataset": standardized,
-                    "detected_format": "unknown",
-                    "final_format": f"chatml_{detected['chat_column']}",
-                    "chat_column": detected["chat_column"],
-                    "is_standardized": True,
-                    "requires_manual_mapping": False,
-                    "is_multimodal": multimodal_info["is_multimodal"],
-                    "multimodal_info": multimodal_info,
-                    "warnings": warnings
-                }
-            except Exception as e:
-                warnings.append(f"Standardization failed: {e}")
-                return {
-                    "dataset": dataset,
-                    "detected_format": "unknown",
-                    "final_format": "unknown",
-                    "chat_column": detected["chat_column"],
-                    "is_standardized": False,
-                    "requires_manual_mapping": True,
-                    "is_multimodal": multimodal_info["is_multimodal"],
-                    "multimodal_info": multimodal_info,
-                    "warnings": warnings
-                }
+            if detected["chat_column"]:
+                try:
+                    standardized = standardize_chat_format(
+                        dataset, tokenizer, aliases_for_system,
+                        aliases_for_user, aliases_for_assistant,
+                        batch_size, num_proc
+                    )
+                    return {
+                        "dataset": standardized,
+                        "detected_format": "unknown",
+                        "final_format": f"chatml_{detected['chat_column']}",
+                        "chat_column": detected["chat_column"],
+                        "is_standardized": True,
+                        "requires_manual_mapping": False,
+                        "is_multimodal": multimodal_info["is_multimodal"],
+                        "multimodal_info": multimodal_info,
+                        "warnings": warnings
+                    }
+                except Exception as e:
+                    warnings.append(f"Standardization failed: {e}")
+
+            return {
+                "dataset": dataset,
+                "detected_format": "unknown",
+                "final_format": "unknown",
+                "chat_column": detected["chat_column"],
+                "is_standardized": False,
+                "requires_manual_mapping": True,
+                "is_multimodal": multimodal_info["is_multimodal"],
+                "multimodal_info": multimodal_info,
+                "warnings": warnings
+            }
 
     else:
         raise ValueError(f"Unknown format_type: {format_type}")
@@ -566,6 +593,7 @@ def format_and_template_dataset(
     aliases_for_assistant=["gpt", "assistant", "output",],
     batch_size=1000,
     num_proc=None,
+    progress_callback=None,
 ):
     """
     Convenience function that combines format_dataset and apply_chat_template_to_dataset.
@@ -611,6 +639,7 @@ def format_and_template_dataset(
                         text_column=user_vlm_text_column,
                         image_column=user_vlm_image_column,
                         dataset_name=dataset_name,
+                        progress_callback=progress_callback,
                     )
                     warnings.append(f"Applied user VLM mapping: image='{user_vlm_image_column}', text='{user_vlm_text_column}'")
                     
@@ -707,6 +736,7 @@ def format_and_template_dataset(
                     text_column=vlm_text_column,
                     image_column=vlm_image_column,
                     dataset_name=dataset_name,
+                    progress_callback=progress_callback,
                 )
 
                 if vlm_instruction:
@@ -768,8 +798,10 @@ def format_and_template_dataset(
         )
 
         # Step 2: Apply chat template
-        if "gemma" in model_name.lower() and not dataset_info["is_multimodal"] and (format_type != "alpaca" or (format_type == "auto" and dataset_info["detected_format"] != "alpaca")):
-            print("remove_bos_prefix is true")
+        # Gemma emits a leading <bos> that must be stripped for text-only chatml/sharegpt.
+        is_alpaca = format_type == "alpaca" or (format_type == "auto" and dataset_info["detected_format"] == "alpaca")
+        is_gemma = "gemma" in model_name.lower()
+        if is_gemma and not dataset_info["is_multimodal"] and not is_alpaca:
             remove_bos_prefix = True
         template_result = apply_chat_template_to_dataset(
             dataset_info=dataset_info,
@@ -791,14 +823,24 @@ def format_and_template_dataset(
         all_warnings = dataset_info.get("warnings", []) + template_result.get("warnings", [])
         all_errors = template_result.get("errors", [])
 
+        # If format_dataset returned "unknown" but apply_chat_template rescued
+        # it via heuristic detection, update final_format to reflect reality.
+        final_format = dataset_info["final_format"]
+        requires_manual = dataset_info.get("requires_manual_mapping", False)
+        if final_format == "unknown" and template_result["success"]:
+            out_ds = template_result["dataset"]
+            if hasattr(out_ds, "column_names") and "text" in out_ds.column_names:
+                final_format = "chatml_conversations"
+                requires_manual = False
+
         return {
             "dataset": template_result["dataset"],
             "detected_format": dataset_info["detected_format"],
-            "final_format": dataset_info["final_format"],
+            "final_format": final_format,
             "chat_column": dataset_info.get("chat_column"),
             "is_vlm": False,  # This is LLM flow
             "success": template_result["success"],
-            "requires_manual_mapping": dataset_info.get("requires_manual_mapping", False),
+            "requires_manual_mapping": requires_manual,
             "warnings": all_warnings,
             "errors": all_errors,
             "summary": summary,

studio/backend/utils/datasets/format_conversion.py

@@ -107,10 +107,12 @@ def standardize_chat_format(
     }
 
     if not isinstance(dataset, IterableDataset):
-        from multiprocessing import cpu_count
+        from utils.hardware import safe_num_proc
 
         if num_proc is None or type(num_proc) is not int:
-            num_proc = max(1, cpu_count() // 3)
+            num_proc = safe_num_proc()
+        else:
+            num_proc = safe_num_proc(num_proc)
 
         dataset_map_kwargs['num_proc'] = num_proc
         dataset_map_kwargs['desc'] = "Standardizing chat format"
@@ -173,10 +175,12 @@ def convert_chatml_to_alpaca(dataset, batch_size=1000, num_proc=None):
     }
 
     if not isinstance(dataset, IterableDataset):
-        from multiprocessing import cpu_count
+        from utils.hardware import safe_num_proc
 
         if num_proc is None or type(num_proc) is not int:
-            num_proc = max(1, cpu_count() // 3)
+            num_proc = safe_num_proc()
+        else:
+            num_proc = safe_num_proc(num_proc)
 
         dataset_map_kwargs['num_proc'] = num_proc
         dataset_map_kwargs['desc'] = "Converting ChatML to Alpaca format"
@@ -221,10 +225,12 @@ def convert_alpaca_to_chatml(dataset, batch_size=1000, num_proc=None):
     }
 
     if not isinstance(dataset, IterableDataset):
-        from multiprocessing import cpu_count
+        from utils.hardware import safe_num_proc
 
         if num_proc is None or type(num_proc) is not int:
-            num_proc = max(1, cpu_count() // 3)
+            num_proc = safe_num_proc()
+        else:
+            num_proc = safe_num_proc(num_proc)
 
         dataset_map_kwargs['num_proc'] = num_proc
         dataset_map_kwargs['desc'] = "Converting Alpaca to ChatML format"
@@ -232,24 +238,51 @@ def convert_alpaca_to_chatml(dataset, batch_size=1000, num_proc=None):
     return dataset.map(_convert, **dataset_map_kwargs)
 
 
+def _format_eta(seconds):
+    """Format seconds into a human-readable ETA string."""
+    if seconds < 60:
+        return f"{seconds:.0f}s"
+    elif seconds < 3600:
+        m, s = divmod(int(seconds), 60)
+        return f"{m}m {s}s"
+    else:
+        h, remainder = divmod(int(seconds), 3600)
+        m, _ = divmod(remainder, 60)
+        return f"{h}h {m}m"
+
+
 def convert_to_vlm_format(
     dataset,
     instruction=None,
     text_column="text",
     image_column="image",
     dataset_name=None,
+    progress_callback=None,
 ):
     """
     Converts simple {image, text} format to VLM messages format.
 
     Returns a LIST, not a HuggingFace Dataset (to preserve PIL Images).
 
+    For URL-based image datasets, runs a 200-sample parallel probe first to
+    estimate download speed and failure rate, then reports time estimate or
+    warning through progress_callback before proceeding with the full conversion.
+
+    Args:
+        progress_callback: Optional callable(status_message=str) to report
+                          progress to the training overlay.
+
     Returns:
         list: List of dicts with 'messages' field
     """
     from PIL import Image
     from .vlm_processing import generate_smart_vlm_instruction
 
+    def _notify(msg):
+        """Send status update to the training overlay if callback is available."""
+        if progress_callback:
+            progress_callback(status_message=msg)
+
     # Generate smart instruction if not provided
     if instruction is None:
         instruction_info = generate_smart_vlm_instruction(
@@ -275,12 +308,17 @@ def convert_to_vlm_format(
 
     def _convert_single_sample(sample):
         """Convert a single sample to VLM format."""
-        # Get image (might be PIL Image or path)
+        # Get image (might be PIL Image, local path, or URL)
         image_data = sample[image_column]
 
-        # Handle image paths
         if isinstance(image_data, str):
-            image_data = Image.open(image_data).convert("RGB")
+            if image_data.startswith(("http://", "https://")):
+                import fsspec
+                from io import BytesIO
+                with fsspec.open(image_data, "rb", expand=True) as f:
+                    image_data = Image.open(BytesIO(f.read())).convert("RGB")
+            else:
+                image_data = Image.open(image_data).convert("RGB")
 
         # Get text
         text_data = sample[text_column]
@@ -311,11 +349,143 @@ def convert_to_vlm_format(
         # Return dict with messages
         return {"messages": messages}
 
-    # Use list comprehension and return the LIST directly
-    print(f"🔄 Converting {len(dataset)} samples to VLM format...")
-    converted_list = [_convert_single_sample(sample) for sample in dataset]
+    total = len(dataset)
+    first_image = next(iter(dataset))[image_column]
+    has_urls = isinstance(first_image, str) and first_image.startswith(("http://", "https://"))
 
-    print(f"✅ Converted {len(converted_list)} samples")
+    # ── URL probe: 200 samples with parallel workers to estimate speed + failure rate ──
+    PROBE_SIZE = 200
+    MAX_FAIL_RATE = 0.3
+
+    if has_urls and total > PROBE_SIZE:
+        import time
+        from concurrent.futures import ThreadPoolExecutor, as_completed
+        from utils.hardware import safe_num_proc
+
+        num_workers = safe_num_proc()
+        _notify(f"Probing {PROBE_SIZE} image URLs with {num_workers} workers...")
+        print(f"🔍 Probing {PROBE_SIZE}/{total} image URLs with {num_workers} workers...")
+
+        probe_samples = [dataset[i] for i in range(PROBE_SIZE)]
+        probe_ok = 0
+        probe_fail = 0
+        probe_start = time.time()
+
+        with ThreadPoolExecutor(max_workers=num_workers) as executor:
+            futures = {executor.submit(_convert_single_sample, s): s for s in probe_samples}
+            for future in as_completed(futures):
+                try:
+                    future.result()
+                    probe_ok += 1
+                except Exception:
+                    probe_fail += 1
+
+        probe_elapsed = time.time() - probe_start
+        probe_total = probe_ok + probe_fail
+        fail_rate = probe_fail / probe_total if probe_total > 0 else 0
+        throughput = probe_total / probe_elapsed if probe_elapsed > 0 else 0
+
+        if fail_rate >= MAX_FAIL_RATE:
+            msg = (
+                f"⚠️ {fail_rate:.0%} of the first {PROBE_SIZE} images failed to download "
+                f"({probe_fail}/{probe_total}). "
+                "This dataset has too many broken or unreachable image URLs. "
+                "Consider using a dataset with embedded images instead."
+            )
+            print(msg)
+            _notify(msg)
+            raise ValueError(msg)
+
+        # Estimate total time for remaining samples
+        remaining = total - PROBE_SIZE
+        estimated_seconds = remaining / throughput if throughput > 0 else 0
+        eta_str = _format_eta(estimated_seconds)
+
+        info_msg = (
+            f"Downloading {total:,} images ({num_workers} workers, ~{throughput:.1f} img/s). "
+            f"Estimated time: ~{eta_str}"
+        )
+        if probe_fail > 0:
+            info_msg += f" | {fail_rate:.0%} broken URLs will be skipped"
+
+        print(f"✅ Probe passed: {probe_ok}/{probe_total} ok, {probe_fail} failed ({fail_rate:.0%}), {throughput:.1f} img/s")
+        print(f"⏱️ Estimated time for {total:,} samples: ~{eta_str}")
+        _notify(info_msg)
+
+    # ── Full conversion with progress ──
+    from tqdm import tqdm
+
+    print(f"🔄 Converting {total} samples to VLM format...")
+    converted_list = []
+    failed_count = 0
+
+    if has_urls:
+        # Parallel conversion for URL-based datasets
+        import time
+        from concurrent.futures import ThreadPoolExecutor, as_completed
+        from utils.hardware import safe_num_proc
+
+        num_workers = safe_num_proc()
+        batch_size = 500
+        start_time = time.time()
+
+        for batch_start in range(0, total, batch_size):
+            batch_end = min(batch_start + batch_size, total)
+            batch_samples = [dataset[i] for i in range(batch_start, batch_end)]
+
+            with ThreadPoolExecutor(max_workers=num_workers) as executor:
+                futures = {executor.submit(_convert_single_sample, s): i for i, s in enumerate(batch_samples)}
+                batch_results = [None] * len(batch_samples)
+                for future in as_completed(futures):
+                    idx = futures[future]
+                    try:
+                        batch_results[idx] = future.result()
+                    except Exception:
+                        failed_count += 1
+
+            converted_list.extend(r for r in batch_results if r is not None)
+
+            # Progress update every batch
+            elapsed = time.time() - start_time
+            done = batch_end
+            rate = done / elapsed if elapsed > 0 else 0
+            remaining_time = (total - done) / rate if rate > 0 else 0
+            eta_str = _format_eta(remaining_time)
+            progress_msg = f"Downloading images: {done:,}/{total:,} ({done*100//total}%) | ~{eta_str} remaining | {failed_count} skipped"
+            print(f"  [{done}/{total}] {rate:.1f} img/s, {failed_count} failed, ETA {eta_str}")
+            _notify(progress_msg)
+    else:
+        # Sequential conversion for local/embedded images (fast, no I/O bottleneck)
+        pbar = tqdm(dataset, total=total, desc="Converting VLM samples", unit="sample")
+        for sample in pbar:
+            try:
+                converted_list.append(_convert_single_sample(sample))
+            except Exception:
+                failed_count += 1
+            pbar.set_postfix(ok=len(converted_list), failed=failed_count, refresh=False)
+        pbar.close()
+
+    if failed_count > 0:
+        fail_rate = failed_count / total
+        print(f"⚠️ Skipped {failed_count}/{total} ({fail_rate:.0%}) samples with broken/unreachable images")
+        # For datasets that skipped the probe (small URL datasets), check fail rate now
+        if has_urls and fail_rate >= MAX_FAIL_RATE:
+            msg = (
+                f"⚠️ {fail_rate:.0%} of images failed to download ({failed_count}/{total}). "
+                "This dataset has too many broken or unreachable image URLs. "
+                "Consider using a dataset with embedded images instead."
+            )
+            _notify(msg)
+            raise ValueError(msg)
+
+    if len(converted_list) == 0:
+        raise ValueError(
+            f"All {total} samples failed during VLM conversion — no usable images found. "
+            "This dataset may contain only image URLs that are no longer accessible."
+        )
+
+    print(f"✅ Converted {len(converted_list)}/{total} samples")
+    _notify(f"Converted {len(converted_list):,}/{total:,} images successfully")
 
     # Return list, NOT Dataset
     return converted_list

studio/backend/utils/hardware/__init__.py

@@ -13,6 +13,8 @@ from .hardware import (
     get_gpu_summary,
     get_package_versions,
     get_gpu_utilization,
+    get_physical_gpu_count,
+    safe_num_proc,
 )
 
 __all__ = [
@@ -27,4 +29,6 @@ __all__ = [
     'get_gpu_summary',
     'get_package_versions',
     'get_gpu_utilization',
+    'get_physical_gpu_count',
+    'safe_num_proc',
 ]

studio/backend/utils/hardware/hardware.py

@@ -385,3 +385,80 @@ def get_gpu_utilization() -> Dict[str, Any]:
         "power_limit_w": power_limit,
         "power_utilization_pct": power_pct,
     }
+
+
+# ========== Multi-GPU Detection & Safe num_proc ==========
+
+_physical_gpu_count: Optional[int] = None
+
+def get_physical_gpu_count() -> int:
+    """
+    Return the number of physical NVIDIA GPUs on the machine.
+
+    Uses ``nvidia-smi -L`` which is NOT affected by CUDA_VISIBLE_DEVICES,
+    so it always reflects the true hardware count.
+    Result is cached after the first call.
+    """
+    global _physical_gpu_count
+    if _physical_gpu_count is not None:
+        return _physical_gpu_count
+
+    try:
+        import subprocess
+        result = subprocess.run(
+            ["nvidia-smi", "-L"],
+            capture_output=True, text=True, timeout=5,
+        )
+        if result.returncode == 0 and result.stdout.strip():
+            _physical_gpu_count = len(result.stdout.strip().splitlines())
+        else:
+            _physical_gpu_count = 1
+    except Exception:
+        _physical_gpu_count = 1
+
+    return _physical_gpu_count
+
+
+def safe_num_proc(desired: Optional[int] = None) -> int:
+    """
+    Return a safe ``num_proc`` for ``dataset.map()`` calls.
+
+    On Windows, always returns 1 because Python uses ``spawn`` instead of
+    ``fork`` for multiprocessing — the overhead of re-importing torch,
+    transformers, unsloth etc. per worker is typically slower than
+    single-process for normal dataset sizes.
+
+    On multi-GPU machines the NVIDIA driver spawns extra background threads,
+    making ``os.fork()`` prone to deadlocks when many workers are created.
+    This helper caps ``num_proc`` to 4 on such machines.
+
+    On single-GPU (or CPU-only) machines the original value is returned
+    unchanged.
+
+    Args:
+        desired: The num_proc you *want*. If None, auto-computes from
+                 ``os.cpu_count()``.
+
+    Returns:
+        A safe integer ≥ 1.
+    """
+    import os
+    import sys
+
+    # Windows uses 'spawn' for multiprocessing — the overhead of re-importing
+    # torch/transformers/unsloth per worker is typically slower than single-process.
+    if sys.platform == "win32":
+        return 1
+
+    if desired is None or not isinstance(desired, int):
+        desired = max(1, os.cpu_count() // 3)
+
+    if get_physical_gpu_count() > 1:
+        capped = min(4, desired)
+        print(
+            f"⚙️ Multi-GPU detected ({get_physical_gpu_count()} GPUs) — "
+            f"capping num_proc {desired} → {capped} to avoid fork deadlocks"
+        )
+        return capped
+
+    return desired

studio/backend/utils/models/__init__.py

@@ -3,11 +3,14 @@ Model and LoRA configuration handling
 """
 from .model_config import (
     ModelConfig,
+    GgufVariantInfo,
     is_vision_model,
     scan_trained_loras,
+    scan_exported_models,
     load_model_defaults,
     get_base_model_from_lora,
     load_model_config,
+    list_gguf_variants,
     MODEL_NAME_MAPPING,
     UI_STATUS_INDICATORS,
 )
@@ -15,11 +18,14 @@ from .checkpoints import scan_checkpoints
 
 __all__ = [
     'ModelConfig',
+    'GgufVariantInfo',
     'is_vision_model',
     'scan_trained_loras',
+    'scan_exported_models',
     'load_model_defaults',
     'get_base_model_from_lora',
     'load_model_config',
+    'list_gguf_variants',
     'MODEL_NAME_MAPPING',
     'UI_STATUS_INDICATORS',
     'scan_checkpoints',

studio/backend/utils/models/model_config.py

@@ -422,6 +422,228 @@ def is_vision_model(model_name: str, hf_token: Optional[str] = None) -> bool:
 pass
 
 
+def _is_mmproj(filename: str) -> bool:
+    """Check if a GGUF filename is a vision projection (mmproj) file."""
+    return "mmproj" in filename.lower()
+
+
+def detect_mmproj_file(path: str) -> Optional[str]:
+    """
+    Find the mmproj (vision projection) GGUF file in a directory.
+
+    Args:
+        path: Directory to search — or a .gguf file (uses its parent dir).
+
+    Returns:
+        Full path to the mmproj .gguf file, or None if not found.
+    """
+    p = Path(path)
+    search_dir = p.parent if p.is_file() else p
+    if not search_dir.is_dir():
+        return None
+
+    for f in search_dir.glob("*.gguf"):
+        if _is_mmproj(f.name):
+            return str(f.resolve())
+    return None
+
+
+def detect_gguf_model(path: str) -> Optional[str]:
+    """
+    Check if the given local path is or contains a GGUF model file.
+
+    Handles two cases:
+    1. path is a direct .gguf file path
+    2. path is a directory containing .gguf files
+
+    Skips mmproj (vision projection) files — those must be passed via
+    ``--mmproj``, not ``-m``.  Use :func:`detect_mmproj_file` instead.
+
+    Returns the full path to the .gguf file if found, None otherwise.
+    For HuggingFace repo detection, use detect_gguf_model_remote() instead.
+    """
+    p = Path(path)
+
+    # Case 1: direct .gguf file
+    if p.suffix == ".gguf" and p.is_file():
+        if _is_mmproj(p.name):
+            return None
+        return str(p.resolve())
+
+    # Case 2: directory containing .gguf files (skip mmproj)
+    if p.is_dir():
+        gguf_files = sorted(
+            (f for f in p.glob("*.gguf") if not _is_mmproj(f.name)),
+            key=lambda f: f.stat().st_size, reverse=True,
+        )
+        if gguf_files:
+            return str(gguf_files[0].resolve())
+
+    return None
+
+
+# Preferred GGUF quantization levels, in descending priority.
+# Q4_K_M is a good default: small, fast, acceptable quality.
+_GGUF_QUANT_PREFERENCE = [
+    "Q4_K_M", "Q4_K_S", "Q5_K_M", "Q5_K_S",
+    "Q6_K", "Q8_0", "Q3_K_M", "Q3_K_L", "Q2_K",
+    "F16", "BF16", "F32",
+]
+
+
+def _pick_best_gguf(filenames: list[str]) -> Optional[str]:
+    """
+    Pick the best GGUF file from a list of filenames.
+
+    Prefers quantization levels in _GGUF_QUANT_PREFERENCE order.
+    Falls back to the first .gguf file found.
+    """
+    gguf_files = [f for f in filenames if f.endswith(".gguf")]
+    if not gguf_files:
+        return None
+
+    # Try preferred quantization levels
+    for quant in _GGUF_QUANT_PREFERENCE:
+        for f in gguf_files:
+            if quant in f:
+                return f
+
+    # Fallback: first GGUF file
+    return gguf_files[0]
+
+
+@dataclass
+class GgufVariantInfo:
+    """A single GGUF quantization variant from a HuggingFace repo."""
+    filename: str       # e.g., "gemma-3-4b-it-Q4_K_M.gguf"
+    quant: str          # e.g., "Q4_K_M" (extracted from filename)
+    size_bytes: int     # file size
+
+
+def _extract_quant_label(filename: str) -> str:
+    """
+    Extract quantization label like Q4_K_M, IQ4_XS, BF16 from a GGUF filename.
+
+    Examples:
+        "gemma-3-4b-it-Q4_K_M.gguf"          → "Q4_K_M"
+        "model-IQ4_NL.gguf"                   → "IQ4_NL"
+        "model-BF16.gguf"                     → "BF16"
+        "model-UD-IQ1_S.gguf"                 → "UD-IQ1_S"
+        "model-UD-TQ1_0.gguf"                 → "UD-TQ1_0"
+        "MXFP4_MOE/model-MXFP4_MOE-0001.gguf"→ "MXFP4_MOE"
+    """
+    import re
+    # Use only the basename (rfilename may include directory)
+    basename = filename.rsplit("/", 1)[-1]
+    # Strip .gguf and any shard suffix (-00001-of-00010)
+    stem = re.sub(r'-\d{3,}-of-\d{3,}', '', basename.rsplit(".", 1)[0])
+    # Match known quantization patterns
+    match = re.search(
+        r'(UD-)?'  # Optional UD- prefix (Ultra Discrete)
+        r'(MXFP[0-9]+(?:_[A-Z0-9]+)*'         # MXFP variants: MXFP4, MXFP4_MOE
+        r'|IQ[0-9]+_[A-Z]+(?:_[A-Z0-9]+)?'    # IQ variants: IQ4_XS, IQ4_NL, IQ1_S
+        r'|TQ[0-9]+_[0-9]+'                    # Ternary quant: TQ1_0, TQ2_0
+        r'|Q[0-9]+_K_[A-Z]+'                   # K-quant: Q4_K_M, Q3_K_S
+        r'|Q[0-9]+_[0-9]+'                     # Standard: Q8_0, Q5_1
+        r'|Q[0-9]+_K'                           # Short K-quant: Q6_K
+        r'|BF16|F16|F32)',                      # Full precision
+        stem, re.IGNORECASE,
+    )
+    if match:
+        prefix = match.group(1) or ""
+        return f"{prefix}{match.group(2)}"
+    # Fallback: last segment after hyphen
+    return stem.split("-")[-1]
+
+
+def list_gguf_variants(
+    repo_id: str,
+    hf_token: Optional[str] = None,
+) -> tuple[list[GgufVariantInfo], bool]:
+    """
+    List all GGUF quantization variants in a HuggingFace repo.
+
+    Separates main model files from mmproj (vision projection) files.
+    The presence of mmproj files indicates a vision-capable model.
+
+    Returns:
+        (variants, has_vision): list of non-mmproj GGUF variants + vision flag.
+    """
+    from huggingface_hub import model_info as hf_model_info
+
+    info = hf_model_info(repo_id, token=hf_token, files_metadata=True)
+    variants: list[GgufVariantInfo] = []
+    has_vision = False
+
+    quant_totals: dict[str, int] = {}      # quant -> total bytes
+    quant_first_file: dict[str, str] = {}  # quant -> first filename (for display)
+
+    for sibling in info.siblings:
+        fname = sibling.rfilename
+        if not fname.endswith(".gguf"):
+            continue
+        size = sibling.size or 0
+
+        # mmproj files are vision projection models, not main model files
+        if "mmproj" in fname.lower():
+            has_vision = True
+            continue
+
+        quant = _extract_quant_label(fname)
+        quant_totals[quant] = quant_totals.get(quant, 0) + size
+        if quant not in quant_first_file:
+            quant_first_file[quant] = fname
+
+    for quant, total_size in quant_totals.items():
+        variants.append(GgufVariantInfo(
+            filename=quant_first_file[quant],
+            quant=quant,
+            size_bytes=total_size,
+        ))
+
+    return variants, has_vision
+
+
+def detect_gguf_model_remote(
+    repo_id: str,
+    hf_token: Optional[str] = None,
+) -> Optional[str]:
+    """
+    Check if a HuggingFace repo contains GGUF files.
+
+    Returns the filename of the best GGUF file in the repo, or None.
+    """
+    try:
+        from huggingface_hub import model_info as hf_model_info
+
+        info = hf_model_info(repo_id, token=hf_token)
+        repo_files = [s.rfilename for s in info.siblings]
+        return _pick_best_gguf(repo_files)
+    except Exception as e:
+        logger.debug(f"Could not check GGUF files for '{repo_id}': {e}")
+        return None
+
+
+def download_gguf_file(
+    repo_id: str,
+    filename: str,
+    hf_token: Optional[str] = None,
+) -> str:
+    """
+    Download a specific GGUF file from a HuggingFace repo.
+
+    Returns the local path to the downloaded file.
+    """
+    from huggingface_hub import hf_hub_download
+
+    local_path = hf_hub_download(
+        repo_id=repo_id,
+        filename=filename,
+        token=hf_token,
+    )
+    return local_path
+
+
 def scan_trained_loras(outputs_dir: str = "./outputs") -> List[Tuple[str, str]]:
     """
     Scan outputs folder for trained LoRA adapters.
@@ -465,6 +687,129 @@ def scan_trained_loras(outputs_dir: str = "./outputs") -> List[Tuple[str, str]]:
         logger.error(f"Error scanning outputs folder: {e}")
         return []
 
+def scan_exported_models(exports_dir: str = "./exports") -> List[Tuple[str, str, str, Optional[str]]]:
+    """
+    Scan exports folder for exported models (merged, LoRA, GGUF).
+
+    Supports two directory layouts:
+      - Two-level: {run}/{checkpoint}/  (merged & LoRA exports)
+      - Flat:      {name}-finetune-gguf/  (GGUF exports)
+
+    Returns:
+        List of tuples: [(display_name, model_path, export_type, base_model), ...]
+        export_type: "lora" | "merged" | "gguf"
+    """
+    results = []
+    exports_path = Path(exports_dir)
+
+    if not exports_path.exists():
+        return results
+
+    try:
+        for run_dir in exports_path.iterdir():
+            if not run_dir.is_dir():
+                continue
+
+            # Check for flat GGUF export (e.g. exports/gemma-3-4b-it-finetune-gguf/)
+            # Filter out mmproj (vision projection) files — they aren't loadable as main models
+            gguf_files = [f for f in run_dir.glob("*.gguf") if not _is_mmproj(f.name)]
+            if gguf_files:
+                base_model = None
+                export_meta = run_dir / "export_metadata.json"
+                try:
+                    if export_meta.exists():
+                        meta = json.loads(export_meta.read_text())
+                        base_model = meta.get("base_model")
+                except Exception:
+                    pass
+
+                display_name = run_dir.name
+                model_path = str(gguf_files[0])  # path to the .gguf file
+                results.append((display_name, model_path, "gguf", base_model))
+                logger.debug(f"Found GGUF export: {display_name}")
+                continue
+
+            # Two-level: {run}/{checkpoint}/
+            for checkpoint_dir in run_dir.iterdir():
+                if not checkpoint_dir.is_dir():
+                    continue
+
+                adapter_config = checkpoint_dir / "adapter_config.json"
+                config_file = checkpoint_dir / "config.json"
+                has_weights = (
+                    any(checkpoint_dir.glob("*.safetensors"))
+                    or any(checkpoint_dir.glob("*.bin"))
+                )
+                has_gguf = any(checkpoint_dir.glob("*.gguf"))
+
+                base_model = None
+                export_type = None
+
+                if adapter_config.exists():
+                    export_type = "lora"
+                    try:
+                        cfg = json.loads(adapter_config.read_text())
+                        base_model = cfg.get("base_model_name_or_path")
+                    except Exception:
+                        pass
+                elif config_file.exists() and has_weights:
+                    export_type = "merged"
+                    export_meta = checkpoint_dir / "export_metadata.json"
+                    try:
+                        if export_meta.exists():
+                            meta = json.loads(export_meta.read_text())
+                            base_model = meta.get("base_model")
+                    except Exception:
+                        pass
+                elif has_gguf:
+                    export_type = "gguf"
+                    gguf_list = list(checkpoint_dir.glob("*.gguf"))
+                    # Check checkpoint_dir first, then fall back to parent run_dir
+                    # (export.py writes metadata to the top-level export directory)
+                    for meta_dir in (checkpoint_dir, run_dir):
+                        export_meta = meta_dir / "export_metadata.json"
+                        try:
+                            if export_meta.exists():
+                                meta = json.loads(export_meta.read_text())
+                                base_model = meta.get("base_model")
+                                if base_model:
+                                    break
+                        except Exception:
+                            pass
+
+                    display_name = f"{run_dir.name} / {checkpoint_dir.name}"
+                    model_path = str(gguf_list[0]) if gguf_list else str(checkpoint_dir)
+                    results.append((display_name, model_path, export_type, base_model))
+                    logger.debug(f"Found GGUF export: {display_name}")
+                    continue
+                else:
+                    continue
+
+                # Fallback: read base model from the original training run's
+                # adapter_config.json in ./outputs/{run_name}/
+                if not base_model:
+                    outputs_adapter_cfg = Path("./outputs") / run_dir.name / "adapter_config.json"
+                    try:
+                        if outputs_adapter_cfg.exists():
+                            cfg = json.loads(outputs_adapter_cfg.read_text())
+                            base_model = cfg.get("base_model_name_or_path")
+                    except Exception:
+                        pass
+
+                display_name = f"{run_dir.name} / {checkpoint_dir.name}"
+                model_path = str(checkpoint_dir)
+                results.append((display_name, model_path, export_type, base_model))
+                logger.debug(f"Found exported model: {display_name} ({export_type})")
+
+        results.sort(key=lambda x: Path(x[1]).stat().st_mtime, reverse=True)
+        logger.info(f"Found {len(results)} exported models in {exports_dir}")
+        return results
+
+    except Exception as e:
+        logger.error(f"Error scanning exports folder: {e}")
+        return []
+
+
 def get_base_model_from_lora(lora_path: str) -> Optional[str]:
     """
     Read the base model name from a LoRA adapter's config.
@@ -595,6 +940,11 @@ class ModelConfig:
     is_cached: bool     # Is this already in HF cache?
     is_vision: bool     # Is this a vision model?
     is_lora: bool       # Is this a lora adapter?
+    is_gguf: bool = False       # Is this a GGUF model?
+    gguf_file: Optional[str] = None  # Full path to the .gguf file (local mode)
+    gguf_mmproj_file: Optional[str] = None  # Full path to the mmproj .gguf file (vision projection)
+    gguf_hf_repo: Optional[str] = None  # HF repo ID for -hf mode (e.g. "unsloth/gemma-3-4b-it-GGUF")
+    gguf_variant: Optional[str] = None  # Quantization variant (e.g. "Q4_K_M")
     base_model: Optional[str] = None  # Base model (for LoRAs)
 
     @classmethod
@@ -650,37 +1000,130 @@ class ModelConfig:
         cls,
         model_id: str,
         hf_token: Optional[str] = None,
-        is_lora: bool = False
+        is_lora: bool = False,
+        gguf_variant: Optional[str] = None,
     ) -> Optional['ModelConfig']:
         """
         Create ModelConfig from a clean model identifier.
-        
+
         For FastAPI routes where the frontend sends sanitized model paths.
         No Gradio dropdown parsing - expects clean identifiers like:
         - "unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit"
         - "./outputs/my_lora_adapter"
         - "/absolute/path/to/model"
-        
+
         Args:
             model_id: Clean model identifier (HF repo name or local path)
             hf_token: Optional HF token for vision detection on gated models
             is_lora: Whether this is a LoRA adapter
-        
+            gguf_variant: Optional GGUF quantization variant (e.g. "Q4_K_M").
+                For remote GGUF repos, specifies which quant to load via -hf.
+                If None, auto-selects using _pick_best_gguf().
+
         Returns:
             ModelConfig or None if configuration cannot be created
         """
         if not model_id or not model_id.strip():
             return None
-        
+
         identifier = model_id.strip()
         is_local = is_local_path(identifier)
         path = normalize_path(identifier) if is_local else identifier
-        
+
         # Add unsloth/ prefix for shorthand HF models
         if not is_local and "/" not in identifier:
             identifier = f"unsloth/{identifier}"
             path = identifier
-        
+
+        # Auto-detect GGUF models (check before LoRA/vision detection)
+        if is_local:
+            gguf_file = detect_gguf_model(path)
+            if gguf_file:
+                display_name = Path(gguf_file).stem
+                logger.info(f"Detected local GGUF model: {gguf_file}")
+
+                # Detect vision: check if base model is vision, then look for mmproj
+                mmproj_file = None
+                gguf_is_vision = False
+                gguf_dir = Path(gguf_file).parent
+
+                # Determine if this is a vision model from export metadata
+                base_is_vision = False
+                meta_path = gguf_dir / "export_metadata.json"
+                if meta_path.exists():
+                    try:
+                        meta = json.loads(meta_path.read_text())
+                        base = meta.get("base_model")
+                        if base and is_vision_model(base, hf_token=hf_token):
+                            base_is_vision = True
+                            logger.info(f"GGUF base model '{base}' is a vision model")
+                    except Exception as e:
+                        logger.debug(f"Could not read export metadata: {e}")
+
+                # If vision (or mmproj happens to exist), find the mmproj file
+                mmproj_file = detect_mmproj_file(gguf_file)
+                if mmproj_file:
+                    gguf_is_vision = True
+                    logger.info(f"Detected mmproj for vision: {mmproj_file}")
+                elif base_is_vision:
+                    logger.warning(f"Base model is vision but no mmproj file found in {gguf_dir}")
+
+                return cls(
+                    identifier=identifier,
+                    display_name=display_name,
+                    path=path,
+                    is_local=True,
+                    is_cached=True,
+                    is_vision=gguf_is_vision,
+                    is_lora=False,
+                    is_gguf=True,
+                    gguf_file=gguf_file,
+                    gguf_mmproj_file=mmproj_file,
+                )
+        else:
+            # Check if the HF repo contains GGUF files
+            gguf_filename = detect_gguf_model_remote(identifier, hf_token=hf_token)
+            if gguf_filename:
+                # Preflight: verify llama-server binary exists BEFORE user waits
+                # for a multi-GB download that llama-server handles natively
+                from core.inference.llama_cpp import LlamaCppBackend
+                if not LlamaCppBackend._find_llama_server_binary():
+                    raise RuntimeError(
+                        "llama-server binary not found — cannot load GGUF models. "
+                        "Run setup.sh to build it, or set LLAMA_SERVER_PATH."
+                    )
+
+                # Use list_gguf_variants() to detect vision & resolve variant
+                variants, has_vision = list_gguf_variants(identifier, hf_token=hf_token)
+                variant = gguf_variant
+                if not variant:
+                    # Auto-select best quantization
+                    variant_filenames = [v.filename for v in variants]
+                    best = _pick_best_gguf(variant_filenames)
+                    if best:
+                        variant = _extract_quant_label(best)
+                    else:
+                        variant = "Q4_K_M"  # Fallback — llama-server's own default
+
+                display_name = f"{identifier.split('/')[-1]} ({variant})"
+                logger.info(
+                    f"Detected remote GGUF repo '{identifier}', "
+                    f"variant={variant}, vision={has_vision}"
+                )
+                return cls(
+                    identifier=identifier,
+                    display_name=display_name,
+                    path=identifier,
+                    is_local=False,
+                    is_cached=False,
+                    is_vision=has_vision,
+                    is_lora=False,
+                    is_gguf=True,
+                    gguf_file=None,
+                    gguf_hf_repo=identifier,
+                    gguf_variant=variant,
+                )
+
         # Auto-detect LoRA for local paths (check adapter_config.json on disk)
         if not is_lora and is_local:
             detected_base = get_base_model_from_lora(path)

studio/frontend/.gitignore

@@ -1,28 +1,31 @@
-# Logs
-logs
-*.log
-npm-debug.log*
-yarn-debug.log*
-yarn-error.log*
-pnpm-debug.log*
-lerna-debug.log*
-
-node_modules
+# Logs
+logs
+*.log
+npm-debug.log*
+yarn-debug.log*
+yarn-error.log*
+pnpm-debug.log*
+lerna-debug.log*
+
+node_modules
 dist
 dist-ssr
 test/
 *.local
 .env
 .env.*
-
-# Editor directories and files
-.vscode/*
-!.vscode/extensions.json
-.idea
-.DS_Store
-._*
-*.suo
-*.ntvs*
-*.njsproj
-*.sln
-*.sw?
+.omx/
+
+# Editor directories and files
+.vscode/*
+!.vscode/extensions.json
+.idea
+.DS_Store
+._*
+*.suo
+*.ntvs*
+*.njsproj
+*.sln
+*.sw?
+/src/features/recipe-studio/AGENTS.md
+/docs

studio/frontend/AGENTS.md

@@ -0,0 +1,37 @@
+# Repository Guidelines
+
+## Project Structure & Module Organization
+- `src/` is app code; entry is `src/main.tsx`, global styles in `src/index.css`.
+- `src/app/` holds app shell and routing; `src/features/` is feature slices w/ public `index.ts` exports.
+- Shared UI lives in `src/components/` (shadcn in `src/components/ui/`).
+- Shared logic in `src/hooks/`, `src/stores/`, `src/utils/`, `src/lib/`, and types in `src/types/`.
+- Static assets: `src/assets/` and `public/`.
+- `test/` is a Python harness for payload validation and preview; not a JS test suite.
+
+## Build, Test, and Development Commands
+- `bun run dev`: start Vite dev server.
+- `bun run build`: typecheck + build to `dist/`.
+- `bun run preview`: serve the production build locally.
+- `bun run lint`: ESLint checks for TS/React.
+- `bun run typecheck`: `tsc` no-emit verification.
+- `bun run biome:check` / `bun run biome:fix`: format + lint w/ Biome.
+- Optional harness: `python test/scripts/validate_payload.py test/data/ui_payload.json`.
+
+## Coding Style & Naming Conventions
+- TypeScript + React, 2-space indent (Biome).
+- Prefer explicit, compact code; avoid heavy abstraction.
+- Use path alias `@/` for app imports.
+- Feature boundaries enforced: import from `@/features/<name>` only, not deep paths.
+- Components in `PascalCase`, hooks in `useCamelCase`, files in `kebab-case` or `camelCase` per local convention.
+
+## Testing Guidelines
+- No frontend test runner configured yet; add one if needed.
+- `test/` is for API payload validation and preview flows; add samples as `test/data/ui_payload_*.json`.
+
+## Commit & Pull Request Guidelines
+- Commit history shows short, imperative messages; optional prefix like `refactor:`; keep it terse.
+- PRs should include: clear summary, linked issue (if any), and UI screenshots/gifs for visual changes.
+- Call out new deps, config, or required env changes in the PR body.
+
+## Agent Notes
+- Keep changes minimal, focused, and easy to review.

studio/frontend/CLAUDE.md

@@ -1,66 +0,0 @@
-# CLAUDE.md
-
-This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
-
-## Project Overview
-
-LLM fine-tuning platform UI (Unsloth-branded). React/TypeScript frontend with a skeletal Python backend. The frontend is the active development focus.
-
-## Commands
-
-All commands run from `frontend/`:
-
-```bash
-bun install          # install dependencies
-bun run dev          # start Vite dev server
-bun run build        # typecheck + production build
-bun run typecheck    # TypeScript type checking only
-bun run lint         # ESLint
-bun run biome:check  # Biome linter + formatter check
-bun run biome:fix    # Biome auto-fix
-```
-
-Package manager is **Bun** (not npm/yarn).
-
-## Architecture
-
-### Frontend (`frontend/src/`)
-
-
-**Feature-based module architecture** with enforced boundaries:
-
-- `features/` — self-contained feature modules (chat, onboarding, studio)
-- `components/ui/` — shadcn/ui primitives (linting/formatting disabled for these)
-- `components/assistant-ui/` — AI chat thread components
-- `components/layout/` — layout shells (dashboard)
-- `stores/` — Zustand stores (training wizard state)
-- `config/` — constants (model lists, hyperparameters, env)
-- `types/` — shared TypeScript types
-- `app/` — router and root layout (TanStack React Router)
-
-### Import Rules (ESLint-enforced)
-
-Cross-feature imports are **prohibited**. Import from feature barrel (`@/features/[name]`), never from internal paths (`@/features/chat/some-component`).
-
-### Key Technology Choices
-
-| Concern | Choice |
-|---------|--------|
-| Routing | TanStack React Router |
-| State | Zustand |
-| Styling | Tailwind CSS + shadcn/ui (radix-maia style, HugeIcons) |
-| Animation | Framer Motion |
-| Chat UI | @assistant-ui/react with streaming |
-| Local DB | Dexie (IndexedDB) for chat threads/messages |
-| Charts | Recharts |
-
-### Backend (`backend/`)
-
-Placeholder Python structure. Frontend expects an inference server at the URL in `frontend/.env` (`VITE_INFERENCE_URL`) serving POST `/api/chat/generate` with streaming responses.
-
-## Code Style
-
-- Biome handles formatting (2-space indent) and import organization
-- `src/components/ui/**` is excluded from Biome linting/formatting (generated shadcn code)
-- Path alias: `@` maps to `frontend/src/`
-- Prefer KISS and DRY

studio/frontend/biome.json

@@ -4,6 +4,8 @@
     "ignore": [
       "dist",
       "node_modules",
+      "test",
+      "test/**",
       "**/._*",
       "._*",
       "**/.DS_Store",

studio/frontend/package.json

@@ -13,77 +13,81 @@
     "biome:fix": "biome check . --write"
   },
   "dependencies": {
-    "@assistant-ui/react": "^0.12.3",
-    "@assistant-ui/react-markdown": "^0.12.1",
-    "@assistant-ui/react-streamdown": "^0.1.0",
-    "@base-ui/react": "^1.1.0",
+    "@assistant-ui/react": "^0.12.10",
+    "@assistant-ui/react-markdown": "^0.12.3",
+    "@assistant-ui/react-streamdown": "^0.1.2",
+    "@base-ui/react": "^1.2.0",
     "@dagrejs/dagre": "^2.0.4",
     "@dagrejs/graphlib": "^3.0.4",
     "@fontsource-variable/figtree": "^5.2.10",
     "@fontsource-variable/inter": "^5.2.8",
     "@fontsource-variable/space-grotesk": "^5.2.10",
     "@hugeicons/core-free-icons": "^3.1.1",
-    "@hugeicons/react": "^1.1.4",
-    "@huggingface/hub": "^2.8.0",
+    "@hugeicons/react": "^1.1.5",
+    "@huggingface/hub": "^2.9.0",
+    "@langchain/core": "^1.1.27",
+    "@langchain/textsplitters": "^1.0.1",
     "@radix-ui/react-checkbox": "^1.3.3",
     "@radix-ui/react-label": "^2.1.8",
     "@radix-ui/react-select": "^2.2.6",
     "@radix-ui/react-separator": "^1.1.8",
     "@radix-ui/react-slot": "^1.2.4",
-    "@streamdown/cjk": "^1.0.1",
-    "@streamdown/code": "^1.0.1",
-    "@streamdown/math": "^1.0.1",
-    "@streamdown/mermaid": "^1.0.1",
-    "@tailwindcss/vite": "^4.1.17",
-    "@tanstack/react-router": "^1.156.0",
+    "@streamdown/cjk": "^1.0.2",
+    "@streamdown/code": "^1.0.2",
+    "@streamdown/math": "^1.0.2",
+    "@streamdown/mermaid": "^1.0.2",
+    "@tailwindcss/vite": "^4.1.18",
+    "@tanstack/react-router": "^1.159.10",
     "@tanstack/react-table": "^8.21.3",
     "@toolwind/corner-shape": "^0.0.8-3",
     "@types/canvas-confetti": "^1.9.0",
     "@xyflow/react": "^12.10.0",
-    "assistant-stream": "^0.3.0",
+    "assistant-stream": "^0.3.2",
     "canvas-confetti": "^1.9.4",
     "class-variance-authority": "^0.7.1",
     "clsx": "^2.1.1",
     "cmdk": "^1.1.1",
     "date-fns": "^4.1.0",
-    "dexie": "^4.2.1",
-    "framer-motion": "^11.15.0",
-    "katex": "^0.16.22",
-    "lucide-react": "^0.563.0",
+    "dexie": "^4.3.0",
+    "framer-motion": "^11.18.2",
+    "js-yaml": "^4.1.1",
+    "katex": "^0.16.28",
+    "lucide-react": "^0.575.0",
     "mammoth": "^1.11.0",
-    "motion": "^12.29.2",
+    "motion": "^12.34.0",
     "next": "^16.1.6",
     "next-themes": "^0.4.6",
     "radix-ui": "^1.4.3",
-    "react": "^19.2.0",
-    "react-day-picker": "^9.13.0",
-    "react-dom": "^19.2.0",
-    "react-resizable-panels": "^4.4.1",
-    "recharts": "2.15.4",
+    "react": "^19.2.4",
+    "react-day-picker": "^9.13.2",
+    "react-dom": "^19.2.4",
+    "react-resizable-panels": "^4.6.4",
+    "recharts": "3.7.0",
     "remark-gfm": "^4.0.1",
-    "shadcn": "^3.7.0",
+    "shadcn": "^3.8.4",
     "sonner": "^2.0.7",
-    "streamdown": "^2.1.0",
+    "streamdown": "^2.3.0",
     "tailwind-merge": "^3.4.0",
-    "tailwindcss": "^4.1.17",
+    "tailwindcss": "^4.1.18",
     "tw-animate-css": "^1.4.0",
-    "tw-shimmer": "^0.4.4",
+    "tw-shimmer": "^0.4.6",
     "unpdf": "^1.4.0",
-    "zustand": "^5.0.10"
+    "zustand": "^5.0.11"
   },
   "devDependencies": {
     "@biomejs/biome": "^1.9.4",
     "@eslint/js": "^9.39.1",
+    "@types/js-yaml": "^4.0.9",
     "@types/node": "^24.10.1",
     "@types/react": "^19.2.5",
     "@types/react-dom": "^19.2.3",
     "@vitejs/plugin-react": "^5.1.1",
     "eslint": "^9.39.1",
     "eslint-plugin-react-hooks": "^7.0.1",
-    "eslint-plugin-react-refresh": "^0.4.24",
+    "eslint-plugin-react-refresh": "^0.4.26",
     "globals": "^16.5.0",
     "typescript": "~5.9.3",
-    "typescript-eslint": "^8.46.4",
-    "vite": "^7.2.4"
+    "typescript-eslint": "^8.55.0",
+    "vite": "^7.3.1"
   }
 }

studio/frontend/src/app/provider.tsx

@@ -10,7 +10,7 @@ export function AppProvider({ children }: AppProviderProps) {
   return (
     <ThemeProvider attribute="class" defaultTheme="light">
       {children}
-      <Toaster position="top-right" />
+      <Toaster position="top-right" visibleToasts={2} expand={true} />
     </ThemeProvider>
   );
 }

studio/frontend/src/app/router.tsx

@@ -1,11 +1,13 @@
 import { createRouter } from "@tanstack/react-router";
 import { Route as rootRoute } from "./routes/__root";
+import { Route as dataRecipesRoute } from "./routes/data-recipes";
+import { Route as dataRecipeRoute } from "./routes/data-recipes.$recipeId";
 import { Route as chatRoute } from "./routes/chat";
+import { Route as exportRoute } from "./routes/export";
 import { Route as gridTestRoute } from "./routes/grid-test";
 import { Route as indexRoute } from "./routes/index";
 import { Route as loginRoute } from "./routes/login";
 import { Route as onboardingRoute } from "./routes/onboarding";
-import { Route as exportRoute } from "./routes/export";
 import { Route as signupRoute } from "./routes/signup";
 import { Route as studioRoute } from "./routes/studio";
 
@@ -18,6 +20,8 @@ const routeTree = rootRoute.addChildren([
   studioRoute,
   chatRoute,
   exportRoute,
+  dataRecipesRoute,
+  dataRecipeRoute,
 ]);
 
 export const router = createRouter({ routeTree });

studio/frontend/src/app/routes/data-recipes.$recipeId.tsx

@@ -0,0 +1,23 @@
+import { createRoute } from "@tanstack/react-router";
+import type { ReactElement } from "react";
+import { lazy } from "react";
+import { requireAuth } from "../auth-guards";
+import { Route as rootRoute } from "./__root";
+
+const EditRecipePage = lazy(() =>
+  import("@/features/data-recipes").then((m) => ({
+    default: m.EditRecipePage,
+  })),
+);
+
+export const Route = createRoute({
+  getParentRoute: () => rootRoute,
+  path: "/data-recipes/$recipeId",
+  beforeLoad: () => requireAuth(),
+  component: DataRecipeEditorRoute,
+});
+
+function DataRecipeEditorRoute(): ReactElement {
+  const { recipeId } = Route.useParams();
+  return <EditRecipePage recipeId={recipeId} />;
+}

studio/frontend/src/app/routes/data-recipes.tsx

@@ -0,0 +1,17 @@
+import { createRoute } from "@tanstack/react-router";
+import { lazy } from "react";
+import { requireAuth } from "../auth-guards";
+import { Route as rootRoute } from "./__root";
+
+const DataRecipesPage = lazy(() =>
+  import("@/features/data-recipes").then((m) => ({
+    default: m.DataRecipesPage,
+  })),
+);
+
+export const Route = createRoute({
+  getParentRoute: () => rootRoute,
+  path: "/data-recipes",
+  beforeLoad: () => requireAuth(),
+  component: DataRecipesPage,
+});

studio/frontend/src/components/assistant-ui/attachment.tsx

@@ -149,10 +149,8 @@ const AttachmentUI: FC = () => {
         return "Document";
       case "file":
         return "File";
-      default: {
-        const _exhaustiveCheck: never = type;
-        throw new Error(`Unknown attachment type: ${_exhaustiveCheck}`);
-      }
+      default:
+        throw new Error(`Unknown attachment type: ${type as string}`);
     }
   });
 

studio/frontend/src/components/assistant-ui/markdown-text.tsx

@@ -1,20 +1,107 @@
 "use client";
 
-import { INTERNAL } from "@assistant-ui/react";
-import { StreamdownTextPrimitive } from "@assistant-ui/react-streamdown";
+import { INTERNAL, useMessagePartText } from "@assistant-ui/react";
+import { copyToClipboard } from "@/lib/copy-to-clipboard";
+import { Copy02Icon, Tick02Icon } from "@hugeicons/core-free-icons";
+import { HugeiconsIcon } from "@hugeicons/react";
 import { code } from "@streamdown/code";
 import { math } from "@streamdown/math";
 import { mermaid } from "@streamdown/mermaid";
+import { Block, type BlockProps, Streamdown } from "streamdown";
+import { useEffect, useRef, useState } from "react";
 import "katex/dist/katex.min.css";
 
-const { withSmoothContextProvider } = INTERNAL;
+const { withSmoothContextProvider, useSmoothStatus } = INTERNAL;
+
+function getMermaidSource(blockContent: string): string | null {
+  const source = blockContent.match(/```mermaid\s*([\s\S]*?)```/i)?.[1]?.trim();
+  return source && source.length > 0 ? source : null;
+}
+
+const COPY_RESET_MS = 2000;
+
+function MermaidCopyButton({ source }: { source: string }) {
+  const [copied, setCopied] = useState(false);
+  const resetTimeoutRef = useRef<ReturnType<typeof setTimeout> | null>(null);
+
+  useEffect(() => {
+    return () => {
+      if (resetTimeoutRef.current) {
+        clearTimeout(resetTimeoutRef.current);
+      }
+    };
+  }, []);
+
+  return (
+    <button
+      type="button"
+      className="absolute top-3.5 right-20 z-20 cursor-pointer text-muted-foreground transition-all hover:text-foreground"
+      title="Copy Mermaid source"
+      onClick={() => {
+        if (!copyToClipboard(source)) return;
+        setCopied(true);
+        if (resetTimeoutRef.current) {
+          clearTimeout(resetTimeoutRef.current);
+        }
+        resetTimeoutRef.current = setTimeout(() => {
+          setCopied(false);
+          resetTimeoutRef.current = null;
+        }, COPY_RESET_MS);
+      }}
+    >
+      <HugeiconsIcon icon={copied ? Tick02Icon : Copy02Icon} className="size-5" />
+    </button>
+  );
+}
+
+function StreamdownBlock(props: BlockProps) {
+  const hasMermaidFence = props.content.includes("```mermaid");
+  const mermaidSource = getMermaidSource(props.content);
+
+  if (props.isIncomplete && hasMermaidFence) {
+    return (
+      <div className="my-4 flex h-48 items-center justify-center rounded-xl border border-border bg-muted/30 text-sm text-muted-foreground animate-pulse">
+        Loading diagram...
+      </div>
+    );
+  }
+
+  if (mermaidSource) {
+    return (
+      <div className="relative">
+        <Block {...props} />
+        <MermaidCopyButton source={mermaidSource} />
+      </div>
+    );
+  }
+
+  return <Block {...props} />;
+}
 
 const MarkdownTextImpl = () => {
+  const { text } = useMessagePartText();
+  const status = useSmoothStatus();
+
   return (
-    <StreamdownTextPrimitive
-      plugins={{ code, math, mermaid }}
-      controls={true}
-    />
+    <div data-status={status.type}>
+      <Streamdown
+        mode="streaming"
+        isAnimating={status.type === "running"}
+        plugins={{ code, math, mermaid }}
+        controls={{
+          mermaid: {
+            fullscreen: true,
+            download: true,
+            copy: false,
+            panZoom: true,
+          },
+        }}
+        shikiTheme={["github-light", "github-dark"]}
+        BlockComponent={StreamdownBlock}
+      >
+        {text}
+      </Streamdown>
+    </div>
   );
 };
 

studio/frontend/src/components/assistant-ui/model-selector.tsx

@@ -27,6 +27,7 @@ interface ModelSelectorProps {
   loraModels?: LoraModelOption[];
   value?: string;
   defaultValue?: string;
+  activeGgufVariant?: string | null;
   onValueChange?: (value: string, meta: ModelSelectorChangeMeta) => void;
   onEject?: () => void;
   variant?: "outline" | "ghost" | "muted";
@@ -62,7 +63,7 @@ function ModelSelectorTrigger({
         className={cn(
           "flex items-center gap-2 transition-colors",
           variant === "outline" &&
-            "rounded-full border border-border/60 hover:bg-accent",
+          "rounded-full border border-border/60 hover:bg-accent",
           variant === "ghost" && "rounded-md hover:bg-accent",
           variant === "muted" && "rounded-md bg-muted hover:bg-muted/80",
           size === "sm" && "h-8 px-3 text-xs",
@@ -158,6 +159,7 @@ export function ModelSelector({
   loraModels = [],
   value,
   defaultValue,
+  activeGgufVariant,
   onValueChange,
   onEject,
   variant = "outline",
@@ -183,17 +185,34 @@ export function ModelSelector({
       all.set(model.id, model);
     }
     for (const lora of loraModels) {
+      // Strip "/ suffix" from display name (e.g. "foo_123/foo" → "foo_123")
+      const displayName = lora.name.includes("/")
+        ? lora.name.split("/")[0].trim()
+        : lora.name;
+      // Show type tag instead of base model name
+      const isExported = lora.source === "exported";
+      const isMerged = lora.exportType === "merged";
+      const tag = isExported
+        ? isMerged ? "Merged · Exported" : "LoRA"
+        : "LoRA";
       all.set(lora.id, {
         ...lora,
-        description: lora.baseModel || lora.description,
+        name: displayName,
+        description: tag,
       });
     }
     return all;
   }, [loraModels, models]);
 
-  const currentModel = selected
-    ? optionById.get(selected) ?? { id: selected, name: selected }
-    : undefined;
+  const currentModel = useMemo(() => {
+    if (!selected) return undefined;
+    const found = optionById.get(selected);
+    if (activeGgufVariant) {
+      const desc = `GGUF · ${activeGgufVariant}`;
+      return found ? { ...found, description: desc } : { id: selected, name: selected, description: desc };
+    }
+    return found ?? { id: selected, name: selected };
+  }, [selected, optionById, activeGgufVariant]);
 
   function handleSelect(id: string, meta: ModelSelectorChangeMeta) {
     if (onValueChange) {

studio/frontend/src/components/assistant-ui/model-selector/pickers.tsx

@@ -5,6 +5,8 @@ import {
   TooltipContent,
   TooltipTrigger,
 } from "@/components/ui/tooltip";
+import { listGgufVariants } from "@/features/chat/api/chat-api";
+import type { GgufVariantDetail } from "@/features/chat/types/api";
 import {
   useDebouncedValue,
   useGpuInfo,
@@ -17,7 +19,7 @@ import type { VramFitStatus } from "@/lib/vram";
 import { checkVramFit, estimateLoadingVram } from "@/lib/vram";
 import { Search01Icon } from "@hugeicons/core-free-icons";
 import { HugeiconsIcon } from "@hugeicons/react";
-import { useMemo, useState, type ReactNode } from "react";
+import { useCallback, useEffect, useMemo, useState, type ReactNode } from "react";
 import type {
   LoraModelOption,
   ModelOption,
@@ -36,6 +38,15 @@ function ListLabel({ children }: { children: ReactNode }) {
   );
 }
 
+/** Format bytes to a human-readable size string. */
+function formatBytes(bytes: number): string {
+  if (bytes === 0) return "0 B";
+  const units = ["B", "KB", "MB", "GB", "TB"];
+  const i = Math.floor(Math.log(bytes) / Math.log(1024));
+  const value = bytes / 1024 ** i;
+  return `${value.toFixed(value < 10 ? 1 : 0)} ${units[i]}`;
+}
+
 function ModelRow({
   label,
   meta,
@@ -44,6 +55,7 @@ function ModelRow({
   vramStatus,
   vramEst,
   gpuGb,
+  tooltipText,
 }: {
   label: string;
   meta?: string;
@@ -52,6 +64,7 @@ function ModelRow({
   vramStatus?: VramFitStatus | null;
   vramEst?: number;
   gpuGb?: number;
+  tooltipText?: ReactNode;
 }) {
   const exceeds = vramStatus === "exceeds";
   const showVramTooltip =
@@ -70,29 +83,26 @@ function ModelRow({
       type="button"
       onClick={onClick}
       className={cn(
-        "flex w-full items-center justify-between gap-2 rounded-md px-2.5 py-1.5 text-left text-sm transition-colors hover:bg-accent",
+        "flex w-full items-center gap-2 rounded-md px-2.5 py-1.5 text-left text-sm transition-colors hover:bg-accent",
         selected && "bg-accent/60",
         exceeds && "opacity-50",
       )}
     >
       <span
         className={cn(
-          "min-w-0 flex-1 truncate",
+          "block min-w-0 flex-1 truncate",
           exceeds && "line-through decoration-muted-foreground/50",
         )}
       >
         {label}
       </span>
-      <span className="flex items-center gap-1.5 shrink-0">
+      <span className="ml-auto flex items-center gap-1.5 shrink-0">
         {vramStatus === "exceeds" && (
           <span className="text-[9px] font-medium text-red-400">OOM</span>
         )}
         {vramStatus === "tight" && (
           <span className="text-[9px] font-medium text-amber-400">TIGHT</span>
         )}
-        {vramStatus === "fits" && (
-          <span className="text-[9px] font-medium text-emerald-500/90">FIT</span>
-        )}
         {meta ? (
           <span className="text-[10px] text-muted-foreground">{meta}</span>
         ) : null}
@@ -111,9 +121,154 @@ function ModelRow({
       </Tooltip>
     );
   }
+
+  if (tooltipText) {
+    return (
+      <Tooltip>
+        <TooltipTrigger asChild>{content}</TooltipTrigger>
+        <TooltipContent side="left" className="max-w-xs break-all">
+          {tooltipText}
+        </TooltipContent>
+      </Tooltip>
+    );
+  }
   return content;
 }
 
+// ── GGUF Variant Expander ────────────────────────────────────
+
+function GgufVariantExpander({
+  repoId,
+  onSelect,
+  gpuGb,
+}: {
+  repoId: string;
+  onSelect: (id: string, meta: ModelSelectorChangeMeta) => void;
+  gpuGb?: number;
+}) {
+  const [variants, setVariants] = useState<GgufVariantDetail[] | null>(null);
+  const [defaultVariant, setDefaultVariant] = useState<string | null>(null);
+  const [hasVision, setHasVision] = useState(false);
+  const [loading, setLoading] = useState(true);
+  const [error, setError] = useState<string | null>(null);
+
+  useEffect(() => {
+    let canceled = false;
+    setLoading(true);
+    setError(null);
+
+    listGgufVariants(repoId)
+      .then((res) => {
+        if (canceled) return;
+        setVariants(res.variants);
+        setDefaultVariant(res.default_variant);
+        setHasVision(res.has_vision);
+      })
+      .catch((err) => {
+        if (canceled) return;
+        setError(err instanceof Error ? err.message : "Failed to load variants");
+      })
+      .finally(() => {
+        if (!canceled) setLoading(false);
+      });
+
+    return () => {
+      canceled = true;
+    };
+  }, [repoId]);
+
+  const handleVariantClick = useCallback(
+    (quant: string) => {
+      onSelect(repoId, {
+        source: "hub",
+        isLora: false,
+        ggufVariant: quant,
+      });
+    },
+    [repoId, onSelect],
+  );
+
+  if (loading) {
+    return (
+      <div className="flex items-center gap-2 px-5 py-2">
+        <Spinner className="size-3 text-muted-foreground" />
+        <span className="text-xs text-muted-foreground">Loading variants…</span>
+      </div>
+    );
+  }
+
+  if (error) {
+    return (
+      <div className="px-5 py-2 text-xs text-destructive">{error}</div>
+    );
+  }
+
+  if (!variants || variants.length === 0) {
+    return (
+      <div className="px-5 py-2 text-xs text-muted-foreground">
+        No GGUF variants found.
+      </div>
+    );
+  }
+
+  return (
+    <div className="pl-4 border-l-2 border-accent/50 ml-3 my-1">
+      <div className="px-2 py-1 flex items-center gap-1.5">
+        <span className="text-[10px] font-semibold uppercase tracking-wider text-muted-foreground">
+          Quantizations
+        </span>
+        {hasVision && (
+          <span className="text-[9px] font-medium text-blue-400">Vision</span>
+        )}
+      </div>
+      {variants.map((v) => {
+        const sizeGb = v.size_bytes / (1024 ** 3);
+        const fitStatus = gpuGb != null && gpuGb > 0 && sizeGb > 0
+          ? checkVramFit(sizeGb, gpuGb)
+          : null;
+        return (
+          <button
+            key={v.filename}
+            type="button"
+            onClick={() => handleVariantClick(v.quant)}
+            className={cn(
+              "flex w-full items-center justify-between gap-2 rounded-md px-2.5 py-1 text-left text-sm transition-colors hover:bg-accent",
+            )}
+          >
+            <span className="min-w-0 flex-1 truncate font-mono text-xs">
+              {v.quant}
+              {v.quant === defaultVariant && (
+                <span className="ml-1.5 text-[9px] font-sans font-medium text-primary/70">
+                  recommended
+                </span>
+              )}
+            </span>
+            <span className="flex items-center gap-1.5 shrink-0">
+              {fitStatus === "exceeds" && (
+                <span className="text-[9px] font-medium text-red-400">OOM</span>
+              )}
+              {fitStatus === "tight" && (
+                <span className="text-[9px] font-medium text-amber-400">TIGHT</span>
+              )}
+              <span className="text-[10px] text-muted-foreground">
+                {formatBytes(v.size_bytes)}
+              </span>
+            </span>
+          </button>
+        );
+      })}
+    </div>
+  );
+}
+
+// ── Detect GGUF repos by naming convention ────────────────────
+
+function isGgufRepo(id: string): boolean {
+  return id.toUpperCase().includes("-GGUF");
+}
+
+// ── Hub Model Picker ──────────────────────────────────────────
+
 export function HubModelPicker({
   models,
   value,
@@ -130,6 +285,9 @@ export function HubModelPicker({
     debouncedQuery,
   );
 
+  // Track which GGUF repo is expanded for variant selection
+  const [expandedGguf, setExpandedGguf] = useState<string | null>(null);
+
   const recommendedIds = useMemo(
     () => dedupe([...models.map((model) => model.id), value ?? ""]),
     [models, value],
@@ -199,6 +357,19 @@ export function HubModelPicker({
 
   const { scrollRef, sentinelRef } = useInfiniteScroll(fetchMore, results.length);
 
+  /** Handle clicking a model row — GGUF repos expand, others load directly. */
+  const handleModelClick = useCallback(
+    (id: string) => {
+      if (isGgufRepo(id)) {
+        // Toggle GGUF variant expander
+        setExpandedGguf((prev) => (prev === id ? null : id));
+      } else {
+        onSelect(id, { source: "hub", isLora: false });
+      }
+    },
+    [onSelect],
+  );
+
   return (
     <div className="space-y-2">
       <div className="relative">
@@ -230,18 +401,24 @@ export function HubModelPicker({
                 recommendedIds.map((id) => {
                   const vram = recommendedVramMap.get(id);
                   return (
-                    <ModelRow
-                      key={id}
-                      label={id}
-                      meta={vram?.detail ?? undefined}
-                      selected={value === id}
-                      onClick={() =>
-                        onSelect(id, { source: "hub", isLora: false })
-                      }
-                      vramStatus={vram?.status ?? null}
-                      vramEst={vram?.est}
-                      gpuGb={gpu.available ? gpu.memoryTotalGb : undefined}
-                    />
+                    <div key={id}>
+                      <ModelRow
+                        label={id}
+                        meta={
+                          isGgufRepo(id)
+                            ? "GGUF"
+                            : vram?.detail ?? undefined
+                        }
+                        selected={value === id}
+                        onClick={() => handleModelClick(id)}
+                        vramStatus={isGgufRepo(id) ? null : vram?.status ?? null}
+                        vramEst={isGgufRepo(id) ? undefined : vram?.est}
+                        gpuGb={gpu.available ? gpu.memoryTotalGb : undefined}
+                      />
+                      {expandedGguf === id && (
+                        <GgufVariantExpander repoId={id} onSelect={onSelect} gpuGb={gpu.available ? gpu.memoryTotalGb : undefined} />
+                      )}
+                    </div>
                   );
                 })
               )}
@@ -259,18 +436,24 @@ export function HubModelPicker({
                 hfIds.map((id) => {
                   const vram = vramMap.get(id);
                   return (
-                    <ModelRow
-                      key={id}
-                      label={id}
-                      meta={metricsById.get(id)}
-                      selected={value === id}
-                      onClick={() =>
-                        onSelect(id, { source: "hub", isLora: false })
-                      }
-                      vramStatus={vram?.status ?? null}
-                      vramEst={vram?.est}
-                      gpuGb={gpu.available ? gpu.memoryTotalGb : undefined}
-                    />
+                    <div key={id}>
+                      <ModelRow
+                        label={id}
+                        meta={
+                          isGgufRepo(id)
+                            ? "GGUF"
+                            : metricsById.get(id)
+                        }
+                        selected={value === id}
+                        onClick={() => handleModelClick(id)}
+                        vramStatus={isGgufRepo(id) ? null : vram?.status ?? null}
+                        vramEst={isGgufRepo(id) ? undefined : vram?.est}
+                        gpuGb={gpu.available ? gpu.memoryTotalGb : undefined}
+                      />
+                      {expandedGguf === id && (
+                        <GgufVariantExpander repoId={id} onSelect={onSelect} gpuGb={gpu.available ? gpu.memoryTotalGb : undefined} />
+                      )}
+                    </div>
                   );
                 })
               )}
@@ -365,15 +548,37 @@ export function LoraModelPicker({
               <div key={baseModel}>
                 {index > 0 ? <div className="my-1" /> : null}
                 <ListLabel>{baseModel}</ListLabel>
-                {adapters.map((adapter) => (
-                  <ModelRow
-                    key={adapter.id}
-                    label={adapter.name}
-                    meta="LoRA"
-                    selected={value === adapter.id}
-                    onClick={() => onSelect(adapter.id, { source: "lora", isLora: true })}
-                  />
-                ))}
+                {adapters.map((adapter) => {
+                  const isExported = adapter.source === "exported";
+                  const isMerged = adapter.exportType === "merged";
+                  const isGguf = adapter.exportType === "gguf";
+                  const tag = isGguf
+                    ? "GGUF"
+                    : isExported
+                      ? isMerged ? "Merged" : "LoRA"
+                      : "LoRA";
+                  const meta = isExported ? `${tag} · Exported` : tag;
+                  return (
+                    <ModelRow
+                      key={adapter.id}
+                      label={adapter.name}
+                      meta={meta}
+                      selected={value === adapter.id}
+                      onClick={() => onSelect(adapter.id, {
+                        source: isExported ? "exported" : "lora",
+                        isLora: !isMerged && !isGguf,
+                      })}
+                      tooltipText={
+                        <>
+                          <span className="block break-words">{adapter.name}</span>
+                          <span className="block mt-1 text-[10px] text-muted-foreground break-all">
+                            {adapter.id}
+                          </span>
+                        </>
+                      }
+                    />
+                  );
+                })}
               </div>
             ))
           )}
@@ -382,4 +587,3 @@ export function LoraModelPicker({
     </div>
   );
 }
-

studio/frontend/src/components/assistant-ui/model-selector/types.ts

@@ -10,10 +10,13 @@ export interface ModelOption {
 export interface LoraModelOption extends ModelOption {
   baseModel?: string;
   updatedAt?: number;
+  source?: "training" | "exported";
+  exportType?: "lora" | "merged" | "gguf";
 }
 
 export interface ModelSelectorChangeMeta {
-  source: "hub" | "lora";
+  source: "hub" | "lora" | "exported";
   isLora: boolean;
+  ggufVariant?: string;
 }
 

studio/frontend/src/components/assistant-ui/thread.tsx

@@ -70,7 +70,7 @@ export const Thread: FC<{ hideComposer?: boolean; hideWelcome?: boolean }> = ({
           }}
         />
 
-        <ThreadPrimitive.ViewportFooter className="aui-thread-viewport-footer sticky bottom-0 mt-auto flex w-full flex-col gap-4 overflow-visible bg-background pb-4 md:pb-4 before:pointer-events-none before:absolute before:inset-x-0 before:bottom-full before:h-20 before:bg-gradient-to-t before:from-background before:to-transparent">
+        <ThreadPrimitive.ViewportFooter className="aui-thread-viewport-footer sticky bottom-0 mt-auto flex w-full flex-col gap-4 overflow-visible bg-background pb-4 md:pb-4">
           <ThreadScrollToBottom />
           <AuiIf condition={({ thread }) => !thread.isEmpty}>
             {!hideComposer && <ComposerAnimated />}

studio/frontend/src/components/markdown/markdown-preview.tsx

@@ -0,0 +1,46 @@
+import { cn } from "@/lib/utils";
+import { code } from "@streamdown/code";
+import { math } from "@streamdown/math";
+import { mermaid } from "@streamdown/mermaid";
+import { memo, type ReactElement } from "react";
+import { Streamdown } from "streamdown";
+import "katex/dist/katex.min.css";
+
+const MARKDOWN_PLUGINS = { code, math, mermaid } as const;
+
+type MarkdownPreviewProps = {
+  markdown: string;
+  className?: string;
+  plain?: boolean;
+};
+
+function MarkdownPreviewImpl({
+  markdown,
+  className,
+  plain = false,
+}: MarkdownPreviewProps): ReactElement {
+  const markdownClassName =
+    "w-full max-w-none min-w-0 space-y-2 [overflow-wrap:anywhere] [&_*]:max-w-none [&_p]:w-full [&_ul]:w-full [&_ol]:w-full [&_li]:w-full [&_h1]:w-full [&_h2]:w-full [&_h3]:w-full [&_h4]:w-full [&_h5]:w-full [&_h6]:w-full [&_pre]:w-full [&_table]:w-full [&_p]:break-words [&_li]:break-words [&_code]:break-words [&_pre]:whitespace-pre-wrap [&_pre]:break-words";
+
+  return (
+    <div
+      className={cn(
+        plain
+          ? "h-full w-full min-w-0 overflow-auto p-2 text-xs leading-relaxed pointer-events-none select-none"
+          : "nodrag max-h-56 w-full min-w-0 overflow-auto rounded-md border border-border/60 bg-muted/20 p-2 text-xs leading-relaxed",
+        className,
+      )}
+    >
+      <Streamdown
+        mode="static"
+        plugins={MARKDOWN_PLUGINS}
+        controls={false}
+        className={markdownClassName}
+      >
+        {markdown.trim() ? markdown : "_Empty note_"}
+      </Streamdown>
+    </div>
+  );
+}
+
+export const MarkdownPreview = memo(MarkdownPreviewImpl);

studio/frontend/src/components/markdown/mermaid-error.tsx

@@ -0,0 +1,28 @@
+import type { MermaidErrorComponentProps } from "streamdown";
+
+function hasSlashComment(chart: string): boolean {
+  return /(^|[^:])\/\/.*/m.test(chart);
+}
+
+export function MermaidError({
+  error,
+  chart,
+  retry,
+}: MermaidErrorComponentProps) {
+  return (
+    <div className="my-4 rounded-lg border border-red-300 bg-red-50 p-3 text-red-800">
+      <p className="text-sm font-semibold">Mermaid render failed</p>
+      <p className="mt-1 break-words font-mono text-xs">{error}</p>
+      {hasSlashComment(chart) ? (
+        <p className="mt-1 text-xs">Hint: Mermaid comments use `%%`, not `//`.</p>
+      ) : null}
+      <button
+        type="button"
+        onClick={retry}
+        className="mt-2 rounded border border-red-300 px-2 py-1 text-xs hover:bg-red-100"
+      >
+        Retry
+      </button>
+    </div>
+  );
+}

studio/frontend/src/components/navbar.tsx

@@ -3,6 +3,7 @@ import {
   HoverCardContent,
   HoverCardTrigger,
 } from "@/components/ui/hover-card";
+import { AnimatedThemeToggler } from "@/components/ui/animated-theme-toggler";
 import {
   Sheet,
   SheetContent,
@@ -13,9 +14,9 @@ import {
 import { cn } from "@/lib/utils";
 import {
   AiChat02Icon,
-  Analytics01Icon,
   ArrowRight01Icon,
   Book03Icon,
+  ChefHatIcon,
   CursorInfo02Icon,
   PackageIcon,
   ZapIcon,
@@ -23,31 +24,30 @@ import {
 import { HugeiconsIcon } from "@hugeicons/react";
 import { useTrainingRuntimeStore } from "@/features/training";
 import { Link, useRouterState } from "@tanstack/react-router";
-import { AnimatePresence, motion } from "motion/react";
+import { motion } from "motion/react";
 import { useState } from "react";
 import { TOUR_OPEN_EVENT } from "@/features/tour";
 
 const NAV_ITEMS = [
   { label: "Studio", href: "/studio", icon: ZapIcon, enabled: true },
-  { label: "Evaluate", href: "/evaluate", icon: Analytics01Icon, enabled: false },
+  { label: "Recipes", href: "/data-recipes", icon: ChefHatIcon, enabled: true },
   { label: "Export", href: "/export", icon: PackageIcon, enabled: true },
   { label: "Chat", href: "/chat", icon: AiChat02Icon, enabled: true },
 ];
 
+function getTourId(pathname: string): "studio" | "chat" | "export" | null {
+  if (pathname === "/studio") return "studio";
+  if (pathname === "/chat") return "chat";
+  if (pathname === "/export") return "export";
+  return null;
+}
+
 export function Navbar() {
   const pathname = useRouterState({ select: (s) => s.location.pathname });
   const isTrainingRunning = useTrainingRuntimeStore((s) => s.isTrainingRunning);
-  const [logoHovered, setLogoHovered] = useState(false);
   const [mobileOpen, setMobileOpen] = useState(false);
 
-  const tourId =
-    pathname === "/studio"
-      ? "studio"
-      : pathname === "/chat"
-        ? "chat"
-        : pathname === "/export"
-          ? "export"
-          : null;
+  const tourId = getTourId(pathname);
 
   const openTour = () => {
     if (!tourId) return;
@@ -58,37 +58,20 @@ export function Navbar() {
 
   return (
     <header className="relative top-0 z-40 h-16 w-full">
-      <div className="mx-auto flex h-full max-w-7xl items-center justify-between px-4 sm:px-6">
+      <div className="mx-auto grid h-full max-w-7xl grid-cols-[1fr_auto_1fr] items-center px-4 sm:px-6">
         {/* Left: logo */}
-        <div
-          className="relative flex items-center gap-2.5 cursor-pointer select-none"
-          onMouseEnter={() => setLogoHovered(true)}
-          onMouseLeave={() => setLogoHovered(false)}
-        >
-          <motion.img
-            src="https://unsloth.ai/cgi/image/unsloth_sticker_no_shadow_ldN4V4iydw00qSIIWDCUv.png?width=96&quality=80&format=auto"
-            alt="unsloth"
-            className="size-10"
-            animate={{ rotate: logoHovered ? 360 : 0 }}
-            transition={{ duration: 0.5, ease: [0.165, 0.84, 0.44, 1] }}
+        <Link to="/studio" className="flex items-center justify-self-start select-none">
+          <img
+            src="/blacklogo.png"
+            alt="Unsloth"
+            className="h-9 w-auto dark:hidden"
           />
-          <span className="text-xl font-bold tracking-wide font-heading sm:text-2xl">
-            unsloth
-          </span>
-          <AnimatePresence>
-            {logoHovered && (
-              <motion.img
-                src="/Sloth emojis/large sloth wave.png"
-                alt="hi!"
-                className="absolute -bottom-10 left-1 size-10 pointer-events-none"
-                initial={{ y: 10, opacity: 0 }}
-                animate={{ y: 0, opacity: 1 }}
-                exit={{ y: 10, opacity: 0 }}
-                transition={{ duration: 0.25, ease: [0.215, 0.61, 0.355, 1] }}
-              />
-            )}
-          </AnimatePresence>
-        </div>
+          <img
+            src="/whitelogo.png"
+            alt="Unsloth"
+            className="hidden h-9 w-auto dark:block"
+          />
+        </Link>
 
         {/* Center: pill nav */}
         <nav
@@ -96,7 +79,8 @@ export function Navbar() {
           className="hidden items-center rounded-full border border-border bg-card p-1 ring-1 ring-foreground/5 md:flex"
         >
           {NAV_ITEMS.map((item) => {
-            const active = pathname === item.href;
+            const active =
+              pathname === item.href || pathname.startsWith(`${item.href}/`);
             const disabledByTraining =
               isTrainingRunning && item.href !== "/studio";
             if (!item.enabled || disabledByTraining) {
@@ -133,23 +117,21 @@ export function Navbar() {
                   />
                 )}
                 <span className="relative z-10 flex items-center gap-1.5">
-                  <AnimatePresence mode="popLayout">
-                    {active && item.icon && (
-                      <motion.span
-                        key={item.href}
-                        initial={{ width: 0, opacity: 0 }}
-                        animate={{ width: "auto", opacity: 1 }}
-                        exit={{ width: 0, opacity: 0 }}
-                        transition={{ duration: 0.2, ease: [0.165, 0.84, 0.44, 1] }}
-                        className="overflow-hidden"
-                      >
-                        <HugeiconsIcon
-                          icon={item.icon}
-                          className="size-3.5 -mt-px fill-current"
-                        />
-                      </motion.span>
-                    )}
-                  </AnimatePresence>
+                  <span className="inline-flex size-3.5 items-center justify-center overflow-hidden">
+                    <motion.span
+                      initial={false}
+                      animate={{
+                        opacity: active ? 1 : 0,
+                        scale: active ? 1 : 0.9,
+                      }}
+                      transition={{ duration: 0.2, ease: [0.165, 0.84, 0.44, 1] }}
+                    >
+                      <HugeiconsIcon
+                        icon={item.icon}
+                        className="size-3.5 -mt-px"
+                      />
+                    </motion.span>
+                  </span>
                   {item.label}
                 </span>
               </Link>
@@ -158,7 +140,12 @@ export function Navbar() {
         </nav>
 
         {/* Right: docs/tour desktop */}
-        <div className="hidden items-center gap-2 md:flex">
+        <div className="hidden items-center justify-self-end gap-2 md:flex">
+          <AnimatedThemeToggler
+            className="flex h-9 w-9 items-center justify-center rounded-md text-muted-foreground transition-colors hover:bg-accent hover:text-foreground [&_svg]:size-4"
+            title="Toggle theme"
+            aria-label="Toggle theme"
+          />
           <HoverCard openDelay={200} closeDelay={100}>
             <HoverCardTrigger asChild={true}>
               <a
@@ -193,16 +180,20 @@ export function Navbar() {
             </HoverCardContent>
           </HoverCard>
 
-          {tourId ? (
-            <button
-              type="button"
-              onClick={openTour}
-              className="flex h-9 w-9 items-center justify-center rounded-md text-muted-foreground transition-colors hover:bg-accent hover:text-foreground"
-              title="Tour"
-            >
-              <HugeiconsIcon icon={CursorInfo02Icon} className="size-4" />
-            </button>
-          ) : null}
+          <button
+            type="button"
+            onClick={tourId ? openTour : undefined}
+            className={cn(
+              "flex h-9 items-center gap-1.5 rounded-md px-3 text-muted-foreground transition-colors hover:bg-accent hover:text-foreground",
+              !tourId && "invisible pointer-events-none",
+            )}
+            title="Tour"
+            aria-hidden={!tourId}
+            tabIndex={tourId ? 0 : -1}
+          >
+            <HugeiconsIcon icon={CursorInfo02Icon} className="size-4" />
+            <span className="text-sm font-medium">Tour</span>
+          </button>
         </div>
 
         {/* Right: mobile */}

studio/frontend/src/components/section-card.tsx

@@ -52,7 +52,7 @@ export function SectionCard({
   return (
     <div
       className={cn(
-        "bg-card corner-squircle rounded-3xl ring-1 ring-foreground/10 flex flex-col gap-5 p-5 relative transition-all duration-300 ease-in-out",
+        "bg-card corner-squircle rounded-3xl ring-1 ring-foreground/10 flex flex-col gap-5 p-5 relative overflow-hidden transition-all duration-300 ease-in-out",
         featured && styles.border,
         className,
       )}

studio/frontend/src/components/ui/animated-theme-toggler.tsx

@@ -0,0 +1,82 @@
+import { useCallback, useEffect, useRef, useState } from "react"
+import { Moon, Sun } from "lucide-react"
+import { flushSync } from "react-dom"
+
+import { cn } from "@/lib/utils"
+
+interface AnimatedThemeTogglerProps extends React.ComponentPropsWithoutRef<"button"> {
+  duration?: number
+}
+
+export const AnimatedThemeToggler = ({
+  className,
+  duration = 400,
+  ...props
+}: AnimatedThemeTogglerProps) => {
+  const [isDark, setIsDark] = useState(false)
+  const buttonRef = useRef<HTMLButtonElement>(null)
+
+  useEffect(() => {
+    const updateTheme = () => {
+      setIsDark(document.documentElement.classList.contains("dark"))
+    }
+
+    updateTheme()
+
+    const observer = new MutationObserver(updateTheme)
+    observer.observe(document.documentElement, {
+      attributes: true,
+      attributeFilter: ["class"],
+    })
+
+    return () => observer.disconnect()
+  }, [])
+
+  const toggleTheme = useCallback(async () => {
+    if (!buttonRef.current) return
+
+    await document.startViewTransition(() => {
+      flushSync(() => {
+        const newTheme = !isDark
+        setIsDark(newTheme)
+        document.documentElement.classList.toggle("dark")
+        localStorage.setItem("theme", newTheme ? "dark" : "light")
+      })
+    }).ready
+
+    const { top, left, width, height } =
+      buttonRef.current.getBoundingClientRect()
+    const x = left + width / 2
+    const y = top + height / 2
+    const maxRadius = Math.hypot(
+      Math.max(left, window.innerWidth - left),
+      Math.max(top, window.innerHeight - top)
+    )
+
+    document.documentElement.animate(
+      {
+        clipPath: [
+          `circle(0px at ${x}px ${y}px)`,
+          `circle(${maxRadius}px at ${x}px ${y}px)`,
+        ],
+      },
+      {
+        duration,
+        easing: "ease-in-out",
+        pseudoElement: "::view-transition-new(root)",
+      }
+    )
+  }, [isDark, duration])
+
+  return (
+    <button
+      ref={buttonRef}
+      onClick={toggleTheme}
+      className={cn(className)}
+      {...props}
+    >
+      {isDark ? <Sun /> : <Moon />}
+      <span className="sr-only">Toggle theme</span>
+    </button>
+  )
+}

studio/frontend/src/components/ui/chart.tsx

@@ -100,30 +100,30 @@ ${colorConfig
   );
 };
 
-const ChartTooltip = RechartsPrimitive.Tooltip;
-
-function ChartTooltipContent({
-  active,
-  payload,
-  className,
+const ChartTooltip = RechartsPrimitive.Tooltip;
+
+function ChartTooltipContent({
+  active,
+  payload,
+  className,
   indicator = "dot",
   hideLabel = false,
   hideIndicator = false,
   label,
   labelFormatter,
   labelClassName,
-  formatter,
-  color,
-  nameKey,
-  labelKey,
-}: React.ComponentProps<typeof RechartsPrimitive.Tooltip> &
-  React.ComponentProps<"div"> & {
-    hideLabel?: boolean;
-    hideIndicator?: boolean;
-    indicator?: "line" | "dot" | "dashed";
-    nameKey?: string;
-    labelKey?: string;
-  }) {
+  formatter,
+  color,
+  nameKey,
+  labelKey,
+}: Partial<RechartsPrimitive.TooltipContentProps<any, any>> &
+  React.ComponentProps<"div"> & {
+    hideLabel?: boolean;
+    hideIndicator?: boolean;
+    indicator?: "line" | "dot" | "dashed";
+    nameKey?: string;
+    labelKey?: string;
+  }) {
   const { config } = useChart();
 
   const tooltipLabel = React.useMemo(() => {
@@ -248,20 +248,20 @@ function ChartTooltipContent({
   );
 }
 
-const ChartLegend = RechartsPrimitive.Legend;
-
-function ChartLegendContent({
-  className,
-  hideIcon = false,
-  payload,
-  verticalAlign = "bottom",
-  nameKey,
-}: React.ComponentProps<"div"> &
-  Pick<RechartsPrimitive.LegendProps, "payload" | "verticalAlign"> & {
-    hideIcon?: boolean;
-    nameKey?: string;
-  }) {
-  const { config } = useChart();
+const ChartLegend = RechartsPrimitive.Legend;
+
+function ChartLegendContent({
+  className,
+  hideIcon = false,
+  payload,
+  verticalAlign = "bottom",
+  nameKey,
+}: React.ComponentProps<"div"> &
+  Pick<RechartsPrimitive.DefaultLegendContentProps, "payload" | "verticalAlign"> & {
+    hideIcon?: boolean;
+    nameKey?: string;
+  }) {
+  const { config } = useChart();
 
   if (!payload?.length) {
     return null;

studio/frontend/src/components/ui/combobox.tsx

@@ -7,6 +7,7 @@ import * as React from "react";
 import { createContext, useContext, useState } from "react";
 
 import { Button } from "@/components/ui/button";
+import { useDialogPortalContainer } from "@/components/ui/dialog";
 import {
   InputGroup,
   InputGroupAddon,
@@ -139,38 +140,42 @@ function ComboboxInput({
   );
 }
 
-function ComboboxContent({
-  className,
-  side = "bottom",
-  sideOffset = 6,
+function ComboboxContent({
+  className,
+  side = "bottom",
+  sideOffset = 6,
   align = "start",
   alignOffset = 0,
   anchor,
+  container,
   ...props
 }: ComboboxPrimitive.Popup.Props &
   Pick<
     ComboboxPrimitive.Positioner.Props,
     "side" | "align" | "sideOffset" | "alignOffset" | "anchor"
-  >): React.ReactElement {
+  > & {
+    container?: HTMLElement | null;
+  }): React.ReactElement {
+  const dialogContainer = useDialogPortalContainer();
   return (
-    <ComboboxPrimitive.Portal>
-      <ComboboxPrimitive.Positioner
-        side={side}
-        sideOffset={sideOffset}
-        align={align}
-        alignOffset={alignOffset}
-        anchor={anchor}
-        className="isolate z-50"
-      >
-        <ComboboxPrimitive.Popup
-          data-slot="combobox-content"
-          data-chips={!!anchor}
-          className={cn(
-            "bg-popover text-popover-foreground data-open:animate-in data-closed:animate-out data-closed:fade-out-0 data-open:fade-in-0 data-closed:zoom-out-95 data-open:zoom-in-95 data-[side=bottom]:slide-in-from-top-2 data-[side=left]:slide-in-from-right-2 data-[side=right]:slide-in-from-left-2 data-[side=top]:slide-in-from-bottom-2 shadow-border ring-1 ring-border *:data-[slot=input-group]:bg-input/30 max-h-72 min-w-36 overflow-hidden rounded-xl corner-squircle duration-100 *:data-[slot=input-group]:m-1 *:data-[slot=input-group]:mb-0 *:data-[slot=input-group]:h-9 *:data-[slot=input-group]:border-none *:data-[slot=input-group]:shadow-none group/combobox-content relative max-h-(--available-height) w-(--anchor-width) max-w-(--available-width) min-w-[calc(var(--anchor-width)+--spacing(7))] origin-(--transform-origin) data-[chips=true]:min-w-(--anchor-width)",
-            className,
-          )}
-          {...props}
-        />
+    <ComboboxPrimitive.Portal container={container ?? dialogContainer ?? undefined}>
+      <ComboboxPrimitive.Positioner
+        side={side}
+        sideOffset={sideOffset}
+        align={align}
+        alignOffset={alignOffset}
+        anchor={anchor}
+        className="isolate z-[120] pointer-events-auto"
+      >
+        <ComboboxPrimitive.Popup
+          data-slot="combobox-content"
+          data-chips={!!anchor}
+          className={cn(
+            "bg-popover text-popover-foreground data-open:animate-in data-closed:animate-out data-closed:fade-out-0 data-open:fade-in-0 data-closed:zoom-out-95 data-open:zoom-in-95 data-[side=bottom]:slide-in-from-top-2 data-[side=left]:slide-in-from-right-2 data-[side=right]:slide-in-from-left-2 data-[side=top]:slide-in-from-bottom-2 shadow-border ring-1 ring-border *:data-[slot=input-group]:bg-input/30 max-h-72 min-w-36 overflow-hidden rounded-xl corner-squircle duration-100 *:data-[slot=input-group]:m-1 *:data-[slot=input-group]:mb-0 *:data-[slot=input-group]:h-9 *:data-[slot=input-group]:border-none *:data-[slot=input-group]:shadow-none group/combobox-content relative pointer-events-auto max-h-(--available-height) w-(--anchor-width) max-w-(--available-width) min-w-[calc(var(--anchor-width)+--spacing(7))] origin-(--transform-origin) data-[chips=true]:min-w-(--anchor-width)",
+            className,
+          )}
+          {...props}
+        />
       </ComboboxPrimitive.Positioner>
     </ComboboxPrimitive.Portal>
   );
@@ -201,7 +206,7 @@ function ComboboxItem({
     <ComboboxPrimitive.Item
       data-slot="combobox-item"
       className={cn(
-        "data-highlighted:bg-accent data-highlighted:text-accent-foreground not-data-[variant=destructive]:data-highlighted:**:text-accent-foreground gap-2.5 rounded-xl corner-squircle py-2 pr-8 pl-3 text-sm [&_svg:not([class*='size-'])]:size-4 relative flex w-full cursor-pointer items-center outline-hidden select-none data-[disabled]:pointer-events-none data-[disabled]:opacity-50 [&_svg]:pointer-events-none [&_svg]:shrink-0",
+        "data-highlighted:bg-accent data-highlighted:text-accent-foreground not-data-[variant=destructive]:data-highlighted:**:text-accent-foreground gap-2 rounded-xl corner-squircle py-2 pr-2 pl-3 text-sm [&[aria-selected=true]]:pr-7 [&_svg:not([class*='size-'])]:size-4 relative flex w-full cursor-pointer items-center outline-hidden select-none data-[disabled]:pointer-events-none data-[disabled]:opacity-50 [&_svg]:pointer-events-none [&_svg]:shrink-0",
         className,
       )}
       {...props}

studio/frontend/src/components/ui/data-table.tsx

@@ -22,15 +22,24 @@ interface DataTableProps<TData, TValue> {
   columns: ColumnDef<TData, TValue>[];
   data: TData[];
   className?: string;
+  onRowClick?: (row: TData, rowIndex: number, rowId: string) => void;
+  getRowClassName?: (
+    row: TData,
+    rowIndex: number,
+    rowId: string,
+  ) => string | undefined;
 }
 
 export function DataTable<TData, TValue>({
   columns,
   data,
   className,
+  onRowClick,
+  getRowClassName,
 }: DataTableProps<TData, TValue>) {
   const [sorting, setSorting] = useState<SortingState>([]);
 
+  // eslint-disable-next-line react-hooks/incompatible-library
   const table = useReactTable({
     data,
     columns,
@@ -81,7 +90,9 @@ export function DataTable<TData, TValue>({
                     ? "bg-background"
                     : "bg-muted/20",
                   "hover:bg-primary/[0.03]",
+                  getRowClassName?.(row.original, idx, row.id),
                 )}
+                onClick={() => onRowClick?.(row.original, idx, row.id)}
               >
                 {row.getVisibleCells().map((cell) => (
                   <TableCell

studio/frontend/src/components/ui/dialog.tsx

@@ -1,165 +1,191 @@
-"use client";
-
-import { Dialog as DialogPrimitive } from "radix-ui";
-import type * as React from "react";
-
-import { Button } from "@/components/ui/button";
-import { cn } from "@/lib/utils";
-import { Cancel01Icon } from "@hugeicons/core-free-icons";
-import { HugeiconsIcon } from "@hugeicons/react";
-
-function Dialog({
-  ...props
-}: React.ComponentProps<typeof DialogPrimitive.Root>) {
-  return <DialogPrimitive.Root data-slot="dialog" {...props} />;
-}
-
-function DialogTrigger({
-  ...props
-}: React.ComponentProps<typeof DialogPrimitive.Trigger>) {
-  return <DialogPrimitive.Trigger data-slot="dialog-trigger" {...props} />;
-}
-
-function DialogPortal({
-  ...props
-}: React.ComponentProps<typeof DialogPrimitive.Portal>) {
-  return <DialogPrimitive.Portal data-slot="dialog-portal" {...props} />;
-}
-
-function DialogClose({
-  ...props
-}: React.ComponentProps<typeof DialogPrimitive.Close>) {
-  return <DialogPrimitive.Close data-slot="dialog-close" {...props} />;
-}
-
-function DialogOverlay({
-  className,
-  ...props
-}: React.ComponentProps<typeof DialogPrimitive.Overlay>) {
-  return (
-    <DialogPrimitive.Overlay
-      data-slot="dialog-overlay"
-      className={cn(
-        "data-open:animate-in data-closed:animate-out data-closed:fade-out-0 data-open:fade-in-0 bg-black/80 duration-100 supports-backdrop-filter:backdrop-blur-xs fixed inset-0 isolate z-50",
-        className,
-      )}
-      {...props}
-    />
-  );
-}
-
-function DialogContent({
-  className,
-  children,
-  showCloseButton = true,
-  ...props
-}: React.ComponentProps<typeof DialogPrimitive.Content> & {
-  showCloseButton?: boolean;
-}) {
-  return (
-    <DialogPortal>
-      <DialogOverlay />
-      <DialogPrimitive.Content
-        data-slot="dialog-content"
-        className={cn(
-          "bg-background data-open:animate-in data-closed:animate-out data-closed:fade-out-0 data-open:fade-in-0 data-closed:zoom-out-95 data-open:zoom-in-95 ring-foreground/5 grid max-w-[calc(100%-2rem)] gap-6 rounded-4xl p-6 text-sm ring-1 duration-100 sm:max-w-md fixed top-1/2 left-1/2 z-50 w-full -translate-x-1/2 -translate-y-1/2",
-          className,
-        )}
-        {...props}
-      >
-        {children}
-        {showCloseButton && (
-          <DialogPrimitive.Close data-slot="dialog-close" asChild>
-            <Button
-              variant="ghost"
-              className="absolute top-4 right-4"
-              size="icon-sm"
-            >
-              <HugeiconsIcon icon={Cancel01Icon} strokeWidth={2} />
-              <span className="sr-only">Close</span>
-            </Button>
-          </DialogPrimitive.Close>
-        )}
-      </DialogPrimitive.Content>
-    </DialogPortal>
-  );
-}
-
-function DialogHeader({ className, ...props }: React.ComponentProps<"div">) {
-  return (
-    <div
-      data-slot="dialog-header"
-      className={cn("gap-2 flex flex-col", className)}
-      {...props}
-    />
-  );
-}
-
-function DialogFooter({
-  className,
-  showCloseButton = false,
-  children,
-  ...props
-}: React.ComponentProps<"div"> & {
-  showCloseButton?: boolean;
-}) {
-  return (
-    <div
-      data-slot="dialog-footer"
-      className={cn(
-        "gap-2 flex flex-col-reverse gap-2 sm:flex-row sm:justify-end",
-        className,
-      )}
-      {...props}
-    >
-      {children}
-      {showCloseButton && (
-        <DialogPrimitive.Close asChild>
-          <Button variant="outline">Close</Button>
-        </DialogPrimitive.Close>
-      )}
-    </div>
-  );
-}
-
-function DialogTitle({
-  className,
-  ...props
-}: React.ComponentProps<typeof DialogPrimitive.Title>) {
-  return (
-    <DialogPrimitive.Title
-      data-slot="dialog-title"
-      className={cn("text-base leading-none font-medium", className)}
-      {...props}
-    />
-  );
-}
-
-function DialogDescription({
-  className,
-  ...props
-}: React.ComponentProps<typeof DialogPrimitive.Description>) {
-  return (
-    <DialogPrimitive.Description
-      data-slot="dialog-description"
-      className={cn(
-        "text-muted-foreground *:[a]:hover:text-foreground text-sm *:[a]:underline *:[a]:underline-offset-3",
-        className,
-      )}
-      {...props}
-    />
-  );
-}
-
-export {
-  Dialog,
-  DialogClose,
-  DialogContent,
-  DialogDescription,
-  DialogFooter,
-  DialogHeader,
-  DialogOverlay,
-  DialogPortal,
-  DialogTitle,
-  DialogTrigger,
-};
+"use client";
+
+import { Dialog as DialogPrimitive } from "radix-ui";
+import type * as React from "react";
+import { createContext, useContext } from "react";
+
+import { Button } from "@/components/ui/button";
+import { cn } from "@/lib/utils";
+import { Cancel01Icon } from "@hugeicons/core-free-icons";
+import { HugeiconsIcon } from "@hugeicons/react";
+
+const DialogPortalContainerContext = createContext<HTMLElement | null>(null);
+
+export function useDialogPortalContainer(): HTMLElement | null {
+  return useContext(DialogPortalContainerContext);
+}
+
+function Dialog({
+  ...props
+}: React.ComponentProps<typeof DialogPrimitive.Root>) {
+  return <DialogPrimitive.Root data-slot="dialog" {...props} />;
+}
+
+function DialogTrigger({
+  ...props
+}: React.ComponentProps<typeof DialogPrimitive.Trigger>) {
+  return <DialogPrimitive.Trigger data-slot="dialog-trigger" {...props} />;
+}
+
+function DialogPortal({
+  ...props
+}: React.ComponentProps<typeof DialogPrimitive.Portal>) {
+  return <DialogPrimitive.Portal data-slot="dialog-portal" {...props} />;
+}
+
+function DialogClose({
+  ...props
+}: React.ComponentProps<typeof DialogPrimitive.Close>) {
+  return <DialogPrimitive.Close data-slot="dialog-close" {...props} />;
+}
+
+function DialogOverlay({
+  className,
+  position = "fixed",
+  ...props
+}: React.ComponentProps<typeof DialogPrimitive.Overlay> & {
+  position?: "fixed" | "absolute";
+}) {
+  return (
+    <DialogPrimitive.Overlay
+      data-slot="dialog-overlay"
+      className={cn(
+        "data-open:animate-in data-closed:animate-out data-closed:fade-out-0 data-open:fade-in-0 bg-black/80 duration-100  inset-0 isolate z-50",
+        position === "fixed" ? "fixed" : "absolute",
+        className,
+      )}
+      {...props}
+    />
+  );
+}
+
+function DialogContent({
+  className,
+  children,
+  showCloseButton = true,
+  container,
+  position = "fixed",
+  overlayClassName,
+  overlayPosition,
+  ...props
+}: React.ComponentProps<typeof DialogPrimitive.Content> & {
+  showCloseButton?: boolean;
+  container?: HTMLElement | null;
+  position?: "fixed" | "absolute";
+  overlayClassName?: string;
+  overlayPosition?: "fixed" | "absolute";
+}) {
+  const resolvedContainer = container ?? null;
+  return (
+    <DialogPortalContainerContext.Provider value={resolvedContainer}>
+      <DialogPortal container={resolvedContainer ?? undefined}>
+        <DialogOverlay
+          className={overlayClassName}
+          position={overlayPosition ?? position}
+        />
+        <DialogPrimitive.Content
+          data-slot="dialog-content"
+          className={cn(
+            "bg-background data-open:animate-in data-closed:animate-out data-closed:fade-out-0 data-open:fade-in-0 data-closed:zoom-out-95 data-open:zoom-in-95 ring-foreground/5 grid max-w-[calc(100%-2rem)] gap-6 rounded-4xl p-6 text-sm ring-1 duration-100 sm:max-w-md top-1/2 left-1/2 z-50 w-full -translate-x-1/2 -translate-y-1/2",
+            position === "fixed" ? "fixed" : "absolute",
+            className,
+          )}
+          {...props}
+        >
+          {children}
+          {showCloseButton && (
+            <DialogPrimitive.Close data-slot="dialog-close" asChild>
+              <Button
+                variant="ghost"
+                className="absolute top-4 right-4"
+                size="icon-sm"
+              >
+                <HugeiconsIcon icon={Cancel01Icon} strokeWidth={2} />
+                <span className="sr-only">Close</span>
+              </Button>
+            </DialogPrimitive.Close>
+          )}
+        </DialogPrimitive.Content>
+      </DialogPortal>
+    </DialogPortalContainerContext.Provider>
+  );
+}
+
+function DialogHeader({ className, ...props }: React.ComponentProps<"div">) {
+  return (
+    <div
+      data-slot="dialog-header"
+      className={cn("gap-2 flex flex-col", className)}
+      {...props}
+    />
+  );
+}
+
+function DialogFooter({
+  className,
+  showCloseButton = false,
+  children,
+  ...props
+}: React.ComponentProps<"div"> & {
+  showCloseButton?: boolean;
+}) {
+  return (
+    <div
+      data-slot="dialog-footer"
+      className={cn(
+        "flex flex-col-reverse gap-2 sm:flex-row sm:justify-end",
+        className,
+      )}
+      {...props}
+    >
+      {children}
+      {showCloseButton && (
+        <DialogPrimitive.Close asChild>
+          <Button variant="outline">Close</Button>
+        </DialogPrimitive.Close>
+      )}
+    </div>
+  );
+}
+
+function DialogTitle({
+  className,
+  ...props
+}: React.ComponentProps<typeof DialogPrimitive.Title>) {
+  return (
+    <DialogPrimitive.Title
+      data-slot="dialog-title"
+      className={cn("text-base leading-none font-medium", className)}
+      {...props}
+    />
+  );
+}
+
+function DialogDescription({
+  className,
+  ...props
+}: React.ComponentProps<typeof DialogPrimitive.Description>) {
+  return (
+    <DialogPrimitive.Description
+      data-slot="dialog-description"
+      className={cn(
+        "text-muted-foreground *:[a]:hover:text-foreground text-sm *:[a]:underline *:[a]:underline-offset-3",
+        className,
+      )}
+      {...props}
+    />
+  );
+}
+
+export {
+  Dialog,
+  DialogClose,
+  DialogContent,
+  DialogDescription,
+  DialogFooter,
+  DialogHeader,
+  DialogOverlay,
+  DialogPortal,
+  DialogTitle,
+  DialogTrigger,
+};

studio/frontend/src/components/ui/empty.tsx

@@ -0,0 +1,104 @@
+import { cva, type VariantProps } from "class-variance-authority"
+
+import { cn } from "@/lib/utils"
+
+function Empty({ className, ...props }: React.ComponentProps<"div">) {
+  return (
+    <div
+      data-slot="empty"
+      className={cn(
+        "gap-4 rounded-lg border-dashed p-12 flex w-full min-w-0 flex-1 flex-col items-center justify-center text-center text-balance",
+        className
+      )}
+      {...props}
+    />
+  )
+}
+
+function EmptyHeader({ className, ...props }: React.ComponentProps<"div">) {
+  return (
+    <div
+      data-slot="empty-header"
+      className={cn(
+        "gap-2 flex max-w-sm flex-col items-center",
+        className
+      )}
+      {...props}
+    />
+  )
+}
+
+const emptyMediaVariants = cva(
+  "mb-2 flex shrink-0 items-center justify-center [&_svg]:pointer-events-none [&_svg]:shrink-0",
+  {
+    variants: {
+      variant: {
+        default: "bg-transparent",
+        icon: "bg-muted text-foreground flex size-10 shrink-0 items-center justify-center rounded-lg [&_svg:not([class*='size-'])]:size-6",
+      },
+    },
+    defaultVariants: {
+      variant: "default",
+    },
+  }
+)
+
+function EmptyMedia({
+  className,
+  variant = "default",
+  ...props
+}: React.ComponentProps<"div"> & VariantProps<typeof emptyMediaVariants>) {
+  return (
+    <div
+      data-slot="empty-icon"
+      data-variant={variant}
+      className={cn(emptyMediaVariants({ variant, className }))}
+      {...props}
+    />
+  )
+}
+
+function EmptyTitle({ className, ...props }: React.ComponentProps<"div">) {
+  return (
+    <div
+      data-slot="empty-title"
+      className={cn("text-lg font-medium tracking-tight", className)}
+      {...props}
+    />
+  )
+}
+
+function EmptyDescription({ className, ...props }: React.ComponentProps<"p">) {
+  return (
+    <div
+      data-slot="empty-description"
+      className={cn(
+        "text-sm/relaxed text-muted-foreground [&>a:hover]:text-primary [&>a]:underline [&>a]:underline-offset-4",
+        className
+      )}
+      {...props}
+    />
+  )
+}
+
+function EmptyContent({ className, ...props }: React.ComponentProps<"div">) {
+  return (
+    <div
+      data-slot="empty-content"
+      className={cn(
+        "gap-4 text-sm flex w-full max-w-sm min-w-0 flex-col items-center text-balance",
+        className
+      )}
+      {...props}
+    />
+  )
+}
+
+export {
+  Empty,
+  EmptyHeader,
+  EmptyTitle,
+  EmptyDescription,
+  EmptyContent,
+  EmptyMedia,
+}

studio/frontend/src/components/ui/select.tsx

@@ -5,6 +5,7 @@ import type * as React from "react";
 import { createContext, useContext, useState } from "react";
 
 import { cn } from "@/lib/utils";
+import { useDialogPortalContainer } from "@/components/ui/dialog";
 import {
   ArrowDown01Icon,
   ArrowUp01Icon,
@@ -96,10 +97,14 @@ function SelectContent({
   children,
   position = "item-aligned",
   align = "center",
+  container,
   ...props
-}: React.ComponentProps<typeof SelectPrimitive.Content>) {
+}: React.ComponentProps<typeof SelectPrimitive.Content> & {
+  container?: HTMLElement | null;
+}) {
+  const dialogContainer = useDialogPortalContainer();
   return (
-    <SelectPrimitive.Portal>
+    <SelectPrimitive.Portal container={container ?? dialogContainer ?? undefined}>
       <SelectPrimitive.Content
         data-slot="select-content"
         data-align-trigger={position === "item-aligned"}

studio/frontend/src/components/ui/sheet.tsx

@@ -1,146 +1,158 @@
-import { Dialog as SheetPrimitive } from "radix-ui";
-import type * as React from "react";
-
-import { Button } from "@/components/ui/button";
-import { cn } from "@/lib/utils";
-import { Cancel01Icon } from "@hugeicons/core-free-icons";
-import { HugeiconsIcon } from "@hugeicons/react";
-
-function Sheet({ ...props }: React.ComponentProps<typeof SheetPrimitive.Root>) {
-  return <SheetPrimitive.Root data-slot="sheet" {...props} />;
-}
-
-function SheetTrigger({
-  ...props
-}: React.ComponentProps<typeof SheetPrimitive.Trigger>) {
-  return <SheetPrimitive.Trigger data-slot="sheet-trigger" {...props} />;
-}
-
-function SheetClose({
-  ...props
-}: React.ComponentProps<typeof SheetPrimitive.Close>) {
-  return <SheetPrimitive.Close data-slot="sheet-close" {...props} />;
-}
-
-function SheetPortal({
-  ...props
-}: React.ComponentProps<typeof SheetPrimitive.Portal>) {
-  return <SheetPrimitive.Portal data-slot="sheet-portal" {...props} />;
-}
-
-function SheetOverlay({
-  className,
-  ...props
-}: React.ComponentProps<typeof SheetPrimitive.Overlay>) {
-  return (
-    <SheetPrimitive.Overlay
-      data-slot="sheet-overlay"
-      className={cn(
-        "data-open:animate-in data-closed:animate-out data-closed:fade-out-0 data-open:fade-in-0 bg-black/80 duration-100 data-ending-style:opacity-0 data-starting-style:opacity-0 supports-backdrop-filter:backdrop-blur-xs fixed inset-0 z-50",
-        className,
-      )}
-      {...props}
-    />
-  );
-}
-
-function SheetContent({
-  className,
-  children,
-  side = "right",
-  showCloseButton = true,
-  overlayClassName,
-  container,
-  ...props
-}: React.ComponentProps<typeof SheetPrimitive.Content> & {
-  side?: "top" | "right" | "bottom" | "left";
-  showCloseButton?: boolean;
-  overlayClassName?: string;
-  container?: HTMLElement | null;
-}) {
-  return (
-    <SheetPortal container={container ?? undefined}>
-      <SheetOverlay className={overlayClassName} />
-      <SheetPrimitive.Content
-        data-slot="sheet-content"
-        data-side={side}
-        className={cn(
-          "bg-background data-open:animate-in data-closed:animate-out data-[side=right]:data-closed:slide-out-to-right-10 data-[side=right]:data-open:slide-in-from-right-10 data-[side=left]:data-closed:slide-out-to-left-10 data-[side=left]:data-open:slide-in-from-left-10 data-[side=top]:data-closed:slide-out-to-top-10 data-[side=top]:data-open:slide-in-from-top-10 data-closed:fade-out-0 data-open:fade-in-0 data-[side=bottom]:data-closed:slide-out-to-bottom-10 data-[side=bottom]:data-open:slide-in-from-bottom-10 fixed z-50 flex flex-col bg-clip-padding text-sm shadow-lg transition duration-200 ease-in-out data-[side=bottom]:inset-x-0 data-[side=bottom]:bottom-0 data-[side=bottom]:h-auto data-[side=bottom]:border-t data-[side=left]:inset-y-0 data-[side=left]:left-0 data-[side=left]:h-full data-[side=left]:w-3/4 data-[side=left]:border-r data-[side=right]:inset-y-0 data-[side=right]:right-0 data-[side=right]:h-full data-[side=right]:w-3/4 data-[side=right]:border-l data-[side=top]:inset-x-0 data-[side=top]:top-0 data-[side=top]:h-auto data-[side=top]:border-b data-[side=left]:sm:max-w-sm data-[side=right]:sm:max-w-sm",
-          className,
-        )}
-        {...props}
-      >
-        {children}
-        {showCloseButton && (
-          <SheetPrimitive.Close data-slot="sheet-close" asChild>
-            <Button
-              variant="ghost"
-              className="absolute top-4 right-4"
-              size="icon-sm"
-            >
-              <HugeiconsIcon icon={Cancel01Icon} strokeWidth={2} />
-              <span className="sr-only">Close</span>
-            </Button>
-          </SheetPrimitive.Close>
-        )}
-      </SheetPrimitive.Content>
-    </SheetPortal>
-  );
-}
-
-function SheetHeader({ className, ...props }: React.ComponentProps<"div">) {
-  return (
-    <div
-      data-slot="sheet-header"
-      className={cn("gap-1.5 p-6 flex flex-col", className)}
-      {...props}
-    />
-  );
-}
-
-function SheetFooter({ className, ...props }: React.ComponentProps<"div">) {
-  return (
-    <div
-      data-slot="sheet-footer"
-      className={cn("gap-2 p-6 mt-auto flex flex-col", className)}
-      {...props}
-    />
-  );
-}
-
-function SheetTitle({
-  className,
-  ...props
-}: React.ComponentProps<typeof SheetPrimitive.Title>) {
-  return (
-    <SheetPrimitive.Title
-      data-slot="sheet-title"
-      className={cn("text-foreground text-base font-medium", className)}
-      {...props}
-    />
-  );
-}
-
-function SheetDescription({
-  className,
-  ...props
-}: React.ComponentProps<typeof SheetPrimitive.Description>) {
-  return (
-    <SheetPrimitive.Description
-      data-slot="sheet-description"
-      className={cn("text-muted-foreground text-sm", className)}
-      {...props}
-    />
-  );
-}
-
-export {
-  Sheet,
-  SheetTrigger,
-  SheetClose,
-  SheetContent,
-  SheetHeader,
-  SheetFooter,
-  SheetTitle,
-  SheetDescription,
-};
+import { Dialog as SheetPrimitive } from "radix-ui";
+import type * as React from "react";
+
+import { Button } from "@/components/ui/button";
+import { cn } from "@/lib/utils";
+import { Cancel01Icon } from "@hugeicons/core-free-icons";
+import { HugeiconsIcon } from "@hugeicons/react";
+
+function Sheet({ ...props }: React.ComponentProps<typeof SheetPrimitive.Root>) {
+  return <SheetPrimitive.Root data-slot="sheet" {...props} />;
+}
+
+function SheetTrigger({
+  ...props
+}: React.ComponentProps<typeof SheetPrimitive.Trigger>) {
+  return <SheetPrimitive.Trigger data-slot="sheet-trigger" {...props} />;
+}
+
+function SheetClose({
+  ...props
+}: React.ComponentProps<typeof SheetPrimitive.Close>) {
+  return <SheetPrimitive.Close data-slot="sheet-close" {...props} />;
+}
+
+function SheetPortal({
+  ...props
+}: React.ComponentProps<typeof SheetPrimitive.Portal>) {
+  return <SheetPrimitive.Portal data-slot="sheet-portal" {...props} />;
+}
+
+function SheetOverlay({
+  className,
+  position = "fixed",
+  ...props
+}: React.ComponentProps<typeof SheetPrimitive.Overlay> & {
+  position?: "fixed" | "absolute";
+}) {
+  return (
+    <SheetPrimitive.Overlay
+      data-slot="sheet-overlay"
+      className={cn(
+        "data-open:animate-in data-closed:animate-out data-closed:fade-out-0 data-open:fade-in-0 bg-black/80 duration-100 data-ending-style:opacity-0 data-starting-style:opacity-0 supports-backdrop-filter:backdrop-blur-xs inset-0 z-50",
+        position === "fixed" ? "fixed" : "absolute",
+        className,
+      )}
+      {...props}
+    />
+  );
+}
+
+function SheetContent({
+  className,
+  children,
+  side = "right",
+  showCloseButton = true,
+  container,
+  position = "fixed",
+  overlayClassName,
+  overlayPosition,
+  ...props
+}: React.ComponentProps<typeof SheetPrimitive.Content> & {
+  side?: "top" | "right" | "bottom" | "left";
+  showCloseButton?: boolean;
+  container?: HTMLElement | null;
+  position?: "fixed" | "absolute";
+  overlayClassName?: string;
+  overlayPosition?: "fixed" | "absolute";
+}) {
+  return (
+    <SheetPortal container={container ?? undefined}>
+      <SheetOverlay
+        className={overlayClassName}
+        position={overlayPosition ?? position}
+      />
+      <SheetPrimitive.Content
+        data-slot="sheet-content"
+        data-side={side}
+        className={cn(
+          "bg-background data-open:animate-in data-closed:animate-out data-[side=right]:data-closed:slide-out-to-right-10 data-[side=right]:data-open:slide-in-from-right-10 data-[side=left]:data-closed:slide-out-to-left-10 data-[side=left]:data-open:slide-in-from-left-10 data-[side=top]:data-closed:slide-out-to-top-10 data-[side=top]:data-open:slide-in-from-top-10 data-closed:fade-out-0 data-open:fade-in-0 data-[side=bottom]:data-closed:slide-out-to-bottom-10 data-[side=bottom]:data-open:slide-in-from-bottom-10 z-50 flex flex-col bg-clip-padding text-sm shadow-lg transition duration-200 ease-in-out data-[side=bottom]:inset-x-0 data-[side=bottom]:bottom-0 data-[side=bottom]:h-auto data-[side=bottom]:border-t data-[side=left]:inset-y-0 data-[side=left]:left-0 data-[side=left]:h-full data-[side=left]:w-3/4 data-[side=left]:border-r data-[side=right]:inset-y-0 data-[side=right]:right-0 data-[side=right]:h-full data-[side=right]:w-3/4 data-[side=right]:border-l data-[side=top]:inset-x-0 data-[side=top]:top-0 data-[side=top]:h-auto data-[side=top]:border-b data-[side=left]:sm:max-w-sm data-[side=right]:sm:max-w-sm",
+          position === "fixed" ? "fixed" : "absolute",
+          className,
+        )}
+        {...props}
+      >
+        {children}
+        {showCloseButton && (
+          <SheetPrimitive.Close data-slot="sheet-close" asChild>
+            <Button
+              variant="ghost"
+              className="absolute top-4 right-4"
+              size="icon-sm"
+            >
+              <HugeiconsIcon icon={Cancel01Icon} strokeWidth={2} />
+              <span className="sr-only">Close</span>
+            </Button>
+          </SheetPrimitive.Close>
+        )}
+      </SheetPrimitive.Content>
+    </SheetPortal>
+  );
+}
+
+function SheetHeader({ className, ...props }: React.ComponentProps<"div">) {
+  return (
+    <div
+      data-slot="sheet-header"
+      className={cn("gap-1.5 p-6 flex flex-col", className)}
+      {...props}
+    />
+  );
+}
+
+function SheetFooter({ className, ...props }: React.ComponentProps<"div">) {
+  return (
+    <div
+      data-slot="sheet-footer"
+      className={cn("gap-2 p-6 mt-auto flex flex-col", className)}
+      {...props}
+    />
+  );
+}
+
+function SheetTitle({
+  className,
+  ...props
+}: React.ComponentProps<typeof SheetPrimitive.Title>) {
+  return (
+    <SheetPrimitive.Title
+      data-slot="sheet-title"
+      className={cn("text-foreground text-base font-medium", className)}
+      {...props}
+    />
+  );
+}
+
+function SheetDescription({
+  className,
+  ...props
+}: React.ComponentProps<typeof SheetPrimitive.Description>) {
+  return (
+    <SheetPrimitive.Description
+      data-slot="sheet-description"
+      className={cn("text-muted-foreground text-sm", className)}
+      {...props}
+    />
+  );
+}
+
+export {
+  Sheet,
+  SheetTrigger,
+  SheetClose,
+  SheetContent,
+  SheetHeader,
+  SheetFooter,
+  SheetTitle,
+  SheetDescription,
+};

studio/frontend/src/components/ui/shine-border.tsx

@@ -0,0 +1,61 @@
+import * as React from "react"
+
+import { cn } from "@/lib/utils"
+
+interface ShineBorderProps extends React.HTMLAttributes<HTMLDivElement> {
+  /**
+   * Width of the border in pixels
+   * @default 1
+   */
+  borderWidth?: number
+  /**
+   * Duration of the animation in seconds
+   * @default 14
+   */
+  duration?: number
+  /**
+   * Color of the border, can be a single color or an array of colors
+   * @default "#000000"
+   */
+  shineColor?: string | string[]
+}
+
+/**
+ * Shine Border
+ *
+ * An animated background border effect component with configurable properties.
+ */
+export function ShineBorder({
+  borderWidth = 1,
+  duration = 14,
+  shineColor = "#000000",
+  className,
+  style,
+  ...props
+}: ShineBorderProps) {
+  return (
+    <div
+      style={
+        {
+          "--border-width": `${borderWidth}px`,
+          "--duration": `${duration}s`,
+          backgroundImage: `radial-gradient(transparent,transparent, ${
+            Array.isArray(shineColor) ? shineColor.join(",") : shineColor
+          },transparent,transparent)`,
+          backgroundSize: "300% 300%",
+          mask: `linear-gradient(#fff 0 0) content-box, linear-gradient(#fff 0 0)`,
+          WebkitMask: `linear-gradient(#fff 0 0) content-box, linear-gradient(#fff 0 0)`,
+          WebkitMaskComposite: "xor",
+          maskComposite: "exclude",
+          padding: "var(--border-width)",
+          ...style,
+        } as React.CSSProperties
+      }
+      className={cn(
+        "motion-safe:animate-shine pointer-events-none absolute inset-0 size-full rounded-[inherit] will-change-[background-position]",
+        className
+      )}
+      {...props}
+    />
+  )
+}

studio/frontend/src/components/ui/tooltip.tsx

@@ -43,10 +43,10 @@ function TooltipContent({
       <TooltipPrimitive.Content
         data-slot="tooltip-content"
         sideOffset={sideOffset}
-        className={cn(
-          "data-open:animate-in data-open:fade-in-0 data-open:zoom-in-95 data-[state=delayed-open]:animate-in data-[state=delayed-open]:fade-in-0 data-[state=delayed-open]:zoom-in-95 data-closed:animate-out data-closed:fade-out-0 data-closed:zoom-out-95 data-[side=bottom]:slide-in-from-top-2 data-[side=left]:slide-in-from-right-2 data-[side=right]:slide-in-from-left-2 data-[side=top]:slide-in-from-bottom-2 rounded-2xl px-3 py-1.5 text-xs **:data-[slot=kbd]:rounded-4xl bg-foreground text-background border border-foreground/40 shadow-lg z-50 w-fit max-w-xs origin-(--radix-tooltip-content-transform-origin)",
-          className,
-        )}
+        className={cn(
+          "data-open:animate-in data-open:fade-in-0 data-open:zoom-in-95 data-[state=delayed-open]:animate-in data-[state=delayed-open]:fade-in-0 data-[state=delayed-open]:zoom-in-95 data-closed:animate-out data-closed:fade-out-0 data-closed:zoom-out-95 data-[side=bottom]:slide-in-from-top-2 data-[side=left]:slide-in-from-right-2 data-[side=right]:slide-in-from-left-2 data-[side=top]:slide-in-from-bottom-2 rounded-2xl corner-squircle px-3 py-1.5 text-xs **:data-[slot=kbd]:rounded-4xl bg-foreground text-background border border-foreground/40 shadow-lg z-50 w-fit max-w-xs origin-(--radix-tooltip-content-transform-origin)",
+          className,
+        )}
         {...props}
       >
         {children}

studio/frontend/src/config/training.ts

@@ -39,6 +39,11 @@ export const MODEL_TYPES: ReadonlyArray<{
   label: string;
   description: string;
 }> = [
+  {
+    value: "text",
+    label: "Text",
+    description: "Language models",
+  },
     {
       value: "vision",
       label: "Vision",
@@ -54,11 +59,6 @@ export const MODEL_TYPES: ReadonlyArray<{
       label: "Embeddings",
       description: "Text embedding models",
     },
-    {
-      value: "text",
-      label: "Text",
-      description: "Language models",
-    },
   ];
 
 export const CONTEXT_LENGTHS = [512, 1024, 2048, 4096, 8192, 16384, 32768];
@@ -103,7 +103,7 @@ export const DEFAULT_HYPERPARAMS = {
   warmupSteps: 5,
   maxSteps: 0,
   saveSteps: 0,
-  evalSteps: 0.01,
+  evalSteps: 0.00,
   packing: false,
   trainOnCompletions: false,
   gradientCheckpointing: "unsloth" as const,

studio/frontend/src/features/chat/api/chat-api.ts

@@ -1,5 +1,6 @@
 import { authFetch } from "@/features/auth";
 import type {
+  GgufVariantsResponse,
   InferenceStatusResponse,
   ListLorasResponse,
   ListModelsResponse,
@@ -74,6 +75,16 @@ export async function unloadModel(payload: UnloadModelRequest): Promise<void> {
   await parseJsonOrThrow<unknown>(response);
 }
 
+export async function listGgufVariants(
+  repoId: string,
+  hfToken?: string,
+): Promise<GgufVariantsResponse> {
+  const params = new URLSearchParams({ repo_id: repoId });
+  if (hfToken) params.set("hf_token", hfToken);
+  const response = await authFetch(`/api/models/gguf-variants?${params}`);
+  return parseJsonOrThrow<GgufVariantsResponse>(response);
+}
+
 function parseSseEvent(rawEvent: string): string[] {
   const dataLines: string[] = [];
   for (const line of rawEvent.split(/\r?\n/)) {

studio/frontend/src/features/chat/chat-page.tsx

@@ -33,6 +33,7 @@ import {
   useRef,
   useState,
 } from "react";
+import { toast } from "sonner";
 import { GuidedTour, useGuidedTourController } from "@/features/tour";
 import { ChatSettingsPanel } from "./chat-settings-sheet";
 import { db } from "./db";
@@ -50,7 +51,7 @@ import {
   SharedComposer,
 } from "./shared-composer";
 import { ThreadSidebar } from "./thread-sidebar";
-import type { ChatView } from "./types";
+import type { ChatView, MessageRecord } from "./types";
 import { buildChatTourSteps } from "./tour";
 
 type LoraCandidate = {
@@ -90,6 +91,40 @@ function pickBestLoraForBase(
   return partial ?? sorted[0];
 }
 
+function messageHasImage(message: MessageRecord): boolean {
+  const contentParts = Array.isArray(message.content) ? message.content : [];
+  if (contentParts.some((part) => part.type === "image")) {
+    return true;
+  }
+  const attachments = Array.isArray(message.attachments) ? message.attachments : [];
+  for (const attachment of attachments) {
+    const parts = Array.isArray(attachment.content) ? attachment.content : [];
+    for (const part of parts as Array<{ type?: string }>) {
+      if (part?.type === "image") {
+        return true;
+      }
+    }
+  }
+  return false;
+}
+
+async function resolveActiveSingleThreadId(view: ChatView): Promise<string | undefined> {
+  if (view.mode !== "single") {
+    return undefined;
+  }
+  if (view.threadId) {
+    return view.threadId;
+  }
+
+  // New-thread flow keeps threadId undefined in local view state.
+  // Fall back to most recent regular base thread.
+  const candidates = await db.threads.where("modelType").equals("base").toArray();
+  const latest = candidates
+    .filter((thread) => !thread.archived && !thread.pairId)
+    .sort((a, b) => b.createdAt - a.createdAt)[0];
+  return latest?.id;
+}
+
 const SingleContent = memo(function SingleContent({
   threadId,
   newThreadNonce,
@@ -208,7 +243,7 @@ function InlineSidebar({
 
   return (
     <div
-      className="group shrink-0 h-full"
+      className="group shrink-0 h-full pb-3.5"
       data-state={state}
       data-collapsible={collapsed ? "offcanvas" : ""}
       data-side={side}
@@ -216,11 +251,11 @@ function InlineSidebar({
       <aside
         data-sidebar="sidebar"
         className={cn(
-          "bg-sidebar text-sidebar-foreground h-full overflow-hidden rounded-2xl corner-squircle transition-[width] duration-200 ease-linear",
+          "bg-muted/70 text-sidebar-foreground h-full overflow-hidden rounded-2xl corner-squircle transition-[width] duration-200 ease-linear",
           !collapsed &&
           (side === "left"
-            ? "border-r border-0 border-sidebar-border"
-            : "border-l border-0 border-sidebar-border"),
+            ? "border-r border-sidebar-border/70"
+            : "border-l border-sidebar-border/70"),
           collapsed ? "w-0" : "w-(--sidebar-width)",
         )}
       >
@@ -279,6 +314,7 @@ export function ChatPage(): ReactElement {
   );
   const inferenceParams = useChatRuntimeStore((state) => state.params);
   const setInferenceParams = useChatRuntimeStore((state) => state.setParams);
+  const activeGgufVariant = useChatRuntimeStore((state) => state.activeGgufVariant);
   const autoTitle = useChatRuntimeStore((state) => state.autoTitle);
   const setAutoTitle = useChatRuntimeStore((state) => state.setAutoTitle);
   const modelsFromStore = useChatRuntimeStore((state) => state.models);
@@ -300,19 +336,47 @@ export function ChatPage(): ReactElement {
   }, [inferenceParams.checkpoint, lorasFromStore]);
 
   const handleCheckpointChange = useCallback(
-    (value: string, meta?: { isLora: boolean }) => {
-      const currentCheckpoint =
-        useChatRuntimeStore.getState().params.checkpoint;
-      if (!value || value === currentCheckpoint) return;
-      setView({ mode: "single", newThreadNonce: crypto.randomUUID() });
+    (value: string, meta?: { isLora: boolean; ggufVariant?: string }) => {
+      const store = useChatRuntimeStore.getState();
+      const currentCheckpoint = store.params.checkpoint;
+      const currentVariant = store.activeGgufVariant;
+      if (!value || (value === currentCheckpoint && (meta?.ggufVariant ?? null) === (currentVariant ?? null))) return;
       void (async () => {
-        if (currentCheckpoint) {
-          await ejectModel();
+        let switchNote: string | undefined;
+        const activeThreadId = await resolveActiveSingleThreadId(view);
+        if (activeThreadId) {
+          const thread = await db.threads.get(activeThreadId);
+          if (thread?.modelId && thread.modelId !== value) {
+            const messages = await db.messages
+              .where("threadId")
+              .equals(activeThreadId)
+              .toArray();
+            const hasImage = messages.some(messageHasImage);
+            const targetModel = modelsFromStore.find((model) => model.id === value);
+            const nonVisionWithImages = hasImage && targetModel?.isVision === false;
+
+            switchNote = nonVisionWithImages
+              ? "Full chat history will be sent to the new model. This chat has images; text-only models may fail."
+              : hasImage
+                ? "Full chat history will be sent to the new model. This chat includes images."
+                : "Full chat history will be sent to the new model.";
+          }
         }
-        await selectModel({ id: value, isLora: meta?.isLora });
+
+        if (switchNote) {
+          toast.warning("Model changed for this chat", {
+            description: switchNote,
+            duration: 6000,
+          });
+        }
+        await selectModel({
+          id: value,
+          isLora: meta?.isLora,
+          ggufVariant: meta?.ggufVariant,
+        });
       })();
     },
-    [selectModel, ejectModel],
+    [modelsFromStore, selectModel, view],
   );
   const handleEject = useCallback(() => {
     void ejectModel();
@@ -361,36 +425,8 @@ export function ChatPage(): ReactElement {
   const handleThreadSelect = useCallback(
     (nextView: ChatView) => {
       setView(nextView);
-
-      const threadId =
-        nextView.mode === "single" ? nextView.threadId : undefined;
-      const pairId =
-        nextView.mode === "compare" ? nextView.pairId : undefined;
-
-      void (async () => {
-        let thread: import("./types").ThreadRecord | undefined;
-        if (threadId) {
-          thread = await db.threads.get(threadId);
-        } else if (pairId) {
-          thread = await db.threads
-            .where("pairId")
-            .equals(pairId)
-            .first();
-        }
-        const threadModelId = thread?.modelId;
-        if (!threadModelId) return;
-
-        const currentCheckpoint =
-          useChatRuntimeStore.getState().params.checkpoint;
-        if (threadModelId === currentCheckpoint) return;
-
-        if (currentCheckpoint) {
-          await ejectModel();
-        }
-        await selectModel({ id: threadModelId });
-      })();
     },
-    [ejectModel, selectModel],
+    [],
   );
 
   const models = useMemo<ModelOption[]>(
@@ -410,6 +446,8 @@ export function ChatPage(): ReactElement {
         name: lora.name,
         baseModel: lora.baseModel,
         updatedAt: lora.updatedAt,
+        source: lora.source,
+        exportType: lora.exportType,
       })),
     [lorasFromStore],
   );
@@ -555,6 +593,7 @@ export function ChatPage(): ReactElement {
                 models={models}
                 loraModels={loraModels}
                 value={inferenceParams.checkpoint}
+                activeGgufVariant={activeGgufVariant}
                 onValueChange={handleCheckpointChange}
                 onEject={handleEject}
                 variant="ghost"

studio/frontend/src/features/chat/chat-settings-sheet.tsx

@@ -157,6 +157,7 @@ export function ChatSettingsPanel({
 }: ChatSettingsPanelProps) {
   const [presets, setPresets] = useState<Preset[]>(BUILTIN_PRESETS);
   const [activePreset, setActivePreset] = useState("Default");
+  const isBuiltinPreset = BUILTIN_PRESETS.some((p) => p.name === activePreset);
 
   function set<K extends keyof InferenceParams>(key: K) {
     return (v: InferenceParams[K]) => onParamsChange({ ...params, [key]: v });
@@ -165,7 +166,11 @@ export function ChatSettingsPanel({
   function applyPreset(name: string) {
     const p = presets.find((pr) => pr.name === name);
     if (p) {
-      onParamsChange({ ...p.params, systemPrompt: params.systemPrompt });
+      onParamsChange({
+        ...p.params,
+        systemPrompt: params.systemPrompt,
+        checkpoint: params.checkpoint,
+      });
       setActivePreset(name);
     }
   }
@@ -195,7 +200,7 @@ export function ChatSettingsPanel({
 
   return (
     <aside
-      className={`shrink-0 h-full overflow-hidden bg-sidebar rounded-2xl corner-squircle transition-[width] duration-200 ease-linear ${open ? "w-[17rem] border-sidebar-border" : "w-0"}`}
+      className={`shrink-0 self-start h-[calc(100%-0.875rem)] overflow-hidden bg-muted/70 rounded-2xl corner-squircle transition-[width] duration-200 ease-linear ${open ? "w-[17rem] border-l border-sidebar-border/70" : "w-0"}`}
     >
       <div className="flex h-full w-[17rem] flex-col">
         <div className="flex items-center gap-2 px-4 py-3">
@@ -219,24 +224,7 @@ export function ChatSettingsPanel({
                 <SelectContent>
                   {presets.map((p) => (
                     <SelectItem key={p.name} value={p.name}>
-                      <div className="flex w-full items-center justify-between gap-2">
-                        <span>{p.name}</span>
-                        {!BUILTIN_PRESETS.some((bp) => bp.name === p.name) && (
-                          <button
-                            type="button"
-                            onClick={(e) => {
-                              e.stopPropagation();
-                              deletePreset(p.name);
-                            }}
-                            className="text-muted-foreground hover:text-destructive"
-                          >
-                            <HugeiconsIcon
-                              icon={Delete02Icon}
-                              className="size-3"
-                            />
-                          </button>
-                        )}
-                      </div>
+                      {p.name}
                     </SelectItem>
                   ))}
                 </SelectContent>
@@ -250,6 +238,20 @@ export function ChatSettingsPanel({
                 <HugeiconsIcon icon={FloppyDiskIcon} className="size-3.5" />
                 Save
               </button>
+              <button
+                type="button"
+                onClick={() => deletePreset(activePreset)}
+                disabled={isBuiltinPreset}
+                className="flex h-8 items-center gap-1.5 rounded-md border px-2.5 text-xs text-muted-foreground transition-colors hover:bg-accent disabled:cursor-not-allowed disabled:opacity-50"
+                title={
+                  isBuiltinPreset
+                    ? "Built-in presets cannot be deleted"
+                    : "Delete selected preset"
+                }
+              >
+                <HugeiconsIcon icon={Delete02Icon} className="size-3.5" />
+                Delete
+              </button>
             </div>
           </div>
 
@@ -320,7 +322,7 @@ export function ChatSettingsPanel({
                 label="Max Tokens"
                 value={params.maxTokens}
                 min={64}
-                max={4096}
+                max={4092}
                 step={64}
                 onChange={set("maxTokens")}
               />

studio/frontend/src/features/chat/hooks/use-chat-model-runtime.ts

@@ -20,6 +20,8 @@ const DEFAULT_MODEL_MAX_SEQ_LENGTH = 2048;
 type SelectedModelInput = {
   id: string;
   isLora?: boolean;
+  ggufVariant?: string;
+  loadingDescription?: string;
 };
 
 const LORA_SUFFIX_RE = /_(\d{9,})$/;
@@ -41,11 +43,13 @@ function stripTrailingEpoch(input: string): string {
 function describeModel(model: {
   is_lora?: boolean;
   is_vision?: boolean;
+  is_gguf?: boolean;
 }): string | undefined {
   const tags: string[] = [];
+  if (model.is_gguf) tags.push("GGUF");
   if (model.is_lora) tags.push("LoRA");
   if (model.is_vision) tags.push("Vision");
-  if (!model.is_lora && !model.is_vision) tags.push("Base");
+  if (!model.is_lora && !model.is_vision && !model.is_gguf) tags.push("Base");
   return tags.join(" · ");
 }
 
@@ -54,6 +58,7 @@ function toChatModelSummary(model: {
   name?: string | null;
   is_lora?: boolean;
   is_vision?: boolean;
+  is_gguf?: boolean;
 }): ChatModelSummary {
   return {
     id: model.id,
@@ -61,6 +66,7 @@ function toChatModelSummary(model: {
     description: describeModel(model),
     isLora: Boolean(model.is_lora),
     isVision: Boolean(model.is_vision),
+    isGguf: Boolean(model.is_gguf),
   };
 }
 
@@ -68,6 +74,8 @@ function toLoraSummary(lora: {
   display_name: string;
   adapter_path: string;
   base_model?: string | null;
+  source?: "training" | "exported" | null;
+  export_type?: "lora" | "merged" | "gguf" | null;
 }): ChatLoraSummary {
   const idTail = lora.adapter_path.split("/").filter(Boolean).at(-1) ?? "";
   const updatedAt =
@@ -78,6 +86,8 @@ function toLoraSummary(lora: {
     name: stripTrailingEpoch(lora.display_name),
     baseModel: lora.base_model || "Unknown base model",
     updatedAt,
+    source: lora.source ?? undefined,
+    exportType: lora.export_type ?? undefined,
   };
 }
 
@@ -134,7 +144,7 @@ export function useChatModelRuntime() {
       setLoras(lorasRes.loras.map(toLoraSummary));
 
       if (statusRes.active_model) {
-        setCheckpoint(statusRes.active_model);
+        setCheckpoint(statusRes.active_model, statusRes.gguf_variant);
       }
     } catch (error) {
       const message =
@@ -149,17 +159,31 @@ export function useChatModelRuntime() {
   const selectModel = useCallback(
     async (selection: string | SelectedModelInput) => {
       const modelId = typeof selection === "string" ? selection : selection.id;
-      if (!modelId || params.checkpoint === modelId) {
+      const ggufVariant =
+        typeof selection === "string" ? undefined : selection.ggufVariant;
+      const currentVariant = useChatRuntimeStore.getState().activeGgufVariant;
+      if (!modelId || (params.checkpoint === modelId && (ggufVariant ?? null) === (currentVariant ?? null))) {
         return;
       }
 
       const explicitIsLora =
         typeof selection === "string" ? undefined : selection.isLora;
+      const extraLoadingDescription =
+        typeof selection === "string" ? undefined : selection.loadingDescription;
       const model = models.find((entry) => entry.id === modelId);
       const lora = loras.find((entry) => entry.id === modelId);
       const isLora =
         explicitIsLora ?? model?.isLora ?? (lora ? true : false);
       const displayName = model?.name || lora?.name || modelId;
+      const currentCheckpoint =
+        useChatRuntimeStore.getState().params.checkpoint;
+      const loadingDescription = [
+        currentCheckpoint ? "Unloading previous model first." : null,
+        extraLoadingDescription ?? null,
+        "This may include downloading. Large models can take a while.",
+      ]
+        .filter(Boolean)
+        .join(" ");
 
       setModelsError(null);
       setLoadingModel({ id: modelId, displayName });
@@ -177,6 +201,7 @@ export function useChatModelRuntime() {
             max_seq_length: DEFAULT_MODEL_MAX_SEQ_LENGTH,
             load_in_4bit: true,
             is_lora: isLora,
+            gguf_variant: ggufVariant ?? null,
           });
 
           const currentParams = useChatRuntimeStore.getState().params;
@@ -193,8 +218,7 @@ export function useChatModelRuntime() {
           success: `${displayName} loaded`,
           error: (err) =>
             err instanceof Error ? err.message : "Failed to load model",
-          description:
-            "This may include downloading. Large models can take a while.",
+          description: loadingDescription,
         });
       } catch (error) {
         setLoadingModel(null);
@@ -220,7 +244,7 @@ export function useChatModelRuntime() {
 
       await toast.promise(performUnload(), {
         loading: "Unloading model",
-        success: "Model unloaded",
+        success: { message: "Model unloaded", duration: 1200 },
         error: (err) =>
           err instanceof Error ? err.message : "Failed to unload model",
         description: "Releases VRAM and resets inference state.",

studio/frontend/src/features/chat/stores/chat-runtime-store.ts

@@ -39,13 +39,14 @@ type ChatRuntimeStore = {
   runningByThreadId: Record<string, boolean>;
   autoTitle: boolean;
   modelsError: string | null;
+  activeGgufVariant: string | null;
   setParams: (params: InferenceParams) => void;
   setModels: (models: ChatModelSummary[]) => void;
   setLoras: (loras: ChatLoraSummary[]) => void;
   setThreadRunning: (threadId: string, running: boolean) => void;
   setAutoTitle: (enabled: boolean) => void;
   setModelsError: (error: string | null) => void;
-  setCheckpoint: (modelId: string) => void;
+  setCheckpoint: (modelId: string, ggufVariant?: string | null) => void;
   clearCheckpoint: () => void;
 };
 
@@ -56,6 +57,7 @@ export const useChatRuntimeStore = create<ChatRuntimeStore>((set) => ({
   runningByThreadId: {},
   autoTitle: loadBool(AUTO_TITLE_KEY, false),
   modelsError: null,
+  activeGgufVariant: null,
   setParams: (params) => set({ params }),
   setModels: (models) => set({ models }),
   setLoras: (loras) => set({ loras }),
@@ -75,12 +77,13 @@ export const useChatRuntimeStore = create<ChatRuntimeStore>((set) => ({
       return { autoTitle };
     }),
   setModelsError: (modelsError) => set({ modelsError }),
-  setCheckpoint: (modelId) =>
+  setCheckpoint: (modelId, ggufVariant) =>
     set((state) => ({
       params: {
         ...state.params,
         checkpoint: modelId,
       },
+      activeGgufVariant: ggufVariant ?? null,
     })),
   clearCheckpoint: () =>
     set((state) => ({
@@ -88,5 +91,6 @@ export const useChatRuntimeStore = create<ChatRuntimeStore>((set) => ({
         ...state.params,
         checkpoint: "",
       },
+      activeGgufVariant: null,
     })),
 }));

studio/frontend/src/features/chat/types/api.ts

@@ -3,6 +3,7 @@ export interface BackendModelDetails {
   name?: string | null;
   is_vision?: boolean;
   is_lora?: boolean;
+  is_gguf?: boolean;
 }
 
 export interface ListModelsResponse {
@@ -14,6 +15,8 @@ export interface BackendLoraInfo {
   display_name: string;
   adapter_path: string;
   base_model?: string | null;
+  source?: "training" | "exported" | null;
+  export_type?: "lora" | "merged" | "gguf" | null;
 }
 
 export interface ListLorasResponse {
@@ -27,6 +30,20 @@ export interface LoadModelRequest {
   max_seq_length: number;
   load_in_4bit: boolean;
   is_lora: boolean;
+  gguf_variant?: string | null;
+}
+
+export interface GgufVariantDetail {
+  filename: string;
+  quant: string;
+  size_bytes: number;
+}
+
+export interface GgufVariantsResponse {
+  repo_id: string;
+  variants: GgufVariantDetail[];
+  has_vision: boolean;
+  default_variant: string | null;
 }
 
 export interface LoadModelResponse {
@@ -35,6 +52,7 @@ export interface LoadModelResponse {
   display_name: string;
   is_vision: boolean;
   is_lora: boolean;
+  is_gguf?: boolean;
   inference?: {
     temperature?: number;
     top_p?: number;
@@ -50,6 +68,8 @@ export interface UnloadModelRequest {
 export interface InferenceStatusResponse {
   active_model: string | null;
   is_vision: boolean;
+  is_gguf?: boolean;
+  gguf_variant?: string | null;
   loading: string[];
   loaded: string[];
 }

studio/frontend/src/features/chat/types/runtime.ts

@@ -15,7 +15,7 @@ export const DEFAULT_INFERENCE_PARAMS: InferenceParams = {
   topK: 50,
   minP: 0.01,
   repetitionPenalty: 1.1,
-  maxTokens: 512,
+  maxTokens: 4092,
   systemPrompt: "",
   checkpoint: "",
 };
@@ -26,6 +26,7 @@ export interface ChatModelSummary {
   description?: string;
   isVision: boolean;
   isLora: boolean;
+  isGguf?: boolean;
 }
 
 export interface ChatLoraSummary {
@@ -33,4 +34,6 @@ export interface ChatLoraSummary {
   name: string;
   baseModel: string;
   updatedAt?: number;
+  source?: "training" | "exported";
+  exportType?: "lora" | "merged" | "gguf";
 }

studio/frontend/src/features/data-recipes/data/recipes-db.ts

@@ -0,0 +1,79 @@
+import { createEmptyRecipePayload } from "@/features/recipe-studio";
+import { normalizeNonEmptyName } from "@/utils";
+import Dexie, { type EntityTable, liveQuery } from "dexie";
+import { useEffect, useState } from "react";
+import type { RecipeRecord, SaveRecipeInput } from "../types";
+
+const db = new Dexie("unsloth-data-recipes") as Dexie & {
+  recipes: EntityTable<RecipeRecord, "id">;
+};
+
+db.version(1).stores({
+  recipes: "id, name, updatedAt, createdAt",
+});
+
+export function listRecipes(): Promise<RecipeRecord[]> {
+  return db.recipes.orderBy("updatedAt").reverse().toArray();
+}
+
+export function getRecipe(id: string): Promise<RecipeRecord | undefined> {
+  return db.recipes.get(id);
+}
+
+export async function saveRecipe(
+  input: SaveRecipeInput,
+): Promise<RecipeRecord> {
+  const now = Date.now();
+  const id = input.id ?? crypto.randomUUID();
+  const existing = input.id ? await db.recipes.get(input.id) : undefined;
+  const record: RecipeRecord = {
+    id,
+    name: normalizeNonEmptyName(input.name),
+    payload: input.payload,
+    createdAt: existing?.createdAt ?? now,
+    updatedAt: now,
+    learningRecipeId: input.learningRecipeId ?? existing?.learningRecipeId,
+    learningRecipeTitle:
+      input.learningRecipeTitle ?? existing?.learningRecipeTitle,
+  };
+  await db.recipes.put(record);
+  return record;
+}
+
+export async function deleteRecipe(id: string): Promise<void> {
+  await db.recipes.delete(id);
+}
+
+export function createRecipeDraft(): Promise<RecipeRecord> {
+  return saveRecipe({
+    name: "Unnamed",
+    payload: createEmptyRecipePayload(),
+  });
+}
+
+export function createRecipeFromLearningRecipe(input: {
+  templateId: string;
+  templateTitle: string;
+  payload: RecipeRecord["payload"];
+}): Promise<RecipeRecord> {
+  return saveRecipe({
+    name: input.templateTitle,
+    payload: input.payload,
+    learningRecipeId: input.templateId,
+    learningRecipeTitle: input.templateTitle,
+  });
+}
+
+export function useRecipes(): RecipeRecord[] {
+  const [recipes, setRecipes] = useState<RecipeRecord[]>([]);
+
+  useEffect(() => {
+    const sub = liveQuery(() => listRecipes()).subscribe({
+      next: (value) => setRecipes(value),
+      error: (error) => console.error("data-recipes liveQuery:", error),
+    });
+    return () => sub.unsubscribe();
+  }, []);
+
+  return recipes;
+}

studio/frontend/src/features/data-recipes/index.ts

@@ -0,0 +1,2 @@
+export { DataRecipesPage } from "./pages/data-recipes-page";
+export { EditRecipePage } from "./pages/edit-recipe-page";

studio/frontend/src/features/data-recipes/learning-recipes/conversation.json

@@ -0,0 +1,280 @@
+{
+  "recipe": {
+    "model_providers": [
+      {
+        "name": "provider_1",
+        "endpoint": "https://openrouter.ai/api/v1",
+        "provider_type": "openai",
+        "extra_headers": {},
+        "extra_body": {}
+      }
+    ],
+    "mcp_providers": [],
+    "model_configs": [
+      {
+        "alias": "model_1",
+        "model": "mistralai/ministral-8b-2512",
+        "provider": "provider_1",
+        "inference_parameters": {
+          "temperature": 0.7,
+          "max_tokens": 2048
+        }
+      }
+    ],
+    "tool_configs": [],
+    "columns": [
+      {
+        "column_type": "sampler",
+        "name": "domain",
+        "drop": true,
+        "sampler_type": "category",
+        "params": {
+          "values": [
+            "Tech Support",
+            "Personal Finance",
+            "Learning"
+          ]
+        }
+      },
+      {
+        "column_type": "sampler",
+        "name": "topic",
+        "drop": true,
+        "sampler_type": "subcategory",
+        "params": {
+          "category": "domain",
+          "values": {
+            "Tech Support": [
+              "Wi-Fi keeps disconnecting",
+              "Laptop running very slow",
+              "Cannot install app update"
+            ],
+            "Personal Finance": [
+              "Monthly budget planning",
+              "Credit card debt payoff",
+              "Emergency fund setup"
+            ],
+            "Learning": [
+              "Exam study plan",
+              "Learn Python basics",
+              "Improve English writing"
+            ]
+          }
+        }
+      },
+      {
+        "column_type": "sampler",
+        "name": "conversation_length",
+        "drop": true,
+        "sampler_type": "category",
+        "params": {
+          "values": [
+            "4",
+            "6"
+          ]
+        }
+      },
+      {
+        "column_type": "llm-text",
+        "name": "user_goal",
+        "drop": false,
+        "model_alias": "model_1",
+        "prompt": "Write one user goal for a chat assistant.\nDomain: {{ domain }}\nTopic: {{ topic }}\nConversation length target: {{ conversation_length }} messages total.\nRules:\n- 1 sentence.\n- Specific and practical.\n- Output only the goal text.",
+        "system_prompt": "You write realistic user goals for assistant conversations.\n",
+        "with_trace": "none"
+      },
+      {
+        "column_type": "llm-structured",
+        "name": "output_format",
+        "drop": false,
+        "model_alias": "model_1",
+        "prompt": "Generate a realistic multi-turn conversation.\nUser goal:\n{{ user_goal }}\nConstraints:\n- Exactly {{ conversation_length }} messages total.\n- Alternate roles strictly: user, assistant, user, assistant...\n- First message must be user.\n- Last message must be assistant.\n- Keep responses grounded in {{ domain }} / {{ topic }}.\n- End naturally with resolution or clear next step.\n- No markdown, no extra keys.",
+        "output_format": {
+          "type": "object",
+          "properties": {
+            "conversation": {
+              "type": "array",
+              "minItems": 4,
+              "maxItems": 6,
+              "items": {
+                "type": "object",
+                "properties": {
+                  "role": {
+                    "type": "string",
+                    "enum": [
+                      "user",
+                      "assistant"
+                    ]
+                  },
+                  "content": {
+                    "type": "string",
+                    "minLength": 1
+                  }
+                },
+                "required": [
+                  "role",
+                  "content"
+                ],
+                "additionalProperties": false
+              }
+            }
+          },
+          "required": [
+            "conversation"
+          ],
+          "additionalProperties": false
+        }
+      }
+    ],
+    "processors": []
+  },
+  "run": {
+    "rows": 5,
+    "preview": true,
+    "output_formats": [
+      "jsonl"
+    ]
+  },
+  "ui": {
+    "nodes": [
+      {
+        "id": "provider_1",
+        "x": -1056.848383841495,
+        "y": 519.6373927070263,
+        "width": 400
+      },
+      {
+        "id": "model_1",
+        "x": -543.7221365246206,
+        "y": 488.2975724283656,
+        "width": 400
+      },
+      {
+        "id": "domain",
+        "x": 0,
+        "y": 140,
+        "width": 400
+      },
+      {
+        "id": "topic",
+        "x": 0,
+        "y": 280,
+        "width": 400
+      },
+      {
+        "id": "conversation_length",
+        "x": 466.61510192672256,
+        "y": 139.68271861864798,
+        "width": 400
+      },
+      {
+        "id": "user_goal",
+        "x": 1.412158386197035,
+        "y": 508.77123580445596,
+        "width": 400
+      },
+      {
+        "id": "output_format",
+        "x": 1.1486983549970375,
+        "y": 754.4221089431811,
+        "width": 400
+      },
+      {
+        "id": "note_1",
+        "x": 210.01377182764494,
+        "y": -262.9440547613487,
+        "width": 400,
+        "node_type": "markdown_note",
+        "name": "note_1",
+        "markdown": "###  Start with controlled chat context\nThis recipe uses sampler columns to shape each conversation:\n\n- `domain`\n- `topic`\n- `conversation_length` (4 or 6 messages)\n\n**Why this helps**:\n\n- You get varied conversations without manual writing\n- Each row stays grounded in a clear scenario\n- You can scale quickly while keeping data quality consistent",
+        "note_color": "#FFE4E6",
+        "note_opacity": "35"
+      },
+      {
+        "id": "note_2",
+        "x": 515.9369583007435,
+        "y": 454.3936030274385,
+        "width": 400,
+        "node_type": "markdown_note",
+        "name": "note_2",
+        "markdown": "The **LLM Text** block (`user_goal`) creates one realistic user intent from sampler context.\n\n**It should be**:\n\n- **specific**\n- **practical**\n- **short**\n\nThis goal becomes the anchor for the full multi-turn conversation.",
+        "note_color": "#FFE4E6",
+        "note_opacity": "35"
+      },
+      {
+        "id": "note_3",
+        "x": -12.952616065779239,
+        "y": 912.1316336111515,
+        "width": 400,
+        "node_type": "markdown_note",
+        "name": "note_3",
+        "markdown": "The **LLM Structured** block (`output_format`) generates the conversation as strict JSON.\n\nIn this recipe, schema enforces:\n\n- `conversation` array\n- message objects with `role` + `content`\n- role enum: `user` / `assistant`\n- no extra keys\n\nPrompt constraints also enforce:\n\n- exact length (`{{ conversation_length }}`)\n- alternating roles\n- first user message, last assistant message\n- natural ending\n\nThis is key for training data: same shape, less cleanup.",
+        "note_color": "#FFE4E6",
+        "note_opacity": "35"
+      },
+      {
+        "id": "note_4",
+        "x": -519.9585237323188,
+        "y": 81.84144119564277,
+        "width": 400,
+        "node_type": "markdown_note",
+        "name": "note_4",
+        "markdown": "Sampler columns are useful during generation but usually noisy in final export.\n\nSet helper columns to `drop=true`, keep only core outputs such as:\n\n- `user_goal`\n- `output_format`\n\nTip: Keep final schema close to your training format, not your generation scaffolding.\n",
+        "note_color": "#FFE4E6",
+        "note_opacity": "35"
+      }
+    ],
+    "edges": [
+      {
+        "from": "domain",
+        "to": "topic",
+        "type": "canvas",
+        "source_handle": "data-out-bottom",
+        "target_handle": "data-in-top"
+      },
+      {
+        "from": "domain",
+        "to": "conversation_length",
+        "type": "canvas",
+        "source_handle": "data-out",
+        "target_handle": "data-in"
+      },
+      {
+        "from": "topic",
+        "to": "user_goal",
+        "type": "canvas",
+        "source_handle": "data-out-bottom",
+        "target_handle": "data-in-top"
+      },
+      {
+        "from": "user_goal",
+        "to": "output_format",
+        "type": "canvas",
+        "source_handle": "data-out-bottom",
+        "target_handle": "data-in-top"
+      },
+      {
+        "from": "provider_1",
+        "to": "model_1",
+        "type": "semantic",
+        "source_handle": "semantic-out",
+        "target_handle": "semantic-in"
+      },
+      {
+        "from": "model_1",
+        "to": "user_goal",
+        "type": "semantic",
+        "source_handle": "semantic-out",
+        "target_handle": "data-in"
+      },
+      {
+        "from": "model_1",
+        "to": "output_format",
+        "type": "semantic",
+        "source_handle": "semantic-out-bottom",
+        "target_handle": "data-in"
+      }
+    ],
+    "layout_direction": "LR"
+  }
+}

studio/frontend/src/features/data-recipes/learning-recipes/index.ts

@@ -0,0 +1,134 @@
+import type { RecipePayload } from "@/features/recipe-studio";
+
+const structuredOutputsJinjaUrl = new URL(
+  "./structured-outputs-jinja.json",
+  import.meta.url,
+).href;
+const pdfGroundedQaUrl = new URL("./pdf-grounded-qa.json", import.meta.url)
+  .href;
+const instructionFromAnswerUrl = new URL(
+  "./instruction-from-answer.json",
+  import.meta.url,
+).href;
+const textToPythonUrl = new URL("./text-to-python.json", import.meta.url).href;
+const textToSqlUrl = new URL("./text-to-sql.json", import.meta.url).href;
+const conversationUrl = new URL("./conversation.json", import.meta.url).href;
+
+function isRecord(value: unknown): value is Record<string, unknown> {
+  return !!value && typeof value === "object" && !Array.isArray(value);
+}
+
+function toRecordArray(value: unknown): Record<string, unknown>[] {
+  if (!Array.isArray(value)) {
+    return [];
+  }
+  return value.filter((item): item is Record<string, unknown> =>
+    isRecord(item),
+  );
+}
+
+function coerceRecipePayload(value: unknown): RecipePayload {
+  if (!isRecord(value)) {
+    throw new Error("Template payload is invalid JSON object.");
+  }
+
+  const recipeSource = isRecord(value.recipe) ? value.recipe : value;
+  if (!Array.isArray(recipeSource.columns)) {
+    throw new Error("Template payload must include recipe.columns.");
+  }
+
+  if (isRecord(value.recipe) && isRecord(value.run) && isRecord(value.ui)) {
+    return value as unknown as RecipePayload;
+  }
+
+  const recipe: RecipePayload["recipe"] = {
+    // biome-ignore lint/style/useNamingConvention: api schema
+    model_providers: toRecordArray(recipeSource.model_providers),
+    // biome-ignore lint/style/useNamingConvention: api schema
+    mcp_providers: toRecordArray(recipeSource.mcp_providers),
+    // biome-ignore lint/style/useNamingConvention: api schema
+    model_configs: toRecordArray(recipeSource.model_configs),
+    // biome-ignore lint/style/useNamingConvention: api schema
+    seed_config: isRecord(recipeSource.seed_config)
+      ? recipeSource.seed_config
+      : undefined,
+    // biome-ignore lint/style/useNamingConvention: api schema
+    tool_configs: toRecordArray(recipeSource.tool_configs),
+    columns: toRecordArray(recipeSource.columns),
+    processors: toRecordArray(recipeSource.processors),
+  };
+
+  return {
+    recipe,
+    run: {
+      rows: 5,
+      preview: true,
+      // biome-ignore lint/style/useNamingConvention: api schema
+      output_formats: ["jsonl"],
+    },
+    ui: {
+      nodes: [],
+      edges: [],
+    },
+  };
+}
+
+async function loadPayloadFromUrl(url: string): Promise<RecipePayload> {
+  const response = await fetch(url);
+  if (!response.ok) {
+    throw new Error(`Failed to fetch template payload (${response.status})`);
+  }
+  const json = (await response.json()) as unknown;
+  return coerceRecipePayload(json);
+}
+
+export type LearningRecipeDef = {
+  id: string;
+  title: string;
+  description: string;
+  loadPayload: () => Promise<RecipePayload>;
+};
+
+export const LEARNING_RECIPES: LearningRecipeDef[] = [
+  {
+    id: "structured-outputs-jinja",
+    title: "Structured Outputs + Jinja Expressions",
+    description:
+      "Support ticket triage with structured JSON outputs and Jinja conditionals.",
+    loadPayload: () => loadPayloadFromUrl(structuredOutputsJinjaUrl),
+  },
+  {
+    id: "pdf-grounded-qa",
+    title: "PDF Document QA",
+    description: "Build grounded question-answer examples from PDF chunks.",
+    loadPayload: () => loadPayloadFromUrl(pdfGroundedQaUrl),
+  },
+  {
+    id: "instruction-from-answer",
+    title: "Instruction from Answer",
+    description:
+      "Use seed answer columns to generate high-quality instruction targets.",
+    loadPayload: () => loadPayloadFromUrl(instructionFromAnswerUrl),
+  },
+  {
+    id: "text-to-python",
+    title: "Text to Python",
+    description:
+      "Generate instruction-to-code data with category sampling and LLM judging.",
+    loadPayload: () => loadPayloadFromUrl(textToPythonUrl),
+  },
+  {
+    id: "text-to-sql",
+    title: "Text to SQL",
+    description:
+      "Generate SQL tasks and runnable SQL outputs with prompt-driven generation.",
+    loadPayload: () => loadPayloadFromUrl(textToSqlUrl),
+  },
+  {
+    id: "conversation",
+    title: "Multi-Turn Chat",
+    description:
+      "Generate realistic user-assistant conversations with structured message output.",
+    loadPayload: () => loadPayloadFromUrl(conversationUrl),
+  },
+];

studio/frontend/src/features/data-recipes/learning-recipes/instruction-from-answer.json

@@ -0,0 +1,194 @@
+{
+  "recipe": {
+    "model_providers": [
+      {
+        "name": "openai_provider",
+        "endpoint": "https://openrouter.ai/api/v1",
+        "provider_type": "openai",
+        "extra_headers": {},
+        "extra_body": {}
+      }
+    ],
+    "mcp_providers": [],
+    "model_configs": [
+      {
+        "alias": "ministral",
+        "model": "mistralai/ministral-8b-2512",
+        "provider": "openai_provider",
+        "inference_parameters": {
+          "temperature": 0.7,
+          "max_tokens": 1024
+        }
+      }
+    ],
+    "seed_config": {
+      "source": {
+        "seed_type": "hf",
+        "path": "datasets/unsloth/alpaca-cleaned/**/*.json",
+        "token": null,
+        "endpoint": "https://huggingface.co"
+      },
+      "sampling_strategy": "ordered",
+      "selection_strategy": {
+        "start": 1,
+        "end": 100
+      }
+    },
+    "tool_configs": [],
+    "columns": [
+      {
+        "column_type": "llm-text",
+        "name": "generated_instruction",
+        "drop": false,
+        "model_alias": "ministral",
+        "prompt": "Based on this target answer:\n{{ output }}\n\nWrite one high-quality plain text short and brief user instruction that this answer would satisfy.\nReturn only the instruction.",
+        "with_trace": "none"
+      }
+    ],
+    "processors": []
+  },
+  "run": {
+    "rows": 5,
+    "preview": true,
+    "output_formats": [
+      "jsonl"
+    ]
+  },
+  "ui": {
+    "nodes": [
+      {
+        "id": "note_1",
+        "x": -567.3566303099885,
+        "y": 38.88875727651093,
+        "width": 400,
+        "node_type": "markdown_note",
+        "name": "note_1",
+        "markdown": "#### Hugginface seed block\nThis recipe uses a ** HuggingFace dataset ** as seed data.\nYou provide dataset identity, load columns, then generate new fields from seed columns.\n\n##### Setup:\n\n1. Paste dataset id as `org/repo` (example: `unsloth/alpaca-cleaned`)\n2. Add token only if dataset is gated/private\n3. Load columns + preview rows so variables are available in prompts\n\n##### Why this matters:\n- Seed columns can drive generation quality\n- You can reference seed values directly in prompts (for example `{{ output }}`)",
+        "note_color": "#DCFCE7",
+        "note_opacity": "35"
+      },
+      {
+        "id": "note_2",
+        "x": -74.04047072330651,
+        "y": -265.3540670633283,
+        "width": 400,
+        "node_type": "markdown_note",
+        "name": "note_2",
+        "markdown": "##### Drop columns behavior:\n\n- You can mark specific seed columns to **drop from final output**\n- Those columns are still used during generation\n- They are removed only from exported final dataset\n\n##### Example:\n- Keep `generated_instruction` from llm-text block\n- Drop original `instruction`, `input`, `output` from the hugginface dataset from final artifact\n- Result: clean training output while still using source columns as generation context\n",
+        "note_color": "#DCFCE7",
+        "note_opacity": "35"
+      },
+      {
+        "id": "seed",
+        "x": -76.07288662013991,
+        "y": 143.39449780463954,
+        "width": 400
+      },
+      {
+        "id": "openai_provider",
+        "x": 461.00000000000006,
+        "y": -489.8750000000001,
+        "width": 400
+      },
+      {
+        "id": "ministral",
+        "x": 463.272022949692,
+        "y": -191.13601147484601,
+        "width": 400
+      },
+      {
+        "id": "generated_instruction",
+        "x": 464,
+        "y": 109.00000000000003,
+        "width": 400
+      }
+    ],
+    "edges": [
+      {
+        "from": "seed",
+        "to": "generated_instruction",
+        "type": "canvas",
+        "source_handle": "data-out",
+        "target_handle": "data-in"
+      },
+      {
+        "from": "ministral",
+        "to": "generated_instruction",
+        "type": "semantic",
+        "source_handle": "semantic-out-bottom",
+        "target_handle": "data-in-top"
+      },
+      {
+        "from": "openai_provider",
+        "to": "ministral",
+        "type": "semantic",
+        "source_handle": "semantic-out-bottom",
+        "target_handle": "semantic-in-top"
+      }
+    ],
+    "layout_direction": "LR",
+    "seed_source_type": "hf",
+    "seed_columns": [
+      "instruction",
+      "input",
+      "output"
+    ],
+    "seed_drop_columns": [],
+    "seed_preview_rows": [
+      {
+        "instruction": "Give three tips for staying healthy.",
+        "input": "",
+        "output": "1. Eat a balanced and nutritious diet: Make sure your meals are inclusive of a variety of fruits and vegetables, lean protein, whole grains, and healthy fats. This helps to provide your body with the essential nutrients to function at its best and can help prevent chronic diseases.\n\n2. Engage in regular physical activity: Exercise is crucial for maintaining strong bones, muscles, and cardiovascular health. Aim for at least 150 minutes of moderate aerobic exercise or 75 minutes of vigorous exercise each week.\n\n3. Get enough sleep: Getting enough quality sleep is crucial for physical and mental well-being. It helps to regulate mood, improve cognitive function, and supports healthy growth and immune function. Aim for 7-9 hours of sleep each night."
+      },
+      {
+        "instruction": "What are the three primary colors?",
+        "input": "",
+        "output": "The three primary colors are red, blue, and yellow. These colors are called primary because they cannot be created by mixing other colors and all other colors can be made by combining them in various proportions. In the additive color system, used for light, the primary colors are red, green, and blue (RGB)."
+      },
+      {
+        "instruction": "Describe the structure of an atom.",
+        "input": "",
+        "output": "An atom is the basic building block of all matter and is made up of three types of particles: protons, neutrons, and electrons. The structure of an atom can be described as a nucleus at the center surrounded by a cloud of electrons.\n\nThe nucleus of an atom is made up of protons and neutrons. Protons are positively charged particles and neutrons are neutral particles with no charge. Both of these particles are located in the nucleus of the atom, which is at the center of the atom and contains most of the atom's mass.\n\nSurrounding the nucleus of the atom is a cloud of electrons. Electrons are negatively charged particles that are in constant motion around the nucleus. The electron cloud is divided into shells or orbitals, and each shell can hold a certain number of electrons. The number of electrons in the outermost shell, called the valence shell, determines the chemical properties of the atom. \n\nIn a neutral atom, the number of protons in the nucleus is equal to the number of electrons in the electron cloud, so the positive and negative charges balance out and the atom has no overall charge. The number of protons, also called the atomic number, determines what element the atom is."
+      },
+      {
+        "instruction": "How can we reduce air pollution?",
+        "input": "",
+        "output": "There are several ways to reduce air pollution, including:\n\n1. Reduce energy consumption: By conserving energy, we reduce the amount of pollution emitted from power plants. Some ways to achieve this include: using energy-efficient appliances and lighting, insulating homes, and shutting off lights and electronics when not in use.\n\n2. Use public transportation or carpool: Cars and trucks are major sources of pollution. By using public transportation, carpooling or biking, you can reduce the number of cars on the road, which in turn reduces emissions.\n\n3. Be responsible with waste: When waste is burned or decomposed, they can release harmful gases into the air. Dispose of hazardous waste correctly and recycle materials like plastics, paper, and glass when possible.\n\n4. Support clean energy sources: Renewable energy sources like solar and wind power generate little or no pollution. Support clean energy companies and projects, and consider installing renewable energy systems in your home.\n\n5. Drive efficiently: If you need to drive, make sure your vehicle is well maintained and drive efficiently by avoiding rapid acceleration and braking, and reducing idling.\n\n6. Be mindful of product choices: Some products and practices release more pollution than others. Choose products with eco-friendly packaging and try to minimize your use of aerosol sprays, which can release harmful chemicals into the air."
+      },
+      {
+        "instruction": "Pretend you are a project manager of a construction company. Describe a time when you had to make a difficult decision.",
+        "input": "",
+        "output": "I had to make a difficult decision when I was working as a project manager at a construction company. I was in charge of a project that needed to be completed by a certain date in order to meet the client’s expectations. However, due to unexpected delays, we were not able to meet the deadline and so I had to make a difficult decision. I decided to extend the deadline, but I had to stretch the team’s resources even further and increase the budget. Although it was a risky decision, I ultimately decided to go ahead with it to ensure that the project was completed on time and that the client’s expectations were met. The project was eventually successfully completed and this was seen as a testament to my leadership and decision-making abilities."
+      },
+      {
+        "instruction": "Write a concise summary of the following:\n\"Commodore 64 (commonly known as the C64 or CBM 64) was manufactured by Commodore Business Machine (CBM) in August 1982 with a starting price of $595. It was an 8-bit home computer with remarkable market success. Between 1983-1986, C64 sales amounted to about 17 million units sold, making them the best-selling single personal computer model of all time in 1983-1986. \n\nAdditionally, the Commodore 64 dominated the market with between 30% and 40% share and 2 million units sold per year, outselling the IBM PC clones, Apple Computers, and Atari computers. Adding to their success, Sam Tramiel (former Atari president), during an interview in 1989, said they were building 400,000 C64s a month for a couple of years. \"",
+        "input": "",
+        "output": "The Commodore 64 was a highly successful 8-bit home computer manufactured by Commodore Business Machine (CBM) in 1982, with sales amounting to approximately 17 million units sold between 1983-1986. It dominated the market with between 30% and 40% share and outsold its competitors, including IBM PC clones, Apple Computers, and Atari computers. At its peak, CBM was building 400,000 C64s a month for a couple of years."
+      },
+      {
+        "instruction": "Explain why the following fraction is equivalent to 1/4",
+        "input": "4/16",
+        "output": "The fraction 4/16 is equivalent to 1/4 because both fractions represent the same value. A fraction can be simplified by dividing both the numerator and the denominator by a common factor. In this case, 4 is a common factor of both the numerator and the denominator of 4/16. When we divide both by 4, we get 4/4 = 1 and 16/4 = 4, so the simplified fraction is 1/4. Alternatively, we can think of this in terms of multiplication. For example, if we multiply the numerator and denominator of the fraction 1/4 by 4, we get (1x4)/(4x4), or 4/16. Since both fractions can be derived from the other through multiplication or division by the same number, they represent the same value and are equivalent."
+      },
+      {
+        "instruction": "Write a short story in third person narration about a protagonist who has to make an important career decision.",
+        "input": "",
+        "output": "Sophie sat at her desk, staring blankly at the computer screen. Her mind was racing as she weighed the options in front of her. She had been offered a promotion to a senior management position within her company, but it meant moving across the country and leaving her friends and family behind. At the same time, her dream job at a rival company had finally opened up, and she had been invited for an interview. \n\nThe thought of leaving her comfortable life behind, to take a chance on something new and uncertain, made Sophie feel anxious, but also excited. She had always been one to take risks, and it had paid off in the past. But now, with such a big decision to make, Sophie couldn't help but feel overwhelmed.\n\nAfter much deliberation, Sophie made her choice. She packed her bags, said goodbye to her loved ones, and moved across the country for the new job. It was a difficult first few months, adjusting to a new city and a new company culture. But as time went by, Sophie realized that she had made the best decision for her career. The job was challenging, but also rewarding, and she was surrounded by passionate, driven colleagues who inspired her to grow and develop.\n\nIn the end, Sophie's bold move paid off. She was thriving in her new role, and had made new friends and connections in her new city. Looking back, Sophie was grateful for the opportunity to take a risk and make a change, and was confident that she had made the right decision for her career."
+      },
+      {
+        "instruction": "Evaluate this sentence for spelling and grammar mistakes",
+        "input": "He finnished his meal and left the resturant",
+        "output": "There are two spelling errors in the sentence. The corrected sentence should be: \"He finished his meal and left the restaurant.\""
+      },
+      {
+        "instruction": "How did Julius Caesar die?",
+        "input": "",
+        "output": "Julius Caesar, the Roman Military general, and statesman was assassinated on March 15, 44 BC. He was stabbed to death at a meeting of the Senate in the Theatre of Pompey by a group of senators who called themselves the Liberatores. Led by Gaius Cassius Longinus, and Marcus Junius Brutus, they believed that Caesar's dictatorship was subverting the Roman Republic and sought to restore the traditional republican government. The exact number of assailants is not known, but it is believed that a group of as many as 60 conspirators participated in the assassination, and each of them took turns in stabbing Caesar. The day of his death is still known as the Ides of March."
+      }
+    ],
+    "local_file_name": "",
+    "unstructured_file_name": "",
+    "unstructured_chunk_size": "1200",
+    "unstructured_chunk_overlap": "200"
+  }
+}

studio/frontend/src/features/data-recipes/learning-recipes/pdf-grounded-qa.json

@@ -0,0 +1,197 @@
+{
+  "recipe": {
+    "model_providers": [
+      {
+        "name": "provider_1",
+        "endpoint": "https://openrouter.ai/api/v1",
+        "provider_type": "openai",
+        "extra_headers": {},
+        "extra_body": {}
+      }
+    ],
+    "mcp_providers": [],
+    "model_configs": [
+      {
+        "alias": "provider_column",
+        "model": "mistralai/ministral-8b-2512",
+        "provider": "provider_1",
+        "inference_parameters": {
+          "temperature": 0.7,
+          "max_tokens": 256
+        }
+      }
+    ],
+    "seed_config": {
+      "source": {
+        "seed_type": "local",
+        "path": "/home/wasim/.cache/unsloth/data-recipe/seed-uploads/dc8ef7c169ed43399b7d37164495150f_50 page sample PDF.indd.jsonl"
+      },
+      "sampling_strategy": "ordered",
+      "selection_strategy": null
+    },
+    "tool_configs": [],
+    "columns": [
+      {
+        "column_type": "llm-structured",
+        "name": "llm_structured_1",
+        "drop": false,
+        "model_alias": "provider_column",
+        "prompt": "Given ONLY this chunk: {{ chunk_text }} generate one answerable question, answer, and exact supporting quote from chunk. If not answerable, skip.",
+        "output_format": {
+          "type": "object",
+          "additionalProperties": false,
+          "required": [
+            "question",
+            "answer",
+            "evidence_quote"
+          ],
+          "properties": {
+            "question": {
+              "type": "string"
+            },
+            "answer": {
+              "type": "string"
+            },
+            "evidence_quote": {
+              "type": "string"
+            }
+          }
+        }
+      }
+    ],
+    "processors": []
+  },
+  "run": {
+    "rows": 5,
+    "preview": true,
+    "output_formats": [
+      "jsonl"
+    ]
+  },
+  "ui": {
+    "nodes": [
+      {
+        "id": "note_1",
+        "x": -180.01113025994076,
+        "y": 43.27382247773167,
+        "width": 400,
+        "node_type": "markdown_note",
+        "name": "note_1",
+        "markdown": "This recipe uses **seed data** from external documents.\nInstead of starting from empty generation, we load real source text first.\n\nIn this flow, the seed source is **Unstructured Documents**:\n\n- Upload: `.pdf`, `.docx`, `.txt`\n- Text is extracted and split on client into chunks\n- Each chunk becomes a row-like seed record (`chunk_text`) that you can reference in prompts with `{{ chunk_text }} `",
+        "note_color": "#F3E8FF",
+        "note_opacity": "35"
+      },
+      {
+        "id": "note_2",
+        "x": 283.8131688769869,
+        "y": -333.10847089567505,
+        "width": 400,
+        "node_type": "markdown_note",
+        "name": "note_2",
+        "markdown": "##### Chunking settings:\n\n- **Chunk size**: how much text per chunk\n- **Chunk overlap**: shared text between neighboring chunks to preserve context\n\n##### Sampling settings:\n\n- **Ordered**: keep original document order\n- **Shuffle**: randomize chunk order\n- **Selection index / selection settings**: choose which part/subset of seed data to use",
+        "note_color": "#F3E8FF",
+        "note_opacity": "35"
+      },
+      {
+        "id": "note_3",
+        "x": 303.52241671566657,
+        "y": 299.62272507131615,
+        "width": 400,
+        "node_type": "markdown_note",
+        "name": "note_3",
+        "markdown": "- LLM prompt: `{{ chunk_text }}`\n- Expression block: combine/format values using `{{ chunk_text }}`\n- Processor templates: use `{{ chunk_text }}` during transforms\n\nTip:\n- Start with medium chunk size + small overlap.\n- Increase overlap only if answers lose context between chunks.",
+        "note_color": "#F3E8FF",
+        "note_opacity": "35"
+      },
+      {
+        "id": "seed",
+        "x": 295.56977201312833,
+        "y": 108.19964868337735,
+        "width": 400
+      },
+      {
+        "id": "provider_1",
+        "x": 960.0115722892822,
+        "y": -465.06410256410254,
+        "width": 400
+      },
+      {
+        "id": "provider_column",
+        "x": 959.90231990232,
+        "y": -180.56654456654456,
+        "width": 400
+      },
+      {
+        "id": "llm_structured_1",
+        "x": 960,
+        "y": 108.25,
+        "width": 400
+      }
+    ],
+    "edges": [
+      {
+        "from": "seed",
+        "to": "llm_structured_1",
+        "type": "canvas",
+        "source_handle": "data-out",
+        "target_handle": "data-in"
+      },
+      {
+        "from": "provider_1",
+        "to": "provider_column",
+        "type": "semantic",
+        "source_handle": "semantic-out-bottom",
+        "target_handle": "semantic-in-top"
+      },
+      {
+        "from": "provider_column",
+        "to": "llm_structured_1",
+        "type": "semantic",
+        "source_handle": "semantic-out-bottom",
+        "target_handle": "data-in-top"
+      }
+    ],
+    "layout_direction": "LR",
+    "seed_source_type": "unstructured",
+    "seed_columns": [
+      "chunk_text"
+    ],
+    "seed_drop_columns": [],
+    "seed_preview_rows": [
+      {
+        "chunk_text": "[Citation Needed] The Best of Wikipedia’s Worst Writing Conor Lastowka and Josh Fruhlinger Boring Legal Fine Print Each entry in this book contains material from Wikipedia, although the text we use may not represent the current version of any article. The URL at the bottom of each page will direct you to the source Wikipedia article; use the article’s History tab to find a list of contributors. All material in this book that is taken from Wikipedia is licensed under the Creative Commons-Attribution Share Alike 3.0 license. Here’s a quick human-readable summary of your rights to use this content: You are free: to Share—to copy, distribute and transmit the work, and to Remix—to adapt the work Under the following conditions: Attribution—You must attribute the work in the manner specified by the author or licensor (but not in any way that suggests that they endorse you or your use of the work.) Share Alike—If you alter, transform, or build upon this work, you may distribute the resulting work only under the same, similar or a compatible license. With the understanding that: Waiver—Any of the above conditions can be waived if you get permission from the copyright holder. Other Rights—In"
+      },
+      {
+        "chunk_text": "only under the same, similar or a compatible license. With the understanding that: Waiver—Any of the above conditions can be waived if you get permission from the copyright holder. Other Rights—In no way are any of the following rights affected by the license: your fair dealing or fair use rights; the author’s moral rights; and rights other persons may have either in the work itself or in how the work is used, such as publicity or privacy rights. Notice—For any reuse or distribution, you must make clear to others the license terms of this work. The best way to do this is with a link to: http://creativecommons.org/licenses/by-sa/3.0/ Italicized material beneath each Wikipedia entry is © 2011 Conor Lastowka and Josh Fruhlinger. Copy-edited by Lauren Lastowka Cover design by Jaime Robinson ISBN # 978-1466346987 This book is dedicated to every person who wrote an entry that appears in it. May your citations always be needed. 6 Introduction Wikipedia. Whether you’ve used it to settle an argument, plagiarized a history report from it, or simply replaced the entire text of the biography of a respected humanitarian with the single word “dogballs,” it’s an inescapable part of the Internet"
+      },
+      {
+        "chunk_text": "plagiarized a history report from it, or simply replaced the entire text of the biography of a respected humanitarian with the single word “dogballs,” it’s an inescapable part of the Internet experience. Since its launch in 2001, it has rapidly risen to become the seventh most popular website, with over 365 million readers (Source: Wikipedia). If you’re like us, when you want to know the name of the kangaroo on Shirt Tales or just want to confirm that Mother Teresa was a dogballs who helped the farts (Source: Wikipedia), The Encyclopedia That Anyone Can Edit will probably be the first place you check. But here’s the thing about letting anybody edit your encyclopedia: it means that anybody can edit your encyclopedia. And while in theory this means that one day Stephen Hawking might decide to weigh in on the entry for string theory, in reality it means that somebody who deeply cares about pro wrestling is going to call someone else a Nazi when they revert his edits about Wrestlemania XI on Razor Ramon’s page. And so we arrive at a cosmic intersection, where an obscure topic of dubious relevance is written about by the type of weirdo who logs on to Wikipedia to write about obscure"
+      },
+      {
+        "chunk_text": "XI on Razor Ramon’s page. And so we arrive at a cosmic intersection, where an obscure topic of dubious relevance is written about by the type of weirdo who logs on to Wikipedia to write about obscure topics of dubious relevance. Were these authors re-watching their video of Wrestlemania XI instead of completing basic 8th grade English assignments? It’s very likely. Does this 7 stop them from attempting to emulate the academic tone of the great encyclopedias of the past as they describe a large mammalian species from the Star Wars universe that shares a common ancestor with the Wookies? It does not. The result? Some really terrible Wikipedia writing. For the past two years, we have collected this writing on our blog, [Citation Needed]. Fascinated and delighted by the brilliantly bad writing we encountered in our Wikipedia browsing, we set out to curate The Best of Wikipedia’s Worst Writing. Starting the blog was a no-brainer; our only concern was whether, after a few months of our daily mining, the well of awful Wikipedia writing would eventually run dry. By the time you read this, we will have published our thousandth entry. We started a podcast. Instead of drying up, the ocean of"
+      },
+      {
+        "chunk_text": "mining, the well of awful Wikipedia writing would eventually run dry. By the time you read this, we will have published our thousandth entry. We started a podcast. Instead of drying up, the ocean of ineptitude has proven far more vast than we ever could have imagined. Through our own browsing, and with the help of a dedicated group of readers who are exploring the topics they submit for God knows what reason, we’ve continually lowered and re-lowered the bar for bad Wikipedia writing. Now, let’s get one thing straight: we love each and every entry written in this book. If you are one of the authors who have chosen to use your valuable time on this planet to write straight-faced exegeses on the subject of forgotten action figures from the seventies, we hope you don’t take offense. And if you do, we have an acceptable retort prepared for you: “You guys ran a blog about Wikipedia for two years, who the hell are you to talk?” Feel free to use it! Others may criticize us for not doing our part to help Wikipedia become “better” by revising these passages. Nothing that does not involve electrodes near our genitals would make us more miserable. In our opinion, many of the passages in this"
+      },
+      {
+        "chunk_text": "to help Wikipedia become “better” by revising these passages. Nothing that does not involve electrodes near our genitals would make us more miserable. In our opinion, many of the passages in this book stand alone as works of art. Think of us as photographers preserving the memory of the great street art of the world before the joyless police come and whitewash over it. (Is that an official police responsibility? It seems beneath them. If it’s not, but they’re still forced to do it, that might explain the joylessness.) The point is, if you’re moved to correct these entries, we’re powerless to stop you. They’ve already given us joy, and we’re just happy to have encountered them. Enough introduction. Here are over two hundred of our favorite bad Wikipedia articles of all time. Comments in italics are ours. Everything else is a faithful reproduction of the way the entry stood at the moment we or our informants encountered it. We hope you will laugh, cry, maybe even learn something, and always remember to dogballs. —Conor Lastowka & Josh Fruhlinger citationneeded.tumblr.com 8 9 In barely one decade, Jimmy Wales has succeeded in establishing a worldwide network of knowledge. Wikipedia,"
+      },
+      {
+        "chunk_text": "remember to dogballs. —Conor Lastowka & Josh Fruhlinger citationneeded.tumblr.com 8 9 In barely one decade, Jimmy Wales has succeeded in establishing a worldwide network of knowledge. Wikipedia, his online encyclopaedia, accessible on the Internet for free, has become a symbol of a radical change in the media economy. Moreover, it revolutionized the access to knowledge as man’s most important resource and thus contributed to democratizing knowledge. The Gottlieb Duttweiler Institute, awarding the 2011 Gottlieb Duttweiler Prize to Wikipedia founder Jimmy Wales I saw the Beavis and Butt-Head episode that had Hogan’s “Real American” music on there. I don’t quite remembering it being critiqued by Beavis and Butt-Head. They sounded more like they liked the music and I don’t really remember any criticism of it (except for when it was going, when Butt-Head said “homework sucks”, but I’m not quite sure if he was referring to music or not). Wikipedia discussion page for Hulk Hogan 10 11 http://en.wikipedia.org/wiki/Polybius_(video_game) Want a Citation for this one? Please see the following 206 pages. The Roach story contained a number of inconsistencies: some of it seems to be directly"
+      },
+      {
+        "chunk_text": "Want a Citation for this one? Please see the following 206 pages. The Roach story contained a number of inconsistencies: some of it seems to be directly sourced from Wikipedia- all in all, an entirely untrustworthy source. Polybius (video game) 12 http://en.wikipedia.org/wiki/General_Mills_monster-themed_breakfast_cereals You can imagine the marketing team having their first meeting after the cereal’s release. “We have good news and bad news. The good news is, your latest cereal is very, very popular. The bad news is, it’s not in any way due to the character you came up with, the box design you slaved over, the costly ad campaign, or the hours you put in coming up with free toy ideas. Gentlemen, you should probably sit down....” Franken Berry was very popular when first introduced possibly because the initial batches of the cereal used a dye that didn’t break down in the body, causing many children’s feces to be bright pink, a symptom sometimes referred to as “Frankenberry Stool.” General Mills monster-themed breakfast cereals 13 http://en.wikipedia.org/wiki/Skiffle Because if your encyclopedia can’t provide you with an unsourced claim that it admits is only one of several"
+      },
+      {
+        "chunk_text": "General Mills monster-themed breakfast cereals 13 http://en.wikipedia.org/wiki/Skiffle Because if your encyclopedia can’t provide you with an unsourced claim that it admits is only one of several theories put forth about the subject, and then go on to inform you in the very same sentence that other unidentified parties disagree with that claim, then what the hell good is it? Skiffle is often said to have developed from New Orleans jazz, but this has been disputed. Skiffle 14 http://en.wikipedia.org/wiki/House_Party_%28film%29 “HP4: Coda or Mistake?” was by far the most contentious panel at HoPaCon 2009, with impassioned arguments echoing through the halls of the Kansas City International Airport Days Inn. In 2001, Immature (now going by IMx) starred in a direct-to-video sequel, House Party 4: Down to the Last Minute, which does not feature Kid or Play. The film is not considered a part of the House Party canon amongst fans. [citation needed] House Party (film) 15 http://en.wikipedia.org/wiki/Inglewood,_California So it would appear that, due to the presence of the anti- drug organization D.A.R.E. in Inglewood, there are in fact parts of Inglewood that are attempting to do good."
+      },
+      {
+        "chunk_text": "So it would appear that, due to the presence of the anti- drug organization D.A.R.E. in Inglewood, there are in fact parts of Inglewood that are attempting to do good. Thus the claim that Inglewood is “always” up to no good can be assumed to be false, or at the very least a gross exaggeration. Also, many have speculated that the so-called “Doctor” Dre never actually received a PhD. D.A.R.E. America has its headquarters in Inglewood. Despite this, in the 1996 rap hit “California Love”, Dr. Dre remarks that Inglewood is “always up to no good”. Inglewood, California 16 http://en.wikipedia.org/wiki/Tyler_Perry People who feel that Tyler Perry’s 6’5” stature is severely diminished by his wearing a wig? The line in the sand has been drawn. Another comical aspect is provided by Perry’s 6’- 5” stature, which is in no way diminished by his wearing a wig. Tyler Perry 17 http://en.wikipedia.org/wiki/Bondage_bed Do not attempt to affix your bondage partner to this question mark using chains and shackles! It is purely metaphorical! It is possible to buy inflatable bondage beds; however, a question mark must remain over how effective they are. Bondage bed 18"
+      }
+    ],
+    "local_file_name": "",
+    "unstructured_file_name": "50 page sample PDF.indd.pdf",
+    "unstructured_chunk_size": "1200",
+    "unstructured_chunk_overlap": "200"
+  }
+}