Combine studio setup fixes: frontend caching, venv isolation, Windows CPU support (#4413)
* Allow Windows setup to complete without NVIDIA GPU setup.ps1 previously hard-exited if nvidia-smi was not found, blocking setup entirely on CPU-only or non-NVIDIA machines. The backend already supports CPU and MLX (Apple Silicon) in chat-only GGUF mode, and the Linux/Mac setup.sh handles missing GPUs gracefully. Changes: - Convert the GPU check from a hard exit to a warning - Guard CUDA toolkit installation behind $HasNvidiaSmi - Install CPU-only PyTorch when no GPU is detected - Build llama.cpp without CUDA flags when no GPU is present - Update doc comment to reflect CPU support * Cache frontend build across setup runs Skip the frontend npm install + build if frontend/dist already exists. Previously setup.ps1 nuked node_modules and package-lock.json on every run, and both scripts always rebuilt even when dist/ was already present. On a git clone editable install, the first setup run still builds the frontend as before. Subsequent runs skip it, saving several minutes. To force a rebuild, delete frontend/dist and re-run setup. * Show pip progress for PyTorch download on Windows The torch CUDA wheel is ~2.8 GB and the CPU wheel is ~300 MB. With | Out-Null suppressing all output, the install appeared completely frozen with no feedback. Remove | Out-Null for the torch install lines so pip's download progress bar is visible. Add a size hint so users know the download is expected to take a while. Also moves the Triton success message inside the GPU branch so it only prints when Triton was actually installed. * Guard CUDA env re-sanitization behind GPU check in llama.cpp build The CUDA_PATH re-sanitization block (lines 1020-1033) references $CudaToolkitRoot which is only set when $HasNvidiaSmi is true and the CUDA Toolkit section runs. On CPU-only machines, $CudaToolkitRoot is null, causing Split-Path to throw: Split-Path : Cannot bind argument to parameter 'Path' because it is null. Wrap the entire block in `if ($HasNvidiaSmi -and $CudaToolkitRoot)`. * Rebuild frontend when source files are newer than dist/ Instead of only checking if dist/ exists, compare source file timestamps against the dist/ directory. If any file in frontend/src/ is newer than dist/, trigger a rebuild. This handles the case where a developer pulls new frontend changes and re-runs setup -- stale assets get rebuilt automatically. * Fix cmake not found on Windows after winget install Two issues fixed: 1. After winget installs cmake, Refresh-Environment may not pick up the new PATH entry (MSI PATH changes sometimes need a new shell). Added a fallback that probes cmake's default install locations (Program Files, LocalAppData) and adds the directory to PATH explicitly if found. 2. If cmake is still unavailable when the llama.cpp build starts (e.g. winget failed silently or PATH was not updated), the build now skips gracefully with a [SKIP] warning instead of crashing with "cmake : The term 'cmake' is not recognized". * Fix frontend rebuild detection and decouple oxc-validator install Address review feedback: - Check entire frontend/ directory for changes, not just src/. The build also depends on package.json, vite.config.ts, tailwind.config.ts, public/, and other config files. A change to any of these now triggers a rebuild. - Move oxc-validator npm install outside the frontend build gate in setup.sh so it always runs on setup, matching setup.ps1 which already had it outside the gate. * Show cmake errors on failure and retry CUDA VS integration with elevation Two fixes for issue #4405 (Windows setup fails at cmake configure): 1. cmake configure: capture output and display it on failure instead of piping to Out-Null. When the error mentions "No CUDA toolset found", print a hint about the CUDA VS integration files. 2. CUDA VS integration copy: when the direct Copy-Item fails (needs admin access to write to Program Files), retry with Start-Process -Verb RunAs to prompt for elevation. This is the root cause of the "No CUDA toolset found" cmake failure -- the .targets files that let MSBuild compile .cu files are missing from the VS BuildCustomizations directory. * Address reviewer feedback: cmake PATH persistence, stale cache, torch error check 1. Persist cmake PATH to user registry so Refresh-Environment cannot drop it later in the same setup run. Previously the process-only PATH addition at phase 1 could vanish when Refresh-Environment rebuilt PATH from registry during phase 2/3 installs. 2. Clean stale CMake cache before configure. If a previous run built with CUDA and the user reruns without a GPU (or vice versa), the cached GGML_CUDA value would persist. Now the build dir is removed before configure. 3. Explicitly set -DGGML_CUDA=OFF for CPU-only builds instead of just omitting CUDA flags. This prevents cmake from auto-detecting a partial CUDA installation. 4. Fix CUDA cmake flag indentation -- was misaligned from the original PR, now consistently indented inside the if/else block. 5. Fail hard if pip install torch returns a non-zero exit code instead of silently continuing with a broken environment. * Remove extra CUDA cmake flags to align Windows with Linux build Drop GGML_CUDA_FA_ALL_QUANTS, GGML_CUDA_F16, GGML_CUDA_GRAPHS, GGML_CUDA_FORCE_CUBLAS, and GGML_CUDA_PEER_MAX_BATCH_SIZE flags. The Linux build in setup.sh only sets GGML_CUDA=ON and lets llama.cpp use its defaults for everything else. Keep Windows consistent. * Address reviewer round 2: GPU probe fallback, Triton check, stale binary rebuild 1. GPU detection: fallback to default nvidia-smi install locations (Program Files\NVIDIA Corporation\NVSMI, System32) when nvidia-smi is not on PATH. Prevents silent CPU-only provisioning on machines that have a GPU but a broken PATH. 2. Triton: check $LASTEXITCODE after pip install and print [WARN] on failure instead of unconditional [OK]. 3. Stale llama-server: check CMakeCache.txt for GGML_CUDA setting and rebuild if the existing binary does not match the current GPU mode (e.g. CUDA binary on a now-CPU-only rerun, or vice versa). * Fix frontend rebuild detection and npm dependency issues Addresses reviewer feedback on the frontend caching logic: 1. setup.sh: Fix broken find command that caused exit under pipefail. The piped `find | xargs find -newer` had paths after the expression which GNU find rejects. Replaced with a simpler `find -maxdepth 1 -type f -newer dist/` that checks ALL top-level files (catches index.html, bun.lock, etc. that the extension allowlist missed). 2. setup.sh: Guard oxc-validator npm install behind `command -v npm` check. When the frontend build is skipped (dist/ is cached), Node bootstrap is also skipped, so npm may not be available. 3. setup.ps1: Replace Get-ChildItem -Include with explicit path probing for src/ and public/. PowerShell's -Include without a trailing wildcard silently returns nothing, so src/public changes were never detected. Also check ALL top-level files instead of just .json/.ts/.js/.mjs extensions. * Fix studio setup: venv isolation, centralized .venv_t5, uv targeting - All platforms (including Colab) now create ~/.unsloth/studio/.venv with --without-pip fallback for broken ensurepip environments - Add --python sys.executable to uv pip install in install_python_stack.py so uv targets the correct venv instead of system Python - Centralize .venv_t5 bootstrap in transformers_version.py with proper validation (checks required packages exist, not just non-empty dir) - Replace ~150 lines of duplicated install code across 3 worker files with calls to the shared _ensure_venv_t5_exists() helper - Use uv-if-present with pip fallback; do not install uv at runtime - Add site.addsitedir() shim in colab.py so notebook cells can import studio packages from the venv without system-Python double-install - Update .venv_t5 packages: huggingface_hub 1.3.0->1.7.1, add hf_xet - Bump transformers pin 4.57.1->4.57.6 in requirements + constraints - Add Fast-Install helper to setup.ps1 with uv+pip fallback - Keep Colab-specific completion banner in setup.sh * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix nvidia-smi PATH persistence and cmake requirement for CPU-only 1. Store nvidia-smi as an absolute path ($NvidiaSmiExe) on first detection. All later calls (Get-CudaComputeCapability, Get-PytorchCudaTag, CUDA toolkit detection) use this absolute path instead of relying on PATH. This survives Refresh-Environment which rebuilds PATH from the registry and drops process-only additions. 2. Make cmake fatal for CPU-only installs. CPU-only machines depend entirely on llama-server for GGUF chat mode, so reporting "Setup Complete!" without it is misleading. GPU machines can still skip the llama-server build since they have other inference paths. * Fix broken frontend freshness detection in setup scripts - setup.sh: Replace broken `find | xargs find -newer` pipeline with single `find ... -newer` call. The old pipeline produced "paths must precede expression" errors (silently suppressed by 2>/dev/null), causing top-level config changes to never trigger a rebuild. - setup.sh: Add `command -v npm` guard to oxc-validator block so it does not fail when Node was not installed (build-skip path). - setup.ps1: Replace `Get-ChildItem -Include` (unreliable without -Recurse on PS 5.1) with explicit directory paths for src/ and public/ scanning. - Both: Add *.html to tracked file patterns so index.html (Vite entry point) changes trigger a rebuild. - Both: Use -print -quit instead of piping to head -1 for efficiency. * Fix bugs found during review of PRs #4404, #4400, #4399 - setup.sh: Add || true guard to find command that checks frontend/src and frontend/public dirs, preventing script abort under set -euo pipefail when either directory is missing - colab.py: Use sys.path.insert(0, ...) instead of site.addsitedir() so Studio venv packages take priority over system copies. Add warning when venv is missing instead of silently failing. - transformers_version.py: _venv_t5_is_valid() now checks installed package versions via .dist-info metadata, not just directory presence. Prevents false positives from stale or wrong-version packages. - transformers_version.py: _install_to_venv_t5() now passes --upgrade so pip replaces existing stale packages in the target directory. - setup.ps1: CPU-only PyTorch install uses --index-url for cpu wheel and all install commands use Fast-Install (uv with pip fallback). * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix _venv_t5_is_valid dist-info loop exiting after first directory Remove premature break that caused the loop over .dist-info directories to exit after the first match even if it had no METADATA file. Now continues iterating until a valid METADATA is found or all dirs are exhausted. * Capture error output on failure instead of discarding with Out-Null setup.ps1: 6 locations changed from `| Out-Null` to `| Out-String` with output shown on failure -- PyTorch GPU/CPU install, Triton install, venv_t5 package loop, cmake llama-server and llama-quantize builds. transformers_version.py: clean stale .venv_t5 directory before reinstall when validation detects missing or version-mismatched packages. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix ModuleNotFoundError when CLI imports studio.backend.core The backend uses bare "from utils.*" imports everywhere, relying on backend/ being on sys.path. Workers and routes add it at startup, but the CLI imports studio.backend.core as a package -- backend/ was never added. Add sys.path setup at the top of core/__init__.py so lazy imports resolve correctly regardless of entry point. Fixes: unsloth inference unsloth/Qwen3-8B "who are you" crashing with "No module named 'utils'" * Fix frontend freshness check to detect all top-level file changes The extension allowlist (*.json, *.ts, *.js, *.mjs, *.html) missed files like bun.lock, so lockfile-only dependency changes could skip the frontend rebuild. Check all top-level files instead. * Add tiktoken to .venv_t5 for Qwen-family tokenizers Qwen models use tiktoken-based tokenizers which fail when routed through the transformers 5.x overlay without tiktoken installed. Add it to the setup scripts (with deps for Windows) and runtime fallback list. Integrates PR #4418. * Fix tiktoken crash in _venv_t5_is_valid and stray brace in setup.ps1 _venv_t5_is_valid() crashed with ValueError on unpinned packages like "tiktoken" (no ==version). Handle by splitting safely and skipping version check for unpinned packages (existence check only). Also remove stray closing brace in setup.ps1 tiktoken install block. --------- Co-authored-by: Daniel Han <danielhanchen@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
This commit is contained in:
Daniel Han
GitHub
parent
7b07ad0fa3
commit
1f12ba16df
11 changed files with 579 additions and 296 deletions
|
|
@ -9,6 +9,32 @@ Uses Colab's built-in proxy - no external tunneling needed!
|
|||
from pathlib import Path
|
||||
import sys
|
||||
|
||||
|
||||
def _bootstrap_studio_venv() -> None:
|
||||
"""Expose the Studio venv's site-packages to the current interpreter.
|
||||
|
||||
On Colab, notebook cells run outside the venv subshell. Instead of
|
||||
installing the full stack into system Python, we prepend the venv's
|
||||
site-packages so that packages like structlog, fastapi, etc. are
|
||||
importable from notebook cells and take priority over system copies.
|
||||
"""
|
||||
venv_lib = Path.home() / ".unsloth" / "studio" / ".venv" / "lib"
|
||||
if not venv_lib.exists():
|
||||
import warnings
|
||||
|
||||
warnings.warn(
|
||||
f"Studio venv not found at {venv_lib.parent} -- run 'unsloth studio setup' first",
|
||||
stacklevel = 2,
|
||||
)
|
||||
return
|
||||
for sp in venv_lib.glob("python*/site-packages"):
|
||||
sp_str = str(sp)
|
||||
if sp_str not in sys.path:
|
||||
sys.path.insert(0, sp_str)
|
||||
|
||||
|
||||
_bootstrap_studio_venv()
|
||||
|
||||
# Add backend to path early so local modules like loggers can be imported
|
||||
backend_path = str(Path(__file__).parent)
|
||||
if backend_path not in sys.path:
|
||||
|
|
|
|||
|
|
@ -10,6 +10,16 @@ like unsloth, transformers, or torch before the version activation
|
|||
code has a chance to run.
|
||||
"""
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
# Ensure the backend directory is on sys.path so that bare "from utils.*"
|
||||
# imports used throughout the backend work when core is imported as a package
|
||||
# (e.g. from the CLI: "from studio.backend.core import ModelConfig").
|
||||
_backend_dir = str(Path(__file__).resolve().parent.parent)
|
||||
if _backend_dir not in sys.path:
|
||||
sys.path.insert(0, _backend_dir)
|
||||
|
||||
__all__ = [
|
||||
# Inference
|
||||
"InferenceBackend",
|
||||
|
|
|
|||
|
|
@ -40,59 +40,25 @@ def _activate_transformers_version(model_name: str) -> None:
|
|||
if backend_path not in sys.path:
|
||||
sys.path.insert(0, backend_path)
|
||||
|
||||
from utils.transformers_version import needs_transformers_5, _resolve_base_model
|
||||
from utils.transformers_version import (
|
||||
needs_transformers_5,
|
||||
_resolve_base_model,
|
||||
_ensure_venv_t5_exists,
|
||||
_VENV_T5_DIR,
|
||||
)
|
||||
|
||||
resolved = _resolve_base_model(model_name)
|
||||
if needs_transformers_5(resolved):
|
||||
venv_t5 = os.path.join(
|
||||
os.path.expanduser("~"), ".unsloth", "studio", ".venv_t5"
|
||||
)
|
||||
if os.path.isdir(venv_t5):
|
||||
sys.path.insert(0, venv_t5)
|
||||
logger.info("Activated transformers 5.x from %s", venv_t5)
|
||||
else:
|
||||
# Fallback: pip install at runtime (slower, ~10-15s)
|
||||
logger.warning(".venv_t5 not found at %s — installing at runtime", venv_t5)
|
||||
import subprocess as sp
|
||||
|
||||
os.makedirs(venv_t5, exist_ok = True)
|
||||
r1 = sp.run(
|
||||
[
|
||||
sys.executable,
|
||||
"-m",
|
||||
"pip",
|
||||
"install",
|
||||
"--target",
|
||||
venv_t5,
|
||||
"--no-deps",
|
||||
"transformers==5.3.0",
|
||||
],
|
||||
stdout = sp.PIPE,
|
||||
stderr = sp.STDOUT,
|
||||
if not _ensure_venv_t5_exists():
|
||||
raise RuntimeError(
|
||||
f"Cannot activate transformers 5.x: .venv_t5 missing at {_VENV_T5_DIR}"
|
||||
)
|
||||
r2 = sp.run(
|
||||
[
|
||||
sys.executable,
|
||||
"-m",
|
||||
"pip",
|
||||
"install",
|
||||
"--target",
|
||||
venv_t5,
|
||||
"--no-deps",
|
||||
"huggingface_hub==1.3.0",
|
||||
],
|
||||
stdout = sp.PIPE,
|
||||
stderr = sp.STDOUT,
|
||||
)
|
||||
if r1.returncode != 0 or r2.returncode != 0:
|
||||
raise RuntimeError(
|
||||
f"Failed to install transformers 5.x into {venv_t5}. "
|
||||
f"pip returncode: transformers={r1.returncode}, huggingface_hub={r2.returncode}"
|
||||
)
|
||||
sys.path.insert(0, venv_t5)
|
||||
if _VENV_T5_DIR not in sys.path:
|
||||
sys.path.insert(0, _VENV_T5_DIR)
|
||||
logger.info("Activated transformers 5.x from %s", _VENV_T5_DIR)
|
||||
# Propagate to child subprocesses (e.g. GGUF converter)
|
||||
_pp = os.environ.get("PYTHONPATH", "")
|
||||
os.environ["PYTHONPATH"] = venv_t5 + (os.pathsep + _pp if _pp else "")
|
||||
os.environ["PYTHONPATH"] = _VENV_T5_DIR + (os.pathsep + _pp if _pp else "")
|
||||
else:
|
||||
logger.info("Using default transformers (4.57.x) for %s", model_name)
|
||||
|
||||
|
|
|
|||
|
|
@ -42,59 +42,25 @@ def _activate_transformers_version(model_name: str) -> None:
|
|||
if backend_path not in sys.path:
|
||||
sys.path.insert(0, backend_path)
|
||||
|
||||
from utils.transformers_version import needs_transformers_5, _resolve_base_model
|
||||
from utils.transformers_version import (
|
||||
needs_transformers_5,
|
||||
_resolve_base_model,
|
||||
_ensure_venv_t5_exists,
|
||||
_VENV_T5_DIR,
|
||||
)
|
||||
|
||||
resolved = _resolve_base_model(model_name)
|
||||
if needs_transformers_5(resolved):
|
||||
venv_t5 = os.path.join(
|
||||
os.path.expanduser("~"), ".unsloth", "studio", ".venv_t5"
|
||||
)
|
||||
if os.path.isdir(venv_t5):
|
||||
sys.path.insert(0, venv_t5)
|
||||
logger.info("Activated transformers 5.x from %s", venv_t5)
|
||||
else:
|
||||
# Fallback: pip install at runtime (slower, ~10-15s)
|
||||
logger.warning(".venv_t5 not found at %s — installing at runtime", venv_t5)
|
||||
import subprocess as sp
|
||||
|
||||
os.makedirs(venv_t5, exist_ok = True)
|
||||
r1 = sp.run(
|
||||
[
|
||||
sys.executable,
|
||||
"-m",
|
||||
"pip",
|
||||
"install",
|
||||
"--target",
|
||||
venv_t5,
|
||||
"--no-deps",
|
||||
"transformers==5.3.0",
|
||||
],
|
||||
stdout = sp.PIPE,
|
||||
stderr = sp.STDOUT,
|
||||
if not _ensure_venv_t5_exists():
|
||||
raise RuntimeError(
|
||||
f"Cannot activate transformers 5.x: .venv_t5 missing at {_VENV_T5_DIR}"
|
||||
)
|
||||
r2 = sp.run(
|
||||
[
|
||||
sys.executable,
|
||||
"-m",
|
||||
"pip",
|
||||
"install",
|
||||
"--target",
|
||||
venv_t5,
|
||||
"--no-deps",
|
||||
"huggingface_hub==1.3.0",
|
||||
],
|
||||
stdout = sp.PIPE,
|
||||
stderr = sp.STDOUT,
|
||||
)
|
||||
if r1.returncode != 0 or r2.returncode != 0:
|
||||
raise RuntimeError(
|
||||
f"Failed to install transformers 5.x into {venv_t5}. "
|
||||
f"pip returncode: transformers={r1.returncode}, huggingface_hub={r2.returncode}"
|
||||
)
|
||||
sys.path.insert(0, venv_t5)
|
||||
if _VENV_T5_DIR not in sys.path:
|
||||
sys.path.insert(0, _VENV_T5_DIR)
|
||||
logger.info("Activated transformers 5.x from %s", _VENV_T5_DIR)
|
||||
# Propagate to child subprocesses (e.g. GGUF converter)
|
||||
_pp = os.environ.get("PYTHONPATH", "")
|
||||
os.environ["PYTHONPATH"] = venv_t5 + (os.pathsep + _pp if _pp else "")
|
||||
os.environ["PYTHONPATH"] = _VENV_T5_DIR + (os.pathsep + _pp if _pp else "")
|
||||
else:
|
||||
logger.info("Using default transformers (4.57.x) for %s", model_name)
|
||||
|
||||
|
|
|
|||
|
|
@ -36,59 +36,25 @@ def _activate_transformers_version(model_name: str) -> None:
|
|||
if backend_path not in sys.path:
|
||||
sys.path.insert(0, backend_path)
|
||||
|
||||
from utils.transformers_version import needs_transformers_5, _resolve_base_model
|
||||
from utils.transformers_version import (
|
||||
needs_transformers_5,
|
||||
_resolve_base_model,
|
||||
_ensure_venv_t5_exists,
|
||||
_VENV_T5_DIR,
|
||||
)
|
||||
|
||||
resolved = _resolve_base_model(model_name)
|
||||
if needs_transformers_5(resolved):
|
||||
venv_t5 = os.path.join(
|
||||
os.path.expanduser("~"), ".unsloth", "studio", ".venv_t5"
|
||||
)
|
||||
if os.path.isdir(venv_t5):
|
||||
sys.path.insert(0, venv_t5)
|
||||
logger.info("Activated transformers 5.x from %s", venv_t5)
|
||||
else:
|
||||
# Fallback: pip install at runtime (slower, ~10-15s)
|
||||
logger.warning(".venv_t5 not found at %s — installing at runtime", venv_t5)
|
||||
import subprocess as sp
|
||||
|
||||
os.makedirs(venv_t5, exist_ok = True)
|
||||
r1 = sp.run(
|
||||
[
|
||||
sys.executable,
|
||||
"-m",
|
||||
"pip",
|
||||
"install",
|
||||
"--target",
|
||||
venv_t5,
|
||||
"--no-deps",
|
||||
"transformers==5.3.0",
|
||||
],
|
||||
stdout = sp.PIPE,
|
||||
stderr = sp.STDOUT,
|
||||
if not _ensure_venv_t5_exists():
|
||||
raise RuntimeError(
|
||||
f"Cannot activate transformers 5.x: .venv_t5 missing at {_VENV_T5_DIR}"
|
||||
)
|
||||
r2 = sp.run(
|
||||
[
|
||||
sys.executable,
|
||||
"-m",
|
||||
"pip",
|
||||
"install",
|
||||
"--target",
|
||||
venv_t5,
|
||||
"--no-deps",
|
||||
"huggingface_hub==1.3.0",
|
||||
],
|
||||
stdout = sp.PIPE,
|
||||
stderr = sp.STDOUT,
|
||||
)
|
||||
if r1.returncode != 0 or r2.returncode != 0:
|
||||
raise RuntimeError(
|
||||
f"Failed to install transformers 5.x into {venv_t5}. "
|
||||
f"pip returncode: transformers={r1.returncode}, huggingface_hub={r2.returncode}"
|
||||
)
|
||||
sys.path.insert(0, venv_t5)
|
||||
if _VENV_T5_DIR not in sys.path:
|
||||
sys.path.insert(0, _VENV_T5_DIR)
|
||||
logger.info("Activated transformers 5.x from %s", _VENV_T5_DIR)
|
||||
# Propagate to child subprocesses (e.g. GGUF converter)
|
||||
_pp = os.environ.get("PYTHONPATH", "")
|
||||
os.environ["PYTHONPATH"] = venv_t5 + (os.pathsep + _pp if _pp else "")
|
||||
os.environ["PYTHONPATH"] = _VENV_T5_DIR + (os.pathsep + _pp if _pp else "")
|
||||
else:
|
||||
logger.info("Using default transformers (4.57.x) for %s", model_name)
|
||||
|
||||
|
|
|
|||
|
|
@ -11,4 +11,4 @@ git+https://github.com/meta-pytorch/OpenEnv.git
|
|||
# executorch>=1.0.1 # 41.5 MB - no imports in unsloth/zoo/studio
|
||||
torch-c-dlpack-ext
|
||||
sentence_transformers==5.2.0
|
||||
transformers==4.57.1
|
||||
transformers==4.57.6
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
# Single-env pins for unsloth + studio + data-designer
|
||||
# Keep compatible with unsloth transformers bounds.
|
||||
transformers==4.57.1
|
||||
transformers==4.57.6
|
||||
trl==0.23.1
|
||||
huggingface-hub==0.36.2
|
||||
|
||||
|
|
|
|||
|
|
@ -26,6 +26,7 @@ import json
|
|||
import structlog
|
||||
from loggers import get_logger
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
|
@ -58,7 +59,7 @@ _tokenizer_class_cache: dict[str, bool] = {}
|
|||
|
||||
# Versions
|
||||
TRANSFORMERS_5_VERSION = "5.3.0"
|
||||
TRANSFORMERS_DEFAULT_VERSION = "4.57.1"
|
||||
TRANSFORMERS_DEFAULT_VERSION = "4.57.6"
|
||||
|
||||
# Pre-installed directory for transformers 5.x — created by setup.sh / setup.ps1
|
||||
_VENV_T5_DIR = str(Path.home() / ".unsloth" / "studio" / ".venv_t5")
|
||||
|
|
@ -216,15 +217,87 @@ def _purge_modules() -> int:
|
|||
return len(to_remove)
|
||||
|
||||
|
||||
def _ensure_venv_t5_exists() -> bool:
|
||||
"""Ensure .venv_t5/ exists. Install at runtime if missing."""
|
||||
if os.path.isdir(_VENV_T5_DIR) and os.listdir(_VENV_T5_DIR):
|
||||
return True
|
||||
_VENV_T5_PACKAGES = (
|
||||
f"transformers=={TRANSFORMERS_5_VERSION}",
|
||||
"huggingface_hub==1.7.1",
|
||||
"hf_xet==1.4.2",
|
||||
"tiktoken",
|
||||
)
|
||||
|
||||
logger.warning(".venv_t5 not found at %s — installing at runtime", _VENV_T5_DIR)
|
||||
os.makedirs(_VENV_T5_DIR, exist_ok = True)
|
||||
for pkg in (f"transformers=={TRANSFORMERS_5_VERSION}", "huggingface_hub==1.3.0"):
|
||||
cmd = [
|
||||
|
||||
def _venv_t5_is_valid() -> bool:
|
||||
"""Return True if .venv_t5/ has all required packages at the correct versions."""
|
||||
if not os.path.isdir(_VENV_T5_DIR) or not os.listdir(_VENV_T5_DIR):
|
||||
return False
|
||||
# Check that the key package directories exist AND match the required version
|
||||
for pkg_spec in _VENV_T5_PACKAGES:
|
||||
parts = pkg_spec.split("==")
|
||||
pkg_name = parts[0]
|
||||
pkg_version = parts[1] if len(parts) > 1 else None
|
||||
pkg_name_norm = pkg_name.replace("-", "_")
|
||||
# Check directory exists
|
||||
if not any(
|
||||
(Path(_VENV_T5_DIR) / d).is_dir()
|
||||
for d in (pkg_name_norm, pkg_name_norm.replace("_", "-"))
|
||||
):
|
||||
return False
|
||||
# For unpinned packages, existence is enough
|
||||
if pkg_version is None:
|
||||
continue
|
||||
# Check version via .dist-info metadata
|
||||
dist_info_found = False
|
||||
for di in Path(_VENV_T5_DIR).glob(f"{pkg_name_norm}-*.dist-info"):
|
||||
metadata = di / "METADATA"
|
||||
if not metadata.is_file():
|
||||
continue
|
||||
for line in metadata.read_text(errors = "replace").splitlines():
|
||||
if line.startswith("Version:"):
|
||||
installed_ver = line.split(":", 1)[1].strip()
|
||||
if installed_ver != pkg_version:
|
||||
logger.info(
|
||||
".venv_t5 has %s==%s but need %s",
|
||||
pkg_name,
|
||||
installed_ver,
|
||||
pkg_version,
|
||||
)
|
||||
return False
|
||||
dist_info_found = True
|
||||
break
|
||||
if dist_info_found:
|
||||
break
|
||||
if not dist_info_found:
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def _install_to_venv_t5(pkg: str) -> bool:
|
||||
"""Install a single package into .venv_t5/, preferring uv then pip."""
|
||||
# Try uv first (faster) if already on PATH -- do NOT install uv at runtime
|
||||
if shutil.which("uv"):
|
||||
result = subprocess.run(
|
||||
[
|
||||
"uv",
|
||||
"pip",
|
||||
"install",
|
||||
"--python",
|
||||
sys.executable,
|
||||
"--target",
|
||||
_VENV_T5_DIR,
|
||||
"--no-deps",
|
||||
"--upgrade",
|
||||
pkg,
|
||||
],
|
||||
stdout = subprocess.PIPE,
|
||||
stderr = subprocess.STDOUT,
|
||||
text = True,
|
||||
)
|
||||
if result.returncode == 0:
|
||||
return True
|
||||
logger.warning("uv install of %s failed, falling back to pip", pkg)
|
||||
|
||||
# Fallback to pip
|
||||
result = subprocess.run(
|
||||
[
|
||||
sys.executable,
|
||||
"-m",
|
||||
"pip",
|
||||
|
|
@ -232,13 +305,31 @@ def _ensure_venv_t5_exists() -> bool:
|
|||
"--target",
|
||||
_VENV_T5_DIR,
|
||||
"--no-deps",
|
||||
"--upgrade",
|
||||
pkg,
|
||||
]
|
||||
result = subprocess.run(
|
||||
cmd, stdout = subprocess.PIPE, stderr = subprocess.STDOUT, text = True
|
||||
)
|
||||
if result.returncode != 0:
|
||||
logger.error("pip install failed:\n%s", result.stdout)
|
||||
],
|
||||
stdout = subprocess.PIPE,
|
||||
stderr = subprocess.STDOUT,
|
||||
text = True,
|
||||
)
|
||||
if result.returncode != 0:
|
||||
logger.error("install failed:\n%s", result.stdout)
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def _ensure_venv_t5_exists() -> bool:
|
||||
"""Ensure .venv_t5/ exists with all required packages. Install if missing."""
|
||||
if _venv_t5_is_valid():
|
||||
return True
|
||||
|
||||
logger.warning(
|
||||
".venv_t5 not found or incomplete at %s -- installing at runtime", _VENV_T5_DIR
|
||||
)
|
||||
shutil.rmtree(_VENV_T5_DIR, ignore_errors = True)
|
||||
os.makedirs(_VENV_T5_DIR, exist_ok = True)
|
||||
for pkg in _VENV_T5_PACKAGES:
|
||||
if not _install_to_venv_t5(pkg):
|
||||
return False
|
||||
logger.info("Installed transformers 5.x to %s", _VENV_T5_DIR)
|
||||
return True
|
||||
|
|
|
|||
|
|
@ -140,15 +140,15 @@ def _bootstrap_uv() -> bool:
|
|||
global UV_NEEDS_SYSTEM
|
||||
if not shutil.which("uv"):
|
||||
return False
|
||||
# Probe: try a dry-run install without --system.
|
||||
# If uv can't find a venv it exits with code 2.
|
||||
# Probe: try a dry-run install targeting the current Python explicitly.
|
||||
# Without --python, uv can ignore the activated venv on some platforms.
|
||||
probe = subprocess.run(
|
||||
["uv", "pip", "install", "--dry-run", "pip"],
|
||||
["uv", "pip", "install", "--dry-run", "--python", sys.executable, "pip"],
|
||||
stdout = subprocess.PIPE,
|
||||
stderr = subprocess.STDOUT,
|
||||
)
|
||||
if probe.returncode != 0:
|
||||
# Retry with --system to confirm it works
|
||||
# Retry with --system (some envs need it when uv can't find a venv)
|
||||
probe_sys = subprocess.run(
|
||||
["uv", "pip", "install", "--dry-run", "--system", "pip"],
|
||||
stdout = subprocess.PIPE,
|
||||
|
|
@ -204,6 +204,10 @@ def _build_uv_cmd(args: tuple[str, ...]) -> list[str]:
|
|||
cmd = ["uv", "pip", "install"]
|
||||
if UV_NEEDS_SYSTEM:
|
||||
cmd.append("--system")
|
||||
# Always pass --python so uv targets the correct environment.
|
||||
# Without this, uv can ignore an activated venv and install into
|
||||
# the system Python (observed on Colab and similar environments).
|
||||
cmd.extend(["--python", sys.executable])
|
||||
cmd.extend(_translate_pip_args_for_uv(args))
|
||||
cmd.append("--torch-backend=auto")
|
||||
return cmd
|
||||
|
|
|
|||
375
studio/setup.ps1
375
studio/setup.ps1
|
|
@ -8,7 +8,7 @@
|
|||
Always installs Node.js if needed. When running from pip install:
|
||||
skips frontend build (already bundled). When running from git repo:
|
||||
full setup including frontend build.
|
||||
Requires an NVIDIA GPU -- CPU-only machines are not supported.
|
||||
Supports NVIDIA GPU (full training + inference) and CPU-only (GGUF chat mode).
|
||||
.NOTES
|
||||
Usage: powershell -ExecutionPolicy Bypass -File setup.ps1
|
||||
#>
|
||||
|
|
@ -107,11 +107,15 @@ function Find-Nvcc {
|
|||
# Returns e.g. "80" for A100 (8.0), "89" for RTX 4090 (8.9), etc.
|
||||
# Returns $null if detection fails.
|
||||
function Get-CudaComputeCapability {
|
||||
$nvSmi = Get-Command nvidia-smi -ErrorAction SilentlyContinue
|
||||
if (-not $nvSmi) { return $null }
|
||||
# Use the resolved absolute path ($NvidiaSmiExe) to survive Refresh-Environment
|
||||
$smiExe = if ($script:NvidiaSmiExe) { $script:NvidiaSmiExe } else {
|
||||
$cmd = Get-Command nvidia-smi -ErrorAction SilentlyContinue
|
||||
if ($cmd) { $cmd.Source } else { $null }
|
||||
}
|
||||
if (-not $smiExe) { return $null }
|
||||
|
||||
try {
|
||||
$raw = & nvidia-smi --query-gpu=compute_cap --format=csv,noheader 2>$null
|
||||
$raw = & $smiExe --query-gpu=compute_cap --format=csv,noheader 2>$null
|
||||
if ($LASTEXITCODE -ne 0 -or -not $raw) { return $null }
|
||||
|
||||
# nvidia-smi may return multiple GPUs; take the first one
|
||||
|
|
@ -168,14 +172,17 @@ function Get-NvccMaxArch {
|
|||
# https://download.pytorch.org/whl/<tag>. The tag must not exceed the driver's
|
||||
# capability: e.g. driver "CUDA Version: 12.9" → cu128 (not cu130).
|
||||
function Get-PytorchCudaTag {
|
||||
$nvSmi = Get-Command nvidia-smi -ErrorAction SilentlyContinue
|
||||
if (-not $nvSmi) { return "cu124" }
|
||||
$smiExe = if ($script:NvidiaSmiExe) { $script:NvidiaSmiExe } else {
|
||||
$cmd = Get-Command nvidia-smi -ErrorAction SilentlyContinue
|
||||
if ($cmd) { $cmd.Source } else { $null }
|
||||
}
|
||||
if (-not $smiExe) { return "cu124" }
|
||||
|
||||
try {
|
||||
# 2>&1 | Out-String merges stderr into stdout then converts to a single
|
||||
# string. Plain 2>$null doesn't fully suppress stderr in PS 5.1 —
|
||||
# string. Plain 2>$null doesn't fully suppress stderr in PS 5.1 --
|
||||
# ErrorRecord objects leak into $output and break the -match.
|
||||
$output = & nvidia-smi 2>&1 | Out-String
|
||||
$output = & $smiExe 2>&1 | Out-String
|
||||
if ($output -match 'CUDA Version:\s+(\d+)\.(\d+)') {
|
||||
$major = [int]$Matches[1]
|
||||
$minor = [int]$Matches[2]
|
||||
|
|
@ -251,23 +258,50 @@ Write-Host "+==============================================+" -ForegroundColor G
|
|||
# ==========================================================================
|
||||
|
||||
# ============================================
|
||||
# 1a. GPU requirement check
|
||||
# 1a. GPU detection
|
||||
# ============================================
|
||||
$HasNvidiaSmi = $false
|
||||
$NvidiaSmiExe = $null # Absolute path -- survives Refresh-Environment
|
||||
try {
|
||||
nvidia-smi 2>&1 | Out-Null
|
||||
if ($LASTEXITCODE -eq 0) { $HasNvidiaSmi = $true }
|
||||
$nvSmiCmd = Get-Command nvidia-smi -ErrorAction SilentlyContinue
|
||||
if ($nvSmiCmd) {
|
||||
& $nvSmiCmd.Source 2>&1 | Out-Null
|
||||
if ($LASTEXITCODE -eq 0) {
|
||||
$HasNvidiaSmi = $true
|
||||
$NvidiaSmiExe = $nvSmiCmd.Source
|
||||
}
|
||||
}
|
||||
} catch {}
|
||||
# Fallback: nvidia-smi may not be on PATH even though a GPU + driver exist.
|
||||
# Check the default install location and the Windows driver store.
|
||||
if (-not $HasNvidiaSmi) {
|
||||
$nvSmiDefaults = @(
|
||||
"$env:ProgramFiles\NVIDIA Corporation\NVSMI\nvidia-smi.exe",
|
||||
"$env:SystemRoot\System32\nvidia-smi.exe"
|
||||
)
|
||||
foreach ($p in $nvSmiDefaults) {
|
||||
if (Test-Path $p) {
|
||||
try {
|
||||
& $p 2>&1 | Out-Null
|
||||
if ($LASTEXITCODE -eq 0) {
|
||||
$HasNvidiaSmi = $true
|
||||
$NvidiaSmiExe = $p
|
||||
Write-Host " Found nvidia-smi at $(Split-Path $p -Parent)" -ForegroundColor Gray
|
||||
break
|
||||
}
|
||||
} catch {}
|
||||
}
|
||||
}
|
||||
}
|
||||
if (-not $HasNvidiaSmi) {
|
||||
Write-Host ""
|
||||
Write-Host "[ERROR] Unsloth Studio requires an NVIDIA GPU." -ForegroundColor Red
|
||||
Write-Host " CPU-only machines are not supported." -ForegroundColor Red
|
||||
Write-Host "[WARN] No NVIDIA GPU detected. Studio will run in chat-only (GGUF) mode." -ForegroundColor Yellow
|
||||
Write-Host " Training and GPU inference require an NVIDIA GPU with drivers installed." -ForegroundColor Yellow
|
||||
Write-Host " https://www.nvidia.com/Download/index.aspx" -ForegroundColor Yellow
|
||||
Write-Host ""
|
||||
Write-Host " If you have an NVIDIA GPU, ensure the driver is installed:" -ForegroundColor Yellow
|
||||
Write-Host " https://www.nvidia.com/Download/index.aspx" -ForegroundColor Yellow
|
||||
exit 1
|
||||
} else {
|
||||
Write-Host "[OK] NVIDIA GPU detected" -ForegroundColor Green
|
||||
}
|
||||
Write-Host "[OK] NVIDIA GPU detected" -ForegroundColor Green
|
||||
|
||||
# ============================================
|
||||
# 1a.5. Windows Long Paths (required for deep node_modules / Python paths)
|
||||
|
|
@ -341,6 +375,30 @@ if (-not $HasCmake) {
|
|||
$HasCmake = $null -ne (Get-Command cmake -ErrorAction SilentlyContinue)
|
||||
} catch { }
|
||||
}
|
||||
# winget may succeed but cmake isn't on PATH yet (MSI PATH changes need a
|
||||
# new shell). Try the default install location as a fallback.
|
||||
if (-not $HasCmake) {
|
||||
$cmakeDefaults = @(
|
||||
"$env:ProgramFiles\CMake\bin",
|
||||
"${env:ProgramFiles(x86)}\CMake\bin",
|
||||
"$env:LOCALAPPDATA\CMake\bin"
|
||||
)
|
||||
foreach ($d in $cmakeDefaults) {
|
||||
if (Test-Path (Join-Path $d "cmake.exe")) {
|
||||
$env:Path = "$d;$env:Path"
|
||||
# Persist to user PATH so Refresh-Environment does not drop it later
|
||||
$userPath = [Environment]::GetEnvironmentVariable('Path', 'User')
|
||||
if (-not $userPath -or $userPath -notlike "*$d*") {
|
||||
[Environment]::SetEnvironmentVariable('Path', "$d;$userPath", 'User')
|
||||
}
|
||||
$HasCmake = $null -ne (Get-Command cmake -ErrorAction SilentlyContinue)
|
||||
if ($HasCmake) {
|
||||
Write-Host " Found cmake at $d (added to PATH)" -ForegroundColor Gray
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
if ($HasCmake) {
|
||||
Write-Host "[OK] CMake installed" -ForegroundColor Green
|
||||
} else {
|
||||
|
|
@ -389,6 +447,7 @@ if ($vsResult) {
|
|||
# ============================================
|
||||
# 1e. CUDA Toolkit (nvcc for llama.cpp build + env vars)
|
||||
# ============================================
|
||||
if ($HasNvidiaSmi) {
|
||||
# IMPORTANT: The CUDA Toolkit version must be <= the max CUDA version the
|
||||
# NVIDIA driver supports. nvidia-smi reports this as "CUDA Version: X.Y".
|
||||
# If we install a toolkit newer than the driver supports, llama-server will
|
||||
|
|
@ -397,7 +456,7 @@ if ($vsResult) {
|
|||
# -- Detect max CUDA version the driver supports --
|
||||
$DriverMaxCuda = $null
|
||||
try {
|
||||
$smiOut = nvidia-smi 2>&1 | Out-String
|
||||
$smiOut = & $NvidiaSmiExe 2>&1 | Out-String
|
||||
if ($smiOut -match "CUDA Version:\s+([\d]+)\.([\d]+)") {
|
||||
$DriverMaxCuda = "$($Matches[1]).$($Matches[2])"
|
||||
Write-Host " Driver supports up to CUDA $DriverMaxCuda" -ForegroundColor Gray
|
||||
|
|
@ -624,11 +683,24 @@ if ($VsInstallPath -and $CudaToolkitRoot) {
|
|||
Copy-Item "$cudaExtras\*" $vsCustomizations -Force -ErrorAction Stop
|
||||
Write-Host " [OK] CUDA VS integration files installed" -ForegroundColor Green
|
||||
} catch {
|
||||
Write-Host " [WARN] Could not copy CUDA VS integration files (may need admin)" -ForegroundColor Yellow
|
||||
Write-Host " Manual fix: copy contents of" -ForegroundColor Yellow
|
||||
Write-Host " $cudaExtras" -ForegroundColor Cyan
|
||||
Write-Host " into:" -ForegroundColor Yellow
|
||||
Write-Host " $vsCustomizations" -ForegroundColor Cyan
|
||||
# Direct copy failed (needs admin). Try elevated copy via Start-Process.
|
||||
try {
|
||||
$copyCmd = "Copy-Item '$cudaExtras\*' '$vsCustomizations' -Force"
|
||||
Start-Process powershell -ArgumentList "-NoProfile -Command $copyCmd" -Verb RunAs -Wait -ErrorAction Stop
|
||||
$hasTargetsRetry = Get-ChildItem $vsCustomizations -Filter "CUDA *.targets" -ErrorAction SilentlyContinue
|
||||
if ($hasTargetsRetry) {
|
||||
Write-Host " [OK] CUDA VS integration files installed (elevated)" -ForegroundColor Green
|
||||
} else {
|
||||
throw "Copy did not produce .targets files"
|
||||
}
|
||||
} catch {
|
||||
Write-Host " [WARN] Could not copy CUDA VS integration files" -ForegroundColor Yellow
|
||||
Write-Host " The llama.cpp build may fail with 'No CUDA toolset found'." -ForegroundColor Yellow
|
||||
Write-Host " Manual fix: copy contents of" -ForegroundColor Yellow
|
||||
Write-Host " $cudaExtras" -ForegroundColor Cyan
|
||||
Write-Host " into:" -ForegroundColor Yellow
|
||||
Write-Host " $vsCustomizations" -ForegroundColor Cyan
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -643,6 +715,9 @@ Write-Host " CudaToolkitDir = $CudaToolkitRoot\" -ForegroundColor Gray
|
|||
if (-not $CudaArch) {
|
||||
Write-Host " [WARN] Could not detect compute capability -- cmake will use defaults" -ForegroundColor Yellow
|
||||
}
|
||||
} else {
|
||||
Write-Host "[SKIP] CUDA Toolkit -- no NVIDIA GPU detected" -ForegroundColor Yellow
|
||||
}
|
||||
|
||||
# ============================================
|
||||
# 1f. Node.js / npm (skip if pip-installed -- only needed for frontend build)
|
||||
|
|
@ -748,9 +823,39 @@ Write-Host ""
|
|||
# ==========================================================================
|
||||
# PHASE 2: Frontend build (skip if pip-installed -- already bundled)
|
||||
# ==========================================================================
|
||||
$DistDir = Join-Path $FrontendDir "dist"
|
||||
# Skip build if dist/ exists and no tracked input is newer than dist/.
|
||||
# Checks src/, public/, package.json, config files -- not just src/.
|
||||
$NeedFrontendBuild = $true
|
||||
if ($IsPipInstall) {
|
||||
$NeedFrontendBuild = $false
|
||||
Write-Host "[OK] Running from pip install - frontend already bundled, skipping build" -ForegroundColor Green
|
||||
} else {
|
||||
} elseif (Test-Path $DistDir) {
|
||||
$DistTime = (Get-Item $DistDir).LastWriteTime
|
||||
$NewerFile = $null
|
||||
# Check src/ and public/ recursively (probe paths directly, not via -Include)
|
||||
foreach ($subDir in @("src", "public")) {
|
||||
$subPath = Join-Path $FrontendDir $subDir
|
||||
if (Test-Path $subPath) {
|
||||
$NewerFile = Get-ChildItem -Path $subPath -Recurse -File -ErrorAction SilentlyContinue |
|
||||
Where-Object { $_.LastWriteTime -gt $DistTime } | Select-Object -First 1
|
||||
if ($NewerFile) { break }
|
||||
}
|
||||
}
|
||||
# Also check all top-level files (package.json, bun.lock, vite.config.ts, index.html, etc.)
|
||||
if (-not $NewerFile) {
|
||||
$NewerFile = Get-ChildItem -Path $FrontendDir -File -ErrorAction SilentlyContinue |
|
||||
Where-Object { $_.LastWriteTime -gt $DistTime } |
|
||||
Select-Object -First 1
|
||||
}
|
||||
if (-not $NewerFile) {
|
||||
$NeedFrontendBuild = $false
|
||||
Write-Host "[OK] Frontend already built and up to date -- skipping build" -ForegroundColor Green
|
||||
} else {
|
||||
Write-Host "[INFO] Frontend source changed since last build -- rebuilding..." -ForegroundColor Yellow
|
||||
}
|
||||
}
|
||||
if ($NeedFrontendBuild -and -not $IsPipInstall) {
|
||||
Write-Host ""
|
||||
Write-Host "Building frontend..." -ForegroundColor Cyan
|
||||
# npm writes warnings to stderr; lower ErrorActionPreference so PS doesn't
|
||||
|
|
@ -758,9 +863,6 @@ if ($IsPipInstall) {
|
|||
$prevEAP_npm = $ErrorActionPreference
|
||||
$ErrorActionPreference = "Continue"
|
||||
Push-Location $FrontendDir
|
||||
# Remove stale node_modules and package-lock.json to avoid version conflicts
|
||||
if (Test-Path "node_modules") { Remove-Item -Recurse -Force "node_modules" }
|
||||
if (Test-Path "package-lock.json") { Remove-Item -Force "package-lock.json" }
|
||||
npm install 2>&1 | Out-Null
|
||||
if ($LASTEXITCODE -ne 0) {
|
||||
Pop-Location
|
||||
|
|
@ -845,7 +947,35 @@ $ErrorActionPreference = "Continue"
|
|||
|
||||
$ActivateScript = Join-Path $VenvDir "Scripts\Activate.ps1"
|
||||
. $ActivateScript
|
||||
pip install --upgrade pip 2>&1 | Out-Null
|
||||
|
||||
# Try to use uv (much faster than pip), fall back to pip if unavailable
|
||||
$UseUv = $false
|
||||
if (Get-Command uv -ErrorAction SilentlyContinue) {
|
||||
$UseUv = $true
|
||||
} else {
|
||||
Write-Host " Installing uv package manager..." -ForegroundColor Cyan
|
||||
try {
|
||||
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex" 2>&1 | Out-Null
|
||||
Refresh-Environment
|
||||
# Re-activate venv since Refresh-Environment rebuilds PATH from
|
||||
# registry and drops the venv's Scripts directory
|
||||
. $ActivateScript
|
||||
if (Get-Command uv -ErrorAction SilentlyContinue) { $UseUv = $true }
|
||||
} catch { }
|
||||
}
|
||||
|
||||
# Helper: install a package, preferring uv with pip fallback
|
||||
function Fast-Install {
|
||||
param([Parameter(ValueFromRemainingArguments=$true)]$Args_)
|
||||
if ($UseUv) {
|
||||
$VenvPy = (Get-Command python).Source
|
||||
$result = & uv pip install --python $VenvPy @Args_ 2>&1
|
||||
if ($LASTEXITCODE -eq 0) { return }
|
||||
}
|
||||
& python -m pip install @Args_ 2>&1
|
||||
}
|
||||
|
||||
Fast-Install --upgrade pip | Out-Null
|
||||
|
||||
# if (-not $IsPipInstall) {
|
||||
# # Running from repo: copy requirements and do editable install
|
||||
|
|
@ -880,16 +1010,37 @@ $env:TORCHINDUCTOR_CACHE_DIR = $TorchCacheDir
|
|||
[Environment]::SetEnvironmentVariable('TORCHINDUCTOR_CACHE_DIR', $TorchCacheDir, 'User')
|
||||
Write-Host "[OK] TORCHINDUCTOR_CACHE_DIR set to $TorchCacheDir (avoids MAX_PATH issues)" -ForegroundColor Green
|
||||
|
||||
$CuTag = Get-PytorchCudaTag
|
||||
Write-Host " Installing PyTorch with CUDA support ($CuTag)..." -ForegroundColor Cyan
|
||||
pip install torch torchvision torchaudio --index-url "https://download.pytorch.org/whl/$CuTag" 2>&1 | Out-Null
|
||||
if ($HasNvidiaSmi) {
|
||||
$CuTag = Get-PytorchCudaTag
|
||||
Write-Host " Installing PyTorch with CUDA support ($CuTag)..." -ForegroundColor Cyan
|
||||
Write-Host " (This download is ~2.8 GB -- may take a few minutes)" -ForegroundColor Gray
|
||||
$output = Fast-Install torch torchvision torchaudio --index-url "https://download.pytorch.org/whl/$CuTag" | Out-String
|
||||
if ($LASTEXITCODE -ne 0) {
|
||||
Write-Host "[FAILED] PyTorch CUDA install failed (exit code $LASTEXITCODE)" -ForegroundColor Red
|
||||
Write-Host $output -ForegroundColor Red
|
||||
exit 1
|
||||
}
|
||||
|
||||
# Install Triton for Windows (enables torch.compile — without it training can hang)
|
||||
Write-Host " Installing Triton for Windows..." -ForegroundColor Cyan
|
||||
pip install "triton-windows<3.7" 2>&1 | Out-Null
|
||||
Write-Host "[OK] Triton for Windows installed (enables torch.compile)" -ForegroundColor Green
|
||||
# Install Triton for Windows (enables torch.compile -- without it training can hang)
|
||||
Write-Host " Installing Triton for Windows..." -ForegroundColor Cyan
|
||||
$output = Fast-Install "triton-windows<3.7" | Out-String
|
||||
if ($LASTEXITCODE -ne 0) {
|
||||
Write-Host "[WARN] Triton install failed -- torch.compile may not work" -ForegroundColor Yellow
|
||||
Write-Host $output -ForegroundColor Yellow
|
||||
} else {
|
||||
Write-Host "[OK] Triton for Windows installed (enables torch.compile)" -ForegroundColor Green
|
||||
}
|
||||
} else {
|
||||
Write-Host " Installing PyTorch (CPU-only)..." -ForegroundColor Cyan
|
||||
$output = Fast-Install torch torchvision torchaudio --index-url "https://download.pytorch.org/whl/cpu" | Out-String
|
||||
if ($LASTEXITCODE -ne 0) {
|
||||
Write-Host "[FAILED] PyTorch install failed (exit code $LASTEXITCODE)" -ForegroundColor Red
|
||||
Write-Host $output -ForegroundColor Red
|
||||
exit 1
|
||||
}
|
||||
}
|
||||
|
||||
# Ordered heavy dependency installation — shared cross-platform script
|
||||
# Ordered heavy dependency installation -- shared cross-platform script
|
||||
Write-Host " Running ordered dependency installation..." -ForegroundColor Cyan
|
||||
python "$PSScriptRoot\install_python_stack.py"
|
||||
# Restore ErrorActionPreference after pip/python work
|
||||
|
|
@ -898,7 +1049,7 @@ $ErrorActionPreference = $prevEAP
|
|||
# ── Pre-install transformers 5.x into .venv_t5/ ──
|
||||
# Models like GLM-4.7-Flash need transformers>=5.3.0. Instead of pip-installing
|
||||
# at runtime (slow, ~10-15s), we pre-install into a separate directory.
|
||||
# The training subprocess just prepends .venv_t5/ to sys.path — instant switch.
|
||||
# The training subprocess just prepends .venv_t5/ to sys.path -- instant switch.
|
||||
Write-Host ""
|
||||
Write-Host " Pre-installing transformers 5.x for newer model support..." -ForegroundColor Cyan
|
||||
$VenvT5Dir = Join-Path $env:USERPROFILE ".unsloth\studio\.venv_t5"
|
||||
|
|
@ -906,17 +1057,20 @@ if (Test-Path $VenvT5Dir) { Remove-Item -Recurse -Force $VenvT5Dir }
|
|||
New-Item -ItemType Directory -Path $VenvT5Dir -Force | Out-Null
|
||||
$prevEAP_t5 = $ErrorActionPreference
|
||||
$ErrorActionPreference = "Continue"
|
||||
pip install --target $VenvT5Dir --no-deps "transformers==5.3.0" 2>&1 | Out-Null
|
||||
if ($LASTEXITCODE -ne 0) {
|
||||
Write-Host "[FAIL] Could not install transformers 5.3.0 into .venv_t5/" -ForegroundColor Red
|
||||
$ErrorActionPreference = $prevEAP_t5
|
||||
exit 1
|
||||
foreach ($pkg in @("transformers==5.3.0", "huggingface_hub==1.7.1", "hf_xet==1.4.2")) {
|
||||
$output = Fast-Install --target $VenvT5Dir --no-deps $pkg | Out-String
|
||||
if ($LASTEXITCODE -ne 0) {
|
||||
Write-Host "[FAIL] Could not install $pkg into .venv_t5/" -ForegroundColor Red
|
||||
Write-Host $output -ForegroundColor Red
|
||||
$ErrorActionPreference = $prevEAP_t5
|
||||
exit 1
|
||||
}
|
||||
}
|
||||
pip install --target $VenvT5Dir --no-deps "huggingface_hub==1.3.0" 2>&1 | Out-Null
|
||||
# tiktoken is needed by Qwen-family tokenizers -- install with deps since
|
||||
# regex/requests may be missing on Windows
|
||||
$output = Fast-Install --target $VenvT5Dir tiktoken | Out-String
|
||||
if ($LASTEXITCODE -ne 0) {
|
||||
Write-Host "[FAIL] Could not install huggingface_hub 1.3.0 into .venv_t5/" -ForegroundColor Red
|
||||
$ErrorActionPreference = $prevEAP_t5
|
||||
exit 1
|
||||
Write-Host "[WARN] Could not install tiktoken into .venv_t5/ -- Qwen tokenizers may fail" -ForegroundColor Yellow
|
||||
}
|
||||
$ErrorActionPreference = $prevEAP_t5
|
||||
Write-Host "[OK] Transformers 5.x pre-installed to .venv_t5/" -ForegroundColor Green
|
||||
|
|
@ -982,12 +1136,46 @@ $LlamaCppDir = Join-Path $UnslothHome "llama.cpp"
|
|||
$BuildDir = Join-Path $LlamaCppDir "build"
|
||||
$LlamaServerBin = Join-Path $BuildDir "bin\Release\llama-server.exe"
|
||||
|
||||
$HasCmakeForBuild = $null -ne (Get-Command cmake -ErrorAction SilentlyContinue)
|
||||
|
||||
# Check if existing llama-server matches current GPU mode. A CUDA-built binary
|
||||
# on a now-CPU-only machine (or vice versa) needs to be rebuilt.
|
||||
$NeedRebuild = $false
|
||||
if (Test-Path $LlamaServerBin) {
|
||||
$CmakeCacheFile = Join-Path $BuildDir "CMakeCache.txt"
|
||||
if (Test-Path $CmakeCacheFile) {
|
||||
$cachedCuda = Select-String -Path $CmakeCacheFile -Pattern 'GGML_CUDA:BOOL=ON' -Quiet
|
||||
if ($HasNvidiaSmi -and -not $cachedCuda) {
|
||||
Write-Host " Existing llama-server is CPU-only but GPU is available -- rebuilding" -ForegroundColor Yellow
|
||||
$NeedRebuild = $true
|
||||
} elseif (-not $HasNvidiaSmi -and $cachedCuda) {
|
||||
Write-Host " Existing llama-server was built with CUDA but no GPU detected -- rebuilding" -ForegroundColor Yellow
|
||||
$NeedRebuild = $true
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if ((Test-Path $LlamaServerBin) -and -not $NeedRebuild) {
|
||||
Write-Host ""
|
||||
Write-Host "[OK] llama-server already exists at $LlamaServerBin" -ForegroundColor Green
|
||||
} elseif (-not $HasCmakeForBuild) {
|
||||
Write-Host ""
|
||||
if (-not $HasNvidiaSmi) {
|
||||
# CPU-only machines depend entirely on llama-server for GGUF chat -- cmake is required
|
||||
Write-Host "[ERROR] CMake is required to build llama-server for GGUF chat mode." -ForegroundColor Red
|
||||
Write-Host " Install CMake from https://cmake.org/download/ and re-run setup." -ForegroundColor Yellow
|
||||
exit 1
|
||||
}
|
||||
Write-Host "[SKIP] llama-server build -- cmake not available" -ForegroundColor Yellow
|
||||
Write-Host " GGUF inference and export will not be available." -ForegroundColor Yellow
|
||||
Write-Host " Install CMake from https://cmake.org/download/ and re-run setup." -ForegroundColor Yellow
|
||||
} else {
|
||||
Write-Host ""
|
||||
Write-Host "Building llama.cpp with CUDA support..." -ForegroundColor Cyan
|
||||
if ($HasNvidiaSmi) {
|
||||
Write-Host "Building llama.cpp with CUDA support..." -ForegroundColor Cyan
|
||||
} else {
|
||||
Write-Host "Building llama.cpp (CPU-only, no NVIDIA GPU detected)..." -ForegroundColor Cyan
|
||||
}
|
||||
Write-Host " This typically takes 5-10 minutes on first build." -ForegroundColor Gray
|
||||
Write-Host ""
|
||||
|
||||
|
|
@ -1007,17 +1195,19 @@ if (Test-Path $LlamaServerBin) {
|
|||
# Re-sanitize CUDA_PATH_V* vars — Refresh-Environment (called during
|
||||
# Node/Python installs above) may have repopulated conflicting versioned
|
||||
# vars from the Machine registry.
|
||||
$cudaPathVars2 = @([Environment]::GetEnvironmentVariables('Process').Keys | Where-Object { $_ -match '^CUDA_PATH_V' })
|
||||
foreach ($v2 in $cudaPathVars2) {
|
||||
[Environment]::SetEnvironmentVariable($v2, $null, 'Process')
|
||||
if ($HasNvidiaSmi -and $CudaToolkitRoot) {
|
||||
$cudaPathVars2 = @([Environment]::GetEnvironmentVariables('Process').Keys | Where-Object { $_ -match '^CUDA_PATH_V' })
|
||||
foreach ($v2 in $cudaPathVars2) {
|
||||
[Environment]::SetEnvironmentVariable($v2, $null, 'Process')
|
||||
}
|
||||
$tkDirName2 = Split-Path $CudaToolkitRoot -Leaf
|
||||
if ($tkDirName2 -match '^v(\d+)\.(\d+)') {
|
||||
[Environment]::SetEnvironmentVariable("CUDA_PATH_V$($Matches[1])_$($Matches[2])", $CudaToolkitRoot, 'Process')
|
||||
}
|
||||
# Also re-assert CUDA_PATH and CudaToolkitDir in case they were overwritten
|
||||
[Environment]::SetEnvironmentVariable('CUDA_PATH', $CudaToolkitRoot, 'Process')
|
||||
[Environment]::SetEnvironmentVariable('CudaToolkitDir', "$CudaToolkitRoot\", 'Process')
|
||||
}
|
||||
$tkDirName2 = Split-Path $CudaToolkitRoot -Leaf
|
||||
if ($tkDirName2 -match '^v(\d+)\.(\d+)') {
|
||||
[Environment]::SetEnvironmentVariable("CUDA_PATH_V$($Matches[1])_$($Matches[2])", $CudaToolkitRoot, 'Process')
|
||||
}
|
||||
# Also re-assert CUDA_PATH and CudaToolkitDir in case they were overwritten
|
||||
[Environment]::SetEnvironmentVariable('CUDA_PATH', $CudaToolkitRoot, 'Process')
|
||||
[Environment]::SetEnvironmentVariable('CudaToolkitDir', "$CudaToolkitRoot\", 'Process')
|
||||
|
||||
# -- Step A: Clone or pull llama.cpp --
|
||||
|
||||
|
|
@ -1037,7 +1227,14 @@ if (Test-Path $LlamaServerBin) {
|
|||
}
|
||||
}
|
||||
|
||||
# -- Step B: cmake configure (CUDA + Unsloth flags) --
|
||||
# -- Step B: cmake configure --
|
||||
# Clean stale CMake cache to prevent previous CUDA settings from leaking
|
||||
# into a CPU-only rebuild (or vice versa).
|
||||
$CmakeCacheFile = Join-Path $BuildDir "CMakeCache.txt"
|
||||
if (Test-Path $CmakeCacheFile) {
|
||||
Remove-Item -Recurse -Force $BuildDir
|
||||
}
|
||||
|
||||
if ($BuildOk) {
|
||||
Write-Host ""
|
||||
Write-Host "--- cmake configure ---" -ForegroundColor Cyan
|
||||
|
|
@ -1066,37 +1263,45 @@ if (Test-Path $LlamaServerBin) {
|
|||
$CmakeArgs += '-DLLAMA_CURL=OFF'
|
||||
}
|
||||
$CmakeArgs += '-DCMAKE_EXE_LINKER_FLAGS=/NODEFAULTLIB:LIBCMT'
|
||||
# CUDA flags (Unsloth-aligned)
|
||||
$CmakeArgs += '-DGGML_CUDA=ON'
|
||||
$CmakeArgs += "-DCUDAToolkit_ROOT=$CudaToolkitRoot"
|
||||
$CmakeArgs += "-DCUDA_TOOLKIT_ROOT_DIR=$CudaToolkitRoot"
|
||||
$CmakeArgs += "-DCMAKE_CUDA_COMPILER=$NvccPath"
|
||||
$CmakeArgs += '-DGGML_CUDA_FA_ALL_QUANTS=ON'
|
||||
$CmakeArgs += '-DGGML_CUDA_F16=OFF'
|
||||
$CmakeArgs += '-DGGML_CUDA_GRAPHS=OFF'
|
||||
$CmakeArgs += '-DGGML_CUDA_FORCE_CUBLAS=OFF'
|
||||
$CmakeArgs += '-DGGML_CUDA_PEER_MAX_BATCH_SIZE=8192'
|
||||
if ($CudaArch) {
|
||||
# Validate nvcc actually supports this architecture
|
||||
if (Test-NvccArchSupport -NvccExe $NvccPath -Arch $CudaArch) {
|
||||
$CmakeArgs += "-DCMAKE_CUDA_ARCHITECTURES=$CudaArch"
|
||||
} else {
|
||||
# GPU arch too new for this toolkit — fall back to highest supported.
|
||||
# PTX forward-compatibility will JIT-compile for the actual GPU at runtime.
|
||||
$maxArch = Get-NvccMaxArch -NvccExe $NvccPath
|
||||
if ($maxArch) {
|
||||
$CmakeArgs += "-DCMAKE_CUDA_ARCHITECTURES=$maxArch"
|
||||
Write-Host " [WARN] GPU is sm_$CudaArch but nvcc only supports up to sm_$maxArch" -ForegroundColor Yellow
|
||||
Write-Host " Building with sm_$maxArch (PTX will JIT for your GPU at runtime)" -ForegroundColor Yellow
|
||||
# CUDA flags -- only if GPU available, otherwise explicitly disable
|
||||
if ($HasNvidiaSmi -and $NvccPath) {
|
||||
$CmakeArgs += '-DGGML_CUDA=ON'
|
||||
$CmakeArgs += "-DCUDAToolkit_ROOT=$CudaToolkitRoot"
|
||||
$CmakeArgs += "-DCUDA_TOOLKIT_ROOT_DIR=$CudaToolkitRoot"
|
||||
$CmakeArgs += "-DCMAKE_CUDA_COMPILER=$NvccPath"
|
||||
if ($CudaArch) {
|
||||
# Validate nvcc actually supports this architecture
|
||||
if (Test-NvccArchSupport -NvccExe $NvccPath -Arch $CudaArch) {
|
||||
$CmakeArgs += "-DCMAKE_CUDA_ARCHITECTURES=$CudaArch"
|
||||
} else {
|
||||
# GPU arch too new for this toolkit -- fall back to highest supported.
|
||||
# PTX forward-compatibility will JIT-compile for the actual GPU at runtime.
|
||||
$maxArch = Get-NvccMaxArch -NvccExe $NvccPath
|
||||
if ($maxArch) {
|
||||
$CmakeArgs += "-DCMAKE_CUDA_ARCHITECTURES=$maxArch"
|
||||
Write-Host " [WARN] GPU is sm_$CudaArch but nvcc only supports up to sm_$maxArch" -ForegroundColor Yellow
|
||||
Write-Host " Building with sm_$maxArch (PTX will JIT for your GPU at runtime)" -ForegroundColor Yellow
|
||||
}
|
||||
# else: omit flag entirely, let cmake pick defaults
|
||||
}
|
||||
# else: omit flag entirely, let cmake pick defaults
|
||||
}
|
||||
} else {
|
||||
$CmakeArgs += '-DGGML_CUDA=OFF'
|
||||
}
|
||||
|
||||
cmake @CmakeArgs 2>&1 | Out-Null
|
||||
$cmakeOutput = cmake @CmakeArgs 2>&1 | Out-String
|
||||
if ($LASTEXITCODE -ne 0) {
|
||||
$BuildOk = $false
|
||||
$FailedStep = "cmake configure"
|
||||
Write-Host $cmakeOutput -ForegroundColor Red
|
||||
if ($cmakeOutput -match 'No CUDA toolset found|CUDA_TOOLKIT_ROOT_DIR|nvcc') {
|
||||
Write-Host ""
|
||||
Write-Host " Hint: CUDA VS integration may be missing. Try running as admin:" -ForegroundColor Yellow
|
||||
Write-Host " Copy contents of:" -ForegroundColor Yellow
|
||||
Write-Host " <CUDA_PATH>\extras\visual_studio_integration\MSBuildExtensions" -ForegroundColor Yellow
|
||||
Write-Host " into:" -ForegroundColor Yellow
|
||||
Write-Host " <VS_PATH>\MSBuild\Microsoft\VC\v170\BuildCustomizations" -ForegroundColor Yellow
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -1110,10 +1315,11 @@ if (Test-Path $LlamaServerBin) {
|
|||
Write-Host " Parallel jobs: $NumCpu" -ForegroundColor Gray
|
||||
Write-Host ""
|
||||
|
||||
cmake --build $BuildDir --config Release --target llama-server -j $NumCpu 2>&1 | Out-Null
|
||||
$output = cmake --build $BuildDir --config Release --target llama-server -j $NumCpu 2>&1 | Out-String
|
||||
if ($LASTEXITCODE -ne 0) {
|
||||
$BuildOk = $false
|
||||
$FailedStep = "cmake build (llama-server)"
|
||||
Write-Host $output -ForegroundColor Red
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -1121,9 +1327,10 @@ if (Test-Path $LlamaServerBin) {
|
|||
if ($BuildOk) {
|
||||
Write-Host ""
|
||||
Write-Host "--- cmake build (llama-quantize) ---" -ForegroundColor Cyan
|
||||
cmake --build $BuildDir --config Release --target llama-quantize -j $NumCpu 2>&1 | Out-Null
|
||||
$output = cmake --build $BuildDir --config Release --target llama-quantize -j $NumCpu 2>&1 | Out-String
|
||||
if ($LASTEXITCODE -ne 0) {
|
||||
Write-Host " [WARN] llama-quantize build failed (GGUF export may be unavailable)" -ForegroundColor Yellow
|
||||
Write-Host $output -ForegroundColor Yellow
|
||||
}
|
||||
}
|
||||
|
||||
|
|
|
|||
143
studio/setup.sh
143
studio/setup.sh
|
|
@ -41,13 +41,26 @@ if [[ "$keynames" == *$'\nCOLAB_'* ]]; then
|
|||
fi
|
||||
|
||||
# ── Detect whether frontend needs building ──
|
||||
# Only skip when BOTH conditions are true:
|
||||
# 1. We're inside site-packages (PyPI / pip install, not editable)
|
||||
# 2. dist/ already exists (pre-built in the wheel)
|
||||
# Otherwise always (re)build — handles upgrades, editable installs, and
|
||||
# pip-from-source where dist/ was never built.
|
||||
if [[ "$SCRIPT_DIR" == */site-packages/* ]] && [ -d "$SCRIPT_DIR/frontend/dist" ]; then
|
||||
echo "✅ Frontend pre-built (PyPI) — skipping Node/npm check."
|
||||
# Skip if dist/ exists AND no tracked input is newer than dist/.
|
||||
# Checks top-level config/entry files and src/, public/ recursively.
|
||||
# This handles: PyPI installs (dist/ bundled), repeat runs (no changes),
|
||||
# and upgrades/pulls (source newer than dist/ triggers rebuild).
|
||||
_NEED_FRONTEND_BUILD=true
|
||||
if [ -d "$SCRIPT_DIR/frontend/dist" ]; then
|
||||
# Check all top-level files (package.json, bun.lock, vite.config.ts, index.html, etc.)
|
||||
_changed=$(find "$SCRIPT_DIR/frontend" -maxdepth 1 -type f \
|
||||
-newer "$SCRIPT_DIR/frontend/dist" -print -quit 2>/dev/null)
|
||||
# Check src/ and public/ recursively (|| true guards against set -e when dirs are missing)
|
||||
if [ -z "$_changed" ]; then
|
||||
_changed=$(find "$SCRIPT_DIR/frontend/src" "$SCRIPT_DIR/frontend/public" \
|
||||
-type f -newer "$SCRIPT_DIR/frontend/dist" -print -quit 2>/dev/null) || true
|
||||
fi
|
||||
if [ -z "$_changed" ]; then
|
||||
_NEED_FRONTEND_BUILD=false
|
||||
fi
|
||||
fi
|
||||
if [ "$_NEED_FRONTEND_BUILD" = false ]; then
|
||||
echo "✅ Frontend already built and up to date -- skipping Node/npm check."
|
||||
else
|
||||
NEED_NODE=true
|
||||
if command -v node &>/dev/null && command -v npm &>/dev/null; then
|
||||
|
|
@ -146,12 +159,17 @@ run_quiet "npm run build" npm run build
|
|||
|
||||
_restore_gitignores
|
||||
trap - EXIT
|
||||
cd "$SCRIPT_DIR/backend/core/data_recipe/oxc-validator"
|
||||
run_quiet "npm install (oxc validator runtime)" npm install
|
||||
cd "$SCRIPT_DIR"
|
||||
echo "✅ Frontend built to frontend/dist"
|
||||
|
||||
fi # end frontend dist check
|
||||
fi # end frontend build check
|
||||
|
||||
# ── oxc-validator runtime (needs npm -- skip if not available) ──
|
||||
if [ -d "$SCRIPT_DIR/backend/core/data_recipe/oxc-validator" ] && command -v npm &>/dev/null; then
|
||||
cd "$SCRIPT_DIR/backend/core/data_recipe/oxc-validator"
|
||||
run_quiet "npm install (oxc validator runtime)" npm install
|
||||
cd "$SCRIPT_DIR"
|
||||
fi
|
||||
|
||||
# ── 6. Python venv + deps ──
|
||||
|
||||
|
|
@ -223,50 +241,76 @@ install_python_stack() {
|
|||
python "$SCRIPT_DIR/install_python_stack.py"
|
||||
}
|
||||
|
||||
if [ "$IS_COLAB" = true ]; then
|
||||
# Colab: install packages directly without venv
|
||||
install_python_stack
|
||||
else
|
||||
# Local: create venv under ~/.unsloth/studio/ (shared location, not in repo)
|
||||
STUDIO_HOME="$HOME/.unsloth/studio"
|
||||
VENV_DIR="$STUDIO_HOME/.venv"
|
||||
VENV_T5_DIR="$STUDIO_HOME/.venv_t5"
|
||||
mkdir -p "$STUDIO_HOME"
|
||||
# Create venv under ~/.unsloth/studio/ (shared location, not in repo).
|
||||
# All platforms (including Colab) use the same isolated venv so that
|
||||
# studio dependencies are never installed into the system Python.
|
||||
STUDIO_HOME="$HOME/.unsloth/studio"
|
||||
VENV_DIR="$STUDIO_HOME/.venv"
|
||||
VENV_T5_DIR="$STUDIO_HOME/.venv_t5"
|
||||
mkdir -p "$STUDIO_HOME"
|
||||
|
||||
# Clean up legacy in-repo venvs if they exist
|
||||
[ -d "$REPO_ROOT/.venv" ] && rm -rf "$REPO_ROOT/.venv"
|
||||
[ -d "$REPO_ROOT/.venv_overlay" ] && rm -rf "$REPO_ROOT/.venv_overlay"
|
||||
[ -d "$REPO_ROOT/.venv_t5" ] && rm -rf "$REPO_ROOT/.venv_t5"
|
||||
# Clean up legacy in-repo venvs if they exist
|
||||
[ -d "$REPO_ROOT/.venv" ] && rm -rf "$REPO_ROOT/.venv"
|
||||
[ -d "$REPO_ROOT/.venv_overlay" ] && rm -rf "$REPO_ROOT/.venv_overlay"
|
||||
[ -d "$REPO_ROOT/.venv_t5" ] && rm -rf "$REPO_ROOT/.venv_t5"
|
||||
|
||||
rm -rf "$VENV_DIR"
|
||||
rm -rf "$VENV_T5_DIR"
|
||||
"$BEST_PY" -m venv "$VENV_DIR"
|
||||
rm -rf "$VENV_DIR"
|
||||
rm -rf "$VENV_T5_DIR"
|
||||
# Try creating venv with pip; fall back to --without-pip + bootstrap
|
||||
# (some environments like Colab have broken ensurepip)
|
||||
if ! "$BEST_PY" -m venv "$VENV_DIR" 2>/dev/null; then
|
||||
"$BEST_PY" -m venv --without-pip "$VENV_DIR"
|
||||
source "$VENV_DIR/bin/activate"
|
||||
cd "$SCRIPT_DIR"
|
||||
install_python_stack
|
||||
curl -sS https://bootstrap.pypa.io/get-pip.py | python > /dev/null
|
||||
else
|
||||
source "$VENV_DIR/bin/activate"
|
||||
fi
|
||||
|
||||
# ── 6b. Pre-install transformers 5.x into .venv_t5/ ──
|
||||
# Models like GLM-4.7-Flash need transformers>=5.3.0. Instead of pip-installing
|
||||
# at runtime (slow, ~10-15s), we pre-install into a separate directory.
|
||||
# The training subprocess just prepends .venv_t5/ to sys.path — instant switch.
|
||||
echo ""
|
||||
echo " Pre-installing transformers 5.x for newer model support..."
|
||||
mkdir -p "$VENV_T5_DIR"
|
||||
run_quiet "pip install transformers 5.x" pip install --target "$VENV_T5_DIR" --no-deps "transformers==5.3.0"
|
||||
run_quiet "pip install huggingface_hub for t5" pip install --target "$VENV_T5_DIR" --no-deps "huggingface_hub==1.3.0"
|
||||
echo "✅ Transformers 5.x pre-installed to $VENV_T5_DIR/"
|
||||
# ── Ensure uv is available (much faster than pip) ──
|
||||
USE_UV=false
|
||||
if command -v uv &>/dev/null; then
|
||||
USE_UV=true
|
||||
elif curl -LsSf https://astral.sh/uv/install.sh | sh > /dev/null 2>&1; then
|
||||
export PATH="$HOME/.local/bin:$PATH"
|
||||
command -v uv &>/dev/null && USE_UV=true
|
||||
fi
|
||||
|
||||
# ── 7. WSL: pre-install GGUF build dependencies ──
|
||||
# On WSL, sudo requires a password and can't be entered during GGUF export
|
||||
# (runs in a non-interactive subprocess). Install build deps here instead.
|
||||
if grep -qi microsoft /proc/version 2>/dev/null; then
|
||||
echo ""
|
||||
echo "⚠️ WSL detected — installing build dependencies for GGUF export..."
|
||||
echo " You may be prompted for your password."
|
||||
sudo apt-get update -y
|
||||
sudo apt-get install -y build-essential cmake curl git libcurl4-openssl-dev
|
||||
echo "✅ GGUF build dependencies installed"
|
||||
# Helper: install a package, preferring uv with pip fallback
|
||||
fast_install() {
|
||||
if [ "$USE_UV" = true ]; then
|
||||
uv pip install --python "$(command -v python)" "$@" && return 0
|
||||
fi
|
||||
python -m pip install "$@"
|
||||
}
|
||||
|
||||
cd "$SCRIPT_DIR"
|
||||
install_python_stack
|
||||
|
||||
# ── 6b. Pre-install transformers 5.x into .venv_t5/ ──
|
||||
# Models like GLM-4.7-Flash need transformers>=5.3.0. Instead of pip-installing
|
||||
# at runtime (slow, ~10-15s), we pre-install into a separate directory.
|
||||
# The training subprocess just prepends .venv_t5/ to sys.path -- instant switch.
|
||||
echo ""
|
||||
echo " Pre-installing transformers 5.x for newer model support..."
|
||||
mkdir -p "$VENV_T5_DIR"
|
||||
run_quiet "install transformers 5.x" fast_install --target "$VENV_T5_DIR" --no-deps "transformers==5.3.0"
|
||||
run_quiet "install huggingface_hub for t5" fast_install --target "$VENV_T5_DIR" --no-deps "huggingface_hub==1.7.1"
|
||||
run_quiet "install hf_xet for t5" fast_install --target "$VENV_T5_DIR" --no-deps "hf_xet==1.4.2"
|
||||
# tiktoken is needed by Qwen-family tokenizers. Install with deps since
|
||||
# regex/requests may be missing on Windows.
|
||||
run_quiet "install tiktoken for t5" fast_install --target "$VENV_T5_DIR" "tiktoken"
|
||||
echo "✅ Transformers 5.x pre-installed to $VENV_T5_DIR/"
|
||||
|
||||
# ── 7. WSL: pre-install GGUF build dependencies ──
|
||||
# On WSL, sudo requires a password and can't be entered during GGUF export
|
||||
# (runs in a non-interactive subprocess). Install build deps here instead.
|
||||
if grep -qi microsoft /proc/version 2>/dev/null; then
|
||||
echo ""
|
||||
echo "⚠️ WSL detected -- installing build dependencies for GGUF export..."
|
||||
echo " You may be prompted for your password."
|
||||
sudo apt-get update -y
|
||||
sudo apt-get install -y build-essential cmake curl git libcurl4-openssl-dev
|
||||
echo "✅ GGUF build dependencies installed"
|
||||
fi
|
||||
|
||||
# ── 8. Build llama.cpp binaries for GGUF inference + export ──
|
||||
|
|
@ -405,6 +449,9 @@ if [ "$IS_COLAB" = true ]; then
|
|||
echo "╠══════════════════════════════════════╣"
|
||||
echo "║ Unsloth Studio is ready to start ║"
|
||||
echo "║ in your Colab notebook! ║"
|
||||
echo "║ ║"
|
||||
echo "║ from colab import start ║"
|
||||
echo "║ start() ║"
|
||||
echo "╚══════════════════════════════════════╝"
|
||||
else
|
||||
echo "╔══════════════════════════════════════╗"
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue