Replace _in_virtualenv() heuristic with a runtime probe. At
bootstrap time, try a dry-run uv install without --system. If
that fails (exit code 2, "No virtual environment found"), retry
with --system to confirm it works. This handles all environments
correctly: venvs, Colab (system Python), local machines, containers.
Three fixes based on review:
1. Make uv truly optional: _bootstrap_uv() now only checks if uv is
already on PATH. It no longer tries to pip install uv. If uv is
not present, pip is used with zero changes to behavior.
2. Add --system flag for Colab: on Colab there is no venv (packages
install into system Python). uv requires --system in this case,
otherwise it errors with "No virtual environment found". Added
_in_virtualenv() check that detects VIRTUAL_ENV, sys.real_prefix,
or sys.base_prefix != sys.prefix.
3. Fix label printed twice on uv fallback: when uv fails and falls
back to pip, the label now says "(pip)" to distinguish from the
initial uv attempt, instead of printing the same label twice.
Tested:
- venv path: no --system flag, uv installs correctly
- no-venv path (Colab sim): --system flag added automatically
- full unsloth studio setup + training run (Llama-3.2-1B, 10 steps)
install_python_stack.py:
- Print uv error output on failure for debuggability
- Refactor pip_install() to use early return after uv success,
removing duplicated pip command path
setup.sh:
- Guard nvidia-smi command substitution with || true so it does
not abort the script under set -euo pipefail when nvidia-smi
fails (e.g., containerized environments, driver quirks)
- Read all GPU compute capabilities and deduplicate, so
mixed-GPU hosts get kernels built for all present architectures
instead of only the first GPU
Restore separate cmake --build calls for llama-server and
llama-quantize on both setup.sh and setup.ps1. The combined
approach made llama-quantize failure fatal, but it was originally
best-effort (|| true on Linux, [WARN] on Windows). The timing
savings from combining was only ~2.7s, not worth the semantic
change.
The Ninja + arch detection speedups are preserved (55s vs 1m 37s).
Build llama-server and llama-quantize in a single cmake --build
invocation on Windows, matching the same optimization done in
setup.sh. This allows MSBuild to better parallelize the two targets.
The Visual Studio generator is kept as-is (not switching to Ninja on
Windows since VS generator is the standard approach and interacts
with MSBuild).
Three improvements to the llama.cpp build step in setup.sh:
1. Detect GPU compute capability via nvidia-smi and limit
CMAKE_CUDA_ARCHITECTURES to the current GPU. Without this, cmake
builds for all default CUDA architectures which is very slow.
2. Use Ninja build generator when available. Ninja has better
parallelism than Make for CUDA compilation.
3. Build both llama-server and llama-quantize targets in a single
cmake --build invocation for better parallelism.
4. Add --threads=0 to CMAKE_CUDA_FLAGS for multi-threaded nvcc
compilation.
Measured on 192-core machine with B200 (sm_100):
Make (all archs): very slow (minutes for each arch)
Make (single arch): 1m 37s
Ninja (single arch): 55s
Speedup: ~1.7x
Combined with the uv change, total setup goes from ~4m 35s to ~1m 40s.
Replace pip with uv in install_python_stack.py to speed up the Python
dependency installation phase of `unsloth studio setup`.
- Add _bootstrap_uv() that checks for uv on PATH, and if not found,
installs it via pip. Falls back to pip if uv is unavailable.
- Translate pip flags to uv equivalents (--no-cache-dir dropped since
uv caching is fast, --force-reinstall becomes --reinstall).
- Add --torch-backend=auto so uv auto-detects CUDA version for
PyTorch ecosystem packages.
- Per-install fallback: if any uv install step fails, it retries that
step with pip before exiting.
Measured on clean venv setup:
Python packages (pip): 2m 28s
Python packages (uv): 18s
Speedup: ~8x
Total setup time goes from ~4m 35s to ~2m 30s (llama.cpp build is
now the bottleneck at 1m 40s).
Related to #1615
Add documentation and function for exporting models from Colab to local machines.
* **README.md**: Add a new section titled "Exporting Models from Colab to Local Machine" under "✨ Finetune for Free" with detailed steps for exporting models from Colab to local machines.
* **CONTRIBUTING.md**: Add a note about the new documentation section for exporting models from Colab.
* **unsloth/save.py**: Add a new function `export_model_to_local` to handle exporting models from Colab to local machines.
(cherry picked from commit 0361bd658f)
Editable installs (-e) work via a .pth file that is only processed at
Python startup. In Colab the kernel is already running when setup.sh
installs the plugin, so the .pth file never gets picked up and
data_designer_unstructured_seed is not importable.
Remove -e so pip copies the package files directly into site-packages,
which the live kernel can find immediately. Local venv installs are
unaffected since the venv is always created fresh before install.
* fix(seed): disable remote code execution for seed inspect loads
* fix(test): use __file__-relative path in seed test
The test used a CWD-relative path (`studio/backend/routes/...`) which
only resolved when pytest was invoked from the repo root. Use
`Path(__file__).resolve()` so the test passes regardless of CWD.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Test <test@test.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix: disable remote code loading for ai-assist model hint lookup
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Roland Tannous <115670425+rolandtannous@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
The studio was disabling flex attention entirely on Blackwell+ GPUs
(sm_120 and above) by setting UNSLOTH_ENABLE_FLEX_ATTENTION=0 at
startup. This was a workaround for the flex_attention backward kernel
exceeding shared memory limits on these GPUs.
The root cause is now fixed in unsloth-zoo (PR #542) which patches the
backward kernel config selection to generate safe fallback configs that
fit within the GPU's shared memory limit. With that fix, flex attention
works correctly on Blackwell GPUs and provides a ~1.3x speedup over
the SDPA fallback.
Fix `llm_int8_skip_modules` not being respected for VLMs with dynamic quantization on transformers 5.x.
Dynamic quant checkpoints (e.g. `gemma-3-4b-it-unsloth-bnb-4bit`) encode skip paths as `language_model.model.layers.*`, but the live module tree on 5.x surfaces them as `model.language_model.layers.*`. This prefix mismatch causes `should_convert_module` to miss the skip list, so 22 modules meant to stay in 16-bit get wrapped in `Linear4bit` without a `quant_state`, producing "Skipping ... no quant_state found" warnings.
Patches `should_convert_module` to expand both the module name and the skip patterns into all equivalent alias forms before matching. Guarded by `hasattr` so it is a no-op on transformers 4.x where the bug does not exist.
Closes#4208
* Update CODEOWNERS for studio and cli
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
