Unsloth is a local UI for training and running Gemma 4, Qwen3.6, DeepSeek, Kimi, GLM and other models.
https://unsloth.ai/docs
agent
deepseek
fine-tuning
gemma
gemma3
gpt-oss
llama
llama3
llm
llms
mistral
openai
qwen
reinforcement-learning
self-hosted
text-to-speech
tts
ui
unsloth
- Python 71.5%
- TypeScript 22.7%
- Shell 1.9%
- PowerShell 1.6%
- Rust 1.5%
- Other 0.7%
6d27160dcc* Studio: fall back to text-only when a vision projector hard-crashes llama-server The text-only mmproj fallback (#6075) only fired when llama-server printed a recognizable projector-format error ("Unknown projector type", exit -6). An installed llama.cpp that predates a model's projector can instead SIGSEGV (exit -11) with no parseable output, e.g. unsloth/Qwen3.5-4B-MTP-GGUF + mmproj-F16 on an older gfx1151 prebuilt: llama-server crashes on load, the --fit off retry crashes the same way, and load_model gives up with a hard 500 instead of dropping vision. Generalize the decision: a vision (--mmproj) launch killed by a signal (POSIX returncode < 0, e.g. -11 SIGSEGV / -6 SIGABRT; Windows 0xC0000000+ access violation) is treated like a projector incompatibility, so the load retries once text-only. The retry is skipped if a cancel/unload is pending, mirroring the MTP guard. Clean non-zero exits (bad GGUF, port bind) and hung processes keep their own handling; non-vision launches are unaffected. Reproduced and verified on gfx1151 (Radeon 8060S, ROCm 7.2.1): a current prebuilt (llama.cpp b9596) loads the exact model + args fine, confirming the crash is a stale prebuilt. With a wrapper that SIGSEGVs on --mmproj, Studio now recovers: the load returns 200 (is_vision=false) and serves at ~31 tok/s text-only instead of failing. New _is_signal_crash helper plus tests pin the decision. Also normalize a few em-dashes to ASCII punctuation in existing comments. * Studio: refine mmproj hard-crash fallback (signal scope + last argv) - Limit _is_signal_crash to genuine program faults (SIGSEGV, SIGABRT, SIGILL, SIGFPE, SIGBUS) and Windows 0xC0000000+ statuses. SIGKILL, SIGTERM and SIGINT no longer count, so an OOM-killer, unload or supervisor kill is not masked as a projector incompatibility. - Strip --mmproj from the last attempted argv so the text-only retry keeps --fit off / --spec-default instead of resurrecting the original spec flags (matters for MTP vision models on an older llama.cpp). - Drop stray temp files committed by mistake and gitignore the "~" dir so they cannot be re-added. * Studio: tighten comments in mmproj hard-crash fallback * Studio: retry --flash-attn off before dropping vision on a startup crash When llama-server hard-crashes at startup, the recovery chain now tries the least-destructive mitigation first. Flash-attention kernels SIGSEGV at load on some ROCm/GPU builds (often inside the vision tower's attention); disabling flash attention keeps BOTH vision and MTP, so a hard program fault with --flash-attn on now retries once with --flash-attn off before the MTP-drop or the text-only (mmproj-strip) fallbacks. _is_signal_crash already gates this to genuine faults (SIGSEGV/SIGABRT/SIGILL/SIGFPE/SIGBUS), so an OOM-kill or unload (SIGKILL/SIGTERM/SIGINT) does not trigger a retry. Field context: a gfx1151 user crashes loading a vision GGUF even on the latest prebuilt, so an update cannot help, and the same model and args load fine on another gfx1151 box, pointing at a runtime/flash-attn fault. New _with_flash_attn_off helper plus tests. Verified on hardware with a wrapper that SIGSEGVs on --flash-attn on: Studio recovers with is_vision=true (vision and MTP intact) instead of failing or losing vision. * Studio: name the OOM kill on a too-large model load When the OS kills llama-server with no diagnostic output (SIGKILL/SIGTERM, almost always the OOM killer, e.g. a BF16 model too large for the WSL VM's RAM cap), the recovery ladder correctly does not retry an external kill, so this is the message the user sees. It fell through to the generic "is the GGUF valid / out of memory" text. Make it actionable: name the signal and point at a smaller or more quantized GGUF, a lower context length, or raising the WSL memory limit. Output-based diagnoses still win and a hard fault keeps the generic fallback. * Studio: refuse a model too large for system RAM on a unified-memory APU On gfx1150/gfx1151 APUs the weights load into shared system RAM (GGML unified memory). _get_gpu_free_memory reports the full ROCm/APU budget as free (often ~100 GB), but under WSL the VM's RAM cap is the real ceiling. Studio trusted the budget, spawned a load larger than RAM, and the OS killed it mid-flight, taking the Studio process with it (a silent "Terminated" with no error, the model resident in RAM not VRAM). Add a pre-flight guard on the APU path: if the weights exceed available system RAM (psutil, then /proc/meminfo), refuse before spawning with a clear message (smaller/more-quantized GGUF, lower context, or raise the WSL memory limit). Weights only so KV/context auto-reduction is not double-counted; unknown RAM never refuses; non-APU and discrete-GPU paths are untouched. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: refine recovery ladder (keep diagnosed errors, flip all flash-attn) Two review points on the hard-crash recovery ladder: 1. The signal-only text-only fallback stripped --mmproj on any hard fault, even when llama-server had already printed a non-projector cause (an OOM such as "cudaMalloc failed: out of memory", an unsupported architecture, or a tensor-parallel limit). That masked the real error and told the user to update llama.cpp for vision. New _output_has_nonprojector_diagnostic gates the signal path: it fires only when no such marker is present, so a bare SIGSEGV with no output still retries text-only, but a diagnosed OOM surfaces the real error instead of silently dropping vision. 2. _with_flash_attn_off only flipped the first --flash-attn. llama.cpp is last-wins, so a leftover enable from extra_args (--flash-attn on, -fa on, or the = form) could keep flash attention on and re-crash the retry. It now flips every occurrence and returns None only when nothing is flippable. test_llama_cpp_mmproj_fallback.py and the classification/APU suites: 103 passed. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: pass the text-only retry's exit code to the failure classifier When the text-only fallback retry itself fails, read its exit code before _kill_process() clears it and forward it to _classify_llama_start_failure, so an OS-killed retry surfaces the actionable out-of-memory message instead of the generic one (matching the primary failure path). * Studio: scope APU RAM guard to selected GPUs, count MTP drafter, neutral SIGTERM Three refinements to the startup recovery work in this PR: - The unified-memory APU RAM guard fired whenever any visible GPU was a gfx1150/gfx1151 APU, so on a mixed APU+dGPU host it could refuse a valid load placed on the discrete GPU. Scope _amd_apu_wants_unified_memory to the selected gpu_indices (physical ids, mapped via CUDA_VISIBLE_DEVICES like _is_datacenter_gpu); None still means every visible GPU. Applied to both the RAM guard and the GGML_CUDA_ENABLE_UNIFIED_MEMORY env set. - The RAM guard counted only the main GGUF plus mmproj, so a separate MTP drafter (also resident in unified system RAM, even when offloaded to CPU) could push the load past the RAM cap and still get OS-killed mid-load. Add the drafter weights to the APU RAM total. - The startup classifier reported SIGTERM (-15) as 'most likely out of memory', but SIGTERM is also how an unload/cancel or a supervisor stops the server. Keep the OOM wording for SIGKILL (-9, the OOM killer) and report -15 neutrally. Tests updated/added accordingly. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: address review on the APU guard and decode-probe ladder - Map APU physical ids via the active ROCm mask (HIP, then ROCR, then CUDA), mirroring _get_gpu_memory, so a HIP_VISIBLE_DEVICES-selected APU is matched. - Only add the MTP drafter to the APU RAM total when MTP will actually engage, so a stale LLAMA_ARG_SPEC_DRAFT_MODEL cannot refuse a non-MTP load. - After an MTP first-decode hard fault, retry --flash-attn off (keeps MTP) before dropping speculative decoding, matching the startup rung. - Fold the --flash-attn= / -fa= rewrite into one branch. Tests: tensor-parallel decode-probe assertion updated for the FA-off rung. * Studio: tighten two comments in the APU guard and RAM preflight * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: refine flash-attn retry and APU RAM guard per review - _with_flash_attn_off now decides on the effective last-wins value: it returns None when FA is already off (no wasted retry), and neutralizes a bare --flash-attn / -fa (which llama.cpp reads as on) so the retry cannot re-enable it. Length is preserved so downstream index slices stay valid. - _amd_apu_wants_unified_memory uses 'gpu_indices is not None' so an empty selection is respected (not treated as all-visible). - The APU RAM refusal now checks the base model only (main + mmproj); an optional MTP drafter is dropped by the existing MTP-drop fallback rather than causing a hard pre-spawn refusal of an otherwise loadable model. Tests: bare-flag / effective-off / empty-selection / HIP-mask cases added. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|---|---|---|
| .github | ||
| images | ||
| scripts | ||
| studio | ||
| tests | ||
| unsloth | ||
| unsloth_cli | ||
| .git-blame-ignore-revs | ||
| .gitattributes | ||
| .gitignore | ||
| .pre-commit-ci.yaml | ||
| .pre-commit-config.yaml | ||
| build.sh | ||
| cli.py | ||
| CODE_OF_CONDUCT.md | ||
| CONTRIBUTING.md | ||
| COPYING | ||
| install.ps1 | ||
| install.sh | ||
| LICENSE | ||
| pyproject.toml | ||
| README.md | ||
| unsloth-cli.py | ||
Unsloth Studio lets you run and train models locally.
Features • Quickstart • Notebooks • Documentation
⚡ Get started
macOS, Linux, WSL:
curl -fsSL https://unsloth.ai/install.sh | sh
Windows:
irm https://unsloth.ai/install.ps1 | iex
Community:
⭐ Features
Unsloth Studio (Beta) lets you run and train text, audio, embedding, vision models on Windows, Linux and macOS.
Inference
- Search + download + run models including GGUF, LoRA adapters, safetensors
- Export models: Save or export models to GGUF, 16-bit safetensors and other formats.
- Tool calling: Support for self-healing tool calling and web search
- Code execution: lets LLMs test code in Claude artifacts and sandbox environments
- API inference endpoint: Deploy and run local LLMs in Claude Code, Codex tools with Unsloth
- Auto set inference settings and customize chat templates.
- We work directly with teams behind gpt-oss, Qwen3, Llama 4, Mistral, Gemma 1-3, and Phi-4, where we’ve fixed bugs that improve model accuracy.
- Chat with images, audio, PDFs, code, DOCX and more. Connect API providers (OpenAI, Anthropic) or servers (vLLM, Ollama).
Training
- Train and RL 500+ models up to 2x faster with up to 70% less VRAM, with no accuracy loss.
- Custom Triton and mathematical kernels. See some collabs we did with PyTorch and Hugging Face.
- Data Recipes: Auto-create datasets from PDF, CSV, DOCX etc. Edit data in a visual-node workflow.
- Reinforcement Learning (RL): The most efficient RL library, using 80% less VRAM for GRPO, FP8 etc.
- Supports full fine-tuning, RL, pretraining, 4-bit, 16-bit and, FP8 training.
- Observability: Monitor training live, track loss and GPU usage and customize graphs.
- Multi-GPU training is supported, with major improvements coming soon.
📥 Install
Unsloth can be used in two ways: through Unsloth Studio, the web UI, or through Unsloth Core, the code-based version. Each has different requirements.
Unsloth Studio (web UI)
Unsloth Studio (Beta) works on Windows, Linux, WSL and macOS.
- CPU: Supported for Chat and Data Recipes currently
- NVIDIA: Training works on RTX 30/40/50, Blackwell, DGX Spark, Station and more
- macOS: Training, MLX and GGUF inference are ALL supported.
- AMD: Chat + Data works. Train with Unsloth Core. Studio support is out soon.
- Multi-GPU: Available now, with a major upgrade on the way
macOS, Linux, WSL:
curl -fsSL https://unsloth.ai/install.sh | sh
Use the same command to update.
Windows:
irm https://unsloth.ai/install.ps1 | iex
Use the same command to update.
Launch
unsloth studio -p 8888
For cloud or global access, add -H 0.0.0.0. By default, Unsloth is accessible only locally.
Docker
Use our Docker image unsloth/unsloth container. Run:
docker run -d -e JUPYTER_PASSWORD="mypassword" \
-p 8888:8888 -p 8000:8000 -p 2222:22 \
-v $(pwd)/work:/workspace/work \
--gpus all \
unsloth/unsloth
Developer, Nightly, Uninstall
To see developer, nightly and uninstallation etc. instructions, see advanced installation.
Unsloth Core (code-based)
Linux, WSL:
curl -LsSf https://astral.sh/uv/install.sh | sh
uv venv unsloth_env --python 3.13
source unsloth_env/bin/activate
uv pip install unsloth --torch-backend=auto
Windows:
winget install -e --id Python.Python.3.13
winget install --id=astral-sh.uv -e
uv venv unsloth_env --python 3.13
.\unsloth_env\Scripts\activate
uv pip install unsloth --torch-backend=auto
For Windows, pip install unsloth works only if you have PyTorch installed. Read our Windows Guide.
You can use the same Docker image as Unsloth Studio.
AMD, Intel:
For RTX 50x, B200, 6000 GPUs: uv pip install unsloth --torch-backend=auto. Read our guides for: Blackwell and DGX Spark.
To install Unsloth on AMD and Intel GPUs, follow our AMD Guide and Intel Guide.
📒 Free Notebooks
Train for free with our notebooks. You can use our new free Unsloth Studio notebook to run and train models for free in a web UI. Read our guide. Add dataset, run, then deploy your trained model.
| Model | Free Notebooks | Performance | Memory use |
|---|---|---|---|
| Gemma 4 (E2B) | ▶️ Start for free | 1.5x faster | 50% less |
| Qwen3.5 (4B) | ▶️ Start for free | 1.5x faster | 60% less |
| gpt-oss (20B) | ▶️ Start for free | 2x faster | 70% less |
| Qwen3.5 GSPO | ▶️ Start for free | 2x faster | 70% less |
| gpt-oss (20B): GRPO | ▶️ Start for free | 2x faster | 80% less |
| Qwen3: Advanced GRPO | ▶️ Start for free | 2x faster | 70% less |
| embeddinggemma (300M) | ▶️ Start for free | 2x faster | 20% less |
| Mistral Ministral 3 (3B) | ▶️ Start for free | 1.5x faster | 60% less |
| Llama 3.1 (8B) Alpaca | ▶️ Start for free | 2x faster | 70% less |
| Llama 3.2 Conversational | ▶️ Start for free | 2x faster | 70% less |
| Orpheus-TTS (3B) | ▶️ Start for free | 1.5x faster | 50% less |
- See all our notebooks for: Kaggle, GRPO, TTS, embedding & Vision
- See all our models and all our notebooks
- See detailed documentation for Unsloth here
🦥 Unsloth News
- Connections: Connect any API provider (OpenAI, Anthropic) or server (vLLM, Ollama). Guide
- MTP: Run Qwen3.6 MTP in Unsloth. MTP settings are autoset specific to your hardware. Guide
- API inference endpoint: Deploy and run local LLMs in Claude Code, Codex tools. Guide
- Qwen3.6: Qwen3.6-35B-A3B can now be trained and run in Unsloth Studio. Blog
- Gemma 4: Run and train Google’s new models directly in Unsloth. Blog
- Introducing Unsloth Studio: our new web UI for running and training LLMs. Blog
- Qwen3.5 - 0.8B, 2B, 4B, 9B, 27B, 35-A3B, 112B-A10B are now supported. Guide + notebooks
- Train MoE LLMs 12x faster with 35% less VRAM - DeepSeek, GLM, Qwen and gpt-oss. Blog
- Embedding models: Unsloth now supports ~1.8-3.3x faster embedding fine-tuning. Blog • Notebooks
- New 7x longer context RL vs. all other setups, via our new batching algorithms. Blog
- New RoPE & MLP Triton Kernels & Padding Free + Packing: 3x faster training & 30% less VRAM. Blog
- 500K Context: Training a 20B model with >500K context is now possible on an 80GB GPU. Blog
- FP8 & Vision RL: You can now do FP8 & VLM GRPO on consumer GPUs. FP8 Blog • Vision RL
📥 Advanced Installation
The below advanced instructions are for Unsloth Studio. For Unsloth Core advanced installation, view our docs.
Developer installs: macOS, Linux, WSL:
git clone https://github.com/unslothai/unsloth
cd unsloth
./install.sh --local
unsloth studio -p 8888
Then to update :
cd unsloth && git pull
./install.sh --local
unsloth studio -p 8888
Developer installs: Windows PowerShell:
git clone https://github.com/unslothai/unsloth.git
cd unsloth
Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass
.\install.ps1 --local
unsloth studio -p 8888
Then to update :
cd unsloth && git pull
./install.sh --local
unsloth studio -p 8888
Nightly: MacOS, Linux, WSL:
git clone https://github.com/unslothai/unsloth
cd unsloth
git checkout nightly
./install.sh --local
unsloth studio -p 8888
Then to launch every time:
unsloth studio -p 8888
Nightly: Windows:
Run in Windows Powershell:
git clone https://github.com/unslothai/unsloth.git
cd unsloth
git checkout nightly
Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass
.\install.ps1 --local
unsloth studio -p 8888
Then to launch every time:
unsloth studio -p 8888
Advanced launch options
Installer options can be passed as environment variables. On macOS, Linux and WSL place the variable after the pipe so the shell passes it to sh; on Windows set it with $env: before piping to iex.
Skip PyTorch (GGUF-only mode):
curl -fsSL https://unsloth.ai/install.sh | UNSLOTH_NO_TORCH=1 sh
$env:UNSLOTH_NO_TORCH=1; irm https://unsloth.ai/install.ps1 | iex
Pin the Python version:
curl -fsSL https://unsloth.ai/install.sh | UNSLOTH_PYTHON=3.12 sh
$env:UNSLOTH_PYTHON='3.12'; irm https://unsloth.ai/install.ps1 | iex
Install to a custom location with UNSLOTH_STUDIO_HOME:
curl -fsSL https://unsloth.ai/install.sh | UNSLOTH_STUDIO_HOME=/abs/path sh
$env:UNSLOTH_STUDIO_HOME='C:\path'; irm https://unsloth.ai/install.ps1 | iex
Cap Studio's native CPU thread pools on high-core hosts: UNSLOTH_CPU_THREADS=8 unsloth studio -p 8888.
Uninstall
The recommended way to fully remove Unsloth Studio is the matching uninstall script for your OS. It stops any running servers, removes the install dir, the launcher data dir, the desktop shortcut, and any platform-specific entries (macOS .app bundle + Launch Services on Mac; Start Menu, HKCU\Software\Unsloth registry key and user PATH entries on Windows):
- MacOS, WSL, Linux:
curl -fsSL https://raw.githubusercontent.com/unslothai/unsloth/main/scripts/uninstall.sh | sh - Windows (PowerShell):
irm https://raw.githubusercontent.com/unslothai/unsloth/main/scripts/uninstall.ps1 | iex
If you only want to drop the install dir and keep the launcher/shortcut for a later reinstall, you can instead run rm -rf ~/.unsloth/studio (Mac/Linux/WSL) or Remove-Item -Recurse -Force "$HOME\.unsloth\studio" (Windows). The model cache at ~/.cache/huggingface is not touched by any of these.
For more info, see our docs.
Deleting model files
You can delete old model files either from the bin icon in model search or by removing the relevant cached model folder from the default Hugging Face cache directory. By default, HF uses:
- MacOS, Linux, WSL:
~/.cache/huggingface/hub/ - Windows:
%USERPROFILE%\.cache\huggingface\hub\
💚 Community and Links
| Type | Links |
|---|---|
 Discord |
Join Discord server |
 r/unsloth Reddit |
Join Reddit community |
| 📚 Documentation & Wiki | Read Our Docs |
_logo_late_2025.svg){kind=logo} Twitter (aka X) |
Follow us on X |
| 🔮 Our Models | Unsloth Catalog |
| ✍️ Blog | Read our Blogs |
Citation
You can cite the Unsloth repo as follows:
@software{unsloth,
author = {Daniel Han, Michael Han and Unsloth team},
title = {Unsloth},
url = {https://github.com/unslothai/unsloth},
year = {2023}
}
If you trained a model with 🦥Unsloth, you can use this cool sticker!
License
Unsloth uses a dual-licensing model of Apache 2.0 and AGPL-3.0. The core Unsloth package remains licensed under Apache 2.0, while certain optional components, such as the Unsloth Studio UI are licensed under the open-source license AGPL-3.0.
This structure helps support ongoing Unsloth development while keeping the project open source and enabling the broader ecosystem to continue growing.
Thank You to
- The llama.cpp library that lets users run and save models with Unsloth
- The Hugging Face team and their libraries: transformers and TRL
- The Pytorch and Torch AO team for their contributions
- NVIDIA for their NeMo DataDesigner library and their contributions
- And of course for every single person who has contributed or has used Unsloth!