Unsloth is a local UI for training and running Gemma 4, Qwen3.6, DeepSeek, Kimi, GLM and other models. https://unsloth.ai/docs
  • Python 71.5%
  • TypeScript 22.7%
  • Shell 1.9%
  • PowerShell 1.6%
  • Rust 1.5%
  • Other 0.7%
Find a file
Ricardo-M-L 2ef98d382e
fix: use % 8 instead of // 8 in FP8 weight shape check (#5243)
* fix: preserve bf16 GGUF file when explicitly requested in quantization list

When users request multiple quantization methods including the base format
(e.g., ["q4_k_m", "bf16"]), the bf16 GGUF serves as both the intermediate
conversion and a user-requested output. The cleanup step unconditionally
deleted this file, losing the explicitly requested bf16 output.

Only delete the intermediate base GGUF when the user did not request it.

Fixes #4932

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: keep reverse() outside conditional deletion to preserve VLM ordering

Address review feedback: the reverse() call must always execute when
quants_created is True to maintain correct [text_model, mmproj] ordering
for VLMs. Only the file deletion should be conditional on whether the
user requested the base format.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Ricardo-M-L <ricardoporsche001@icloud.com>

* fix: move preserved base GGUF away from list boundaries for correct example commands

Address review from @Datta0: when the base format (e.g. bf16) is kept
in all_saved_locations, it could end up at [-1], causing the VLM example
command to use bf16 as --mmproj instead of the actual projector file.

Move the preserved base file to index 1 (after the primary quantized
model, before mmproj) so [0] and [-1] remain correct for example
commands.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Ricardo-M-L <ricardoporsche001@icloud.com>

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix: use % 8 instead of // 8 in FP8 weight shape check

weight.shape[X] // 8 != 0 is True for any non-zero dimension, causing
incorrect fallback to dequantization for small weights. Using % 8
correctly checks non-divisibility: weights not divisible by 8 should
dequantize, while those with % 8 == 0 stay on the fast kernel path.

* Preserve sharded base GGUF files during cleanup

convert_to_gguf can return multiple base text shards plus an mmproj
entry when llama.cpp splits the output. The previous cleanup only
removed/repositioned base_gguf=initial_files[0]:

- when the base format is NOT in quantization_method, sibling shards
  were left both in all_saved_locations and on disk as orphans
- when the base IS preserved, the reverse + insert(1, base_gguf)
  step left a sibling base shard at all_saved_locations[-1] for VLMs,
  so the example llama-mtmd-cli command ended up with --mmproj
  pointing at a text shard instead of the projector

Treat every initial file whose basename does not contain "-mmproj"
as part of the base set, then remove/unlink or reposition all of
them together. Drop the redundant frozenset() construction at both
call sites and the dead `base_gguf in all_saved_locations` clause
in the reorder guard.

* Apply bias in FP8 dequant fallback and dedupe full-precision flag

unsloth/kernels/fp8.py:
  FbgemmFp8Linear_matmul.forward had a dequant fallback that called
  torch_matmul without adding bias. The fast row-wise branch and the
  block FP8 branch both apply `output = output + bias if bias is not
  None else output` immediately after the matmul; the fallback now
  matches. This silently dropped bias for any FP8 layer routed to the
  fallback (Qwen 2.5 VL gate/up_proj 3420x1280, transposed-weight
  backward dispatch, and the small-shape cases newly routed here by
  the recent `% 8` divisibility fix).

unsloth/save.py:
  preserved_base inside the cleanup block and want_full_precision below
  it computed the identical expression `first_conversion in
  quantization_method`. Hoist want_full_precision above the cleanup
  block, reuse it for the not-preserved deletion and the preserved
  reposition, and assign True directly in the GPT-OSS branch.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: Ricardo-M-L <ricardoporsche001@icloud.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
2026-05-05 03:48:04 -07:00
.github Fix Studio desktop tray installer and titlebar and bux fixes (#5179) 2026-04-30 08:40:39 -07:00
images Add files via upload 2026-04-02 03:00:10 -07:00
scripts Add qwen3.6 script (#5084) 2026-04-17 01:21:30 -07:00
studio unsloth run: add --enable-tools/--disable-tools server-side tool policy (#5277) 2026-05-05 12:45:15 +04:00
tests unsloth run: add --enable-tools/--disable-tools server-side tool policy (#5277) 2026-05-05 12:45:15 +04:00
unsloth fix: use % 8 instead of // 8 in FP8 weight shape check (#5243) 2026-05-05 03:48:04 -07:00
unsloth_cli unsloth run: add --enable-tools/--disable-tools server-side tool policy (#5277) 2026-05-05 12:45:15 +04:00
.gitattributes EOL LF (unix line endings) normalization (#3478) 2025-10-17 16:22:42 -07:00
.gitignore Add tauri (#5144) 2026-04-23 04:50:10 -07:00
.pre-commit-ci.yaml pre-commit CI config (#3565) 2025-11-07 14:44:18 -08:00
.pre-commit-config.yaml [pre-commit.ci] pre-commit autoupdate (#5204) 2026-04-27 14:17:03 -07:00
build.sh perf(studio): upgrade to Vite 8 + auto-install bun for faster frontend builds (#4522) 2026-03-25 04:27:41 -07:00
cli.py Rename cli/ to unsloth_cli/ to fix namespace collision with stringzilla (#4393) 2026-03-17 20:40:21 -07:00
CODE_OF_CONDUCT.md Update CODE_OF_CONDUCT.md 2025-10-25 19:31:05 -07:00
CONTRIBUTING.md Revert "Improve documentation on how to export model from Colab" 2026-03-13 22:38:41 -07:00
COPYING Rename cli/ to unsloth_cli/ to fix namespace collision with stringzilla (#4393) 2026-03-17 20:40:21 -07:00
install.ps1 Default Studio host to 127.0.0.1 and prompt before auto-start (#5267) 2026-05-04 13:03:16 +04:00
install.sh Default Studio host to 127.0.0.1 and prompt before auto-start (#5267) 2026-05-04 13:03:16 +04:00
LICENSE Rename cli/ to unsloth_cli/ to fix namespace collision with stringzilla (#4393) 2026-03-17 20:40:21 -07:00
pyproject.toml Studio: add github_repo seed reader and GitHub Support Bot recipe (#5169) 2026-04-24 12:02:03 -07:00
README.md Default Studio host to 127.0.0.1 and prompt before auto-start (#5267) 2026-05-04 13:03:16 +04:00
unsloth-cli.py Merge pull request #3612 from Vangmay/feature/raw-text-dataprep 2026-01-08 03:38:15 -08:00

Unsloth logo

Unsloth Studio lets you run and train models locally.

FeaturesQuickstartNotebooksDocumentation


unsloth studio ui homepage

Get started

macOS, Linux, WSL:

curl -fsSL https://unsloth.ai/install.sh | sh

Windows:

irm https://unsloth.ai/install.ps1 | iex

Community:

Features

Unsloth Studio (Beta) lets you run and train text, audio, embedding, vision models on Windows, Linux and macOS.

Inference

Training

  • Train and RL 500+ models up to 2x faster with up to 70% less VRAM, with no accuracy loss.
  • Custom Triton and mathematical kernels. See some collabs we did with PyTorch and Hugging Face.
  • Data Recipes: Auto-create datasets from PDF, CSV, DOCX etc. Edit data in a visual-node workflow.
  • Reinforcement Learning (RL): The most efficient RL library, using 80% less VRAM for GRPO, FP8 etc.
  • Supports full fine-tuning, RL, pretraining, 4-bit, 16-bit and, FP8 training.
  • Observability: Monitor training live, track loss and GPU usage and customize graphs.
  • Multi-GPU training is supported, with major improvements coming soon.

📥 Install

Unsloth can be used in two ways: through Unsloth Studio, the web UI, or through Unsloth Core, the code-based version. Each has different requirements.

Unsloth Studio (web UI)

Unsloth Studio (Beta) works on Windows, Linux, WSL and macOS.

  • CPU: Supported for Chat and Data Recipes currently
  • NVIDIA: Training works on RTX 30/40/50, Blackwell, DGX Spark, Station and more
  • macOS: Currently supports chat and Data Recipes. MLX training is coming very soon
  • AMD: Chat + Data works. Train with Unsloth Core. Studio support is out soon.
  • Coming soon: Training support for Apple MLX, AMD, and Intel.
  • Multi-GPU: Available now, with a major upgrade on the way

macOS, Linux, WSL:

curl -fsSL https://unsloth.ai/install.sh | sh

Windows:

irm https://unsloth.ai/install.ps1 | iex

Launch

unsloth studio -p 8888

For cloud VMs or LAN access, add -H 0.0.0.0 to bind on all interfaces.

Update

To update, use the same install commands as above. Or run (does not work on Windows):

unsloth studio update

Docker

Use our Docker image unsloth/unsloth container. Run:

docker run -d -e JUPYTER_PASSWORD="mypassword" \
  -p 8888:8888 -p 8000:8000 -p 2222:22 \
  -v $(pwd)/work:/workspace/work \
  --gpus all \
  unsloth/unsloth

Developer, Nightly, Uninstall

To see developer, nightly and uninstallation etc. instructions, see advanced installation.

Unsloth Core (code-based)

Linux, WSL:

curl -LsSf https://astral.sh/uv/install.sh | sh
uv venv unsloth_env --python 3.13
source unsloth_env/bin/activate
uv pip install unsloth --torch-backend=auto

Windows:

winget install -e --id Python.Python.3.13
winget install --id=astral-sh.uv  -e
uv venv unsloth_env --python 3.13
.\unsloth_env\Scripts\activate
uv pip install unsloth --torch-backend=auto

For Windows, pip install unsloth works only if you have PyTorch installed. Read our Windows Guide. You can use the same Docker image as Unsloth Studio.

AMD, Intel:

For RTX 50x, B200, 6000 GPUs: uv pip install unsloth --torch-backend=auto. Read our guides for: Blackwell and DGX Spark.
To install Unsloth on AMD and Intel GPUs, follow our AMD Guide and Intel Guide.

📒 Free Notebooks

Train for free with our notebooks. You can use our new free Unsloth Studio notebook to run and train models for free in a web UI. Read our guide. Add dataset, run, then deploy your trained model.

Model Free Notebooks Performance Memory use
Gemma 4 (E2B) ▶️ Start for free 1.5x faster 50% less
Qwen3.5 (4B) ▶️ Start for free 1.5x faster 60% less
gpt-oss (20B) ▶️ Start for free 2x faster 70% less
Qwen3.5 GSPO ▶️ Start for free 2x faster 70% less
gpt-oss (20B): GRPO ▶️ Start for free 2x faster 80% less
Qwen3: Advanced GRPO ▶️ Start for free 2x faster 70% less
embeddinggemma (300M) ▶️ Start for free 2x faster 20% less
Mistral Ministral 3 (3B) ▶️ Start for free 1.5x faster 60% less
Llama 3.1 (8B) Alpaca ▶️ Start for free 2x faster 70% less
Llama 3.2 Conversational ▶️ Start for free 2x faster 70% less
Orpheus-TTS (3B) ▶️ Start for free 1.5x faster 50% less

🦥 Unsloth News

  • Qwen3.6: Qwen3.6-35B-A3B can now be trained and run in Unsloth Studio. Blog
  • Gemma 4: Run and train Googles new models directly in Unsloth. Blog
  • Introducing Unsloth Studio: our new web UI for running and training LLMs. Blog
  • Qwen3.5 - 0.8B, 2B, 4B, 9B, 27B, 35-A3B, 112B-A10B are now supported. Guide + notebooks
  • Train MoE LLMs 12x faster with 35% less VRAM - DeepSeek, GLM, Qwen and gpt-oss. Blog
  • Embedding models: Unsloth now supports ~1.8-3.3x faster embedding fine-tuning. BlogNotebooks
  • New 7x longer context RL vs. all other setups, via our new batching algorithms. Blog
  • New RoPE & MLP Triton Kernels & Padding Free + Packing: 3x faster training & 30% less VRAM. Blog
  • 500K Context: Training a 20B model with >500K context is now possible on an 80GB GPU. Blog
  • FP8 & Vision RL: You can now do FP8 & VLM GRPO on consumer GPUs. FP8 BlogVision RL
  • gpt-oss by OpenAI: Read our RL blog, Flex Attention blog and Guide.

📥 Advanced Installation

The below advanced instructions are for Unsloth Studio. For Unsloth Core advanced installation, view our docs.

Developer installs: macOS, Linux, WSL:

git clone https://github.com/unslothai/unsloth
cd unsloth
./install.sh --local
unsloth studio -p 8888

Then to update :

unsloth studio update

Developer installs: Windows PowerShell:

git clone https://github.com/unslothai/unsloth.git
cd unsloth
Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass
.\install.ps1 --local
unsloth studio -p 8888

Then to update :

unsloth studio update

Nightly: MacOS, Linux, WSL:

git clone https://github.com/unslothai/unsloth
cd unsloth
git checkout nightly
./install.sh --local
unsloth studio -p 8888

Then to launch every time:

unsloth studio -p 8888

Nightly: Windows:

Run in Windows Powershell:

git clone https://github.com/unslothai/unsloth.git
cd unsloth
git checkout nightly
Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass
.\install.ps1 --local
unsloth studio -p 8888

Then to launch every time:

unsloth studio -p 8888

Uninstall

You can uninstall Unsloth Studio by deleting its install folder usually located under $HOME/.unsloth/studio on Mac/Linux/WSL and %USERPROFILE%\.unsloth\studio on Windows. Using the rm -rf commands will delete everything, including your history, cache:

  • MacOS, WSL, Linux: rm -rf ~/.unsloth/studio
  • Windows (PowerShell): Remove-Item -Recurse -Force "$HOME\.unsloth\studio"

For more info, see our docs.

Deleting model files

You can delete old model files either from the bin icon in model search or by removing the relevant cached model folder from the default Hugging Face cache directory. By default, HF uses:

  • MacOS, Linux, WSL: ~/.cache/huggingface/hub/
  • Windows: %USERPROFILE%\.cache\huggingface\hub\
Type Links
  Discord Join Discord server
  r/unsloth Reddit Join Reddit community
📚 Documentation & Wiki Read Our Docs
  Twitter (aka X) Follow us on X
🔮 Our Models Unsloth Catalog
✍️ Blog Read our Blogs

Citation

You can cite the Unsloth repo as follows:

@software{unsloth,
  author = {Daniel Han, Michael Han and Unsloth team},
  title = {Unsloth},
  url = {https://github.com/unslothai/unsloth},
  year = {2023}
}

If you trained a model with 🦥Unsloth, you can use this cool sticker!  

License

Unsloth uses a dual-licensing model of Apache 2.0 and AGPL-3.0. The core Unsloth package remains licensed under Apache 2.0, while certain optional components, such as the Unsloth Studio UI are licensed under the open-source license AGPL-3.0.

This structure helps support ongoing Unsloth development while keeping the project open source and enabling the broader ecosystem to continue growing.

Thank You to

  • The llama.cpp library that lets users run and save models with Unsloth
  • The Hugging Face team and their libraries: transformers and TRL
  • The Pytorch and Torch AO team for their contributions
  • NVIDIA for their NeMo DataDesigner library and their contributions
  • And of course for every single person who has contributed or has used Unsloth!