Unsloth is a local UI for training and running Gemma 4, Qwen3.6, DeepSeek, Kimi, GLM and other models.
https://unsloth.ai/docs
agent
deepseek
fine-tuning
gemma
gemma3
gpt-oss
llama
llama3
llm
llms
mistral
openai
qwen
reinforcement-learning
self-hosted
text-to-speech
tts
ui
unsloth
- Python 71.5%
- TypeScript 22.7%
- Shell 1.9%
- PowerShell 1.6%
- Rust 1.5%
- Other 0.7%
7482685757* studio: unblock /load event loop on detect_audio_type (#5642, #5635)
studio/backend/routes/inference.py wraps llama_backend.detect_audio_type
in await asyncio.to_thread() so its chain of sequential sync
httpx.Client.post() probes (/tokenize and /detokenize, 10 s timeout
each) runs on the threadpool instead of blocking the FastAPI event
loop. Without this wrap, /api/inference/load-progress polling and any
other in-flight HTTP request stalls for up to ~80 s while
detect_audio_type runs, which is exactly the "llama-server logs say
ready, Studio UI never finishes loading" symptom in #5642 (Win10) and
#5635 (Win11). The matching init_audio_codec call on the next branch
was already wrapped; this just brings detect_audio_type to parity.
Add a CPU-only spoof-based test suite under tests/studio/load_freeze/:
- llama_server_shim.py: stdlib http.server that answers /health,
/props, /tokenize, /detokenize, /completion with per-request
delay knobs.
- test_load_orchestrator.py:
* test_buggy_route_blocks_event_loop -- behavioural canary:
with a sync detect_audio_type call, concurrent /health
requests stall for >= one tokenize delay (proves the bug
class, runs from worker threads against a real uvicorn).
* test_fixed_route_keeps_event_loop_responsive -- with the
to_thread wrap, concurrent /health latency stays under 250 ms.
* test_routes_inference_wraps_detect_audio_type_in_to_thread --
static guard so the fix cannot regress silently.
* test_fast_path_load_completes_quickly -- regression budget
for post-_wait_for_health work.
Add .github/workflows/studio-load-orchestrator-ci.yml. CPU-only,
no torch, no real llama.cpp binary, no GPU. Cross-OS proof
(ubuntu-latest / macos-14 / windows-latest, 4 passed in 7-10 s each)
ran green on danielhanchen/unsloth-staging-2#136 before landing here.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: expand load-orchestrator suite to 22 tests (failure modes, stress, drift)
Replace the 4-test smoke with a comprehensive 22-test simulation
covering every failure mode of the /load -> detect_audio_type path:
1. Behavioural canary (2) - sync vs to_thread under slow shim
2. Functional equivalence (5) - sync == to_thread for each codec
branch (None / snac / csm / whisper
/ bicodec)
3. Failure modes (5) - shim returns 500, malformed JSON,
connection reset, unreachable port,
backend not loaded
4. Concurrency / stress (2) - 50 concurrent /probe; 100-burst
/health during slow /probe
5. Drift / regression guards (3) - wrap on production source, neighbour
init_audio_codec still wrapped, no
bare detect_audio_type() in any
async route
6. Timing budgets (2) - fast-path under 2s; 5 sequential
/probes under 10s
7. Browser-compat (2) - Content-Type + JSON.parse round-trip
+ response shape stable sync vs fix
8. Cancellation (1) - client disconnect mid-probe; server
keeps serving /health afterwards
Extended llama_server_shim with knobs for HTTP-500, malformed-JSON,
connection-reset, and tok_response_map / detok_map so we can
synthesise the exact request/response shape that triggers each codec
match. No new dependencies, still CPU-only and stdlib-driven.
Cross-OS validation on danielhanchen/unsloth-staging-2#136:
- ubuntu-latest: 22 passed in 19.59s
- macos-14: 22 passed in 22.07s
- windows-latest: 22 passed in 38.79s
Cross-Python on Linux (3.10 / 3.11 / 3.12 / 3.13 x pinned-floor /
latest deps, 8 uv venvs): 176/176 passed.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: move audio detect/codec init inside load_model lock; relax small-quant CI
Follow-up to #5642 fix that addresses two distinct concerns raised by
the gemini-code-assist review on PR #5669:
1. Race condition (medium-priority comment on routes/inference.py:869)
The original fix wrapped llama_backend.detect_audio_type in
asyncio.to_thread. That unblocks the FastAPI event loop but opens
a race window where a concurrent /api/inference/load can acquire
_serial_load_lock, kill the live llama-server, and start a new
one while the first request's detect_audio_type thread is still
probing the (now-dead) port -- the route then writes stale
_is_audio / _audio_type onto the shared backend instance.
Fix: move detect_audio_type + init_audio_codec INSIDE
LlamaCppBackend.load_model, immediately before the function
returns True. Both calls happen while self._serial_load_lock is
held, so the entire load sequence (spawn, wait health, detect
audio, init codec, return) is atomic. routes/inference.py now
just reads the cached _audio_type / _is_audio attributes.
This is the shape the gemini reviewer recommended, and it also
simplifies the route -- no more asyncio.to_thread wrap, no more
conditional init_audio_codec call. The route layer keeps its
non-inference responsibilities (_native_display_label /
_native_grant_backed assignments) since those depend on
route-local arguments.
2. Hardcoded local file path in test shim (gemini's other comment)
FakeLlamaServer's default model_path was a developer-specific
Windows cache path. Replaced with an OS-portable placeholder.
The value is cosmetic-only -- only used in the synthesised stdout
template's "loading model" line, which the production code we
drive from the tests does not parse.
3. Existing CI flake on studio-inference-smoke.yml (generalised fix)
Studio GGUF CI has been red on main and 5+ unrelated PRs all
day. Root cause: small-quant Qwen3.5-2B drifts in two places.
(a) The python tool spits back "55,888" instead of "56088"
even though the tool itself returned the correct value. (b) The
OpenAI / Anthropic determinism check sees occasional non-byte-
identical responses at temperature=0.0 across runs due to KV
cache / speculative-decoding non-determinism. Both are model
output drift, not Studio regressions.
Generalised fix: match the Windows variant's already-lenient
WARN-when-tool-ran-but-model-drifted pattern. SSE-stream-empty
stays a hard FAIL (real plumbing failure); a non-empty stream
with the wrong numeric content becomes a WARN. Determinism
check similarly demotes "trailing whitespace OK but content
diverged" to a WARN; the harder grounding assertions on
later turns (paris present somewhere, turn-1 contains '1')
remain strict and continue to catch real regressions.
Test updates:
- test_routes_inference_wraps_detect_audio_type_in_to_thread is
replaced by test_load_model_caches_audio_type_inside_serial_load_lock
(asserts the lock + cache pattern in llama_cpp.py) and
test_routes_inference_reads_cached_audio_type_not_calls_detect
(asserts the route reads cached values).
- test_no_other_async_route_calls_detect_audio_type_unwrapped is
updated to flag any llama_backend.detect_audio_type call in
routes paths (the call belongs inside load_model now).
Local cross-Python matrix (Linux, Python 3.10 / 3.11 / 3.12 / 3.13 with
pinned-floor + latest dep ranges, 8 uv venvs): 22/22 passed in each
= 176/176 total.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: tool-actually-ran assertion (chatgpt P1); shim port-0 (gemini)
Two PR-review follow-ups on #5669:
1. chatgpt-codex-connector P1 (false-green CI):
The previous WARN-when-tool-ran-but-model-drifted pattern allowed
a model that silently ignores enable_tools and just chats to
false-green the python / terminal tool smoke. Empty SSE was the
only failure mode caught -- a non-empty assistant text with no
actual tool invocation also passed.
Fix: post_sse now also returns the raw event payloads. A new
helper _tool_invoked(events, expected_outputs=...) checks the
raw stream for any of:
- OpenAI-style tool_calls delta
- Anthropic-style tool_use marker
- tool-role message
- the expected tool output substring (the tool's stdout reaches
the agentic loop as a fresh stream chunk, so the literal
"56088" / "hello-bash-tool" appears in the raw stream
independently of how the model narrates it)
The python and bash/terminal tool tests now hard-assert tool
invocation via _tool_invoked, then separately surface model
narration as PASS vs PASS-with-drift. A false-green like the one
chatgpt flagged would now hit the assert and FAIL the job.
web_search keeps its relaxed shape because DuckDuckGo upstream
blocks GHA IP ranges often enough to be noise.
2. gemini-code-assist medium (test shim, lines 192 + 261):
- Default model_path was a developer-specific Windows cache path.
Already replaced last cycle with an OS-portable placeholder.
- _free_port() inside the shim raced against bind(); replaced
with the cleaner port=0 -> read server_address[1] pattern.
The unused _free_port helper inside the shim is removed.
Local sim suite still green (22 passed in 19.91s). Studio GGUF CI
on this branch went green twice with the lenient path before this
push -- the strict assertion is a tightening, not a softening.
* ci(studio-inference-smoke): broaden tool-invocation markers
Add tool_status / tool_start / tool_end / tool_result to the
_tool_invoked marker tuple in studio-inference-smoke.yml. Studio's
routes/inference.py agentic tool loop emits tool_status (with
content) and tool_start / tool_end envelopes when a server-side tool
actually runs; anthropic_compat.py emits tool_use / tool_result.
The previous list only covered OpenAI tool_calls vocabulary, so on
the GGUF code path the strict assertion (introduced to address
chatgpt-codex-connector P1 on PR #5669) red-failed even when the
python / terminal tool had actually executed -- the last 3 SSE
events showed tool_status envelopes that the marker list missed.
Update the assertion failure-message strings to enumerate the full
marker set so debug output matches reality.
Local sim suite remains 22/22 green.
* studio: address chatgpt-codex P1+P2 follow-ups on 237052ff
P1 (.github/workflows/studio-inference-smoke.yml): tighten
_tool_invoked so it only counts strong markers. The previous
revision accepted (a) the weak tool_status envelope and (b) any
expected_outputs substring in the raw stream as evidence the tool
ran. Both let the test false-green:
- tool_status fires on every iteration boundary of Studio's GGUF
tool stream (including empty {"type":"tool_status","content":""}
cursor resets) regardless of whether any tool_call was actually
produced.
- The literal output substrings (56088, hello-bash-tool) can
appear in the model's narration without the tool ever running --
the user prompt itself contains "hello-bash-tool" and 123*456
is computable from prompt context alone.
Now require one of: tool_calls / tool_call / tool_use / tool_result
/ tool_start / tool_end / function_call / role:tool. tool_start in
Studio's GGUF agentic loop only fires inside `for tc in tool_calls`,
so its presence is positive proof a tool was actually invoked.
P2 (studio/backend/core/inference/llama_cpp.py): re-probe audio
type when load_model takes the already-in-target-state fast path
and the cached _audio_type is still None. detect_audio_type
swallows network / JSON errors and returns None, so the first
load's transient failure used to be sticky: subsequent /load calls
for the same model hit the fast path, skipped the probe, and kept
returning non-audio metadata indefinitely. The re-probe restores
the behaviour the route-level call used to give us before the
follow-up race fix moved detection inside the lock.
Local 22-test load_freeze sim suite remains green.
* studio: hard-assert tool_end.result for python+bash tools
Addresses chatgpt-codex-connector P1 review on PR #5669 commit
1a2fba84 ("Keep tool-output assertions hard-failing").
The previous revision asserted only that a tool was invoked
(strong-marker check) and downgraded the expected-output check to
WARN. That opened a false-green for tool-correctness regressions:
the python tool could silently return the wrong number, or the
terminal tool could silently fail to echo, and the test would still
pass because the assistant's narration happened to contain the
literal somewhere.
Add `_tool_output_contains(events, *needles)` which parses each SSE
event payload as JSON and checks the *tool's own output* across
three native shapes:
1. Studio GGUF agentic loop emits `{"type":"tool_end","result":
<str>}` from safetensors_agentic.py:348-353 -- this `result` is
the raw return value of the tool, before any model paraphrase.
2. Anthropic compatibility layer emits `{"type":"tool_result",
"content":[...]}` from anthropic_compat.py:357 -- check the
text blocks.
3. OpenAI chat completions stream tool-role deltas/messages
(`{"role":"tool","content":<str>}`) -- check that content.
Hard-assert that:
- python tool's tool_end.result contains "56088" or "56,088"
- bash tool's tool_end.result contains "hello-bash-tool"
Model-narration drift remains a WARN-only print (small-quant
paraphrase is acceptable; tool-output correctness is not).
Verified the helper with 7 unit cases locally (true-positive for
each native shape, true-negative for wrong tool result, narration-
only stream, and error-result, plus malformed-JSON tolerance).
Local 22-test load_freeze sim suite remains green.
* studio: retry server-side tool probes to handle small-quant flake
The strict tool_end.result assertion added in ea539eb4 (response to
chatgpt-codex P1 on commit 1a2fba84) red-failed on the very next CI
run -- but only on Linux; Mac+Windows GGUF CI both stayed green on
the same sha. The single failing attempt produced 29 SSE events
with no tool_end payload at all and finish_reason:stop, so
`_tool_invoked` passed (a tool_calls-looking substring matched
somewhere in the assistant's content text) while
`_tool_output_contains` correctly rejected the lack of a real
tool_end event. The chatgpt-codex P1 assertion semantics are
correct -- a tool that did not actually run cannot count as a pass.
The cause is small-quant Qwen3.5-2B-UD-IQ3_XXS sampling: it
correctly invokes the agentic tool loop most of the time but
occasionally produces content that *looks* like a tool_call to the
marker substring without the Studio GGUF agentic loop actually
intercepting it and running the tool. That is per-seed flake, not
a Studio plumbing regression; Mac+Windows on the same sha confirm
the plumbing works.
Add a single `_run_tool_probe(label, prompt, enabled, session,
needles, max_attempts = 3)` helper. Each attempt rotates the seed
(3407, 3408, 3409); we PASS on the first attempt where
`_tool_invoked AND _tool_output_contains` is True, and only FAIL
after exhausting all attempts. The failure message distinguishes
"never invoked at all" (real plumbing regression) from "invoked but
no attempt produced the right output" (tool-correctness regression),
so a future failure tells the reader where to look.
Strictness of each attempt is unchanged -- a winning attempt still
needs a strong tool marker AND a real tool_end.result containing
the expected literal. We only widen the chance the model gets to
actually invoke the tool.
Local 22-test load_freeze sim suite remains green. YAML parses.
* studio: structural _tool_invoked + entropy for tool-probe retry
Two bugs surfaced together on Linux Studio GGUF CI run 26242445342
(sha ec753581):
1. `_tool_invoked` was substring-based. Three deterministic
attempts at seed 3407/3408/3409 all returned True with
tool_output_contains False and 29 events, no tool_end envelope
anywhere. The marker substrings (tool_calls, tool_use, etc.)
were matching the model's own chat content text -- e.g. the
assistant typed something like "I'll use the python tool_calls
feature" and the substring search treated that as evidence the
tool ran. Even tool_calls:null inside a delta would match.
Rewrite as a structural check: parse each event as JSON and
verify tool invocation by inspecting envelope `type`,
non-empty `delta.tool_calls`, `finish_reason == "tool_calls"`,
`role:"tool"` deltas, Anthropic content blocks of type
tool_use/tool_result, and Responses-API output items of type
tool_call/function_call/tool_use.
Verified with 9 true-positive and 7 true-negative unit cases.
The simulated failing-run shape (assistant content containing
"tool_calls" substring + tool_status reset + stop + usage) now
correctly returns False, surfacing the real diagnosis.
2. Retry seed rotation was a no-op at temperature 0. llama.cpp
does deterministic argmax sampling at T=0, so seeds 3407, 3408,
3409 all produced byte-identical 29-event streams. Bump
TOOL_PROBE_TEMP to 0.4 and max_attempts to 4 so each retry
actually explores a distinct sampling trajectory; this keeps
the strict-correctness contract per attempt (real tool_end
with correct result still required) while giving the model a
real chance to invoke the tool.
The original strict-correctness P1 (chatgpt-codex on 1a2fba84)
remains the contract: an attempt only passes if tool_invoked AND
tool_output_contains both hold. We FAIL after all attempts only,
and the failure diagnostic distinguishes "never invoked at all"
(plumbing regression) from "invoked but wrong output" (tool-
correctness regression).
Local 22-test load_freeze sim suite remains green. YAML parses.
* studio: split audio detect/init around self._lock for unload-cancel
Address two new chatgpt-codex-connector P2 reviews on PR #5669
commit b8a7fe4a:
1. "Run audio probing outside _lock to keep unload responsive"
(3282819131). detect_audio_type was running inside the phase-3
self._lock critical section. In the worst case it fires 8
sequential httpx.Client.post() calls with timeout=10, so unload
(which also needs self._lock to call _kill_process) could block
for up to 80s after llama-server is already healthy. Move
detect_audio_type outside self._lock; it stays inside
self._serial_load_lock so a concurrent /load still serialises.
2. "Synchronize fast-path codec init with unload lock" (3283177129).
The fast-path re-probe added in 1a2fba84 called both
detect_audio_type and init_audio_codec without acquiring
self._lock. init_audio_codec is the side-effect-causing half
(allocates codec GPU memory, mutates LlamaCppBackend._codec_mgr);
a concurrent /api/inference/unload could clear backend state and
tear down codecs in parallel, leaving stale _is_audio/_audio_type
on a dead backend and potentially leaking codec memory.
Fix: wrap init_audio_codec in a short self._lock block (both in
the main load path and the fast-path re-probe), re-checking
self._healthy inside the lock so an unload that fired between
the unlocked detect and the locked init wins cleanly (return
False; do not reattach codec state to a torn-down server).
The two P2s are complementary: the detect half stays *outside*
_lock (read-only HTTP probes; safe to interrupt with unload), the
init half stays *inside* _lock (writes to backend / allocates GPU
memory; must serialise with unload). Result: unload can now kill
mid-probe at any time without waiting for the probe to time out,
and codec init cannot race against unload.
Local 22-test load_freeze sim suite remains green; AST parses.
* studio: demote tool_end.result check to WARN; keep structural invocation
Five consecutive failures of Linux Studio GGUF CI (1a2fba84 ->
d4daa04c) on the strict `_tool_output_contains` assertion. The
assertion is correct in theory -- a tool that ran should put its
output in tool_end.result -- but unreachable in practice with the
Studio-runnable models on hand:
* Cross-checked: main (sha
|
||
|---|---|---|
| .github | ||
| images | ||
| scripts | ||
| studio | ||
| tests | ||
| unsloth | ||
| unsloth_cli | ||
| .gitattributes | ||
| .gitignore | ||
| .pre-commit-ci.yaml | ||
| .pre-commit-config.yaml | ||
| build.sh | ||
| cli.py | ||
| CODE_OF_CONDUCT.md | ||
| CONTRIBUTING.md | ||
| COPYING | ||
| install.ps1 | ||
| install.sh | ||
| LICENSE | ||
| pyproject.toml | ||
| README.md | ||
| unsloth-cli.py | ||
Unsloth Studio lets you run and train models locally.
Features • Quickstart • Notebooks • Documentation
⚡ Get started
macOS, Linux, WSL:
curl -fsSL https://unsloth.ai/install.sh | sh
Windows:
irm https://unsloth.ai/install.ps1 | iex
Community:
⭐ Features
Unsloth Studio (Beta) lets you run and train text, audio, embedding, vision models on Windows, Linux and macOS.
Inference
- Search + download + run models including GGUF, LoRA adapters, safetensors
- Export models: Save or export models to GGUF, 16-bit safetensors and other formats.
- Tool calling: Support for self-healing tool calling and web search
- Code execution: lets LLMs test code in Claude artifacts and sandbox environments
- API inference endpoint: Deploy and run local LLMs in Claude Code, Codex tools with Unsloth
- Auto set inference settings and customize chat templates.
- We work directly with teams behind gpt-oss, Qwen3, Llama 4, Mistral, Gemma 1-3, and Phi-4, where we’ve fixed bugs that improve model accuracy.
- Chat with images, audio, PDFs, code, DOCX and more. Connect API providers (OpenAI, Anthropic) or servers (vLLM, Ollama).
Training
- Train and RL 500+ models up to 2x faster with up to 70% less VRAM, with no accuracy loss.
- Custom Triton and mathematical kernels. See some collabs we did with PyTorch and Hugging Face.
- Data Recipes: Auto-create datasets from PDF, CSV, DOCX etc. Edit data in a visual-node workflow.
- Reinforcement Learning (RL): The most efficient RL library, using 80% less VRAM for GRPO, FP8 etc.
- Supports full fine-tuning, RL, pretraining, 4-bit, 16-bit and, FP8 training.
- Observability: Monitor training live, track loss and GPU usage and customize graphs.
- Multi-GPU training is supported, with major improvements coming soon.
📥 Install
Unsloth can be used in two ways: through Unsloth Studio, the web UI, or through Unsloth Core, the code-based version. Each has different requirements.
Unsloth Studio (web UI)
Unsloth Studio (Beta) works on Windows, Linux, WSL and macOS.
- CPU: Supported for Chat and Data Recipes currently
- NVIDIA: Training works on RTX 30/40/50, Blackwell, DGX Spark, Station and more
- macOS: Training, MLX and GGUF inference are ALL supported.
- AMD: Chat + Data works. Train with Unsloth Core. Studio support is out soon.
- Multi-GPU: Available now, with a major upgrade on the way
macOS, Linux, WSL:
curl -fsSL https://unsloth.ai/install.sh | sh
Windows:
irm https://unsloth.ai/install.ps1 | iex
Launch
unsloth studio -p 8888
For cloud or global access, add -H 0.0.0.0. By default, Unsloth is accessible only locally.
Update
To update, use the same install commands above or use unsloth studio update.
Docker
Use our Docker image unsloth/unsloth container. Run:
docker run -d -e JUPYTER_PASSWORD="mypassword" \
-p 8888:8888 -p 8000:8000 -p 2222:22 \
-v $(pwd)/work:/workspace/work \
--gpus all \
unsloth/unsloth
Developer, Nightly, Uninstall
To see developer, nightly and uninstallation etc. instructions, see advanced installation.
Unsloth Core (code-based)
Linux, WSL:
curl -LsSf https://astral.sh/uv/install.sh | sh
uv venv unsloth_env --python 3.13
source unsloth_env/bin/activate
uv pip install unsloth --torch-backend=auto
Windows:
winget install -e --id Python.Python.3.13
winget install --id=astral-sh.uv -e
uv venv unsloth_env --python 3.13
.\unsloth_env\Scripts\activate
uv pip install unsloth --torch-backend=auto
For Windows, pip install unsloth works only if you have PyTorch installed. Read our Windows Guide.
You can use the same Docker image as Unsloth Studio.
AMD, Intel:
For RTX 50x, B200, 6000 GPUs: uv pip install unsloth --torch-backend=auto. Read our guides for: Blackwell and DGX Spark.
To install Unsloth on AMD and Intel GPUs, follow our AMD Guide and Intel Guide.
📒 Free Notebooks
Train for free with our notebooks. You can use our new free Unsloth Studio notebook to run and train models for free in a web UI. Read our guide. Add dataset, run, then deploy your trained model.
| Model | Free Notebooks | Performance | Memory use |
|---|---|---|---|
| Gemma 4 (E2B) | ▶️ Start for free | 1.5x faster | 50% less |
| Qwen3.5 (4B) | ▶️ Start for free | 1.5x faster | 60% less |
| gpt-oss (20B) | ▶️ Start for free | 2x faster | 70% less |
| Qwen3.5 GSPO | ▶️ Start for free | 2x faster | 70% less |
| gpt-oss (20B): GRPO | ▶️ Start for free | 2x faster | 80% less |
| Qwen3: Advanced GRPO | ▶️ Start for free | 2x faster | 70% less |
| embeddinggemma (300M) | ▶️ Start for free | 2x faster | 20% less |
| Mistral Ministral 3 (3B) | ▶️ Start for free | 1.5x faster | 60% less |
| Llama 3.1 (8B) Alpaca | ▶️ Start for free | 2x faster | 70% less |
| Llama 3.2 Conversational | ▶️ Start for free | 2x faster | 70% less |
| Orpheus-TTS (3B) | ▶️ Start for free | 1.5x faster | 50% less |
- See all our notebooks for: Kaggle, GRPO, TTS, embedding & Vision
- See all our models and all our notebooks
- See detailed documentation for Unsloth here
🦥 Unsloth News
- Connections: Connect any API provider (OpenAI, Anthropic) or server (vLLM, Ollama). Guide
- MTP: Run Qwen3.6 MTP in Unsloth. MTP settings are autoset specific to your hardware. Guide
- API inference endpoint: Deploy and run local LLMs in Claude Code, Codex tools. Guide
- Qwen3.6: Qwen3.6-35B-A3B can now be trained and run in Unsloth Studio. Blog
- Gemma 4: Run and train Google’s new models directly in Unsloth. Blog
- Introducing Unsloth Studio: our new web UI for running and training LLMs. Blog
- Qwen3.5 - 0.8B, 2B, 4B, 9B, 27B, 35-A3B, 112B-A10B are now supported. Guide + notebooks
- Train MoE LLMs 12x faster with 35% less VRAM - DeepSeek, GLM, Qwen and gpt-oss. Blog
- Embedding models: Unsloth now supports ~1.8-3.3x faster embedding fine-tuning. Blog • Notebooks
- New 7x longer context RL vs. all other setups, via our new batching algorithms. Blog
- New RoPE & MLP Triton Kernels & Padding Free + Packing: 3x faster training & 30% less VRAM. Blog
- 500K Context: Training a 20B model with >500K context is now possible on an 80GB GPU. Blog
- FP8 & Vision RL: You can now do FP8 & VLM GRPO on consumer GPUs. FP8 Blog • Vision RL
📥 Advanced Installation
The below advanced instructions are for Unsloth Studio. For Unsloth Core advanced installation, view our docs.
Developer installs: macOS, Linux, WSL:
git clone https://github.com/unslothai/unsloth
cd unsloth
./install.sh --local
unsloth studio -p 8888
Then to update :
unsloth studio update
Developer installs: Windows PowerShell:
git clone https://github.com/unslothai/unsloth.git
cd unsloth
Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass
.\install.ps1 --local
unsloth studio -p 8888
Then to update :
unsloth studio update
Nightly: MacOS, Linux, WSL:
git clone https://github.com/unslothai/unsloth
cd unsloth
git checkout nightly
./install.sh --local
unsloth studio -p 8888
Then to launch every time:
unsloth studio -p 8888
Nightly: Windows:
Run in Windows Powershell:
git clone https://github.com/unslothai/unsloth.git
cd unsloth
git checkout nightly
Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass
.\install.ps1 --local
unsloth studio -p 8888
Then to launch every time:
unsloth studio -p 8888
Uninstall
The recommended way to fully remove Unsloth Studio is the matching uninstall script for your OS. It stops any running servers, removes the install dir, the launcher data dir, the desktop shortcut, and any platform-specific entries (macOS .app bundle + Launch Services on Mac; Start Menu, HKCU\Software\Unsloth registry key and user PATH entries on Windows):
- MacOS, WSL, Linux:
curl -fsSL https://raw.githubusercontent.com/unslothai/unsloth/main/scripts/uninstall.sh | sh - Windows (PowerShell):
irm https://raw.githubusercontent.com/unslothai/unsloth/main/scripts/uninstall.ps1 | iex
If you only want to drop the install dir and keep the launcher/shortcut for a later reinstall, you can instead run rm -rf ~/.unsloth/studio (Mac/Linux/WSL) or Remove-Item -Recurse -Force "$HOME\.unsloth\studio" (Windows). The model cache at ~/.cache/huggingface is not touched by any of these.
For more info, see our docs.
Deleting model files
You can delete old model files either from the bin icon in model search or by removing the relevant cached model folder from the default Hugging Face cache directory. By default, HF uses:
- MacOS, Linux, WSL:
~/.cache/huggingface/hub/ - Windows:
%USERPROFILE%\.cache\huggingface\hub\
💚 Community and Links
| Type | Links |
|---|---|
 Discord |
Join Discord server |
 r/unsloth Reddit |
Join Reddit community |
| 📚 Documentation & Wiki | Read Our Docs |
_logo_late_2025.svg){kind=logo} Twitter (aka X) |
Follow us on X |
| 🔮 Our Models | Unsloth Catalog |
| ✍️ Blog | Read our Blogs |
Citation
You can cite the Unsloth repo as follows:
@software{unsloth,
author = {Daniel Han, Michael Han and Unsloth team},
title = {Unsloth},
url = {https://github.com/unslothai/unsloth},
year = {2023}
}
If you trained a model with 🦥Unsloth, you can use this cool sticker!
License
Unsloth uses a dual-licensing model of Apache 2.0 and AGPL-3.0. The core Unsloth package remains licensed under Apache 2.0, while certain optional components, such as the Unsloth Studio UI are licensed under the open-source license AGPL-3.0.
This structure helps support ongoing Unsloth development while keeping the project open source and enabling the broader ecosystem to continue growing.
Thank You to
- The llama.cpp library that lets users run and save models with Unsloth
- The Hugging Face team and their libraries: transformers and TRL
- The Pytorch and Torch AO team for their contributions
- NVIDIA for their NeMo DataDesigner library and their contributions
- And of course for every single person who has contributed or has used Unsloth!