ceos/unsloth: Unsloth is a local UI for training and running Gemma 4, Qwen3.6, DeepSeek, Kimi, GLM and other models.

Unsloth is a local UI for training and running Gemma 4, Qwen3.6, DeepSeek, Kimi, GLM and other models. https://unsloth.ai/docs

agent deepseek fine-tuning gemma gemma3 gpt-oss llama llama3 llm llms mistral openai qwen reinforcement-learning self-hosted text-to-speech tts ui unsloth

Python 71.5%
TypeScript 22.7%
Shell 1.9%
PowerShell 1.6%
Rust 1.5%
Other 0.7%

Find a file

Eyera 27b6d553fe Feat/model picker per model config v2 (#7207 ) * refactor(studio): move chat model picker into features/model-picker Relocate model-selector + its support files from components/assistant-ui into a self-contained features/model-picker feature (own barrel), mirroring the modular Hub layout. Pure move + import repoint; no behaviour change. * feat(model-picker): add per-model config persistence layer Superset PerModelConfig (customContextLength, kvCacheDtype, speculativeType, specDraftNMax, tensorParallel, chatTemplateOverride, trustRemoteCode) persisted to localStorage (unsloth_model_configs) with schema versioning + LRU budget. KV-dtype and speculative value sets match main's sidebar (no q4_0/ngram-simple). Reuses features/hub/lib/model-identity for normalization; adds storage-key layer and applyPerModelConfigToRuntime (sets tensorParallel, which the old PR omitted). * feat(picker): modular backend for chat-template validate + default fetch New studio/backend/picker package (schemas/service/routes) mounted at /api/picker: - POST /api/picker/validate-chat-template (Jinja syntax validation, no false positives) - GET /api/picker/chat-template/{model_name} (default template from tokenizer_config.json, reusing get_cache_path/resolve_cached_repo_id_case; graceful null, no model-code exec) Frontend api/templates.ts client + hooks/use-model-defaults lazy cache. No backend changes to the existing inference load route (per-model load fields already supported). * feat(model-picker): bind picker on-device list to shared hub inventory Picker now sources cached + local models from useHubInventory (the Hub's shared store) via a thin adapter, replacing its own /api/models/* fetchers + module caches. Hub, download manager, and picker now share one source of truth, so completed downloads reflect in the picker automatically. Partial/live-download rows are filtered from the cached lists (unchanged rendering). Local naming/search preserved via additive LocalInventoryRow modelId/displayName. Variant expander, scan-folder management, recommended-fit, search, external providers untouched. Known minor: cached 'Downloaded date' sort tiebreak degrades to alphabetical (hub cached rows carry no mtime); default 'recent' (load-time) sort preserved. * feat(model-picker): per-model config step inside the picker Picking a (non-external) model now opens an in-picker config view built from main's current load controls (context length, KV cache dtype, speculative decoding, draft tokens, tensor parallel) plus a chat-template editor backed by the picker validate/default endpoints. 'Remember for this model' persists the config per model+variant; Run forwards the config to the existing load flow via meta.config. External models bypass the step. Two-view orchestration lives in model-selector (single interception point); pickers.tsx call sites untouched. trustRemoteCode dropped from PerModelConfig to preserve main's per-load consent. * feat(chat): apply/persist per-model config through the load flow handleCheckpointChange threads meta.config into the selection; stageOrLoad and the autoload/Hub-run paths now apply the picker config (explicit pick or saved remembered config) via applyPerModelConfigToRuntime before staging/loading, with keepSpeculative set so a remembered speculative mode survives the model switch. Replaces the old remembered-load-settings seeding (resolveInitialConfig now the single source). SelectedModelInput carries config. * refactor(chat): remove per-model load config from the right sidebar The load knobs (context, KV cache, speculative, draft tokens, tensor parallel) and the chat-template editor now live only in the picker config step. The sheet's Model section keeps the staged Load/Cancel flow (config is applied at pick time); sampling params, system prompt, and RAG are unchanged. Deletes the superseded remembered-load-settings module + the store's applyRememberedLoadSettings action, removes the now-dead sheet state/imports, and points the settings reset at unsloth_model_configs. Delete-cleanup deferred (stale config is LRU-capped). * fix(model-picker): remove leftover sidebar-staging cogwheel + empty Model section The downloaded-variant gear (ModelLoadSettingsAction) staged a model straight into the right-sidebar Run-settings flow -- the old 'configure before load' path now fully replaced by the in-picker config step. Removed the gear + its component. Also gate the sheet's 'Model' section to staged picks only (pendingSelection): after the load-knob strip its content is staged-only, so it was rendering an empty section header whenever a model was merely loaded. * chore(chat): remove dead per-model-config setters + modelControlsDisabled After the load-config UI moved into the picker, the store's per-model setters (setKvCacheDtype/setSpeculativeType/setSpecDraftNMax/setTensorParallel/ setCustomContextLength/setChatTemplateOverride) had zero callers (applyPerModelConfigToRuntime writes via setState), and the sheet's modelControlsDisabled was unreferenced. Verified dead across the whole tree. * fix(chat): config-step Load actually loads (ignore Load-on-selection) Root cause: with Settings > Chat > 'Load on selection' turned OFF, the config step's load went down the deferred-staging path -- opening the right sidebar with '<model> is staged, not loaded yet / Choose Load model'. The in-picker config step IS the deliberate load action, so its Load now loads immediately (or downloads + auto-loads when not cached) regardless of the toggle. Renamed the button 'Run model' -> 'Load model' to match. Native/dropped picks still honor the toggle. * refactor(chat,hub): retire 'Load on selection' — config step is the only load flow The in-picker config step (and the Hub Run button) now fully supersede the old stage-to-sidebar flow, so the Load-on-selection toggle is removed everywhere: - chat stageOrLoad: every pick loads immediately, or downloads + auto-loads when not cached (the previous default behaviour, now universal). - hub Run: drops the stage branch; downloaded GGUFs load directly with their saved per-model config (no collision with the chat config step — both end at selectModel). - store: removed loadOnSelection field/setter/key/default; Settings>Chat toggle and its settings-reset entry removed. - staged sidebar section is now a download-progress view (auto-loads on completion). No manual staging remains; stageModel is used only for background auto-load downloads. * feat(model-picker): default chat template from GGUF + thread variant through config flow Read the embedded tokenizer.chat_template from GGUF files (read_gguf_chat_template in gguf_metadata) and use it as the per-model default. Plumb gguf_variant through the picker service, /api/picker/chat-template route, frontend templates API, and use-model-defaults so the right variant's template is fetched. Also refine the picker config-page/model-selector wiring, drop the dead ggufNativeContextLength runtime path, and add the per-model-config storage keys to the settings prefs export. * feat(model-picker): read safetensors chat template + hide editor where it has no effect Resolve the default chat template for safetensors models: prefer the modern chat_template.jinja, fall back to the tokenizer_config.json chat_template field, then chat_template.json (multimodal processor), then the GGUF embedded template. Applied to local dirs, the HF cache snapshot scan, and the HF remote fetch. Hide the chat-template editor in the picker for safetensors models — the override is only applied at load by the GGUF/llama.cpp backend, so editing it on safetensors currently has no effect. GGUF keeps the editor. Nothing removed; the dialog stays for when the safetensors apply path is wired up in a later branch. * fix(model-picker): set legacy-migration flag only after the write succeeds Set unsloth_model_configs_migrated only once writeMap confirms the migrated map persisted, so a quota/storage failure no longer marks migration done and silently drops the user's pre-existing remembered settings — the next load retries. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * MVP model picker fixes * MVP picker config fix * MVP safetensors config * MVP max seq config * MVP max seq fix * Fix static max tokens cap ignoring model context * Fix picker GGUF scan parity * fix(studio): harden model picker config loading Apply remembered per-model configs consistently from picker and Hub loads, keep default configs from overriding standing speculative settings, add config access for direct local GGUF files, and support saving or forgetting active model settings without a reload. * Fix model picker config flow * Fix model picker config loads * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Avoid recursive per-model config migration reads * Apply the displayed context length when loading a GGUF * Fix template validation, cached template lookup, and failed load rollback - Validate chat templates with the loopcontrols extension so templates that use break or continue tags pass the picker validator, matching the inference renderer that already accepts them. - Read the default chat template from the newest cache snapshot rather than an arbitrary iterdir order, so an older cached revision no longer prefills a stale template. - Capture the runtime per-model config before a load and reapply it when the load fails, so a failed switch leaves the active model context, KV cache, template, and speculative settings as they were. * Make chat template view only for safetensors models Custom chat template overrides are applied at inference only for GGUF models, which pass the template to llama-server. The safetensors backend renders with the model built-in template and ignores the override, so editing it would save a value that never loads. For safetensors the config page now opens the template as a read-only preview with a note that editing is not available yet. This can become editable once inference support for custom safetensors templates lands in main. * Fix model picker config edge cases - Restore prior runtime config when a load no-ops for the active model - Cap the picker validator request body via the protected prefixes - Keep the GGUF context slider max above the loaded context - Fetch subfolder chat templates for uncached Hub repos - Show the compare side config when reopening the picker * Keep saved GGUF context above the fallback ceiling * Show the model config in the run settings sidebar * Fix model config sidebar reset and context slider - Stack the remember toggle and action buttons in the sidebar - Reset the config to defaults instead of the loaded values - Fetch the native context so the slider max is not the loaded value * Fix model picker config and download regressions - Run picker chat template routes off the event loop - Depth and root guard local template directory scans - Restore download manager flow for uncached hub picks - Apply per model context length on reload - Import model picker symbols from the feature barrel * Fix model picker config and cached download sorting - Restore load settings when a Hub run is rejected mid load - Reuse one NumericValueInput instead of a duplicate copy - Fix double decode of the model name in the template route - Remove the unused reset-to-loaded settings action - Fix cached model download sorting * Fix model picker per-model config edge cases Honor a saved or typed max seq length above the model's native context so RoPE extended values are no longer clamped and silently overwritten. Allow typing past native while the slider keeps native as a soft ceiling. Guard the fetch success paths in use-model-defaults against an aborted signal, and refetch when the HF token changes. Hash the chat template content in the sidebar remount key instead of its length. Enable reset for a GGUF whose native context is unknown, and floor the context slider max so it can never fall below the min. * Fix GGUF context auto-fit and gated model config token Stop forcing a 32768 context when a GGUF native context is unknown so the backend auto-fits to VRAM again, while still honoring an explicit context edit. Send the HF token as a query param so gated safetensors models resolve their max position embeddings. Derive model default state during render to drop the set-state-in-effect calls. * Fix native GGUF context ceiling and guard picker template reads Restore the native context store field so the sidebar slider keeps the full ceiling for drag and drop GGUFs. Limit local chat template reads to the browse allowlist, skip malformed repo ids, and drop unused model picker exports. * Fix model picker lint boundaries * Fix model picker review findings Chat template editor never seeded its draft. Radix only calls onOpenChange from internal events, so the seed in the nextOpen branch was dead and a model with a saved override opened empty. Saving then cleared the override. Drop the dead branch, treat draft as an untouched sentinel, and reset it on every close. Uncached Hub picks could auto load a model after the user left the chat. Main detached the staged pick on route exit and on chat context change. Carry the context key on the pending pick and skip the load when it no longer matches. Also clear configTarget when the picker closes, restore the onUpdated ref so variant rows stop resubscribing on every parent render, skip the LRU write when the entry is already most recent, import NumericValueInput relatively, and drop the unused ModelUpdateAction barrel export. * Preserve GGUF context on active reload * Fix model picker per-model config regressions - Stop reloading the already loaded model on re-pick - Hide infra models from the chat picker - Detect vision support on cached GGUF repos - Honor saved maxSeqLength on auto load - Restore default chat template for local GGUFs - Warn on save failure and revert config on cancel - Refetch picker inventory on open - Persist read only per model config safely * Fix stale model auto load * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix model picker numeric input sizing and constraints Size value inputs to their content so long context lengths are not clipped, restrict them to numeric characters, and stop the speculative decoding label from truncating in the sidebar. * Fix picker CI tests and harden chat template resolution for PR #6647 - tests: point the descender guard at the moved model-selector.tsx path - tests: exclude the disabled Reload model button from the regenerate locator so .first targets the real Regenerate - picker/service.py: reject symlinked template/gguf leaves that resolve outside the browse allowlist (HF cache reads unchanged) - compare mode: resolve each pane's own remembered chat template instead of inheriting the other pane's from the store * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Protect future-schema per-model configs from deletion for PR #6647 savePerModelConfig already refuses to overwrite a stored config whose schema version is newer than this client understands, but deletePerModelConfig did not. Unchecking Remember on an older client therefore silently destroyed a newer client's saved config. Apply the same guard on delete and surface the blocked case through the existing saveFailed toast. * Protect future-schema per-model configs from quota eviction for PR #6647 The save and delete guards already refuse to touch a stored config whose schema version is newer than this client understands, but the quota-eviction path did not, so a full store on an older client could still evict a newer client's config. Skip future-schema entries when evicting and fail the save if the budget cannot be met without them. * Fix GGUF context persistence, compare context, and rollback settings for PR #6647 Persist a GGUF context override from the user's intent instead of collapsing it against the loaded context, which reintroduced the context-reset (`f4838782cb` reverted the native-baseline fix). model-config-page now collapses the saved value against native, and use-chat-model-runtime and chat-adapter retain the requested context on load so re-saving another setting keeps the override; a null request stays null so a VRAM auto-fit never becomes a stored override. shared-composer: a compare pane with no explicit GGUF context now loads at native (0) like single-view, not the session maxSeqLength that silently shrank the shown context. use-chat-model-runtime: restore the previous model's KV cache dtype and chat template on a failed-load rollback so it runs as it was, not with backend defaults. * Preserve native path token when reloading the active model for PR #6647 handleReloadActiveModel rebuilt the selection without the store's activeNativePathToken, so reloading a file-picked GGUF after a settings change validated the display label as a repo/path and failed. Thread the active native token through the reload selection so native-loaded models reopen correctly. * Make picker template validation resilient and accept HF generation tags for PR #6647 Import Jinja lazily inside validate_chat_template so a backend without the optional jinja2 package (GGUF-only installs) still starts instead of raising ModuleNotFoundError at import time. Register a no-op extension for the Transformers {% generation %} assistant-mask tag so pasting a valid HF chat template validates, matching the renderer, rather than being rejected as an unknown tag. * Honor remembered compare config and parse processor chat_template.json for PR #6647 * Fix failed-load rollback context and processor template map fallback for PR #6647 * Restore speculative decoding config on failed-switch rollback When a model switch fails after the previous model was unloaded, the rollback reload restored tensor_parallel, KV cache dtype and the chat template override, but omitted speculative_type and spec_draft_n_max and cleared their loaded shadows to null. The previous model therefore came back running at backend defaults (speculation off) while the UI still showed it enabled, and the status resync confirmed the off state. Resend the previous model's speculative settings in the rollback load and keep the store's active and loaded speculative fields in sync with them. * Reset max sequence length when a model has no saved config applyPerModelConfigToRuntime reset every per-model field except maxSeqLength, which it only wrote when the incoming config had one. maxSeqLength is the sole field carried on store.params, so selecting a model with no remembered config left the previous model's value in place and later loaded the new model at that leaked length. Fall back to the standing default so an unremembered model loads at its own default. * Surface a message when a variant update cannot start startManagedUpdate handled the conflict and error start outcomes but let busy fall through as if the update began, so the confirm dialog closed with no job created and the cached variant stayed stale. Show an info message when the repo is busy with a sibling transfer so the click is not silently dropped. * Keep per-model speculative choices out of the global default A staged load with a per-model or one-off config sets keepSpeculative, which already skips reading the global speculative preference. The matching save still ran unconditionally, so the model-specific choice was written to the global unsloth_chat_speculative_type and a later model with no saved config started from it instead of Auto. Skip saveSpeculativeType when keepSpeculative so the per-model choice stays isolated. * Seed non-active model settings from the app default max length The Run settings page captured initialMaxSeqLength from the loaded model's runtime params and fell back to it for a model with no saved config. Opening settings for a different, unloaded model and clicking Load then sent the active model's context (for example 64k) instead of the 4096 default, risking validation failures or OOMs. Seed the default for non-active models and keep the runtime value only for the active one. * Prefer sidecar tokenizer chat template over the GGUF copy for variants _chat_template_from_dir returned the embedded GGUF template first when a variant was selected, reversing the tokenizer-first precedence of the no-variant path. A model whose chat_template.jinja or tokenizer_config.json supersedes a stale embedded template then got the wrong template on variant selection. Keep tokenizer files first regardless of variant; the variant only picks which GGUF is the fallback. Adds regression tests for both the tokenizer-wins and gguf-fallback cases. * Keep per-model speculative choices load-local in autoload and compare The interactive load path treats a per-model speculative choice as load-local and skips writing it to the global default. Autoload and generalized compare still called saveSpeculativeType unconditionally, so a remembered off or ngram setting leaked into unsloth_chat_speculative_type and later models with no saved config inherited it. Persist the global preference only when the value came from the global settings. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: record the compare pane's loaded context in runtime state so the active model's settings and any reload or save use it, not the previous context * Studio: notify the user when a Hub autoload can't start because another download for the model is already running, instead of silently dropping it * Studio: drop the merge's orphaned staged-model store helpers and unused alert imports The main merge left isPendingGguf and pendingSelectionMatches referencing the removed PendingModelSelection type, and the alert-dialog/alert imports unused after the permission-mode dropdown replaced the bypass dialog, so tsc -b failed. * Studio: cache a null default chat template so the viewer stops re-fetching it A model with no sidecar or embedded template resolves to a terminal null, but that result was never cached, so reopening the template viewer re-ran the backend and Hugging Face lookup every time. * Studio: detect direct-file GGUFs in run settings so Max Tokens uses their context A GGUF loaded from a local file or custom folder has no variant label, so the run-settings panel treated it as non-GGUF and clamped Max Tokens to the session max_seq_length instead of the loaded GGUF context. Detect it via the reported GGUF context and the .gguf checkpoint suffix, matching the chat page. * Studio: prompt to re-select a local model file when its lease expired before reload A file-picked GGUF is reachable only through a native path token that the desktop host prunes after a TTL. Reloading reused that token blindly, so a reload long after the initial load failed with an opaque error. Track the token's expiry and, when it has passed, ask the user to re-select the file instead of attempting a doomed reload. * Fix descender-clipping test to tolerate sidebar layout utilities The sidebar account-block div carries layout utilities (min-w-0, flex-1) between 'flex' and 'flex-col', so the descender-clipping guard's regex, which required 'flex' immediately followed by 'flex-col', no longer matched and the test failed to locate the account-block div. Generalize the prefix to allow intervening flex utilities while still capturing the leading-* class before the collapsible visibility utility and asserting leading-tight, so the guard against clipped glyph descenders is fully preserved. * Harden picker chat-template resolution Enforce the 64 KiB chat-template contract at the validate endpoint's request model so a direct caller cannot submit a template far larger than the frontend allows (MaxBodyMiddleware only bounds the whole request body, not this field); oversized templates now return a clean 422. Apply sidecar-over-GGUF template precedence globally across cached snapshots instead of per snapshot. A repo with multiple cached revisions previously returned the first snapshot's template, so a newer GGUF-only revision could win over an older revision's maintained chat_template.jinja sidecar, which contradicted the documented intent that sidecars supersede the embedded copy. * Guard per-model config against future-schema and lossy migration Two forward-compatibility gaps in the versioned per-model config store: - The load/apply path returned and normalized a stored record without checking its schema version, so a record written by a newer client was reinterpreted under the current schema and applied to a live model load, even though save, delete and eviction all refuse to touch future-schema records. Reject future-schema records on load too. - The one-time legacy migration enforced the storage budget without protecting the entries it had just migrated and set the completion flag unconditionally. When storage was already full of future-schema records (which are unevictable by an older client), the migrated entries were the only evictable ones and could be dropped while migration was still marked complete. Protect the migrated keys during eviction and only mark migration complete when they survive, so it retries once space frees up. * Discard chat-template validation results after the dialog closes Server-side template validation is async, but closing or cancelling the editor did not abort it, so a late-arriving valid response still called onSave and applied a template the user had already dismissed. Track a validation token that is bumped on close and ignore any validation result whose token is stale. * Record native lease expiry when loading a picked GGUF from the chip The pending-native-model chip loaded via stageOrLoad directly, bypassing loadNativeModelIntent, so activeNativePathExpiresAtMs was never recorded for a chip-loaded file. A later reload then either skipped the lease-expiry guard entirely (expiry left null) or compared against a previously loaded file's stale expiry, so reload could reuse an already-pruned token or wrongly block a still-valid one. Route the chip through loadNativeModelIntent, which builds the same selection and records the expiry. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Prefer sidecar template for a directly selected local GGUF file A direct .gguf file path read its embedded chat template without checking the parent directory for a maintained sidecar (chat_template.jinja / tokenizer_config.json), while directory and variant selections already prefer the sidecar. That let the config editor preview or save a stale embedded template for the same model depending on how it was selected. Check the parent directory sidecars first, then fall back to the embedded copy, and cover both paths with tests. * Resolve cached chat template per revision, newest first The earlier change searched every cached snapshot for a sidecar before considering any snapshot's embedded GGUF template, which let an obsolete sidecar from an older revision override the newest revision's template. Restore per-snapshot resolution (newest first): a revision's sidecar still supersedes its own embedded GGUF copy, but a newer revision is no longer overridden by an older revision's sidecar. * Preserve autoload transport conflicts and surface background busy downloads - When a Hub autoload hits a transport conflict, keep pendingHubAutoLoad bound instead of clearing it. Clearing it re-keyed the download surface and its cleanup cancelled the conflict the toast tells the user to resolve, so the Hub resume affordance was gone the moment it appeared. Return early on conflict, mirroring the started branch, so resolving it from the Hub still auto-loads on completion. - The background-download branch handled started and conflict but silently dropped a busy outcome, leaving the user with no feedback when a peer variant of the same repo was already downloading. Surface the same busy toast the autoload path uses. * Fix context length, GGUF template, fetch state and lease expiry bugs Keep explicit context length values instead of collapsing to null at native. The collapse made the slider jump back at the native maximum and made Reload load the previous context instead of the chosen one. Prefer the first split when resolving a GGUF without a variant. Later splits carry no chat template metadata, so picking the largest file could return no template for a sharded model. Clear stale fetch state when template and metadata lookups retry, so a previous terminal error is not shown while a new fetch is running. Record native path lease expiry together with the token when a load commits. The expiry was written by only one load path and even when the load did not start, so a reload could be blocked with an expired file message for a still valid token. * fix(model-picker): resolve review findings across config, inventory, and templates - Apply remembered per-model config in the training-compare chat handoff so a prior model's customContextLength no longer leaks into the next load - Match GGUF variant labels with the inventory extractor too, so cached no-quant-token files resolve their default chat template - Show "Auto" instead of a fabricated 32768 when native context is unknown - Reuse the identical staged auto-load object on same-pick so a re-pick during download pre-flight no longer disarms auto-load via "busy" - Union supports_vision when deduping cross-cache inventory rows - Serve hidden-model needles from a new GET /api/hub/hidden-models endpoint and merge them client-side, covering runtime-configured RAG embedders - Clamp GET chat templates to MAX_CHAT_TEMPLATE_BYTES (route + jinja sidecar), matching the validate endpoint's contract - Lower-clamp stored customContextLength to shared CONTEXT_LENGTH_MIN - Wipe unsloth_chat_load_on_selection in Settings "Reset all" - Drop stale pendingHasContext comment describing deleted staging machinery * Fix stale defaults cache, token in query string and rounded up context ceiling Refresh cached chat template and max position data when a model update completes. Send the HF token for model config requests in the dedicated header instead of the URL. Snap the native sequence length ceiling down to the nearest step so the slider cannot exceed the declared maximum. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix compare pane reverting active checkpoint on non-GGUF load Re-read runtime params after setCheckpoint so the fresh checkpoint is kept instead of being overwritten by the pre-setCheckpoint snapshot. * Send the HF token via header for the vision and embedding checks checkVisionModel and checkEmbeddingModel still passed the HuggingFace token as a ?hf_token= query parameter, so it landed in server access logs, proxy logs, and browser history. Move them to the X-Unsloth-HF-Token header like getModelConfig already does, and accept the header on the check-vision and check-embedding routes with the existing query parameter kept as a fallback for older clients. * Cap the chat template on the model load path The load endpoint accepted an unbounded chat_template_override, so a direct caller could hand llama.cpp an arbitrarily large Jinja template even though the frontend, the validate endpoint, and the read paths all enforce the 64 KiB limit. Reuse MAX_CHAT_TEMPLATE_BYTES in the LoadRequest validator, rejecting oversized templates with a fast character-count check before the exact UTF-8 byte check. * Protect existing per-model configs during legacy migration When the one-time legacy import pushes the store over budget, eviction now protects the entries the user already has and drops only the just-migrated legacy entries, so importing old load settings can never discard a newer per-model config. * Reset clears the context override instead of pinning the native value Reset wrote the discovered native context into customContextLength for GGUF models, but isDefaultConfig treats any non-null customContextLength as an explicit pin, so Reset with Remember enabled persisted a fixed context and future loads stopped using the native auto context. Reset now restores the full default (customContextLength null); the native value is still shown through the existing display fallback. * Bound chat-template sidecar reads to a size limit The chat_template.json, tokenizer_config.json, and Hub-downloaded sidecar readers decoded and json-parsed the whole file before the extracted template hit the 64 KiB response cap, so an oversized metadata file could exhaust memory. Read them through a bounded reader (4 MiB envelope) that returns None when the file is larger, matching the existing chat_template.jinja size guard. Adds tests for oversized tokenizer_config.json and chat_template.json. * Keep the native-path token and lease expiry in sync Rollback after a failed reload restored the previous token but left the failed load's expiry in the store, so a later reload could be falsely blocked as expired (token A paired with load B's lease). Restore the previous lease alongside the token, and clear the expiry wherever the token is cleared on a non-GGUF transition, so the two never diverge. * Clear the native file lease on compare-pane loads * Studio: add regression tests for the model-picker per-model-config Guard the specific regressions that reverted the predecessor change: - backend pytest (studio/backend/tests/test_model_picker_regression.py): infra-model hiding, HF token via header with query fallback, and the chat-template byte caps. - source contracts (tests/studio/test_model_picker_contracts.py): the token stays out of the URL, the context ceiling is floored, the native lease is cleared on compare-load and restored on rollback, the default caches key on the inventory version, and the hidden needles stay present. - Playwright E2E (tests/studio/playwright_model_config.py) wired into studio-ui-smoke.yml on port 18898: Context Length persists across a reload, Reset clears the stored override, and infra models are absent from the picker. - optional GPU-gated inference smoke (tests/studio/test_gpu_inference_smoke.py) that auto-skips on GPU-less CI and stays short on a GPU. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: model pinning, row menus, hub inference settings, and inventory filters Pinning - Add a pinned models store (localStorage) with repo and per-quant pins - Pinned section in the model selector's On Device list and the hub inventory, with newest pins first so Pin to top lands on top - Deleting a repo drops its pins Row menus - Replace loose row icons with a shared 3-dots menu (pin, reveal in file manager, copy identifier, copy path, delete) on picker rows, hub quant rows, the hub run bar, and on-device inventory rows - Menus only render for models actually on disk; platform-aware reveal labels - Backend: cached-model-path and reveal-cached-model endpoints resolving managed HF-cache repos only Hub inference settings - Gear in the GGUF run bar opens an Inference settings dialog reusing the chat page's controls: model config (context length, KV cache, speculative decoding, chat template), system prompt, reasoning, sampling, tools and retrieval Inventory - Model-type filter (text, vision, embedding, STT, TTS, diffusion) beside the sort pill, both with a sort icon, capped widths and truncation so the On device heading never wraps - Unsloth-owned repos without an upstream provider logo fall back to the Unsloth mascot avatar - Discover / On Device tabs widened; hub search bar narrowed to match * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: revert the Unsloth mascot avatar fallback Unsloth-owned repos without an upstream provider match go back to the colored-initial tile, and unslothai is no longer a relabeled owner. * Studio: run-bar options on single models, and aligned type/capability filters - Give single-model (non-GGUF) run bars the same 3-dots options menu and settings gear as GGUF, at repo level - Drop Pin to top from the run-bar menus; pinning stays in the On Device list - Add an Image to text (diffusion) capability with detection, and surface it in both the hub Discover capability filter and the On Device type filter - Align the On Device type filter with the Discover capability options and share the same detection so both dropdowns match * Studio: apply hub inference config on reload, eject action, and run-bar polish - Fix inference settings not applying: the hub dialog now writes the config to the runtime before reload, matching the chat page (selectModel reads runtime state, not the selection) - Order the settings gear before the 3-dots menu in the run bars - Replace the loaded-model run-bar action (New Chat) with Eject, wired through the inspector to the hub's ejectModel - Truncate the results heading so a long search query clips instead of overlapping the header pills in split view - Use a plain magnifying-glass icon for the no-results empty state * Studio: fix GPU settings loss, load guards, pins, filters, and cached paths Reloading a model from the chat sidebar or the hub gear dialog rebuilt the per-model config without the GPU memory fields, so manual GPU layers, MoE placement, and the GPU pick were reset on every reload and could be saved over a remembered config. The active config now comes from a shared useActiveModelConfig hook that carries the GPU fields for GGUF models, and the sidebar remount signature tracks them through a shared gpuFieldsSignature helper. The in-flight load guard lived in a ref inside each useChatModelRuntime instance, so the chat page, hub page, and gear dialog could not see each other's loads. A load started from the gear dialog left the hub page free to eject the model mid-reload or start a second concurrent load. The runtime store now records the loading pick, selectModel checks it across instances, and ejectModel refuses with a toast while any load is in flight. The cached-model-path endpoint matched GGUF files by basename and excluded only mmproj, so Copy path and Reveal could return an MTP drafter for a quant and returned 404 for directory layouts like BF16/model-00001.gguf. Variant files are now resolved from snapshot-relative paths with the same drafter, mmproj, and big-endian exclusions as the load path, shared through a new _main_variant_gguf_label helper. Hub and picker fixes: - rename the diffusion capability label from "Image to text" to "Image generation", since it detects image generators - validate pinned quants through the cached variant listing, keep the last verified set while revalidating, and drop deleted quants immediately - pass a measured scroll margin to the on-device virtual list so rows past the overscan stay visible below the pinned block - keep the delete menu for stopped partial safetensors downloads - give the inventory type filter a reset in Clear filters, a truthful empty state with a Show all types action, and hide it on the datasets view - order picker pinned rows by pin recency, include pinned matches in the empty-state check, and sync pins across browser tabs - count only the visible rows in the On device list header Tests: contract checks for each fix in test_model_picker_contracts.py and a backend test for the variant label selection. * Studio: reveal cached models in Windows Explorer under WSL The reveal endpoint only branched on macOS, Windows, and generic Linux. Under WSL the Linux branch spawned xdg-open, which is missing on a stock distro without a Linux desktop, so the request failed with a 500 and the UI showed a failed to open file manager error. WSL is now detected with the existing helper and the path is converted with wslpath before opening explorer.exe, selecting the file the same way native Windows does. Directories open directly. When interop is unavailable the old xdg-open fallback still runs. The macOS, native Windows, and native Linux branches are unchanged, and the Tauri app is covered since its hub reveal calls this same local endpoint. Tests: platform guards for the WSL reveal, the interop fallback, and the unchanged native Linux behavior in tests/studio/test_reveal_file_manager.py. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Adjust model picker row spacing and cogwheel hover consistency * Studio: exact hidden model ids and newest revision cached paths A custom RAG embedder repo was published to the frontend as a basename substring needle, so a generic name like org/model could hide unrelated models in the pickers. The hidden-models endpoint now sends full repo ids that are matched exactly. Copy path and Reveal picked a GGUF variant from an arbitrary cache revision when the same file existed in more than one. The newest revision now wins, matching the whole repo lookup. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix model picker GPU config, metadata, and cache selection Load each compare model with its saved GPU memory mode, GPU layers, CPU MoE layers, and selected GPU IDs. Reconcile saved GPU IDs with the current hardware. Include the active native GGUF path token in metadata checks. Search all Hugging Face cache roots when resolving cached models and select the largest visible cache entry. Remove obsolete barrel exports and the staging-only GPU memory helper. * Studio: hide hub inference settings gear for now The cogwheel in the hub download cards is out of scope for this PR. The dialog component stays in place and a TODO marks where the button returns in a future PR. * Refresh hidden model matchers * Fix GGUF detection, compare context pin, and picker delete staleness Treat any pick with a GGUF variant as GGUF in selectModel so the first load after downloading an uncached quant validates and sizes with the right GPU settings instead of unloading the current model on a wrong preflight. Variant picks now also set isGguf on their selection meta. Stop compare panes from inheriting the active model's context pin when their own saved config says Auto. Null context in a remembered config now means no pin, matching how the pane settings are shown. Route picker deletes through the hub inventory client, which invalidates the HF cache scan and the variants cache. The legacy delete route left the scan cache warm, so deleted models reappeared in the picker until the TTL expired. Removed the now unused legacy delete client and updated the contract test to match. * Studio: fix stale GGUF load-marker ordering test The load-in-flight marker still precedes the hub-download guard and the unload, but the llama_extra_args inheritance that used to sit between the marker and the guard now runs ahead of the GGUF branch, so it is no longer a landmark inside the sliced source. Drop it from the ordering assertion and keep the marker -> guard -> unload invariant. * Studio: fix per-model config edge cases in compare loads and saved defaults - chat-settings-sheet: gate the MTP fallback note and context/VRAM warning on the broader isGguf (variant, loaded gguf context, or .gguf suffix) instead of isLoadedGguf, so direct-file and custom-folder GGUF loads still surface those diagnostics. - shared-composer: a compare pane's context now comes from its own config only (a saved pin, else null for Auto/native). It no longer inherits the active model's shared snapshot, which resolveFitMaxSeqLength treated as an explicit pin and could load a pane at another model's context (VRAM/OOM), matching the single-model load path. - model-config-page: when an auto-fit GGUF is saved with fixed GPU layers (Manual) and Remember, pin the displayed fitted context so a later fresh load keeps the placement instead of sending native/0 and recreating the OOM. - per-model-config: treat Auto GPU memory mode and Auto/default speculative type as follow-global defaults; do not persist them as per-model overrides so later global preference changes keep applying. * Studio: gate vision capability on GGUF projectors and bound remote template downloads - cache_inventory: only mark a cached repo vision-capable when it holds an actual GGUF mmproj projector, not any file whose name merely contains "mmproj" (e.g. mmproj_config.json), matching the runtime's GGUF-only projector detection. - picker/service: pre-check the remote file size before downloading an uncached repo's chat template / tokenizer config, so a maliciously large sidecar is skipped instead of fetched and retained in full, mirroring the size gate the local-file path already applies. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: add source-contract guards for the per-model-config edge-case fixes Guard the four per-model-config fixes against silent regression in CI: - local GGUF diagnostics gate on the broad isGguf, not the variant-only isLoadedGguf - fixed-layer GGUF saves pin the displayed context - Auto GPU mode and Auto/default speculative are not persisted as per-model overrides - a compare pane's context comes from its own config, not the active model's snapshot * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: clear manual GPU knobs on Default and resolve local embedders before repo-id - model-config-page: switching GPU Memory back to Default now clears the Manual-only knobs (gpuLayers/nCpuMoe/selectedGpuIds); otherwise a remembered config kept stale pins that a later load re-applied when the global GPU preference was Manual, despite the page showing Default. - routes/models hidden_model_matchers: resolve an existing local path before the repo-id regex, mirroring is_hidden_model, so a local embedder shaped like "models/embedder" is hidden by exact path instead of leaking as a chat model. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: add _is_mtp_drafter to the model_config stub in the export-paths test routes/models.py imports _is_mtp_drafter from utils.models.model_config at module load, but the lightweight stub in test_export_absolute_paths.py did not provide it, so loading the module under the stub raised ImportError on Backend CI. Add the stub. * Studio: read a picked GGUF's chat template through the native path lease The picker chat-template GET has no native-path-lease plumbing, so a desktop-picked (drag-drop) GGUF could not show its default chat template in Run Settings until the model was loaded: the endpoint only receives the display label, not the leased file path. Read the embedded template through the existing lease-aware /api/inference/validate probe instead. A new include_chat_template flag resolves the granted canonical path and returns the GGUF's own embedded template, never a sibling sidecar (the grant authorizes just that one file); it skips the training guard like include_context_length and is bounded by MAX_CHAT_TEMPLATE_BYTES. The frontend fetch mints a one-shot validate-model lease when a native token is present and keeps the plain GET path for HF and allowlisted local models. Adds backend and source-contract regression tests. * Studio: call worker.direct_wheel_url in the ROCm wheel-url test The ROCm Mamba/SSM test referenced worker.py's private _direct_wheel_url, but the worker imports the wheel helper under its public name direct_wheel_url (utils.wheel_utils). When the worker module loads (its imports resolve in CI), worker_mod._direct_wheel_url raised AttributeError; the test only masked it by skipping when the worker could not be imported. Call the name that actually exists so the assertion runs; it still returns None for an empty cuda_major (ROCm). * Studio: reset max sequence length to the app default, not the loaded value For a non-GGUF active model, the per-model config seeds maxSeqLength from the loaded runtime value so the panel opens showing the running context. Reset set config.maxSeqLength to null, but the null fallback resolved back to that captured runtime value, so the field kept showing the old custom length and the config saved/reloaded it again. A remembered or active max-length override therefore could not be cleared from Run settings. Fall the null/default case back to the app default (clamped to the model's native ceiling) instead of the active runtime snapshot, so Reset actually clears the override. The initial view is unaffected: an active model's config.maxSeqLength is already non-null, so it still shows the loaded value. Adds a source-contract regression guard. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: persist default max length, refresh deleted quants, hide non-chat locals Three follow-up fixes from review of the per-model-config picker: - Max sequence length: the persisted per-model record now keeps config's maxSeqLength (null after Reset) so isDefaultConfig can clear a remembered override; the resolved app-default is substituted only into the load request, never the saved record. Previously Reset saved the concrete default and left the model pinned/remembered. - GGUF variant expander: deleting a downloaded quant from a repo that still has other cached quants now bumps the expander refresh key, so the removed quant stops showing as downloaded and clickable (which would try to reload the deleted file) until the repo is collapsed and reopened. - Local picker rows: require capabilities.canChat before listing a local models-folder / LM Studio row. A weightless folder (only config.json) is classified non-chat, and toLocalModelInfo drops capabilities, so selecting such a row would try to load a path the inventory already marked non-chat. Adds source-contract regression guards for all three. * Fix compare-pane and Reset context defaults in model picker Two related per-model-config default regressions: - A non-GGUF compare pane with no saved maxSeqLength fell back to the active model's shared runtime snapshot, so comparing a saved 128K model against an unconfigured pane loaded the latter at 128K and could OOM. It now falls back to the shared app default (DEFAULT_MAX_SEQ_LENGTH), the same fallback the single-model config path uses. - contextAtDefault treated an explicit customContextLength equal to the native ceiling as a default, which wedged the Reset button disabled for a deliberate pin-to-native. It now counts as default only when there is no override at all. DEFAULT_MAX_SEQ_LENGTH becomes a single exported constant in per-model-config.ts so the single-model config and the compare path share one source of truth. Adds source-contract guards for both fixes. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Skip over-cap remote Jinja templates so the tokenizer template wins The remote chat-template resolver bounded raw chat_template.jinja downloads only by MAX_TEMPLATE_METADATA_BYTES (4 MiB), then returned the first non-empty Jinja unconditionally. The picker route drops any template larger than MAX_CHAT_TEMPLATE_BYTES (64 KiB), so an uncached repo whose chat_template.jinja sits between 64 KiB and 4 MiB returned no template at all, even when a valid smaller tokenizer_config.json template existed. The local path already skips oversized .jinja files and falls through. Gate the extracted Jinja on MAX_CHAT_TEMPLATE_BYTES and continue searching when it exceeds the cap, matching _chat_template_from_jinja_file. The 4 MiB download bound stays for JSON files that merely embed a small template. Adds a regression test that a big Jinja plus a valid tokenizer config resolves to the tokenizer template. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Guard legacy per-model-config migration idempotency The v1->v2 localStorage migration (unsloth_load_settings -> unsloth_model_configs) runs on every store read, so it must migrate exactly once and never re-run, duplicate, or clobber a newer per-model config on a reload or restart. That was covered only by a manual proof, so add durable guards: - Source-contract test pinning the three idempotency layers (the in-memory legacyMigrationChecked guard, the persistent unsloth_model_configs_migrated flag set in every terminal branch, and the non-overwriting Object.hasOwn merge-skip) plus the readMap invocation. Reddens if any layer is dropped. - Playwright model-config E2E: promote the legacy-migration step to a gating check (soft_fail, which gates under the CI STUDIO_UI_STRICT=1) that the migrated value is preserved and the flag is set, then reload again with a fresh legacy seed present and assert the stored key set is unchanged, so a second reload cannot re-migrate, duplicate, or clobber. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Note the migration E2E now gates idempotency under STUDIO_UI_STRICT * Tighten model-picker per-model-config code comments --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Daniel Han <danielhanchen@gmail.com> Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com> Co-authored-by: danielhanchen <michaelhan2050@gmail.com> Co-authored-by: shimmyshimmer <shimmyshimmer@users.noreply.github.com> Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local> Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>		2026-07-20 22:53:22 -07:00
.github	Feat/model picker per model config v2 (#7207 )	2026-07-20 22:53:22 -07:00
images	images: use narrower Discord button and drop duplicate (#5552 )	2026-05-18 05:00:59 -07:00
scripts	Replace standalone Studio wording with Unsloth (#7221 )	2026-07-19 00:47:04 -07:00
studio	Feat/model picker per model config v2 (#7207 )	2026-07-20 22:53:22 -07:00
tests	Feat/model picker per model config v2 (#7207 )	2026-07-20 22:53:22 -07:00
unsloth	AMD	2026-07-20 07:07:30 -07:00
unsloth_cli	Fix local CLI streamed generation error handling (#7135 )	2026-07-20 23:14:58 -03:00
.gitattributes	Replace standalone Studio wording with Unsloth (#7221 )	2026-07-19 00:47:04 -07:00
.gitignore	studio: tool calling for DeepSeek (R1/V3/V3.1), GLM 4.x, Kimi K2 on safetensors + MLX (#5624 )	2026-07-06 15:40:46 -07:00
.pre-commit-ci.yaml	pre-commit CI config (#3565 )	2025-11-07 14:44:18 -08:00
.pre-commit-config.yaml	[pre-commit.ci] pre-commit autoupdate (#6587 )	2026-06-23 03:01:11 -07:00
build.sh	Replace standalone Studio wording with Unsloth (#7221 )	2026-07-19 00:47:04 -07:00
cli.py	Rename cli/ to unsloth_cli/ to fix namespace collision with stringzilla (#4393 )	2026-03-17 20:40:21 -07:00
CODE_OF_CONDUCT.md	Update CODE_OF_CONDUCT.md	2025-10-25 19:31:05 -07:00
CONTRIBUTING.md	docs: repository cleanup (#5617 )	2026-06-12 11:07:04 +01:00
COPYING	Rename cli/ to unsloth_cli/ to fix namespace collision with stringzilla (#4393 )	2026-03-17 20:40:21 -07:00
install.ps1	Bump install.sh / install.ps1 pin to unsloth>=2026.7.4 (#7263 )	2026-07-20 07:17:31 -07:00
install.sh	Bump install.sh / install.ps1 pin to unsloth>=2026.7.4 (#7263 )	2026-07-20 07:17:31 -07:00
LICENSE	Rename cli/ to unsloth_cli/ to fix namespace collision with stringzilla (#4393 )	2026-03-17 20:40:21 -07:00
pyproject.toml	AMD	2026-07-20 07:07:30 -07:00
README.md	Update README with latest features and Unsloth Start (#7258 )	2026-07-20 05:57:14 -07:00
unsloth-cli.py	Add MLX backend support for CLI unsloth train (#6709 )	2026-07-08 03:25:26 -07:00

README.md

Unsloth Studio lets you run and train models locally.

Features • News • Quickstart • Notebooks • Documentation

⚡ Get started

macOS, Linux, WSL:

curl -fsSL https://unsloth.ai/install.sh | sh

Windows:

irm https://unsloth.ai/install.ps1 | iex

Community:

⭐ Features

Unsloth Studio (Beta) lets you run and train text, audio, embedding, vision models on Windows, Linux and macOS.

Inference

Search + download + run models including GGUF, LoRA adapters, safetensors
Export models: Save or export models to GGUF, 16-bit safetensors and other formats.
Tool calling: Support for self-healing tool calling and web search
Code execution: lets LLMs test code in Claude artifacts and sandbox environments
API inference endpoint: Deploy and run local LLMs in Claude Code, Codex tools with Unsloth
Auto set inference settings and customize chat templates.
We work directly with teams behind gpt-oss, Qwen3, Llama 4, Mistral, Gemma 1-3, and Phi-4, where we’ve fixed bugs that improve model accuracy.
Chat with images, audio, PDFs, code, DOCX and more. Connect API providers (OpenAI, Anthropic) or servers (vLLM, Ollama).
Compare any two models side by side with the same prompt.
OpenAI/Anthropic-compatible APIs: Serve local models through /v1/chat/completions, /v1/responses and /v1/messages.
Connect local models to agents: Use unsloth start with Claude Code, Codex, Hermes and more.
Web/PDF search can read PDF papers, manuals and other PDF results.
GGUF hardware controls: Choose GPUs/layers, offload MoE experts, use multi-GPU or Tensor Parallelism.
The opt-in MCP control endpoint lets AI clients manage models, training, recipes and exports.

Training

Train and RL 500+ models up to 2x faster with 70% less VRAM; MoE up to 12x faster.
Train and run RL on AMD GPUs across Windows, WSL and Linux.
Data Recipes: Auto-create datasets from PDF, CSV, DOCX etc. Edit data in a visual-node workflow.
Reinforcement Learning uses 80% less VRAM for GRPO, FP8 and vision RL, with 7x longer contexts.
Long-context training: 3x faster, 30% less VRAM and 500K+ context.
Supports LoRA/QLoRA, full fine-tuning, RL, pretraining, 4-bit, 16-bit and FP8.
Custom Triton and mathematical kernels built with PyTorch and Hugging Face.
Observability: Monitor training live, track loss and GPU usage and customize graphs.
Multi-GPU training is supported, with major improvements coming soon.

🚀 Unsloth Start

Unsloth Start connects Claude Code, Codex and other agents to local models with one command.

Start Unsloth, load a model, open your project folder, then run:

unsloth start claude

Replace claude with any supported agent:

Agent	Command
Claude Code	`unsloth start claude`
OpenAI Codex	`unsloth start codex`
Hermes Agent	`unsloth start hermes`
OpenClaw	`unsloth start openclaw`
OpenCode	`unsloth start opencode`
Pi Coding Agent	`unsloth start pi`

📥 Install

Unsloth can be used in two ways: through Unsloth Studio, the web UI, or through Unsloth Core, the code-based version. Each has different requirements.

Unsloth Studio (web UI)

Unsloth Studio (Beta) works on Windows, Linux, WSL and macOS.

CPU: Supported for Chat and Data Recipes currently
NVIDIA: Training works on RTX 30/40/50, Blackwell, DGX Spark, Station and more
macOS: Training, MLX and GGUF inference are ALL supported.
AMD: Training, RL, chat and deployment work on Windows, WSL and Linux. Read the AMD guide.
Vulkan: GGUF inference is supported on compatible GPUs, including Intel GPUs.
Multi-GPU: Available now, with a major upgrade on the way

macOS, Linux, WSL:

curl -fsSL https://unsloth.ai/install.sh | sh

Use the same command to update.

Windows:

irm https://unsloth.ai/install.ps1 | iex

Use the same command to update.

Launch

unsloth studio -p 8888

For LAN or cloud access, add -H 0.0.0.0 (raw port only; add --cloudflare for a public URL). By default, Unsloth is accessible only locally.

To reach Unsloth over HTTPS, use unsloth studio --secure. Unsloth stays bound to localhost and is reached only through a free Cloudflare tunnel, which publishes it at a public https://*.trycloudflare.com URL (it fails closed if the tunnel can't start, so the raw port is never exposed). This makes Unsloth reachable from the internet, so anyone with the link and API key can use it and run code: keep your API key private (see Remote access below).

Docker

Use our Docker image unsloth/unsloth container. Run:

docker run -d -e JUPYTER_PASSWORD="mypassword" \
  -p 8888:8888 -p 8000:8000 -p 2222:22 \
  -v $(pwd)/work:/workspace/work \
  --gpus all \
  unsloth/unsloth

Developer, Nightly, Uninstall

To see developer, nightly and uninstallation etc. instructions, see advanced installation.

Unsloth Core (code-based)

Linux, WSL:

curl -LsSf https://astral.sh/uv/install.sh | sh
uv venv unsloth_env --python 3.13
source unsloth_env/bin/activate
uv pip install unsloth --torch-backend=auto

Windows:

winget install -e --id Python.Python.3.13
winget install --id=astral-sh.uv  -e
uv venv unsloth_env --python 3.13
.\unsloth_env\Scripts\activate
uv pip install unsloth --torch-backend=auto

For Windows, pip install unsloth works only if you have PyTorch installed. Read our Windows Guide. You can use the same Docker image as Unsloth Studio.

AMD, Intel:

For RTX 50x, B200, 6000 GPUs: uv pip install unsloth --torch-backend=auto. Read our guides for: Blackwell and DGX Spark.
To install Unsloth on AMD and Intel GPUs, follow our AMD Guide and Intel Guide.

📒 Free Notebooks

Train for free with our notebooks. You can use our new free Unsloth Studio notebook to run and train models for free in a web UI. Read our guide. Add dataset, run, then deploy your trained model.

Model	Free Notebooks	Performance	Memory use
Gemma 4 (E2B)	▶️ Start for free	1.5x faster	50% less
Qwen3.5 (4B)	▶️ Start for free	1.5x faster	60% less
gpt-oss (20B)	▶️ Start for free	2x faster	70% less
Qwen3.5 GSPO	▶️ Start for free	2x faster	70% less
gpt-oss (20B): GRPO	▶️ Start for free	2x faster	80% less
Qwen3: Advanced GRPO	▶️ Start for free	2x faster	70% less
embeddinggemma (300M)	▶️ Start for free	2x faster	20% less
Mistral Ministral 3 (3B)	▶️ Start for free	1.5x faster	60% less
Llama 3.1 (8B) Alpaca	▶️ Start for free	2x faster	70% less
Llama 3.2 Conversational	▶️ Start for free	2x faster	70% less
Orpheus-TTS (3B)	▶️ Start for free	1.5x faster	50% less

See all our notebooks for: Kaggle, GRPO, TTS, embedding & Vision
See all our models and all our notebooks
See detailed documentation for Unsloth here

🦥 Unsloth News

AMD training: Train, run RL, chat and deploy on AMD GPUs across Windows, WSL and Linux. Guide
GGUF hardware controls: Choose GPU/layer placement, offload MoE experts and use multi-GPU or Tensor Parallelism. #6414
Local models for any agent: Use unsloth start with Claude Code, Codex, Hermes, OpenCode, OpenClaw, Pi and more through Unsloth's OpenAI- and Anthropic-compatible APIs. Guide
MCP control endpoint: Let compatible clients manage models, training, recipes, checkpoints and exports. #7191
Local inference reliability: Resume long chats faster, recover stalled downloads and reuse existing GGUF files. #7204 • #6858 • #7209
New models: Qwen-AgentWorld, Ornith, Kimi K2.7 Code and MiniMax M3
GLM-5.2: Run Z.ai's 744B-parameter, 1M-context open model locally with Unsloth Dynamic GGUFs. Guide
DeepSeek-V4: Run DeepSeek-V4-Flash locally with corrected multi-turn and tool-calling behavior. Guide
DiffusionGemma: Run and fine-tune Google's diffusion language model with 1.8x faster inference in Unsloth Studio. Guide
Qwen3.6: Run and train Qwen3.6 with MTP for 1.4-2.2x faster inference and NVFP4 quants for supported GPUs. Guide
Gemma 4: Run and train Gemma 4 text, image and audio models with QAT, MTP, GGUF and MLX support. Guide
MCP servers: Connect local models to files, apps, databases and external tools through Model Context Protocol. Guide
Connections: Mix local models with API providers (OpenAI, Anthropic) or servers (vLLM, Ollama) in the same interface. Guide
Introducing Unsloth Studio: our new web UI for running and training LLMs. Blog
Train MoE LLMs 12x faster with 35% less VRAM - DeepSeek, GLM, Qwen and gpt-oss. Blog
Embedding models: Unsloth now supports ~1.8-3.3x faster embedding fine-tuning. Blog • Notebooks
New 7x longer context RL vs. all other setups, via our new batching algorithms. Blog
New RoPE & MLP Triton Kernels & Padding Free + Packing: 3x faster training & 30% less VRAM. Blog
500K Context: Training a 20B model with >500K context is now possible on an 80GB GPU. Blog
FP8 & Vision RL: You can now do FP8 & VLM GRPO on consumer GPUs. FP8 Blog • Vision RL

📥 Advanced Installation

The below advanced instructions are for Unsloth Studio. For Unsloth Core advanced installation, view our docs.

Developer / Nightly / Experimental installs: macOS, Linux, WSL:

The developer install builds from the main branch, which is the latest (nightly) source.

git clone https://github.com/unslothai/unsloth
cd unsloth
./install.sh --local
unsloth studio -p 8888

To install into an isolated location (its own virtual env, auth/, studio.db, cache and llama.cpp build), set UNSLOTH_STUDIO_HOME and pass it again at launch:

UNSLOTH_STUDIO_HOME="$PWD/.studio" ./install.sh --local
UNSLOTH_STUDIO_HOME="$PWD/.studio" unsloth studio -p 8888

Then to update :

cd unsloth && git pull
./install.sh --local
unsloth studio -p 8888

Developer / Nightly / Experimental installs: Windows PowerShell:

The developer install builds from the main branch, which is the latest (nightly) source.

git clone https://github.com/unslothai/unsloth.git
cd unsloth
Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass
.\install.ps1 --local
unsloth studio -p 8888

To install into an isolated location (its own virtual env, auth/, studio.db, cache and llama.cpp build), set UNSLOTH_STUDIO_HOME and pass it again at launch:

$env:UNSLOTH_STUDIO_HOME="$PWD\.studio"; .\install.ps1 --local
$env:UNSLOTH_STUDIO_HOME="$PWD\.studio"; unsloth studio -p 8888

Then to update :

cd unsloth; git pull
.\install.ps1 --local
unsloth studio -p 8888

Remote access: `--secure` (HTTPS tunnel) vs raw port

By default unsloth studio binds to 127.0.0.1 (this machine only). To reach it from another device, pick one of:

--secure (recommended): serve only through a free Cloudflare HTTPS link. Unsloth stays bound to localhost and the tunnel provides the public URL; it fails closed (does not start) if the tunnel can't come up, so the raw port is never exposed.

unsloth studio --secure -p 8888

-H 0.0.0.0: bind the raw port on all network interfaces, reachable from anywhere on the network (subject to your firewall). It does not create a public internet URL; add --cloudflare to also publish an internet-reachable https://*.trycloudflare.com link even behind a firewall. Only use this on a network you trust.

unsloth studio -H 0.0.0.0 -p 8888

The Cloudflare tunnel is off by default: -H 0.0.0.0 exposes the raw port only, not a public internet URL. Pair the wildcard bind with --cloudflare (unsloth studio -H 0.0.0.0 --cloudflare) to also publish a public https://*.trycloudflare.com link, or prefer --secure (above), which keeps the raw port private. --cloudflare has no effect on a loopback bind.

The first time Unsloth is published on a public URL (--secure or --cloudflare) with the auto-generated admin password still in place, it asks for a new admin password in the terminal (masked input with confirmation) before the public link goes up. Without an attached terminal it warns instead and keeps the bootstrap deadline: Unsloth shuts down after UNSLOTH_STUDIO_BOOTSTRAP_TIMEOUT (default 1 hour) unless the password is changed in the web UI.

For headless setups that cannot answer that prompt, set the initial admin password non-interactively with --password (only takes effect when no password is set yet; if one already exists it is a hard error, so rotate later with unsloth studio reset-password):

unsloth studio --secure --password 'your-strong-password'        # visible in `ps`/history
UNSLOTH_STUDIO_PASSWORD='your-strong-password' unsloth studio --secure   # via env var
printf '%s\n' 'your-strong-password' | unsloth studio --secure --password -   # via stdin

A literal --password VALUE is visible in the process list and shell history, so prefer the UNSLOTH_STUDIO_PASSWORD env var or --password - (stdin) for automation. This applies to any launch (public or a headless -H 0.0.0.0 bind), and the password is set in the parent before the server binds, so it never reaches a re-executed child process.

Server-side tools (web search, Python and terminal code execution) run as your user and are on by default. Anyone who can reach the server with the API key can run code on this machine, so keep your API key private and pass --disable-tools when exposing Unsloth.

Advanced launch options

Installer options can be passed as environment variables. On macOS, Linux and WSL place the variable after the pipe so the shell passes it to sh; on Windows set it with $env: before piping to iex.

Skip PyTorch (GGUF-only mode):

curl -fsSL https://unsloth.ai/install.sh | UNSLOTH_NO_TORCH=1 sh

$env:UNSLOTH_NO_TORCH=1; irm https://unsloth.ai/install.ps1 | iex

Skip the post-install prompt that starts Unsloth (useful for automated installs):

curl -fsSL https://unsloth.ai/install.sh | UNSLOTH_SKIP_AUTOSTART=1 sh

$env:UNSLOTH_SKIP_AUTOSTART=1; irm https://unsloth.ai/install.ps1 | iex

Pin the Python version:

curl -fsSL https://unsloth.ai/install.sh | UNSLOTH_PYTHON=3.12 sh

$env:UNSLOTH_PYTHON='3.12'; irm https://unsloth.ai/install.ps1 | iex

Install to a custom location with UNSLOTH_STUDIO_HOME:

curl -fsSL https://unsloth.ai/install.sh | UNSLOTH_STUDIO_HOME=/abs/path sh

$env:UNSLOTH_STUDIO_HOME='C:\path'; irm https://unsloth.ai/install.ps1 | iex

On macOS, the installer defaults to the system certificate store (UV_SYSTEM_CERTS=1) so uv trusts the CAs in your Keychain, needed behind TLS-inspecting proxies (Cisco Umbrella, Zscaler, etc.). Opt out with:

curl -fsSL https://unsloth.ai/install.sh | UV_SYSTEM_CERTS=0 sh

Point the frontend build at a corporate npm mirror/proxy with UNSLOTH_NPM_REGISTRY (for the developer install behind a firewall that blocks registry.npmjs.org):

UNSLOTH_NPM_REGISTRY=https://artifactory.example.com/api/npm/npm/ ./install.sh --local

$env:UNSLOTH_NPM_REGISTRY='https://artifactory.example.com/api/npm/npm/'; .\install.ps1 --local

It is threaded as --registry into the Unsloth frontend npm/bun installs; the supply-chain locks (7-day min-release-age, exact version pins) stay in force.

Cap Unsloth's native CPU thread pools on high-core hosts: UNSLOTH_CPU_THREADS=8 unsloth studio -p 8888.

Uninstall

The recommended way to fully remove Unsloth Studio is the matching uninstall script for your OS. It stops any running servers, removes the install dir, the launcher data dir, the desktop shortcut, and any platform-specific entries (macOS .app bundle + Launch Services on Mac; Start Menu, HKCU\Software\Unsloth registry key and user PATH entries on Windows):

MacOS, WSL, Linux: curl -fsSL https://raw.githubusercontent.com/unslothai/unsloth/main/scripts/uninstall.sh | sh
Windows (PowerShell): irm https://raw.githubusercontent.com/unslothai/unsloth/main/scripts/uninstall.ps1 | iex

If you only want to drop the install dir and keep the launcher/shortcut for a later reinstall, you can instead run rm -rf ~/.unsloth/studio (Mac/Linux/WSL) or Remove-Item -Recurse -Force "$HOME\.unsloth\studio" (Windows). The model cache at ~/.cache/huggingface is not touched by any of these.

For more info, see our docs.

Deleting model files

You can delete old model files either from the bin icon in model search or by removing the relevant cached model folder from the default Hugging Face cache directory. By default, HF uses:

MacOS, Linux, WSL: ~/.cache/huggingface/hub/
Windows: %USERPROFILE%\.cache\huggingface\hub\

💚 Community and Links

Type	Links
Discord	Join Discord server
r/unsloth Reddit	Join Reddit community
📚 Documentation & Wiki	Read Our Docs
Twitter (aka X)	Follow us on X
🔮 Our Models	Unsloth Catalog
✍️ Blog	Read our Blogs

Citation

You can cite the Unsloth repo as follows:

@software{unsloth,
  author = {Daniel Han, Michael Han and Unsloth team},
  title = {Unsloth},
  url = {https://github.com/unslothai/unsloth},
  year = {2023}
}

If you trained a model with 🦥Unsloth, you can use this cool sticker!

License

Unsloth uses a dual-licensing model of Apache 2.0 and AGPL-3.0. The core Unsloth package remains licensed under Apache 2.0, while certain optional components, such as the Unsloth Studio UI are licensed under the open-source license AGPL-3.0.

This structure helps support ongoing Unsloth development while keeping the project open source and enabling the broader ecosystem to continue growing.

Thank You to

The llama.cpp library that lets users run and save models with Unsloth
The Hugging Face team and their libraries: transformers and TRL
The Pytorch and Torch AO team for their contributions
NVIDIA for their NeMo DataDesigner library and their contributions
And of course for every single person who has contributed or has used Unsloth!

README.md Unescape Escape

Unsloth Studio lets you run and train models locally.

⚡ Get started

macOS, Linux, WSL:

Windows:

Community:

⭐ Features

Inference

Training

🚀 Unsloth Start

📥 Install

Unsloth Studio (web UI)

macOS, Linux, WSL:

Windows:

Launch

Docker

Developer, Nightly, Uninstall

Unsloth Core (code-based)

Linux, WSL:

Windows:

AMD, Intel:

📒 Free Notebooks

🦥 Unsloth News

📥 Advanced Installation

Developer / Nightly / Experimental installs: macOS, Linux, WSL:

Developer / Nightly / Experimental installs: Windows PowerShell:

Remote access: --secure (HTTPS tunnel) vs raw port

Advanced launch options

Uninstall

Deleting model files

💚 Community and Links

Citation

License

Thank You to

README.md

Remote access: `--secure` (HTTPS tunnel) vs raw port