* refactor(studio): move chat model picker into features/model-picker
Relocate model-selector + its support files from components/assistant-ui
into a self-contained features/model-picker feature (own barrel), mirroring
the modular Hub layout. Pure move + import repoint; no behaviour change.
* feat(model-picker): add per-model config persistence layer
Superset PerModelConfig (customContextLength, kvCacheDtype, speculativeType,
specDraftNMax, tensorParallel, chatTemplateOverride, trustRemoteCode) persisted
to localStorage (unsloth_model_configs) with schema versioning + LRU budget.
KV-dtype and speculative value sets match main's sidebar (no q4_0/ngram-simple).
Reuses features/hub/lib/model-identity for normalization; adds storage-key layer
and applyPerModelConfigToRuntime (sets tensorParallel, which the old PR omitted).
* feat(picker): modular backend for chat-template validate + default fetch
New studio/backend/picker package (schemas/service/routes) mounted at /api/picker:
- POST /api/picker/validate-chat-template (Jinja syntax validation, no false positives)
- GET /api/picker/chat-template/{model_name} (default template from tokenizer_config.json,
reusing get_cache_path/resolve_cached_repo_id_case; graceful null, no model-code exec)
Frontend api/templates.ts client + hooks/use-model-defaults lazy cache. No backend
changes to the existing inference load route (per-model load fields already supported).
* feat(model-picker): bind picker on-device list to shared hub inventory
Picker now sources cached + local models from useHubInventory (the Hub's shared
store) via a thin adapter, replacing its own /api/models/* fetchers + module
caches. Hub, download manager, and picker now share one source of truth, so
completed downloads reflect in the picker automatically. Partial/live-download
rows are filtered from the cached lists (unchanged rendering). Local naming/search
preserved via additive LocalInventoryRow modelId/displayName. Variant expander,
scan-folder management, recommended-fit, search, external providers untouched.
Known minor: cached 'Downloaded date' sort tiebreak degrades to alphabetical
(hub cached rows carry no mtime); default 'recent' (load-time) sort preserved.
* feat(model-picker): per-model config step inside the picker
Picking a (non-external) model now opens an in-picker config view built from
main's current load controls (context length, KV cache dtype, speculative
decoding, draft tokens, tensor parallel) plus a chat-template editor backed by
the picker validate/default endpoints. 'Remember for this model' persists the
config per model+variant; Run forwards the config to the existing load flow via
meta.config. External models bypass the step. Two-view orchestration lives in
model-selector (single interception point); pickers.tsx call sites untouched.
trustRemoteCode dropped from PerModelConfig to preserve main's per-load consent.
* feat(chat): apply/persist per-model config through the load flow
handleCheckpointChange threads meta.config into the selection; stageOrLoad and
the autoload/Hub-run paths now apply the picker config (explicit pick or saved
remembered config) via applyPerModelConfigToRuntime before staging/loading, with
keepSpeculative set so a remembered speculative mode survives the model switch.
Replaces the old remembered-load-settings seeding (resolveInitialConfig now the
single source). SelectedModelInput carries config.
* refactor(chat): remove per-model load config from the right sidebar
The load knobs (context, KV cache, speculative, draft tokens, tensor parallel)
and the chat-template editor now live only in the picker config step. The sheet's
Model section keeps the staged Load/Cancel flow (config is applied at pick time);
sampling params, system prompt, and RAG are unchanged. Deletes the superseded
remembered-load-settings module + the store's applyRememberedLoadSettings action,
removes the now-dead sheet state/imports, and points the settings reset at
unsloth_model_configs. Delete-cleanup deferred (stale config is LRU-capped).
* fix(model-picker): remove leftover sidebar-staging cogwheel + empty Model section
The downloaded-variant gear (ModelLoadSettingsAction) staged a model straight
into the right-sidebar Run-settings flow -- the old 'configure before load' path
now fully replaced by the in-picker config step. Removed the gear + its component.
Also gate the sheet's 'Model' section to staged picks only (pendingSelection):
after the load-knob strip its content is staged-only, so it was rendering an
empty section header whenever a model was merely loaded.
* chore(chat): remove dead per-model-config setters + modelControlsDisabled
After the load-config UI moved into the picker, the store's per-model setters
(setKvCacheDtype/setSpeculativeType/setSpecDraftNMax/setTensorParallel/
setCustomContextLength/setChatTemplateOverride) had zero callers
(applyPerModelConfigToRuntime writes via setState), and the sheet's
modelControlsDisabled was unreferenced. Verified dead across the whole tree.
* fix(chat): config-step Load actually loads (ignore Load-on-selection)
Root cause: with Settings > Chat > 'Load on selection' turned OFF, the config
step's load went down the deferred-staging path -- opening the right sidebar with
'<model> is staged, not loaded yet / Choose Load model'. The in-picker config step
IS the deliberate load action, so its Load now loads immediately (or downloads +
auto-loads when not cached) regardless of the toggle. Renamed the button
'Run model' -> 'Load model' to match. Native/dropped picks still honor the toggle.
* refactor(chat,hub): retire 'Load on selection' — config step is the only load flow
The in-picker config step (and the Hub Run button) now fully supersede the old
stage-to-sidebar flow, so the Load-on-selection toggle is removed everywhere:
- chat stageOrLoad: every pick loads immediately, or downloads + auto-loads when
not cached (the previous default behaviour, now universal).
- hub Run: drops the stage branch; downloaded GGUFs load directly with their saved
per-model config (no collision with the chat config step — both end at selectModel).
- store: removed loadOnSelection field/setter/key/default; Settings>Chat toggle and
its settings-reset entry removed.
- staged sidebar section is now a download-progress view (auto-loads on completion).
No manual staging remains; stageModel is used only for background auto-load downloads.
* feat(model-picker): default chat template from GGUF + thread variant through config flow
Read the embedded tokenizer.chat_template from GGUF files (read_gguf_chat_template
in gguf_metadata) and use it as the per-model default. Plumb gguf_variant through
the picker service, /api/picker/chat-template route, frontend templates API, and
use-model-defaults so the right variant's template is fetched.
Also refine the picker config-page/model-selector wiring, drop the dead
ggufNativeContextLength runtime path, and add the per-model-config storage keys to
the settings prefs export.
* feat(model-picker): read safetensors chat template + hide editor where it has no effect
Resolve the default chat template for safetensors models: prefer the modern
chat_template.jinja, fall back to the tokenizer_config.json chat_template field,
then chat_template.json (multimodal processor), then the GGUF embedded template.
Applied to local dirs, the HF cache snapshot scan, and the HF remote fetch.
Hide the chat-template editor in the picker for safetensors models — the override
is only applied at load by the GGUF/llama.cpp backend, so editing it on safetensors
currently has no effect. GGUF keeps the editor. Nothing removed; the dialog stays
for when the safetensors apply path is wired up in a later branch.
* fix(model-picker): set legacy-migration flag only after the write succeeds
Set unsloth_model_configs_migrated only once writeMap confirms the migrated
map persisted, so a quota/storage failure no longer marks migration done and
silently drops the user's pre-existing remembered settings — the next load retries.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* MVP model picker fixes
* MVP picker config fix
* MVP safetensors config
* MVP max seq config
* MVP max seq fix
* Fix static max tokens cap ignoring model context
* Fix picker GGUF scan parity
* fix(studio): harden model picker config loading
Apply remembered per-model configs consistently from picker and Hub loads, keep default configs from overriding standing speculative settings, add config access for direct local GGUF files, and support saving or forgetting active model settings without a reload.
* Fix model picker config flow
* Fix model picker config loads
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Avoid recursive per-model config migration reads
* Apply the displayed context length when loading a GGUF
* Fix template validation, cached template lookup, and failed load rollback
- Validate chat templates with the loopcontrols extension so templates
that use break or continue tags pass the picker validator, matching the
inference renderer that already accepts them.
- Read the default chat template from the newest cache snapshot rather than
an arbitrary iterdir order, so an older cached revision no longer prefills
a stale template.
- Capture the runtime per-model config before a load and reapply it when the
load fails, so a failed switch leaves the active model context, KV cache,
template, and speculative settings as they were.
* Make chat template view only for safetensors models
Custom chat template overrides are applied at inference only for GGUF
models, which pass the template to llama-server. The safetensors backend
renders with the model built-in template and ignores the override, so
editing it would save a value that never loads. For safetensors the
config page now opens the template as a read-only preview with a note
that editing is not available yet. This can become editable once
inference support for custom safetensors templates lands in main.
* Fix model picker config edge cases
- Restore prior runtime config when a load no-ops for the active model
- Cap the picker validator request body via the protected prefixes
- Keep the GGUF context slider max above the loaded context
- Fetch subfolder chat templates for uncached Hub repos
- Show the compare side config when reopening the picker
* Keep saved GGUF context above the fallback ceiling
* Show the model config in the run settings sidebar
* Fix model config sidebar reset and context slider
- Stack the remember toggle and action buttons in the sidebar
- Reset the config to defaults instead of the loaded values
- Fetch the native context so the slider max is not the loaded value
* Fix model picker config and download regressions
- Run picker chat template routes off the event loop
- Depth and root guard local template directory scans
- Restore download manager flow for uncached hub picks
- Apply per model context length on reload
- Import model picker symbols from the feature barrel
* Fix model picker config and cached download sorting
- Restore load settings when a Hub run is rejected mid load
- Reuse one NumericValueInput instead of a duplicate copy
- Fix double decode of the model name in the template route
- Remove the unused reset-to-loaded settings action
- Fix cached model download sorting
* Fix model picker per-model config edge cases
Honor a saved or typed max seq length above the model's native context so
RoPE extended values are no longer clamped and silently overwritten. Allow
typing past native while the slider keeps native as a soft ceiling.
Guard the fetch success paths in use-model-defaults against an aborted
signal, and refetch when the HF token changes.
Hash the chat template content in the sidebar remount key instead of its
length. Enable reset for a GGUF whose native context is unknown, and floor
the context slider max so it can never fall below the min.
* Fix GGUF context auto-fit and gated model config token
Stop forcing a 32768 context when a GGUF native context is unknown so the backend auto-fits to VRAM again, while still honoring an explicit context edit.
Send the HF token as a query param so gated safetensors models resolve their max position embeddings.
Derive model default state during render to drop the set-state-in-effect calls.
* Fix native GGUF context ceiling and guard picker template reads
Restore the native context store field so the sidebar slider keeps the
full ceiling for drag and drop GGUFs. Limit local chat template reads to
the browse allowlist, skip malformed repo ids, and drop unused model
picker exports.
* Fix model picker lint boundaries
* Fix model picker review findings
Chat template editor never seeded its draft. Radix only calls onOpenChange
from internal events, so the seed in the nextOpen branch was dead and a model
with a saved override opened empty. Saving then cleared the override. Drop the
dead branch, treat draft as an untouched sentinel, and reset it on every close.
Uncached Hub picks could auto load a model after the user left the chat. Main
detached the staged pick on route exit and on chat context change. Carry the
context key on the pending pick and skip the load when it no longer matches.
Also clear configTarget when the picker closes, restore the onUpdated ref so
variant rows stop resubscribing on every parent render, skip the LRU write when
the entry is already most recent, import NumericValueInput relatively, and drop
the unused ModelUpdateAction barrel export.
* Preserve GGUF context on active reload
* Fix model picker per-model config regressions
- Stop reloading the already loaded model on re-pick
- Hide infra models from the chat picker
- Detect vision support on cached GGUF repos
- Honor saved maxSeqLength on auto load
- Restore default chat template for local GGUFs
- Warn on save failure and revert config on cancel
- Refetch picker inventory on open
- Persist read only per model config safely
* Fix stale model auto load
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix model picker numeric input sizing and constraints
Size value inputs to their content so long context lengths are not clipped,
restrict them to numeric characters, and stop the speculative decoding label
from truncating in the sidebar.
* Fix picker CI tests and harden chat template resolution for PR #6647
- tests: point the descender guard at the moved model-selector.tsx path
- tests: exclude the disabled Reload model button from the regenerate locator so .first targets the real Regenerate
- picker/service.py: reject symlinked template/gguf leaves that resolve outside the browse allowlist (HF cache reads unchanged)
- compare mode: resolve each pane's own remembered chat template instead of inheriting the other pane's from the store
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Protect future-schema per-model configs from deletion for PR #6647
savePerModelConfig already refuses to overwrite a stored config whose schema version is newer than this client understands, but deletePerModelConfig did not. Unchecking Remember on an older client therefore silently destroyed a newer client's saved config. Apply the same guard on delete and surface the blocked case through the existing saveFailed toast.
* Protect future-schema per-model configs from quota eviction for PR #6647
The save and delete guards already refuse to touch a stored config whose schema version is newer than this client understands, but the quota-eviction path did not, so a full store on an older client could still evict a newer client's config. Skip future-schema entries when evicting and fail the save if the budget cannot be met without them.
* Fix GGUF context persistence, compare context, and rollback settings for PR #6647
Persist a GGUF context override from the user's intent instead of collapsing it against the loaded context, which reintroduced the context-reset (f4838782cb reverted the native-baseline fix). model-config-page now collapses the saved value against native, and use-chat-model-runtime and chat-adapter retain the requested context on load so re-saving another setting keeps the override; a null request stays null so a VRAM auto-fit never becomes a stored override.
shared-composer: a compare pane with no explicit GGUF context now loads at native (0) like single-view, not the session maxSeqLength that silently shrank the shown context.
use-chat-model-runtime: restore the previous model's KV cache dtype and chat template on a failed-load rollback so it runs as it was, not with backend defaults.
* Preserve native path token when reloading the active model for PR #6647
handleReloadActiveModel rebuilt the selection without the store's activeNativePathToken, so reloading a file-picked GGUF after a settings change validated the display label as a repo/path and failed. Thread the active native token through the reload selection so native-loaded models reopen correctly.
* Make picker template validation resilient and accept HF generation tags for PR #6647
Import Jinja lazily inside validate_chat_template so a backend without the optional jinja2 package (GGUF-only installs) still starts instead of raising ModuleNotFoundError at import time. Register a no-op extension for the Transformers {% generation %} assistant-mask tag so pasting a valid HF chat template validates, matching the renderer, rather than being rejected as an unknown tag.
* Honor remembered compare config and parse processor chat_template.json for PR #6647
* Fix failed-load rollback context and processor template map fallback for PR #6647
* Restore speculative decoding config on failed-switch rollback
When a model switch fails after the previous model was unloaded, the
rollback reload restored tensor_parallel, KV cache dtype and the chat
template override, but omitted speculative_type and spec_draft_n_max and
cleared their loaded shadows to null. The previous model therefore came
back running at backend defaults (speculation off) while the UI still
showed it enabled, and the status resync confirmed the off state. Resend
the previous model's speculative settings in the rollback load and keep
the store's active and loaded speculative fields in sync with them.
* Reset max sequence length when a model has no saved config
applyPerModelConfigToRuntime reset every per-model field except
maxSeqLength, which it only wrote when the incoming config had one.
maxSeqLength is the sole field carried on store.params, so selecting a
model with no remembered config left the previous model's value in place
and later loaded the new model at that leaked length. Fall back to the
standing default so an unremembered model loads at its own default.
* Surface a message when a variant update cannot start
startManagedUpdate handled the conflict and error start outcomes but let
busy fall through as if the update began, so the confirm dialog closed
with no job created and the cached variant stayed stale. Show an info
message when the repo is busy with a sibling transfer so the click is
not silently dropped.
* Keep per-model speculative choices out of the global default
A staged load with a per-model or one-off config sets keepSpeculative,
which already skips reading the global speculative preference. The
matching save still ran unconditionally, so the model-specific choice was
written to the global unsloth_chat_speculative_type and a later model with
no saved config started from it instead of Auto. Skip saveSpeculativeType
when keepSpeculative so the per-model choice stays isolated.
* Seed non-active model settings from the app default max length
The Run settings page captured initialMaxSeqLength from the loaded
model's runtime params and fell back to it for a model with no saved
config. Opening settings for a different, unloaded model and clicking
Load then sent the active model's context (for example 64k) instead of
the 4096 default, risking validation failures or OOMs. Seed the default
for non-active models and keep the runtime value only for the active one.
* Prefer sidecar tokenizer chat template over the GGUF copy for variants
_chat_template_from_dir returned the embedded GGUF template first when a
variant was selected, reversing the tokenizer-first precedence of the
no-variant path. A model whose chat_template.jinja or tokenizer_config.json
supersedes a stale embedded template then got the wrong template on
variant selection. Keep tokenizer files first regardless of variant; the
variant only picks which GGUF is the fallback. Adds regression tests for
both the tokenizer-wins and gguf-fallback cases.
* Keep per-model speculative choices load-local in autoload and compare
The interactive load path treats a per-model speculative choice as
load-local and skips writing it to the global default. Autoload and
generalized compare still called saveSpeculativeType unconditionally, so a
remembered off or ngram setting leaked into unsloth_chat_speculative_type
and later models with no saved config inherited it. Persist the global
preference only when the value came from the global settings.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: record the compare pane's loaded context in runtime state so the active model's settings and any reload or save use it, not the previous context
* Studio: notify the user when a Hub autoload can't start because another download for the model is already running, instead of silently dropping it
* Studio: drop the merge's orphaned staged-model store helpers and unused alert imports
The main merge left isPendingGguf and pendingSelectionMatches referencing the
removed PendingModelSelection type, and the alert-dialog/alert imports unused
after the permission-mode dropdown replaced the bypass dialog, so tsc -b failed.
* Studio: cache a null default chat template so the viewer stops re-fetching it
A model with no sidecar or embedded template resolves to a terminal null, but
that result was never cached, so reopening the template viewer re-ran the
backend and Hugging Face lookup every time.
* Studio: detect direct-file GGUFs in run settings so Max Tokens uses their context
A GGUF loaded from a local file or custom folder has no variant label, so the
run-settings panel treated it as non-GGUF and clamped Max Tokens to the session
max_seq_length instead of the loaded GGUF context. Detect it via the reported
GGUF context and the .gguf checkpoint suffix, matching the chat page.
* Studio: prompt to re-select a local model file when its lease expired before reload
A file-picked GGUF is reachable only through a native path token that the
desktop host prunes after a TTL. Reloading reused that token blindly, so a
reload long after the initial load failed with an opaque error. Track the
token's expiry and, when it has passed, ask the user to re-select the file
instead of attempting a doomed reload.
* Fix descender-clipping test to tolerate sidebar layout utilities
The sidebar account-block div carries layout utilities (min-w-0, flex-1)
between 'flex' and 'flex-col', so the descender-clipping guard's regex,
which required 'flex' immediately followed by 'flex-col', no longer matched
and the test failed to locate the account-block div. Generalize the prefix
to allow intervening flex utilities while still capturing the leading-*
class before the collapsible visibility utility and asserting leading-tight,
so the guard against clipped glyph descenders is fully preserved.
* Harden picker chat-template resolution
Enforce the 64 KiB chat-template contract at the validate endpoint's request
model so a direct caller cannot submit a template far larger than the frontend
allows (MaxBodyMiddleware only bounds the whole request body, not this field);
oversized templates now return a clean 422.
Apply sidecar-over-GGUF template precedence globally across cached snapshots
instead of per snapshot. A repo with multiple cached revisions previously
returned the first snapshot's template, so a newer GGUF-only revision could
win over an older revision's maintained chat_template.jinja sidecar, which
contradicted the documented intent that sidecars supersede the embedded copy.
* Guard per-model config against future-schema and lossy migration
Two forward-compatibility gaps in the versioned per-model config store:
- The load/apply path returned and normalized a stored record without checking
its schema version, so a record written by a newer client was reinterpreted
under the current schema and applied to a live model load, even though save,
delete and eviction all refuse to touch future-schema records. Reject
future-schema records on load too.
- The one-time legacy migration enforced the storage budget without protecting
the entries it had just migrated and set the completion flag unconditionally.
When storage was already full of future-schema records (which are unevictable
by an older client), the migrated entries were the only evictable ones and
could be dropped while migration was still marked complete. Protect the
migrated keys during eviction and only mark migration complete when they
survive, so it retries once space frees up.
* Discard chat-template validation results after the dialog closes
Server-side template validation is async, but closing or cancelling the editor
did not abort it, so a late-arriving valid response still called onSave and
applied a template the user had already dismissed. Track a validation token
that is bumped on close and ignore any validation result whose token is stale.
* Record native lease expiry when loading a picked GGUF from the chip
The pending-native-model chip loaded via stageOrLoad directly, bypassing
loadNativeModelIntent, so activeNativePathExpiresAtMs was never recorded for a
chip-loaded file. A later reload then either skipped the lease-expiry guard
entirely (expiry left null) or compared against a previously loaded file's
stale expiry, so reload could reuse an already-pruned token or wrongly block a
still-valid one. Route the chip through loadNativeModelIntent, which builds the
same selection and records the expiry.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Prefer sidecar template for a directly selected local GGUF file
A direct .gguf file path read its embedded chat template without checking the
parent directory for a maintained sidecar (chat_template.jinja /
tokenizer_config.json), while directory and variant selections already prefer
the sidecar. That let the config editor preview or save a stale embedded
template for the same model depending on how it was selected. Check the parent
directory sidecars first, then fall back to the embedded copy, and cover both
paths with tests.
* Resolve cached chat template per revision, newest first
The earlier change searched every cached snapshot for a sidecar before
considering any snapshot's embedded GGUF template, which let an obsolete sidecar
from an older revision override the newest revision's template. Restore
per-snapshot resolution (newest first): a revision's sidecar still supersedes
its own embedded GGUF copy, but a newer revision is no longer overridden by an
older revision's sidecar.
* Preserve autoload transport conflicts and surface background busy downloads
- When a Hub autoload hits a transport conflict, keep pendingHubAutoLoad bound
instead of clearing it. Clearing it re-keyed the download surface and its
cleanup cancelled the conflict the toast tells the user to resolve, so the
Hub resume affordance was gone the moment it appeared. Return early on
conflict, mirroring the started branch, so resolving it from the Hub still
auto-loads on completion.
- The background-download branch handled started and conflict but silently
dropped a busy outcome, leaving the user with no feedback when a peer variant
of the same repo was already downloading. Surface the same busy toast the
autoload path uses.
* Fix context length, GGUF template, fetch state and lease expiry bugs
Keep explicit context length values instead of collapsing to null at
native. The collapse made the slider jump back at the native maximum
and made Reload load the previous context instead of the chosen one.
Prefer the first split when resolving a GGUF without a variant. Later
splits carry no chat template metadata, so picking the largest file
could return no template for a sharded model.
Clear stale fetch state when template and metadata lookups retry, so
a previous terminal error is not shown while a new fetch is running.
Record native path lease expiry together with the token when a load
commits. The expiry was written by only one load path and even when
the load did not start, so a reload could be blocked with an expired
file message for a still valid token.
* fix(model-picker): resolve review findings across config, inventory, and templates
- Apply remembered per-model config in the training-compare chat handoff so a
prior model's customContextLength no longer leaks into the next load
- Match GGUF variant labels with the inventory extractor too, so cached
no-quant-token files resolve their default chat template
- Show "Auto" instead of a fabricated 32768 when native context is unknown
- Reuse the identical staged auto-load object on same-pick so a re-pick during
download pre-flight no longer disarms auto-load via "busy"
- Union supports_vision when deduping cross-cache inventory rows
- Serve hidden-model needles from a new GET /api/hub/hidden-models endpoint and
merge them client-side, covering runtime-configured RAG embedders
- Clamp GET chat templates to MAX_CHAT_TEMPLATE_BYTES (route + jinja sidecar),
matching the validate endpoint's contract
- Lower-clamp stored customContextLength to shared CONTEXT_LENGTH_MIN
- Wipe unsloth_chat_load_on_selection in Settings "Reset all"
- Drop stale pendingHasContext comment describing deleted staging machinery
* Fix stale defaults cache, token in query string and rounded up context ceiling
Refresh cached chat template and max position data when a model update
completes. Send the HF token for model config requests in the dedicated
header instead of the URL. Snap the native sequence length ceiling down
to the nearest step so the slider cannot exceed the declared maximum.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix compare pane reverting active checkpoint on non-GGUF load
Re-read runtime params after setCheckpoint so the fresh checkpoint is
kept instead of being overwritten by the pre-setCheckpoint snapshot.
* Send the HF token via header for the vision and embedding checks
checkVisionModel and checkEmbeddingModel still passed the HuggingFace
token as a ?hf_token= query parameter, so it landed in server access
logs, proxy logs, and browser history. Move them to the
X-Unsloth-HF-Token header like getModelConfig already does, and accept
the header on the check-vision and check-embedding routes with the
existing query parameter kept as a fallback for older clients.
* Cap the chat template on the model load path
The load endpoint accepted an unbounded chat_template_override, so a
direct caller could hand llama.cpp an arbitrarily large Jinja template
even though the frontend, the validate endpoint, and the read paths all
enforce the 64 KiB limit. Reuse MAX_CHAT_TEMPLATE_BYTES in the
LoadRequest validator, rejecting oversized templates with a fast
character-count check before the exact UTF-8 byte check.
* Protect existing per-model configs during legacy migration
When the one-time legacy import pushes the store over budget, eviction
now protects the entries the user already has and drops only the
just-migrated legacy entries, so importing old load settings can never
discard a newer per-model config.
* Reset clears the context override instead of pinning the native value
Reset wrote the discovered native context into customContextLength for
GGUF models, but isDefaultConfig treats any non-null customContextLength
as an explicit pin, so Reset with Remember enabled persisted a fixed
context and future loads stopped using the native auto context. Reset
now restores the full default (customContextLength null); the native
value is still shown through the existing display fallback.
* Bound chat-template sidecar reads to a size limit
The chat_template.json, tokenizer_config.json, and Hub-downloaded sidecar
readers decoded and json-parsed the whole file before the extracted
template hit the 64 KiB response cap, so an oversized metadata file could
exhaust memory. Read them through a bounded reader (4 MiB envelope) that
returns None when the file is larger, matching the existing chat_template.jinja
size guard. Adds tests for oversized tokenizer_config.json and chat_template.json.
* Keep the native-path token and lease expiry in sync
Rollback after a failed reload restored the previous token but left the
failed load's expiry in the store, so a later reload could be falsely
blocked as expired (token A paired with load B's lease). Restore the
previous lease alongside the token, and clear the expiry wherever the
token is cleared on a non-GGUF transition, so the two never diverge.
* Clear the native file lease on compare-pane loads
* Studio: add regression tests for the model-picker per-model-config
Guard the specific regressions that reverted the predecessor change:
- backend pytest (studio/backend/tests/test_model_picker_regression.py):
infra-model hiding, HF token via header with query fallback, and the
chat-template byte caps.
- source contracts (tests/studio/test_model_picker_contracts.py): the token
stays out of the URL, the context ceiling is floored, the native lease is
cleared on compare-load and restored on rollback, the default caches key on
the inventory version, and the hidden needles stay present.
- Playwright E2E (tests/studio/playwright_model_config.py) wired into
studio-ui-smoke.yml on port 18898: Context Length persists across a reload,
Reset clears the stored override, and infra models are absent from the picker.
- optional GPU-gated inference smoke (tests/studio/test_gpu_inference_smoke.py)
that auto-skips on GPU-less CI and stays short on a GPU.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: model pinning, row menus, hub inference settings, and inventory filters
Pinning
- Add a pinned models store (localStorage) with repo and per-quant pins
- Pinned section in the model selector's On Device list and the hub inventory,
with newest pins first so Pin to top lands on top
- Deleting a repo drops its pins
Row menus
- Replace loose row icons with a shared 3-dots menu (pin, reveal in file
manager, copy identifier, copy path, delete) on picker rows, hub quant rows,
the hub run bar, and on-device inventory rows
- Menus only render for models actually on disk; platform-aware reveal labels
- Backend: cached-model-path and reveal-cached-model endpoints resolving
managed HF-cache repos only
Hub inference settings
- Gear in the GGUF run bar opens an Inference settings dialog reusing the chat
page's controls: model config (context length, KV cache, speculative
decoding, chat template), system prompt, reasoning, sampling, tools and
retrieval
Inventory
- Model-type filter (text, vision, embedding, STT, TTS, diffusion) beside the
sort pill, both with a sort icon, capped widths and truncation so the
On device heading never wraps
- Unsloth-owned repos without an upstream provider logo fall back to the
Unsloth mascot avatar
- Discover / On Device tabs widened; hub search bar narrowed to match
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: revert the Unsloth mascot avatar fallback
Unsloth-owned repos without an upstream provider match go back to the
colored-initial tile, and unslothai is no longer a relabeled owner.
* Studio: run-bar options on single models, and aligned type/capability filters
- Give single-model (non-GGUF) run bars the same 3-dots options menu and
settings gear as GGUF, at repo level
- Drop Pin to top from the run-bar menus; pinning stays in the On Device list
- Add an Image to text (diffusion) capability with detection, and surface it
in both the hub Discover capability filter and the On Device type filter
- Align the On Device type filter with the Discover capability options and
share the same detection so both dropdowns match
* Studio: apply hub inference config on reload, eject action, and run-bar polish
- Fix inference settings not applying: the hub dialog now writes the config to
the runtime before reload, matching the chat page (selectModel reads runtime
state, not the selection)
- Order the settings gear before the 3-dots menu in the run bars
- Replace the loaded-model run-bar action (New Chat) with Eject, wired through
the inspector to the hub's ejectModel
- Truncate the results heading so a long search query clips instead of
overlapping the header pills in split view
- Use a plain magnifying-glass icon for the no-results empty state
* Studio: fix GPU settings loss, load guards, pins, filters, and cached paths
Reloading a model from the chat sidebar or the hub gear dialog rebuilt the
per-model config without the GPU memory fields, so manual GPU layers, MoE
placement, and the GPU pick were reset on every reload and could be saved
over a remembered config. The active config now comes from a shared
useActiveModelConfig hook that carries the GPU fields for GGUF models, and
the sidebar remount signature tracks them through a shared gpuFieldsSignature
helper.
The in-flight load guard lived in a ref inside each useChatModelRuntime
instance, so the chat page, hub page, and gear dialog could not see each
other's loads. A load started from the gear dialog left the hub page free to
eject the model mid-reload or start a second concurrent load. The runtime
store now records the loading pick, selectModel checks it across instances,
and ejectModel refuses with a toast while any load is in flight.
The cached-model-path endpoint matched GGUF files by basename and excluded
only mmproj, so Copy path and Reveal could return an MTP drafter for a quant
and returned 404 for directory layouts like BF16/model-00001.gguf. Variant
files are now resolved from snapshot-relative paths with the same drafter,
mmproj, and big-endian exclusions as the load path, shared through a new
_main_variant_gguf_label helper.
Hub and picker fixes:
- rename the diffusion capability label from "Image to text" to
"Image generation", since it detects image generators
- validate pinned quants through the cached variant listing, keep the last
verified set while revalidating, and drop deleted quants immediately
- pass a measured scroll margin to the on-device virtual list so rows past
the overscan stay visible below the pinned block
- keep the delete menu for stopped partial safetensors downloads
- give the inventory type filter a reset in Clear filters, a truthful empty
state with a Show all types action, and hide it on the datasets view
- order picker pinned rows by pin recency, include pinned matches in the
empty-state check, and sync pins across browser tabs
- count only the visible rows in the On device list header
Tests: contract checks for each fix in test_model_picker_contracts.py and a
backend test for the variant label selection.
* Studio: reveal cached models in Windows Explorer under WSL
The reveal endpoint only branched on macOS, Windows, and generic Linux.
Under WSL the Linux branch spawned xdg-open, which is missing on a stock
distro without a Linux desktop, so the request failed with a 500 and the
UI showed a failed to open file manager error.
WSL is now detected with the existing helper and the path is converted
with wslpath before opening explorer.exe, selecting the file the same
way native Windows does. Directories open directly. When interop is
unavailable the old xdg-open fallback still runs. The macOS, native
Windows, and native Linux branches are unchanged, and the Tauri app is
covered since its hub reveal calls this same local endpoint.
Tests: platform guards for the WSL reveal, the interop fallback, and
the unchanged native Linux behavior in tests/studio/test_reveal_file_manager.py.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Adjust model picker row spacing and cogwheel hover consistency
* Studio: exact hidden model ids and newest revision cached paths
A custom RAG embedder repo was published to the frontend as a basename
substring needle, so a generic name like org/model could hide unrelated
models in the pickers. The hidden-models endpoint now sends full repo ids
that are matched exactly.
Copy path and Reveal picked a GGUF variant from an arbitrary cache
revision when the same file existed in more than one. The newest revision
now wins, matching the whole repo lookup.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix model picker GPU config, metadata, and cache selection
Load each compare model with its saved GPU memory mode, GPU layers, CPU MoE layers, and selected GPU IDs. Reconcile saved GPU IDs with the current hardware. Include the active native GGUF path token in metadata checks. Search all Hugging Face cache roots when resolving cached models and select the largest visible cache entry. Remove obsolete barrel exports and the staging-only GPU memory helper.
* Studio: hide hub inference settings gear for now
The cogwheel in the hub download cards is out of scope for this PR. The
dialog component stays in place and a TODO marks where the button
returns in a future PR.
* Refresh hidden model matchers
* Fix GGUF detection, compare context pin, and picker delete staleness
Treat any pick with a GGUF variant as GGUF in selectModel so the first
load after downloading an uncached quant validates and sizes with the
right GPU settings instead of unloading the current model on a wrong
preflight. Variant picks now also set isGguf on their selection meta.
Stop compare panes from inheriting the active model's context pin when
their own saved config says Auto. Null context in a remembered config
now means no pin, matching how the pane settings are shown.
Route picker deletes through the hub inventory client, which
invalidates the HF cache scan and the variants cache. The legacy
delete route left the scan cache warm, so deleted models reappeared
in the picker until the TTL expired. Removed the now unused legacy
delete client and updated the contract test to match.
* Studio: fix stale GGUF load-marker ordering test
The load-in-flight marker still precedes the hub-download guard and the
unload, but the llama_extra_args inheritance that used to sit between the
marker and the guard now runs ahead of the GGUF branch, so it is no
longer a landmark inside the sliced source. Drop it from the ordering
assertion and keep the marker -> guard -> unload invariant.
* Studio: fix per-model config edge cases in compare loads and saved defaults
- chat-settings-sheet: gate the MTP fallback note and context/VRAM warning on
the broader isGguf (variant, loaded gguf context, or .gguf suffix) instead of
isLoadedGguf, so direct-file and custom-folder GGUF loads still surface
those diagnostics.
- shared-composer: a compare pane's context now comes from its own config only
(a saved pin, else null for Auto/native). It no longer inherits the active
model's shared snapshot, which resolveFitMaxSeqLength treated as an explicit
pin and could load a pane at another model's context (VRAM/OOM), matching the
single-model load path.
- model-config-page: when an auto-fit GGUF is saved with fixed GPU layers
(Manual) and Remember, pin the displayed fitted context so a later fresh load
keeps the placement instead of sending native/0 and recreating the OOM.
- per-model-config: treat Auto GPU memory mode and Auto/default speculative type
as follow-global defaults; do not persist them as per-model overrides so later
global preference changes keep applying.
* Studio: gate vision capability on GGUF projectors and bound remote template downloads
- cache_inventory: only mark a cached repo vision-capable when it holds an actual
GGUF mmproj projector, not any file whose name merely contains "mmproj" (e.g.
mmproj_config.json), matching the runtime's GGUF-only projector detection.
- picker/service: pre-check the remote file size before downloading an uncached
repo's chat template / tokenizer config, so a maliciously large sidecar is
skipped instead of fetched and retained in full, mirroring the size gate the
local-file path already applies.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: add source-contract guards for the per-model-config edge-case fixes
Guard the four per-model-config fixes against silent regression in CI:
- local GGUF diagnostics gate on the broad isGguf, not the variant-only isLoadedGguf
- fixed-layer GGUF saves pin the displayed context
- Auto GPU mode and Auto/default speculative are not persisted as per-model overrides
- a compare pane's context comes from its own config, not the active model's snapshot
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: clear manual GPU knobs on Default and resolve local embedders before repo-id
- model-config-page: switching GPU Memory back to Default now clears the Manual-only
knobs (gpuLayers/nCpuMoe/selectedGpuIds); otherwise a remembered config kept stale
pins that a later load re-applied when the global GPU preference was Manual, despite
the page showing Default.
- routes/models hidden_model_matchers: resolve an existing local path before the repo-id
regex, mirroring is_hidden_model, so a local embedder shaped like "models/embedder" is
hidden by exact path instead of leaking as a chat model.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: add _is_mtp_drafter to the model_config stub in the export-paths test
routes/models.py imports _is_mtp_drafter from utils.models.model_config at module
load, but the lightweight stub in test_export_absolute_paths.py did not provide it,
so loading the module under the stub raised ImportError on Backend CI. Add the stub.
* Studio: read a picked GGUF's chat template through the native path lease
The picker chat-template GET has no native-path-lease plumbing, so a
desktop-picked (drag-drop) GGUF could not show its default chat template
in Run Settings until the model was loaded: the endpoint only receives
the display label, not the leased file path.
Read the embedded template through the existing lease-aware
/api/inference/validate probe instead. A new include_chat_template flag
resolves the granted canonical path and returns the GGUF's own embedded
template, never a sibling sidecar (the grant authorizes just that one
file); it skips the training guard like include_context_length and is
bounded by MAX_CHAT_TEMPLATE_BYTES. The frontend fetch mints a one-shot
validate-model lease when a native token is present and keeps the plain
GET path for HF and allowlisted local models.
Adds backend and source-contract regression tests.
* Studio: call worker.direct_wheel_url in the ROCm wheel-url test
The ROCm Mamba/SSM test referenced worker.py's private _direct_wheel_url,
but the worker imports the wheel helper under its public name
direct_wheel_url (utils.wheel_utils). When the worker module loads (its
imports resolve in CI), worker_mod._direct_wheel_url raised AttributeError;
the test only masked it by skipping when the worker could not be imported.
Call the name that actually exists so the assertion runs; it still returns
None for an empty cuda_major (ROCm).
* Studio: reset max sequence length to the app default, not the loaded value
For a non-GGUF active model, the per-model config seeds maxSeqLength from
the loaded runtime value so the panel opens showing the running context.
Reset set config.maxSeqLength to null, but the null fallback resolved back
to that captured runtime value, so the field kept showing the old custom
length and the config saved/reloaded it again. A remembered or active
max-length override therefore could not be cleared from Run settings.
Fall the null/default case back to the app default (clamped to the model's
native ceiling) instead of the active runtime snapshot, so Reset actually
clears the override. The initial view is unaffected: an active model's
config.maxSeqLength is already non-null, so it still shows the loaded value.
Adds a source-contract regression guard.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: persist default max length, refresh deleted quants, hide non-chat locals
Three follow-up fixes from review of the per-model-config picker:
- Max sequence length: the persisted per-model record now keeps config's
maxSeqLength (null after Reset) so isDefaultConfig can clear a remembered
override; the resolved app-default is substituted only into the load
request, never the saved record. Previously Reset saved the concrete
default and left the model pinned/remembered.
- GGUF variant expander: deleting a downloaded quant from a repo that still
has other cached quants now bumps the expander refresh key, so the removed
quant stops showing as downloaded and clickable (which would try to reload
the deleted file) until the repo is collapsed and reopened.
- Local picker rows: require capabilities.canChat before listing a local
models-folder / LM Studio row. A weightless folder (only config.json) is
classified non-chat, and toLocalModelInfo drops capabilities, so selecting
such a row would try to load a path the inventory already marked non-chat.
Adds source-contract regression guards for all three.
* Fix compare-pane and Reset context defaults in model picker
Two related per-model-config default regressions:
- A non-GGUF compare pane with no saved maxSeqLength fell back to the
active model's shared runtime snapshot, so comparing a saved 128K model
against an unconfigured pane loaded the latter at 128K and could OOM. It
now falls back to the shared app default (DEFAULT_MAX_SEQ_LENGTH), the
same fallback the single-model config path uses.
- contextAtDefault treated an explicit customContextLength equal to the
native ceiling as a default, which wedged the Reset button disabled for
a deliberate pin-to-native. It now counts as default only when there is
no override at all.
DEFAULT_MAX_SEQ_LENGTH becomes a single exported constant in
per-model-config.ts so the single-model config and the compare path share
one source of truth. Adds source-contract guards for both fixes.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Skip over-cap remote Jinja templates so the tokenizer template wins
The remote chat-template resolver bounded raw chat_template.jinja downloads
only by MAX_TEMPLATE_METADATA_BYTES (4 MiB), then returned the first
non-empty Jinja unconditionally. The picker route drops any template larger
than MAX_CHAT_TEMPLATE_BYTES (64 KiB), so an uncached repo whose
chat_template.jinja sits between 64 KiB and 4 MiB returned no template at
all, even when a valid smaller tokenizer_config.json template existed. The
local path already skips oversized .jinja files and falls through.
Gate the extracted Jinja on MAX_CHAT_TEMPLATE_BYTES and continue searching
when it exceeds the cap, matching _chat_template_from_jinja_file. The 4 MiB
download bound stays for JSON files that merely embed a small template. Adds
a regression test that a big Jinja plus a valid tokenizer config resolves to
the tokenizer template.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Guard legacy per-model-config migration idempotency
The v1->v2 localStorage migration (unsloth_load_settings ->
unsloth_model_configs) runs on every store read, so it must migrate exactly
once and never re-run, duplicate, or clobber a newer per-model config on a
reload or restart. That was covered only by a manual proof, so add durable
guards:
- Source-contract test pinning the three idempotency layers (the in-memory
legacyMigrationChecked guard, the persistent unsloth_model_configs_migrated
flag set in every terminal branch, and the non-overwriting Object.hasOwn
merge-skip) plus the readMap invocation. Reddens if any layer is dropped.
- Playwright model-config E2E: promote the legacy-migration step to a gating
check (soft_fail, which gates under the CI STUDIO_UI_STRICT=1) that the
migrated value is preserved and the flag is set, then reload again with a
fresh legacy seed present and assert the stored key set is unchanged, so a
second reload cannot re-migrate, duplicate, or clobber.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Note the migration E2E now gates idempotency under STUDIO_UI_STRICT
* Tighten model-picker per-model-config code comments
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
Co-authored-by: shimmyshimmer <shimmyshimmer@users.noreply.github.com>
Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local>
Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
* Studio: make repeated tab switches feel immediate
* Keep cached Studio navigation data fresh
* Make first Studio tab visits responsive
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Serve range requests uncompressed for immutable assets (PR #7271)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: test <test@test.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Studio: validate Hugging Face tokens before use
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: keep token validation failures non-blocking
* Studio: harden Hugging Face token preflight
* Studio: make token validation effect lint-safe
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
* Studio: fix per-GPU VRAM reporting on Windows ROCm
On Windows ROCm without a HIP SDK, amd-smi is disabled and the System tab fell
back to torch mem_get_info, which reports free==total there (ROCm/ROCm#1909), so
used VRAM showed as 0. The perf-counter fallback also summed every adapter into a
single device with only GPU 0's total, hiding the second GPU.
Read per-adapter Dedicated Usage (LUID-instanced) for used and take each GPU's
total from torch properties, and treat the free==total case as unknown rather
than 0, so every GPU shows real usage. NVIDIA, Linux ROCm, Apple and CPU paths
are unchanged. Final validation needs a real Windows AMD box.
Fixes#7072
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: report unknown VRAM instead of fabricating or zeroing it
Two gaps in the Windows ROCm VRAM path. When more adapters are actively using
VRAM than are visible to the process (a GPU outside the visibility mask), the
per-adapter attribution paired usage by size and fabricated a per-GPU value;
report unknown for every device in that case rather than mis-assign. And the
System API turned an unknown (None) used value into 0 with ``or 0``, then
reported the full card as free, re-hiding the exact case this change surfaces;
keep None so the UI shows unknown.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Render unknown VRAM as Unknown instead of zero in the System tab
The backend reports null usage when it is unknown (e.g. the Windows ROCm
perf counter is unavailable or localized), but the System tab coerced
null to 0 and derived free from it, fabricating a 0-used/full-free total.
Preserve null and render the translated Unknown for per-device used, free
and utilization, and mark the aggregate VRAM tile unknown when any device
is unknown.
* Render unknown VRAM as Unknown in the floating monitor and the util tile
The floating VRAM monitor and the aggregate utilization ring both still
coerced a null usage to 0, showing a fabricated 0.00 GiB / full free / 0%
on the same Windows ROCm no-counter case the resources tab already
handles. Guard both on whether every device reports a finite usage and
render Unknown (value and percent) instead of a concrete 0.
* Attribute per-adapter VRAM usage only when capacity forces the mapping
On Windows/ROCm there is no shared key between LUID performance-counter
instances and torch ordinals, so usage was paired to devices purely by capacity
ranking. That pairing is only trustworthy when capacity forces it (a usage
larger than every smaller device can sit on one card). When a smaller-capacity
device could equally hold a strictly larger usage (for example an 8 GiB card
near full beside a lightly used 48 GiB card), the two values are swappable
without violating any capacity, so the ranking is a guess with no key to break
the tie. A wrong guess both mislabels the System tab and feeds
routes/training_vram.py a wrong per-index free value, driving a wrong
keep-resident decision.
Report unknown for every device when the assignment is ambiguous, keeping the
attribution only for the capacity-forced case. Returning None is the
conservative direction: training_vram treats a missing index as zero free, so it
never keeps a chat model into an OOM. Add regression tests for the
not-capacity-ordered, same-capacity, single-fits-both, and capacity-forced
cases.
* Report unknown VRAM usage when a hidden adapter survives the noise filter
When HIP_VISIBLE_DEVICES exposes a subset of the physical adapters, the LUID
usage counters cover cards outside the visibility mask too. The sub-64 MiB noise
filter could drop a genuinely-idle visible card's real usage while keeping a
hidden larger card's high usage, which was then clamped onto the smaller visible
device and reported as fully used (for example a hidden 48 GiB card at 40 GiB
shown as a visible 8 GiB card fully used, with its true 10 MiB usage filtered
out). That fabricated reading also feeds routes/training_vram.py a wrong
per-index free value.
Flag extra adapters on the raw counter count (before the noise filter, since an
idle visible card can itself fall below the floor) and, when a kept usage exceeds
its ranked visible capacity, report unknown rather than clamp a hidden card's
usage onto a visible device. The genuinely-idle-noise and capacity-forced
single-model cases are unchanged. Add a regression test for the hidden
high-use-adapter case in both counter orders.
* Report unknown when only a placeholder adapter counter survives the noise filter
When more raw counters than visible devices are present but every counter sits
below the 64 MiB noise floor (an idle real GPU alongside a Windows Basic Render
Driver placeholder), the non_trivial-or-raw fallback resurrected the raw
magnitude-sorted counters and could attribute the placeholder to a real GPU while
dropping a real card's reading. With a single visible device the swap-ambiguity
check cannot catch it (it needs at least two ranks), so the fabricated value
reached the System tab and automatic GPU selection.
Return unknown for every device in that case instead of falling back to raw
counters. With the earlier guards this completes the invariant: a concrete
per-GPU usage is emitted only when the assignment is capacity-forced, and every
ambiguous, extra-adapter, placeholder-fallback, or count-mismatch path reports
unknown. Add a regression test for the placeholder fallback in both counter
orders and the two-idle-GPU case.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: attribute Windows/ROCm VRAM only when capacity forces a clean bijection
With more raw adapter counters than visible devices, a survivor that merely
fits a visible card was pinned to it by magnitude ranking, fabricating a hidden
GPU's usage onto an idle visible card whose true reading was dropped by the
sub-threshold noise filter (two visible 48/8 GiB cards using 40 GiB / 10 MiB
beside a hidden 6 GiB adapter returned [40, 6]). Emit a concrete per-device
value only when the supra-threshold counters number exactly the visible devices
(every visible card has one real reading, the extras were sub-threshold
placeholders) AND the ranked usage strictly exceeds every smaller visible card's
capacity. When a visible card is idle (fewer supra-threshold counters than
devices) a survivor could be the hidden GPU's usage, so every device reports
unknown; more active counters than visible cards, the smallest card, and any
merely-fitting usage stay unknown too. The reporter's loaded-card display is
preserved (40 GiB / 0.5 GiB across 48/8 GiB -> [40, None]). Adds a regression
test for the reported case plus an exhaustive capacity-forced/bijection matrix.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: keep the unified-memory total when Windows-ROCm used is unknown
_apply_unified_memory_correction gated both the total and the used update on
torch_used_gb being known, so on a unified-memory APU (Strix Halo) where torch
reports used=None (the Windows-ROCm free==total sentinel) but an authoritative
full-GTT total, the device kept amd-smi's small dedicated carve-out and
underreported its capacity on the System tab. Adopt torch's larger total
independently of used; overwrite used only when torch's is known (otherwise keep
amd-smi's dedicated-usage figure) and recompute utilization against the
corrected total. Adds regression tests.
* Tighten comments in the ROCm/Windows VRAM reporting path
* Tighten comments further in the ROCm/Windows VRAM reporting path
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <unslothshared@gmail.com>
* Studio: persist llama.cpp KV cache across idle auto-unload (slot save/restore)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: address KV persistence review feedback
* Studio: guard KV restore on launch config
* Studio: fix KV resume purge race, fingerprint requested ctx, purge on disable
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: re-check idle/keep-KV settings after slot save, ns file identity
* Studio: shard-aware KV guard, honor user --no-cache-prompt, early save cap
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: honor LLAMA_ARG_CACHE_PROMPT env in slot-save guard
* Studio: derive prompt-cache state from final argv for slot saves
* Studio: stat LoRA/control-vector sidecars in KV restore fingerprint
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: parse csv and FNAME:SCALE sidecar syntax in KV fingerprint
* Studio: address codex review on idle-unload KV resume
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: harden slot-save cleanup, cap accounting, stale-KV guard, save timeout
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: treat unavailable KV estimate as full-cap for slot-save disk check
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Studio: GPU memory dropdown — llama.cpp --fit on and manual gpu-layers/cpu-moe
* Studio: simplify GPU memory changes (reuse ParamSlider, GPU_LAYERS_ALL, loadedGpuMemoryFields helper)
* Studio: GPU picker — choose which GPUs a GGUF model loads on (gpu_ids)
* Studio: simplify GPU picker (share /api/system fetch, validate gpu_ids)
* Studio: GPU picker review fixes (gate relative indices, no cross-model leak, validate, types)
* Studio: group GPU controls under a collapsible GPU section
* Studio: GPU feature review fixes (fix fit-ctx test, behavior-test the floor, comment accuracy)
* Studio: make GPU a top-level settings section (not nested under Model)
* Studio: flatten GPU controls into the Model section, group by GPU/context/generation
* Studio: move GPU Memory to the bottom of Model with its dependent controls beneath it
* Studio: move GPU Memory below Tensor Parallelism and GPUs below GPU Memory
* Studio: tighten GPU Memory and GPU Layers tooltip copy
* Studio: fix fit-mode context slider track-click, restore GPU Memory tooltip, shorten fit dropdown label
* Studio: GPU Memory tooltip one mode per line, briefer
* Studio: note HIP_VISIBLE_DEVICES (ROCm) in the GPUs picker tooltip
* Studio: narrow the GPU Memory dropdown to fit the shortened label
* Studio: use 'llama.cpp --fit' in the GPU Memory tooltip for consistency
* Studio: allow Tensor Parallelism in Manual GPU mode
* Studio: graduated MoE-on-CPU offload (--n-cpu-moe) replacing the all-or-nothing toggle
* Studio: size the MoE-offload slider for staged (deferred-load) models
* Studio: share one GGUF header walk for the context-length and MoE-count readers
* Studio: size the GPU Layers slider for staged models (one staged-header read)
* Studio: move Tensor Parallelism below the GPUs picker
* Studio: GPU split (--tensor-split) per-GPU model share in Manual mode
* Studio: tolerate whitespace in GPU split input, move it below GPU Layers
* Studio: rename the GPU split control to "Split ratio"
* Studio: Split ratio sends explicit even input; fix blank=free-VRAM (not even) copy
* Studio: tighten llama.cpp --fit VRAM margin with --fit-target 512
* Studio: GPU memory review fixes (rollback re-baseline, single-GPU TP gate, accurate copy)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: move Split ratio below MoE Layers on CPU
* Studio: address PR review (fix GPU-info hydration race, share fit context-length across load paths)
* Studio: address codex review (manual single-GPU TP guard, GPU-aware spec defaults in fit/manual, GGUF-only context/preference)
* Studio: address codex review round 2 (gpu_present seed, single-GPU tensor-split guard, staged manual-knob reset, strip inherited offload flags)
* Studio: address codex review round 3 (strip inherited --n-cpu-moe, CPU-fallback warning in Manual mode)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: address codex review round 4 (preserve pinned fit context across a later Apply)
* Studio: address codex review round 5 (honor GPU picker for diffusion GGUFs, clear fit pin on cross-model switch)
* Studio: preserve the pending GPU Memory mode when staging a model
* Studio: pin diffusion GPU device order and reset GPU-memory state for diffusion loads
* Studio: address codex review round 6 (fit-Auto rollback context, preserve manual non-tensor split modes, persist GPU mode on load not select)
* Studio: persist the applied GPU Memory mode, not the requested one (skip diffusion loads)
* Studio: replace Manual-mode split-ratio field with per-GPU layer sliders
* Studio: clarify per-GPU layer split hint for tensor-parallel mode
* Studio: address codex review round 7 (allow GGUF gpu_ids past the legacy guard, replay GPU-memory fields on respawn)
* Studio: address codex review round 8 (size the validate preflight like the load in fit mode, across both load paths)
* Studio: skip the training-OOM guard for llama.cpp --fit GGUF loads (they spill to RAM)
* Studio: drop the now-redundant compare-path validate sizing (the --fit guard skip makes it moot)
* Studio: address codex review round 9 (keep the training guard for fit loads, forward gpu_ids to validate, strip inherited manual tensor-split)
* Studio: address codex review round 10 (gate GPU-memory adoption on is_gguf, record manual knobs only in Manual mode)
* Studio: handle diffusion GGUFs symmetrically in the GPU Memory controls (preserve the standing mode preference, hide the inapplicable mode/TP controls)
* Studio: remember the GPU Memory settings per model
* Studio: consolidate --fit mode and Manual mode into a single Manual mode
* Studio: preserve the per-GPU layer split across GPU Layers changes
* Studio: trim overly long GPU Memory comments
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* address GPU memory config review comments
* trim redundant GPU memory tests
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Reconcile manual-mode TP drops with the #6659 drop-site invariants
* Preserve quantized KV in manual --fit, charge GGUF companions in full, reconcile GPU pick on load
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Clear stale GPU baseline on non-GGUF loads so it can't read as dirty
* Fix no-context-shift test for the conditional -c flag
* Credit manual GPU-layer offload for cached HF GGUFs
* Reset per-model load knobs on GGUF quant switch
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Strip inherited tensor-split when manual ratio is cleared
* Match auto-load validation to safetensors placement
* Reset editable manual knobs after Auto GGUF loads
* Record a single device for diffusion GPU picks
* Reset per-model GPU knobs before applying saved settings
* Address review comments
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Guard manual tensor splits and keep remembered context on auto-load
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Snapshot compare knobs, seed splits from free VRAM, flag zero-offload loads
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Exempt CPU-only loads from the guard floor and harden compare and reseed paths
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Reach full offload from the layers slider and charge extras drafters in the guard
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Warm the GPU device cache before pick reconciles and disable staged GPU controls
* Align the training guard with inherited extras, spec mode, and compare targets
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Hide GPUs from companion-less zero-offload loads
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Size diffusion picks per device, own manual offload flags, reject XPU picks
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Drop tensor flags at zero layers and exempt CPU-pinned drafters
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Allowlist the zero-layer tensor parallel drop site
* Keep validate and load guards on the same extras and refresh stale baselines
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Drop mismatched manual tensor splits before launch
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Gate XPU picks on the real backend field and harden split and hydration paths
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Weight full GPUs as zero, clamp split shares, and refine the zero-layer mask gate
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Carry fit context across mode changes and align drafter and picker gates
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Catch variant switches, uncached diffusion repos, and text-only mmproj skips
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Check companions on the first device and size native and remote zero-layer loads
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Replace the training guard's precise VRAM modeling with a conservative bound
* Baseline context pins on non-GGUF hydration and reprobe list-seeded staged GGUFs
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Size manual splits by their largest share and preserve resolved context from Default
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Default-deny unsized required companions and price KV at the effective cache dtype
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Reserve MTP draft KV and MLA target-copy in the training guard
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Size tensor-parallel loads per device and show GPU controls for native GGUFs
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Reserve MTP overhead for uncached remote GGUFs and the mmproj runtime factor
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Drop the training-coexistence VRAM estimation this PR added
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Gate remembered load settings to GGUF picks
* Lock the remaining load-time controls during a staged load
* Clear the stale native-path token on compare loads
* Drop a stale guard reference from the zero-offload masking comment
* Seed GPU baselines from the rollback response and drop never-emitted offload flags
* Match validate's training guard to load and keep the native reload token
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Trim verbose GPU-memory comments
* Thread the variants header walk off the event loop, honor device pins on zero-offload, and hold staged GPU edits
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Honor manual placement and classify pinned zero-offload loads
* Close diffusion admission and status hydration gaps
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Check the actual diffusion GPU during training
* Align staged baselines and manual reload dedupe
* Fix GGUF placement and rollback state
* Harden manual GGUF placement boundaries
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Remove unused resolve_tensor_parallel import in llama_cpp.py
The name is used only in llama_server_args.py, routes/inference.py, and tests,
not in llama_cpp.py; the unused hoisted import trips the import-hoist verifier
in the source-lint CI job.
* Fix diffusion GPU dedup and training guard for non-numeric device tokens
The diffusion runner drives only its single lowest device and the backend
records that one device (self._gpu_ids = [sorted(gpu_ids)[0]]), but the reload
dedupe compared it against the full requested list, so a multi-GPU pick that
resolves to the same device forced a needless reload. Normalize the request the
same way for a loaded diffusion model in both _already_in_target_state and the
route _request_matches_loaded_settings.
The chat-during-training coexistence guard called int() on the single-device
token and hard-rejected when it could not parse. A non-numeric token (a CUDA
UUID / MIG handle) now sizes against the whole visible pool like the GGUF guard
instead of falsely blocking the load, and an empty token (a CPU-only runner such
as a CPU diffusion GGUF) is allowed outright since it uses no GPU VRAM.
* Tighten comments added by the GPU memory config changes
* Harden GGUF placement from independent review: VRAM sizing, diffusion TP reset, tensor_split validation
- Training coexistence guard: a single-device runner pinned through an
unresolvable UUID/MIG token was sized against the aggregate visible-VRAM pool,
so a load could pass on capacity it cannot use and then OOM active training.
Size against the worst-case visible device (min free) instead, keeping the
guard's documented default-deny contract. The empty-token (CPU-only runner)
allow path is unchanged.
- Diffusion startup: _start_diffusion_server now resets self._tensor_parallel to
False alongside the other placement resets. A prior tensor-parallel chat load
(process killed but not fully unload-reset) otherwise left /status misreporting
tensor parallelism and made an identical diffusion re-Apply reload against the
stale state.
- tensor_split: reject negative / non-finite / all-zero splits up front. They
were dropped at launch but still compared raw in the reload dedupe, so an
identical Apply reloaded indefinitely.
- Tests: the shared httpx stub was incomplete and, installed via setdefault
before real httpx loaded, broke a combined pytest run (collection errors on
httpx.Response). Import the real installed httpx instead.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <unslothshared@gmail.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
* Replace standalone Studio wording with Unsloth
Replace the single word Studio with Unsloth wherever it is used as
shorthand for Unsloth Studio in docs, CLI output, UI strings, i18n
locales, workflow display names, comments and docstrings.
Kept unchanged: the full name Unsloth Studio, third party product
names (LM Studio, Visual Studio, Mac Studio), feature names
(Recipe Studio, Fine-tuning Studio and its translations), and all
identifiers such as env vars, commands, paths and filenames.
* Address review feedback on the Studio wording rename
Use "an" before Unsloth where the rename left the article as "a".
Restore the split brand where Unsloth and Studio render as two halves
of the full product name: the onboarding sidebar subtitle and the
IPv6 localhost warning. Scope two messages to the full name Unsloth
Studio where plain Unsloth was misleading: the AMD README bullet and
the CLI studio setup error.
* feat(studio): expose opt-in MCP control plane
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Harden Studio MCP tools: byte-safe auth, page clamping, forward export/checkpoint fields
Follow-up hardening on the opt-in MCP control plane. All changes are additive
and backwards compatible.
- BearerTokenMiddleware now compares the Authorization header on raw bytes.
A non-ASCII bearer value previously reached str-based hmac.compare_digest,
which raises TypeError and surfaced as a 500 instead of a clean 401. The
constructor also rejects an empty or whitespace-only token so an empty token
can never match an empty "Bearer " header.
- MCP tools call the route functions directly, which skips FastAPI Query
validation. list_training_runs and get_recipe_job_dataset now clamp limit and
offset to the same bounds the HTTP routes enforce (a negative SQLite LIMIT
otherwise means "no limit").
- export_gguf forwards hf_token (the backend rejects a Hub upload without it),
accepts a list of quantization methods, and exposes imatrix / imatrix_path so
the IQ low-bit quants are reachable.
- load_checkpoint forwards hf_token and approved_remote_code_fingerprint so
gated checkpoints and the remote-code approval retry work. Its docstring is
corrected: the export backend coexists with training and inference rather than
freeing GPU work.
- start_training passes via_api_key=False explicitly instead of relying on the
unfilled Depends default.
Tests: add coverage for non-ASCII and empty-token auth, the correct-token pass
through, non-http scope pass through, pagination clamping, and the forwarded
export/checkpoint fields.
* Harden Studio MCP: cap /mcp request bodies, reject unusable tokens, fix docs
Follow-up hardening from a full review pass. All changes are additive and
backwards compatible.
- Add "/mcp" to _BODY_PROTECTED_PREFIXES so MaxBodyMiddleware enforces the same
request-body cap it already applies to every other write endpoint (/api/train,
/api/export, /api/data-recipe, ...). The MCP endpoint accepts authenticated
POST tool-call bodies; without this an authenticated client could send an
unbounded body. The middleware only buffers the request body (not the SSE
response), so streaming is unaffected, and the 500MB default cap never affects
a real JSON-RPC tool call (verified live).
- Reject a non-ASCII UNSLOTH_STUDIO_MCP_TOKEN at construction. HTTP header values
are ASCII, so a non-ASCII token cannot be sent by a standard client and would
silently lock out the endpoint; fail fast instead.
- MCP.md: document the canonical /mcp/ endpoint and note that /mcp redirects to
it, so clients that do not follow redirected POSTs still connect.
Tests: add non-ASCII token rejection coverage.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten MCP server comments
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <unslothshared@gmail.com>
* refactor(studio): move chat model picker into features/model-picker
Relocate model-selector + its support files from components/assistant-ui
into a self-contained features/model-picker feature (own barrel), mirroring
the modular Hub layout. Pure move + import repoint; no behaviour change.
* feat(model-picker): add per-model config persistence layer
Superset PerModelConfig (customContextLength, kvCacheDtype, speculativeType,
specDraftNMax, tensorParallel, chatTemplateOverride, trustRemoteCode) persisted
to localStorage (unsloth_model_configs) with schema versioning + LRU budget.
KV-dtype and speculative value sets match main's sidebar (no q4_0/ngram-simple).
Reuses features/hub/lib/model-identity for normalization; adds storage-key layer
and applyPerModelConfigToRuntime (sets tensorParallel, which the old PR omitted).
* feat(picker): modular backend for chat-template validate + default fetch
New studio/backend/picker package (schemas/service/routes) mounted at /api/picker:
- POST /api/picker/validate-chat-template (Jinja syntax validation, no false positives)
- GET /api/picker/chat-template/{model_name} (default template from tokenizer_config.json,
reusing get_cache_path/resolve_cached_repo_id_case; graceful null, no model-code exec)
Frontend api/templates.ts client + hooks/use-model-defaults lazy cache. No backend
changes to the existing inference load route (per-model load fields already supported).
* feat(model-picker): bind picker on-device list to shared hub inventory
Picker now sources cached + local models from useHubInventory (the Hub's shared
store) via a thin adapter, replacing its own /api/models/* fetchers + module
caches. Hub, download manager, and picker now share one source of truth, so
completed downloads reflect in the picker automatically. Partial/live-download
rows are filtered from the cached lists (unchanged rendering). Local naming/search
preserved via additive LocalInventoryRow modelId/displayName. Variant expander,
scan-folder management, recommended-fit, search, external providers untouched.
Known minor: cached 'Downloaded date' sort tiebreak degrades to alphabetical
(hub cached rows carry no mtime); default 'recent' (load-time) sort preserved.
* feat(model-picker): per-model config step inside the picker
Picking a (non-external) model now opens an in-picker config view built from
main's current load controls (context length, KV cache dtype, speculative
decoding, draft tokens, tensor parallel) plus a chat-template editor backed by
the picker validate/default endpoints. 'Remember for this model' persists the
config per model+variant; Run forwards the config to the existing load flow via
meta.config. External models bypass the step. Two-view orchestration lives in
model-selector (single interception point); pickers.tsx call sites untouched.
trustRemoteCode dropped from PerModelConfig to preserve main's per-load consent.
* feat(chat): apply/persist per-model config through the load flow
handleCheckpointChange threads meta.config into the selection; stageOrLoad and
the autoload/Hub-run paths now apply the picker config (explicit pick or saved
remembered config) via applyPerModelConfigToRuntime before staging/loading, with
keepSpeculative set so a remembered speculative mode survives the model switch.
Replaces the old remembered-load-settings seeding (resolveInitialConfig now the
single source). SelectedModelInput carries config.
* refactor(chat): remove per-model load config from the right sidebar
The load knobs (context, KV cache, speculative, draft tokens, tensor parallel)
and the chat-template editor now live only in the picker config step. The sheet's
Model section keeps the staged Load/Cancel flow (config is applied at pick time);
sampling params, system prompt, and RAG are unchanged. Deletes the superseded
remembered-load-settings module + the store's applyRememberedLoadSettings action,
removes the now-dead sheet state/imports, and points the settings reset at
unsloth_model_configs. Delete-cleanup deferred (stale config is LRU-capped).
* fix(model-picker): remove leftover sidebar-staging cogwheel + empty Model section
The downloaded-variant gear (ModelLoadSettingsAction) staged a model straight
into the right-sidebar Run-settings flow -- the old 'configure before load' path
now fully replaced by the in-picker config step. Removed the gear + its component.
Also gate the sheet's 'Model' section to staged picks only (pendingSelection):
after the load-knob strip its content is staged-only, so it was rendering an
empty section header whenever a model was merely loaded.
* chore(chat): remove dead per-model-config setters + modelControlsDisabled
After the load-config UI moved into the picker, the store's per-model setters
(setKvCacheDtype/setSpeculativeType/setSpecDraftNMax/setTensorParallel/
setCustomContextLength/setChatTemplateOverride) had zero callers
(applyPerModelConfigToRuntime writes via setState), and the sheet's
modelControlsDisabled was unreferenced. Verified dead across the whole tree.
* fix(chat): config-step Load actually loads (ignore Load-on-selection)
Root cause: with Settings > Chat > 'Load on selection' turned OFF, the config
step's load went down the deferred-staging path -- opening the right sidebar with
'<model> is staged, not loaded yet / Choose Load model'. The in-picker config step
IS the deliberate load action, so its Load now loads immediately (or downloads +
auto-loads when not cached) regardless of the toggle. Renamed the button
'Run model' -> 'Load model' to match. Native/dropped picks still honor the toggle.
* refactor(chat,hub): retire 'Load on selection' — config step is the only load flow
The in-picker config step (and the Hub Run button) now fully supersede the old
stage-to-sidebar flow, so the Load-on-selection toggle is removed everywhere:
- chat stageOrLoad: every pick loads immediately, or downloads + auto-loads when
not cached (the previous default behaviour, now universal).
- hub Run: drops the stage branch; downloaded GGUFs load directly with their saved
per-model config (no collision with the chat config step — both end at selectModel).
- store: removed loadOnSelection field/setter/key/default; Settings>Chat toggle and
its settings-reset entry removed.
- staged sidebar section is now a download-progress view (auto-loads on completion).
No manual staging remains; stageModel is used only for background auto-load downloads.
* feat(model-picker): default chat template from GGUF + thread variant through config flow
Read the embedded tokenizer.chat_template from GGUF files (read_gguf_chat_template
in gguf_metadata) and use it as the per-model default. Plumb gguf_variant through
the picker service, /api/picker/chat-template route, frontend templates API, and
use-model-defaults so the right variant's template is fetched.
Also refine the picker config-page/model-selector wiring, drop the dead
ggufNativeContextLength runtime path, and add the per-model-config storage keys to
the settings prefs export.
* feat(model-picker): read safetensors chat template + hide editor where it has no effect
Resolve the default chat template for safetensors models: prefer the modern
chat_template.jinja, fall back to the tokenizer_config.json chat_template field,
then chat_template.json (multimodal processor), then the GGUF embedded template.
Applied to local dirs, the HF cache snapshot scan, and the HF remote fetch.
Hide the chat-template editor in the picker for safetensors models — the override
is only applied at load by the GGUF/llama.cpp backend, so editing it on safetensors
currently has no effect. GGUF keeps the editor. Nothing removed; the dialog stays
for when the safetensors apply path is wired up in a later branch.
* fix(model-picker): set legacy-migration flag only after the write succeeds
Set unsloth_model_configs_migrated only once writeMap confirms the migrated
map persisted, so a quota/storage failure no longer marks migration done and
silently drops the user's pre-existing remembered settings — the next load retries.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* MVP model picker fixes
* MVP picker config fix
* MVP safetensors config
* MVP max seq config
* MVP max seq fix
* Fix static max tokens cap ignoring model context
* Fix picker GGUF scan parity
* fix(studio): harden model picker config loading
Apply remembered per-model configs consistently from picker and Hub loads, keep default configs from overriding standing speculative settings, add config access for direct local GGUF files, and support saving or forgetting active model settings without a reload.
* Fix model picker config flow
* Fix model picker config loads
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Avoid recursive per-model config migration reads
* Apply the displayed context length when loading a GGUF
* Fix template validation, cached template lookup, and failed load rollback
- Validate chat templates with the loopcontrols extension so templates
that use break or continue tags pass the picker validator, matching the
inference renderer that already accepts them.
- Read the default chat template from the newest cache snapshot rather than
an arbitrary iterdir order, so an older cached revision no longer prefills
a stale template.
- Capture the runtime per-model config before a load and reapply it when the
load fails, so a failed switch leaves the active model context, KV cache,
template, and speculative settings as they were.
* Make chat template view only for safetensors models
Custom chat template overrides are applied at inference only for GGUF
models, which pass the template to llama-server. The safetensors backend
renders with the model built-in template and ignores the override, so
editing it would save a value that never loads. For safetensors the
config page now opens the template as a read-only preview with a note
that editing is not available yet. This can become editable once
inference support for custom safetensors templates lands in main.
* Fix model picker config edge cases
- Restore prior runtime config when a load no-ops for the active model
- Cap the picker validator request body via the protected prefixes
- Keep the GGUF context slider max above the loaded context
- Fetch subfolder chat templates for uncached Hub repos
- Show the compare side config when reopening the picker
* Keep saved GGUF context above the fallback ceiling
* Show the model config in the run settings sidebar
* Fix model config sidebar reset and context slider
- Stack the remember toggle and action buttons in the sidebar
- Reset the config to defaults instead of the loaded values
- Fetch the native context so the slider max is not the loaded value
* Fix model picker config and download regressions
- Run picker chat template routes off the event loop
- Depth and root guard local template directory scans
- Restore download manager flow for uncached hub picks
- Apply per model context length on reload
- Import model picker symbols from the feature barrel
* Fix model picker config and cached download sorting
- Restore load settings when a Hub run is rejected mid load
- Reuse one NumericValueInput instead of a duplicate copy
- Fix double decode of the model name in the template route
- Remove the unused reset-to-loaded settings action
- Fix cached model download sorting
* Fix model picker per-model config edge cases
Honor a saved or typed max seq length above the model's native context so
RoPE extended values are no longer clamped and silently overwritten. Allow
typing past native while the slider keeps native as a soft ceiling.
Guard the fetch success paths in use-model-defaults against an aborted
signal, and refetch when the HF token changes.
Hash the chat template content in the sidebar remount key instead of its
length. Enable reset for a GGUF whose native context is unknown, and floor
the context slider max so it can never fall below the min.
* Fix GGUF context auto-fit and gated model config token
Stop forcing a 32768 context when a GGUF native context is unknown so the backend auto-fits to VRAM again, while still honoring an explicit context edit.
Send the HF token as a query param so gated safetensors models resolve their max position embeddings.
Derive model default state during render to drop the set-state-in-effect calls.
* Fix native GGUF context ceiling and guard picker template reads
Restore the native context store field so the sidebar slider keeps the
full ceiling for drag and drop GGUFs. Limit local chat template reads to
the browse allowlist, skip malformed repo ids, and drop unused model
picker exports.
* Fix model picker lint boundaries
* Fix model picker review findings
Chat template editor never seeded its draft. Radix only calls onOpenChange
from internal events, so the seed in the nextOpen branch was dead and a model
with a saved override opened empty. Saving then cleared the override. Drop the
dead branch, treat draft as an untouched sentinel, and reset it on every close.
Uncached Hub picks could auto load a model after the user left the chat. Main
detached the staged pick on route exit and on chat context change. Carry the
context key on the pending pick and skip the load when it no longer matches.
Also clear configTarget when the picker closes, restore the onUpdated ref so
variant rows stop resubscribing on every parent render, skip the LRU write when
the entry is already most recent, import NumericValueInput relatively, and drop
the unused ModelUpdateAction barrel export.
* Preserve GGUF context on active reload
* Fix model picker per-model config regressions
- Stop reloading the already loaded model on re-pick
- Hide infra models from the chat picker
- Detect vision support on cached GGUF repos
- Honor saved maxSeqLength on auto load
- Restore default chat template for local GGUFs
- Warn on save failure and revert config on cancel
- Refetch picker inventory on open
- Persist read only per model config safely
* Fix stale model auto load
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix model picker numeric input sizing and constraints
Size value inputs to their content so long context lengths are not clipped,
restrict them to numeric characters, and stop the speculative decoding label
from truncating in the sidebar.
* Fix picker CI tests and harden chat template resolution for PR #6647
- tests: point the descender guard at the moved model-selector.tsx path
- tests: exclude the disabled Reload model button from the regenerate locator so .first targets the real Regenerate
- picker/service.py: reject symlinked template/gguf leaves that resolve outside the browse allowlist (HF cache reads unchanged)
- compare mode: resolve each pane's own remembered chat template instead of inheriting the other pane's from the store
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Protect future-schema per-model configs from deletion for PR #6647
savePerModelConfig already refuses to overwrite a stored config whose schema version is newer than this client understands, but deletePerModelConfig did not. Unchecking Remember on an older client therefore silently destroyed a newer client's saved config. Apply the same guard on delete and surface the blocked case through the existing saveFailed toast.
* Protect future-schema per-model configs from quota eviction for PR #6647
The save and delete guards already refuse to touch a stored config whose schema version is newer than this client understands, but the quota-eviction path did not, so a full store on an older client could still evict a newer client's config. Skip future-schema entries when evicting and fail the save if the budget cannot be met without them.
* Fix GGUF context persistence, compare context, and rollback settings for PR #6647
Persist a GGUF context override from the user's intent instead of collapsing it against the loaded context, which reintroduced the context-reset (f4838782cb reverted the native-baseline fix). model-config-page now collapses the saved value against native, and use-chat-model-runtime and chat-adapter retain the requested context on load so re-saving another setting keeps the override; a null request stays null so a VRAM auto-fit never becomes a stored override.
shared-composer: a compare pane with no explicit GGUF context now loads at native (0) like single-view, not the session maxSeqLength that silently shrank the shown context.
use-chat-model-runtime: restore the previous model's KV cache dtype and chat template on a failed-load rollback so it runs as it was, not with backend defaults.
* Preserve native path token when reloading the active model for PR #6647
handleReloadActiveModel rebuilt the selection without the store's activeNativePathToken, so reloading a file-picked GGUF after a settings change validated the display label as a repo/path and failed. Thread the active native token through the reload selection so native-loaded models reopen correctly.
* Make picker template validation resilient and accept HF generation tags for PR #6647
Import Jinja lazily inside validate_chat_template so a backend without the optional jinja2 package (GGUF-only installs) still starts instead of raising ModuleNotFoundError at import time. Register a no-op extension for the Transformers {% generation %} assistant-mask tag so pasting a valid HF chat template validates, matching the renderer, rather than being rejected as an unknown tag.
* Honor remembered compare config and parse processor chat_template.json for PR #6647
* Fix failed-load rollback context and processor template map fallback for PR #6647
* Restore speculative decoding config on failed-switch rollback
When a model switch fails after the previous model was unloaded, the
rollback reload restored tensor_parallel, KV cache dtype and the chat
template override, but omitted speculative_type and spec_draft_n_max and
cleared their loaded shadows to null. The previous model therefore came
back running at backend defaults (speculation off) while the UI still
showed it enabled, and the status resync confirmed the off state. Resend
the previous model's speculative settings in the rollback load and keep
the store's active and loaded speculative fields in sync with them.
* Reset max sequence length when a model has no saved config
applyPerModelConfigToRuntime reset every per-model field except
maxSeqLength, which it only wrote when the incoming config had one.
maxSeqLength is the sole field carried on store.params, so selecting a
model with no remembered config left the previous model's value in place
and later loaded the new model at that leaked length. Fall back to the
standing default so an unremembered model loads at its own default.
* Surface a message when a variant update cannot start
startManagedUpdate handled the conflict and error start outcomes but let
busy fall through as if the update began, so the confirm dialog closed
with no job created and the cached variant stayed stale. Show an info
message when the repo is busy with a sibling transfer so the click is
not silently dropped.
* Keep per-model speculative choices out of the global default
A staged load with a per-model or one-off config sets keepSpeculative,
which already skips reading the global speculative preference. The
matching save still ran unconditionally, so the model-specific choice was
written to the global unsloth_chat_speculative_type and a later model with
no saved config started from it instead of Auto. Skip saveSpeculativeType
when keepSpeculative so the per-model choice stays isolated.
* Seed non-active model settings from the app default max length
The Run settings page captured initialMaxSeqLength from the loaded
model's runtime params and fell back to it for a model with no saved
config. Opening settings for a different, unloaded model and clicking
Load then sent the active model's context (for example 64k) instead of
the 4096 default, risking validation failures or OOMs. Seed the default
for non-active models and keep the runtime value only for the active one.
* Prefer sidecar tokenizer chat template over the GGUF copy for variants
_chat_template_from_dir returned the embedded GGUF template first when a
variant was selected, reversing the tokenizer-first precedence of the
no-variant path. A model whose chat_template.jinja or tokenizer_config.json
supersedes a stale embedded template then got the wrong template on
variant selection. Keep tokenizer files first regardless of variant; the
variant only picks which GGUF is the fallback. Adds regression tests for
both the tokenizer-wins and gguf-fallback cases.
* Keep per-model speculative choices load-local in autoload and compare
The interactive load path treats a per-model speculative choice as
load-local and skips writing it to the global default. Autoload and
generalized compare still called saveSpeculativeType unconditionally, so a
remembered off or ngram setting leaked into unsloth_chat_speculative_type
and later models with no saved config inherited it. Persist the global
preference only when the value came from the global settings.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: record the compare pane's loaded context in runtime state so the active model's settings and any reload or save use it, not the previous context
* Studio: notify the user when a Hub autoload can't start because another download for the model is already running, instead of silently dropping it
* Studio: drop the merge's orphaned staged-model store helpers and unused alert imports
The main merge left isPendingGguf and pendingSelectionMatches referencing the
removed PendingModelSelection type, and the alert-dialog/alert imports unused
after the permission-mode dropdown replaced the bypass dialog, so tsc -b failed.
* Studio: cache a null default chat template so the viewer stops re-fetching it
A model with no sidecar or embedded template resolves to a terminal null, but
that result was never cached, so reopening the template viewer re-ran the
backend and Hugging Face lookup every time.
* Studio: detect direct-file GGUFs in run settings so Max Tokens uses their context
A GGUF loaded from a local file or custom folder has no variant label, so the
run-settings panel treated it as non-GGUF and clamped Max Tokens to the session
max_seq_length instead of the loaded GGUF context. Detect it via the reported
GGUF context and the .gguf checkpoint suffix, matching the chat page.
* Studio: prompt to re-select a local model file when its lease expired before reload
A file-picked GGUF is reachable only through a native path token that the
desktop host prunes after a TTL. Reloading reused that token blindly, so a
reload long after the initial load failed with an opaque error. Track the
token's expiry and, when it has passed, ask the user to re-select the file
instead of attempting a doomed reload.
* Fix descender-clipping test to tolerate sidebar layout utilities
The sidebar account-block div carries layout utilities (min-w-0, flex-1)
between 'flex' and 'flex-col', so the descender-clipping guard's regex,
which required 'flex' immediately followed by 'flex-col', no longer matched
and the test failed to locate the account-block div. Generalize the prefix
to allow intervening flex utilities while still capturing the leading-*
class before the collapsible visibility utility and asserting leading-tight,
so the guard against clipped glyph descenders is fully preserved.
* Harden picker chat-template resolution
Enforce the 64 KiB chat-template contract at the validate endpoint's request
model so a direct caller cannot submit a template far larger than the frontend
allows (MaxBodyMiddleware only bounds the whole request body, not this field);
oversized templates now return a clean 422.
Apply sidecar-over-GGUF template precedence globally across cached snapshots
instead of per snapshot. A repo with multiple cached revisions previously
returned the first snapshot's template, so a newer GGUF-only revision could
win over an older revision's maintained chat_template.jinja sidecar, which
contradicted the documented intent that sidecars supersede the embedded copy.
* Guard per-model config against future-schema and lossy migration
Two forward-compatibility gaps in the versioned per-model config store:
- The load/apply path returned and normalized a stored record without checking
its schema version, so a record written by a newer client was reinterpreted
under the current schema and applied to a live model load, even though save,
delete and eviction all refuse to touch future-schema records. Reject
future-schema records on load too.
- The one-time legacy migration enforced the storage budget without protecting
the entries it had just migrated and set the completion flag unconditionally.
When storage was already full of future-schema records (which are unevictable
by an older client), the migrated entries were the only evictable ones and
could be dropped while migration was still marked complete. Protect the
migrated keys during eviction and only mark migration complete when they
survive, so it retries once space frees up.
* Discard chat-template validation results after the dialog closes
Server-side template validation is async, but closing or cancelling the editor
did not abort it, so a late-arriving valid response still called onSave and
applied a template the user had already dismissed. Track a validation token
that is bumped on close and ignore any validation result whose token is stale.
* Record native lease expiry when loading a picked GGUF from the chip
The pending-native-model chip loaded via stageOrLoad directly, bypassing
loadNativeModelIntent, so activeNativePathExpiresAtMs was never recorded for a
chip-loaded file. A later reload then either skipped the lease-expiry guard
entirely (expiry left null) or compared against a previously loaded file's
stale expiry, so reload could reuse an already-pruned token or wrongly block a
still-valid one. Route the chip through loadNativeModelIntent, which builds the
same selection and records the expiry.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Prefer sidecar template for a directly selected local GGUF file
A direct .gguf file path read its embedded chat template without checking the
parent directory for a maintained sidecar (chat_template.jinja /
tokenizer_config.json), while directory and variant selections already prefer
the sidecar. That let the config editor preview or save a stale embedded
template for the same model depending on how it was selected. Check the parent
directory sidecars first, then fall back to the embedded copy, and cover both
paths with tests.
* Resolve cached chat template per revision, newest first
The earlier change searched every cached snapshot for a sidecar before
considering any snapshot's embedded GGUF template, which let an obsolete sidecar
from an older revision override the newest revision's template. Restore
per-snapshot resolution (newest first): a revision's sidecar still supersedes
its own embedded GGUF copy, but a newer revision is no longer overridden by an
older revision's sidecar.
* Preserve autoload transport conflicts and surface background busy downloads
- When a Hub autoload hits a transport conflict, keep pendingHubAutoLoad bound
instead of clearing it. Clearing it re-keyed the download surface and its
cleanup cancelled the conflict the toast tells the user to resolve, so the
Hub resume affordance was gone the moment it appeared. Return early on
conflict, mirroring the started branch, so resolving it from the Hub still
auto-loads on completion.
- The background-download branch handled started and conflict but silently
dropped a busy outcome, leaving the user with no feedback when a peer variant
of the same repo was already downloading. Surface the same busy toast the
autoload path uses.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
* Studio: make the Cloudflare tunnel opt-in (off by default)
A wildcard bind (`-H 0.0.0.0`) auto-started a public trycloudflare.com
tunnel, so exposing Studio on the LAN also published it to the public
internet. Flip the default so the tunnel is opt-in.
- `--cloudflare` is now tri-state (Optional[bool], default None = off),
mirroring the existing --enable-tools/--disable-tools handling. Pass
--cloudflare to expose a public HTTPS link for a wildcard bind; --secure
still implies the tunnel.
- --secure + --no-cloudflare is still rejected as a contradiction.
- Update the parent-command guard, re-exec forwarding, startup-banner
wording, the colab comment, README, and tests.
* Studio: update installer/setup launch hints for opt-in Cloudflare
The post-install launch hints only mentioned --secure for a public link.
Now that the tunnel is opt-in, clarify that -H 0.0.0.0 exposes the raw
port on the LAN (not a public URL), and surface --cloudflare as the
explicit opt-in for a public HTTPS link (--secure keeps the raw port
private). Applied to install.ps1, install.sh, and studio/setup.sh.
* Studio: address review - keep cloudflare tri-state + harden run re-exec
Two review points from the bots:
- Gemini: keep `cloudflare` as Optional[bool] in run_server instead of
casting None -> False, so the startup banner can distinguish "OFF (default)"
(unset) from "OFF (--no-cloudflare)" (explicit). `_cloudflare_flag` and the
banner branch now carry the tri-state.
- Codex (P1): `unsloth studio run` re-execs the studio venv's console script,
which can be an older build whose --cloudflare defaulted on; omitting the
flag let it re-enable the tunnel. That path now forwards the default polarity
explicitly (--no-cloudflare, or nothing under --secure since --secure implies
the tunnel). The plain `unsloth studio` path runs the same-version in-tree
run.py (resolved via _find_run_py), so it keeps forwarding only an explicit
polarity and still shows the accurate "(default)" banner.
Tests updated for the tri-state banner labels, the None gate cases, and the
new re-exec forwarding.
* Studio: forward --no-cloudflare on plain re-exec too (mixed install)
Codex follow-up: _find_run_py falls back to STUDIO_HOME/.../studio/backend/
run.py when the package copy is absent, so the plain `unsloth studio` re-exec
can land on an older run.py whose --cloudflare defaults on. Forward the default
polarity explicitly there too (--no-cloudflare, or nothing under --secure),
matching the run subcommand. The common in-venv launch skips the re-exec and
still shows the tri-state "(default)" banner.
* Studio: fix launch hint - --cloudflare needs the wildcard bind
Codex P3: the launch hint listed --cloudflare next to the loopback
`unsloth studio -p 8888` command, but the tunnel only starts for wildcard
binds, so `--cloudflare` alone on 127.0.0.1 does nothing. Show
`-H 0.0.0.0 --cloudflare` in the hints (install.ps1, install.sh,
studio/setup.sh) and clarify the same in the README.
* Studio: cross-platform masked terminal password prompt helper
Per-keystroke '*' echo (POSIX termios cbreak / Windows msvcrt.getwch),
backspace editing, Ctrl-C abort, EOF handling, confirmation loop with
re-prompt on mismatch or policy failure. Pure should_prompt gate for the
--secure/--cloudflare exposure paths.
* Studio CLI: force a terminal password change before public tunnel exposure
When a launch will start the Cloudflare tunnel (--secure, or --cloudflare on
a non-api-only wildcard bind) and the admin account still has its seeded
bootstrap password, prompt for a new password in the terminal (masked with
'*', confirmed, re-prompting until valid) before any re-exec or server
exists. The change is committed in the parent so it never crosses argv or
the environment and older studio-venv children see it immediately. Without
a terminal, warn and fall back to the backend bootstrap shutdown timer.
Mirrors backend update_password semantics in one transaction: rehash,
rotate the JWT secret, clear must_change_password, revoke refresh tokens,
drop the desktop secret, then remove the stale credential files.
* Studio: terminal password gate before the public tunnel (backend backstop)
Never publish a trycloudflare URL while the seeded admin password is
active: run_server now runs a terminal password-change gate after the
tunnel decision and strictly before start_studio_tunnel. Interactive
refusal fails closed (shutdown + exit 1, mirroring the secure gate);
without a tty it warns and keeps the bootstrap deadline. Success applies
the same effects as the change-password route (update_password +
revoke_user_refresh_tokens) and drops the stale
app.state.bootstrap_password. MIN_PASSWORD_LENGTH centralised in
auth/storage.py and referenced by the HTTP schema. terminal_prompt.py
carries the pure gate helper (interactive loop stubbed; supplied by the
masked-input module). Also migrates the studio/setup.ps1 launch footer
that still showed the bare wildcard hint.
* README: reconcile remote-access section with opt-in Cloudflare tunnel
* Studio: harden the terminal password gate after review
- run.py: run the gate BEFORE the uvicorn socket binds. On a wildcard
--cloudflare launch the served HTML injects the bootstrap credential
for first login, so a pre-gate listener would hand the default
password to anyone who reaches the raw port while the operator is
still typing. The gate now also seeds the admin row itself (it can
run before lifespan startup).
- Headless launches that nothing would protect now fail closed: the
bootstrap deadline never arms for api-only serving and
UNSLOTH_STUDIO_BOOTSTRAP_TIMEOUT=0 disables it, so warn-and-proceed
would have promised a shutdown that never comes. Both the CLI and the
backend refuse to publish in that case; the ordinary headless path
still warns and relies on the 1h deadline, and no longer auto-fills
the default credential into HTML served on a public URL.
- storage.update_password gains revoke_refresh_tokens to delete the
user's refresh tokens in the SAME transaction as the password commit;
the change-password route and the backend gate use it (a separable
follow-up delete could fail after the commit and leave a stale
refresh token able to mint access tokens under the rotated secret).
- clear_bootstrap_password is best-effort: a locked/undeletable file
must not surface as a failed password change.
- CLI masked reader: disable ISIG like the backend so Ctrl-Z cannot
suspend the process with the shared terminal stuck in no-echo mode;
handle Ctrl-C/Ctrl-Z as characters; treat stream EOF mid-line as an
abort instead of submitting a partial password. Both readers restore
terminal attrs from a SIGTERM/SIGHUP handler since a finally block
cannot run when a default-disposition signal terminates the process.
- Backend reader: decode byte-at-a-time through an incremental UTF-8
decoder so multi-byte characters split across read boundaries are no
longer dropped; isatty checks tolerate closed/None streams.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: persist bootstrap suppression through lifespan startup
The pre-bind password gate nulled app.state.bootstrap_password, but the
FastAPI lifespan runs after it and re-reads the bootstrap password into
app.state on both admin paths, so a headless public launch could still
serve the injected credential in HTML. Carry a persistent
suppress_bootstrap_injection flag that the lifespan honors instead.
Also drop the quoted Tuple annotation on _terminal_password_gate that
tripped the import-hoist lint (the typing import looked unused).
* Studio CLI: keep the pre-exec auth DB private (0700 dir, 0600 db)
On a fresh install the pre-exposure password gate creates auth/ and
auth.db through the CLI before the backend ever runs, and
sqlite3.connect leaves the DB 0644 under a 022 umask. Mirror backend
storage.get_connection's chmod so the committed password hash and JWT
secret are never world-readable, even if the launch aborts before the
backend applies its own modes.
* Tighten pre-exposure password gate comments
* Studio: delete seeded bootstrap password before headless public re-exec
The headless warn-and-proceed path returns with the default admin
password still active, then re-execs a child Studio process. An old
studio-venv child (mixed-version install) predates the pre-bind gate and
its injection-suppress flag, so its lifespan reads .bootstrap_password
and injects the seeded credential into the public HTML for up to the
bootstrap deadline. A CLI-flag handshake cannot fix this uniformly: the
studio run path uses ignore_unknown_options and an old in-venv child
runs in-process, so it would never reject the flag.
Delete the seeded .bootstrap_password file in the parent before re-exec
so a fresh child of any version reads None and never serves it. This
covers both re-exec paths and both child versions. must_change_password
stays set, so the login page still forces a change and the bootstrap
shutdown timer still arms; only the plaintext-on-disk copy is removed.
Recovery is via a terminal-attached run or reset-password. Backend gate
and CLI warnings updated to match.
* Studio: commit the seeded admin before headless public re-exec
The headless-warn path deletes the seeded .bootstrap_password so a
re-exec'd child cannot inject it, but _ensure_cli_default_admin's INSERT
was never committed and rolled back on conn.close(). On a fresh
STUDIO_HOME an old studio-venv child then found no admin, regenerated a
fresh bootstrap password + file, and injected THAT into the public page,
defeating the deletion.
Commit the seeded admin right after _ensure_cli_default_admin so any
re-exec'd child sees the existing account and does not regenerate.
Regression tests cover both re-exec paths on a fresh (unseeded) DB.
* Studio: fail closed when the bootstrap password file cannot be removed
On the headless public path, deleting .bootstrap_password is the
protection against an old re-exec'd child injecting the seeded
credential. If unlink fails (locked file, read-only auth dir) the file
is still on disk, so warning and proceeding would still leak it for the
bootstrap-timeout window. Abort with a clear error instead. Regression
test covers the unlink-failure fail-closed path.
* Studio: hold no-echo for the whole password line, not per keystroke
The POSIX masked reader set cbreak/no-echo inside _getch_posix and restored
the terminal to echo-on in a finally after every single keystroke, because
_read_password calls _getch once per character. Between one char returning and
the next call re-entering cbreak, ECHO was on, so a keystroke arriving in that
window echoed the password in cleartext.
Move the terminal mode into a _prompt_raw_mode context that _read_password
holds around the entire line (mirroring unsloth_cli/commands/_password_prompt.py,
which already did this), restoring once when the line completes or aborts.
_getch_posix now only reads, since the mode is held by the caller. The context
is a no-op when stdin is not a real terminal, keeping the _getch test seam.
Add a regression test asserting the raw-mode context wraps the read exactly
once and every keystroke is read while it is active.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: strip the seeded bootstrap password when the auth DB check fails
The pre-exposure gate returned early on two auth-DB inspection failures and
proceeded to re-exec without removing the seeded .bootstrap_password:
- _connect_auth_db() failure: a seeded credential from a prior run may still
be on disk.
- the must_change_password read-back failure: worse, _ensure_cli_default_admin
had already seeded the admin and the code committed it (writing
.bootstrap_password) right before the failing SELECT.
In the mixed-version case (a new outer CLI re-execing an old studio-venv child
that predates the pre-bind gate), that child would read the file back and
inject the default admin credential into the public Cloudflare page. The
sibling headless branch already deletes the file for exactly this reason, so
these returns were an inconsistent gap.
Factor the delete-or-fail-closed logic into
_strip_seeded_bootstrap_password_or_exit and call it on both inspection
failures (and reuse it in the headless branch): strip the seeded file first
(version-independent protection), failing closed if the removal itself fails.
must_change_password stays set, so the login page still forces a change and the
bootstrap shutdown timer still arms.
Add tests for both new paths (connect failure and post-commit read-back
failure strip the file and proceed; a failed strip fails closed).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: fail closed when the seeded admin cannot be committed before exposure
The pre-exposure gate wrapped _ensure_cli_default_admin (the INSERT), its
conn.commit(), and the must_change_password read-back in one try, and the
except recovered by stripping .bootstrap_password and proceeding to re-exec on
the assumption the admin was already committed. That assumption only holds when
the failing statement is the SELECT. When the INSERT or the commit itself fails
(e.g. a write lock held past the busy timeout on a fresh install), no admin row
is committed: it rolls back on conn.close(), and a re-exec'd old studio-venv
child (no pre-bind gate) then finds no admin, regenerates a fresh bootstrap
password + file, and serves that default credential on the public Cloudflare
page. Stripping the file cannot stop a regeneration.
Split the seed+commit into its own try that fails closed (refuse the public
launch, best-effort removing any half-written seed file) since we cannot prove a
committed admin; keep the separate read-back failure on the strip-and-proceed
path, where the admin is committed so an old child finds it and will not
regenerate. Add a test for the seed-commit-failure path.
* Studio: decode the CLI masked password reader with errors="replace"
The CLI reader read keystrokes with text-mode sys.stdin.read(1), which raises
UnicodeDecodeError on a pasted non-UTF-8 password (e.g. Latin-1 bytes), or under
PYTHONUTF8 yields a lone surrogate that later crashes the pbkdf2 encode -- either
aborts the launch with a traceback. The backend mirror (terminal_prompt.py)
already reads raw bytes through an incremental decoder with errors="replace".
Mirror that here: read with os.read and an incremental decoder so invalid bytes
map to U+FFFD, iterating over each emitted char (one byte can complete a
replacement plus the next char).
* Studio: resolve the child launcher before the pre-exposure gate
The gate strips the seeded .bootstrap_password on a headless public launch, and
it ran before the re-exec launchability check (studio venv / run.py / console
script present). So a headless launch with an incomplete studio setup would seed
the admin, delete the bootstrap password, then abort because the child could not
be found, leaving the admin at must_change_password=1 with no password ever
shown or injectable: locked out until `unsloth studio reset-password`.
Resolve and validate the child launcher first, in both `studio` (studio_default)
and `studio run`, and only then run the gate, so an unlaunchable setup exits
before anything is stripped. Add a regression test that a missing venv exits
without removing the seeded file.
* Studio: fail closed when the auth DB cannot be opened before exposure
The connect-failure branch of the pre-exposure gate stripped .bootstrap_password
and proceeded, on the assumption a committed admin from a prior run made an old
child find it and not regenerate. But on a fresh public launch whose
_connect_auth_db() itself fails (transient lock during the schema/seed step, or
an unwritable home), no admin is committed, so a mixed-version re-exec child that
predates the backend gate can find no user, generate a fresh bootstrap password,
and serve it on the public Cloudflare page. Stripping a file we cannot vouch for
cannot stop a regeneration.
Make this branch fail closed like the seed/commit failure path: we only continue
past the DB inspection once a committed admin is confirmed. The existing file is
left untouched so a retry (after a transient lock clears) can still prompt.
Update the connect-failure test to assert fail-closed, and give the in-venv
--secure flag test a real STUDIO_HOME with an already-changed admin so the gate
is a no-op rather than relying on a DB-open failure.
* Studio: invalidate seeded bootstrap files before deleting auth.db on reset
reset-password deleted auth.db first, then best-effort unlinked the seeded
.bootstrap_password and desktop secret. unlink() only ignores
FileNotFoundError, so a locked or read-only file (Windows AV, read-only auth
dir) survived while auth.db was gone. The next server start then re-seeded
from that stale plaintext and re-validated the exact credential the reset was
meant to revoke.
Invalidate the credential files first, truncating any that cannot be
unlinked, then delete the DB, so a surviving file can never carry a reusable
secret. clear_bootstrap_password now truncates on unlink failure for the same
reason, and its warning says the contents were cleared rather than claiming
the stale password is already invalid.
* Studio: require a servable frontend before the pre-exposure gate can strip the seeded password
A headless public launch strips the seeded .bootstrap_password before the
re-exec'd child starts. If the child then cannot serve the login page (the only
in-band way to change the seeded password) the admin is locked out
(must_change_password=1, no file, no UI) until reset-password.
Add _require_servable_frontend_or_exit and call it before the gate on both
`unsloth studio` and `unsloth studio run` public launches: fail closed if a
non-api-only public launch has no built frontend dist, before anything is
stripped. A user-supplied --frontend is validated to contain index.html so a
bad path cannot silently bypass the check; an auto-resolved dist is trusted
(_find_frontend_dist already requires index.html) and forwarded to the child.
Model-load aborts on `studio run` remain a residual: the parent must strip for
mixed-version safety (an old studio-venv child has no pre-bind gate) and model
loadability cannot be proven before exec, so that path stays recoverable via
reset-password.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: harden reset-password ordering and validate the in-venv backend before the strip
Three follow-ups to the pre-exposure hardening:
reset-password now deletes auth.db FIRST and proves it is gone before touching
the seeded credential files. If the DB cannot be removed (a running Studio or
Windows holds it open, or a read-only auth dir) it aborts with the credential
files untouched, so a forgotten-password reset is not left half-done with the
recovery credentials deleted while an un-resettable must_change_password=1 DB
survives. After the DB is gone it invalidates the stale credential files
(unlink, else truncate) and fails closed if a file can be neither removed nor
truncated, since a surviving plaintext would be re-seeded by
generate_bootstrap_password() and re-validate the revoked password.
The in-venv (in-process) launch path had no analogue of the re-exec launcher
check: a headless public launch would seed the admin and strip the seeded
.bootstrap_password in the gate before _load_run_module() later failed on a
broken/partial venv, leaving must_change_password=1 with no password to log in.
Add _validate_inproc_backend_before_strip, called on the in-venv path (both
`unsloth studio` and `unsloth studio run`) before the gate on the headless
public path, so a broken backend fails cleanly before anything is stripped. It
is scoped to the headless path so an interactive prompt is not delayed behind a
full backend import.
* Studio: validate the frontend and tunnel before the strip on every public path
Five follow-ups closing the remaining pre-exposure-strip lockouts:
The in-venv (in-process) paths of both `unsloth studio` and `unsloth studio
run` validated the backend but not the frontend before the gate, so a headless
public launch with a missing/bad dist would strip the seeded .bootstrap_password
and then abort in run_server() during frontend setup, leaving
must_change_password=1 with no login page. Both now validate a servable frontend
before the strip (cheap check first, backend import after) and serve the
resolved dist in-process.
The `studio run` re-exec discarded the dist that satisfied the pre-strip check
and only forwarded a user-supplied --frontend. In a shadowed install where the
parent finds a built dist the child cannot, it stripped and exec'd without the
path, and the child aborted during frontend setup. It now forwards the resolved
dist, matching `unsloth studio`.
On a headless --secure launch the bind is loopback, so the Cloudflare tunnel is
the only public exposure. If cloudflared is provably unavailable (found nowhere
and undownloadable) the tunnel cannot start, so stripping the recovery
credential would just lock the user out with no public URL ever served. Add
_tunnel_binary_confirmed_unavailable and, on --secure only, refuse the launch
with the credential preserved rather than strip. Wildcard --cloudflare binds
0.0.0.0 publicly regardless of the tunnel, so it still strips; any uncertainty
(helper not loadable) also still strips, since a possible credential leak
outweighs a recoverable lockout.
clear_bootstrap_password no longer claims it cleared the file's contents when
both unlink and truncate failed; it now reports the stale password is still on
disk and asks the user to remove it manually.
* Studio: fix cloudflared probe path and skip the bootstrap strip for a self-suppressing child
Two follow-ups to the --secure pre-exposure hardening:
The cloudflared availability probe loaded cloudflare_tunnel by file path but not
its backend deps: ensure_cloudflared() -> _cache_path() lazily imports
utils.paths.storage_roots, which only resolves when studio/backend is on
sys.path. From the outer CLI it is not, so the probe saw ensure_cloudflared()
return None (cache unresolvable) and wrongly treated the tunnel as unavailable,
refusing --secure even when cloudflared was cached or downloadable. Add the
backend dir to sys.path for the probe (and remove it after) so the cache path
resolves as it will in the child.
A headless --secure launch stripped the seeded .bootstrap_password before the
child proved the tunnel could actually connect, so a cloudflared that is present
but cannot establish the tunnel (blocked connectivity, Cloudflare outage) left
must_change_password=1 with no recovery credential. But the strip is only needed
when the re-exec'd child is an OLD studio-venv backend with no pre-bind
suppression: this install's own run.py sets app.state.suppress_bootstrap_injection
before binding and never serves the seeded credential publicly. Add
_child_self_suppresses (true in-process, or when the re-exec target is this
install's own run.py by path identity) and skip the strip in that case, keeping
.bootstrap_password as a local recovery credential; the strip stays fully in
force for the studio-venv console-script path and any venv-fallback run.py, where
an old child is actually possible.
* Studio: reword the pre-exposure terminal password prompt
* Studio: warn when -H is overridden by --secure; align pre-exposure prompt wording
- --secure/--secure run: emit a Note (not an error) when -H is a non-loopback
host, since --secure forces the loopback bind and would otherwise discard -H
silently.
- Reword the pre-exposure terminal prompt to 'exposed on the public internet'
in both the backend gate and the CLI mirror.
- Align the CLI success line with the backend ("Password updated for '<user>'.").
- Tests for the new -H warning (present when overridden, absent on loopback).
* Studio: add non-interactive --password to set the initial admin password
Headless hosts (CI, containers, systemd units) have no TTY, so the forced
first-exposure password change could not be completed unattended. Add a
non-interactive way to set the INITIAL admin password before the server binds:
- --password <value>, the UNSLOTH_STUDIO_PASSWORD env var, or --password -
(read one line from stdin). Off by default; unset falls back to the normal
interactive terminal prompt / browser setup.
- Applies on any launch (public --secure/--cloudflare or a headless -H 0.0.0.0
bind), only when the account still has its seeded bootstrap password. An
already-set password is a hard error, never an override; an invalid value
(too short, or equal to the bootstrap) fails closed before bind.
- The CLI applies the change in the parent, never forwards --password to the
re-exec child, and strips UNSLOTH_STUDIO_PASSWORD from the child env so the
secret never crosses to the child. run.py does the same on the direct path and
strips the env var so spawned subprocesses (cloudflared, llama-server, tools)
cannot inherit it.
Mirrors resolve_supplied_password across the CLI and backend, documents the
option in the README (including the argv-visibility caveat), and covers all
flows (env/stdin/literal, fail-closed cases, no-forward, env-strip,
reset-password roundtrip) in the CLI, backend, and unit suites.
* Studio: truncate the stale bootstrap file when unlink fails on a CLI password change
The post-change cleanup in _cli_update_password only warned when
.bootstrap_password could not be unlinked but was still writable (locked file,
read-only auth dir), leaving the old plaintext on disk. If auth.db is later
reset or removed, generate_bootstrap_password() reads that file back and
re-validates the revoked bootstrap password. Truncate the file on unlink
failure so its stale plaintext cannot be re-seeded, mirroring the backend
clear_bootstrap_password(); the password change is already committed, so this
never rolls it back. The warning now states truthfully whether the contents
were cleared or the file must be removed manually.
* Studio: tighten comments
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: scope the seeded bootstrap password auto-fill to loopback clients
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: block bootstrap injection through Cloudflare tunnels
* Studio: require loopback host for bootstrap injection
* Studio: add regression test for unparseable Host in bootstrap loopback gate
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Scope bootstrap auto-fill to a direct-loopback client (block proxy/tunnel headers and malformed Host)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Reject scope-id addresses in loopback check (fail closed on ::1%zone Host)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Reject malformed bracketed Host in loopback check (e.g. [::1]evil)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: danielhanchen <23090290+danielhanchen@users.noreply.github.com>
* Studio: startup loading banner and mute the benign bitsandbytes ROCm warning
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: shorten startup banner wording
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: expose full compressed-tensors scheme set in an export formats dropdown
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: multi-select export formats, portable torchao FP8/INT8, GGUF LoRA, source parity
Export page overhaul on top of the formats dropdown:
- Unify merged precision into one sorted multi-select list (16-bit first, then
8-bit, then 4-bit). Drop "vLLM" from labels, add INT8 (W8A8), INT8 (W8A16),
INT4 (W4A16), MXFP4, MXFP8. Quick formats render as toggle pills; the rest live
in a multi-select "More formats" dropdown, so several formats export in one run.
- Add a portable torchao FP8/INT8 save path (Float8WeightOnlyConfig /
Int8WeightOnlyConfig) that needs no NVIDIA GPU to produce and loads in vLLM.
FP8 serializes to safetensors, INT8 to .bin. Wired into save_pretrained_merged
and push_to_hub_merged via a TORCHAO_EXPORT_SCHEMES registry and
_unsloth_save_torchao, parallel to the compressed-tensors path.
- Hide NVIDIA-only compressed-tensors formats when no NVIDIA GPU is present; keep
16-bit and portable FP8/INT8. The backend also rejects a compressed request on
non-NVIDIA hardware so it stays authoritative.
- Relax merged export to non-PEFT models so Local Model and Hugging Face sources
get the same 16-bit / compressed / portable options.
- GGUF: send the whole quant list in one call (merge once, quantize many).
- LoRA: add a GGUF adapter option (convert_lora_to_gguf.py) with an outtype
select (f16/bf16/f32/q8_0/auto), alongside the safetensors adapter.
- Thread the new fields through models, routes, orchestrator, and worker; extend
the export tests.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: gate export by accelerator with a torch-aware reason; fix export save dir naming
Export runs through Unsloth, which requires a compute accelerator (NVIDIA/AMD/Intel
GPU or Apple MLX) and has no CPU code path, so a bare-CPU host cannot export even
with PyTorch installed. Add export_capability() in utils/hardware that reports
export_supported plus a precise reason so the UI stops showing a generic "no GPU":
- pytorch_not_installed: a --no-torch install (even a physical GPU is unusable)
- no_accelerator: PyTorch present but no supported accelerator (bare CPU)
- mlx_unavailable: Apple Silicon where the MLX stack is missing or too old
Expose the fields on /api/system/hardware and /api/system, and guard the mutating
export routes (load-checkpoint, export/merged|base|gguf|lora) with HTTP 400 and the
reason, leaving read-only endpoints usable so the Export page still renders.
Make core/export/export.py import without PyTorch and without a usable accelerator
(the Unsloth import is caught) so the export worker degrades to a clear message
instead of crashing at import.
Frontend: keep /export reachable on chat-only hosts and gray out the method and
format options with the backend reason (Alert plus disabled MethodPicker) instead
of silently redirecting to /chat, so users see why export is unavailable.
Also fix the export save directory producing "model/null" for Local Model and
Hugging Face sources that have no run/checkpoint, naming the folder from the model id.
* CI: validate Studio export capability gating on Linux, Windows and macOS
Add a small pytest matrix that runs studio/backend/tests/test_export_capability.py
on ubuntu-latest, windows-latest and macos-latest. It confirms, on each real OS,
that hardware.export_capability() reports the right decision and reason
(pytorch_not_installed, no_accelerator, or mlx_unavailable) and that the export
backend imports without PyTorch and degrades to a clear message instead of crashing.
Hosted runners have no GPU/MLX, so this covers the "export unavailable, here is why"
path a Mac/Windows user without an accelerator sees; a real accelerator export is
validated separately. The job installs only a CPU PyTorch plus the backend import
deps (no unsloth, triton, or llama.cpp), so it runs in seconds with no GPU.
* Studio export: address Codex review (source-aware gating, GGUF LoRA token/MLX/guard)
Frontend (export-page):
- Gate LoRA and quantized-model restrictions on the active source. isAdapter /
isQuantized come from the selected checkpoint; in Local Model / Hugging Face
("model") source mode they were stale, so LoRA stayed wrongly enabled for a
direct base model (backend then rejects "No adapter to export") and a stale
"quantized" flag disabled every method for an unrelated, exportable model. Add
effectiveIsAdapter / effectiveIsQuantized (false outside checkpoint mode) and use
them in the method-reset effect and the MethodPicker disabled state.
- Hide the GGUF LoRA option on a macOS/MLX host (the backend rejects GGUF LoRA on
MLX), so users no longer pick it, wait through the load, and always fail. Disable
the "GGUF adapter" button on a Mac host and never send loraGguf there.
Backend (core/export/export.py):
- Pass the HF token into the GGUF LoRA conversion (save_pretrained_gguf), so a
gated/private base model's config fetch in convert_lora_to_gguf.py is
authenticated; without it the load can succeed but the conversion fails.
- Guard the save_pretrained_gguf capability check with getattr so an older Unsloth
model that lacks the method returns the clean "not supported" message instead of
an AttributeError that surfaces as a generic 500.
* Studio export: address 2nd Codex review (CI index, empty merged, test import)
- studio-export-capability-ci.yml: add --extra-index-url https://pypi.org/simple to
the torch install so torch's transitive deps still resolve; --index-url alone
replaces PyPI with only the CPU wheel index, which does not serve all of them.
- export-page handleStart: reject an empty merged selection (mirrors canExport), so
clicking the panel's Start button with every precision pill deselected no longer
submits mergedSelections: [] and launches an unintended default 16-bit export.
- test_export_imatrix_compressed: the torchao-registry test now reads unsloth/save.py
as text (like the other ast/string checks) instead of `import unsloth.save`, which
raised ModuleNotFoundError in the CPU studio-backend suite that has no unsloth
installed.
* Studio export: make comments succinct across the export changes
* Studio export: use load token for local GGUF LoRA export of gated bases
* Studio export: harden portable torchao path and gate multi-format Hub push
torchao (_unsloth_save_torchao):
- merge to an isolated temp staging dir so a co-selected 16-bit output at save_directory is not deleted
- narrow VLM detection to vision_config / ForVisionText2Text so T5/BART/Whisper are not misrouted
- forward trust_remote_code (from auto_map) to the reload so custom-code models export
Export UI:
- hide portable torchao formats on macOS/MLX (backend rejects quantized export there)
- restrict a Hub merged export to a single format (each writes to the repo root)
* Studio export: torchao tokenizer remote-code + XPU offload, scale GGUF timeout
torchao (_unsloth_save_torchao):
- honor auto_map in the staged tokenizer/processor configs (not just model.config) when
deriving trust_remote_code, so custom-code tokenizers reload after the merge
- offload single-device XPU models to CPU (and empty the XPU cache) before the reload, matching
the CUDA path, so an Intel GPU that fits the model once does not OOM on the second copy
Export orchestrator:
- scale the GGUF wait timeout by the number of requested quants so a multi-quant list export of a
large model does not time out at a flat 3600s
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio export: show portable torchao formats only on non-NVIDIA (CPU) hosts
Portable torchao FP8/INT8 is the fallback for hosts without the NVIDIA compressed-tensors path.
On an NVIDIA GPU the compressed-tensors FP8/FP4/INT formats are the intended path (llm-compressor
auto-installs), so hide the portable duplicates there; keep them on CPU / non-NVIDIA hosts and
continue hiding them on macOS/MLX.
* Studio export: report all output folders and the exported formats
- Multi-format merged export now collects every sibling output directory (one per selected
precision) instead of only the last; the success banner lists them all.
- Show the selected precision formats in the run summary (a Formats row, like GGUF Quantizations),
so the panel says what is being exported rather than just 'Merged Model'.
- Persist the selected formats in the run summary and seed them on mount, so navigating away and
back (or toggling the export method) restores the selection instead of resetting to 16-bit.
* Studio export: list all output formats, add GGUF LoRA target, default Q8_0, auto-select newest checkpoint
- Progress/summary panel now shows a Formats row with the selected merged
formats, and the success banner lists every output folder a multi-format
merged run creates (one line per format) instead of only the last one.
- Merged format selection is seeded from the active run, so navigating away
and back (or switching method cards) no longer resets it to 16-bit.
- GGUF / Llama.cpp now offers an Export target toggle (Full model or LoRA
adapter) for adapter checkpoints, reusing the LoRA GGUF export path.
- Removed the Auto GGUF LoRA output type and defaulted to Q8_0 in the UI,
the request model, and the backend defaults; the outtype list is now
Q8_0/F16/BF16/F32. Core save.py still accepts auto for external callers.
- When a finetune has no checkpoint selected, auto-select the newest one.
* Studio torchao export: robust reload class + optional VLM import
Two fixes to the portable torchao FP8/INT8 export reload, from review of the
narrowed VLM detection:
- Encoder-decoder seq2seq checkpoints (T5/BART/Whisper) are not causal LMs.
With the narrowed is_vlm test they now correctly skip the image-text class,
but fell through to AutoModelForCausalLM and failed to reload after the merge.
Reload them with their own architecture class from the config instead.
- AutoModelForImageTextToText was imported unconditionally at the top of the
torchao path, so on Transformers builds without that class the import aborted
every torchao export (even text-only). Import it lazily only for a VLM, with
the AutoModelForVision2Seq fallback used elsewhere in Unsloth.
* Studio: enable FP8/FP4 compressed export for newer-transformers models
The shipped llm-compressor 0.10.x pins transformers<=4.57.6, so FP8/FP4 export failed
for models needing a transformers 5.x sidecar (Qwen3.5, Gemma-4, Qwen3-Next): the
quantization subprocess crashed importing the removed TORCH_INIT_FUNCTIONS.
Run the quantization against a dedicated llm-compressor-main "shadow": a --target
package dir (transformers 5.10.2 + llm-compressor main + compressed-tensors) layered
over the existing torch. It installs --no-deps so torch is never touched (works on any
Studio torch build), is provisioned lazily and fingerprint-cached, and can be turned
off with UNSLOTH_DISABLE_LLMCOMPRESSOR_MAIN.
- transformers_version.py: provision + validate .venv_llmcompressor.
- export.py: route all compressed exports through the shadow when available; else keep
the workspace 0.10.x path and fail fast past its transformers ceiling.
- save.py: launch _compressed_quantize.py with a clean PYTHONPATH = shadow.
- _compressed_quantize.py: skip linear_attn / vision tower / MTP modules (matches the
RedHatAI and NVIDIA reference quants, and is required by the grouped schemes).
Verified all four schemes (fp8, w8a8, w4a16, mxfp4) on Qwen3.5-9B and Llama-3.2-1B, and
fp8 on Gemma-4, end to end through Studio.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix GGUF LoRA export tests
* Fix export CI expectations
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: wasimysaid <112766706+wasimysaid@users.noreply.github.com>
Co-authored-by: Wasim Yousef Said <wasimysdev@gmail.com>
* feat: Implementation of the Portuguese (Brazil) language and VRAM/RAM monitor.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update studio/frontend/src/hooks/use-gpu-utilization.ts
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* Update studio/backend/main.py
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* Update studio/backend/main.py
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update studio/backend/utils/hardware/hardware.py
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* Update studio/backend/utils/hardware/hardware.py
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update studio/frontend/src/features/settings/components/usage-examples.tsx
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update studio/backend/main.py
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* Update studio/backend/utils/hardware/hardware.py
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* Update studio/frontend/src/features/studio/sections/progress-section.tsx
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* fix: resolve automated review feedback on API shape
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix review issues for PR #6509: Cpu icon, VRAM percent, system polling
- model-inspector: use the exported CpuIcon (Cpu is not a Hugeicons export)
- app-sidebar: guard the VRAM percent on totalVram to avoid Infinity, and
reset the system poll cache only after each request settles so a slow probe
is reused instead of stacking overlapping requests
- use-gpu-info: populate CPU/RAM on hosts without a GPU
- progress-section: label GPUs by visible_ordinal instead of array index
- hub-page: base the RAM label on systemRamTotalGb
- usage-examples: emit JS sampling and tool options at the top level instead
of nesting them under extra_body (the JS SDK does not unwrap extra_body)
- main: read torch and transformers versions from package metadata instead of
importing the libraries on every system poll, and guard the VRAM math
against null values
- hardware: translate a leftover comment to English
* Harden /api/system: guard psutil.boot_time for PR #6509
Simulating restricted containers and some VMs (where psutil.boot_time can raise)
showed the /api/system endpoint would 500 on the unguarded boot_time call, the
same failure class already handled for cpu_freq, disk_usage, and Process. Wrap
boot_time and return uptime_seconds as null when it is unavailable so the sidebar
monitor degrades gracefully instead of breaking. Widen the uptime_seconds type to
number | null to match.
* Studio: make the sidebar hardware monitor a toggle (default on) for PR #6509
Adds a "Show hardware monitor" switch under Settings > Appearance > Layout,
backed by a localStorage preference (default on), mirroring the existing
useSidebarPin pattern. When turned off, the sidebar hides the VRAM/RAM meters
and useSystemInfo stops the 3s /api/system poll entirely, so no nvidia-smi /
SMI probes run while the monitor is disabled. Adds the en and pt-BR strings.
* Studio: default the sidebar hardware monitor to off (opt-in) for PR #6509
* Studio pt-BR: fix three small translation defects for PR #6509
- learningRateDescription: "5e-5 for CPT" -> "5e-5 para CPT" (leftover English)
- exportScopeRecents: "Recents" -> "Recentes" (untranslated)
- relativeMonthsAgo/relativeYearsAgo: add the missing space ("há {count} meses"/
"há {count} anos") so they no longer render as "há 3meses"
* Studio pt-BR: translate the last 10 fallback keys for PR #6509
Adds the settings.general.storage block (Armazenamento) and the
settings.chat.modelDisclaimer pair, so pt-BR now covers all en keys
(679/679) with no English fallbacks.
* Studio: hide sidebar VRAM row on CPU-only hosts for PR #6509
* Studio: tighten and trim code comments for PR #6509
* fix: UI issue in the stop button dialog box (fine-tuning)
* Studio pt-BR: translate 18 new keys from main merge (password dialog, GGUF export, dataset streaming) for PR #6509
* Rounding to GB
* Fix/adjust System resources tab for PR #6509
* Fix/adjust GPU monitor review items for PR #6509
* Fix/adjust remaining GPU monitor review items for PR #6509
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix/adjust MLX resource fallback for PR #6509
* floating window implementation
* resize for floating window
* Fix resource monitor review items
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Restore frontend optional dependency lock entries
* Make GPU selection tests hermetic
* Fix GPU monitor CI test failures
* Bound MLX GGUF reload smoke
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix MLX GGUF reload smoke exit
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
* Studio: opt-in OpenAI /v1 model auto-switch and idle keep-warm
The OpenAI-compatible endpoints serve whichever GGUF is loaded and ignore the
request model field, so an OpenAI client that changes model never reloads. Add
an opt-in setting that, when a /v1 request names a downloaded local GGUF
different from the loaded one, loads it before serving by reusing the existing
/load path (its dedup, tensor fallback, and threading apply). Unknown names
still serve the loaded model, so drop-in compatibility is preserved and no
remote download is triggered.
Also add an optional idle auto-unload (TTL keep-warm): a pure-ASGI middleware
tracks in-flight inference requests so a stream is never unloaded mid-response,
and a lifespan loop unloads the model after the configured idle seconds. Both
settings default off and live in the app_settings store, exposed via
GET/PUT /api/settings/openai-auto-switch.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: variant-aware auto-switch, /v1/responses coverage, keep-warm load stamp
Follow-ups from review of the opt-in OpenAI auto-switch path:
1. Variant-aware dedup. _maybe_auto_switch_model compared only the repo id, so
requesting another quant of the loaded repo (e.g. Q4_K_M loaded, Q8_0 asked)
was served by the old quant. Compare hf_variant too, matching /load dedup.
2. Streaming /v1/responses now calls the auto-switch hook. It went straight into
_responses_stream and only checked is_loaded, so stream=True could serve the
old model or 400. Non-streaming already routed through chat completions; the
hook is idempotent once loaded.
3. resolve_local_gguf tries an exact id match before splitting a trailing
:VARIANT, so local ids that contain a colon (e.g. a Windows path) resolve
instead of being cut at the drive letter.
4. Idle keep-warm stamps activity on a load/swap transition. _last_active was
only refreshed by inference requests, so a model loaded after the server sat
idle past the TTL could be unloaded before its first request.
Tests cover each case.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: make the /v1/responses auto-switch test order-independent
The new streaming-responses test passed in isolation but failed under the CI's
randomized collection order with "object has no attribute 'state'": it passed a
bare object() as the request and stubbed only one dispatcher, so an ordering
where the real dispatcher ran hit request.state. Give the request a state and
stub both dispatchers; the test still asserts the hook fires before dispatch.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: assert /v1/responses auto-switch wiring on source, not at runtime
The behavioral version executed openai_responses and relied on stubbing its
callees, which a randomized collection order in CI could defeat (the real
dispatcher ran and hit request attributes). Assert on the function source that
the hook precedes both dispatchers instead; the hook's runtime behavior is
already covered by the direct _maybe_auto_switch_model tests.
* Studio: auto-switch on /v1/embeddings, GGUF-only targets, idle-unload race gate
Second-pass review follow-ups on the opt-in auto-switch path:
1. /v1/embeddings now calls the auto-switch hook before the loaded-state check,
matching the other model-bearing OpenAI endpoints (the keep-warm middleware
already treats embeddings as inference).
2. The resolver index is now GGUF-only. The local-model scanners also surface
Transformers/safetensors repos; without a filter, auto-switch could unload
the GGUF and route a request into the non-GGUF loader. _has_local_gguf checks
a direct .gguf, a models-dir folder, and the HF-cache snapshots layout.
3. Idle keep-warm now holds an asyncio gate across the idle check and the
unload, and a request bumps inflight under the same gate, so the loop can no
longer unload in the window between "looks idle" and the kill.
Tests cover each. Broader local-model source parity (LM Studio, Ollama, legacy
caches, custom scan folders) is a follow-up; missing one of those today just
falls through to the loaded model.
* Studio: variant-aware local resolver, count_tokens + audio auto-switch coverage
Third-pass review follow-ups on the opt-in auto-switch path:
1. The resolver is now variant-aware via list_local_gguf_variants. It indexes
only the quants actually on disk, recursing snapshots and quant subdirs such
as the nested per-quant folders, so a requested repo:VARIANT resolves only
when that quant is local and a bare repo resolves to a concrete local quant.
This fixes two gaps: the previous shallow glob rejected nested-variant GGUF
repos, and a request for an uncached quant could send /load down the remote
download path, breaking the local-only contract.
2. /v1/messages/count_tokens now auto-switches like its sibling /v1/messages, so
a count uses the requested model's tokenizer.
3. /api/inference/audio/generate (direct GGUF TTS) is now tracked as in-flight
inference, so the idle loop cannot unload the model mid-generation.
Tests cover each. Two reviewer items are left as follow-ups: indexing the
remaining local sources (LM Studio, Ollama, legacy/default caches, custom scan
folders), which fails safe today by falling through to the loaded model; and
fully serializing concurrent different-model requests, an inherent limit of the
single-slot llama backend that the opt-in feature is not designed around.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: make local GGUF resolver fail-safe so a bad model name cannot 500
The auto-switch hook calls resolve_local_gguf without its own guard, and
/v1/completions and /v1/embeddings pass body.get("model") through unchanged.
A non-string model (e.g. {"model": 123}) or any internal scan failure would
then raise out of the resolver and turn a request that would otherwise be
served by the loaded model into a 500, breaking the drop-in compatibility the
feature is built on.
Guard the resolver at its boundary: reject non-string input up front and wrap
the lookup so any failure returns None (fall through to the loaded model).
Add regression tests for both paths.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: per-model launch flags for auto-switched GGUF models
* Studio: list switch-eligible GGUFs in /v1/models when auto-switch is on
* Studio: settings UI for OpenAI model auto-switch and idle auto-unload
* Studio: show save error over the disabled-idle hint in auto-switch settings
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: address gemini review (case-insensitive /v1/models retrieve, idle-input empty guard)
* Studio: address codex review (deterministic override args, exclude probe/embedding models from discovery)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: keep-warm count_tokens, gate idle on auto-switch, drop hidden models
Three hardening fixes to the opt-in auto-switch path surfaced while reviewing
the work that builds on it:
1. count_tokens keep-warm. /v1/messages/count_tokens counts via the loaded
tokenizer and already auto-switches, but the keep-warm middleware did not
track it, so idle auto-unload could free the model mid-count. It is now a
tracked in-flight path.
2. "Off means unchanged" for idle unload. get_auto_unload_idle_seconds now
reports 0 while auto-switch is disabled. Idle unload only makes sense with
auto-switch on (an unloaded model returns only via the next request's swap),
so a stray TTL can no longer trigger a destructive unload while the feature
is off, keeping the disabled state identical to pre-feature behavior.
3. Hidden models are not switch targets. The resolver index now skips what
Studio hides from its own pickers (the llama.cpp validation probe, RAG
embedding weights) via _is_hidden_model, so they can never be auto-switched
to by name.
Tests added for each.
* Studio: bare-id reuse, responses validation order, in-flight tracking
Review follow-ups after folding in the per-model overrides and discovery work:
1. A bare model id (no :VARIANT) is now satisfied by any loaded quant of that
repo. Previously a bare name resolved to the largest local quant, so it could
force a slow reload when a different quant of the same repo was already
serving. An explicit repo:VARIANT request still honors the quant.
2. /v1/responses now runs the auto-switch hook after the empty-input validation
so a request that 400s can no longer trigger a multi-minute model load before
being rejected. The hook still precedes both dispatchers, so streaming
requests switch.
3. The keep-warm middleware now tracks in-flight requests whenever auto-switch
is enabled rather than only when the idle TTL is already positive, so a stream
that starts with the TTL at 0 is still protected if idle-unload is enabled
mid-stream. Off still passes straight through.
Tests added for each.
* Studio: tighten auto-switch code comments
Comment/docstring-only pass over the OpenAI auto-switch feature: collapse
multi-line blocks, drop a comment that restated the gate it sits next to, and
trim verbose docstrings on internal helpers while keeping the load-bearing
rationale (concurrency, API behavior, drop-in compat, gotchas). No logic
change: verified comment-only with the AST/printer signature check.
* Studio: bind auto-switch locks per running loop
Review follow-up. The auto-switch swap lock and the keep-warm unload gate were
module-level asyncio.Lock objects. That is safe under the single uvicorn loop
and on Python 3.10+ (the Lock resolves the running loop lazily on acquire), but
a module-level Lock binds to one loop on pre-3.10, which can raise a loop
mismatch in multi-loop runners. Resolve each lock through a per-loop accessor
backed by a WeakKeyDictionary so every running loop gets its own Lock and stale
loops are collected. No behavior change under the server's single loop.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: auto-switch re-review fixes (body codes, coverage, swap, alias, tracking)
Follow-ups from a second review pass over the opt-in OpenAI auto-switch feature:
1. OFF-state status codes: /v1/completions and /v1/embeddings moved the body
read ahead of the loaded-state check, so a malformed/empty body with no model
loaded returned 500 instead of the prior 503. A shared helper reads the body
defensively (an unparseable/non-dict body yields no model), and the handler
re-reads after the 503 gate to surface the original parse error exactly as
before. OFF behavior is unchanged.
2. Local-model coverage: the resolver index only scanned ./models and the active
HF cache, while the model picker also lists the legacy/default HF caches, LM
Studio dirs, and user scan folders. A request for one of those named models
silently served the loaded model instead. _build_index now scans the same
roots (Ollama's symlink-creating scanner is skipped on the request path), and
resolution is offloaded with asyncio.to_thread so the wider scan never blocks
the event loop.
3. Swap vs in-flight stream: a cross-model swap killed the llama-server while
another client was still streaming from it. The hook now tracks how many
requests are streaming on the loaded model (in-flight minus those still inside
the hook) and returns 409 instead of swapping while one is active. Concurrent
same-model requests never reach this path, so they are unaffected.
4. Idle-unload + alias: after idle-unload freed the model, an unknown/alias name
resolved to nothing and 503'd, though it served the active model before the
TTL. Idle-unload now remembers the freed id and an alias request reloads it
(only an already-local model, so no remote download), cleared once a model is
loaded again.
5. In-flight tracking: the keep-warm middleware tracked in-flight only while the
feature was on, so a stream started while off could be unloaded if idle-unload
was enabled mid-stream. It now tracks on every inference path; counting is
cheap and invisible to clients.
Tests added for each.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Remove stray async_task_outputs files committed by mistake
* Studio: auto-switch review round 3 (revert swap guard, hardening)
Addressing a third review pass:
- Revert the cross-model swap guard. It counted keep-warm in-flight (which
includes external-provider calls that never touch the local model) and so
could 409 a local swap spuriously, and it still left a same-model request able
to start streaming on the model a concurrent swap was unloading. A correct fix
needs a request-lifetime reader/writer barrier; a partial guard was worse than
the honest single-slot behavior, so concurrent different-model use is back to
being serialized (documented), like llama-swap's single slot.
- Non-string request model (e.g. {"model": 123} on a raw-body endpoint) is now
treated as absent, so it falls through instead of raising in the membership
checks once an idle-unload stash exists.
- Idle-unload now stashes and replays the freed quant: an alias reload restores
the exact (id, variant) that was freed rather than the largest local quant.
- Anthropic /v1/messages validates max_tokens before the auto-switch hook, so a
request that 400s never triggers a model load.
- Keep-warm tracks a pending count for requests waiting on the unload gate, so
the idle loop cannot unload the model out from under a request that is blocked
on the gate but not yet counted as in-flight.
- The idle-unload task is awaited after cancel on shutdown to avoid pending-task
warnings.
- The resolver's HF cache scan is None-safe and logs at debug instead of letting
a bad root abort the whole index build.
- upsert_app_setting_map_entry rolls back explicitly on error.
Tests updated/added for each.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: keep saved idle-unload seconds when auto-switch is toggled off
* Studio: auto-switch hardening (thread-safe lock maps, body validation)
Defensive fixes from review:
- Guard the per-loop WeakKeyDictionary get-or-create for both the unload gate and
the auto-switch lock with a threading lock, since WeakKeyDictionary mutation is
not thread-safe when two event loops run on different threads.
- Build the resolver index under the cache lock so concurrent callers with an
expired cache don't all run the multi-dir scan at once.
- /v1/completions and /v1/embeddings return a clean 400 for a valid JSON body
that is not an object (e.g. a list), instead of a 500 from body.get(...).
- The keep-warm middleware only tracks POST requests (inference is always POST),
so CORS preflight (OPTIONS) is not counted, and tolerates a None path.
Tests added for the list-body 400 and the non-POST skip.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: auto-switch review round 4 (local-path load, swap guard, idle fixes)
From a 10-reviewer pass:
- HF-cache entries now load by a concrete local path, not the bare repo id. The
resolver records a load_path (the snapshot dir for a models--* cache repo, the
file/dir otherwise) so /load takes the local branch and can never trigger a
download to satisfy a partial cache. The advertised loader_id (repo id) is kept
as the launch-override key. resolve_local_gguf now returns
(load_path, variant, loader_id).
- Re-add a single-slot swap guard: a cross-model swap returns 409 model_switch_busy
while another inference request is active rather than killing its stream (the
caller is excluded from the count), and holds the keep-warm gate across the load
so no new inference starts mid-swap. Concurrent same-model requests never reach
this path. A residual spurious 409 is possible while a concurrent or external-
provider request is active; that is the documented single-slot tradeoff.
- Idle keep-warm tracks (model_identifier, hf_variant): reloading the same repo at
a different quant counts as a fresh model, so it is not unloaded before one TTL.
- Track Studio's own /api/inference/generate/stream so the idle loop can't unload
the model mid-stream on that route.
- A successful manual /load clears the idle-unload reload stash synchronously, not
only on the next idle poll.
Also merged origin/main (the branch had fallen behind, which would have reverted
unrelated files on merge). Tests added/updated for each.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: auto-switch review round 5 (concurrency, identity, load gate)
From a 10-reviewer pass (9 request-changes, 1 approve):
- Concurrent same-target requests load once instead of each returning 409. The
count-based busy guard could not tell "another request wants the same model"
(safe, load once) from "another request is using the loaded model" (refuse).
Track in-flight auto-switch requests per (target, variant) and subtract
same-target waiters from the busy count; a cross-model swap still 409s while a
genuinely different request is active.
- Fix the identity confusion introduced when round 4 began loading by concrete
local path: the backend identifier became a filesystem path. Record the
advertised repo id on the backend after an auto-switch load and use it so
(a) a model loaded manually by repo id is recognized as already serving
(no spurious reswap/409), (b) /v1/models reports the repo id, never a host
path or a duplicate, and (c) the idle-unload stash keeps the override keyed by
the repo id, so an alias reload after TTL keeps the user's saved launch flags.
- Gate the manual /load route with the keep-warm lifecycle gate so idle
auto-unload can't unload a model mid-load. load_model now wraps _load_model_impl
in the gate; auto-switch calls _load_model_impl directly since it already holds
the gate.
- Restore default-off parity on Anthropic /v1/messages: an unloaded backend with
auto-switch disabled 503s before the max_tokens 400 check, as it did pre-feature.
When the feature is on, request-shape validation still runs before any load.
Tests added for each; full backend suite diff vs baseline is unchanged.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: auto-switch review round 6 (concurrency ordering, leaks, unload gate)
From a second 10-reviewer pass (8 request-changes, 2 approve):
- Same-target concurrency: register a waiter by the raw requested model before
the (slow) resolve, and exclude pending requests from the swap busy count. The
middleware counts a concurrent same-model request as in-flight before it
resolves and joins the resolved-target waiter map, so the prior fix could still
409 it. The guard now subtracts max(same resolved-target, same raw-request)
waiters and ignores pending (a pending request is blocked in the middleware,
not generating, so a swap can't interrupt it).
- External-provider requests no longer block a local swap. The keep-warm
middleware counts every inference-path POST, but external-provider chat returns
before the auto-switch hook and never touches the local GGUF. The chat handler
now untracks itself before proxying, so its in-flight stream can't trip
model_switch_busy on a concurrent local auto-switch. The middleware skips its
own end-decrement for an untracked request.
- Manual /unload is gated like load and idle-unload: it holds the lifecycle gate
and returns 409 rather than tearing down llama-server while an inference request
is in flight.
- Response model id no longer leaks the load path. /v1/models already advertised
the repo id; chat, completions, embeddings, Anthropic messages, and audio
response bodies now use the same _llama_public_model_id helper instead of the
concrete on-disk model_identifier.
- Chat completions validates the non-system-message requirement before the
auto-switch hook (as /responses and /messages already do), so an invalid
request can't swap the resident model before returning 400.
Tests added for each; full backend suite diff vs baseline is unchanged.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: auto-switch review round 7 (teardown policy, Unsloth-active swap, training)
From a third 10-reviewer pass (9 request-changes, 1 approve), all on the same
asymmetric-teardown theme. Resolved per the intended policy that only automatic
paths defer to an active stream; deliberate user actions stay interrupting:
- Revert the manual /unload in-flight guard added last round. A manual /load or
/unload is a deliberate action and tears down immediately, as before; only the
automatic idle-unload loop and auto-switch defer to an active request. This
removes the asymmetry the reviewers flagged (manual /load, the /unload Unsloth
branch, and the opposite-backend swaps inside _load_model_impl) by not
extending the guard to deliberate paths, rather than spreading it.
- Auto-switch now refuses a swap whenever another inference request is in flight,
not only when a GGUF is already loaded. _load_model_impl also unloads an active
Unsloth/transformers backend before loading a GGUF, so the busy guard must cover
that case too; otherwise an Unsloth stream could be killed by an auto-switch.
- Refuse API-initiated training while inference is active. When Studio is driven
as an inference API (sk-unsloth key auth), POST /api/training/start returns 409
if a request is in flight, since training frees VRAM by unloading the chat
model and would kill the stream. The Studio UI (session auth) still starts
training and coexists/frees VRAM as before. A mixed UI+API session is not yet
special-cased. Adds auth.authentication.authenticated_via_api_key.
Tests added/updated for each; full backend suite diff vs baseline is unchanged.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: add UNSLOTH_MODEL_IDLE_TTL env override for idle-unload
Borrowed from PR 6517: a startup env var that sets the idle-unload TTL without
the settings UI. Unlike the stored setting (gated on auto-switch), the env value
is a standalone default that enables idle-unload even with auto-switch off, for
headless/container deploys. An explicit UI/API value still overrides it and stays
gated. The settings GET reflects the env default when nothing is stored.
* Studio: auto-switch fixes from review (paths, embeddings input, env idle reload)
- /v1/models advertises a client-facing alias instead of a filesystem path:
the ./models and LM Studio scanners report the on-disk path as the model id,
so the index now prefers model_id/display_name as the advertised/override id
and keeps the concrete path internal as load_path, still resolvable by path.
- /v1/embeddings validates input before auto-switch: a request with a model but
no input now 400s before the hook (like chat/responses/messages), so an
invalid embeddings request cannot unload or swap the resident model.
- Standalone UNSLOTH_MODEL_IDLE_TTL reloads the freed model: the hook now runs
when auto-switch or idle-unload is active, and with auto-switch off it skips
the resolver and only restores the idle-unloaded model, so the first idle
timeout no longer leaves later /v1 requests with nothing loaded.
- Do not resurrect a stale GGUF over an active Unsloth model: the reload-stash
path bails when a non-GGUF backend is loaded, so an unknown /v1 name cannot
tear down a live Transformers/Unsloth model.
- Defensive HF cache scan: each cache root's resolve/dedup is wrapped so a
missing or malformed root skips that root rather than aborting the index.
- Single-model retrieve checks the id is a string before lowercasing.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: fix automatic-load asymmetry, audio reload, preview, idle timer
The standalone UNSLOTH_MODEL_IDLE_TTL reload is a second automatic-load
trigger, but several validate-before-switch guards and reload hooks only
checked the auto-switch toggle. Add a shared _automatic_model_load_may_run()
(auto-switch on, or idle TTL > 0) and route every guard through it.
- /v1/completions validates prompt before any automatic load (it was the one
model-bearing route with no pre-check).
- /v1/chat/completions and /v1/embeddings pre-checks gate on the shared
predicate so a standalone idle TTL cannot reload then reject.
- /v1/messages no longer 503s before the reload hook can restore an idle-freed
model when auto-switch is off.
- Raw completions/embeddings with no model field pass a non-empty sentinel so
the idle-stash reload runs, restoring the legacy "omit model, use loaded" path.
- /api/inference/audio/generate gains the reload hook (after message validation)
so an idle-freed audio GGUF is restored.
- Public preview opts out of auto-switch via a request-scope flag, so a caller's
model field cannot swap away from the pinned checkpoint; preview chat streams
are now matched by _is_inference_path so idle-unload cannot kill them.
- Keep-warm no longer stamps activity on request start, and external-provider
untracking decrements without restamping, so periodic external traffic can no
longer keep the local GGUF warm forever.
Merges origin/main (the branch had fallen behind, which also brought in the
preview route the review flagged).
* Studio: surface model auto-switch in the API tab and demo it in examples
The OpenAI auto-switch toggle previously lived only in Settings -> General.
Add the same toggle to the API tab's usage-examples panel (it shares the
settings cache), and make the examples reflect it: when on, the Python
examples append a second call naming a different downloaded GGUF (so the
model field visibly selects which model serves), and the curl examples gain
a one-line note. Reuses the existing settings API client and i18n keys.
* Studio: harden OpenAI auto-switch reload-only path and Anthropic tool validation
- Omitted-model raw-body requests pass a reload-only sentinel so the idle-stash
reload still restores an idle-freed model, but the resolver never matches a
downloaded GGUF literally named "default".
- Reject malformed Anthropic client tools before _maybe_auto_switch_model so an
invalid request can no longer evict the loaded model.
* Studio: extend auto-switch reload-only and tool validation to schema endpoints
- Schema-backed endpoints (chat completions, responses, count_tokens, messages,
audio) defaulted an omitted model to "default" and passed it to the switch
hook, so a downloaded GGUF named "default" could be swapped to. Route the hook
through a helper that switches only on an explicitly set model, else reload-only.
- Propagate the explicit-set status when building the chat request from a
Responses request, so the non-streaming chat re-check stays reload-only too.
- Validate Responses function tools before the switch hook so a malformed tool
returns 400 without evicting the loaded model.
* Studio: serialize auto-switch swaps across event loops with a process-wide gate
The auto-switch lock is a per-event-loop asyncio.Lock, so two /v1 swaps on
different loops in one process could both pass it and race the single model slot
(the backend and _load_model_impl are process-wide). Add a process-wide
threading gate around the swap, acquired off the loop so a cross-loop wait never
blocks it, layered with the existing per-loop lock. Add a cross-loop test that
fails without the gate (two slow loads overlap) and passes with it.
* Studio: make the auto-switch swap gate wait cancellation-safe
_acquire_swap_gate awaited asyncio.to_thread(lock.acquire) when another loop held
the process-wide gate. to_thread cancellation doesn't stop the worker thread, so a
/v1 request cancelled mid-wait (client disconnect during a cross-loop swap) would
have its thread acquire the gate after the fact, while the finally that releases it
never runs -- permanently deadlocking later auto-switch swaps.
Poll a non-blocking acquire off a short asyncio.sleep instead: it still keeps the
wait off the loop and serializes across loops, but a cancel now lands during the
sleep, when the gate is not held, so nothing leaks. Add a test that deadlocks the
to_thread variant (it times out) and passes with the poll.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: validate modality and tool-confirmation before auto-switch
Two more request shapes could load a named GGUF and only then 400, evicting the
resident model:
- An image request naming a different text-only GGUF. The switch hook now takes
require_vision and rejects a swap to a non-vision target before loading it; a
GGUF's vision capability is its companion mmproj, knowable without a load, and
matches the post-load guard. Only the resolver branch is checked, never the
reload-stash restore.
- confirm_tool_calls=true with stream=false and local tools. /v1/chat/completions
now rejects that shape before the hook, mirroring the local tool path's
bypass_permissions exemption and intent signal.
The vision probe threads the ambient HF token to keep the capability-probe
invariant. Reload-only and idle-reload paths are unaffected.
* Studio: extend validate-before-switch and make the lifecycle gate process-wide
- /v1/messages/count_tokens now rejects malformed client tools before the switch
hook, like /messages (shared _validate_anthropic_client_tools helper), so a
count request can't evict the loaded model.
- /v1/chat/completions rejects a malformed tool_choice forcing object (a
{"type":"function","function":{}} with no name) before the switch hook.
- The inference lifecycle gate that blocks new inference during a swap is now
process-wide (a poll-acquired threading lock, cancellation-safe), not a
per-loop asyncio lock, so a request on another event loop can't start inference
while a swap tears the single backend down.
- Usage examples no longer hard-code a switch-demo repo most users lack; the
model is an explicit placeholder the user replaces.
* Studio: extend the auto-switch modality guard to /v1/responses and /v1/messages
The pre-load vision check that guards /v1/chat/completions now also runs on
/v1/responses and /v1/messages, so an image request naming a text-only GGUF is
rejected before the swap and never evicts the resident vision model. Run the
vision capability probe off the event loop. Make the /v1/models retrieve
loaded fast-path case-insensitive, and never advertise a host path from the
resolver. Remove the dead list_switch_eligible_ids helper, superseded by the
/v1/models catalog.
* Studio: filter /v1/models to GGUF, per-loop catalog lock, reject system-only Responses
Address review findings on the auto-switch path:
- /v1/models advertises only GGUF models the API can actually switch to; a
safetensors/LoRA entry would be selectable but never loadable via llama.cpp.
- The /v1/models catalog cache uses a per-loop lock (like the auto-switch path)
so a second event loop awaiting it can't hang in a multi-loop process.
- /v1/responses rejects system/developer-only input before the switch, mirroring
chat, so an invalid request can't evict the resident model.
- _build_index guards each scan source on its own so one bad root drops only
that source; the vision probe logs a real detection failure instead of
swallowing it.
* Studio: list cached GGUFs in /v1/models by inspecting files, not model_format
The HF-cache scanner leaves model_format unset for GGUF snapshots, so the
previous model_format == "gguf" filter dropped every downloaded HF-cache GGUF
from /v1/models and the retrieve fallback. Decide GGUF-ness from the on-disk
files via the resolver (info_has_local_gguf) instead, run off the event loop, so
the catalog advertises exactly what /v1 can serve.
* Studio: fix /v1/messages/count_tokens route binding plus auto-switch review fixes
The @router.post decorator for /messages/count_tokens had been separated from
anthropic_count_tokens by the _validate_anthropic_client_tools helper, so the
route bound to the validator and dropped its auth dependency. Move the decorator
back onto the handler. Add route-binding tests asserting each /v1 endpoint maps
to its handler with the auth dependency, so a decorator/handler split is caught
at the route level (the direct-call tests missed it).
Also from review:
- update_openai_auto_switch writes both settings keys in one transaction so a PUT
can't leave one updated and the other stale (drop the now-unused single setters).
- max_seq_length override rejects 0 at the boundary (ge=1) instead of accepting
then silently dropping it.
- Document that embeddings auto-switch is best-effort: GGUF pooling has no cheap
pre-load probe like vision's mmproj, so a guard would false-reject GGUF embedders.
- Add a positive idle-unload test (loop frees the model and stashes it for reload).
* Studio: validate Responses tool_choice + Anthropic mixed tools before switch, filter Ollama from catalog
More auto-switch review findings:
- /v1/responses rejects a forcing-function tool_choice with no name before the
switch, mirroring chat, so a malformed request can't evict the resident model.
- /v1/messages rejects mixing Anthropic server tools with custom client tools
before the switch (the check depends only on the payload, so it moves up cleanly).
- /v1/models no longer advertises Ollama-link models: info_has_local_gguf excludes
.studio_links / ollama_links entries, which the resolver skips and can't switch
to, so an advertised id never silently falls through.
* Studio: guard chat audio input before switch; surface env-backed idle unload in settings UI
A chat request carrying audio_base64 rides the same companion mmproj
projector as a vision request, so a text-only target cannot serve it
either. Flag require_vision for audio input as well so the multimodal
probe runs before the switch and a rejected request never evicts the
working model. Generalize the reject message to cover image and audio.
The settings response now reports idle_unload_active (effective TTL > 0)
so the UI can distinguish idle-unload that is active via the
UNSLOTH_MODEL_IDLE_TTL env var from the case where it needs the toggle
enabled.
* Studio: harden auto-switch eviction guards (count_tokens vision, TTS reload-only, mmproj/stash)
Four eviction/correctness fixes on the opt-in /v1 auto-switch path:
- /v1/messages/count_tokens now carries the same require_vision guard as
/messages, so an image count naming a text-only GGUF can't evict a loaded
vision model for a swap that can't serve the request.
- /audio/generate is now reload-only. A local GGUF's audio-input capability
is not a cheap pre-load probe (the companion mmproj signal can't tell an
audio projector from a vision one, and codec TTS ships no projector), so
resolving the client model could load a text/vision-only target and evict
the working audio model before the audio check fails. Only the idle-stash
restore runs here; switching TTS models is an explicit /load.
- The resolver no longer treats a standalone mmproj .gguf as a servable
model. _scan_models_dir's standalone-file pass does not filter mmproj the
way its directory scan does, so /v1/models could advertise a projector and
a switch could load it over the real weights.
- A non-GGUF (Transformers/Unsloth) load and a deliberate /unload now clear
the idle reload stash, so a manual load/unload is never superseded by a
stale idle-freed GGUF that the next /v1 request resurrects.
* Studio: report advertised repo id consistently after an auto-switch
Two model-id reporting fixes so an auto-switched cached HF GGUF is named by
its repo id everywhere, not its snapshot path:
- Streamed /v1/responses envelopes now derive the model id from
_llama_public_model_id (which prefers _openai_advertised_id) instead of the
raw model_identifier. After an auto-switch the identifier is the snapshot
path while the repo id lives in _openai_advertised_id, so the stream used to
report a snapshot basename while /v1/models, chat completions, and
non-streaming Responses all reported the repo id.
- When an advertised alias already resolves to the loaded model (a model
loaded by local path, requested by its repo or LM Studio id), the
already-serving early return now records the alias as the advertised id, so
/v1/models and responses report the alias and mark it loaded instead of the
path-derived basename. Resolver branch only; safe lock-free because an
in-flight request blocks any concurrent swap via the single-slot busy guard.
* Studio: validate request shapes before auto-switch (prompt/input/audio/mcp confirm)
Four more validate-before-switch guards so a deterministic client error never
evicts the resident model on the opt-in /v1 auto-switch path:
- /v1/completions rejects an object/number prompt (only a string or array is
valid) before the switch, instead of loading the named GGUF and letting
llama-server reject the shape afterward.
- /v1/embeddings rejects an object/number input the same way.
- Chat rejects an oversized audio_base64 upload (413) before the switch. The
size cap is a cheap, target-independent length check; the decode itself
stays post-switch to avoid decoding a valid upload twice.
- The chat confirm-without-stream pre-switch guard now mirrors the tool loop's
actual enablement: _effective_enable_tools (honoring a CLI --enable-tools
policy) and mcp_enabled (which opens the tool loop on its own but defers to a
CLI --disable-tools policy). Previously a confirm+no-stream request with only
mcp_enabled slipped past and 400'd after the swap.
* Studio: fix model-id retrieval, streaming n>1, resolver cache TTL, keep-warm auth
Four fixes from review:
- GET /v1/models/{id} legacy raw-path fallback now maps the raw identifier to
the same public id its /v1/models entry uses. After an auto-switch load the
identifier is the snapshot path while the entry is keyed by the advertised
repo id, so a client that cached the old absolute path no longer 404s on a
model that is in fact loaded.
- stream=true with n>1 is now rejected before the switch. Only the
non-streaming GGUF path returns multiple choices, so streaming n>1 is invalid
on every local serving path; both fields are known pre-switch, so it must not
load model B only to 400 and evict model A. Non-streaming n>1 stays
post-switch where the serving path decides.
- The resolver index cache is stamped after _build_index, not with the pre-scan
timestamp. On installs with enough local models for the multi-root scan to
exceed the 5s TTL, the cache was stored already expired and every request
rebuilt it.
- The keep-warm middleware no longer stamps model activity for 401/403
responses. It runs before FastAPI auth, so unauthenticated probes used to
refresh the idle timer without touching llama.cpp; they now decrement the
in-flight count without keeping the model warm.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
Extracted and narrowed from unslothai/unsloth#6543 by @TheJagStudio.
This keeps the startup/banner and text file encoding hardening separate from the already-merged Python code-exec UTF-8 fix in #6548.
Co-authored-by: Jagrat Patel <81472856+TheJagStudio@users.noreply.github.com>
* Speed up Studio desktop startup
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address Studio startup review findings
* Keep orphaned run cleanup before readiness
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: harden the data-recipe and inference consumer loops against pump death
Follow-up to #6643. The same single-unsupervised-consumer pattern the training
pump had lives in two sibling loops, with the same failure mode: one bad event
kills the only thread that updates the in-memory state every UI surface reads,
while the worker subprocess keeps running.
- data_recipe JobManager._pump_loop: a malformed worker log line that makes
parse_log_message raise no longer kills the pump. Guard _handle_event, the
queue read, and the worker-exit finalize, and broaden _drain_queue so a drain
error still finalizes the job instead of leaving it wedged "active" (which also
leaked the workflow-scoped API key until its 24h expiry).
- inference InferenceOrchestrator._dispatcher_loop: guard the routing body so a
malformed response or a mailbox put error can't kill the dispatcher and hang
every in-flight generation (callers key liveness on the subprocess, not on
this thread).
Adds regression tests for both.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: extend consumer-loop hardening to RAG, hub, auth, and stream-reader paths
Continuation of the data-recipe and inference pump hardening: the same
"background producer updates in-memory state that a single unsupervised
consumer surfaces to the UI" pattern shows up in several more Studio paths,
each able to silently freeze a UI surface while the worker keeps running.
RAG ingestion SSE (core/rag/ingestion.py):
- job_events polled the queue with a blocking get and never noticed client
disconnect or a dead worker, so a closed tab or a producer that died
without emitting a terminal event left the stream hanging. It now polls
with a timeout, emits heartbeats, ends on terminal job status, caps idle
time, and always pops the job registry in finally.
- Added _reap_finished_jobs() and call it from start_ingestion so finished
job state does not accumulate.
Startup reconcile (storage/rag_db.py, main.py):
- reconcile_orphaned_ingestion_jobs() marks ingestion jobs (and their
documents) that were left non-terminal by a previous crash as failed, so
the UI does not show jobs stuck "running" forever after a restart. Wired
in at startup next to cleanup_orphaned_runs().
Hub download watcher (hub/services/download_lifecycle.py):
- _watch() could leave a job pinned "running" if finalize raised. Body is
now guarded: on failure it logs and sets the job to error, and always
invalidates the hf cache scan in finally.
External provider stream (core/inference/external_provider.py):
- read timeout was None (no stall ceiling); set to 300s so a wedged
upstream surfaces as an error instead of an indefinitely hung stream.
Auth store (auth/storage.py):
- Enable WAL + busy_timeout on the auth DB so token validation (read on
every request) and login writes stop serialising on the rollback journal.
Matches studio_db / rag_db / providers_db.
Login rate limiter (routes/auth.py):
- _LOGIN_IP_BUCKETS could grow unbounded under spoofed-IP traffic; cap it
and prune stale buckets, mirroring the per-account bucket handling.
Training progress SSE (routes/training.py):
- Break promptly on client disconnect instead of waiting for the next
yield to fail on a closed socket, matching the export / data-recipe SSE
routes.
llama-server stdout drain (core/inference/llama_cpp.py):
- Broaden the drain guard so an unexpected decode/read error logs at debug
and stops the drainer cleanly instead of escaping the thread.
Frontend stream readers (chat-api.ts, rag-api.ts):
- Wrap the SSE read loops in try/finally + reader.cancel() so early return
([DONE]), thrown errors, and consumer aborts release the reader lock
instead of holding it until GC.
Tests:
- test_training_progress_stream_nan: fake request now implements the async
is_disconnected() the route polls, matching the other SSE route fakes.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: address Codex review feedback on the consumer-loop hardening
Four follow-ups from the automated review, all on code this PR introduced:
- Data-recipe pump (manager.py): a queue read that keeps raising an error
outside the read's narrow catch set (e.g. a broken queue pipe after the
child died) hit the `continue` guard and skipped the dead-worker finalize
below, spinning forever and leaving the job wedged "active" with its
workflow key unretired. On a read failure, fall through to finalize when
the worker is no longer alive. Added a regression test.
- RAG ingestion SSE (ingestion.py): the 5-minute idle cap could end the
stream while the job was still pending/running (a large document spends
minutes in embedding/storing with no per-batch progress event). The route
then sends [DONE], and the client treats a no-terminal-frame end as
completion, marking the document indexed mid-ingestion. Drop the idle cap:
while the worker is alive and non-terminal we keep heartbeating; the stream
ends only on terminal DB status, the None sentinel, or client disconnect.
- Login rate limiter (auth.py): the per-IP path pruned but then added the
new IP unconditionally, so a spoofed-source-IP spray kept _LOGIN_IP_BUCKETS
unbounded and made every new IP pay a full-dict prune scan. Gate the add on
the cap, mirroring the account path.
- Hub download watcher (download_lifecycle.py): if finalize raised before it
reaped (proc.wait) and dropped the worker (e.g. an I/O error draining
stderr), the crash path published a terminal state while the live Popen
stayed registered and kept writing the cache, and the terminal set_job let
claim() admit a retry on the same repo. Terminate + drop the worker before
setting the terminal state.
* Studio: keep login throttling working when the per-IP bucket dict saturates
Review follow-up. The previous cap fix skipped creating a bucket for a new IP
once _LOGIN_IP_BUCKETS was full, returning ip_fails=0. Under a sustained spray
that also fills the account dict, every failure from such an IP then looked
first-seen and _login_blocked had no bucket to enforce, so the cap effectively
disabled throttling once saturated.
Bound the dict with a FIFO eviction instead: if the IP is new and the dict is
full, reclaim expired buckets (rate-limited so a burst of distinct IPs can't
make each failure an O(n) sweep) and, if still full, evict the oldest-inserted
IP. The new IP always gets a real bucket, so a saturating (e.g. spoofed
X-Forwarded-For) spray stays throttled while memory stays bounded. Added a
regression test that saturates the dict and asserts a later IP is still blocked.
* Studio: address Codex review (RAG queue lifecycle, stream error, orphan chunks)
Three follow-ups on the Phase 6 changes:
- RAG ingestion SSE (ingestion.py): job_events removed the per-job queue in its
finally on ANY exit, including an early client disconnect while the worker is
still running. That dropped the worker's later events (the queue is the only
one _emit writes to) and made a reconnect find no queue and receive only
[DONE], which the client treats as completion. Only drop the queue on a
terminal exit (None sentinel / terminal DB status); leftover terminal queues
are still swept by _reap_finished_jobs. Added queue-lifecycle tests.
- External provider stream (routes/inference.py): once the 300s read timeout can
fire, the stream's except path failed the monitor but ended without an error
frame or [DONE], so the chat client saw a bare EOF and saved the timed-out
answer as a successful partial with no error. Emit an SSE error frame (and
[DONE]) on stream failure so the client surfaces it.
- RAG startup reconcile (storage/rag_db.py): marking a half-ingested document
failed left its chunks/fts/vec rows intact, and retrieval filters by scope not
status, so a failed document could still be retrieved and cited. Purge the
document's chunks when reconciling it to failed (the doc row stays for
re-ingest).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: release the remaining SSE stream readers (training, data-recipe, export)
reviewer.py follow-up. The chat and RAG SSE readers were wrapped in
try/finally + reader.cancel(), but the other three readers built on the same
response.body.getReader() pattern were left without it: streamTrainingProgress,
streamRecipeJobEvents, and streamExportLogs leak the ReadableStreamDefaultReader
lock (held until GC) when the consumer aborts, returns early, or a parse/callback
throws. Wrap each in try/finally + reader.cancel() (export already had a
try/catch, so it only needed the finally). All five frontend SSE readers now
release the reader symmetrically.
* Tighten resilience comments and docstrings
Condense the verbose explanatory comments and internal-helper docstrings added
in this branch to shorter, clearer forms. Comment/whitespace only; verified no
code changed via AST diff. No behaviour change.
* Studio: keep chunks for completed docs during ingestion reconcile
Startup reconciliation flips orphaned (non-terminal) ingestion jobs to failed and
purges the document's chunks so a failed source can't be retrieved. But it dropped
the chunks unconditionally, so a document the worker had already committed as
'completed' before the crash (only its job row left non-terminal) lost every chunk
while still reporting 'completed'. That leaves an empty source that retrieval can't
return and dedup (status != 'failed') blocks from re-ingest.
Only purge chunks when the document UPDATE actually transitions it to failed; an
already-completed document keeps its chunks. Adds reconcile regression tests for
both the completed-doc and genuine in-flight-orphan cases.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: drop a finished RAG job's queue when the client disconnects
job_events kept the per-job queue until it consumed the None sentinel, so a UI
that stops on the terminal event (its reader.cancel aborts the stream before
[DONE]) left the queue registered until the next _reap_finished_jobs sweep; a
batch of uploads followed by idling retained them all.
_run writes the terminal DB status before emitting the terminal event, so on
generator exit, drop the queue when the job's DB row is already terminal (worker
done, nothing to resume) and keep it only while the worker is still running. Adds
a disconnect-after-terminal-event regression test.
* Remove stray async task output files committed by mistake
* Studio: harden login IP throttle and end progress stream on disconnect
Two Codex review items:
Login per-IP throttle: when the per-IP bucket dict saturated, FIFO eviction could
drop a still-hot (blocked) bucket, so an IP could flood the dict with distinct
(or spoofed) source IPs to push out its own bucket and retry as first-seen. Stop
evicting hot buckets; a new IP that can't fit now shares a bounded overflow
counter that still trips the per-IP threshold, so a saturating spray stays
throttled and no live counter is reset.
Progress SSE: on client disconnect the polling loop only broke and fell through
to the unconditional final 'complete' frame, so a buffered or proxying consumer
could read a still-active run as completed. Return from the generator instead.
Adds regression tests for both (spray cannot reset a hot bucket; disconnect while
active emits no complete frame).
* Studio: shard the login overflow counter and stop cancelling chat stream after [DONE]
Two Codex review items:
Login throttle overflow: the single shared overflow counter meant that once a
saturating spray pushed it past the per-IP threshold, _login_blocked returned 429
for every new unbucketed source IP, before credentials were checked -- a global
login denial. Shard the overflow into a fixed array of counters keyed by hash(ip),
so a hot shard only throttles the IPs that map to it while a single source's
repeated failures still concentrate in one shard and stay throttled. Memory stays
bounded and no live bucket is evicted. Adds a regression test that a hot overflow
shard does not block an unrelated IP.
Chat stream: the reader.cancel() in the SSE finally fired even after a natural
[DONE]/EOF. The backend finalizes its api-monitor entry right after yielding the
sentinel (the local pass-through finishes after the last yield), so a client
cancel there can be observed as a disconnect and mark a completed request as
cancelled. Track natural completion and only cancel on an early/abnormal exit.
(No frontend unit test: the Studio frontend has no test harness.)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: give prep-timeout test fakes an is_disconnected method
The progress stream now ends on client disconnect (await request.is_disconnected()
before falling through to the terminal frame). After merging that into the
prep-timeout tests added later on main, their _FakeRequest/_ReconnectRequest must
provide is_disconnected or the generator raises AttributeError under CI.
* Studio: keep the login overflow throttle when bucket capacity frees up
_login_blocked only consulted the per-IP overflow shard while the bucket dict was
still at capacity. If a slot freed before the 60s window expired (e.g. another
IP's successful login calls _clear_login_bucket), a source counted in a hot shard
stopped being blocked and its next failure got a fresh per-IP bucket, resetting
the throttle the overflow path exists to preserve. Always max in the IP's shard
(shards are empty outside saturation, so it is a no-op in the common case). Adds a
regression test that a hot source stays throttled after a bucket frees.
* Studio: clear a login IP's overflow throttle on successful login
_clear_login_bucket reset the per-IP and per-account buckets on a successful
login but not the overflow shard, so after the dict saturated and an IP was
counted in overflow, a later successful login left those entries behind and the
next failed attempt could immediately return 429.
Store overflow entries as (timestamp, ip) so a source is throttled by its own
count within the shard (also removing cross-IP collateral within a shard), and
drop just that IP's entries in _clear_login_bucket. Adds a regression test that a
successful login clears the overflow throttle.
* Studio: bound the login overflow shard memory under high-cardinality spray
The per-IP overflow tracked failures in a time-pruned deque of (timestamp, ip)
tuples, so a spoofed-X-Forwarded-For spray of distinct one-off IPs grew memory and
the per-check scan with request cardinality for the whole window -- undermining
the bucket cap that exists to bound memory. Replace each shard with a fixed-
capacity dict (ip -> [count, window_start]): O(1) lookups, and when a shard is
full a one-off IP evicts the lowest-count entry (Space-Saving) so memory is hard-
bounded while a persistent attacker keeps a high count and is never evicted. Adds
a regression test that shards stay within the per-shard cap under a 5000-IP spray.
* Studio: purge chunks for already-failed docs during ingestion reconcile
The reconcile chunk-purge was gated on the documents UPDATE actually flipping a
non-terminal doc to failed. A doc the worker had already marked 'failed' before
the crash (job row left non-terminal) was not re-flipped, so its committed chunks
were kept and stayed retrievable/citable, since retrieval filters by scope not
status. Purge chunks whenever the document is not 'completed' (failed, in-flight,
or gone), preserving the completed-doc carve-out. Adds a regression test.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: don't inherit an evicted IP's count onto a new overflow source
When a full overflow shard evicted the lowest-count entry, the new source
inherited that count (Space-Saving base + 1). If a shard was saturated with hot
entries, an unrelated new IP could land at/over the threshold and be 429'd after a
single attempt -- cross-IP collateral despite the per-source-isolation intent.
New entries now start clean at count 1; the only cost is that a heavy hitter that
is the lowest-count entry in a fully saturated shard can briefly reset, which is
preferable to blocking a bystander. Adds a regression test.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: carry overflow failures into a new IP bucket on transition
_login_blocked took max(per-IP bucket, overflow shard) rather than combining them,
so a source could log (threshold-1) failures in overflow during saturation and,
once a bucket slot freed, another (threshold-1) in a fresh bucket within the same
window -- roughly doubling the per-IP limit. When a saturated-era IP first gets a
real bucket, migrate its windowed overflow count into that bucket (and drop the
overflow entry) so the combined failures throttle at the intended limit. Adds a
regression test.
* Studio: reconcile a completed doc's orphaned job to completed, not failed
When a crash left an ingestion job non-terminal after its document was already
committed as completed, reconcile marked the job failed. After restart the upload
UI has no in-memory SSE queue and falls back to getJob(), which treats a failed
job as an indexing failure and removes/toasts a document that is actually
searchable. Mark the job completed (keeping its chunks) when its document is
completed. Extends the completed-doc reconcile test to assert the job status.
* Studio: clamp the overflow failure count migrated into a login bucket
A saturated source could accrue an unbounded overflow count, then materialize
one deque entry per recorded failure when a bucket slot freed, allocating an
arbitrarily large deque under the login lock. Only at-or-above the per-IP
threshold matters for blocking, so cap the count there at the record and take
sites; the migration is now bounded without weakening the limit.
* Studio: keep the RAG job stream alive on a transient status read
The heartbeat poll read the job row unguarded; a momentarily-locked DB would
raise out of job_events, which the SSE route turns into a terminal error frame,
and the UI drops a document whose worker is still running. Treat a failed status
read as non-terminal: heartbeat and retry, and keep the queue so a reconnect can
resume.
* Studio: set busy_timeout before journal_mode on the auth DB
Switching journal_mode needs a lock, so if a refresh-token write already holds
one, journal_mode=WAL raises SQLITE_BUSY and the shared try leaves the
connection on SQLite's default zero lock wait. Set busy_timeout first so the
switch waits instead of failing.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* checkpoint preview endpoint
* harden new preview endpoints
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* address review
* Studio preview: pin adapter, guard streaming submit, robust copy-link
Harden the public per-checkpoint preview surface:
- Pin use_adapter=True in the preview payload sanitizer. Otherwise an
unauthenticated /p caller can POST use_adapter=false, which calls
disable_adapter_layers() on the shared in-memory model without restoring
it; since load_model skips reloads for the same checkpoint, every later
visitor (the page never sends the field) keeps getting base-model output
instead of the fine-tuned checkpoint. Forcing it on also re-enables a
previously disabled adapter and no-ops on merged checkpoints.
- Ignore preview-page submits while a response is streaming. The send
button was disabled but the Enter handler still called requestSubmit(),
so a second request could start before the first reply landed in msgs and
reorder the chat history. Both the keydown and submit handlers now honor
the disabled button.
- Keep the cloudflare-URL polling loop alive across transient startup fetch
errors instead of letting one rejection halt it.
- Build the copy-link from a backend preview_ref (output dir relative to
outputs_root, gated on previewability and the two-segment /p route limit)
so a nested output dir no longer copies a basename-only link that 404s.
Expose preview_ref on training run summaries.
Add route-level security tests (path traversal, payload sanitization,
asset containment, CSP header, HTML title escaping, streaming lock held
until drained) and preview_ref unit tests.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio preview: Safari-safe submit and adapter pin only for LoRA
Follow-ups from cross-browser and route simulations:
- Preview page: send the message from a shared send() helper called by both
the form submit and the Enter key, instead of form.requestSubmit(). The
latter throws on Safari < 16 and older iOS, which broke Enter-to-send there.
Verified across Chromium, Firefox and WebKit with Playwright.
- Only pin use_adapter=True when the resolved checkpoint is a LoRA adapter
(adapter_config.json present); for a merged checkpoint strip it to None.
A merged model has no adapter to toggle, so forcing it on only produced a
per-request "not a PeftModel" warning. The cross-request base-model
contamination fix still holds for LoRA previews.
Add a merged-checkpoint test asserting use_adapter is stripped to None.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio preview: trim verbose comments
Tighten comments across the preview routes, page, checkpoint helpers, and tests
to short single-line notes; drop ones that just restate the code. No behavior
change (verified comment/docstring-only with comment_tools.py check).
* Harden preview routes for PR #6486
- Return a generic 400 detail on a rejected preview path so the public /p
route never echoes the absolute install path (the real reason is logged
server-side instead).
- Strip confirm_tool_calls, session_id and rag_scope in the preview payload
sanitizer so the public surface stays inert regardless of the tool gate.
- Use Path.is_relative_to for the asset containment check, matching the rest
of the codebase.
- Add img-src 'self' and font-src 'self' to the preview page CSP.
- Preview page: on a mid-stream error keep the streamed text, flag the break,
and restore the prompt so the user can retry; drop the unused --font-sans var.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Studio: start the Cloudflare tunnel for --secure even in --api-only, and add --api-only to `unsloth studio run`
--secure exposes ONLY the Cloudflare link (it forces a loopback bind), but
_cloudflare_tunnel_should_start gated the tunnel on `not api_only`, so
`run.py --secure --api-only` started no tunnel and then fail-closed with
"A secure Cloudflare link is not allowed". That blocked the natural headless
use: serve just the API (no web UI) over the authenticated tunnel.
Make --secure start the tunnel regardless of api_only (the non-secure path is
unchanged: tunnel only a 0.0.0.0 bind, never api-only Tauri or Colab). Then
expose --api-only on `unsloth studio run` and forward it through both the
re-exec args and the in-venv run_server call, so
`unsloth studio run --secure --api-only --model ...` is a one-liner secure API
server.
Verified end to end: `run.py --secure --api-only` now brings up the tunnel and
serves /api/health over it (200), with / returning 404 (no UI).
Tests: update the tunnel-gate truth table (secure+api-only now tunnels;
secure+colab still does not) and add --api-only registration + re-exec/in-venv
forwarding coverage to the run CLI tests.
* Trim comments to be succinct (no behavior change)
* studio: address review on parent --api-only and secure api-only CORS
- Reject --api-only on the parent `unsloth studio` group when a subcommand
is invoked, with the same redirect guidance used for --parallel/--secure;
otherwise the flag was silently dropped and the UI served anyway.
- Keep CORS any-origin for secure api-only serving: that mode publishes the
API over Cloudflare for remote browser clients, so the Tauri-only lockdown
(still applied to plain local api-only) would break preflight. Factored the
decision into cors_origins_for_mode() and gate it on api_only and not secure;
run_server exports UNSLOTH_SECURE before importing main.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: suppress TAURI_PORT and de-dup test for headless run --api-only
- run_server gains emit_tauri_port (default True, unchanged for the Tauri/
desktop path). The new headless `run --api-only` path passes False so the
Tauri-only TAURI_PORT= line no longer prepends the documented URL/API key
banner (it ran even under --silent and could break one-liner parsers).
- Remove a duplicate test_reexec_forwards_api_only that shadowed the
parametrized one; fold the --secure --api-only case into it so the secure
headless path is actually collected.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: defer llama.cpp update probes and self-heal MLX on macOS
Two macOS startup problems shared one root area in the FastAPI lifespan:
- The llama.cpp capability + freshness probes ran inline before the server
yielded, so a cold/slow/flaky network on the GitHub freshness check blocked
'Application startup complete' (~34s on CI, longer in the field). Move both
probes to a daemon thread; app.state stays None until ready (status routes
already re-probe at request time). Opt out with UNSLOTH_DISABLE_UPDATE_CHECK=1.
- Train and Export were greyed out because mlx/mlx-lm/mlx-vlm arrive only
transitively and a resolver backtrack silently drops them, so CHAT_ONLY stayed
true. Add utils/mlx_repair.py: when Apple Silicon is detected without MLX,
reinstall mlx/mlx-lm/mlx-vlm by name on a daemon thread and re-run hardware
detection (opt out UNSLOTH_DISABLE_MLX_AUTOREPAIR=1). Surface a chat_only_reason
in /api/health plus a sidebar tooltip so a greyed Train/Export explains itself
instead of failing silently.
* Studio: guard model defaults against a None model name
load_model_defaults(None) called model_name.lower() with no guard, raising
'Error loading model defaults for None' before any model is selected. Return
an empty dict for a falsy/non-str name.
* Studio: drop obsolete upstream macOS + Windows Blackwell prebuilt pins
Both pins worked around gaps in ggml-org upstream prebuilts, but Studio now
routes every GPU host and all of macOS to the unslothai/llama.cpp fork
(published_repo_for_host), which ships the needed bundles, so both pins are
dead code on the default install path:
- macOS b9415: macOS always routes to the fork (its own macOS bundles), and
host_supports_macos_minos() is the backstop. The pin only fired under an
explicit --published-repo ggml-org override.
- Windows Blackwell b9360: Windows-NVIDIA routes to the fork, whose
windows-x64-cuda13 bundle covers Blackwell (manifest max_sm 120, toolkit
13.3), so the pin's self-disable check makes it dormant on every default
install; it could only activate under the same upstream override on a
13.0-13.2 driver.
Remove the pin constants, functions, and call sites. Keep the Blackwell
capability detection (_drop_blackwell_incapable_windows_cuda, _host_is_blackwell,
_windows_cuda_attempt_covers_blackwell) that still drops a non-sm_120 cuda-12.4
build on a Blackwell host. After this, an explicit --published-repo ggml-org
override on a Blackwell 13.0-13.2 host loses its GPU fallback and lands on CPU;
the default fork path is unaffected. Update the install selection-logic and
macOS-compat unit tests for the new no-pin behavior.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: walk back deeper on the macOS upstream prebuilt path
After removing the b9415 macOS pin, the explicit --published-repo ggml-org
upstream path still used the default 2-release fallback, so a pre-macOS-26 host
behind a run of macOS-26-only builds would exhaust two too-new plans (minos is
only checked post-download) and drop to a source build before reaching a
loadable older release. Walk back as deep as the fork macOS path
(DEFAULT_MAX_MACOS_RELEASE_FALLBACKS), turning the removed static pin into
dynamic discovery. Addresses review feedback on the macOS upstream fallback.
* Studio: pin transformers during MLX self-heal so it cannot break Studio
mlx-lm/mlx-vlm declare transformers>=5, but the single-env install pins
transformers==4.57.6. The self-heal used --upgrade with no constraint, so it
could upgrade transformers in the live venv and break the rest of Studio just to
make import mlx.core pass. Pin transformers to the installed version via a
constraint file: the resolver either finds an mlx build compatible with it or
fails (we stay chat-only), never upgrading transformers underneath Studio.
Addresses review feedback on the MLX repair install.
* Studio: harden MLX self-heal against an unsupported mlx-vlm
Pinning transformers alone made uv backtrack mlx-vlm to 0.3.9 (below unsloth-zoo's
mlx-vlm>=0.4.4), which imports but breaks VLM Train/Export -- so the self-heal
could clear chat-only onto a broken stack. Mirror the main installer: set
UV_OVERRIDE=overrides-darwin-arm64.txt so a current mlx-vlm coexists with the
transformers pin, require the same minimum versions unsloth-zoo declares, and
gate/validate on a full mlx_stack_available() check (not a bare import) so an
old or partial stack stays chat-only. Addresses PR review.
* Studio: filter Blackwell-incapable CUDA in resolve_upstream_asset_choice
resolve_upstream_asset_choice returned the first windows-cuda choice unfiltered,
so a Blackwell host could be handed an sm_120-incapable cuda-12.4 build while the
sibling planners drop it. Apply _drop_blackwell_incapable_windows_cuda here too
and fall through to the CPU bundle on a Blackwell host with no capable GPU asset.
Addresses PR review.
* Studio: re-poll health so MLX self-heal reaches an open UI
The sidebar cached the initial /api/health, so a successful background MLX
self-heal (chat_only flips false) did not re-enable Train/Export until a manual
reload. While chat-only for the recoverable mlx_unavailable reason, re-poll
/api/health and stop once Train/Export become available. Addresses PR review.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: make the disabled Train/Export tooltip reachable
The greyed Train/Export items pass a tooltip explaining why (e.g. MLX missing),
but a disabled <button> fires no pointer events and SidebarMenuButton only showed
tooltips while collapsed, so the explanation never appeared. Wrap a disabled
button in a focusable span and show its tooltip while expanded too; enabled items
keep the collapsed-only behavior. Addresses PR review.
* Studio: gate Train/Export on the full MLX stack, not bare mlx.core
detect_hardware enabled MLX training whenever `import mlx.core` worked, but the
MLX self-heal (utils/mlx_repair) treats a stack without mlx-lm/mlx-vlm at the
versions unsloth-zoo requires as inadequate. That asymmetry let the UI enable
Train/Export on exactly the partial/backtracked stack the self-heal is trying to
repair (greyed-in-but-broken VLM export). Gate on the same mlx_stack_available()
criterion so a partial stack stays chat-only (reason mlx_unavailable) and the
background repair restores it. Addresses PR review.
* Fix MLX repair and health auth for PR #6494
* Fix macOS upstream prebuilt fallback for PR #6494
* Fix MLX stack validation for PR #6494
* Fix MLX self-heal validation for PR #6494
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Review fixes: isolate hardware-state test, robust transformers pin
- test_chat_only_reason.py: detect_hardware() assigns module globals directly,
which monkeypatch does not revert; the autouse fixture now saves and restores
DEVICE/CHAT_ONLY/CHAT_ONLY_REASON/IS_ROCM so a chat-only verdict here cannot
leak into other backend tests (e.g. test_utils.py) on a GPU host.
- mlx_repair.py: read the transformers version from importlib.metadata instead of
importing transformers, so the install pin is not silently dropped when
transformers has valid metadata but fails to import.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix CI: model full MLX stack in dispatch tests, keep selection test offline
dispatch (macOS) job:
- detect_hardware now gates MLX on the full stack (mlx_stack_available imports
mlx_lm/mlx_vlm and checks dist versions), so faking only mlx.core makes the
apple_silicon_mlx profile resolve to CPU. The dispatch tests assert the routing
decision when the stack IS usable, so model a complete stack:
test_hardware_dispatch_matrix patches utils.mlx_repair.mlx_stack_available and
test_is_mlx_dispatch_gate patches hardware._has_usable_mlx_stack. The stack
predicate's own internals stay covered by test_mlx_repair.py.
Repo tests (CPU) job:
- test_no_cuda_attempt_on_published_path_for_13_1 fell through to a live
github_release_assets() upstream fetch after the Blackwell filter dropped every
published attempt, which the offline security scanner blocks. Stub that fetch so
the walk-back deterministically finds no usable CUDA build and raises
PrebuiltFallback without network.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Harden MLX self-heal: prepare transformers constraint inside the try
attempt_mlx_repair runs on a daemon thread, but _transformers_constraint_args was
called before the try. A failure there (e.g. tempfile.mkstemp on a full disk or a
bad TMPDIR) would propagate unhandled and silently kill the self-heal thread.
Move the call inside the try and initialize constraint_path so any such failure
is caught and leaves Studio chat-only instead of crashing the thread.
---------
Co-authored-by: Daniel Han <michaelhan2050@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
* Studio: pin CUDA_DEVICE_ORDER=PCI_BUS_ID and list GPUs at startup
On a mixed-GPU host, Studio could load a model onto a different physical
GPU than the one it selected. The free-VRAM probe numbers GPUs via
nvidia-smi (PCI-bus order), but CUDA defaults to FASTEST_FIRST ordering,
so a selected index written into CUDA_VISIBLE_DEVICES resolved to the
wrong card. Example: 5090 + RTX PRO 6000, the picker chose the emptier
RTX PRO 6000 (nvidia-smi index 1) but CUDA read index 1 as the 5090.
Pin CUDA_DEVICE_ORDER=PCI_BUS_ID at import (before any CUDA context is
created) in both the Studio entrypoint and the hardware module, so torch,
nvidia-smi, and CUDA_VISIBLE_DEVICES share one index space. setdefault
keeps an explicit user override intact. Child processes inherit it via
os.environ.
Also list every detected CUDA GPU with its index at startup instead of
naming only device 0, matching nvidia-smi -L and making the selected
index unambiguous on multi-GPU hosts.
* Studio: make CUDA_DEVICE_ORDER tests exercise module import and respect user override
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: guard full _print_cuda_device_list body and fix test PYTHONPATH trailing separator
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: serialize non-streaming responses once and pool the proxy client
Two safe latency wins on the OpenAI/Anthropic-compatible endpoints that leave
the streaming generation paths untouched (they keep Connection: close and
max_keepalive_connections=0 so a client disconnect still stops GPU decode).
1. Non-streaming responses used JSONResponse(content=model.model_dump()), which
builds a dict and then re-runs json.dumps. Serialize once with
model.model_dump_json() via a small _model_json_response helper. The body is
byte-identical (nulls preserved), about 3x faster to encode in a microbench.
2. The non-streaming completions and embeddings proxies built a fresh
httpx.AsyncClient per request. Route them through one pooled client
(core/inference/llama_http) closed on shutdown; streaming generation keeps
its own per-request close-only client. About 5x faster per call to the
local llama-server in a microbench.
The existing API-monitor tests for the non-streaming completions, embeddings
and passthrough paths now patch nonstreaming_client instead of httpx.AsyncClient
to match the pooled client, so they stay deterministic.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: make the pooled non-streaming client per event loop
Review follow-up on the shared httpx client. It was a single module-global
instance, which has two lifecycle problems the per-request client did not:
1. After aclose() in lifespan shutdown, nonstreaming_client() kept handing back
the closed client, so a second lifespan in the same process (repeated
TestClient, embedded restart) failed with "client has been closed".
2. An httpx client binds its transport to the loop it first runs on, so reuse
from another loop could raise "Event loop is closed".
Hold one client per running loop in a WeakKeyDictionary, recreate when missing
or closed, and close all on shutdown. Single-loop production is unchanged.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: convert SecurityHeadersMiddleware to pure ASGI
SecurityHeadersMiddleware was the last BaseHTTPMiddleware in the global stack,
so every response (including SSE streams) was wrapped in an anyio stream that
penalizes streaming. Rewrite it as a pure-ASGI middleware that mutates the
response-start headers, mirroring the logging-middleware rewrite in #6337.
The header logic is unchanged: it uses MutableHeaders over the start message,
so the same get/del/setdefault calls apply (CSP nonce splice and strip,
X-Frame-Options skip on Colab and the artifact-preview frame, the baseline
nosniff/Referrer-Policy/Permissions-Policy/server headers). The existing
middleware tests cover it; added cases assert headers still apply to a
streaming response and that the artifact-preview path omits X-Frame-Options.
* Studio: harden ASGI header coercion in SecurityHeadersMiddleware
Review follow-up. MutableHeaders mutates its raw list in place, so if a server
sends http.response.start with tuple-valued or missing headers the mutation
would raise. Coerce to a list (defaulting to empty) before wrapping, then inject
the same security headers as before. Also drop a stray em dash in a comment.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: make lifespan shutdown resilient to a dead default executor
On an abrupt shutdown (closing the Windows console window, or interpreter
teardown racing uvicorn's graceful stop) the event loop's default thread-pool
executor can already be shut down by the time the FastAPI lifespan shutdown
runs. The first post-yield statement was an unguarded
`await asyncio.to_thread(terminate_hub_downloads)`, so executor.submit raised
`RuntimeError: cannot schedule new futures after shutdown`. That raise
propagated up through every nested merged_lifespan __aexit__, aborted the rest
of the cleanup (DEVICE reset, compiled-cache clear), and surfaced as
"Application shutdown failed. Exiting."
Extract the post-yield cleanup into utils/lifespan_shutdown.run_lifespan_shutdown
and guard each step independently. On the to_thread RuntimeError, fall back to
running the (already best-effort, quick) terminate inline on the loop thread so
shutdown still completes cleanly. The helper is dependency-injected and free of
the heavy backend import graph, so it is unit-tested in isolation.
Add tests/test_lifespan_shutdown.py (4 cases: dead-executor survival, normal
path, terminate error, clear error). Validated on windows-latest and
ubuntu-latest runners on Python 3.13.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: only retry terminate inline when scheduling fails, not when the body raises
Address review feedback on the shutdown helper: the previous
`except RuntimeError` after `asyncio.to_thread(terminate_downloads)` could
not tell a dead-executor scheduling failure from a RuntimeError raised by
terminate_downloads itself, so a body-side RuntimeError on a healthy executor
ran the cleanup a second time inline.
Schedule via loop.run_in_executor and await separately: a dead default executor
raises synchronously at submit time (inline fallback), while a body exception
only surfaces when the future is awaited (logged, never retried). Add a
regression test that a body RuntimeError runs terminate exactly once.
* Studio: run terminate cleanup with a copied context (parity with asyncio.to_thread)
Simulation across the executor-state x exception x DEVICE matrix surfaced the one
behavioural difference from the original implementation: asyncio.to_thread copies
the caller's contextvars into the worker thread, while a bare
run_in_executor(None, fn) does not. Restore exact parity by scheduling
ctx.run(terminate_downloads) from a contextvars.copy_context(), so the refactor
is a behavioural no-op apart from the intended dead-executor recovery. Add a
regression test asserting the copied context is visible to terminate_downloads.
* Studio: tighten comments in lifespan_shutdown helper and tests
Condense the verbose docstrings/comments to the non-obvious rationale and drop
the self-evident ones. Verified comment-only with comment_tools.py check
--strip-docstrings (code signature unchanged); tests and sims still green.
* Studio: address review nits on lifespan shutdown helper
Annotate hw_module as types.ModuleType, reword the schedule/await comment
(inline fallback runs, it does not retry), and add a test for the public
loop.shutdown_default_executor() path (the 'Executor shutdown has been called'
RuntimeError that real uvicorn shutdown takes, distinct from the submit-time
'cannot schedule new futures after shutdown').
---------
Co-authored-by: Daniel Han <michaelhan2050@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: show llama.cpp version and GPU specs in the About panel
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: run the hardware endpoint off the event loop
* Studio: show the full llama.cpp release tag (incl -mix-<sha>) in the About panel
* Studio: order About-panel GPUs by visible ordinal and skip the llama.cpp probe during updates
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix hardware info refresh and endpoint scope
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Etherll <61019402+Etherll@users.noreply.github.com>
* Studio: add --secure Cloudflare-only mode and revamp API usage examples
--secure / --not-secure on `unsloth studio` and `unsloth studio run`:
- --secure binds 127.0.0.1, requires the Cloudflare tunnel, and advertises only
the Cloudflare link. cloudflared reaches the server over localhost, so the raw
port is never exposed on a public interface.
- If the tunnel cannot start, fail closed with a clear message instead of
silently leaving a raw 0.0.0.0 link.
- Default stays not-secure (no behavior change); coexists with the existing
--cloudflare/--no-cloudflare flag. Host defaults are unchanged.
- /api/health (authed) now reports the live tunnel URL.
API usage examples (Profile > API):
- Example tabs for curl, Python, curl + tools, Python + tools, plus an OS row
(Linux/macOS/WSL vs Windows) auto-detected from the platform.
- Windows curl passes the JSON body via a file so PowerShell does not strip the
quotes when calling curl.exe.
- Python + tools forwards enable_tools/enabled_tools through extra_body and
guards chunk.choices, since tool-lifecycle events carry no choices.
- Shows the loaded model name and the real API key while it is still revealed.
- A Cloudflare Tunnel toggle (default on) shows the public tunnel URL and uses
it as the base_url in the examples when a tunnel is running.
Tests cover the tunnel start gate and the --secure flag on both commands.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: gate --secure tools on public exposure and harden API examples
In secure mode the server binds loopback but is reachable via the public
Cloudflare tunnel, so resolve the tool policy against the public exposure
(0.0.0.0) rather than the loopback bind. This keeps server-side tools off by
default and prompts before enabling them, instead of inheriting the loopback
default of on. The startup tool notice now names the public surface.
Also reject --secure with --no-cloudflare directly in run_server and the
run.py argparse (not only the CLI), JSON-encode interpolated model names so
Windows paths and quotes cannot produce invalid JSON or broken snippets, and
force-refresh /api/health on the API panel so a tunnel that starts after the
first health read still surfaces its URL.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: API examples show direct host when tunnel toggle is off; move Copy onto code
The Cloudflare Tunnel toggle had no visible effect when Studio was opened
through the tunnel: the off state fell back to window.location.origin, which
equals the tunnel URL in that case. /api/health now reports the direct
host:port (server_url), and the API panel uses it for the off state so it shows
the real non-tunnel base. Also move the Copy button out of the tab row and onto
the code block.
* Studio: highlight API examples, add advanced tabs, fix tunnel toggle row
Syntax-highlight the curl/PowerShell/Python snippets with the app's shared
shiki plugin (bash/powershell/python). Add 'curl + advanced' and
'Python + advanced' tabs that set temperature/top_p/top_k/min_p/
repetition_penalty/max_tokens, enable thinking, and turn on all tools.
The Cloudflare Tunnel row no longer shifts the code block: the tunnel URL is
always rendered (dimmed when off) so toggling keeps the row height constant.
Key the highlighted block on its content so it remounts when only the base URL
changes (the renderer's block memo otherwise kept a stale URL).
* Studio: rename API tunnel toggle to Secure HTTPS, hint --secure when exposed
Rename the API examples toggle from Cloudflare Tunnel to Secure HTTPS. When the
server was not launched with --secure, show an info tooltip noting the raw
0.0.0.0 port is still globally reachable and pointing at --secure. /api/health
now reports whether --secure was used so the hint is hidden in secure mode.
* Studio: force tools off for plain network/secure launches
The plain 'unsloth studio --secure' (and '-H 0.0.0.0') launcher re-execs run.py
and never installed a tool policy, so the process default (honor per-request
enable_tools) let any API-key holder run Python/terminal tools over the public
endpoint. Force the policy off at the run.py entrypoint when network-reachable
(0.0.0.0 or --secure); 'unsloth studio run' still installs its own resolved
policy and does not go through this path.
* Studio: apply default tool policy in run_server, not the run.py entrypoint
The plain launcher runs from the studio venv and calls run_server directly, so
it never hit the run.py __main__ guard. Move the network/secure default-off tool
policy into run_server so every launch path (plain, --secure, direct run.py)
gets it; the run subcommand still overrides it with its resolved policy.
* Studio: clarify --secure help text on the network exposure tradeoff
Spell out in --help (both unsloth studio and unsloth studio run, plus the
run.py argparse) that --not-secure also serves the raw 0.0.0.0 port reachable
from anywhere on the network, matching the API panel's Secure HTTPS hint.
* Studio: cache API-key PBKDF2 derivation to cut per-request /v1 auth overhead
validate_api_key re-ran the 100k-round PBKDF2 on every authenticated
request, adding ~15ms to each /v1 call made with an sk-unsloth- key.
Benchmarked against the bare llama-server it proxies to, API-key requests
carried ~22ms of fixed overhead vs ~7ms for the JWT path; the gap was
entirely this redundant key derivation (Pydantic validation measured
0.005ms, so it is not a factor).
The raw-key to hash mapping is a pure deterministic function of the fixed
server salt, so memoize it per process, keyed by a salted HMAC of the key
(never the key or a recoverable digest). The cached value equals what is
already stored at rest. Revocation and expiry remain enforced by the
SQLite read on every call, so a cache hit only skips the KDF, never the
active or expiry checks. Only keys that exist in the DB are cached, so
unknown-key spam cannot grow it.
After the change the API-key /v1 overhead drops to ~8ms, at parity with
JWT, while the at-rest PBKDF2 hashing is unchanged.
Adds test_api_key_expiry.py covering API-key and JWT expiry enforcement
and the new cache: it skips the KDF on repeat and still rejects revoked
or expired keys.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: tighten comments across the secure-tunnel and API-key changes
Condense multi-line comments and docstrings to one or two lines, drop the
ones that restate obvious code, and remove an orphaned test section header.
Comment-only: verified with comment_tools.py check (9/9 code unchanged), the
auth/secure-tunnel/CLI test suites, and a clean frontend typecheck and build.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
The studio backend lazily imports unsloth_zoo submodules (export's
llama_cpp, hardware's vllm_utils, mlx, chat_templates). Each imports the
top-level unsloth_zoo, which raises 'Please install Unsloth via pip
install unsloth' unless UNSLOTH_IS_PRESENT is set (normally by import
unsloth). Only hardware.py set it per-site; the rest crash on a clean
install, and it surfaced on Windows. Set it once at backend startup (as
unsloth does on import) so every site, and the DiffusionGemma runner,
works across platforms. Follow-up to #6259 (which covered only the shim
subprocess env).
* Require a found ROCm DLL before forcing BNB_ROCM_VERSION in Studio paths
main.py previously set BNB_ROCM_VERSION=72 whenever HIP_PATH or ROCM_PATH
was set, and the training worker fell back to a blind 72 when DLL
detection found nothing. On a Windows machine with the AMD HIP SDK
installed but CUDA or CPU torch, that forces a ROCm backend onto a
non-ROCm bitsandbytes wheel, which raises at import. Both paths now only
write the override when a libbitsandbytes_rocm DLL actually exists (or a
seeded value is already present), matching the strict gates in
unsloth/import_fixes.py.
Also removes four redundant local import shutil statements in
unsloth/save.py that shadow the module-level import, the same pattern
that caused the UnboundLocalError fixed in #6149.
* Worker: gate the BNB override on a found ROCm DLL, preserving seeded marker
Review follow-ups: track _found_rocm_bnb in the worker like main.py so a
ROCm DLL with an unparsable name still gets the seeded or 72 fallback,
and skip the env write entirely when no DLL exists so a seeded value
keeps its sitecustomize marker and stays redetectable by later import
fixes.
Adds an in-app "Update llama.cpp" banner and button to Unsloth Studio. When the installed prebuilt is behind the latest published release, a non-invasive banner appears; clicking Update downloads the latest prebuilt for this host and swaps it in place in the background, with no restart.
Detection reuses the freshness check from #5529. The update re-runs install_llama_prebuilt.py the same way setup.sh and setup.ps1 do after #5963: it forwards the published repo and the AMD gfx target derived from the install marker, and does not pass the removed --simple-policy or the arm64-only --cpu-fallback.
While the installer swaps binaries the backend enters a maintenance state (flag set under the serial load lock, active server unloaded) so a concurrent load cannot start a server from a half-swapped binary; the next load uses the new build. The banner also handles refused responses and jobs started in another tab so it never sticks on "Updating...".
Verified end to end on an NVIDIA B200: installed b9493, detected the update, applied it, and confirmed the binary at the same path advanced to b9585 in the same process. Hermetic backend tests and the frontend type-check pass.
* Fix Windows installer winget msstore certificate failure
`winget install` was invoked without `--source winget`, so winget also
queried the msstore source. When msstore fails certificate pinning
(error 0x8a15005e, "The server certificate did not match any of the
expected values") winget aborts and demands `--source`, so the Python
(and uv) install fails even though the package exists in the winget
source.
- Pass `--source winget` to all winget install calls (Python x2, uv).
Both packages live in the winget source, so this is strictly correct
and skips the failing msstore round-trip entirely.
- Add a python.org fallback (Install-PythonFromPythonOrg) that downloads
the official installer and runs it silently per-user (no admin/UAC)
when winget is unavailable or fails for any reason. Mirrors the
existing uv -> astral.sh fallback so Python installs without manual
steps. Resolves the latest 3.13.x from python.org with a pinned
fallback, and selects the amd64/arm64/x86 installer per architecture.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Pin remaining setup.ps1 winget calls to --source winget
Two winget invocations in studio/setup.ps1 still queried all sources and
could hit the same msstore certificate-pinning failure (0x8a15005e) that
broke the Python install in install.ps1:
- `winget show Nvidia.CUDA --versions` (CUDA Toolkit version probe)
- `winget install ... ShiningLight.OpenSSL.Dev` (OpenSSL dev for llama-server)
Every other winget call in this file already passes `--source winget`
(Git, CMake, VS Build Tools, CUDA install, Node.js, and setup.ps1's own
Python 3.12 install), so these two were stragglers. Both packages live in
the winget source; pinning it makes setup robust to an unhealthy msstore
source, matching the rest of the file.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Stop amd-smi GPU probe from popping a DiskPart UAC prompt
On Windows, AMD GPU detection in install.ps1 and studio/setup.ps1 runs
`amd-smi list` / `static --asic` / `version`. amd-smi (shipped in
System32 by the Adrenalin driver) auto-elevates to read GPU/APU memory
details, surfacing a confusing DiskPart UAC prompt mid-install. The
Studio backend already documents and circuit-breaks on this in
studio/backend/utils/hardware/amd.py, but the installers did not.
Add an Invoke-AmdSmiNoElevate helper (both scripts) that runs amd-smi via
Start-Process under __COMPAT_LAYER=RunAsInvoker so it cannot auto-elevate
(no prompt), with a 30s timeout (matching amd.py) so a flaky amd-smi
cannot stall the install for minutes. On failure/timeout the existing WMI
name -> gfx fallback still resolves the arch, so detection is unchanged on
working hosts.
Verified on a Strix Halo (Radeon 8060S / gfx1151) box: the prompt is gone
and the probe is bounded.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Add experimental ROCm-on-WSL setup helper for Strix Halo (gfx1151)
install.sh already routes gfx1151 (Radeon 8060S / Strix Halo) to the
repo.amd.com/rocm/whl/gfx1151 wheels once a ROCm runtime is present, but
it does not install AMD's driver/ROCm stack -- a large, admin-gated
prerequisite. scripts/install_rocm_wsl_strixhalo.sh automates the Linux
side on a dedicated Ubuntu 24.04 WSL2 distro: ROCm 7.2 (wsl usecase), the
rocr4wsl HSA runtime, a librocdxg build, env setup, and a PyTorch gfx1151
GPU smoke test. A hard preflight refuses to run until the Adrenalin
>=26.3.1 driver is actually present, so it cannot half-install.
Procedure adapted from AMD's ROCm-on-WSL docs and community gfx1151 notes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Detect AMD GPUs by name so native Windows gets a GPU llama.cpp
The gfx-arch inference from the WMI GPU name was gated behind $HasROCm,
which the hipinfo/amd-smi probe leaves false on the common Windows case
(Adrenalin driver only, no HIP SDK -- and amd-smi often cannot read the
arch without elevation). So an AMD GPU was detected by name but never
mapped to a gfx target, --rocm-gfx was not forwarded, and studio setup
fell back to a CPU llama.cpp build.
Un-gate the inference (install.ps1 + studio/setup.ps1) so it runs whenever
an AMD GPU name is available. The inferred gfx is forwarded as --rocm-gfx,
which makes install_llama_prebuilt.py download the matching lemonade-sdk
ROCm prebuilt (e.g. llama-bNNNN-windows-rocm-gfx1151-x64.zip) -- a
GPU-accelerated llama.cpp that bundles its own ROCm runtime, so it runs
with just the Adrenalin driver. PyTorch's ROCm wheels still require a
confirmed HIP SDK ($HasROCm), so this only affects llama.cpp / inference
and never pulls broken ROCm torch.
Also broaden the name->arch table to every family lemonade ships Windows
assets for: gfx120X (RDNA 4), gfx110X (RDNA 3), gfx1151/gfx1150
(RDNA 3.5), and gfx103X (RDNA 2). Unknown names still fall back to CPU.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Suppress amd-smi DiskPart UAC prompt in the Python install/runtime paths
The earlier PowerShell guard covered install.ps1 / setup.ps1, but the
Python installer (install_llama_prebuilt.py detect_host,
install_python_stack.py ROCm probes) and the Studio backend monitor
(amd.py) also shell out to amd-smi on Windows, where it auto-elevates and
pops the same DiskPart UAC prompt mid-install / at runtime.
Inject __COMPAT_LAYER=RunAsInvoker into the amd-smi subprocess env on
Windows so it runs un-elevated (no prompt). Callers already tolerate an
empty/failed result and fall back to WMI / name detection (installer) or
the existing circuit breaker (amd.py). Gated to Windows so Linux/macOS
amd-smi behaviour is unchanged.
- install_llama_prebuilt.py: handled centrally in run_capture (covers
detect_host's `amd-smi list` and the version probe).
- install_python_stack.py: new _amd_smi_env() helper on its 3 raw
subprocess.run amd-smi calls.
- amd.py: merge RunAsInvoker into the existing child env.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Tighten AMD GPU name->arch patterns to avoid mismatches
The W9[0-9]{3} and RX 90[0-9]{2} patterns added for RDNA 4 were
speculative and over-broad: W9xxx would also match old GCN FirePro
W9100/W9000 cards (wrong gfx1201 -> a lemonade gfx120X download that
fails validation), and RX 90[0-9]{2} was redundant with the explicit
9070/9060 entries. Drop both; keep only confirmed RDNA 4 SKUs. Unmatched
AMD names still fall back cleanly to CPU.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Fetch the llama.cpp validation model via huggingface_hub
The prebuilt validation downloads a tiny GGUF test model from huggingface
via bare urllib. On Windows / proxy setups where the server sends an
incomplete TLS chain, urllib cannot complete the Amazon CA chain (it does
no AIA intermediate fetching) and fails with CERTIFICATE_VERIFY_FAILED, so
a perfectly good GPU prebuilt is rejected and the installer falls back to a
CPU source build.
Route the validation-model download through huggingface_hub
(hf_hub_download) -- the same mechanism Studio uses for model downloads,
which completes the chain where urllib cannot -- keeping the direct URL as
a fallback. This lets the lemonade ROCm prebuilt validate and install on
cert-restricted machines (verified: hf_hub_download succeeds where urllib
returns CERTIFICATE_VERIFY_FAILED).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Guard the remaining raw amd-smi version probe via run_capture
A ROCm-version detector in install_llama_prebuilt.py called amd-smi version through a raw subprocess.run that bypassed run_capture's Windows RunAsInvoker guard, so it still triggered the DiskPart UAC prompt during setup. Route it through run_capture like the other amd-smi calls.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Forward --rocm-gfx even when the ROCm runtime is unconfirmed
setup.ps1 forwarded --rocm-gfx (and picked the windows-hip llama.cpp
prebuilt) only inside `if ($HasROCm)`. On Adrenalin-only hosts (amd-smi
present but no HIP SDK, so $HasROCm stays false) the gfx arch was
name-inferred but never forwarded, so install_llama_prebuilt.py saw
has_rocm=False and installed the CPU build -- even though the lemonade
gfx1151 GPU prebuilt runs fine there (it bundles its own ROCm runtime;
verified: llama-cli --list-devices -> ROCm0: AMD Radeon 8060S, 69 GB).
Forward --rocm-gfx whenever a gfx arch is known (it is authoritative and
implies ROCm in install_llama_prebuilt.py), and treat a known gfx arch as
windows-hip in the existing-install mismatch check. --has-rocm stays gated
on the confirmed-runtime signal.
Verified on Radeon 8060S / gfx1151: the installer now selects, validates,
and installs llama-b1286-windows-rocm-gfx1151-x64.zip (ROCm DLLs present)
instead of the CPU build.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Install AMD ROCm PyTorch on name-inferred gfx hosts (enables Train/Export)
setup.ps1 picked the AMD ROCm PyTorch wheels only inside `if ($HasROCm ...)`.
On Adrenalin-only hosts (amd-smi present but no HIP SDK, so $HasROCm is
false) the gfx arch was name-inferred but the ROCm-wheel branch never ran,
so the host got torch+cpu. With CPU torch, torch.cuda.is_available() is
False, so the Studio backend sets CHAT_ONLY=True and hides Train/Export.
Un-gate the ROCm PyTorch index resolution on a known gfx arch (mirrors the
llama.cpp --rocm-gfx fix). AMD's per-arch Windows wheels
(repo.amd.com/rocm/whl/<gfx>) bundle the ROCm runtime, so they work without
a HIP SDK; a failed install still falls back to CPU.
Verified on Radeon 8060S / gfx1151: torch 2.11.0+rocm7.13.0 installs and
torch.cuda.is_available() -> True, device "AMD Radeon(TM) 8060S Graphics",
GPU matmul OK -> CHAT_ONLY=False -> Train/Export enabled.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Force amd-smi un-elevated process-wide in the Python installers
Guarding individual amd-smi call sites kept missing some (install_python_stack.py's probe loop and its Windows GPU re-check), so the DiskPart UAC prompt kept reappearing. Set __COMPAT_LAYER=RunAsInvoker process-wide at the top of install_python_stack.py and install_llama_prebuilt.py on Windows so every amd-smi subprocess (current and future) runs un-elevated with no per-call guard. Safe: these scripts only spawn amd-smi/rocminfo/hipinfo probes and pip/uv. setup.ps1 keeps per-call guards because it also spawns winget installers that need elevation.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Fix Invoke-AmdSmiNoElevate exit code on PS 5.1 + RX 7700S arch match
Start-Process -PassThru leaves the returned process object's .ExitCode
$null after WaitForExit on Windows PowerShell 5.1, so the helper set
$LASTEXITCODE to $null and every caller's `if ($LASTEXITCODE -eq 0 ...)`
was always false -- the amd-smi GPU / gfx-token / ROCm-version detection
branch was effectively dead (masked only because the un-gated WMI
name->gfx inference still ran). Reproduced on PS 5.1.26100.
Rewrite the helper to use [System.Diagnostics.Process]::Start with a
ProcessStartInfo (UseShellExecute=false), whose .ExitCode is reliable,
with async stream reads (ReadToEndAsync) to avoid a pipe-buffer deadlock
and WaitForExit(timeout) to bound a flaky amd-smi. __COMPAT_LAYER=
RunAsInvoker (inherited via the process env) still suppresses the
auto-elevation / DiskPart prompt. Also drops the temp files and the
empty-ArgumentList edge case. Verified: exit code propagates
(7 -> $LASTEXITCODE=7), output captured, env restored.
Also fix the gfx1100 name pattern `RX 7700(?! S)` -> `RX 7700(?!S)` so the
spaceless retail name "RX 7700S" is correctly excluded (it belongs to the
gfx1102 row). Both found by PR review.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Address PR review follow-ups (install.sh table, update path, tests, WSL)
From the multi-agent PR review:
- install.sh: sync the AMD name->arch table with install.ps1 / setup.ps1
(the bash table had drifted to the old narrow patterns). Adds RDNA 2
(gfx103X), workstation PRO W SKUs, and more Strix Halo/Point names, and
orders gfx1102 before gfx1100 so the spaceless retail name "RX 7700S"
resolves correctly (bash case has no negative lookahead). AMD-ROCm-only:
the name inference stays gated behind _has_amd_rocm_gpu(), so NVIDIA /
CPU / macOS are unaffected.
- setup.ps1: the "dependencies up to date" fast path skipped the torch
reinstall, so an existing user who had CPU torch (installed before
ROCm-wheel support) stayed stuck in CHAT_ONLY. Now, when an AMD gfx arch
is known AND the installed torch is CPU-only, don't skip -- force the
dependency pass so the ROCm wheels install.
- scripts/install_rocm_wsl_strixhalo.sh: resolve the real /opt/rocm dir
instead of hardcoding ROCM_VER for LD_LIBRARY_PATH / the librocdxg
symlink (breaks if amdgpu-install lays ROCm under a patch-version dir);
add a LIBROCDXG_REF pin knob and a "verified against" freshness header.
- tests/studio/install/test_pr5940_followups.py: cover _hf_resolve_url_parts,
_fetch_validation_model_bytes (hf path + urllib fallback), run_capture's
Windows-only amd-smi RunAsInvoker injection, and install.ps1 vs setup.ps1
name-table parity (catches future drift). 14 tests, all passing.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix DiskPart UAC prompt: skip amd-smi on Windows without a HIP SDK
On Windows, amd-smi re-initialises the ROCm runtime on every invocation
(even `amd-smi version`) and, on hosts without a working HIP runtime
(consumer APUs/dGPUs with only the Adrenalin driver), elevates a child
process at runtime -- popping a UAC/DiskPart prompt. amd-smi's own
manifest is asInvoker, so __COMPAT_LAYER=RunAsInvoker cannot suppress
that runtime elevation (verified: even `amd-smi version` hangs and
times out with RunAsInvoker set).
Replace the ineffective RunAsInvoker-only approach with a real gate:
only spawn amd-smi on Windows when a HIP SDK is detectable (hipinfo
present, so amd-smi runs un-elevated) or the user opts in with
UNSLOTH_ENABLE_AMD_SMI=1. The gfx arch is already resolved from WMI
name inference (forwarded via --rocm-gfx), so ROCm wheel + lemonade
llama.cpp selection is unaffected. Linux/macOS amd-smi never elevates
and is untouched (no regression). RunAsInvoker is kept as harmless
belt-and-suspenders for tools that DO use manifest elevation.
Applied consistently across:
- studio/backend/utils/hardware/amd.py (runtime GPU polling)
- install.ps1, studio/setup.ps1 (install-time detection)
- studio/install_llama_prebuilt.py (prebuilt arch probe + version)
- studio/install_python_stack.py (ROCm version + arch probe)
Verified live on AMD Radeon 8060S (gfx1151), native Windows: fresh
install detects the GPU, installs ROCm torch (torch.cuda.is_available()
True), launches Studio with no DiskPart prompt, and inference, tool
calling, web search, LoRA finetuning, and GGUF export all run on the GPU.
Tests: add 6 _amd_smi_allowed() gating tests + PowerShell-installer gate
assertions; update the three amd-smi monitoring tests to opt in (they
mock amd-smi as available). Full suite: 267 passed, 2 skipped.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install.sh: helpful WSL message when the GPU isn't exposed to ROCm
In WSL, an AMD GPU's ROCm-on-WSL runtime is only available with a recent
Adrenalin driver AND a distro AMD supports (currently Ubuntu 24.04). When
neither is in place, GPU detection (rocminfo/_has_amd_rocm_gpu) finds
nothing and we silently fall back to CPU.
Add an actionable hint in the CPU-fallback path, shown only on WSL and
only AFTER detection has already failed -- so it is forward-compatible:
the moment a driver/distro DOES expose the GPU (e.g. if AMD later adds
Ubuntu 26.04 support), detection succeeds and the hint never fires. The
message:
- notes a GPU is plumbed in (/dev/dxg) but no ROCm runtime is exposed,
- lists the two prerequisites (Adrenalin driver + Ubuntu 24.04),
- if the distro is not 24.04, says AMD may not support it yet,
- tells the user to `wsl --install Ubuntu-24.04` and re-run,
- links AMD's ROCm-on-WSL guide + the experimental Strix Halo helper.
Verified live: on Ubuntu-24.04 the hint shows (version-warning omitted)
and the CPU install completes; on Ubuntu-26.04 the extra "this distro may
not be supported" line appears and points to 24.04.
Also fix the experimental scripts/install_rocm_wsl_strixhalo.sh: AMD's
repo.radeon.com/amdgpu-install/ is indexed by unified installer version
(30.30, 31.30, ...), NOT ROCm version, so the hard-coded
amdgpu-install/7.2.0/ path 404'd. Scan the installer dirs newest-first
for a noble .deb matching the target ROCm major.minor (ROCm 7.2 ->
30.30.x/amdgpu-install_7.2.x), falling back to the newest available.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* WSL: fix shortcut collision + pin ROCm-on-WSL driver reqs from AMD docs
Two WSL-related fixes informed by AMD's official ROCm-on-WSL docs and
field reports for Strix Halo / Ryzen AI Max+ (Radeon 8060S, gfx1151):
1. Shortcut collision (real bug). install.sh's WSL branch wrote
"Unsloth Studio.lnk" to the SAME Desktop / Start Menu folder as the
native-Windows installer (install.ps1 New-StudioShortcuts). Running
install.sh in WSL therefore silently retargeted the native shortcut at
the WSL launcher (wt.exe -> wsl.exe), so the desktop/start-menu icon
stopped launching native GPU Studio. Now the WSL shortcut uses a
DISTINCT name -- "Unsloth Studio (WSL - <distro>).lnk" -- and fetches
the Unsloth .ico to %LOCALAPPDATA%\Unsloth Studio so it shows the
proper icon. Native and WSL shortcuts now coexist.
2. Precise ROCm-on-WSL prerequisites. Research (AMD radeon-ryzen WSL
compatibility matrix, gianni.rosagallina.com Feb-2026 guide,
ROCm/ROCm#4952/#5509/#6022) confirms WSL GPU on Strix Halo requires
AMD Adrenalin Edition >= 26.1.1 (26.2.2+ is the first production
ROCDXG/WSL release) + ROCm 7.2.1 + Ubuntu 24.04; an older driver does
not inject the ROCm/DXG runtime into /usr/lib/wsl/lib, so rocminfo sees
only the CPU. install.sh's WSL hint and the experimental
install_rocm_wsl_strixhalo.sh header/preflight now state the exact
driver version (was a guessed ">=26.3.1"), bump ROCM_VER to 7.2.1, link
AMD's radeon-ryzen docs, and document the known librocdxg caveat that
usable VRAM is currently capped at the .wslconfig memory setting.
bash -n clean; install test suite 267 passed, 2 skipped.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* installer: hint when the AMD driver is too old for ROCm-on-WSL
Adds a detect-and-guide hook for the optional WSL-GPU path. An AMD GPU on
native Windows can also be used inside WSL2, but only with AMD Adrenalin
Edition >= 26.2.2 (the first production ROCDXG/WSL release). Native Windows
GPU works with any recent driver, so this is purely about enabling the WSL
path.
We intentionally do NOT auto-install the driver: AMD referrer-gates driver
downloads (scripted curl/Invoke-WebRequest are blocked) and does not publish
Adrenalin via winget, so no installer can reliably fetch it -- and silently
swapping a live display driver is risky. Instead we point the user at AMD's
official download page (one click), after which the existing WSL detection
lights up automatically.
- install.ps1: new Show-AmdWslDriverHint -- when an AMD GPU is present and the
installed driver predates the 26.2.2 release (DriverDate < 2026-02-01),
print a concise tip with the AMD download URL. Handles DriverDate as either
a CIM DateTime or a WMI string. Suppress with UNSLOTH_SKIP_AMD_DRIVER_HINT=1.
- install.sh (WSL hint): add the direct Adrenalin 26.2.2 download URL and note
that AMD downloads are referrer-gated (open in a browser).
Verified: hint fires on a Sept-2025 driver, auto-suppresses on >= 2026-02-01;
install.ps1 parses; install.sh bash -n clean; suite 267 passed, 2 skipped.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* install.ps1: refresh shell icon cache after creating the shortcut
After writing the Desktop / Start Menu .lnk, nudge Explorer to refresh
its icon (ie4uinit.exe -show). Without this, a stale icon cache can show
a blank shortcut icon until the next explorer restart -- most visible
when a shortcut of the same name was rewritten (e.g. a native install
followed by a WSL install, which previously shared the name; now they use
distinct names, but the cache nudge makes the icon appear immediately
regardless). Best-effort and wrapped in try/catch so it never fails the
install. The bundled unsloth.ico itself is valid (verified it renders).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* setup.ps1: don't silently CPU-build llama.cpp on an AMD GPU
For AMD, GPU acceleration comes from the lemonade ROCm prebuilt (it bundles
the ROCm runtime, no HIP SDK needed) and is the preferred/default path. The
source-build fallback is CPU-only -- a HIP/ROCm *source* build would need the
full HIP SDK + ROCm clang toolchain, which the prebuilt exists to avoid.
Previously, if an AMD-GPU host ever fell through to the source build (e.g. the
prebuilt could not be downloaded), it printed "building llama.cpp (CPU-only,
no NVIDIA GPU detected)" and quietly produced a CPU binary -- masking the lost
GPU acceleration. Now that case emits a loud [WARN] explaining the GPU prebuilt
is the AMD path and how to restore it (re-run / check network / set
UNSLOTH_LLAMA_RELEASE_TAG), so AMD never silently degrades to CPU.
No behavior change on the happy path: AMD still gets the GPU prebuilt (verified
on gfx1151: ggml-hip.dll bundled, ~80% GPU compute during inference). NVIDIA
(CUDA source build) and CPU-only hosts are unchanged.
setup.ps1 parses; install suite 267 passed, 2 skipped.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* uninstall: remove shared llama.cpp build, kill lock-holders, match WSL shortcut
Three gaps found by running a real uninstall on a native-Windows + WSL host;
all fixes are scoped to Unsloth-owned paths and no-op on the other pathways
(env/custom-root, NVIDIA/AMD/CPU, Mac) so nothing else regresses.
uninstall.ps1:
- Remove the default-mode SHARED llama.cpp build + cache. setup.ps1 installs
them at ~/.unsloth/llama.cpp and ~/.unsloth/.cache -- SIBLINGS of studio,
not under it -- so deleting <studio> left hundreds of MB behind. Now removed
explicitly, then ~/.unsloth is dropped ONLY if empty (never nukes unrelated
content). No-op in env/custom mode (llama.cpp nests under the custom root,
removed already) and when absent. UNSLOTH_LLAMA_CPP_PATH (user-owned) is kept.
- New _StopProcessesLockingRoots: _StopStudioProcesses only matched the venv
unsloth/python/studio exe, so it missed (a) llama-server.exe under llama.cpp
and (b) an orphaned multiprocessing python fork that ran from the SYSTEM
python but loaded a venv DLL (bitsandbytes) -- on Windows an open DLL handle
blocks the directory delete, leaving a half-removed install. The new helper
kills any process whose image path OR loaded module is under a target root
(module scan scoped to python/unsloth/llama-server names; vendor-agnostic).
- _RemovePath now retries (transient post-kill handle release).
uninstall.sh:
- Remove the default-mode ~/.unsloth/llama.cpp + ~/.unsloth/.cache; rmdir
~/.unsloth only if empty.
- WSL Windows-side shortcut cleanup now matches by TARGET (any
"Unsloth Studio*.lnk" whose target launches wsl.exe), covering both the
legacy "Unsloth Studio.lnk" and the new "Unsloth Studio (WSL - <distro>).lnk"
-- and never removes a native-Windows shortcut (which launches wscript.exe).
uninstall.ps1 parses; uninstall.sh passes sh -n and bash -n.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* install.ps1: invalidate Win11 Start Menu tile cache after creating shortcut
The Start Menu shortcut kept showing a blank/generic icon even after the
Explorer icon-cache rebuild, because Windows 11's StartMenuExperienceHost
keeps its OWN pre-rendered tile-icon cache
(%LOCALAPPDATA%\Packages\Microsoft.Windows.StartMenuExperienceHost_cw5n1h2txyewy\
TempState\TileCache_*.bin + StartUnifiedTileModelCache.dat), separate from
Explorer's iconcache_*.db. ie4uinit and an explorer.exe restart do not touch
it, and they don't recycle the host -- so a rewritten same-name shortcut keeps
showing the first-rendered (often the generic wscript ">") tile until the host
restarts on its own.
Fix: after creating the shortcut, drop only the Start Menu RENDER caches
(TileCache_* + StartUnifiedTileModelCache.dat) and stop StartMenuExperienceHost
(Windows auto-relaunches it), so the tile re-resolves the real icon via the
shell image factory. start2.bin (the user's pinned layout) is deliberately
preserved. Guarded by Test-Path (Windows 10 has no such host -> skipped) and
wrapped in try/catch so it can never fail the install. Windows-only
(install.ps1); no effect on Linux/macOS/Studio.
Verified live: rendering the shortcut via IShellItemImageFactory::GetImage (the
API StartMenuExperienceHost uses) returns the Unsloth sloth icon, color-matched,
after this invalidation -- previously it returned the generic script tile.
install.ps1 parses; install suite 267 passed, 2 skipped.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* ROCm-on-WSL for AMD Strix Halo (gfx1151): auto-setup + runtime enablement
Make Unsloth Studio set up ROCm-on-WSL automatically for AMD Strix Halo
(Radeon 8060S / gfx1151) and use the GPU at runtime, validated end-to-end
on a Ryzen AI Max+ PRO 395 (ROCm 7.2.1 + librocdxg + Adrenalin Apr-2026):
rocminfo enumerates gfx1151, torch.cuda True, ~85.8 GB UMA pool.
Every change is a strict no-op for all other configs (NVIDIA/CUDA,
discrete + native-Linux AMD ROCm, macOS/MLX, Windows, CPU-only, non-Strix
WSL) and can never abort the installer.
- scripts/install_rocm_wsl_strixhalo.sh: rewrite to the validated recipe.
Fixes that would have broken a working box: drop the /usr/lib/wsl/lib
preflight (a working ROCDXG host has only d3d12/dxcore there); remove the
obsolete rocr4wsl step (gone from the 7.2.1 repo; would hard-fail and also
rips out the standard hsa-rocr ROCDXG needs); dynamic librocdxg soname
(was hardcoded 1.1.0; build is 1.2.0); direct apt-repo install; Windows
SDK auto-discovery; persist env to /etc/profile.d + ~/.bashrc; idempotent.
- install.sh: _maybe_bootstrap_rocm_wsl auto-offers/runs the helper when it
detects a Strix Halo APU in WSL (/dev/dxg) with no ROCm runtime, then
loads the env so detection routes to the gfx1151 wheels. Fast-path when
already configured. Fix an inaccurate WSL hint line.
- studio/backend/main.py + worker.py: set HSA_ENABLE_DXG_DETECTION=1
in-process before torch (gated on /dev/dxg AND librocdxg.so), so the
worker uses the GPU even when launched outside a login shell. Mirrors the
existing BNB_ROCM_VERSION injection.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* uninstall: clean up ROCm-on-WSL artifacts + Start Menu tile cache
- uninstall.sh: remove the ROCm-on-WSL helper artifacts -- the librocdxg
build clone (~/.unsloth/librocdxg, which otherwise blocks the empty-dir
rmdir of ~/.unsloth), the throwaway smoke-test venv, the persisted env
(/etc/profile.d/unsloth-rocm-wsl.sh) and the ~/.bashrc block. The system
ROCm userspace is a shared prereq like CUDA and is kept by default;
UNSLOTH_UNINSTALL_ROCM=1 removes it too. No-ops on macOS / non-Strix Linux.
- uninstall.ps1: invalidate the Win11 Start Menu tile cache after removing
the shortcut so its tile disappears promptly (mirrors install.ps1),
preserving start2.bin.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* installer: accurate AMD ROCm messaging (HIP SDK optional, not required)
The Windows installer printed "HIP SDK not found - GPU-accelerated training
unavailable" / "ROCm wheels require the HIP SDK" whenever the HIP SDK was
absent. That is misleading: for a detected AMD GPU arch (gfx1151 etc.),
setup.ps1 installs AMD's bundled-runtime ROCm PyTorch wheels (repo.amd.com)
which ship their own ROCm runtime and do NOT need the HIP SDK -- verified
end-to-end (torch 2.11.0+rocm7.13.0, cuda True, QLoRA training on GPU) on a
Radeon 8060S with no HIP SDK installed.
Gate the GPU-detection + rocm-step messages on a detected gfx arch: when one
is known, state that GPU PyTorch uses bundled-runtime wheels and the HIP SDK
is optional; only when the arch is unknown fall back to the HIP-SDK hint.
Behavior (torch routing) is unchanged; this is messaging only. No-op for
NVIDIA/CUDA, HIP-SDK-present, and CPU paths (they hit earlier branches).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* installer: fix /opt/rocm data-loss + make WSL shortcut create/remove interop-robust
Two fixes from the 3-reviewer regression audit + live testing on a
systemd-enabled WSL distro (interop disabled):
F1 (data-loss, install_rocm_wsl_strixhalo.sh): the /opt/rocm symlink-repair
could force-delete a pre-existing REAL ROCm install. The guard only checked
that /opt/rocm is a real directory, not that it is the stray librocdxg stub.
Now it only touches /opt/rocm when it is NOT a real install (no bin/rocminfo,
bin/hipcc, or .info/version present), and MOVES it aside (rocm.unsloth-stub-bak)
instead of deleting it, so a wrong guess can never lose data.
WSL interop robustness (install.sh + uninstall.sh): both relied on
`command -v powershell.exe`, which is true even when WSL interop cannot EXECUTE
it (on systemd distros powershell.exe fails with "Exec format error"). Result:
the WSL shortcut silently failed to create (install) and to remove (uninstall).
- uninstall.sh: test that powershell.exe actually runs; if not, remove the
"Unsloth Studio (WSL...).lnk" files directly via drvfs (/mnt/<drive>), which
works without interop. The name is WSL-install-specific, so a native install's
"Unsloth Studio.lnk" is never touched.
- install.sh: when the shortcut cannot be created, warn with the manual launch
command + how to re-enable interop, instead of failing silently.
No behavior change on the interop-on path. The regression audit otherwise found
no regressions on Linux/Mac/Windows/CPU/NVIDIA install paths.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* install.sh: fast-path fully restores ROCm-on-WSL env when the drop-in is gone
Reinstall regression found by uninstall->reinstall testing: after a Studio
uninstall that removed /etc/profile.d/unsloth-rocm-wsl.sh but KEPT the shared
ROCm (the default), a non-login reinstall hit the bootstrap fast-path
(librocdxg present) and its else-branch only set HSA_ENABLE_DXG_DETECTION --
NOT PATH/LD_LIBRARY_PATH. So rocminfo was not on PATH, GPU detection failed,
and the installer fell back to CPU-only PyTorch.
Fix: when librocdxg is present but the env drop-in is missing, restore the
FULL env inline (HSA + TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL + PATH +
LD_LIBRARY_PATH) so rocminfo is found and detection routes to the GPU, and
recreate /etc/profile.d/unsloth-rocm-wsl.sh so future shells and the Studio
worker get it too. No change to the env-present fast-path or any other host.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* installer: clear Explorer icon cache so shortcut icons aren't blank
Root cause of the persistent blank Desktop + Start Menu icons: Explorer caches
each shortcut's icon in iconcache_*.db and does NOT re-read the .ico when a
same-name .lnk is recreated across reinstalls. The .ico and .lnk are correct
(the shell renders them non-blank via IShellItemImageFactory; the .ico has real
image data at 16/32/48/128 px), but the stale cache entry wins. The previous
fix only ran a weak `ie4uinit -show` + the Start Menu tile-cache clear -- it
never invalidated Explorer's icon cache, so the desktop icon stayed blank.
Fix (native install.ps1 New-StudioShortcuts AND the WSL shortcut path in
install.sh):
- ie4uinit -ClearIconCache (thorough; replaces -show as the primary refresh)
- SHChangeNotify(SHCNE_ASSOCCHANGED) to force a live desktop/taskbar refresh
WITHOUT restarting explorer
- keep the Win11 Start Menu tile-cache invalidation (and add it to the WSL
shortcut path too, preserving start2.bin)
Non-disruptive (no explorer restart). install.ps1 parses clean; install.sh
passes bash -n + dash -n; the heredoc-generated WSL PowerShell parses clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* installer: per-item SHChangeNotify(UPDATEITEM) reliably fixes blank icons
The blank Desktop/Start Menu shortcut icons are a stale Explorer PER-ITEM icon
cache: when a same-name .lnk is recreated across reinstalls, Explorer caches the
previously-resolved (often generic "white page") icon for that item and won't
re-extract the .ico on its own. The .ico and the .lnk's IconLocation are correct
(every icon API renders the sloth) -- only Explorer's cached display is stale.
The previous refresh (ie4uinit -ClearIconCache + a GLOBAL SHCNE_ASSOCCHANGED
broadcast) does NOT recover a stale item -- confirmed by reproduction. The
reliable, NON-disruptive fix (no explorer restart) is a PER-ITEM
SHChangeNotify(SHCNE_UPDATEITEM, SHCNF_PATHW, <lnk path>) for each created
shortcut, which forces Explorer to re-read that exact item's icon.
Verified end-to-end: deliberately staled a shortcut to the generic icon, ran the
installer's exact new refresh code, and the sloth icon recovered with NO explorer
restart (confirmed by capturing the live desktop via PrintWindow).
Applied to both native install.ps1 (New-StudioShortcuts) and the WSL shortcut
path in install.sh. Still clears the on-disk icon cache (ie4uinit) and the Win11
Start Menu tile cache (preserving start2.bin).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* uninstall: remove leftover llama.cpp .staging root so ~/.unsloth is cleaned
The llama.cpp atomic-install staging root (install_llama_prebuilt.py
INSTALL_STAGING_ROOT_NAME=.staging) is a sibling of the llama.cpp install
dir (~/.unsloth/.staging in default mode). It is normally pruned after a
successful activate, but an interrupted or retained build can leave a
<name>.staging-XXXX tree behind. The uninstallers removed llama.cpp and
.cache but not .staging, so the final empty-dir cleanup of ~/.unsloth failed
and the directory lingered. Reproduced on WSL (Ubuntu-24.04) where an empty
llama.cpp.staging-XXXX dir kept ~/.unsloth alive after uninstall.
Remove ~/.unsloth/.staging in both uninstall.sh and uninstall.ps1. No-op in
env/custom mode (staging nests under the custom root removed already) and
when absent. Cross-platform fix (the staging logic is platform-agnostic).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* installer: WSL-absent hint + fix here-string lint false positive
install.ps1: in the AMD WSL-ROCm driver hint, detect when wsl.exe is absent
and add a one-line "wsl --install -d Ubuntu-24.04" pointer so a Strix Halo
user with no WSL yet gets an actionable next step (the hint previously assumed
an Ubuntu-24.04 distro already existed). Best-effort, informational only.
test_rocm_support.py: test_no_here_strings did a crude substring check that
false-positived on the conda-style block marker
printf '# <<< Unsloth ROCm-on-WSL (gfx1151) <<<' -- a string literal written
into the /etc/profile.d drop-in, also used as a sed delimiter pair by
uninstall.sh, not a here-string. Strip quoted spans before the check so the
lint still catches a real here-string operator but ignores quoted literals.
install.sh remains POSIX-clean (sh -n / dash -n / bash -n all pass).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* installer: address PR review comments (gfx1150 mapping, amd-smi opt-out, WSL bootstrap, SDK path, make)
Apply the valid bot review findings on #5940; reject the ones that don't hold.
Fixed:
- AMD name->gfx table (setup.ps1 + install.ps1): Radeon 890M and Ryzen AI 9 HX
370/375 are Strix POINT (gfx1150), not Strix Halo (gfx1151). Move 890M / HX 37x
/ AI 9 HX to the gfx1150 row and drop the bogus HX 38x pattern (no such Strix
Halo SKU). Matches the runtime classifier in worker.py (890M/880M -> gfx1150;
8060S/8050S -> gfx1151). Prevents Strix Point hosts from getting the wrong ROCm
prebuilt/wheels.
- amd-smi opt-out (setup.ps1 + install.ps1): an explicit UNSLOTH_ENABLE_AMD_SMI=
0/false/no/off now wins over the HIP-SDK heuristic, so a host with a HIP SDK
binary but a broken runtime no longer gets the DiskPart/UAC prompt the opt-out
exists to avoid.
- amd-smi warning probes (install_python_stack.py): _has_rocm_gpu and
_detect_amd_gfx_codes now gate amd-smi behind _amd_smi_allowed() (and pass
_amd_smi_env()), closing the last unguarded amd-smi spawn on Windows.
- WSL ROCm bootstrap (install.sh): the "already-usable ROCm?" early return now
requires rocminfo to enumerate the real gfx1151 agent instead of the generic
_has_amd_rocm_gpu (whose broad gfx[1-9][0-9] match accepts a fallback
"gfx11-generic" ISA), so a Strix Halo box missing the ROCDXG bridge is no longer
skipped. The shared helper is untouched (no gfx90a regression).
- install_rocm_wsl_strixhalo.sh:
* Quote-safe Windows SDK discovery: the old for-in-$(ls -d "...Program Files
(x86)/...") word-split on the space and never matched; use find + read loop.
* Add `make` to apt prereqs (cmake only recommends it; minimal images lacked it
and the librocdxg `make -j` build failed).
* Verification requires gfx1151 exactly (not gfx1[0-9]) so a generic ISA or an
unrelated RDNA GPU can't pass while the real GPU is absent.
Reviewed but NOT changed:
- "Forward inferred ROCm arch without HasROCm" (setup.ps1): already correct --
--rocm-gfx is forwarded under `if ($script:ROCmGfxArch)`, not `if ($HasROCm)`.
- "Route inferred arch into install.ps1 torch path": not a bug -- install.ps1
installs CPU torch as a base by design and setup.ps1 swaps in the ROCm wheel for
the inferred arch (gate `($HasROCm -or $ROCmGfxArch) -and cpu`); verified live
the native install ends on torch 2.11.0+rocm7.13.0.
- "$p null guard after Start-Process" (install.ps1/setup.ps1): redundant -- the
amd-smi runner uses [Process]::Start wrapped in try/catch, so a null process
already returns "" with LASTEXITCODE=1 (no uncaught exception).
- "ls -> find for /usr/lib/wsl/lib" (gemini): stale -- that heuristic was removed;
only a comment about it remains.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* installer(rocm-wsl): auto-install the Windows 11 SDK via winget (fewer manual steps)
librocdxg's build needs the Windows SDK 'shared' headers on the Windows host.
Previously the helper just die()d with "install the Windows 11 SDK and re-run" if
they were missing -- a manual prerequisite that broke the otherwise-seamless
`curl ... install.sh | sh` one-liner on Strix Halo.
Now, when the headers aren't found, the helper installs the Windows 11 SDK on the
Windows host from inside WSL via winget (powershell.exe interop), then
re-discovers them. The SDK installer elevates -> ONE UAC prompt on the Windows
desktop; the headers appear under /mnt/c immediately (drvfs is live, no reboot).
The user already consented to the ROCm-on-WSL setup, so no extra prompt is added
beyond the OS UAC gate.
- New _find_win_sdk (space-safe find of the newest installed SDK 'shared' dir)
and _install_windows_sdk_via_winget helpers.
- winget IDs tried newest-stable first: Microsoft.WindowsSDK.10.0.26100, then
.22621. The presence of the headers (re-check) is the source of truth, not
winget's exit code. </dev/null so winget never consumes a piped `curl|sh` stdin.
- Best-effort + non-fatal: interop-off / no-winget / declined-UAC all fall
through to the existing clear manual-install die(). Opt out with
UNSLOTH_SKIP_WIN_SDK_INSTALL=1.
Removes the last avoidable manual step from the WSL Strix Halo path; only the AMD
Adrenalin driver (AMD referrer-gates the download) remains manual. Verified
_find_win_sdk resolves the spaced "Program Files (x86)" path; bash -n clean; all
winget flags validated against `winget install --help`.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* installer(amd): gate install-time amd-smi probe to fix DiskPart UAC prompt
install_python_stack.py's Windows "AMD GPU detected but ROCm torch missing"
warning probe ran `amd-smi list` whenever amd-smi was on PATH -- and amd-smi
ships in C:\Windows\System32 with the AMD Adrenalin driver -- without the
_amd_smi_allowed() gate that every other amd-smi call site in the file uses.
On Adrenalin-only hosts (no HIP SDK) amd-smi elevates a child at runtime and
pops a UAC/DiskPart prompt that __COMPAT_LAYER=RunAsInvoker cannot suppress
(amd-smi's manifest is asInvoker). The probe also ran before the
ROCm-torch-installed check, so it fired on every Windows AMD install.
Gate it behind _amd_smi_allowed() and pass _amd_smi_env(), matching
_has_rocm_gpu()/_detect_amd_gfx_codes(). When skipped, the only loss is the
best-effort "AMD GPU detected" note on HIP-SDK-less hosts.
Adds a per-function AST regression test asserting every function in
install_python_stack.py that names the amd-smi command and spawns a subprocess
also references _amd_smi_allowed() (flags the pre-fix code; passes after).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* studio(cli): fix `unsloth studio stop` crashing on Windows
`stop` used the POSIX `os.kill(pid, 0)` liveness probe, but on Windows
CPython raises OSError (WinError 87, "The parameter is incorrect") for
*every* pid -- alive or dead. `stop` only catches ProcessLookupError /
PermissionError, so the OSError propagated and the command crashed with
a traceback before ever reaching its (correct) `taskkill /F` path.
Add a cross-platform `_pid_alive(pid)` helper (tasklist on Windows,
signal-0 elsewhere) and use it for both the pre-check and the post-kill
wait loop. The actual kill path is unchanged.
Verified on Windows (Python 3.13): os.kill(pid,0) raises WinError 87 for
both a live and a dead pid; `_pid_alive` returns True/False correctly and
the full stop() flow (alive -> taskkill -> dead -> "stopped") passes
end-to-end against a throwaway process.
Adds tests/studio/test_cli_studio_stop_windows.py (AST guard against a
bare os.kill(pid,0) liveness probe + mock-only _pid_alive behaviour for
the win32 tasklist branch and the POSIX signal-0 branch).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* installer(amd): fix install.sh name->arch table misrouting Strix Point to gfx1151
The bash name->arch inference table in install.sh placed Strix Point
identifiers (Radeon 890M, "Ryzen AI 9 HX 370/375", "AI 9 HX") in the
gfx1151 (Strix Halo) row, diverging from the install.ps1 / setup.ps1
PowerShell tables which correctly map them to gfx1150. It also carried a
stray "HX 38" token absent from the PowerShell source-of-truth.
Align install.sh with the PowerShell tables:
gfx1151 row: 8060S|8050S|8040S|Strix Halo|Ryzen AI Max|AI Max
gfx1150 row: 890M|880M|860M|840M|Strix Point|Krackan|HX 37|AI 9 HX|...
Impact is low (the bash table only feeds the display label _gpu_disp_gfx
and the "set UNSLOTH_ROCM_GFX_ARCH=..." hint; wheel selection is driven
by the detected ROCm version, not this name string) but a Strix Point
user would otherwise see/copy the wrong gfx arch.
Add a parity test (test_install_sh_name_arch_agrees_with_ps_for_strix_and_non_amd)
that parses install.sh's case table and asserts Strix Halo->gfx1151,
Strix Point->gfx1150, RX 7700S->gfx1102, and NVIDIA/Intel->no match,
cross-checking against install.ps1 (the previous parity test only
compared install.ps1 <-> setup.ps1, missing install.sh).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* setup.ps1: keep prebuilt-llama ownership guard within the test's block window
The AMD additions to the prebuilt-llama.cpp block (the windows-hip vs
windows-cpu existing-install kind validation) pushed the
install_llama_prebuilt.py invocation to ~1999 chars after the
"installing prebuilt llama.cpp bundle (preferred path)" anchor, right at
the edge of the 2000-char window that
test_setup_ps1_prebuilt_llama_cpp_has_ownership_guard slices -- so the
helper string was truncated and the test failed with "substring not
found" (CI: Repo tests (CPU)).
The ownership-guard invariant (Assert-StudioOwnedOrAbsent precedes the
install_llama_prebuilt.py call) was already satisfied; only the proximity
to the anchor regressed. Move the "installing prebuilt..." substep to
immediately before the install (after the existing-install pre-cleanup),
which also reads better (validate/clean existing -> then "installing"),
shrinking anchor->helper from 1999 to 413 chars. Behaviour is unchanged
(console message ordering only).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* install.sh: auto-run Strix Halo ROCm-on-WSL setup by default
`curl -fsSL https://unsloth.ai/install.sh | sh` should make a Strix Halo
(gfx1151) GPU usable inside WSL with no extra commands. Previously the
ROCm-on-WSL bootstrap was opt-in: it required UNSLOTH_ROCM_WSL_AUTO=1 or an
interactive [Y/n] at a TTY, and silently skipped under a pipe (no /dev/tty),
so the piped one-liner never set the GPU up automatically.
Flip it to auto-by-default for the single narrow case the existing guards
allow (WSL + Strix Halo + /dev/dxg + no usable ROCm yet) -- exactly the GPU
setup the user ran the installer for. Opt out with
UNSLOTH_SKIP_ROCM_WSL_SETUP=1. The Tauri desktop app keeps its own consent UI
(only auto-runs when it passes UNSLOTH_ROCM_WSL_AUTO=1). All hardware/OS
guards are unchanged, so non-Strix / non-WSL / NVIDIA / native-Linux / macOS /
CPU paths are unaffected.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* PR comments: condense to be succinct (comments/docstrings only)
Shorten the verbose explanatory comments and docstrings this PR added across
the installer, scripts, backend shims, CLI, and tests -- tighter, fewer lines,
while preserving every non-obvious "why" (os.kill WinError 87, amd-smi
RunAsInvoker/UAC, /dev/dxg + librocdxg gating, the ROCm-on-WSL bootstrap guard
chain, ownership guards, etc.). No executable code, string literals, messages,
or behavior changed.
Verified comments-only: docstring-normalized AST equality (Python, 9 files),
non-comment token equality (PowerShell, 3 files), comment-stripped diff +
sh -n / bash -n (shell, 3 files). Behavior re-confirmed: get_torch_index_url +
gfx name->arch table 44/44 under dash & bash; rocm_support / pr5940_followups /
cli_studio_stop tests green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Installer: address PR review (amd-smi opt-out, pipefail, multi-distro, non-root)
Fixes valid findings from the Codex/Gemini PR review:
- install.ps1 / setup.ps1: gate the `amd-smi version` ROCm-version fallback with
$amdSmiAllowed so UNSLOTH_ENABLE_AMD_SMI=0 opt-out is honored (the device
probe was gated but this fallback wasn't), avoiding the DiskPart/UAC prompt.
- install_rocm_wsl_strixhalo.sh: make the post-verification rocminfo summary
best-effort (|| true) so head's early pipe-close under `set -o pipefail` can't
fail the bootstrap after gfx1151 was already enumerated; pin the Windows SDK
`winget install` to --source winget (matches the msstore-cert fix rationale).
- install.ps1: python.org fallback installs the py launcher per-user
(InstallLauncherAllUsers=0, avoids admin), and derives the fallback full
version from the requested minor so a non-default UNSLOTH_PYTHON (e.g. 3.12)
isn't silently replaced with 3.13 when the listing is unreachable.
- install.sh: recreate /etc/profile.d/unsloth-rocm-wsl.sh via `sudo tee` for a
non-root reinstall (a plain redirect failed silently, dropping the ROCm env).
- uninstall.sh: scope WSL Windows-side shortcut removal to the current
WSL_DISTRO_NAME (per-distro name or -d "<distro>" arg) so uninstalling one
distro no longer deletes other distros' launchers.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* Studio ROCm Windows: fix field-reported issues from Strix Halo testers
Four fixes from PR #5940 field reports (Win11 native, gfx1151):
1. bitsandbytes arch-probe spam: bnb's get_rocm_gpu_arch() runs
hipinfo.exe via subprocess PATH at import; the AMD torch wheel ships
hipInfo.exe in the venv Scripts dir, which is only on PATH for
activated venvs. Every bnb import logged "Could not detect ROCm GPU
architecture: [WinError 2]" ERROR + WARNING (even with the HIP SDK
installed, whose bin dir is not on PATH either). Prepend the Scripts
dir to PATH before bnb imports in main.py, worker.py, and
install_python_stack.py, gated on the file existing (only AMD wheels
ship it). Verified on gfx1151: ROCM_GPU_ARCH now resolves to gfx1151
with zero errors.
2. OOM-guard double-tax on native Windows unified APUs: mem_get_info's
total is the WDDM budget the driver grants HIP (BIOS carve + ~half
of remaining RAM) -- the OS share is already outside it. The 0.80
unified cap on top denied loads that fit (field report: 48.49 GiB
budget -> "38.79 GiB allowed" OOM for a 47.29 GiB load with 48.08
free). Use 1.0 on win32 unified; Linux keeps 0.80, discrete 0.90.
3. "Missing VRAM" confusion: log the WDDM budget vs physical RAM with
the fix (BIOS UMA frame buffer / AMD Software Variable Graphics
Memory) when the grant is under 75% of RAM, so a 48 GiB cap on a
96 GiB box reads as policy, not a Studio bug.
4. llama-server fit-step crash (Qwen3.6-27B-MTP + mmproj, lemonade
gfx1151): --fit defaults to 'on' upstream, so the fit step runs even
when Studio already placed the model via -ngl -1, and aborts in
ggml-cuda.cu on some ROCm hosts. Retry the spawn once with --fit off
when the server crashes during startup and Studio's own VRAM math
had placed the model (never when use_fit or an explicit fit flag was
passed). Also keep the TAIL of crash output in the error log (the
diagnostic line prints last; head-truncation cut exactly that) and
reference the full on-disk log.
Verified live on Radeon 8060S: bnb import clean, Qwen3.5-4B-MTP loads
and generates through the new spawn loop, stub-crash retry appends
--fit off and recovers, fraction probes confirm WDDM overcommit and
sub-1.0-only enforcement on current AMD wheels.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio ROCm Windows: GPU-name fallbacks so nothing depends on amd-smi
amd-smi does not reliably exist on Windows: the HIP SDK never ships a
CLI, inbox Windows Update drivers do not, and only some full Adrenalin
packages drop amd-smi.exe into System32 (field report: fresh Win11 +
Adrenalin + HIP SDK, still no amd-smi anywhere). Make every consumer
work without it:
- install_python_stack._detect_windows_gfx_arch: two new probes after
hipinfo/amd-smi -- (2b) the venv Scripts hipInfo.exe shipped by AMD
torch wheels (drives `studio update` on driver-only hosts), and (4) a
last-resort GPU marketing-name -> gfx table via WMI
(Win32_VideoController), mirroring setup.ps1's $nameArchTable so a
standalone repair resolves the arch with zero AMD tooling installed.
- install_llama_prebuilt._resolve_exe: also probe the venv Scripts dir
so a standalone rerun finds hipInfo.exe without HIP_PATH.
- hardware/amd.py _run_amd_smi: which() guard before spawning --
absence now disables the poller in one step instead of burning the
3-strike circuit breaker on FileNotFoundError; corrected the stale
comment claiming Adrenalin ships amd-smi.
Simulated against the real detection functions on gfx1151: amd-smi
absent, present-but-crashing (exit 1), present-but-hanging (60s sleep
vs 5-10s probe timeouts), and hard opt-out -- all resolve gfx1151, no
exceptions, bounded time. Full adversarial install (broken amd-smi
stub first on PATH + UNSLOTH_ENABLE_AMD_SMI=1, fresh uninstall first):
exit 0, name-table arch inference, lemonade gfx1151 b1292 prebuilt,
torch 2.11.0+rocm7.13.0 cuda_avail=True on the 8060S, Studio boots
healthy and stops cleanly.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: per-attempt llama-server log names + amd-smi test portability
Found by cross-platform simulation of the --fit off retry (Windows +
Linux sandboxes, real load_model with stub servers):
- llama-server log filename now carries the spawn-attempt index. The
retry can respawn within the same epoch second; reusing the name
opened the same file with "w" and truncated the crash log the retry
warning had just pointed the user at (proven with a frozen
time.time: one file, crash evidence gone; with the suffix both
attempts keep their logs). Regression-pinned in
test_llama_cpp_wait_for_health.py.
- test_amd_primary_gpu_with_mock now mocks shutil.which alongside
subprocess.run: the amd-smi absence guard which()-checks before
spawning, so on hosts without a real amd-smi (Linux CI, driver-only
Windows) the subprocess mock was never reached and the test failed.
Surfaced by running the suite in a clean Linux sandbox.
Simulation coverage on both OSes: 67-case platform/edge matrix
(real shipped code blocks under win32/linux/darwin spoofs: OOM-guard
fractions + VGM-hint boundary, bnb PATH-prepend gates, retry
eligibility incl. equals-forms and decoy tokens, GPU-name table
adversarial set, WMI fallback without powershell, monitor absence
semantics), 6-scenario live retry matrix (crash-once/crash-always/
exit-zero/explicit-fit/hang/log-collision) against real llama-server
spawns on Windows and WSL (GPU success legs on the 8060S), and a
3-engine browser matrix (chromium/firefox/webkit) driving the live
backend's health + authed /v1 chat completion.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: classify unified-memory via props.is_integrated first
Align the ROCm OOM-guard classifier with PR #5988's UMA gate: consult
hipDeviceProp_t.integrated (props.is_integrated) before the hardcoded
arch set. Strictly additive -- truthy upgrades to unified; 0/absent
falls through to the existing gfx1150/gfx1151 + device-name logic, so
wheels that omit or zero the field cannot downgrade the known APU set.
Extends correct unified-cap treatment to APUs outside that set (e.g.
gfx1103 Phoenix iGPUs) and keeps Studio's two unified-memory consumers
on one driver signal. Verified live on gfx1151 (is_integrated == 1 on
the AMD Windows wheel -> ('gfx1151', True) via the new path).
* AMD detection: probe rocminfo with HSA_ENABLE_DXG_DETECTION and sync setup.sh gfx table
Fleet validation on a Strix Halo WSL2 box showed the system rocminfo
(HSA 1.18, ROCm 7.2.1) only enumerates the GPU over /dev/dxg when
HSA_ENABLE_DXG_DETECTION=1, and that rocminfo can sit at /opt/rocm/bin
off PATH outside login shells. Detection probes that miss either of
these report no GPU on a working ROCDXG host and select the CPU build
even though the lemonade bundle offloads fine (95.7 tok/s measured vs
64.5 CPU on the same laptop). Seed the env (a no-op on bare metal) and
the PATH fallback in install.sh, studio/setup.sh, and the installer's
Linux rocm probe, mirroring what main.py/worker.py already do for the
runtime.
Also sync studio/setup.sh's name->gfx table with install.sh: 890M and
the HX 37/AI 9 HX SKUs are Strix Point (gfx1150, not gfx1151), RX 7700S
must match gfx1102 before the gfx1100 row, and the RDNA2/workstation
rows were missing. New parity test pins the two bash tables together so
they cannot drift again.
* Studio: persist server session logs + native-crash stacks to disk
Field report (Strix Halo, 96 GB UMA carve, WSL and native Windows):
"the studio just terminates without a warning". A native crash in the
GPU runtime kills the process with no Python traceback, and a desktop-
shortcut console closes before anything can be read. The server only
ever logged to the console, so there was nothing to send back.
run_server now tees stdout/stderr to
~/.unsloth/studio/logs/server/server-<ts>-pid<n>.log (console behavior
unchanged; file copy is best-effort), arms faulthandler at the same
file so access violations / SIGSEGV leave a stack trace on disk, and
exports PYTHONFAULTHANDLER=1 so training workers inherit crash dumps
on their captured stderr. Armed before `from main import app` so even
import-time failures leave evidence. Keeps the newest 20 session logs;
opt out with UNSLOTH_STUDIO_NO_FILE_LOG=1. Prints "Session log: <path>"
at startup so users know what to attach.
Verified on this box: a forced real segfault (faulthandler._sigsegv)
leaves the full session output plus "Fatal Python error: Segmentation
fault" and the thread stack in the file while the console shows
nothing; a normal server boot captures the startup banner and serves
health as before.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* AMD probe: honor a pre-set HSA_ENABLE_DXG_DETECTION value
Match the shell helpers, which use the parameter-default form: a user
who exports HSA_ENABLE_DXG_DETECTION=0 to deliberately hide the GPU
from DXG detection should not have the probe override it.
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
Adds a self-contained RAG stack to Studio: knowledge bases with chunked indexing, hybrid (dense + lexical) retrieval, and an automatic first-pass context inject into chat. Embeddings run through a local llama-server GGUF backend (default unsloth/bge-small-en-v1.5-GGUF) with a sentence-transformers fallback. The chat tool loop gains a search_knowledge_base tool, a per-turn re-search cap, and source citation, layered on top of the shared ToolLoopController.
* Studio: fix OpenAI- and Anthropic-compatible API spec compliance
* Studio: fix API spec-compliance gaps on passthrough and streaming paths
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: carry context_length_exceeded through the OpenAI passthrough error path
* Studio: count tool-schema tokens in the Anthropic server-tool stream, and small stream-handling guards
* Studio: guard message_delta usage against None and normalize developer role before proxying
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: honor max_completion_tokens on the external-provider proxy path
* Studio: forward llama-server cached_tokens into OpenAI prompt_tokens_details
* Studio: sanitize messages in count_tokens to match the /v1/messages prompt
* Studio: report max_tokens for truncated tool calls and guard null usage in metadata events
* Studio: drop the request-id middleware (headers aren't declared in either spec)
* Studio: include the required request_id field in Anthropic error bodies
* Studio: honor max_completion_tokens on the audio (TTS / audio-input) paths
* Studio: add the _effective_max_tokens helper and route all max-token sites through it
* Studio: align API compatibility edge cases
* Studio: clarify multi-choice chat support
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: clarify logprobs chat support
* Studio: opt the local chat UI into the streaming usage chunk so the context bar and tok/s repopulate
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: forward seed to llama-server, and fix Anthropic server-tool stop_reason, tool_result id correlation, and parallel-tool execution cap
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: align OpenAI chat completion spec edge cases
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: align backend API compatibility tests
* Studio: honor tool caps and internal stream usage
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: coerce nullable stream usage counts
* Studio: preserve system prompts with developer messages
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
* Studio: make Helper LLM startup pre-cache opt in
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Adds the Studio Hub and download manager: browse Hugging Face models and datasets, download GGUF and safetensors with live progress and cancellation, and manage on-device inventory. The Hub does not require a GPU, so it is available on chat-only hosts.
CI: all substantive checks pass, including the three Core jobs after unsloth-zoo#736. The two red checks are non-code flakes, a transient npm-registry DNS resolution failure in the package scan and one quantized vision-model output assertion whose sibling shards passed.
Trim and tighten code comments and docstrings across the repository. Comment-only: every changed file verified code-identical to main via AST/token comparison.
Trim and tighten code comments and docstrings across studio/ Python. Comment-only: every changed file verified code-identical to main via AST/token comparison.
Raise ruff line-length to 100 and extend the local pre-commit format pipeline (def-signature magic-comma normalization, short multi-line assert collapse, kwarg '=' spacing, blank-line-after-short-import removal, adjacent string-literal / f-string+plain merge, redundant-pass pruning). Every transform re-checks the file AST and is dropped if it would differ; the whole-repo reformat is verified AST-identical per file and idempotent.
* studio: redesign chat composer
Reworks the new-chat composer and the compare composer into a single
rounded pill surface with a softer, lighter look.
- New welcome screen with a time-of-day sloth mascot and a lighter
heading.
- One rounded composer surface with a soft drop shadow. The input grows
inline as you type and collapses back to a single row when cleared.
- Tools and attachments live in a single plus menu; the thinking control
is a compact pill with a reasoning-effort submenu.
- Inlined glyphs for the thinking, send, and dictate controls, kept in
sync across the main and compare composers.
- Toast notifications match the composer surface: no border line, the
same drop shadow, and the same dark surface color, with a ring-less
close button.
- Dark mode: the side-menu shadow blends into the background, hovered
menu rows read clearly, and their roundness matches light mode.
- Composer styles use dedicated unsloth- prefixed classes so compare
mode keeps its own stacked layout.
* studio: sync compare-composer reasoning state and harden compare id
- Compare composer: keep "Preserve thinking" consistent with reasoning,
matching the main composer. Enabling it now turns reasoning on, and
disabling reasoning (the None option or the Thinking toggle) turns it
off, so the invalid "preserve on while thinking off" state can't occur.
- Guard crypto.randomUUID in the Compare action. It is undefined in
non-secure contexts (HTTP over a LAN IP) and would throw; fall back to
a timestamped random id, matching createNavigationNonce.
* studio: reflect pre-selected Search/Code tools when no model is loaded
The Search and Code pills only lit up when the tool was usable right now
(a model loaded and capable), so a tool turned on from the + menu showed
as off in the pill while the menu showed it on. toolsEnabled is persisted
and takes effect once a capable model loads, so the pill should reflect it.
The pills now disable only when a loaded model lacks the capability, and
otherwise reflect the selected state. Applied to the main and compare
composers.
* Studio: link MCP Servers heading to its PR and fix composer pill cursors
Make the "MCP Servers" heading in the chat Configuration sheet link to the
MCP PR, keeping the chevron as the toggle. The label and chevron are rendered
as siblings so we don't nest an <a> inside a <button>.
Also add cursor-pointer to the composer pills and the thinking pill so hovering
a clickable pill shows the hand cursor instead of the default arrow.
* Studio: refine chat composer and add compare-mode parity
- Composer expands to two rows only once the input wraps to a second line,
not on the first keystroke. Re-measure the autosize textarea on the width
swap so expanding no longer leaves a stray blank row.
- Light-mode composer shadow now matches Gemini's soft elevation.
- Plus menu: replace Canvas with a More submenu (Canvas, Compare chat, RAG)
and add Code above MCP. Active Web search/Code items use medium weight.
- Compare mode: the plus side menu, Search/Code toggles, and a Compare exit
pill now match single chat, with the thinking control on the right.
- Projects menu entries link to their tracking PR (#5725).
- Add cursor-pointer to the composer plus button.
* studio: refine composer controls and chat search shadow
- Active tool pills show an x on hover to signal click-to-disable
- Plus button rotates into an x when the tools menu opens
- Composer surface uses a 32px radius and a taller single-line height
- Even, ChatGPT-style spacing between the plus and tool pills in both the single and compare composers
- Send and mic circles resized and spaced, with the arrow centered in the circle
- Chat search box gets a borderless, soft Gemini-style shadow
* studio: size the pill hover x to match the icon it replaces
Cross-engine checks (Chromium, Firefox, WebKit) flagged the active-pill
hover x as a fixed 14px, so it popped smaller than the 19px Code icon.
Fill the glyph slot instead so the x tracks whatever icon it covers.
* studio: do not persist Kimi search/thinking mutual-exclusion in single composer
The single-chat composer flipped the other control off when toggling
search or thinking on Kimi, but without { persist: false }, so it
overwrote the user's saved preference. Match shared-composer and keep
the side effect session-only.
* studio: pointer cursor on model selector trigger and menu items
Add scoped marker classes so the model picker trigger and every
clickable element in its menu (tabs, model rows, delete, eject) show a
pointer cursor; disabled items stay not-allowed.
* studio: pass baseUrl when resolving reasoning caps in single composer
The docked composer omitted baseUrl, so a custom Gemini OpenAI-compat
gateway still advertised the native thinking ladder the backend cannot
honor. Pass selectedExternalProvider.baseUrl like the compare composer
so the resolver hides it.
* studio: grey side-menu hover, green pill hover, thinking hover x
- Plus side-menu items hover grey in light mode, not the green accent
- Thinking pill hovers green like the Search and Code pills
- The plain Thinking toggle shows an x on hover when active, matching
Search and Code; the effort dropdown trigger keeps its bulb
* studio: make the pill hover x a uniform size
The x filled the icon slot, so the wider Code chevron gave a bigger x
than Search and Compare. Pin it to a fixed 15px, centered, so every
pill's x matches.
* studio: broaden chat attachments, fix active hover color, gemini shadow
- Accept svg, source code and many text/config files as drag-and-drop
or picked attachments, matched by extension since their MIME is
unreliable; html keeps its own adapter
- Active (green) side-menu items keep their text and icon color on
hover instead of switching to the accent color
- Composer surface uses Gemini's soft centered shadow 0 0 20px rgba(0,0,0,0.04)
* studio: keep the thinking pill full height when icon-only
The inactive thinking pill has no label, so its flex row collapsed to
the icon height and the hover box looked short. Reserve one text line
(min-height: 1lh + padding) so it matches the Search and Code pills.
* studio: refine composer menu, drop overlay and greetings
- Open the MCP servers dialog directly from the composer plus menu
- Redesign the drag-and-drop affordance Gemini style, drop the badge and border, make the whole chat page a drop target
- Swap in Hugeicons for the RAG, attachment chip and new project icons
- Add time-based randomized welcome greetings, each matched to a fitting sloth
* studio: rename artifacts toggle to Canvas and make it opt-in
- Label the toggle Canvas everywhere, matching the plus menu
- Stop greying out the Canvas menu item; it toggles like the other items
- Only show the Canvas pill in the composer row once it is turned on, since it is less central than Search and Code
* studio: wire Canvas and MCP composer toggles, even out the pill row
- Open the MCP servers dialog from the menu, or toggle MCP on/off once a server is enabled
- Force MCP off when no server is enabled, so the toggle stays honest
- Show Canvas and MCP as opt-in pills that appear in the order they were toggled on
- Expand the composer and light up the pill when Canvas or MCP is on, like Search and Code
- Keep Compare directly after Code in the compare composer
- Use the same Code icon on both composers and give every pill an even icon slot
* studio: tidy composer toggle row and fix MCP enable/disable lifecycle
- Enable MCP automatically after a server is configured via the toggle flow
- Force MCP off everywhere once the last enabled server is removed
- Collapse the pill labels to icons only when more than 4 pills show, keeping Compare labelled
- Order Compare first in compare mode, before Search and Code
- Use the same Code icon and an even 19px icon slot across both composers
- Match the compare composer surface padding and send button inset to normal chat
* studio: revert compare composer padding change that cramped the input
Matching the surface padding to normal chat clipped the textarea text and
left a white strip on top. Restore the compare composer's own padding, which
gives proper top spacing. The send button inset fix stays.
* studio: center welcome greeting and soften composer scrollbar
Center the sloth and title together over the composer instead of
shifting the row left, which left the greeting sitting off to the side.
Keep the composer textarea scroll thumb faint by default and only darken
it when the thumb is hovered or dragged, so a tall draft no longer shows
a heavy dark rail.
* studio: match composer plus-menu tool gating to the pills
The new plus-menu tool entries did not carry the gating the visible pills
already enforce, so the menu and pills could disagree about a loaded
model's capabilities.
- Web search and Code menu items now disable when a loaded model lacks
the capability, while still allowing preselection with no model loaded.
- Enabling Web search from the menu on a Kimi model now flips thinking
off as a session-only change, since Kimi forbids search and thinking
together. This matches the Search pill.
- Added an Images menu item, shown only for image-generation models and
disabled until a model loads, so a short prompt has an entry point.
Applied to both the single-chat and compare composers.
* studio: round the active-pill hover x and even out pill padding
The hover x sat bare and the trailing label was tighter to the pill edge
than the leading icon, so the pill looked lopsided.
- Give the hover x a soft circular background that fills the icon slot,
matching the ChatGPT-style toggle and the icon it replaces.
- Add a little more trailing padding so the label and the leading icon
have even breathing room, and keep icon-only compact pills symmetric.
* studio: nudge the thinking bulb icon up by 0.5px
Bump the thinking lightbulb from 15px to 15.5px in the single-chat and
compare composers so it sits a touch larger next to the other controls.
* studio: drop the hover x circle on icon-only pills
When pills collapse to icon-only, the circle around the hover x is too
cramped in the small chip, so show a bare x there and keep the circle
only on the full-width labelled pills.
* studio: space the compare send button like normal chat
In compare mode the Thinking control sat right against the send button.
Match the normal composer's control spacing (gap-1.5 plus a send margin)
so Thinking has the same breathing room before send. The send button
keeps its 14px inset, so its position is unchanged.
* studio: make collapsed pill hover a circle, not a wide pill
Icon-only pills were wider than tall, so their rounded-full hover
highlight read as a fat rounded rectangle. Make the compact button a
square and center the glyph so the hover (and the x it reveals) sits in
a clean circle.
* studio: fix compare pane drops and audio picker lifetime
- Skip the page-level drop handler when the composer is hidden, so files
dropped on a compare pane are not swallowed by a hidden composer; the
shared compare composer keeps handling drops through its own dropzone.
- Build the audio file input on document.body instead of inside the plus
menu, so the menu closing on select no longer unmounts the input before
the OS picker returns and drops the file.
* studio/chat: stop projects list from white-screening on older backends
The projects list API returned data.projects directly, so a backend that
omits the field handed back undefined. useChatProjects cached that value,
then the next mount read undefined.length and crashed the whole chat page.
Default the projects and threads list APIs to an empty array and keep the
hook null-safe so a bad response can never poison the cache.
* studio/chat: align MCP dropdown with the + menu and add a chevron
Reuse the + menu surface (unsloth-plus-menu) for the MCP dropdown: rounded
corners, narrower width, neutral grey hover, and enabled rows shown as green
text with a right-aligned check instead of the emerald underlay. Add a
chevron to the MCP pill so it reads as openable, matching the Thinking pill.
* studio/chat: make MCP an opt-in pill and fix its dropdown placement
- MCP is back in the + menu as a toggle. The pill now only shows in the
composer when MCP is on, matching Canvas, instead of always sitting there.
- The dropdown follows the composer side like the + menu (opens down in the
welcome composer, up when docked) rather than always opening upward.
- Drop the dropdown caret when pills collapse so the icon is not squished.
- Stop force-syncing mcpEnabledForChat to the server count; the + menu owns it.
* studio/chat: MCP expands the composer, drop sidebar Compare, tidy scrollbars
- Toggling MCP now expands the composer and shows the tool pills, the same as
Canvas, instead of leaving the row collapsed.
- Remove the Compare item from the sidebar now that it lives in the + menu, and
point the compare tour step at the side-by-side view instead of the old button.
- Both sidebars only show their scrollbar on hover, and run settings reserves
the scrollbar gutter so the close button no longer shifts when it appears.
* studio/chat: tighten toggle gap, fix run-settings close button, collapsed Train
- Reduce the composer toggle gap by 2px (gap-1 to gap-0.5) in both composers.
- Move the run settings header out of the scroll area so the close button keeps
its position whether or not the scrollbar shows, and sits flush with the
topbar open button again instead of shifting left.
- Surface Train as an icon in the collapsed sidebar (it already has a labelled
section when expanded).
* studio/chat: tighten Thinking pill X padding, create projects inline
- The Thinking pill used px-2.5, so the hover X sat further in than the left
pills. Match their pl-2 so the X lines up.
- The + menu New project now opens a create dialog and jumps straight to the
new project, instead of routing to the projects list. Shared by both
composers via a small NewProjectDialog.
* studio/chat: soften account menu, hover scrollbars, show collapsed chevrons
- Account menu drops its border ring for the composer's soft shadow and opens
centered over its trigger.
- Settings and search reuse the hover-only scrollbar via a shared
hover-scrollbar class, matching the sidebars.
- Train and Recents keep their chevron visible while collapsed so it is clear
they can be expanded.
* studio/chat: roomier, more rounded account menu
Widen the account menu, add more left and right padding on the rows, bump the
row height and text a touch, and round the corners more, closer to the GPT
account menu.
* studio/chat: trim account menu width and nudge it up 2px
Pull the account menu in slightly on the left and right (narrower box, a touch
less row padding) and lift it 2px higher above the trigger.
* studio/settings: drop outline ring, circular close hover, pointer cursors
- Remove the settings dialog outline ring, keeping just the soft shadow.
- The close button hover is now a circle instead of a rounded rectangle.
- Every clickable control in the settings dialog uses a pointer cursor.
* studio/chat: bump MCP pill icon to 14.5px
Nudge the MCP icon up 0.5px so it sits even with the other pill glyphs.
* studio/chat: bump MCP pill icon to 15px
Nudge the MCP icon up another 0.5px.
* studio/settings: add a Settings title above the tabs
Put a Settings heading at the top of the sidebar so the tabs sit below it,
matching the Claude settings layout. Hidden on mobile where the nav is a row.
* studio/settings: rounder tab hover, bigger title, less-round search dialog
* studio/sidebar: round nav row hover boxes 2px more (10px to 12px)
* studio: drop settings dark shadow + divider, add tab left padding, tune hover roundness
* studio/model-selector: roomier padding, borderless box, rounder hover rows; settings divider light-only
* studio/search: match chat box shadow (soft light, none dark)
* studio/sidebar: borderless chat context menus, rename submenu to Projects with folder-export icon
* studio/model-selector: match light corner radius in dark, drop dark shadow, more visible dark hover
* studio/sidebar: chat context menu matches + side menu styling; relabel submenu Move to project
* studio: borderless message export menu (no dark shadow), match dark corner radius to light on export menu and settings
* studio/sidebar: open chat options menu GPT-style (down-right) and widen so Move to project fits one line
* studio/chat: message export menu uses the chatbox shadow in light mode
* studio/sidebar: narrow chat options menu slightly (w-60 to w-56)
* studio: unify all download icons to Hugeicons download-01; round profile button hover 1px more
* studio/run-settings: bump header to 16px
* studio/sidebar: trim chat options menu width slightly (w-56 to 216px)
* studio/sidebar: trim chat options menu width to w-52
* studio/profile: camera-01 Hugeicons glyph and chatbox shadow on avatar button
* studio: match dark-mode corner radius to light globally (single --radius token)
* studio/recipes: borderless New Recipe menu with chatbox shadow in light, none in dark
* studio: borderless dropdowns globally, chatbox shadow in light, none in dark
* studio: extend borderless + chatbox/none shadow to select, combobox and popover overlays
* studio/mcp: nudge MCP dropdown radius to 20px so its wider box reads as round as the + menu
* studio: restore dark dropdown shadow to avoid same-color merge; greet name ~1/3 of lines; bigger sloth + more gap
* studio/train: active tab is a borderless pill (no underline), roomier padding, more tab gap and bottom spacing
* studio/chat: nudge welcome up ~5px (still vh-based) and trim sloth image to 44px
* studio/train: active tab pill is white with chatbox shadow in light, taller padding
* studio/chat: welcome offset to calc(30vh - 10px)
* studio/chat: welcome offset to 28vh (drop the -10px)
* studio/chat: tighten sloth-to-text gap by 1px (16px to 15px)
* studio/train: revert light active pill to grey fill, drop white bg + shadow
* studio: app-wide hand cursor on every clickable control (disabled excluded)
* studio/chat: welcome offset to 26vh
* studio/chat: welcome offset to 28vh
* studio/chat: harden project and thread list guards against non-array payloads
* studio/sidebar: give the profile row more height and breathing room
* studio/sidebar: trim the profile row top and bottom padding slightly
* studio/sidebar: reduce Train and Recents section label size slightly
* studio/sidebar: trim the profile row top and bottom padding a touch more
* studio/sidebar: enlarge the profile hover area top and bottom
* studio/sidebar: increase profile hover roundness by 1px
* studio/sidebar: trim the profile row top and bottom padding slightly
* studio/sidebar: trim the profile row top and bottom padding slightly
* studio/chat: cache composer line metrics so wrap detection runs once, not per keystroke
* studio/chat: restore the prior view when exiting compare opened from the + menu
* studio/tests: drive Compare from the composer + menu after it moved out of the sidebar
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio/tests: open Compare from the composer + menu in the extra UI suite too
* studio: fix chat dictation microphone access
* studio: snappier plus-to-x spin and steady composer expand gap
Speed up the composer plus icon morph from 480ms to 300ms.
Add row-gap on the expanded composer line so the space between the text
and the controls row stays the same whether the box expanded from
wrapped text or from a toggle being on. The gap sits on the line, not the
input, so the placeholder max-height clamp never crops it.
* studio: only show composer tool pills once a model is loaded
Persisted Search/Code/Canvas/MCP toggles were surfacing the composer pill row on a fresh page load before any model was selected, so an empty composer looked different from the clean just-ejected state. Gate the composerExpanded tool checks on modelLoaded so a model-less composer stays collapsed, while saved preferences still apply the moment a model loads.
* studio: hide RAG composer menu item temporarily
Hide the placeholder RAG entry from the composer plus menu in both single chat and compare until the feature is ready, and drop the now-unused DatabaseIcon import.
* studio: let composer tools pre-select before a model loads
Selecting Web search, Code, Canvas or MCP from the + menu with no model
loaded did nothing visible: the toggle turned on but the composer never
expanded, so the pill stayed hidden. Drop the model-loaded gate from the
expand check so an active tool always surfaces its pill.
Align MCP with the Search/Code pattern too: grey it out only when a loaded
model lacks tool support, so MCP stays toggleable and the pill stays
clickable before a model is loaded instead of looking disabled.
---------
Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: Daniel Han <23090290+danielhanchen@users.noreply.github.com>
* studio: cap training dataset uploads
* studio: clean up failed dataset uploads
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: raise upload limits to 500MB
* studio: make upload limit configurable
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: stream upload routes
* studio: split recipe upload caps
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: tighten upload limit handling
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: import settings router directly
* studio: polish upload cap setting control
* studio: cap settings request bodies
* studio: stub settings route in desktop auth test
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Follow-up cleanups to the merged AMD ROCm support PR #5301:
1. De-duplicate the torchao Windows-ROCm import stub into a single shared
module (studio/backend/core/_torchao_stub.py); both workers call one
install_torchao_windows_rocm_stub() entrypoint.
2. Align the gfx name/arch comment columns in setup.sh and setup.ps1.
3. Isolate the float16 dtype fallback to AMD without native bf16; NVIDIA
keeps dtype=None so unsloth's own bf16/fp16/FORCE_FLOAT32 detection is
honored.
4. Hoist unconditional stdlib imports (gc, glob, re, subprocess, copy,
types, sys, importlib.metadata) from function bodies to module top
across the PR #5301-touched files; heavy/optional/relative imports stay
lazy.
5. bitsandbytes Windows-ROCm install now uses plain pip (force_pip=True)
instead of UV_SKIP_WHEEL_FILENAME_CHECK, per the AMD hackathon docs.
Also adds scripts/verify_import_hoist.py (a scope-aware LEGB AST resolver
that catches dangling-alias and rename-clash bugs in import-hoist
refactors) and wires it into the Lint CI source-lint job as a self-test
plus a pull_request compare gate.
* fix(studio): set HIP_VISIBLE_DEVICES in apply_gpu_ids for ROCm training workers
Training workers are spawned via multiprocessing spawn before detect_hardware()
runs, so IS_ROCM is still False. If the user never set HIP_VISIBLE_DEVICES in
their shell, _inherits_rocm_visibility is also False, leaving the worker with
only CUDA_VISIBLE_DEVICES set. On ROCm hosts the HIP runtime honors
HIP_VISIBLE_DEVICES over CUDA_VISIBLE_DEVICES, so the worker saw the full
device list and torch raised "no usable HIP accelerator" on some setups.
Fall back to probing torch.version.hip (a build-time attribute, safe to read
before GPU init) to detect ROCm when neither IS_ROCM nor inherited env vars
are available. Mirrors the existing fix in llama_cpp.py for llama-server
subprocess GPU pinning.
Fixes https://github.com/unslothai/unsloth/issues/5180
* test: tighten apply_gpu_ids ROCm fallback assertions
Replace loose OR chain with exact string matches, split into three
focused tests, and add a guard check for the try/except wrapper.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: detect ROCm unified memory (Strix Halo / AMD iGPU) via torch fallback
amd-smi on iGPUs with shared/unified memory (e.g. Radeon 8060S on Strix
Halo) reports only the dedicated VRAM slice (~512 MB) in its metric output,
so get_visible_gpu_utilization() was returning usable_gb ≈ 0.35 GB instead
of the full GTT pool (~128 GB). torch.cuda.mem_get_info() already surfaces
the correct unified-pool size.
Add _reconcile_rocm_unified_memory(): after amd-smi returns a valid result
on a ROCm device, cross-check each device's vram_total_gb against
torch.cuda.mem_get_info(). When torch reports a larger total, replace the
amd-smi VRAM fields in-place. No-op for discrete AMD GPUs where the two
sources agree.
Fixes: "Falling back to all visible GPUs -- model may not fit" on AMD iGPU
machines even when 100+ GB of unified memory is available.
* Apply unified-memory reconciliation in get_gpu_utilization too
The visible-GPU path was already corrected for AMD iGPUs with unified memory
(Strix Halo / Radeon 8060S), but get_gpu_utilization was still returning the
raw 512 MB amd-smi VRAM slice. Studio's /api/train/hardware endpoint and the
live GPU monitor read from this primary path, so users continued seeing the
wrong total even after auto_select_gpu_ids picked the right device.
Refactor to share the per-device correction:
* _apply_unified_memory_correction(metrics, torch_info) -- the actual
replacement logic, in-place on a single metrics dict.
* _reconcile_rocm_unified_memory(...) -- multi-device,
iterates utilization["devices"] (visible-GPU path).
* _reconcile_primary_rocm_unified_memory(...) -- single flat
metrics dict (primary-GPU path), uses parent_visible_spec to pick the
primary index, falls back to ordinal 0 when no visibility env is set.
get_gpu_utilization now calls the primary reconciler under IS_ROCM, so both
endpoints surface the real unified-memory pool on iGPUs while leaving
discrete AMD GPUs untouched (torch_total <= smi_total -> no replace).
* Use 'is not None' and log debug on torch.version.hip probe failures
Two small follow-ups to the apply_gpu_ids ROCm fallback:
1. Match detect_hardware()'s 'getattr(torch.version, "hip", None) is not None'
form so the entire codebase has one canonical 'this torch was built with
HIP' check. On every shipping torch wheel hip is either None or a non-empty
version string, so the new form agrees with the old bool() form on every
real install.
2. Log the probe failure at debug level instead of swallowing it silently.
The broad 'except Exception' is intentional (we never want apply_gpu_ids
to crash a worker over a probe), but the silent pass made it impossible
to tell whether the fallback was firing or being skipped.
* fix(studio): honour HIP_VISIBLE_DEVICES in _get_parent_visible_gpu_spec before IS_ROCM is set
When a user has HIP_VISIBLE_DEVICES set in their shell (e.g. "1" to select
GPU 1) but detect_hardware() has not yet run in the Studio parent process,
IS_ROCM is still False. _get_parent_visible_gpu_spec() was gated on IS_ROCM
so it fell through to CUDA_VISIBLE_DEVICES (unset), saw all physical GPUs,
and auto-selected index 0. apply_gpu_ids then overwrote HIP_VISIBLE_DEVICES
with "0", making the intended GPU invisible to ROCm torch in the worker,
which triggered the "no usable HIP accelerator" error (issue #5180).
Apply the same _inherits_rocm_visibility pattern already used in
apply_gpu_ids: check for HIP_VISIBLE_DEVICES / ROCR_VISIBLE_DEVICES in the
environment regardless of IS_ROCM so the correct GPU index is preserved.
* fix(install): harden AMD ROCm GPU detection for multi-GPU and env-filtered setups
The previous rocminfo awk pattern could miss discrete GPUs on machines
where HIP_VISIBLE_DEVICES/ROCR_VISIBLE_DEVICES is used to mask an
integrated GPU — the env vars filter rocminfo output but may not
propagate into the install script subprocess, causing detection to
fail entirely.
Two changes:
- Tighten rocminfo pattern from /gfx[0-9]/ && !/gfx000/ to
/gfx[1-9][0-9]/ — simpler and correctly excludes the CPU agent
(gfx000) without a negative lookahead
- Add sysfs KFD topology fallback: reads
/sys/class/kfd/kfd/topology/nodes/*/gpu_id which is a kernel-level
view unaffected by HIP_VISIBLE_DEVICES or ROCR_VISIBLE_DEVICES
Fixes detection failure reported in Discord by Chains (gfx1201 + iGPU
machine where env var exclusion of the iGPU caused rocminfo to return
no usable device).
* Fix KFD sysfs awk fallback to read properties file
The fallback added by this PR reads /sys/class/kfd/kfd/topology/nodes/*/gpu_id
files but matches the literal token 'gpu_id' against their content. Those
files contain only a single decimal value (e.g. '0' for CPU agents, '50432'
for GPU agents), so the regex never matches and 'found' stays 0, making the
fallback a no-op on every host. The properties file in the same directory
contains key/value lines like 'gpu_id 50432' which is what the existing awk
pattern expects.
Reproduced with a synthetic sysfs layout: against gpu_id files awk exits 1;
against properties files awk exits 0 when any node reports gpu_id > 0.
* fix(setup.ps1): detect AMD ROCm GPU on Windows, bring to parity with setup.sh
setup.ps1 only checked nvidia-smi and fell straight to "gpu: none" on AMD
machines. setup.sh already probed rocminfo/amd-smi/hipconfig/hipinfo.
Add three-tier detection mirroring install_llama_prebuilt.py's detect_host():
1. hipinfo: gcnArchName in output confirms a real HIP GPU (not just SDK)
2. amd-smi list: "GPU: <digit>" data rows as fallback
3. WMI Win32_VideoController: last resort -- detects AMD GPU even without
HIP SDK, then guides user to install it rather than silently going CPU
Also corrects the "none" message to mention AMD ROCm alongside NVIDIA so
users with AMD hardware understand the requirement.
Fixes: rohit-style install where Strix Halo (Radeon 8060S) showed
"gpu: none" even with the HIP SDK present.
* fix(install.ps1): detect AMD ROCm GPU on Windows, bring to parity with setup.ps1
install.ps1 had the same nvidia-smi-only GPU detection as setup.ps1 before
the setup.ps1 fix. Applies the same three-tier AMD detection:
1. hipinfo: gcnArchName confirms real HIP GPU
2. amd-smi list: GPU data rows as fallback
3. WMI Win32_VideoController: detects AMD GPU without HIP SDK and guides
user to install it
Fixes: install.ps1 showing "gpu: none" while setup.ps1 correctly showed
"AMD GPU detected" on the same machine (reported by rohit, RX 7600 XT).
* fix(install.ps1): suppress 'No NVIDIA GPU detected' when AMD GPU is present
* feat: add Windows AMD ROCm PyTorch wheel installation
install_python_stack.py:
- Add _ROCM_WINDOWS_WHEEL_BASE and _ROCM_WINDOWS_RELEASES constants
pointing to AMD repo.radeon.com (ROCm 7.2 -> torch 2.9.1+rocm7.2.1)
- Extend _ensure_rocm_torch() with a Windows branch: detects ROCm via
_has_rocm_gpu() / _detect_rocm_version(), requires Python 3.12 (cp312
is the only ABI AMD publishes for Windows), installs the direct wheel
URL from repo.radeon.com
install.ps1:
- Capture ROCmVersion during AMD detection via hipconfig --version /
amd-smi version (needed for wheel URL selection)
- After Get-TorchIndexUrl, add an AMD wheel override block: when HasROCm
and Python 3.12 detected, set ROCmTorchWheelUrl to AMD wheel URL
- Expand torch install branch to handle ROCmTorchWheelUrl with
uv pip install --force-reinstall --no-cache-dir
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: also install torchvision and torchaudio from AMD Windows repo
AMD publishes matching torchvision-0.24.1+rocm7.2.1 and
torchaudio-2.9.1+rocm7.2.1 cp312 wheels at the same repo.radeon.com
release folder. Install all three in both install.ps1 and
install_python_stack.py Windows ROCm path.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* feat: add ROCm 7.1.1 Windows wheel mapping
AMD uses a different version string for 7.1.1 wheels:
2.9.0+rocmsdk20251116 (date-tagged) instead of +rocm7.1.1.
Adds the 7.1.1 release folder to both install.ps1 and
install_python_stack.py so users with ROCm 7.1 get ROCm
torch instead of falling back to CPU.
* fix: install rocm_sdk_core and rocm_sdk_libraries_custom alongside torch
The AMD Windows torch wheels declare rocm[libraries]==<ver> as a hard
dependency. Without installing rocm_sdk_core and rocm_sdk_libraries_custom
from the same AMD release folder, uv cannot resolve the dependency and
fails with 'No solution found'. Include all 5 wheels in one install call.
* fix: expand ROCm wheel array to scalars for Invoke-InstallCommand
@array splatting inside a scriptblock only works when the native command
is prefixed with '&'. Invoke-InstallCommand uses '& $Command' to run the
block, so @ROCmAllWheelUrls was not being expanded. Extract to scalar
variables $rw0-$rw4 which are captured correctly by the closure.
* fix: use --no-deps for AMD Windows torch wheel install
uv's resolver looks up rocm[libraries]==0.1.dev0 on PyPI during
dependency resolution before downloading any wheels, and fails because
the package doesn't exist on PyPI. --no-deps skips resolution entirely
and installs all 5 AMD wheels directly. The GPU runtime dependency is
satisfied by the HIP SDK, not a Python package.
* fix: setup.ps1 and install_python_stack.py now install ROCm torch on Windows
setup.ps1 was always setting CuTag='cpu' for non-NVIDIA hosts and installing
cpu-only PyTorch, overwriting the ROCm torch installed by install.ps1.
Adds the same AMD wheel selection logic (ROCm version detection, Python 3.12
check, 5-wheel install with --no-deps) to setup.ps1's torch install block.
install_python_stack.py: remove IS_WINDOWS guard from _ensure_rocm_torch()
call site so the Windows path in _ensure_rocm_torch() is reachable during
'unsloth studio update' as well.
* fix: suppress manual-install warning when ROCm torch already present; fix progress counter
- Gate the 'must be installed manually' warning on torch.version.hip being empty
so it doesn't fire when our ROCm torch install succeeded
- Update _TOTAL counter to include the 3 ROCm steps on Windows now that
_ensure_rocm_torch() is called there (fixes 10/9 display)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* feat: add rocm step display in setup.ps1; fix warning and progress counter
- Add 'rocm' step after 'cuda' in setup.ps1 showing ROCm version or HIP SDK missing
- Move ROCm version detection up to GPU detection block so it's available early
- Suppress 'must be installed manually' warning when torch.version.hip is set
- Fix _TOTAL counter to include ROCm steps on Windows (fixes 10/9 display)
* fix: detect AMD SDK ROCm torch via __version__ when torch.version.hip is unset
AMD's repo.radeon.com wheels (e.g. 2.9.0+rocmsdk20251116) do not set
torch.version.hip, leaving it None. All three probes that relied solely on
torch.version.hip now also check for 'rocm' in torch.__version__.lower():
- hardware.py detect_hardware(): IS_ROCM was never set, causing the studio
to report 'Hardware detected: CPU' even after AMD wheels were installed
and HIP DLLs were on PATH.
- install_python_stack.py _ensure_rocm_torch(): skip-if-already-installed
probe would always reinstall on subsequent runs.
- install_python_stack.py Windows AMD warning: suppression check always
failed, so the 'must be installed manually' note kept appearing after
a successful AMD wheel install.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* perf: drop --no-cache-dir from AMD ROCm torch wheel installs
uv caches downloaded wheels by default; passing --no-cache-dir forced a
full redownload of the ~2 GB torch wheel on every install run. CUDA installs
never had this flag -- AMD was the only path affected.
* fix: use install-state flag instead of subprocess probe for AMD Windows warning
Replace the subprocess torch probe in the post-install warning block with a
module-level _rocm_windows_torch_installed flag set by _ensure_rocm_torch().
Subprocess re-import of torch is unnecessary and fragile -- the install
function already knows whether it succeeded.
* fix: hoist global declaration to top of _ensure_rocm_torch
Python requires the global statement to appear before any assignment
to the variable within a function. Moving it to the function top fixes
the SyntaxError on line 354.
* fix: pass AMD torch install status via env var to suppress false warning
setup.ps1 now sets UNSLOTH_ROCM_TORCH_INSTALLED=1 after a successful AMD
wheel install. install_python_stack.py reads this at the top of
_ensure_rocm_torch() to skip both the subprocess probe and the warning --
no re-import of torch needed, and the warning message now correctly says
'could not be auto-installed' rather than 'must be installed manually'.
* fix: register ROCm DLL directory before torch import on Windows
Python 3.8+ ignores PATH for extension DLL loading on Windows; amdhip64.dll
and other HIP runtime DLLs must be registered via os.add_dll_directory().
Without this, torch.cuda.is_available() always returns False on AMD ROCm
Windows even when HIP_PATH is correctly set in system environment variables.
Reads HIP_PATH / ROCM_PATH env vars first, then falls back to scanning
common ROCm install roots (C:\Program Files\AMD\ROCm, F:\ROCm, C:\ROCm).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: remove hardcoded non-standard ROCm paths from DLL directory scan
Only use HIP_PATH/ROCM_PATH (set by AMD installer) and the standard
C:\Program Files\AMD\ROCm\<version>\bin location. Custom drive paths
like F:\ROCm are user-specific and should not be hardcoded.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: prevent torchao overrides step from overwriting AMD ROCm torch
torchao==0.14.0 in overrides.txt declares torch as a dependency. Without
--no-deps, uv resolves torch from PyPI and installs 2.11.0+cpu on top of
the AMD ROCm wheels (2.9.0+rocmsdk20251116). This was the root cause of
'Hardware detected: CPU' -- the AMD wheels were installed but then
immediately overwritten by the overrides step.
When _rocm_windows_torch_installed is True, add --no-deps to the overrides
pip_install call so torchao is installed without pulling in CPU torch.
* fix: add rocm_sdk namespace tarball to Windows ROCm wheel installs
torch/_rocm_init.py calls `import rocm_sdk` at startup, which requires
the rocm namespace tarball (rocm-*.tar.gz) in addition to the SDK wheel
packages. This tarball was missing from both install.ps1 and setup.ps1,
causing ModuleNotFoundError on first torch import.
- Add rocm-0.1.dev0.tar.gz to ROCm 7.1.1 install (provides rocm_sdk namespace)
- Add rocm-7.2.1.tar.gz + rocm_sdk_devel to ROCm 7.2.1 install
- Install tarball in a dedicated step before main SDK/torch wheels
- Switch to @array splatting in install.ps1 scriptblock for dynamic wheel count
- Remove --no-cache-dir from Python-side ROCm wheel install (prevents ~2GB redownload)
* feat: enable ROCm 7.2 torch install + warn on gfx1151 with ROCm < 7.2
Chigoma333 (AMD Radeon 8060S / gfx1151, Strix Halo) confirmed that ROCm
7.1 segfaults when tensors are moved to GPU, but ROCm 7.2 + torch
2.11.0+rocm7.2 works fully including training.
Changes:
- Uncomment (7,2): "rocm7.2" in _ROCM_TORCH_INDEX (was blocked by <2.11.0)
- Add _ROCM_TORCH_PKG_SPECS dict with per-tag version bounds:
rocm7.2 → torch>=2.11.0,<2.12.0; all older tags → <2.11.0
- Add _detect_amd_gfx_codes() helper that parses rocminfo output
- Warn on gfx1151/gfx1150 (Strix Halo) when ROCm < 7.2 is installed,
pointing users at the known segfault and recommending upgrade
- install.sh get_torch_index_url(): enable rocm7.2 case (previously capped
to rocm7.1), cap unknown future tags to rocm7.2
- install.sh: override TORCH_CONSTRAINT to >=2.11.0,<2.12.0 when rocm7.2
index is selected, so pip can actually resolve torch 2.11.0
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: prefer Python 3.12 for AMD ROCm users when 3.13 is also installed
After GPU detection, if ROCm HIP SDK is found and the selected Python
is not 3.12, run a second pass to locate a 3.12 install via py.exe and
PATH (catches uv-managed installs). Switch $DetectedPython to 3.12 so
the venv is created with a compatible interpreter for the cp312-only AMD
Windows torch wheels.
NVIDIA and Intel GPU paths are unaffected -- the re-detection block only
runs when $HasROCm is true.
Fixes: #5301
* fix: also check uv-managed Python 3.12 for AMD ROCm #5301
* fix: hide amd-smi console popups on Windows, guard torch.distributed.is_initialized for ROCm #5301
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: suppress remaining console popups on Windows, patch torch.distributed.is_initialized for ROCm #5301
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: stub all missing torch.distributed attrs for ROCm Windows wheel #5301
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: inject torch.distributed stub when C backend missing in ROCm Windows wheel #5301
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(rocm/windows): pre-stub torch._C._distributed_c10d + raise amd-smi timeout
Two fixes for Windows ROCm regressions reported by electroglyph on #5301:
1. worker.py — torch.distributed stub now fires unconditionally on Windows
The previous stub only injected sys.modules in the except branch, meaning
it was silently skipped when `import torch.distributed` happened to succeed
(the C backend is lazily resolved). The crash then hit later when
transformers/trl triggered the lazy load. Fix: on win32 we pre-populate
sys.modules['torch._C._distributed_c10d'] AND set the attribute on the
torch._C extension module *before* attempting the import, covering both
the early-ImportError and lazy-load failure modes.
2. amd.py — increase amd-smi timeout from 5 s to 30 s on Windows (10 s Linux)
amd-smi on Windows must cold-init the ROCm runtime on first invocation;
5 s was consistently too short, producing repeated 'Command timed out'
warnings in the server log. 30 s gives enough headroom without blocking
indefinitely on broken installs.
3. install.ps1 — widen Python 3.12 enforcement to ROCmGpuLabel (WMI-only path)
Users whose HIP SDK is not on PATH were detected via WMI but not switched
to Python 3.12 before the install started, causing a second pass. Guard
now fires on (HasROCm -or ROCmGpuLabel).
* fix(rocm): guard c10d stub, fix TorchIndexFamily for 7.1, clean dead code + comments
- worker.py: wrap c10d stub injection in `if _c10d_key not in sys.modules` so
Windows NVIDIA users with a real torch.distributed are never affected
- install.ps1: fix Get-TauriTorchIndexFamily receiving hardcoded "rocm7.2"
even when ROCm 7.1 wheels are installed; now branches on $ROCmVersion
- main.py: remove dead `import ctypes as _ctypes` (ctypes is never called)
- hardware.py, install_python_stack.py, worker.py, install.ps1: shorten
verbose multi-line comment blocks throughout
- tests: update 4 stale assertions that expected rocm7.2 to be absent/capped
* fix(tests): match windows AMD warning assertion to actual source string
* chore: trim verbose comment blocks across all ROCm-related files
* fix: guard reconcile call against None numeric_ids; add torchvision lower bounds
* fix(install.ps1): recreate venv with Python 3.12 after ROCm switch
Venv was created with 3.13 before GPU detection ran; switching
$DetectedPython to 3.12 had no effect since $VenvPython still
pointed to the 3.13 interpreter inside the already-created venv.
* ux: detect AMD GPU before Python selection to avoid double venv creation
- Early hipinfo + WMI probe runs before Find-CompatiblePython so Python
3.12 is selected upfront when AMD is detected; venv is now created
exactly once instead of 3.13 then immediately 3.12.
- Post-venv recreation block replaced with a simple warning for the rare
case where AMD was missed by the early probe.
- setup.ps1: show venv's actual Python version (e.g. 3.12) instead of
the system Python found by the pre-activation search (was showing 3.13).
* fix(rocm/win): auto-stub all _distributed_c10d symbols via PEP-562 __getattr__
The bare ModuleType stub caused ImportError when torch._dynamo was imported
(triggered by trainer.py accessing torch._dynamo.config at load time).
torch._dynamo pulls in torch.distributed.fsdp._flat_param which does:
from torch._C._distributed_c10d import FakeProcessGroup
and potentially other symbols. Adding module __getattr__ auto-creates a
stub class for any missing symbol so all such imports succeed without
enumerating every individual symbol. Applied to both the primary stub
and the fallback stub in the except branch.
* chore: trim c10d stub comment
* fix(rocm/win): auto-stub missing torch.distributed attrs (Store, ProcessGroup, …)
* fix(rocm/win): pre-stub fsdp submodules in sys.modules; fix __getattr__ subpackage clash
* feat(rocm/win): arch-aware wheel selector always picks newest ROCm release
Replace HIP-SDK-version-gated wheel selection with GPU arch-based logic.
Select-ROCmWheelRelease (PS) and _select_windows_rocm_release (Python) map
gcnArchName → minimum ROCm version, then pick the newest available release
that satisfies it (currently always rocm-rel-7.2.1 for any supported GPU).
Wheels bundle their own ROCm runtime so the installed HIP SDK 7.1 does not
prevent using 7.2.1 wheels on gfx1200 (RX 9060 XT) and similar RDNA 4 GPUs.
Also installs the bitsandbytes Windows ROCm continuous-release wheel and sets
BNB_ROCM_VERSION=72 in worker.py before ML imports so bnb loads the
libbitsandbytes_rocm72.dll that ships in that wheel.
* fix(rocm/win): stub class metaclass for ProcessGroup.BackendType; amd-smi circuit breaker
torchao.float8.inference accesses ProcessGroup.BackendType as a class-level
attribute. Plain type() stubs have no __getattr__ on the metaclass so this
raises AttributeError. Introduce _StubClassMeta whose __getattr__ returns
child stub classes, fixing the torchao import chain.
Add an amd-smi circuit breaker in amd.py: after 3 consecutive failures the
module stops spawning the process, eliminating the repeated Windows UAC /
DiskPart elevation prompts caused by polling a non-functional amd-smi.
Also guard BNB_ROCM_VERSION=72 behind a DLL existence check so bitsandbytes
fails with its own detection message rather than a harder "DLL not found" when
the Windows ROCm bnb wheel is not yet installed.
* fix: stub __members__ so torchao float8 enum check doesn't crash on ROCm Windows
torchao.float8.inference accesses ProcessGroup.BackendType.__members__
expecting a Python Enum registry dict. _StubClassMeta.__getattr__ was
blocking all dunder attributes, causing AttributeError. Return {} for
__members__ specifically so the isinstance/iteration checks pass cleanly.
* fix: stub distributed tensor/functional_collectives to prevent missing C++ op crash on ROCm Windows
torch._dynamo.trace_rules eagerly loads torch.distributed.tensor at import
time, which pulls in _functional_collectives.py. That file registers Meta
kernels for _c10d_functional C++ ops, but those ops are only registered
by torch._C._distributed_c10d — a C extension absent from ROCm Windows
wheels. Pre-stubbing the affected modules in sys.modules prevents the real
import chain from running and avoids the "operator does not exist" crash.
* fix: give mod stubs __path__ and pre-stub _tensor to fix 'not a package' import error
_make_mod_stub now sets __path__=[] so Python treats stub modules as
packages. Without it, any import of a submodule raises "is not a package".
Also pre-stub torch.distributed._tensor and its submodules so that
_tensor/__init__.py (which re-exports from torch.distributed.tensor) never
runs and torchao's `from torch.distributed._tensor import DTensor` gets a
harmless stub instead of crashing.
* fix: stub torch.ops._c10d_functional namespace with hashable op sentinels
torchao.dtypes.nf4tensor uses _c10d_functional ops as dict keys at import
time (all_gather_into_tensor.default, wait_tensor.default) and
torch.ops.c10d.scatter_.default. None of these ops are registered on ROCm
Windows because torch._C._distributed_c10d (the C extension) doesn't ship.
Replace the whole _c10d_functional namespace with a custom stub whose ops
return hashable .default objects, so dict-key construction doesn't crash.
Also inject a scatter_ stub into torch.ops.c10d if it's missing.
* fix: stub entire torchao package on ROCm Windows instead of individual ops
torchao is not supported on ROCm Windows and its import chain transitively
requires torch._C._distributed_c10d (absent from the ROCm Windows wheel).
Rather than stub each missing op one by one, stub the whole torchao package
upfront. Unsloth uses bitsandbytes for quantization, not torchao, so this
has no functional impact. transformers gracefully handles an importable-but-
empty torchao by disabling TorchAoHfQuantizer.
* fix: set __spec__ on mod stubs so importlib.util.find_spec doesn't raise
Manually-injected sys.modules entries have __spec__=None by default.
importlib.util.find_spec() raises ValueError when it finds a module in
sys.modules with __spec__=None (transformers.utils.import_utils hits this
when checking if torchao is available). Give every stub a minimal
ModuleSpec(name, loader=None, is_package=True) to satisfy find_spec.
* fix: add meta path finder to auto-stub subpackages of stub modules
`import torchao.prototype` goes through the import machinery, not
__getattr__, so an empty __path__ means ModuleNotFoundError. Rather than
list every submodule explicitly, register a MetaPathFinder that intercepts
any import whose parent is one of our stubs (detected by loader=None in the
parent's ModuleSpec). Real installed packages always have a SourceFileLoader
so they are never intercepted. Also register child stubs in sys.modules
from __getattr__ as a belt-and-suspenders measure.
* fix: use _unsloth_stub sentinel instead of loader=None for stub detection
The import machinery overwrites module.__spec__ with the spec returned by
find_spec (which has loader=_StubSubpackageLoader, not None), so the
loader=None check broke for second-level subpackages. Switch to a custom
_unsloth_stub object identity sentinel set directly on each stub module --
it survives __spec__ being replaced and correctly identifies stubs at any
depth (torchao.prototype.safetensors, etc.).
* refactor(rocm/win): switch to repo.amd.com arch-aware index, remove stubs
AMD recommends repo.amd.com/rocm/whl/{arch}/ as the Windows ROCm wheel
source. These wheels bundle their own ROCm runtime, support all Python
versions (not just cp312), and include the full torch._C extension set
(including _distributed_c10d) that the old repo.radeon.com wheel omitted.
Changes:
- install.ps1: remove Select-ROCmWheelRelease + hardcoded cp312 wheel
URLs; remove Python 3.12 forced-preference logic; install via
--index-url repo.amd.com/rocm/whl/{arch-family}/
- studio/setup.ps1: same -- remove Select-ROCmWheelRelease, switch to
repo.amd.com arch-aware index URL
- studio/install_python_stack.py: replace _ROCM_WINDOWS_RELEASES /
_select_windows_rocm_release with _windows_rocm_index_url() using the
_GFX_TO_AMD_INDEX_ARCH map; drop Python 3.12 restriction
- studio/backend/core/training/worker.py: remove all stub machinery
(_make_mod_stub, _StubSubpackageFinder, _StubSubpackageLoader,
_StubClassMeta, torchao/fsdp/dtensor stubs, _c10d_functional ops
stubs, BNB DLL detection) -- no longer needed with new wheel source
* fix(rocm/win): restore _distributed_c10d + torchao stubs; fix BNB install
repo.amd.com torch wheels also omit torch._C._distributed_c10d on Windows
(RCCL is not shipped on Windows). torch/distributed/__init__.py imports
from it unconditionally at module level, so the stub must land in
sys.modules before any torch.distributed import.
torchao (pulled in by transformers.quantizers) walks
torchao.float8.distributed_utils -> torch.distributed._functional_collectives
-> distributed_c10d at import time. Stubbing torchao up-front short-circuits
that chain.
worker.py:
- Restore _make_mod_stub / _StubSubpackageFinder / _StubSubpackageLoader
- Restore _StubClassMeta for ProcessGroup.BackendType attribute access
- Restore _distributed_c10d stub with __getattr__ (Windows only)
- Restore torchao stubs (5 modules, Windows only)
install_python_stack.py:
- BNB AMD wheel install was inside the early-return branch that fires when
torch is already a ROCm build (installed by install.ps1). Move BNB install
outside that branch so it always runs on Windows ROCm — the PyPI
bitsandbytes has only CUDA DLLs and fails to load on ROCm.
* worker: remove _distributed_c10d stub; stub only torchao
The installed torch/distributed/__init__.py from repo.amd.com
(torch==2.10.0+rocm7.12.0) is now properly guarded with
`if is_available():`, so `import torch.distributed` alone is safe.
The crash only comes via torchao's import chain:
torchao.float8.distributed_utils
→ torch.distributed._functional_collectives (unguarded import)
→ torch.distributed.distributed_c10d
→ torch._C._distributed_c10d ← absent on Windows ROCm
Stubbing torchao short-circuits the chain entirely. No need to stub
_distributed_c10d. Remove _StubClassMeta and the _c10d stub block;
keep only _make_mod_stub + _StubSubpackageFinder + torchao seeds.
* fix: BNB AMD wheel skipped + torch.compile segfault on Windows ROCm
install_python_stack.py: the UNSLOTH_ROCM_TORCH_INSTALLED=1 early-return
path (set by setup.ps1 when it installed torch itself) returned before
ever reaching the AMD BNB prerelease wheel install. The PyPI
bitsandbytes==0.49.x ships only CUDA DLLs, so loading it on ROCm fails
with "libbitsandbytes_rocm72.dll not found". Now installs the AMD
Windows BNB wheel before returning on that path too.
worker.py: torch._grouped_mm crashes on gfx1200 (null HIP kernel pointer,
0xC0000005) when torch.compile's JitDecomp system dispatches it during
the first forward pass. Detect Windows ROCm via torch.version.hip
(already in sys.modules from section 1e) and set TORCHDYNAMO_DISABLE=1
to bypass the broken kernel dispatch.
* fix: BNB AMD wheel install fails uv wheel filename check
The bitsandbytes continuous-release wheel is intentionally mismatched:
filename encodes 1.33.7.preview (= 1.33.7rc0 in PEP 440) but wheel
metadata reports 0.50.0.dev0. uv rejects this by default.
Introduce _install_bnb_windows_rocm() helper that sets
UV_SKIP_WHEEL_FILENAME_CHECK=1 only for this specific install, then
restores the previous env value. Both BNB install call sites (the
UNSLOTH_ROCM_TORCH_INSTALLED early-return path and the normal Windows
ROCm path) now use this helper.
* worker: patch _grouped_mm CUDA dispatch on Windows ROCm (gfx1200 null kernel)
TORCHDYNAMO_DISABLE=1 stopped the compiler frontend but not the autograd
JitDecomp system, which also dispatches _grouped_mm and hits the same
null HIP kernel crash (0xC0000005).
Verified that torch.library.Library("aten","IMPL").impl("_grouped_mm", fn,
"CUDA") successfully overrides the broken HIP kernel with a Python mm
fallback on torch==2.10.0+rocm7.12.0.
Schema: _grouped_mm(Tensor self, Tensor mat2, Tensor? offs=None,
Tensor? bias=None, ScalarType? out_dtype=None) -> Tensor
The fallback handles both the simple case (offs=None → torch.mm) and the
grouped case (offs provided → split self by offsets, multiply each group
against the corresponding slice of mat2, then cat results).
Keep _WINDOWS_ROCM_GROUPED_MM_LIB alive at function scope to prevent the
C++ dispatch registration from being freed by GC.
* worker: fix torchao stub — return stub classes not modules for isinstance()
peft/tuners/lora/torchao.py does:
from torchao.dtypes import AffineQuantizedTensor, LinearActivationQuantizedTensor
isinstance(weight, (AffineQuantizedTensor, LinearActivationQuantizedTensor))
The stub __getattr__ was returning stub modules, which isinstance() rejects
with "arg 2 must be a type, a tuple of types, or a union".
Add _StubTypeMeta metaclass whose __instancecheck__ always returns False,
and _make_stub_type() to create stub classes via it. Change _make_mod_stub
__getattr__ to return stub classes instead of stub modules for leaf
attribute access, so isinstance() gets a valid type and returns False.
_StubSubpackageFinder still handles import-style subpackage creation
(those still need module objects in sys.modules); __getattr__ only fires
for from-import or direct attribute access, which are the isinstance paths.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* tests: add coverage for Windows ROCm install paths and worker patches
Add conftest.py to fix pre-existing sys.path issue that prevented
test_rocm_support.py from running at all (install_python_stack.py
imports from backend.utils.wheel_utils which needs studio/ on sys.path).
New test classes cover everything added in this session:
- TestWindowsRocmIndexUrl: arch → AMD pip index URL mapping (gfx120X-all,
gfx1151, gfx1150, gfx110X-all, unknown → None, trailing slash)
- TestDetectWindowsGfxArch: hipinfo output parsing, missing/timeout/bad
returncode/no-gcnArchName paths
- TestInstallBnbWindowsRocm: UV_SKIP_WHEEL_FILENAME_CHECK set+restored,
env restored on exception, no-op when URL missing
- TestRocmTorchInstalledEnvVar: UNSLOTH_ROCM_TORCH_INSTALLED=1 skips
pip_install, calls _install_bnb_windows_rocm, sets flag
- TestWorkerWindowsRocmPatches: _grouped_mm CUDA dispatch override,
offs/grouped variant handling, GC-prevention sentinel,
_StubTypeMeta __instancecheck__, _StubSubpackageFinder registration,
torchao key submodule pre-stubbing, TORCHDYNAMO_DISABLE guard
- TestRocmTorchPkgSpecs: rocm7.2 torch 2.11.x spec, default <2.11 cap,
3-tuple shape, _GFX_TO_AMD_INDEX_ARCH RDNA4/3.5/3 coverage
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* tests: fix encoding, IS_WINDOWS patching, and wrong assertion
- Add encoding="utf-8" to all read_text() calls (54 occurrences) so
tests pass on Windows where the default codec is cp1252 and source
files contain UTF-8 emoji (e.g. ⚠️ in install_python_stack.py)
- Add @patch.object(stack_mod, "IS_WINDOWS", False) to Linux-path
TestEnsureRocmTorch tests so they reach the Linux code path when run
on a Windows machine instead of short-circuiting into the Windows branch
- Fix test_grouped_mm_patch_guarded_by_windows_and_hip_check: the source
uses getattr(_torch_for_rocm, "version", None) not torch.version, so
check for '"version"' and '"hip"' substrings instead
137 passed, 2 skipped
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: pin BNB_ROCM_VERSION=72 for torch==2.11.0+rocm7.13.0 compatibility
AMD's pip index now ships torch==2.11.0+rocm7.13.0 (ROCm 7.13).
bitsandbytes auto-detects HIP 7.13 from torch.version.hip and looks for
libbitsandbytes_rocm713.dll, which the AMD Windows prerelease wheel does
not ship (it only ships rocm72.dll), causing a load error at training start.
Fix:
- worker.py section 1f: set BNB_ROCM_VERSION=72 (via setdefault) before
section 2 ML imports, so bitsandbytes always loads rocm72.dll on Windows ROCm
- install_python_stack.py: set BNB_ROCM_VERSION=72 in _install_bnb_windows_rocm()
for any post-install imports; update comment to document root cause
- tests: 4 new assertions covering the fix (141 passed, 2 skipped)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: detect BNB ROCm DLL suffix dynamically instead of hardcoding '72'
BNB_ROCM_VERSION was pinned to '72' which works today (AMD wheel ships
rocm72.dll) but would break again if AMD ships a future wheel with a
different DLL suffix (e.g. rocm713.dll).
Add _detect_bnb_rocm_dll_ver() to install_python_stack.py: scans the
installed bitsandbytes package dir for libbitsandbytes_rocm{VER}.dll
using importlib.util.find_spec (no BNB import needed) and returns the
suffix. '72' remains the fallback when detection fails.
Apply the same detection inline in worker.py section 1f. Both paths
still respect a pre-set BNB_ROCM_VERSION (caller override wins).
Tests: +8 cases covering detection logic and fallback (147 passed, 2 skipped).
* fix: patch torch.distributed stubs in server process for Windows ROCm
On Windows ROCm, torch.distributed ships without process-group helpers
(is_initialized, is_available, get_rank, get_world_size). The worker
subprocess already patches these in section 1e, but the main server
process calls _determine_attention_impl_for_gpu_estimate() which calls
unsloth's resolve_attention_implementation() → is_initialized(), causing:
"Could not resolve attention implementation for '...':
module 'torch.distributed' has no attribute 'is_initialized'"
Fix: patch the missing attrs onto torch.distributed at the top of
_determine_attention_impl_for_gpu_estimate, matching the same stubs
already applied in worker.py section 1e. No-ops on Linux/CUDA where
torch.distributed is fully populated.
* fix: gate _grouped_mm dispatch patch on HIP < 7.13
AMD fixed the gfx1200 null HIP kernel in ROCm 7.13 (torch 2.11+).
Users on the new wheel now get the real GPU _grouped_mm kernel for
MoE workloads instead of the Python mm fallback.
Changes:
- worker.py: add _hip_ver_at_least() helper; wrap full _grouped_mm
patch in `if not _hip_ver_at_least(7, 13):` with else branch that
logs the skip reason; update section-1f comment to document the fix
- test_rocm_support.py: add 5 tests covering the helper definition,
the (7, 13) gate expression, the else branch, the skip log message,
and the AMD-format version string parsing (.split(".")[:2])
Verified: torch==2.11.0+rocm7.13.0 — 3D batch and grouped (offs)
variants both succeed; null crash only present on rocm7.12 and earlier.
* fix: stub is_torchelastic_launched on torch.distributed for Windows ROCm
resolve_attention_implementation calls is_torchelastic_launched() which
does not exist in the incomplete torch.distributed shipped with the
Windows ROCm wheel, causing a warning on every model config load in the
server process. Add it to the stub table alongside the four helpers
already patched in _determine_attention_impl_for_gpu_estimate.
Also adds two tests: one confirming the new stub and one confirming all
five core distributed helpers are covered.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: explicit warnings on AMD ROCm arch/version fallbacks + Fast-Install arg order
setup.ps1:
- Fix Fast-Install argument order: packages before flags, consistent with
all other Fast-Install calls in the file
(was: Fast-Install --force-reinstall --index-url $url torch ...)
(now: Fast-Install torch torchvision torchaudio --force-reinstall --index-url $url)
- Add explicit [WARN] substep when $HasROCm is true but arch mapping fails:
- GPU arch detected but not in supported wheel list → names the arch and
lists supported families so user knows exactly what to report
- HIP SDK present (amd-smi path) but gcnArchName unreadable → instructs
user to re-install the HIP SDK; previously fell back silently to CPU
install.sh:
- Add [WARN] to stderr before silent CPU fallback when AMD GPU is confirmed
(rocminfo/amd-smi) but ROCm version cannot be read from any source
(amd-smi, /opt/rocm/.info/version, hipconfig, dpkg, rpm)
- Add [WARN] to stderr when ROCm version is too old (< 6.0) with upgrade link
install.ps1 and setup.sh: no changes needed (already handle these paths correctly)
* fix: robust gfx arch detection for Strix Halo / HIP-runtime-only installs
Covers users who have the HIP runtime (amd-smi available) but not the
full HIP SDK (no hipinfo), which is common on Strix Halo iGPU systems.
Without this, $ROCmGfxArch stays null and the installer silently falls
back to CPU-only PyTorch despite a working GPU.
Detection waterfall (setup.ps1 + install.ps1):
1. hipinfo gcnArchName -- full HIP SDK (existing, unchanged)
2. amd-smi list gfx pattern -- newer amd-smi versions embed arch
3. amd-smi static --asic -- ROCm 6+ ASIC details with GFX target
4. UNSLOTH_ROCM_GFX_ARCH env -- manual override escape hatch
5. GPU name → arch table -- best-effort from marketing name:
890M / Strix Halo → gfx1151 (RDNA 3.5 iGPU, Strix Halo)
880M / Strix Point → gfx1150 (RDNA 3.5 iGPU, Strix Point)
780M / Phoenix → gfx1103 (RDNA 3 iGPU)
RX 7900/7800/7700 → gfx1100 (RDNA 3 desktop)
RX 9070 XT / 9080 → gfx1201 (RDNA 4)
RX 9070 / 9060 XT → gfx1200 (RDNA 4)
When arch is inferred from name, a Cyan substep tells the user to set
UNSLOTH_ROCM_GFX_ARCH to skip inference on future installs.
WMI block intentionally does not set $HasROCm (no runtime confirmation).
Tests: 11 new tests in TestStrixHaloGfxArchDetection covering all five
detection levels, WMI safety, and gfx regex in both ps1 files.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: resolve hipinfo/hipconfig via HIP_PATH/ROCM_PATH when not on PATH
AMD HIP SDK sets HIP_PATH on Windows but does not always add the bin
directory to PATH. Get-Command hipinfo therefore silently fails and
detection falls through to WMI, which cannot provide a gfx arch, leaving
the user with a CPU-only PyTorch install and no warning.
Changes:
- setup.ps1 / install.ps1: before falling through to amd-smi, attempt to
locate hipinfo.exe and hipconfig.exe under $env:HIP_PATH\bin (then
$env:ROCM_PATH\bin) when Get-Command returns nothing
- Emit a [WARN] with the resolved path and a one-liner to permanently fix
PATH via SetEnvironmentVariable
- Emit a [WARN] when HIP_PATH/ROCM_PATH is set but the exe is still not
found (incomplete SDK install)
- Emit a [WARN] with the first hipinfo output line when hipinfo runs but
returns a non-zero exit code (e.g. "no ROCm-capable device detected")
- 18 new tests in TestHipSdkEnvPathResolution; total 183 passed, 2 skipped
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* feat: print HIP SDK path and full hipconfig version in terminal on AMD detection
Both install.ps1 and setup.ps1 now emit substeps under the gpu step when
AMD ROCm is detected:
gpu AMD ROCm (gfx1200)
HIP SDK: C:\Program Files\AMD\ROCm\7.1
hipconfig: 7.1.51803-d3a86bd04
Previously only the gpu label (e.g. "AMD ROCm (gfx1200)") was shown with
no indication of where the SDK was found or which exact build was active.
The full hipconfig build string (e.g. 7.1.51803-d3a86bd04 instead of just
7.1) is now stored in ROCmVersionFull and also used in setup.ps1's
'rocm' step label.
9 new tests in TestHipSdkDetectedSubstep; total 192 passed, 2 skipped
* fix: Strix rocm7.1 segfault bypass + Ubuntu 24.04 HIP gcc-install-dir
Issue 1 (install.sh): gfx1151/gfx1150 + ROCm 7.1 causes a segfault in
torch._grouped_mm (moe_utils.py:167). The Radeon repo now ships cp313
wheels for rocm-rel-7.1, so _amd_gpu_radeon=true silently lands on the
broken combo. When Strix Halo/Point is detected and TORCH_INDEX_URL is
rocm7.1, override to rocm7.2 PyTorch index, update TORCH_CONSTRAINT, and
set _amd_gpu_radeon=false to bypass the Radeon repo entirely. Emits a
clear [WARN] explaining the segfault and linking to the ROCm upgrade docs.
Issue 2 (setup.sh): ROCm 7.x ships clang-20 which on Ubuntu 24.04+ picks
/usr/lib/gcc/x86_64-linux-gnu/14/ (runtime dir, no C++ headers), causing
'cstdlib file not found' and a failed llama.cpp HIP build. Iterate gcc
versions 14→11 to find the first install dir that has both runtime and
/usr/include/c++/<ver> headers, then pass --gcc-install-dir to clang via
CMAKE_HIP_FLAGS. Fix confirmed by h34v3nzc0dex (llama.cpp 417/417 clean).
11 new tests across TestStrixRocm71Override and TestSetupShGccInstallDir;
total 203 passed, 2 skipped
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: BNB_ROCM_VERSION in server process + torch._C._distributed_c10d stubs
Two errors visible in training logs on Windows ROCm:
1. Server process bitsandbytes crash:
"Configured ROCm binary not found at libbitsandbytes_rocm713.dll"
The installed BNB wheel ships rocm72.dll (not rocm713.dll). The
training worker already sets BNB_ROCM_VERSION=72 via DLL detection
but the server process (main.py) imported bitsandbytes before that
ran. Fix: add the same DLL-scan + BNB_ROCM_VERSION assignment to
main.py inside the existing win32 guard, before any downstream
import can pull in bitsandbytes.
2. torch.distributed import failure:
"No module named 'torch._C._distributed_c10d'; torch._C is not a package"
torch._C is a C extension on Windows ROCm — Python cannot do
submodule imports from it, so torch.distributed fails to import
before our attribute stubs could ever run. Fix: inject empty
ModuleType stubs for _distributed_c10d, _distributed_autograd and
_distributed_rpc into sys.modules inside the win32 guard in
hardware.py BEFORE importing torch.distributed, so the import
succeeds and our attribute stubs take effect.
9 new tests in TestServerStartupRocmFixes; total 212 passed, 2 skipped
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(win32): populate distributed c10d stub with dummy symbols
torch.distributed tries to `from torch._C._distributed_c10d import
FakeProcessGroup` (and ProcessGroup, Work, Store, etc.). The previous
empty ModuleType stub caused an AttributeError on those names.
Populate every stub with a _Dummy class for each known symbol so the
import chain completes silently on Windows ROCm where torch._C is a
compiled extension and its _distributed_c10d submodule doesn't exist.
Adds four new tests in TestServerStartupRocmFixes covering FakeProcessGroup,
ProcessGroup, setattr population, and all three _distributed_* siblings.
* fix(win32): distinguish HIP SDK installed vs GPU not ROCm-accessible
Previously, when hipinfo was found but exited non-zero (e.g. "no
ROCm-capable device detected"), both install.ps1 and setup.ps1 fell
through to the WMI-label-only branch and printed "AMD GPU detected --
HIP SDK not found" -- factually wrong since the SDK binary is present.
Add $HipSdkInstalled flag (set true when hipinfo binary is found,
regardless of exit code). When HipSdkInstalled && !HasROCm:
- Show "AMD GPU detected -- not ROCm-accessible (HIP <ver>)" instead
- Explain this is a driver issue, not an SDK issue, with a link
- Still run hipconfig version capture so version shows in output
- CPU-only hint now says "GPU not ROCm-accessible" not "require HIP SDK"
Also applies to setup.ps1 (same detection block, same branches).
Adds TestHipSdkInstalledButDeviceInaccessible (11 tests).
* fix(win32): scope ROCm workarounds to AMD hosts only
Three Codex-flagged issues where Windows ROCm workarounds incorrectly
applied to Windows CUDA (NVIDIA) machines:
main.py (P1): BNB_ROCM_VERSION was set unconditionally on all win32
hosts. On NVIDIA, bitsandbytes sees BNB_ROCM_VERSION and looks for a
ROCm DLL that doesn't exist, breaking bitsandbytes initialisation.
Fix: gate the block on HIP_PATH/ROCM_PATH being present (ROCm hosts only).
worker.py (P2): torchao stubs were seeded for all win32 runs, shadowing
real torchao on Windows CUDA and silently disabling torchao quantization
for NVIDIA users. Fix: gate on HIP_PATH/ROCM_PATH (win32 ROCm only).
install_python_stack.py (P1): _detect_windows_gfx_arch() only checked
shutil.which("hipinfo"), skipping the HIP_PATH/ROCM_PATH fallback that
the PowerShell installers use. On installs where the HIP SDK bin dir is
not on PATH, _ensure_rocm_torch() returned early without installing
ROCm wheels or bitsandbytes. Fix: mirror the env-var fallback.
* fix(linux): route Strix + ROCm 7.1 to AMD arch-specific index
Instead of falling back to pytorch.org/rocm7.2, the Strix override now
routes to repo.amd.com/rocm/whl/gfx1151/ (or gfx1150/) which serves
torch 2.11.0+rocm7.13.0 -- AMD's build containing the actual _grouped_mm
kernel fix, verified on real gfx1151 hardware by h34v3nzc0dex.
This exercises the real GPU kernel path rather than the rocm7.2 workaround.
UNSLOTH_AMD_ROCM_MIRROR can override the base URL for air-gapped installs.
Also teaches _tauri_torch_index_family to recognise AMD arch-specific URLs
(repo.amd.com/rocm/whl/gfx*) and return the rocm7.13 family label so
_tauri_gpu_branch correctly classifies these installs as rocm.
Suggested by h34v3nzc0dex based on hardware-verified probe results.
* fix(studio/rocm): gate ROCm-only side-effects on active torch runtime
Address five edge cases flagged during PR review:
1. studio/backend/main.py: BNB_ROCM_VERSION was set whenever HIP_PATH or
ROCM_PATH was present in the environment. A Windows CUDA user who once
installed the HIP SDK and reverted to a CUDA torch wheel still has those
env vars set, so bitsandbytes would try to load libbitsandbytes_rocm72.dll
against a CUDA torch and crash. Now probe torch.version.hip inside the
env-var guard (worker.py already does this).
2. studio/backend/main.py: os.add_dll_directory returned handles were
discarded. Per CPython docs, the directory leaves the DLL search list when
the handle is garbage collected. Retain handles in module-level
_ROCM_DLL_HANDLES list so they survive process lifetime.
3. studio/install_python_stack.py: _install_bnb_windows_rocm() returned None
regardless of pip_install_try outcome, and the caller flipped
_rocm_windows_torch_installed to True unconditionally. On a failed BNB
install the post-install "manual install may be required" warning was
suppressed and the user was misled. Helper now returns bool; caller gates
on it.
4. studio/install_python_stack.py: _detect_windows_gfx_arch returned the raw
capture group, so mixed-case hipinfo output ("Gfx1151") missed the
lowercase keys in _GFX_TO_AMD_INDEX_ARCH and silently fell back to CPU
torch. Lowercase the token.
5. studio/install_python_stack.py: UNSLOTH_ROCM_TORCH_INSTALLED=1 early-
return trusted the env var even when the venv was wiped between runs.
Subprocess-probe torch importability first; fall through to the full
install path if the probe fails.
Tests: 231 passed, 1 skipped in tests/studio/install/test_rocm_support.py
(adds one new test for case 5 fall-through).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio/rocm): worker.py parity + don't roll back ROCm torch on bnb failure
Addresses findings from a 10x reviewer pass on the prior fix commit:
1. studio/backend/core/training/worker.py (parity with main.py):
- Gate the torchao stub block on torch.version.hip / 'rocm' in
torch.__version__ instead of HIP_PATH / ROCM_PATH env-var presence.
Same root cause as main.py: HIP SDK env vars stick around on CUDA hosts.
- Add module-level Windows ROCm DLL registration block. Worker subprocesses
inherit env vars but not the parent's add_dll_directory handles, so the
first `import torch` in the worker could fail to find amdhip64.dll when
HIP_PATH\bin is not on PATH. Mirrors main.py setup. Handles retained at
module scope via _ROCM_DLL_HANDLES.
- Promote _WINDOWS_ROCM_GROUPED_MM_LIB to module scope with `global` in
run_training_process so the torch.library.Library registration survives
past function return / mid-run garbage collection.
- Harden _torch_has_hip() to also accept 'rocm' in torch.__version__
(AMD SDK / Radeon wheels may not set torch.version.hip).
2. studio/install_python_stack.py:
- Don't roll back ROCm torch when bitsandbytes install fails. The prior
commit gated _rocm_windows_torch_installed on _install_bnb_windows_rocm()
returning True; if torch installed successfully but bnb failed, the flag
stayed False and later install steps could overwrite ROCm torch with the
generic CPU torch wheel. Set the flag after torch install; surface bnb
failure as a separate warning instead.
- _detect_windows_gfx_arch now probes in three tiers: UNSLOTH_ROCM_GFX_ARCH
env-var override (matches the PowerShell installer), then hipinfo (PATH
or HIP_PATH\bin), then amd-smi (`static --asic`, `list`). Without the
amd-smi fallback, runtime-only Radeon installs without hipinfo on PATH
made `studio update` return early and leave the venv on CPU torch.
- Linux torch-already-rocm probe in _ensure_rocm_torch now matches the
Windows probe shape: accepts torch.version.hip OR 'rocm' in
torch.__version__ to cover AMD SDK / Radeon Linux wheels.
3. studio/backend/utils/hardware/hardware.py:
- apply_gpu_ids() final-fallback torch probe accepts 'rocm' in
torch.__version__ in addition to torch.version.hip, matching
detect_hardware(). AMD SDK wheels could otherwise leak through with
CUDA-only visibility masks on a spawned ROCm worker.
Tests: 231 passed, 1 skipped in tests/studio/install/test_rocm_support.py
(no test changes needed; the probe shape that prints the hip version (or
'rocm' sentinel) preserves the existing non-empty-string contract).
Not addressed in this commit (deferred or out of scope):
- Tag drift / lemonade checksum (PR 5303 surface, not this PR).
- install.sh rocm7.2.1 URL: small fix, separate.
- install.ps1 / setup.ps1 'Radeon 8060S' marketing-name fallback table.
- Strix Halo + ROCm 7.1 routing asymmetry in Python update path.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio/rocm): robustness pass - rocm tag normalisation, Strix routing parity, hardened detection
Robustness pass on top of 76137b2d. Four targeted fixes:
1. install.sh ROCm-tag routing normalisation.
`rocm7.2.1` would route to https://download.pytorch.org/whl/rocm7.2.1
which does not exist (PyTorch publishes major.minor URLs only). Same
for any future patch-level tag. Normalise every rocm{maj.min}* pattern
to the bare {maj.min} index URL.
2. install.ps1 + studio/setup.ps1 marketing-name fallback.
The gfx1151 row matched 890M / Strix Halo / HX 37x / HX 38x / AI 9 HX
but not the actual retail name 'AMD Radeon 8060S Graphics' shipped by
OEMs (Ryzen AI MAX+ 395). Add '8060S' to the regex.
3. install_python_stack.py Strix + ROCm 7.1 routing parity with install.sh.
The shell installer reroutes Strix Halo / Point + ROCm 7.1 to
repo.amd.com/rocm/whl/{gfx}/ (which serves torch 2.11.0+rocm7.13.0
with the upstream _grouped_mm fix). The Python `studio update` path
only warned and still installed the broken generic rocm7.1 wheel.
Mirror the override: detect gfx1151/gfx1150 on ROCm 7.1, route to
the AMD per-gfx index, honour UNSLOTH_AMD_ROCM_MIRROR override.
4. _detect_windows_gfx_arch amd-smi parsing tightened.
The amd-smi fallback added in the prior commit used a bare
`\bgfx[1-9][0-9a-z]{2,3}\b` match against the lowercased stdout,
which could pick up stray gfx references in warnings / device-name
strings. Anchor on labelled lines first (Target_Graphics_Version,
ASIC, Arch, gfx) and fall back to the bare match only when no
labelled line is present.
Tests: 231 passed, 1 skipped in tests/studio/install/test_rocm_support.py;
sim_5301 23 cases pass (6 new sims for the Strix override + amd-smi parsing).
* fix(studio/rocm): multi-GPU selection, Strix sibling handling, defensive cleanups
Round 4 robustness pass based on 5 parallel Opus reviewers of head 21773215.
Seven items from across regression / edge-case / error-paths / architecture
reviews:
1. studio/backend/main.py BNB gate: aligned with the broad ROCm check used
everywhere else in this PR (torch.version.hip OR 'rocm' in __version__).
AMD SDK / Radeon Linux wheels do not always populate torch.version.hip;
without this, main.py would silently skip BNB_ROCM_VERSION while worker.py
set it.
2. studio/install_python_stack.py _install_bnb_windows_rocm: init _ok = False
before the try block. Without this, if pip_install_try itself raises
(e.g. OSError on uv binary missing), the finally block restored env vars
correctly but the subsequent `if not _ok:` raised UnboundLocalError,
masking the original exception.
3. studio/install_python_stack.py _detect_windows_gfx_arch:
- Rewrote to use re.findall (not re.search) on both hipinfo and amd-smi
output, dedup tokens preserving order, and select via new
_pick_visible_index() helper.
- HIP_VISIBLE_DEVICES / ROCR_VISIBLE_DEVICES (first comma entry, integer)
now picks the right GPU on multi-AMD-GPU hosts. Out-of-range or non-int
values fall back to the first GPU (matches detect_host behaviour in
install_llama_prebuilt.py).
4. studio/install_python_stack.py Strix override now consults the runtime
target before flipping:
- Previous behaviour intersected gfx_codes with {gfx1151, gfx1150} and
picked the first Strix arch, ignoring whether HIP_VISIBLE_DEVICES
selected a non-Strix sibling (e.g. discrete RX 7900 in a mixed APU+dGPU
box). Could install Strix-specific wheels onto a gfx1100 dGPU.
- Now resolves the runtime gfx via _pick_visible_index() and only
overrides when that runtime target is in the Strix set.
5. studio/backend/main.py + studio/backend/core/training/worker.py: ROCm
version dir scan no longer sorts lexically. Previous sort placed "10.0"
before "7.0" alphabetically, which would mis-prioritise ROCm 10.x bin
dirs once AMD ships them. New _ver_key() splits on "." and sorts
numerically with a string fallback.
6. install.sh Strix override URL: replaced ${var%/} (strips one trailing
slash) with a while-loop that strips all trailing slashes, matching
Python's .rstrip("/"). A user setting UNSLOTH_AMD_ROCM_MIRROR with
"http://corp/whl///" no longer ends up with "http://corp/whl///gfx1151/"
which strict pip proxies (artifactory, sonatype) 404 on.
7. studio/install_python_stack.py: bumped torch import probe timeout from
30s to 90s. PyTorch's lazy .so loading can take 60-90s on cold NFS or
USB-backed venvs. The shorter timeout was producing a false "torch
missing" classification and reinstalling a working ROCm torch.
Tests: 231 passed, 1 skipped. sim_5301 30 cases pass (added 7 new sims for
multi-GPU detection, Strix sibling handling, and _ok-init regression).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio/rocm): worker BNB/grouped_mm broad gate, install.sh Strix visibility, runtime-only ROCm detection
Round-5 robustness pass based on 20 parallel reviewers of head 96b9e465.
1. studio/backend/core/training/worker.py - BNB version pin / dynamo disable
/ _grouped_mm fallback block was still gated on torch.version.hip alone
despite the torchao stub block above already using the broad check. AMD
SDK / Radeon Windows wheels (torch.__version__ contains "rocm" but
torch.version.hip is None) silently skipped the Windows ROCm runtime
patches. Aligned to the same broad check (8/20 reviewers).
2. studio/backend/core/training/worker.py - _hip_ver_at_least() now also
parses the ROCm version out of torch.__version__ (e.g. "2.11.0+rocm7.13.0")
when torch.version.hip is missing, so the kernel-fix gate is correct for
SDK / Radeon wheels too.
3. studio/backend/core/training/worker.py - _grouped_mm_safe_impl with
offs=None now picks torch.bmm/matmul for 3-D inputs instead of always
calling torch.mm. The real _grouped_mm accepts 3-D batched matmul; the
prior fallback raised "self must be a matrix" on MoE workloads (2/20).
4. studio/backend/main.py - dropped the HIP_PATH / ROCM_PATH env-var gate
from the BNB block; probe torch directly. Runtime-only Radeon / AMD SDK
Windows installs do not set those SDK env vars but still ship ROCm torch
(5/20 reviewers).
5. install.sh - Strix override now collects every gfx token from
rocminfo / amd-smi (in enumeration order), then indexes by
HIP_VISIBLE_DEVICES / ROCR_VISIBLE_DEVICES so a mixed Strix iGPU + non-
Strix dGPU host where the user selected the dGPU does NOT get rerouted
to the Strix per-gfx index. Mirrors the Python update path (5/20 reviewers).
6. install.sh - Strix detection chain now also probes `amd-smi static --asic`,
matching the PowerShell installer (1/20). Closes the gap on runtime-only
Strix hosts where `amd-smi list` does not surface a gfx token.
7. studio/install_python_stack.py - _has_rocm_gpu() now has the sysfs KFD
topology fallback (/sys/class/kfd/kfd/topology/nodes/*/gpu_id), matching
install.sh. On minimal package-managed installs without rocminfo /
amd-smi GUI tools, `studio update` can now detect the GPU and repair the
venv instead of returning early (2/20).
8. studio/install_python_stack.py - _detect_amd_gfx_codes() now falls back
to `amd-smi list` and `amd-smi static --asic` when rocminfo is missing
(2/20). Strix routing on runtime-only Radeon hosts now matches what
install.sh has done for a while.
9. studio/install_python_stack.py - Strix override now applies even when
has_hip_torch is True. The whole point of the override is to repair an
existing broken torch.version.hip == "7.1" install; skipping the
reinstall left users on the known _grouped_mm segfaulting stack (3/20).
Tests: 231 passed, 1 skipped. sim_5301 30 cases pass. sim_cross 12 pass.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio/rocm): code review hardening pass
- main.py: numeric DLL sort (string sort picked rocm72 over rocm713);
add basename() to regex; log warning on detection failure; log info
when BNB_ROCM_VERSION is set (mirrors worker.py)
- worker.py: explicit len-guard in _hip_ver_at_least() with warning
logs instead of silent IndexError/ValueError swallow
- hardware.py: isinstance(result, dict) guard before result.get() in
_smi_query() to prevent AttributeError on non-dict backend returns
- amd.py: round() before int() on parsed GPU IDs; log warning when
truncation occurs (defensive against malformed amd-smi output)
- setup.sh: quote --gcc-install-dir value in CMAKE_HIP_FLAGS so paths
with spaces do not break the CMake argument
- install.ps1, setup.ps1: apply colon-split + ToLower() to hipinfo
gcnArchName match (consistent with each other and with setup.sh)
- install.sh: tighten ROCm tag case patterns to explicit
rocmX.Y|rocmX.Y.* to avoid unintended prefix matches
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio/training): GPU OOM guard to prevent system freeze on VRAM exhaustion
On RDNA 4 (gfx1200/gfx1201) and other ROCm GPUs, exhausting VRAM can
cause a HIP driver hang that freezes the entire system rather than
raising a recoverable Python exception.
Two-part fix:
- set_per_process_memory_fraction(0.90) caps the HIP/CUDA allocator at
90% of VRAM so PyTorch raises OutOfMemoryError before hitting the
hardware limit, keeping the driver alive and the system responsive
- top-level exception handler detects OOM errors by type and message
and surfaces a clear actionable message to the UI (reduce
max_seq_length, enable gradient_checkpointing, lower batch size)
instead of the raw CUDA/HIP error string
* fix(studio/rocm): OOM guard ROCm-only + unified memory, multi-GPU arch selection
OOM guard (worker.py):
- Scope to _hw.IS_ROCM only -- NVIDIA CUDA has a graceful OOM path and
does not need the allocator cap
- Detect unified memory by comparing torch VRAM against psutil system RAM;
use 0.80 on unified-memory APUs (gfx1151 Strix Halo) where the GPU pool
is carved from host RAM, 0.90 on discrete cards
Multi-GPU arch selection:
- install.ps1 / setup.ps1: replace -match (first hit only) with
[regex]::Matches() to collect all gcnArchName entries, then index by
HIP_VISIBLE_DEVICES / ROCR_VISIBLE_DEVICES
- install_python_stack.py: index into full token list before dedup so
HIP_VISIBLE_DEVICES=2 on [gfx1100, gfx1100, gfx1151] resolves gfx1151
- install.sh: remove awk dedup from gfx token collection for same reason
GCC multiarch (setup.sh):
- Only append -linux-gnu when gcc -print-multiarch does not already return
the full triple, fixing double-suffix on Ubuntu 24.04
* fix(tests): update ROCm version cap expectations from rocm7.1 to rocm7.2
Daniel's normalisation commit updated the cap from rocm7.1 to rocm7.2
since PyTorch now publishes that index and rocm7.2 ships torch 2.11.0.
Test expectations were stale.
* fix(tests): correct MLX smoke test losses_per_step assertion
logging_steps=1 with max_steps=30 produces 30 loss entries, not 7.
The assertion was stale from a previous config.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio/worker): detect unified-memory APU by GPU name not VRAM/RAM ratio
The previous heuristic (VRAM > 50 % of system RAM) false-positived on discrete
cards in low-RAM systems — e.g. RX 9060 XT 16 GB on a 16 GB or 24 GB machine
would trip the unified-memory path and log "unified memory host" when it should
say "discrete".
AMD iGPUs (gfx1150/gfx1151 Strix Halo, Strix Point, etc.) expose names with a
digit+M suffix ("AMD Radeon 890M"), while discrete cards use "RX NNNN [XT|XTX]"
naming. Matching that suffix is reliable across all current ROCm-capable AMD
consumer GPUs and does not require psutil.
Also includes the device name in the log line to ease future debugging.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(install/setup.ps1): force array on hipinfo gcnArchName parse to fix single-GPU arch truncation
When [regex]::Matches() finds exactly one match, PowerShell's pipeline
unwraps the result to a scalar string. Indexing a scalar string with [0]
returns the first *character*, so a one-GPU system would parse
gcnArchName "gfx1200" as "g", which is not in the supported arch map
and triggers the CPU-only fallback.
Wrapping with @() forces the result to remain an array regardless of
match count. On a single-GPU machine the arch is now correctly read as
"gfx1200" (or whatever the full name is) so the ROCm wheel index is
selected.
Reproducer: hipinfo exits 0 and outputs exactly one gcnArchName line.
Without @(), $_hipAllArches = "gfx1200" (String); $_hipAllArches[0] = 'g'.
With @(), $_hipAllArches = @("gfx1200") (Object[]); $_hipAllArches[0] = "gfx1200".
* fix(studio/rocm): classify unified-memory APU via VRAM/RAM ratio, not arch list
Replace the gcnArchName allowlist {gfx1150, gfx1151} with a
psutil-based heuristic: unified APUs expose the entire system RAM
as the HIP pool (ratio ≥ 0.90), discrete cards are well below that.
No arch name required — future APUs classify correctly without code changes.
Also removes the stale import re / \d[Mm]\b device-name regex that
5d84704 left behind, and logs vram/sys GiB for easier on-hardware
verification.
Addresses h34v3nzc0dex review: Radeon 8060S (gfx1151, 128 GiB
unified) now correctly gets 0.80 cap instead of 0.90.
* fix(studio/rocm): revert to gcnArchName for unified-memory APU classification
VRAM/RAM ratio >= 0.90 false-positives on machines where discrete VRAM
equals system RAM (e.g. RX 9060 XT 16 GB + 16 GB system RAM → ratio 1.0,
incorrectly classified as unified → wrong 0.80 cap applied).
gcnArchName is the correct signal: naming-independent, stable within a
product family, and already parsed throughout this PR. Unified set is
{gfx1150, gfx1151} (Strix Point + Strix Halo).
* fix(studio/llama-prebuilt): resolve hipinfo via HIP_PATH/ROCM_PATH on Windows
shutil.which("hipinfo") returns None when the HIP SDK bin dir is not on
PATH -- the HIP SDK installer sets HIP_PATH/ROCM_PATH but does not always
add the bin dir to PATH. This caused has_rocm=False in the prebuilt asset
selector, so AMD ROCm machines got the CPU llama.cpp zip instead of the
HIP one, silently running all chat inference on CPU.
Add _resolve_exe() that falls back to %HIP_PATH%\bin and %ROCM_PATH%\bin
when shutil.which() finds nothing, mirroring the same fallback already
present in setup.ps1.
* fix(studio/llama-prebuilt): pass --has-rocm from setup.ps1 to skip re-detection
The Python prebuilt installer re-detects ROCm independently via
shutil.which("hipinfo"), which fails when hipinfo is not on PATH
(HIP SDK sets HIP_PATH but doesn't always add the bin dir to PATH).
This caused has_rocm=False and downloaded the CPU llama.cpp zip even
on confirmed AMD ROCm machines.
setup.ps1 already performs reliable ROCm detection with its own
HIP_PATH/ROCM_PATH fallback. Add --has-rocm flag to
install_llama_prebuilt.py so setup.ps1 can forward its result directly,
and pass it whenever $HasROCm is true. The Python script then overrides
has_rocm=True in the HostInfo without re-probing.
* fix(studio/llama-prebuilt): add HIP asset to simple-policy Windows path
direct_upstream_release_plan (used by --simple-policy, which setup.ps1
always passes) only checked has_usable_nvidia on Windows and fell
straight to CPU for AMD ROCm machines, ignoring has_rocm entirely.
The --has-rocm override had no effect because the simple-policy code
path never reached resolve_asset_choice where has_rocm was checked.
Add an elif branch for has_rocm that tries the upstream HIP asset
(llama-TAG-bin-win-hip-radeon-x64.zip) before falling through to the
CPU fallback, consistent with the non-simple-policy path.
* fix(studio/setup.ps1): auto-remove mismatched llama.cpp install kind
When an existing llama.cpp install is the wrong kind for the current
GPU (e.g. windows-cpu on an AMD ROCm machine that should have
windows-hip), the prebuilt installer skips on tag match and never
upgrades. Read install_kind from UNSLOTH_PREBUILT_INFO.json before
invoking the installer and remove the directory if the kind doesn't
match, forcing a fresh download of the correct variant.
* fix(studio/setup.ps1): show live PyTorch install output in verbose mode for ROCm
The ROCm torch reinstall (setup.ps1 phase) always silently captured
output, so in --verbose mode the torch downgrade mid-install
(2.11.0+rocm → 2.10.0 → 2.11.0+rocm) looked like the final state was
2.10.0. Match the CPU/CUDA blocks which show live uv output when
$script:UnslothVerbose is set.
* fix(rocm/windows): set ROCBLAS_TENSILE_LIBPATH for bundled rocblas.dll
The llama.cpp ROCm prebuilt bundles rocblas.dll next to the binary but
not the Tensile kernel library files it depends on at runtime
(rocblas/library/TensileLibrary*.dat + *.hsaco). The bundled DLL
searches for these files relative to its own location by default, i.e.
<binary_dir>/rocblas/library/, which does not exist in the prebuilt
install tree. This causes a silent crash on the very first GEMM
(prefill) with no output from llama-server, seen by the caller as
WinError 10054 / 10061. Model load and the single-token warmup pass
because they use simpler code paths that do not trigger rocBLAS GEMM.
Fix: set ROCBLAS_TENSILE_LIBPATH in the subprocess env to
<HIP_PATH>/bin/rocblas/library so the bundled DLL finds the kernel
files from the system ROCm installation. Uses setdefault so a user-
supplied env var is never overwritten. No-ops on CUDA and CPU (no
HIP_PATH) and on Linux (win32 branch only).
Reproducer log:
rocBLAS error: Cannot read .../Release/rocblas/library/TensileLibrary.dat
rocBLAS error: Could not initialize Tensile host:
directory_iterator: The system cannot find the path specified.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(install.sh): restore gfx token dedup in Strix multi-GPU awk indexer
536a54df removed the per-source `| awk '!seen[$0]++'` dedup from the
_gfx_all collection step but left the indexer awk as bare NF, so on a
mixed-arch host (e.g. dGPU gfx1100 + Strix iGPU gfx1151) where
rocminfo emits each gfx token twice (Name: field + ISA triple),
HIP_VISIBLE_DEVICES=1 indexed vals[1] = the second gfx1100 occurrence
instead of gfx1151, triggering the Strix routing on the wrong GPU.
Add !seen[$0]++ to the indexer awk so duplicate tokens from the same
GPU collapse to one entry before the HIP_VISIBLE_DEVICES index is
applied -- matching exactly what the Python side does with dict.fromkeys()
in _detect_amd_gfx_codes(). The comment above the block ("skip
duplicates") already documented this as the intended behaviour.
* fix(studio/install): correct _TOTAL progress count on Windows
base_total += 3 fired for all non-macOS platforms including Windows,
but flash-attn (line 1620) and ROCm torch final (line 1705) are both
guarded by 'not IS_WINDOWS and not IS_MACOS', so on Windows with torch
enabled _TOTAL was 13 while only 11 _progress() calls actually execute.
Split into +1 for the ROCm torch check (all non-macOS) and +2 for the
two Linux-only steps, so Windows gets _TOTAL=11 and Linux gets 14.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(install.ps1): enforce torch>=2.11.0 for gfx120X and Strix on Windows
The AMD arch-specific index (repo.amd.com/rocm/whl/gfx120X-all/ and
gfx1151/) publishes torch wheels from 2.7.1 through 2.11.0. Without a
version floor pip can resolve to torch 2.10.0+rocm7.12 on RDNA 4
(gfx120X) or torch 2.10.0+rocm7.1 on Strix (gfx1151/gfx1150), both of
which have a null-pointer crash in torch._C._grouped_mm (TheRock
issues #5284 / #3284). torch 2.11.0+rocm7.13 contains the fix.
Add $ROCmTorchFloor alongside $ROCmIndexUrl: set to torch>=2.11.0 for
the two affected arch families, null for all others. Wire it into the
uv pip install call so the broken wheels are never selected.
* fix(rocm/windows): address Codex nits - deterministic DLL suffix, CUDA llama.cpp kind, HIP_VISIBLE_DEVICES arch indexing
- install_python_stack.py / worker.py: _detect_bnb_rocm_dll_ver() and the
inline worker probe now collect ALL libbitsandbytes_rocm*.dll suffixes and
return max() by numeric value instead of stopping at the first glob hit.
Filesystem glob order is not guaranteed; this ensures '713' always wins
over '72' when both variants are present in the wheel.
- setup.ps1 (expectedKind): add 'windows-cuda' branch so NVIDIA hosts are
not treated as 'windows-cpu'. Previously an existing windows-cuda prebuilt
was always considered a mismatch on non-ROCm machines, forcing an
unnecessary re-download on every update.
- setup.ps1 (amd-smi gfx arch): collect ALL gfx tokens from amd-smi list
output in GPU order and honour HIP_VISIBLE_DEVICES / ROCR_VISIBLE_DEVICES
when selecting which arch to use. On mixed-arch AMD systems where the
visible GPU is not the first enumerated one, this prevents installing an
incompatible wheel index. Falls back to index 0 (same as before) when the
visibility var is unset or is a comma-separated list.
- test_rocm_support.py: add test_picks_highest_suffix_when_multiple_dlls to
cover the multi-DLL case that was previously untested.
* fix(rocm): misleading amd-smi log, BNB spec consistency, torch ceiling for AMD index
amd.py: split 'returncode != 0 or not stdout' into two separate branches.
Previously, exit-0 with empty output logged 'amd-smi returned code 0' (which
reads as success, not a warning) and incorrectly incremented the circuit-breaker
counter. Now: non-zero exit logs the code and counts toward the limit as before;
empty stdout on exit 0 logs at DEBUG level and does not penalise the counter
(amd-smi --json always emits at least [] on exit 0, so this branch is rare and
is not a tool failure).
main.py: replace spec.origin / os.path.dirname() with
spec.submodule_search_locations to match install_python_stack.py and worker.py.
For normal wheel installs both approaches reach the same directory, but using
submodule_search_locations is the canonical way and handles editable bitsandbytes
installs correctly. Also use max() by numeric suffix (same as the other two sites)
instead of a sort-then-break loop.
install.ps1: add <2.12.0 ceiling to the torch constraint for gfx120X (RDNA 4)
and gfx1151/gfx1150 (Strix). AMD actively publishes new versions on their
per-arch index; without a ceiling, a future 2.12.0+rocmX.Y wheel would be
pulled in automatically before being validated on these architectures. The
ceiling matches the existing Linux install_python_stack.py constraint for the
same arches. Bump both when 2.12.x is confirmed working.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(rocm): torch floor in setup.ps1, torchvision pin for Strix, rocmsdk in _hip_ver_at_least
setup.ps1: add \ (mirrors install.ps1) and derive \
from it. Previously the AMD index install called 'Fast-Install torch torchvision
torchaudio --force-reinstall --index-url \' with no version
constraint, so pip could resolve torch 2.10.0+rocm7.12 for gfx1151/gfx1200 --
the exact broken wheel the PR is meant to avoid. Now gfx120X and Strix enforce
'torch>=2.11.0,<2.12.0', matching install.ps1 and the Linux constraint.
install_python_stack.py: pin torchvision and torchaudio in _strix_override_pkgs.
The Strix Linux override uses --index-url (exclusive, no PyPI fallback); bare
unversioned 'torchvision' and 'torchaudio' could resolve a build from AMD's
index targeting a different torch major, causing ABI/version mismatches at
runtime. Now pinned to '>=0.26.0,<0.27.0' and '>=2.11.0,<2.12.0' respectively,
matching _ROCM_TORCH_CONSTRAINT['rocm7.2'].
worker.py: extend _hip_ver_at_least to handle AMD SDK wheel version strings.
The fallback regex r'rocm(\d+)\.(\d+)' cannot match '2.9.0+rocmsdk20251116'
(no rocmX.Y component), so the function always returned False on SDK/Radeon
wheels -- installing the Python _grouped_mm workaround on wheels that already
have the working HIP kernel. Added a second check: if the version string
contains '+rocmsdk', assume >= 7.13 (the rocmsdk format post-dates the
gfx120X null-kernel fix) and skip the fallback.
* fix(rocm): warn on OOB HIP_VISIBLE_DEVICES, bail on empty numeric_ids mask
- setup.ps1: when HIP/ROCR_VISIBLE_DEVICES names an index beyond the
detected GPU count, emit a yellow warning and fall back to GPU 0
instead of silently reading allGfxArches[-1] (wrong arch)
- hardware.py _reconcile_primary_rocm_unified_memory: distinguish
numeric_ids=None (no env var, use torch ordinal 0) from numeric_ids=[]
(empty mask / HIP_VISIBLE_DEVICES=-1, no GPU visible); bail out early
in the empty case to avoid querying torch.device(0) incorrectly
* fix(rocm): gate StubSubpackageFinder on win32 ROCm, add gcnArchName fallbacks
- worker.py _StubSubpackageFinder: the meta_path append was running on
every platform on every call to run_training_process; moved it inside
the if _is_win32_rocm: block since stubs are only seeded there and the
finder is a pure accumulation on Linux/Windows CUDA
- worker.py OOM guard: AMD SDK / Radeon wheels may not populate
gcnArchName, causing Strix Halo to be misclassified as discrete and
get the 0.90 cap (12.8 GB OS headroom) instead of 0.80 (25.6 GB);
now tries gcn_arch_name / arch_name / gfx_arch_name variants first,
then falls back to device-name matching (890M -> Strix Halo,
880M -> Strix Point) with a debug log when the fallback fires
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(rocm): pin torchvision/torchaudio in setup.ps1, remove -Unique from arch array
- setup.ps1 ROCm torch install: torchvision and torchaudio were passed
bare alongside pinned torch>=2.11.0,<2.12.0 for gfx1151/gfx1200 arches.
AMD publishes packages independently so a future torchvision 0.27 (for
torch 2.12) on the same arch index would cause pip ResolutionImpossible
or an ABI-incompatible install. Added torchvisionFloorMap and
torchaudioFloorMap mirroring install_python_stack.py's strix override
(torchvision>=0.26.0,<0.27.0, torchaudio>=2.11.0,<2.12.0) and derived
ROCmVisionSpec/ROCmAudioSpec used in all three Fast-Install call sites.
- setup.ps1 amd-smi arch detection: Select-Object -Unique was collapsing
same-arch multi-GPU arrays (e.g. two gfx1151 APUs -> 1-element array)
causing HIP_VISIBLE_DEVICES=1 to trigger a false out-of-range warning
and fall back to GPU 0 even though the correct GPU would have been at
index 1. Removed -Unique; added comment noting the positional-index
assumption and its non-contiguous-GPU limitation.
* fix(rocm): add 8060s/8050s to OOM guard device-name fallback, extract classifier helper
Path 3 of the OOM guard device-name fallback only checked for 890m/880m
(gfx1150 Strix Point SKU names). Strix Halo (gfx1151) ships as Radeon 8060S
(Ryzen AI MAX+ 395) and Radeon 8050S (cut-down SKU) -- neither matches, so
the fallback returned is_unified=False and applied the 0.90 fraction instead
of 0.80, leaving ~12.8 GiB OS headroom on a 128 GiB pool instead of ~25.6 GiB.
Fix: add 8060s and 8050s to the name-match set. Also correct the comment that
mislabelled 890M as a Strix Halo name (it is Strix Point).
Refactor: extract the three-path classifier into _rocm_classify_unified_memory()
so it can be unit-tested directly. Add 31 test cases in test_rocm_oom_guard.py
covering all three paths and the regression case (Radeon 8060S Graphics).
Reported-by: h34v3nzc0dex
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(rocm): pass explicit dtype on bf16-unsupported hardware (RDNA2)
dtype=None lets unsloth auto-detect the model dtype. On RDNA2 (gfx103x,
e.g. RX 6600) is_bfloat16_supported() incorrectly returns True, so unsloth
picks bf16 and the first bf16 kernel dispatch triggers:
LLVM ERROR: Cannot select: intrinsic %llvm.amdgcn.fdot2.bf16.bf16
Replace every dtype=None in load_model() with _auto_dtype which resolves
to None when bf16 is supported (all modern NVIDIA + RDNA3+) and
torch.float16 otherwise. This gives RDNA2 users a working float16
training path without touching NVIDIA behaviour at all.
Fixes: https://github.com/unslothai/unsloth/issues/5337
* fix: reduce log noise for expected non-issues on Windows ROCm
Three log lines fired at warning/error level for conditions that are
completely expected on a Windows HIP SDK-only setup:
amd.py
- amd-smi WinError 2 (FileNotFoundError): downgrade warning -> debug.
amd-smi ships with Adrenalin, not the HIP SDK; absence is normal.
- 'disabling' message: downgrade warning -> info with clearer text
'not available (not installed; expected on HIP SDK-only systems);
GPU VRAM polling disabled'
hardware.py
- torch.distributed.Store missing: downgrade warning -> debug.
The distributed stub added in this PR intentionally omits Store; the
attention-impl fallback to eager is expected and non-actionable.
worker.py
- causal-conv1d: add early Windows exit (info) in both
_ensure_causal_conv1d_fast_path and _causal_conv1d_install hook;
no cp313/win_amd64 wheel exists, so the install always fails.
- FLA: add early Windows exit (info) in
_ensure_flash_linear_attention_unconditional; triton dependency has
no cp313/win_amd64 wheel.
- Defense-in-depth: _install_package_wheel_first non-HIP PyPI failure
logs info+debug on Windows instead of error; FLA failure logs
info+debug on Windows instead of warning.
* [AMD] FIx installation of bitsandbytes when it's from .dev and skip rebuilding llama.cpp if we build it manually.
* fix: use force_pip for Windows ROCm bitsandbytes prebuilt wheel install
uv rejects the bnb continuous-release wheel due to filename/metadata
version mismatch (1.33.7.preview vs 0.50.0.dev0). Switch to force_pip=True
(pip bypass) instead of the UV_SKIP_WHEEL_FILENAME_CHECK env var workaround
-- cleaner and consistent with how the Linux path handles it.
BNB_ROCM_VERSION is still set post-install to the detected DLL suffix so
the worker subprocess loads the correct libbitsandbytes_rocm{VER}.dll even
when torch.version.hip reports a newer HIP version than the wheel ships.
* fix: three small correctness fixes found in PR review
- _install_bnb_windows_rocm: use UV_SKIP_WHEEL_FILENAME_CHECK=1 with
try/finally instead of force_pip=True so the env var is always
restored and the failing CI test passes
- _determine_attention_impl_for_gpu_estimate: gate torch._C distributed
stubs on IS_ROCM so Windows CUDA users keep the real extension
- install.ps1 amd-smi fallback: collect all gfx tokens and index by
HIP_VISIBLE_DEVICES, matching the hipinfo path on multi-GPU hosts
* fix: stub torchao in export subprocess on Windows ROCm
On Windows, the ROCm build of PyTorch ships without the distributed
C extension (torch._C._distributed_c10d). torchao, which is pulled in
transitively by transformers.quantizers at import time, walks into
torch.distributed._functional_collectives -> distributed_c10d and
crashes with:
No module named 'torch._C._distributed_c10d'; 'torch._C' is not a package
This only affected the export subprocess because the training subprocess
already applied an identical torchao stub (introduced separately to fix
the same root cause). The export subprocess had no such guard and died
during 'Importing Unsloth...' before any model loading could happen.
Fix: apply the same _StubSubpackageFinder / torchao stub pattern to the
export subprocess entry point, gated on Windows ROCm detection, before
any import of transformers or unsloth_zoo.
Root cause tracked in ROCm/TheRock#3284 (libuv / torch.distributed
missing on Windows ROCm builds).
Ref: https://github.com/ROCm/TheRock/issues/3284
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install.sh, setup.sh: add GPU arch step logging to match PS1 scripts
Both shell scripts were missing the step "gpu" terminal log block that
install.ps1 and setup.ps1 emit. This adds equivalent output: GPU label
with gfx arch (e.g. "AMD ROCm (gfx1151)"), ROCm root path, hipconfig
version, and marketing name substep. Includes the same gfx arch detection
chain (rocminfo → amd-smi list → amd-smi static --asic), UNSLOTH_ROCM_GFX_ARCH
env override, and name-based arch inference table (Strix Halo/Point, RDNA 3/4)
as the PS1 versions. install.sh also replaces bare echo blocks for the AMD
ROCm and CPU-only cases with formatted substep output.
* Fix BNB_ROCM_VERSION gate, ROCm GPU mask preference, APU unified memory and Release build for PR #5301
- main.py: gate BNB_ROCM_VERSION on the rocm bnb DLL or HIP_PATH/ROCM_PATH instead of importing torch on every Windows host
- hardware.py: prefer HIP/ROCR visible-device masks only on ROCm hosts so a stale mask cannot override CUDA_VISIBLE_DEVICES on NVIDIA
- llama_cpp.py: set GGML_CUDA_ENABLE_UNIFIED_MEMORY=1 only for unified-memory APUs (gfx1150/gfx1151)
- setup.sh: pass -DCMAKE_BUILD_TYPE=Release for the HIP source build
- add test_amd_apu_unified_memory.py
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: guard recompile_limit + fix AMD VRAM monitor fallback
trainer.py: torch._dynamo.config.recompile_limit does not exist in
some ROCm torch builds (e.g. pytorch.org/whl/rocm6.2 wheels). Guard
the assignment so training doesn't crash on RDNA2/RDNA3.
hardware.py: when amd-smi/nvidia-smi is unavailable or returns no
usable data (HIP SDK-only Windows, Docker, unexpected JSON format),
the existing fallback used torch.cuda.memory_allocated() which is
process-specific and reads near-zero even with a fully loaded model.
Switch to torch.cuda.mem_get_info() via _torch_get_per_device_info()
which reports system-wide VRAM occupancy so the GPU monitor shows
real usage on all AMD systems without requiring amd-smi.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: Windows VRAM monitor via Performance Counter API
When amd-smi/nvidia-smi is unavailable on Windows, query dedicated GPU
VRAM via Windows Performance Counters (same source as Task Manager).
This gives system-wide cross-process usage, fixing the near-zero reading
caused by torch.cuda.mem_get_info only seeing the Studio server process.
Linux fallback path unchanged (mem_get_info is system-wide on ROCm).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: rename to _rocm_windows_perf_counter_vram_gb, scope to IS_ROCM
Function is AMD ROCm specific — amd-smi absent on Windows when only the
HIP SDK is installed. Scoped to IS_ROCM so NVIDIA Windows path is
untouched (nvidia-smi handles that case).
* fix: AMD VRAM monitor — Linux DRM sysfs + Windows perf counter
Linux: read /sys/class/drm/card*/device/mem_info_vram_used|total for
system-wide GPU memory across all processes. No tools required, always
present on Linux AMD systems.
Windows: Windows Performance Counter API (already added).
Both paths are gated on IS_ROCM and only fire when amd-smi is absent.
torch mem_get_info remains as last resort (process-local).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: AMD GPU monitor — utilization, temperature, and power for Windows and Linux fallback paths
- Windows: GPU utilization via \GPU Engine(*engtype_3D*)\Utilization Percentage perf counter
- Windows: temperature and power via ADL (atiadlxx.dll, ships with Adrenalin)
- Linux: GPU utilization via DRM sysfs gpu_busy_percent
- Linux: temperature via hwmon temp1_input (millidegrees C)
- Linux: power via hwmon power1_average / power1_input (microwatts)
All paths are no-op fallbacks (None) when the source is unavailable.
Mirrors what nvidia-smi provides on the CUDA path.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: remove ADL ctypes — does not support AMD iGPU (Strix Halo)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: Erland366 <erland.pg366@gmail.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
* fix(studio/colab): merge iframe+keepalive into start(), add proxy_headers to uvicorn
- Move serve_kernel_port_as_iframe and keepalive loop into colab.start()
so both run in the same cell execution context, eliminating the race
where the proxy URL was shown before the iframe cell had a chance to run
- Add a 2s sleep after run_server() before show_link() to give Colab's
proxy infrastructure time to register the bound port
- Add proxy_headers=True and forwarded_allow_ips="*" to uvicorn Config
so X-Forwarded-Proto/Host from Colab's reverse proxy are trusted
- Simplify notebook start cell (no more separate iframe cell needed)
* fix(studio/colab): fix iframe blocking and server thread crash in Colab
Two root causes for the long-standing proxy/iframe breakage:
1. SecurityHeadersMiddleware set X-Frame-Options: DENY and
frame-ancestors 'none' unconditionally, blocking
serve_kernel_port_as_iframe regardless of server health.
Fix: detect Colab via COLAB_BACKEND_URL/COLAB_GPU env vars,
relax frame-ancestors to *.prod.colab.dev and omit X-Frame-Options.
2. asyncio.run() in the daemon thread conflicted with nest_asyncio's
global patches applied on the main thread, causing the server to
crash silently after ready_event fired.
Fix: use explicit new_event_loop() + run_until_complete() in the
daemon thread to bypass nest_asyncio's asyncio.run patch.
Also replace blind time.sleep(2) with a health endpoint poll so the
link and iframe are only shown once the server is truly reachable.
* fix(studio/colab): use reliable /content + google.colab path for Colab detection
COLAB_BACKEND_URL and COLAB_GPU env vars aren't consistently set across
all Colab runtime versions. Use /content dir + google.colab package path
as a more reliable signal, computed once at module load.
* fix(studio/colab): fix port mismatch, health-check silence, and CSP framing
Four bugs causing the iframe and URL button to always fail:
1. Port not propagated back: run_server auto-increments when 8888 is taken,
but start() kept using the original port for show_link() and
serve_kernel_port_as_iframe() — now reads app.state.server_port.
2. Silent health-check failure: the poll loop never checked whether any
attempt succeeded; on all-fail it continued and showed a dead link —
now exits early with a clear error message.
3. CSP frame-ancestors too narrow: '*.prod.colab.dev' only matches one
subdomain level; actual Colab proxy URLs are two levels deep
(e.g. foo.region.prod.colab.dev), and the parent frame may also be
colab.research.google.com or a sandboxed null-origin output iframe —
changed to '*' in Colab mode (single-user sandbox, no security loss).
4. _IS_COLAB detection hardcoded python3.10/3.11 paths: Python 3.12+
Colab runtimes wouldn't match when env vars aren't set — replaced with
a glob over python3.*/dist-packages/google/colab.
* fix(studio/colab): harden Colab startup against every known failure mode
colab.py:
- get_colab_url: retry eval_js up to 3x (10s timeout each), validate that
result is a real https:// URL containing the port before accepting it;
log a clear warning when falling back to localhost
- show_link: safe short_url truncation (try/except around str.index so an
unexpected URL shape never blocks the link card from rendering); also
emit the URL via logger so it's visible in cell text output even if
HTML display is suppressed
- start: detect "already running" at entry — on cell re-run Studio is
still healthy on port 8888; skip re-launch and go straight to
show+iframe so the user never ends up with mismatched port state
- start: wrap run_server in try/except (SystemExit + Exception) so
startup errors surface as readable messages rather than cell crashes
- start: check frontend_path/index.html exists, not just the directory
- start: remove unused `import sys`
- start / keepalive: catch KeyboardInterrupt so interrupting the cell
prints a clean "stopped" message instead of a raw traceback
- extract _is_studio_healthy() and _show_and_embed() helpers to
deduplicate the fast-path and normal-path logic
main.py:
- _build_csp: in Colab mode, extend script-src to include
*.prod.colab.dev and *.googleusercontent.com (Colab injects scripts
from these origins into the output iframe scaffolding)
- _build_csp: in Colab mode, extend connect-src with blob:, data:,
wss://*.prod.colab.dev, and wss://*.googleusercontent.com so
WebSocket streams and Colab kernel traffic are not blocked by CSP
* fix(studio/colab): fix iframe width responsiveness and height sizing
Replace serve_kernel_port_as_iframe with a raw CSS iframe for two
reasons:
1. Width responsiveness: serve_kernel_port_as_iframe sets the width as
an HTML attribute (width="100%") which Colab's output machinery can
bake into a fixed pixel value on first render, causing the Studio to
stop following the notebook panel width when it opens/closes or the
window resizes. A CSS style property (style="width:100%") participates
in normal reflow and always tracks the parent container width.
2. Height sizing: the hardcoded height=1200 was too tall on short monitors
(forced outer-page scroll) and wasted space on tall ones. A small JS
snippet reads screen.availHeight and sets height to ~82% of the screen,
clamped to [600, 1100]px, with a resize listener that re-fits on zoom
changes and panel open/close events.
Also eliminate the double eval_js call: _show_and_embed now fetches the
Colab proxy URL once and passes it to show_link via the new _url kwarg,
so google.colab.kernel.proxyPort is only called once per invocation.
Falls back to serve_kernel_port_as_iframe if IPython.display.HTML is
unavailable for any reason.
* fix(studio/colab): fix link button + add fullscreen hover button to iframe
Link button: target="_blank" is blocked by Colab's output sandbox.
Switch to onclick="window.open(url,'_blank')" which the sandbox allows.
Fullscreen: add a small button that appears on hover in the top-right
corner of the iframe. Clicking it calls requestFullscreen() on the
wrapper div and stretches the iframe to 100vh/100vw. Exits back to
normal on fullscreen change.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* revert(studio/colab): remove fullscreen button
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio/colab): address review feedback
- Wrap both urlopen calls in with statements to prevent socket/fd leaks
- Replace JS resize listener with CSS height:82vh — simpler, responsive,
and no risk of leaked window listeners on cell re-runs
- Use importlib.util.find_spec("google.colab") instead of a glob path
to detect Colab; more robust across Python versions and venv layouts
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio/colab): fall back to href navigation when window.open is blocked
window.open from a cross-origin sandboxed Colab output iframe can be
silently blocked by the browser (returns null, no exception). The old
code returned false unconditionally, so a blocked popup left the button
doing nothing. Now: if window.open succeeds the new tab opens and the
href is suppressed; if it returns null the browser follows the href,
navigating the output cell to Studio — always does something useful.
* fix(studio/colab): remove button, give iframe a branded header bar
The "Open Unsloth Studio" button was unreliable in Colab's sandboxed
output context regardless of how window.open was called. Since the
iframe already loads Studio inline, the button added no value and
confused users with a URL that 404s outside the output cell.
Replace the separate link card + bare iframe with a single block:
a slim black header bar (Unsloth logo + truncated URL) flush on top
of the full-height responsive iframe. Cleaner and removes the broken
button entirely.
* studio: gate uvicorn proxy_headers/forwarded_allow_ips behind _IS_COLAB
forwarded_allow_ips="*" was applied unconditionally, so every Studio
deployment trusted X-Forwarded-* headers from any client. Only Colab needs
that, because its reverse proxy fronts the kernel. For a normal
local/standalone Studio this is an unwanted relaxation, especially when bound
to 0.0.0.0.
Now proxy_headers/forwarded_allow_ips are only set when _IS_COLAB. Standalone
runs fall back to uvicorn's defaults (proxy_headers honored from loopback
only), restoring the prior security posture, while Colab keeps the wide trust
its proxy requires.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* added remote MCP server support
* trim
* added tests
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* increased timeout
* disabling MCP chat toggle
* Fix MCP OpenAI function-name validation + cancel propagation for PR #5750
OpenAI requires function.name to match ^[a-zA-Z0-9_-]{1,64}$ before
streaming starts. The existing 64-char length check is necessary but
not sufficient: MCP servers can return tool names containing '.', '/',
spaces, etc. that would 400 the whole chat request. Validate the
composed mcp__<server_id>__<tool> name against the regex, skip + warn
on miss, and drop duplicate tool names from the same server (which
would also 400 the request as "duplicates").
Also propagate the agentic-loop cancel_event into MCP tool execution
so a /cancel POST during a long-running MCP call (e.g. GitHub MCP
search across a large repo) actually interrupts the in-flight HTTP
call instead of waiting out the 300 s timeout. The watcher polls the
threading.Event at 50 ms cadence inside the asyncio loop (matches
routes/inference.py's existing cancel-watcher cadence) and races
against the call task with asyncio.wait FIRST_COMPLETED.
Tests added:
- test_mcp_specs_skip_invalid_openai_function_names: drops bad chars
- test_mcp_specs_skip_empty_tool_name
- test_mcp_specs_drops_duplicate_names
- test_call_tool_sync_respects_pre_set_cancel_event
Also fix test_desktop_auth.py's router stub that listed every existing
router but missed mcp_servers_router, so importing main.py fails after
this PR adds it to routes/__init__.py.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* PR #5750 round 2: OAuth cleanup on delete/url-change + mcp_enabled standalone
Round 2 of cross-platform validation surfaced two more P1 findings:
1. OAuth tokens never get cleared. fastmcp keys tokens by MCP URL, not by
server row, and delete / URL change / use_oauth toggle only updated
the SQLite row. Re-registering the same URL would silently reuse the
old account's credentials. Adds clear_oauth_tokens_async() in
mcp_client.py and calls it from the delete + put route handlers when
the row had use_oauth=True and either the URL changes or OAuth is
turned off.
2. mcp_enabled=true was ignored unless the caller also sent
enable_tools=true. The frontend always sends both together so the UI
path was fine, but a direct API caller sending only mcp_enabled would
silently get no MCP tools, which contradicts the field's documented
"append tools from every enabled MCP server" behavior. Loosens the
use_tools gate in both the GGUF and safetensors paths so mcp_enabled
opens the tool loop on its own; when the caller did not also opt
into built-ins, the built-in list starts empty.
Tests added:
- test_clear_oauth_tokens_async_no_op_safe
- test_delete_server_calls_oauth_cleanup_when_oauth_was_on
- test_delete_server_skips_oauth_cleanup_when_oauth_off
- test_update_server_clears_oauth_on_url_change
- test_update_server_clears_oauth_when_oauth_disabled
26 backend MCP tests pass; full studio/backend suite 1710 passed locally.
Cross-platform CI (Linux, macOS, Windows) green on staging fork.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* PR #5750 round 3: reject null bool updates + /test surfaces 400
Round 3 of cross-platform validation:
1. PUT /api/mcp/servers/<id> would 500 with TypeError when the body
explicitly set is_enabled or use_oauth to null. Pydantic accepts
None for an Optional[bool] and _changes_from_payload then passed
None into mcp_servers_db.update_server, which int(None)d. Reject
explicit null at the validation layer with 400 instead.
2. POST /api/mcp/servers/test caught HTTPException under
"except Exception", so an invalid URL came back as HTTP 200 with
{"ok": false, "error": "400: ..."} instead of a real 400. The
create + update paths return 400 for the same input. Move
validation outside the transport try/except so it surfaces 400.
Tests added:
- test_changes_from_payload_rejects_null_is_enabled
- test_changes_from_payload_rejects_null_use_oauth
- test_test_endpoint_surfaces_url_validation_as_400
* PR #5750 round 4: hyphenated MCP tool names + empty-tool-list gate
Round 4 surfaces two more interaction bugs between the new MCP path
and existing safetensors tool plumbing:
1. OpenAI accepts ^[a-zA-Z0-9_-]{1,64}$ for function.name, and round 1
widened the MCP regex to that set, so MCP tools can now be advertised
as `mcp__srv__list-issues`. But the XML tool-call parser in
tool_call_parser.py used `\w+` (no hyphen), so the model could call
the tool but Studio could not parse the call. Same in
routes/inference.py's `_TOOL_XML_RE` stripper, which would leave
hyphenated tool-call XML in the visible content. Both regexes now
use `[\w-]+`.
2. safetensors_agentic treats `tools=[]` as "allow all" (documented
contract, exercised by test_empty_tools_list_does_not_enforce_allowlist).
When a caller sends `enable_tools=true` + `enabled_tools=[]` +
`mcp_enabled=true` and MCP discovery returns 0, the resolved tool
list is genuinely empty and built-in tools (web_search / python /
terminal) could execute via the model's emitted call. Fix at the
route gate instead of breaking the documented contract: set
`use_tools=False` when the resolved list is empty, in both GGUF and
safetensors paths. Existing callers who omit `enabled_tools` still
get ALL_TOOLS and are unaffected.
Tests added (32 total):
- test_tool_xml_parser_handles_hyphenated_function_names
- test_tool_xml_strip_handles_hyphenated_function_names
- test_safetensors_agentic_empty_allowlist_still_means_allow_all
(documents the contract round 4 preserved)
1716 passed locally; cross-platform CI on staging fork still green.
* PR #5750 round 5: GGUF allow-list + CLI policy + hyphenated params + cancel race
Round 5 of parallel-reviewer aggregation surfaced six additional
findings; five are real and fixed here:
1. Hyphenated MCP parameter names (`<parameter=issue-number>`) were
dropped by the XML parser's `\w+` regex. Extended to `[\w-]+` in
both core/inference/tool_call_parser.py and core/tool_healing.py.
The latter is GGUF's own copy of the parser/strip patterns and was
missed by round 4.
2. core/tool_healing.py's `strip_tool_call_markup` still used
`<function=\w+>` so hyphenated MCP tool-call XML leaked into the
GGUF visible content even after round 4 fixed the shared parser.
3+4. `mcp_enabled` re-opened the tool loop even when the operator
passed `unsloth run --disable-tools` (CLI policy False). Round 2's
`(_tools_on or payload.mcp_enabled)` gate ignored the raw process
policy. Now reads `state.tool_policy.get_tool_policy()` and gates
mcp_enabled on `_cli_policy is not False`. Applied to both GGUF
and safetensors paths.
5. GGUF's agentic loop called `execute_tool(tool_name, ...)` without
checking the model-emitted name against the per-request tool list,
while the safetensors loop already enforces this. Added the same
allow-list check so a model that hallucinates a filtered MCP name
or a built-in the caller opted out of returns "not enabled" instead
of executing.
Bonus P2 fixes:
- `call_tool_sync` now checks `cancel_event.is_set()` BEFORE
creating the call task, so a pre-set cancellation does not open
the HTTP transport.
- `clear_oauth_tokens_async` moved the OAuth import + construction
inside the protected try block; a fastmcp.client.auth load error
used to escape and 500 the delete / update route.
NOT fixed (verified false or out of scope):
- finding #10 "structured_content vs structuredContent": fastmcp's
CallToolResult dataclass uses snake_case (verified live against
structured-only tool result; fields are
`dict_keys(['content', 'structured_content', 'meta', 'data', 'is_error'])`).
- finding #11 "asyncio.run from running loop": call_tool_sync is
invoked from `asyncio.to_thread` worker threads which have no
event loop; asyncio.run() is safe there.
Tests added (37 total): hyphenated param names, tool_healing strip,
GGUF allow-list gate, cancel pre-set short-circuit, OAuth cleanup
constructor-error swallowing. 1721 passed locally, no regressions.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
* Studio: stop leaking seeded admin pw to cross-origin callers
The "/" SPA fallback serves index.html with an inline
``window.__UNSLOTH_BOOTSTRAP__`` script containing the seeded admin
password while a password change is pending. Default web mode runs
``CORSMiddleware`` with ``allow_origins=["*"]`` + ``allow_credentials=
True``, which reflects an attacker-controlled ``Origin`` back on every
request and sets Access-Control-Allow-Credentials true. The combination
let any cross-origin page ``fetch('/')`` with credentials and read the
bootstrap admin password out of the HTML body. The API smoke
``CORS: GET / leaks bootstrap pw to cross-origin caller`` audit already
tracked this (tests/studio/studio_api_smoke.py:224) but did not gate CI.
Gate ``_inject_bootstrap`` on a same-origin check: legitimate top-level
navigations omit ``Origin`` on most engines, so the absence of the
header is treated as same-origin; when the header IS present and does
not match ``request.url.scheme://request.url.netloc`` exactly, we now
skip injecting the bootstrap tag. ``Vary: Origin`` is added so an
intermediary cache cannot serve a same-origin response (with bootstrap)
to a later cross-origin caller (and vice versa).
Coverage: ``test_index_bootstrap_origin.py`` exercises the helper with
missing / matching / evil / scheme-mismatch / port-mismatch origins.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: tighten comments on bootstrap cross-origin helper
* Studio: canonicalise Origin before same-origin gate
A plain string-compare between the Origin header and request.url.netloc
misclassifies legitimate same-origin requests as cross-origin in three
scenarios:
- Browser strips the default port from Origin (https://example.com)
but Starlette's netloc keeps it (example.com:443). Per RFC 6454 the
default port is dropped on the wire, so the strings will not match
even though the requests share an origin.
- Host case differs (Origin: http://Example.com vs netloc:
example.com). Per RFC 3986 host comparison is case-insensitive.
- Scheme case differs (HTTP:// vs http://). Per RFC 3986 the scheme
is also case-insensitive.
These are usability degradations rather than security gaps (legitimate
user denied the bootstrap injection, no attacker gain), but worth
shipping so non-default Studio deployments keep the change-password
auto-fill.
Adds _canonical_origin(scheme, netloc) -> (scheme, host, port) and
compares the canonical tuples. Default-port lookup covers
http/https/ws/wss; userinfo (user:pass@) is stripped per RFC 3986
since Origin never carries credentials. Origin: "null" (sandboxed
iframes, file:// pages) and unparseable values collapse to cross-
origin so the bootstrap pw is never leaked through those paths either.
Tests: 14 cases (was 5). Covers the original same/missing/evil/
scheme/port matrix plus default-port stripping in both directions,
host + scheme case folding, Origin: null, garbage values, and
userinfo-in-netloc.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix IPv6 netloc parsing for PR #5739
The canonical-origin helper used ``netloc.partition(":")`` which
mis-parses bracketed IPv6 hosts (``[::1]:8902`` -> host=``[``,
port-str=``:1]:8902``). The int() then raises and the canonicaliser
returns None, so every IPv6 same-origin request is misclassified as
cross-origin and Studio refuses to inject the bootstrap pw on a
legitimate top-level nav when launched with ``unsloth studio -H ::1``.
Bracket-aware split per RFC 3986 §3.2.2, plus extra regression tests
for IPv6, opaque (data:/blob:/file:), comma-joined multi-Origin and
localhost-vs-127 cases.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Guard urlparse ValueError in same-origin gate
urlparse raises ValueError on malformed bracketed Origin values
(unclosed [, invalid IPv6 hex, text after ]) and on a few NFKC
edge cases since Py 3.8. Without a guard, a request carrying
Origin: http://[malformed surfaced as HTTP 500 from the SPA
handler rather than being treated as cross-origin per the
docstring's safer-default rule. Wrap both urlparse calls in
try/except ValueError and return False on parse failure.
Also distinguish a missing Origin header (top-level same-document
GET, treat as same-origin) from an explicit empty string (not a
valid serialised origin per RFC 6454 §6.1, treat as cross-origin).
Four new regression tests pinned down by the PR audit: malformed
IPv6 bracket, invalid IPv6 hex, bracket with trailing garbage,
and the empty Origin header.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Shorten origin-gate comments for PR #5739
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Eyera