* Studio: persistent per-user trust_remote_code approval cache
The consent gate pins each approval to a content fingerprint (sha256 over every
repo .py), but nothing was persisted, so the dialog reappeared on every fresh
load of the same unchanged repo. This adds an on-disk, per-user approval cache
that lets the gate skip the dialog when the same user reloads the same code,
while keeping the safety guarantees intact.
Two-tier validation, both must hold or the user is re-prompted:
- Commit SHA (cheap, one HfApi.model_info().sha, no download): a match means a
byte-identical tree to the approved revision, so the scan/download is skipped.
- Content fingerprint (authoritative): used whenever the SHA is unavailable
(local path / offline) and always recomputed on a SHA miss. A new or edited
.py changes both the SHA and the fingerprint, so it is caught in every mode.
Safety:
- Keyed per subject; one user's approval never auto-runs code for another.
- CRITICAL is never stored or honored (guarded on both write and read), so a
hand-edited store cannot smuggle in an auto-approval.
- The malware (HF unsafe-file) gate stays unconditional.
- Fail-safe: a corrupt store, an unresolvable SHA, or any error degrades to
"ask again", never to "auto-approve". UNSLOTH_TRC_APPROVAL_CACHE_DISABLE=1
turns the cache off entirely.
New module utils/security/remote_code_approvals.py holds the store
(studio_root()/security/remote_code_approvals.json, atomic write, 0600, RLock)
plus the SHA resolvers. Recording happens at the single gate chokepoint when the
caller supplies the matching fingerprint, so subject is just threaded through
inference/training/export (orchestrators, routes, workers). The scan endpoint
returns already_approved so the frontend can skip the dialog on a cache hit.
Tests: new tests/test_trc_approval_cache.py covers cache miss, SHA-match skip,
SHA-moved re-scan, new-file re-consent, CRITICAL never cached (write + forged
read), disable flag, subject isolation, combined adapter+base key, corrupt
store, and no-subject bypass. Full security suite: 101 passed.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: make the approval cache skip only the prompt, never the scan
Codex found that the SHA "no-scan" fast path could run untrusted code without
re-consent. Removed it; the gate now always re-scans and the cache only seeds the
authoritative fingerprint check, so it can skip the dialog but never the scan.
- CRITICAL is hard-blocked on every load (the scan always runs), so a hand-edited
store that downgrades a CRITICAL repo's severity can no longer auto-run it
(P2: do not trust editable severity for SHA approvals).
- The fingerprint covers external auto_map repos, so changed third-party code
always re-prompts even when the primary commit SHA is unchanged; there is no
longer a SHA path that bypasses the fingerprint (P1: external auto_map repos).
- resolve_commit_sha is resolved fresh on every call (no memoization), so a repo
whose default branch moves after approval re-prompts instead of reusing a stale
cached SHA (P1: revalidate mutable Hub SHAs). The SHA is now only a conservative
secondary gate: a fresh resolvable SHA must match the approved revision, else the
seed is withheld; a None (local/offline) falls back to the fingerprint.
- Approvals record the scanner ruleset version (SCAN_RULES_VERSION); the gate
ignores approvals from an older ruleset so reclassified bytes are re-scanned and
re-shown instead of silently auto-approved (P2: invalidate on scan-policy change).
Tests: test_trc_approval_cache.py rewritten around the prompt-skip semantics
(unchanged repo still scans; SHA move / changed code / scanner-version bump /
disable flag all re-prompt; forged downgraded severity still blocks CRITICAL).
105 passed with test_consent_gate.py.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Trim comments to be more succinct
* Keep run-owner subject out of persisted config; serialize approval writes
Threading subject (the run owner's username / API-key id) into the training
config meant _sanitize_db_config persisted it into config_json, which
training-history GET returns to any authenticated user, leaking who started a run
in multi-user installs. Filter subject alongside the token fields; the worker
still receives it from the live config.
The approval store's RLock only guards one process, but approvals are recorded
from separate inference/export/training subprocesses, so concurrent writers could
clobber each other on os.replace and drop an approval (re-prompt). Hold a
best-effort cross-process file lock around the read-modify-write.
* Fail safe on a malformed approval store
A store with the right version but a non-dict shape (e.g. a hand-edited
"subjects": []) passed _load()'s check, then lookup chained .get() on a list and
raised, breaking every remote-code load until the file was removed. Validate that
subjects is a dict in _load(), and tolerate a non-dict per-subject entry in
lookup/record/forget, so a corrupt store fails safe (re-prompt) instead.
* Keep subject out of the MLX W&B run config
_run_mlx_training uploads the whole training config to W&B minus a sensitive set
that only listed hf_token/wandb_token/s3_config, so the authenticated subject
(username / API-key id) was sent to W&B as run config even though DB history
already strips it. Add subject to the W&B-sensitive filter, mirroring
training._sanitize_db_config.
* Tighten the W&B subject-filter comment
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: add Export to GGUF button on finished training runs
A completed run's 'Current Run' tab greys out, so it was unclear how to
export it: GGUF export lives on the separate Export page and there was no
link to it from a run. Add an 'Export to GGUF' button to the run progress
card (shown for completed/stopped runs) that deep-links to the Export page
with that run preselected via a new ?run= search param. The Export page
reads the param, selects the run, defaults to GGUF, and picks the run's
main checkpoint. No retraining is required to export a finished run.
* Studio: fix export deep-link checkpoint preselect and edge cases
Address review feedback on the Export to GGUF deep link:
- Move the main-checkpoint auto-select effect after the model-change reset
effect so it runs last; previously the reset cleared the checkpoint back to
null in the same commit, leaving the field empty on a deep link.
- Reset the applied-run ref when the ?run= param clears (e.g. navigating to
/export via the sidebar) so a later manual reselect of the same run is not
treated as a deep link.
- Trim trailing slashes before taking the run output-dir basename so a path
like /outputs/run/ still yields a name (the button no longer disappears).
* Studio: hide Export to GGUF on runs superseded by a resume
A stopped run whose output_dir was later reused by a resumed run is marked
resumed_later by the backend; its on-disk contents no longer match the older
run's metrics. Since the export deep link selects by output-dir basename,
showing the button on such a run would export the newer continuation instead
of the run being viewed. Carry resumed_later into the view data and hide the
button when set.
---------
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
The chat and sidebar scroll-fade overlays ended their gradient at the
`transparent` keyword, which is transparent black. Safari 27 Beta
(Liquid Glass) interpolates an opaque colour to transparent black
through a grey midtone, so the fades render as two solid grey bands
(top of the chat and above the composer).
Fade each gradient to the theme colour at zero alpha instead, so every
step keeps the same hue and no grey can appear. Fixes#6457.
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
* Studio: redesign Select model dropdown to match Hub design
Make the chat Select model picker easier to scan by reusing the Hub
on-device card's visual language.
- Rows now split owner/name, add a param chip, a DotTag format pill,
a tabular size, and a Loaded marker on the active model.
- Hub models / Fine-tuned tabs reuse the Hub's exact .hub-tab-toggle
styling (selectors extended in hub.css to the selector menu).
- Add a Downloaded / Recommended / Custom section toggle on the Hub
tab to filter the list.
- Widen the popover and nudge the scrollbar toward the edge.
* Studio: move section toggle below search, size tabs to label
Put Downloaded / Recommended / Custom under the search bar in their own
row so Hub models / Fine-tuned no longer wrap. The section toggle uses a
smaller font and sizes each tab to its label instead of equal widths.
* Studio: extract pure row-meta helpers into their own module
Move splitRepoLabel, classifyMetaToken, and parseMetaTokens out of
pickers.tsx into row-meta.ts. No behaviour change; keeps the presentation
logic free of React/DOM deps so it is easy to test in isolation.
* Studio: content-size the source tabs and add section icons
Size the Hub models / Fine-tuned tabs to their labels (with side
padding) like the section toggle, instead of stretching full width. Add
a leading download, star, and folder icon to Downloaded, Recommended,
and Custom.
* Studio: stop source tabs stretching and hide empty Fine-tuned tab
The popover is a flex column, so the fit toggle stretched full width;
add w-fit/self-start so it sizes to its content. Also hide the
Fine-tuned tab when there are no fine-tuned models, defaulting to Hub
models.
* Studio: keep only fine-tuned models in the Fine-tuned tab
Local models (LM Studio, Ollama, custom folders) carry source "local"
and already show in the Hub tab's Downloaded / Custom sections, so
exclude them from the Fine-tuned tab and from its visibility count.
Extract the tab rules into source-tabs.ts.
* Studio: show local providers under Downloaded, Recommended first
Show LM Studio and other local provider models in the Downloaded
section in all modes (was chat-only). Put Recommended first and make it
the default section. Add a little more space below the search bar.
* Studio: make Recommended a sortable live Unsloth listing
Replace the static Recommended list (and its collapse chevron) with a
sort dropdown over Unsloth's own models: Recommended, Trending, Most
likes, Downloads, Recently updated. Recommended shows recently uploaded
GGUF/MLX models that fit the device (hidden if they do not); the other
sorts list all Unsloth models, badged but never hidden. Adds a sort
option to useHfModelSearch and a pure recommended-fit helper.
* Studio: size Recommended models from the repo name when metadata is missing
GGUF and MLX repos rarely expose safetensors metadata, so a large model
with no size could pass the Recommended fit check because unknown size was
treated as fitting. Parse the parameter count from the repo id, including
the Gemma E series, and hide anything we still cannot size.
* Studio: detect model capabilities and family from HF tags
Thread tags and the pipeline tag through the model search results and add a
pure helper that infers vision, reasoning and audio plus the architecture
family, falling back to repo-name keywords when tags are absent.
* Studio: add row details and inline section sorting to Select model
Give each model row more detail and make the Hub sections easier to scan:
- Show vision, reasoning and audio badges plus the architecture family tag
on each row, alongside the params, format and size.
- Drop the redundant unsloth/ prefix on the Recommended rows.
- Rename the Recommended section tab to Unsloth and enlarge the section tabs.
- Move the sort dropdown inline to the right of the tabs at a fixed width.
- Add Recent, Size and Downloaded sorting to the Downloaded and Custom tabs.
- Remove the header icons, pad the subheadings, and grow the list height.
* Studio: tune the Select model sort dropdown and trim row badges
- Recommended now lists the most recently created Unsloth repos.
- Narrow the sort dropdown, remove its border, and truncate long labels.
- Tighten the gap between the section tab icons and their labels.
- Remove the architecture family tag from rows since it repeats the name.
* Studio: extract the PillTabs toggle into a shared module
Move the segmented pill toggle out of the model selector into its own file so
the Hub picker can reuse it for a format filter without duplicating the markup.
* Studio: fix Recommended infinite scroll and add a format filter
- Re-attach the scroll observer on each loaded page so a filtered Recommended
list keeps paging until the viewport fills instead of spinning forever with
nothing new appearing.
- Add an All / GGUF / MLX / Safetensors toggle on the Unsloth listing that
filters every sort.
* Studio: default Recommended to Trending, rename Downloaded to On Device, and fade the scroll edge
Sort: default the Recommended view to Trending and add a Name option to
the On Device / Custom sort. Recent now orders by last load time while
Downloaded orders by file date, tracked in localStorage (model-usage.ts).
Formats: show the format filter on all three tabs (Unsloth, On Device,
Custom), exclude mobile GGUF builds from Recommended, and flag GGUF rows
that exceed the device with the same OOM badge as safetensors.
Polish: download-icon badge on already-downloaded Recommended rows, the
hugeicons view stroke-rounded vision badge, Search all models placeholder,
matched popover padding, and a top-edge mask fade once the list scrolls.
* Studio: size GGUF repos from gguf metadata so large ones flag OOM
Repos with no <n>B token in the name (Kimi, MiniMax) had no param count
and so never showed an OOM badge. Request the gguf expand field from
Hugging Face and read gguf.total, so those repos get a param chip and an
OOM badge when they exceed the device budget.
Keep the row name full contrast when over budget (the OOM badge already
signals the fit), shorten the format and sort dropdowns, narrow the
popover, and rename Recently updated to Recent and All formats to All.
* Studio: address selector review feedback
Add WAI-ARIA roving tabindex and Arrow Left/Right navigation to the pill
toggle so only the active tab is in the tab order. Keep the chat-only
GGUF/MLX filter for every Recommended sort, not just Recommended, so
chat-only users do not see unrunnable checkpoints under Trending. Feed
both listings' GGUF hints into repo detection so a tag-only GGUF in
Recommended expands variants instead of loading as a checkpoint.
* Studio: scope Select model search per tab and add an MLX tag
Search is now per section. The Unsloth tab searches the Unsloth HF
listing only, On Device filters downloaded and LM Studio models by name,
and Custom filters custom-folder models, each with its own empty state.
MLX repos get an MLX pill mirroring the GGUF tag. Downloaded quants in
the Unsloth and search lists get the same delete action as On Device.
Also: revert the model name to normal weight, narrow the popover to
558px so the format and sort dropdowns sit one gap-2 from the tabs,
tighten the dropdown menus to match the Projects activity Select, and
make the empty On Device state name the active format filter.
* Studio: show local ./models on the On Device tab so they stay selectable
Models under the local models directory (source models_dir) flow in as local
models but were dropped from every list: filtered out of Fine-tuned and never
re-added by the Hub picker, which kept only LM Studio and custom-folder
sources. Capture them in the local refresh and render a Local models group on
the On Device tab, with the same format, search, and chat-only GGUF rules as
the other local groups.
* Studio: add a Hub button beside the Select model search bar
Adds a Hub button next to the search bar that opens the full Hub Discover
page to browse more models. Styled like the section tabs (rounded, no
border, soft shadow with a faint top layer) and darkens on hover. Also
nudges the format and sort dropdown chevrons a touch toward the edge.
* Studio: align Select model padding and tighten the format pills
Sizes the popover to the tab cluster so the left and right padding match,
and drops the top row below the rounded corner so the Hub button lines up
with the Trending dropdown. Gives the Hub button a fixed width, lets the
list scrollbar sit inside the box, and shrinks the format pill dot with a
tighter dot-to-label gap.
* Studio: label the Hub button Search Hub and match the dropdown width
Renames the button to Search Hub, sets its width to the format and sort
dropdown width so it lines up above them, and tightens the icon gap.
* Studio: drop the vision and reasoning row badges to declutter
Removes the vision and reasoning capability icons from the model rows so
they read cleaner. Audio is kept.
* Studio: add a safetensors pill, hide diffusion models, eye on Vision
Gives safetensors rows a format pill and size so their meta matches GGUF
and MLX, drops image and video diffusion models from the listing since they
cannot run in chat, and shows an eye icon next to the Vision tag. Also
removes the em dashes from the Projects export and import labels.
* Studio: gate recommended folders on real weights and polish the selector
Only show a Recommended chip once the well-known dir actually holds
weights, so an empty LM Studio or Ollama scaffold no longer suggests
itself. _dir_has_downloaded_model checks for a GGUF/safetensors file or
a non-empty Ollama manifests store, with a bounded walk.
Selector polish: round the popover and option menus a touch more,
lighten the OOM badge in dark mode, soften the inner dropdown shadow,
even out the padding, and lift the toggle track and field triggers so
their edges read against the popover.
Also catch CogVideoX in the diffusion name fallback.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: align the dark Select model panel with the sidebar
Match the popover, fields, dropdowns, tab toggle and row states to the
sidebar surface and accent so the dropdown reads as one piece in dark
mode. The active tab pill and Search Hub button sit a touch lighter
than the track, and the inner option menus drop their drop shadow for a
flatter look. Light mode is unchanged.
* Studio: re-derive the Select model tab on open
The picker remounts each time the dropdown opens, but the source tab
state did not, so a persisted fine-tuned or connected selection that
only lands in its list after an async load would reopen on Hub. Reset
the active tab to the selection-derived default on the open edge, while
still letting the user switch tabs freely within a session.
* Studio: fold Custom into On Device and polish the picker
Merge the Custom tab into On Device so custom folders sit right below
the downloaded models, with a folder shortcut on the group header.
Rename the first Hub tab to Recommended, give the format dropdown
colored dots, even out the tab row spacing, and tighten the popover
width. Align the folder browser with the app dialogs (soft surface,
roomier padding, green confirm, grey hover).
* Studio: fix On Device controls and nudge the folder browser close
The Hub redesign merge dropped the old Search Hub button styling, so the
On Device search row rendered flat. Point the search input and Search
Hub button at the shared .field-soft surface so they match the rest of
the Hub controls, and lift the folder browser close button slightly.
* Studio: run the Select model search on the Hub search stack
Point the picker at the Hub's useHubModelSearch and useHubInfiniteScroll
instead of its own useHfModelSearch/useInfiniteScroll, scoped to unsloth
so the listing matches the old one. Both the search and the recommended
feed now share the Hub implementation, so there is one search path. The
Hub result folds GGUF params into totalParams, so the dead ggufParams
fallback is dropped.
* Studio: trim the recommended sort to Recommended, Trending, Recent
Drop Downloads and Most likes from the sort dropdown.
* Studio: give the section tabs room off the rounded edge
The fit-mode toggle wrapped the tabs with no inset, so On Device sat
tight against the rounded-full edge. Add a small horizontal inset and
widen the popover a touch to fit it.
* Studio: drop the legacy HF search hooks for the Hub ones
Migrate the training model and dataset sections, export page, onboarding
steps and recipe dataset combobox off useHfModelSearch, useHfDatasetSearch
and useInfiniteScroll onto the Hub equivalents, scoped to unsloth so the
listings match. The picker reads recommended param counts off the search
results it already has instead of a separate fetch. Removes the duplicate
search stack: use-hf-model-search, use-hf-dataset-search,
use-hf-paginated-search, use-infinite-scroll, use-recommended-model-vram
and the old lib/hf-cache.
* Fix model selector section toggle proportions
Remove the fit-mode track inset so the active pill sits flush to the
track edge, matching the Hub's segmented controls.
* Tighten model selector width and tab padding
Reduce the popover width so the right edge aligns with the row, and
widen the fit-mode tab padding so On Device clears the track edge.
* Refine Recommended formats, sort width and tab padding
Recommended now suggests GGUF anywhere and MLX only on Mac, never
safetensors. Size the sort dropdown to its label so Recommended no
longer truncates, and match the On Device trailing gap to the active
pill's leading inset.
* Flush section toggle and match dropdown font to Search Hub
Drop the trailing track pad so the active pill fits the track exactly
at either end. Size the sort and format dropdown text to text-xs like
the Search Hub button, and clip long labels without an ellipsis.
* Fix sort menu checkmark overlap and lock dropdown widths
Keep the option's right padding so the selected checkmark no longer
overlaps the label, and let the open menu expand to fit it. Set the
format and sort triggers to a fixed width matching the Search Hub
button so they always line up.
* Keep section toggle and dropdowns on one row
Drop the wrap and size the Search Hub button, format and sort dropdowns
to a shared 100px so they stay equal width and fit on one row without
widening the box.
* Studio: pre-load inference settings dialog with native context
Add a gear on downloaded GGUF quant rows that opens a settings dialog
to adjust inference parameters before loading a model:
- Context length, KV cache dtype, speculative decoding and tensor
parallelism, all written to the runtime store the load call reads.
- Settings can be remembered per model in localStorage.
- The context slider ceiling and "Model supports up to N tokens" come
from the model's native context, read from GGUF metadata and returned
by /api/models/gguf-variants once a variant is downloaded.
Also drop models Studio can't run for chat (diffusion, image, video)
from the recommended feed and Hub search, plus minor selector polish
on row hover padding, Search Hub and dropdown widths, and tab spacing.
* Studio: model selector polish and memory-aware load warning
Search and listing:
- Drop the "Recommended" and "Hugging Face" section labels while
searching so results read as one list; keep the format and sort
dropdowns visible so search results can still be sorted and filtered.
- Request gguf metadata in the Hub listing so GGUF repos report a
parameter count, restoring the OOM badge for repos without a size
token in the name (Kimi, MiniMax, GLM).
Load settings dialog:
- Warn when weights plus the KV cache at the chosen context exceed
available memory. The KV size is sized by the backend's
architecture-aware estimator via a new kv-cache-estimate endpoint;
the budget uses VRAM plus system RAM. Best-effort, no warning on
failure or on auto context.
- Context Length placeholder reads "auto"; dark background slightly
lighter.
Other:
- Clicking the Custom Folders header opens the folder browser; its
title now reads "Select folder to detect models".
- On Device sort lists Downloaded last.
- Smaller chat template editor font; rounded wrapper clips the prompt
and template editor scrollbars so the right corners stay round.
* Studio: fix load dialog memory warning budget and KV dropdown width
- The memory warning never fired without a discrete GPU. useGpuInfo
returned zero system RAM in that case, so the budget was always zero.
Surface system RAM even when no GPU is present (Mac unified memory),
and have the load dialog read memory directly instead of through props.
- Give the dialog fields shrink-0 so the KV Cache Dtype value (e.g.
q8_0) is not squeezed and clipped by the row.
* Studio: fold fine-tuned models into On Device tab
Remove the Hub models and Fine-tuned source tabs. Fine-tuned models now
show as a section in the Hub tab's On Device view, above Custom Folders,
with the Train icon and a collapse toggle. The section only appears when
the user has fine-tuned models. With no external providers the lone Hub
tab hides its own toggle.
Also: tick-circle Show hidden checkbox and drop the divider above Eject;
keep run settings load params (KV cache dtype, speculative, tensor
parallel) from being clobbered by a mid-load status poll.
* Studio: stage load settings in the sidebar with a Load on selection toggle
Replace the pre-load settings popup with a staging flow in the Run settings
sidebar. The gear on a downloaded quant row now stages the model and opens
Run settings with Load model and Cancel buttons, so options like context
length, KV cache, speculative decoding and tensor parallelism are set before
the model loads. A "Remember these settings" tick reuses them next time.
Add a global Load on selection toggle in Settings, Chat tab (default on).
On: Unsloth auto-picks the best settings for your hardware and loads on
selection. Off: picking a model stages it in Run settings to customize first.
The gear always stages, regardless of the toggle.
Other polish in this change:
- Fine-tuned models live under the On Device tab, with a train icon on the
header that jumps to the Fine-tuned section.
- Default to the On Device tab when downloads exist, otherwise the last used
section.
- Standard Unsloth tooltips on the train, folder and gear icons.
- Request the gguf param count on every Hub listing fetch so Kimi, MiniMax
and GLM show a size badge.
- Search Hub hover state, scrollbar position and minor spacing fixes.
Remove the old inference load settings dialog.
* Studio: always show the fine-tuned shortcut and smooth out the picker
- Fine-tuned section and its train shortcut now always show on On Device,
with an empty state when no fine-tuned models exist yet.
- Folder icon on the header jumps to Custom Folders instead of opening the
browse popup, matching the train shortcut.
- Folder browser keeps the list mounted and dims it while refetching, so
toggling Show hidden or changing folders no longer flashes.
- Drop the tooltip hover grace area in the picker so moving between the
train, folder and gear icons switches the tooltip at once.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: add quantization display options and drop the fine-tuned empty text
- Settings, Chat: 'Expand quantizations' toggle. On expands every On Device
GGUF model's quantizations by default; off keeps them behind a click
(default).
- Settings, Chat: 'Show all quantizations' toggle. On lists every quant
including ones not downloaded (default); off shows downloaded only.
- Remove the empty-state line under the Fine-tuned header; the header still
shows on its own.
* Studio: let expanded quantizations collapse on click and split the On/Off help
- With Expand quantizations on, clicking an On Device model now collapses or
re-expands its quantizations. The collapse state is in memory only, so it
resets on reload and when the setting is toggled.
- Put the Off sentence on its own line in the quantization setting descriptions.
* Studio: reorder chat settings and rename the model section
- Rename the Models section to Select model settings and move it above the
Chat menu section.
- Trim the section and Load on selection descriptions.
* Studio: tighten the On/Off lines in the model setting descriptions
Use a line break instead of separate spans so the On and Off lines sit on
consecutive lines without the extra paragraph gap.
* Studio: top-align the Load on selection toggle
Add an alignTop option to SettingsRow and use it so the toggle sits at the top
of the row next to the label, not centered against the tall description.
* Studio: put the gear hint and example chip on one line
Move the gear example chip inline with its label so it reads as a single line
instead of wrapping onto its own row.
* Studio: move the New badge from API keys to Chat settings
Add the New badge to the Chat settings tab and drop it from API keys.
* Studio: line the Load on selection toggle up with the first description line
Offset the top-aligned control past the label row so it sits next to the On
line instead of the label.
* Studio: label the chat menu item Chat with Files (RAG)
Rename the Chat with Files entry in the chat menu settings to clarify it is RAG.
* Studio: drop the pill around the gear example so it fits on one line
Remove the background and padding from the gear example chip so it sits inline
with its label at a lower height.
* Studio: fold the gear example into the description line spacing
Render the gear example inline in the same text block so its line spacing
matches the On and Off lines instead of an extra flex gap.
* Studio: scope Show all quantizations to On Device only
Gate the downloaded-only filter on an onDevice flag so Recommended and other
browse lists always show every quant, and note On Device in the setting copy.
* Studio: tidy On Device GGUF rows
- Drop the redundant Quantizations subheading under On Device models.
- Relay GGUF vision support up to the model name as a Vision badge instead.
- Drop the repo size from On Device GGUF model rows since the quants already
show their size.
* Studio: pin the eject button and tidy General settings
- Move Eject loaded model out of the scrollable list into a centered footer so
it stays in view no matter how far the list is scrolled.
- Space out and center the gear example in the Load on selection description.
- General: drop the duplicate Unsloth version section, move llama.cpp
notifications above Helper LLM, and note new models in its description.
* Studio: add left padding before the gear example
Nudge the gear example away from its label with a small left margin.
* Studio: make the eject footer a sticky bar over the list
Pin Eject loaded model to the bottom of the scroll area with the menu
background so rows scroll under it, and drop the divider line.
* Studio: drop the eject footer background, keep it a sticky button
Make the sticky eject a centered transparent button so it coexists with the
rows scrolling behind it. The wrapper ignores pointer events so only the button
is clickable.
* Studio: give the eject button a solid background
Add the menu background, a border and a soft shadow to the sticky eject button
so it reads as a floating button over the list.
* Studio: restore the eject footer block, keep hover on the button only
Bring back the full-width menu background behind the sticky eject footer, but
keep the button compact and centered so the hover stays on the button.
* Studio: show the vision badge on On Device rows without expanding
- cached-gguf listing reports has_vision (mmproj present), so the badge shows
on the model name without opening the quantizations.
- Make the vision badge icon-only with a tooltip: "This model can process
image inputs". Falls back to the expander-reported value on older backends.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Make LM Studio and Local models sections collapsible
* Fade the eject footer instead of a solid block
* Wrap the vision badge in a bordered pill
* Taller model list with the eject footer pinned to the bottom
* Use purple for the vision badge to set it apart from GGUF
* Reduce the model list height
* Make the eject button inline with no background block
* Match the vision badge color to the Hub indigo tone
* Shorten the model list and square off the format tags
* Pin the eject button so it floats at the bottom of the list
* Give the floating eject button a tinted background
* Add bottom clearance so the list ends on white space under the eject button
* Match eject button to the menu background and unify the settings gear icon
* Move eject below the list and match its shadow and dark background
* Drop the min height so short model lists leave no white space
* Remove the eject button fill so it never covers the list
* Nest dropdown hover radius inside the menu corners
* Float the eject pill again and fix sort dropdown hover radius
* Make the eject button opaque in both themes on hover and dark
* Trim the model menu bottom padding so it stops clipping the last row
* Match dark eject background to the Search Hub button and pad row indicators
* Fade the model list bottom edge while rows sit below the fold
* Lift the eject button and trim the section toggle right padding
* Nudge the model list taller and run the bottom fade to the box edge
* Nudge the model list slightly taller
* Remove the eject button shadow
* Align the eject button to the right
* Widen the Search Hub and dropdowns and right-align them
* Seat the eject button at the base and restore On Device right padding
* Reduce the Search Hub and dropdown width by 4px
* Widen the model menu so the section toggle keeps its padding
* Make the eject button an icon-only button with shadow
* Tighten section tab padding to cut the grey between tabs
* Revert section tab padding back to px-3
* Remove the section toggle trailing padding
* Add an eject button beside the model selector trigger
* Shrink the in-list eject button to a smaller proportional size
* Raise the in-list eject button
* Make the trigger eject a bare icon next to the dropdown arrow
* Revert eject back to the labeled button on the right
* Place the format and sort dropdowns next to the section toggle
* Raise the eject button and shorten its label to Eject model
* Widen the gap between the toggle and dropdowns slightly
* Align Search Hub with the last dropdown via a shared-width grid
* Narrow the model menu for symmetric padding
* Stretch the search row so Search Hub lines up with the last dropdown
* Inset the list so the right padding matches the left
* Right-align dropdowns and full-width search so Search Hub meets the last dropdown
* Pack section toggle and dropdowns with a uniform gap
* Inset search row so Search Hub aligns with the Trending dropdown
* Trim model menu right padding to match the left
* Nudge model list scrollbar inward
* Move eject button to the bottom left with a light shadow
* Shorten show all quantizations description
* Keep eject button right-aligned, nudged in from the edge
* Move Connected into the section toggle as a cloud-icon tab
* Align eject button with the format tag edge
* Right-align Connected layout so Search Hub meets Trending
* Download selected models through the Hub download manager
* Add Other models section for non-Unsloth downloads
* Add directions icon and shortcut for Other models section
* Space out subheadings and gate Other models on non-Unsloth downloads
* Use direction-right icon for Other models
* Use flag icon for Other models
* Widen Connected menu so dropdowns align with Search Hub
* Model selector: truncate long quant labels and tidy layout
- Hub GGUF card: truncate long file-path quant labels with an ellipsis
instead of overflowing the row.
- Connected layout: left-pack the dropdowns and size the box so the last
dropdown's right gap matches the pill's left gap, with Search Hub on its edge.
- On Device: show MLX/Safetensors with the size on non-GGUF rows.
- Connected list rows use the same grey hover as the tabs; the selected
section tab no longer shows a hover change.
* Model selector: drop stale custom section on restore
A persisted custom section value no longer maps to a tab, so restoring it
opened the picker to an empty view. Fall back to recommended instead.
* Model selector: align the non-connected search bar with the All dropdown
Nudge the non-connected box width so the search bar's right edge meets the
All dropdown, which lands Search Hub on the last dropdown's edge.
* Studio chat model selector: remember last tab, route non-GGUF downloads through Hub, stack overlays
- Restore the last Hub section (Recommended / On Device) on every open instead of always snapping to On Device when downloads exist.
- Route uncached non-GGUF repos (safetensors / MLX) through the Hub download manager via a snapshot download, so every model download shows in the bottom-right indicator and follows Load on selection like GGUF.
- Allow safetensors in Recommended on Mac (they run locally there now), and honor the Safetensors format filter instead of dropping it via the recommendation default.
- Stack bottom-right overlays in one column so the download panel and banners never overlap.
- Add evenly spaced divider lines between the On Device subheadings.
- Pad the bottom of the list so the floating Eject pill never covers the last row.
* Studio downloads panel: widen left padding on header and rows
Bump the left inset to pl-4 while keeping pr-3 so the collapse and cancel buttons stay put.
* Studio: update cached-gguf route tests for the has_vision field
list_cached_gguf now returns has_vision per row (vision badge on On Device);
the expected dicts were missing it. True for the mmproj vision repo, False elsewhere.
* Studio: keep MLX/safetensors selectable in chat-only Mac search
The empty Recommended view allows GGUF plus MLX/safetensors on Mac, but the
curated and HF search lists dropped non-GGUF in chat-only via a GGUF-only filter,
so typing a query hid runnable Mac models. Reuse isRecommendableFormat in both
lists so search matches the empty view (chat-only non-Mac stays GGUF-only).
* Model selector: restore global model search and fix GGUF/device-fit regressions
- Search: training, export and onboarding pickers searched only the unsloth org
on a typed query. Restore the prior behavior (global Hub search with unsloth
floated first when a query is typed, curated unsloth listing when empty).
- Recommended browse: the GGUF/MLX-only gate ran before the format filter, so
the Safetensors filter and the Trending/Recent sorts always came back empty.
Apply that gate only for the Recommended sort and chat-only mode.
- GGUF metadata: request the gguf expand field through listModels so repos with
no size token in the name (Kimi, MiniMax, GLM) report a param count for the
size and OOM badge.
- Local GGUF: custom-folder and standalone ./models/*.gguf files now load
directly with the GGUF marker instead of dead-ending in the variant expander,
and scanned GGUF folders are classified via a backend model_format hint.
- Device fit: use system RAM in the budget on unified-memory hosts, and keep MLX
rows selectable on chat-only Macs.
- kv-cache-estimate: resolve the quant from the snapshot-relative path, skip MTP
drafter files, and prefer the most complete snapshot (mirrors the variant
scanner). Bound the Ollama manifest walk.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Model selector: classify suffixless local GGUF folders consistently
Complete the model_format plumbing so a GGUF folder is detected and loaded
through the same GGUF path that the format filter already uses:
- _scan_models_dir: a config.json no longer disqualifies a folder whose only
weights are .gguf, so HF GGUF repos shipping a config still classify as GGUF.
- _scan_lmstudio_dir: emit model_format for every GGUF row (LM Studio dirs
rarely carry a -GGUF suffix), via a shared _dir_model_format helper.
- Custom Folders and LM Studio rows: use localModelIsGguf (the same helper the
filter uses) so the row label, expand-vs-direct-load, and isGguf flag agree;
a suffixless GGUF folder no longer filters as GGUF but loads as non-GGUF.
Adds tests/test_local_model_format.py covering the classification rule.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio model selector: tighten section spacing
Trim each subheading's gap to its rows (pb-1.5 to pb-1) and pull the On Device
heading block tight to the controls while Recommended keeps a little top room.
* Hub: format filter fix, sort defaults, avatar and layout polish
- Format dropdown now filters the feed's Latest list too, so the default
GGUF hides fp8/safetensors and picking a format changes the rows.
- Latest Unsloth Models sorts by newest created, not recently updated.
- Sort dropdown order: Newest, Trending, Most downloads, Recently
updated, Most likes.
- Unsloth uploads with no upstream provider logo show the Unsloth avatar
instead of a colored initial.
- Owner scope pill gets a little more room before the chevron.
- README detail column lines up with the top bar (both-edges gutter).
- Long file-path quant labels truncate instead of overflowing the row.
- Model list keyboard nav no longer clips the focus ring.
- Run settings sheet: restore the Remember settings toggle and larger
Load/Cancel buttons on the staged load flow.
* Hub: hide the RAG embedding model from browse previews
The Hub discover feed and chat model selector pull from the Hugging Face
listing on the client, which the backend _is_hidden_model filter never
touches, so the RAG embedder (unsloth/bge-small-en-v1.5-GGUF) and the
llama.cpp validation probe leaked into the lists.
Added isHiddenModelId mirroring the backend needles and filtered it out of
the discover rows, the trending feed, and the selector's recommended and
Hugging Face search lists. Per-repo file and download views are untouched,
so the model is never deleted and a reinstall still shows it as already
downloaded.
* Studio: skip hidden dirs when checking a folder for downloaded models
_dir_has_downloaded_model walked the tree with rglob("*") bounded by
max_entries. rglob yields entries in arbitrary order and counts every one, so a
model directory that also holds a large hidden subtree (.git/.cache/venv) could
exhaust the budget before reaching the real weights and falsely report no model,
hiding a valid Recommended-folder chip. Replace the generic-weights pass with a
bounded BFS that skips hidden directories so their entries can't starve the walk.
Adds a regression test (50-entry .git beside the weights, max_entries=10).
* Fix/adjust model selector handling for PR #6364
* Studio: address codex review on the staging/recommended-folder paths
- chat-page auto-load: selectModel only clears pendingSelection on success, so a
failed auto-load left the hidden stage (and its edited load knobs) behind.
Abandon the stage when it still matches the failed pick.
- model picker: count fine-tuned rows in the On Device empty check so a
fine-tuned-only tab no longer shows a false 'No models on device' message
above the Fine-tuned section.
- general settings: add the remembered per-model load settings key to PREFS_KEYS
so 'Reset all local preferences' actually clears it.
- recommended-folders: recognize PyTorch .bin weights (gated by the scanner's
weight-name prefixes) so a .bin-only model folder still earns a chip; add tests.
* Studio: name-gate .bin weight detection and complete selector preference reset
Follow-up to the codex review on the model_format/recommended-folder paths:
- _dir_model_format and _scan_models_dir treated any .bin (incl. tokenizer.bin)
as a non-GGUF weight, so a suffixless GGUF folder shipping a companion .bin was
misclassified as a plain checkpoint and routed through the wrong load path.
Factor the scanner's weight-name gating into shared _is_weight_bin /
_has_non_gguf_weights helpers and use them everywhere (also in
_dir_has_downloaded_model).
- PREFS_KEYS was missing the new 'Select model settings' keys (load on selection,
expand/show-all quantizations), so 'Reset all local preferences' left them set.
- On Device cached search dropped the active format filter while a query was
typed; keep matchesFormatFilter applied so the format dropdown stays consistent.
Adds tests for the tokenizer.bin vs weight-.bin classification.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: validate Ollama blobs, gate staged context, honor RAM budget on no-GPU hosts
- recommended-folders: only count an Ollama dir once its manifest resolves to an
on-disk model blob, so a failed/pruned pull no longer surfaces an empty chip
- GGUF variant click: only seed the staged contextLength for already-downloaded
picks, so choosing an undownloaded quant from a partially cached repo still
starts its download (the staging effect short-circuits on a known context)
- device fit: classify GGUF variants against the system-RAM budget on no-GPU /
unified-memory hosts instead of reporting everything as fits, and pass
systemRamGb to every variant expander regardless of gpu.available
* Studio: scope Hub search to Recommended, fix staged non-GGUF settings, keep local MLX on Mac
- model picker: only run the Hub search hooks on the Recommended section. On
Device / Connected render local data, so typing there no longer fires HF
requests or a spinner and the local/offline flow is preserved
- chat settings: when a pick is staged, decide the GGUF-only controls from the
staged model's type, not the currently loaded model's. A staged non-GGUF Hub
repo no longer inherits a loaded GGUF's context/KV/speculative controls
- On Device: keep local MLX builds in ./models selectable on Mac (chat-only ran
GGUF/MLX only, but the filter dropped MLX before the format toggle)
---------
Co-authored-by: shimmyshimmer <info@unsloth.ai>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
* studio: persist personalization (profile + theme) server-side
Profile name/nickname/avatar and appearance (theme) were stored only in the
browser's localStorage, so every browser or device that connected to the same
Studio started from defaults and forgot the user's personalization.
Persist them server-side (single-account, stored as one JSON blob in
app_settings) so they follow the account:
- utils/personalization_settings.py + GET/PUT /api/settings/personalization,
with validation (theme/shape enums, avatar must be an image data URL capped at
512 KB) and a 'saved' flag.
- Frontend usePersonalizationSync (mounted in the root layout when signed in)
hydrates the profile + theme stores from the server when a blob exists, and
otherwise migrates the existing local settings up once so nothing is lost;
later changes are written through, debounced. Writers keep using the local
stores unchanged.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix/adjust personalization sync for PR #6516
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix Studio personalization sync edge cases
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
* Studio: defer llama.cpp update probes and self-heal MLX on macOS
Two macOS startup problems shared one root area in the FastAPI lifespan:
- The llama.cpp capability + freshness probes ran inline before the server
yielded, so a cold/slow/flaky network on the GitHub freshness check blocked
'Application startup complete' (~34s on CI, longer in the field). Move both
probes to a daemon thread; app.state stays None until ready (status routes
already re-probe at request time). Opt out with UNSLOTH_DISABLE_UPDATE_CHECK=1.
- Train and Export were greyed out because mlx/mlx-lm/mlx-vlm arrive only
transitively and a resolver backtrack silently drops them, so CHAT_ONLY stayed
true. Add utils/mlx_repair.py: when Apple Silicon is detected without MLX,
reinstall mlx/mlx-lm/mlx-vlm by name on a daemon thread and re-run hardware
detection (opt out UNSLOTH_DISABLE_MLX_AUTOREPAIR=1). Surface a chat_only_reason
in /api/health plus a sidebar tooltip so a greyed Train/Export explains itself
instead of failing silently.
* Studio: guard model defaults against a None model name
load_model_defaults(None) called model_name.lower() with no guard, raising
'Error loading model defaults for None' before any model is selected. Return
an empty dict for a falsy/non-str name.
* Studio: drop obsolete upstream macOS + Windows Blackwell prebuilt pins
Both pins worked around gaps in ggml-org upstream prebuilts, but Studio now
routes every GPU host and all of macOS to the unslothai/llama.cpp fork
(published_repo_for_host), which ships the needed bundles, so both pins are
dead code on the default install path:
- macOS b9415: macOS always routes to the fork (its own macOS bundles), and
host_supports_macos_minos() is the backstop. The pin only fired under an
explicit --published-repo ggml-org override.
- Windows Blackwell b9360: Windows-NVIDIA routes to the fork, whose
windows-x64-cuda13 bundle covers Blackwell (manifest max_sm 120, toolkit
13.3), so the pin's self-disable check makes it dormant on every default
install; it could only activate under the same upstream override on a
13.0-13.2 driver.
Remove the pin constants, functions, and call sites. Keep the Blackwell
capability detection (_drop_blackwell_incapable_windows_cuda, _host_is_blackwell,
_windows_cuda_attempt_covers_blackwell) that still drops a non-sm_120 cuda-12.4
build on a Blackwell host. After this, an explicit --published-repo ggml-org
override on a Blackwell 13.0-13.2 host loses its GPU fallback and lands on CPU;
the default fork path is unaffected. Update the install selection-logic and
macOS-compat unit tests for the new no-pin behavior.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: walk back deeper on the macOS upstream prebuilt path
After removing the b9415 macOS pin, the explicit --published-repo ggml-org
upstream path still used the default 2-release fallback, so a pre-macOS-26 host
behind a run of macOS-26-only builds would exhaust two too-new plans (minos is
only checked post-download) and drop to a source build before reaching a
loadable older release. Walk back as deep as the fork macOS path
(DEFAULT_MAX_MACOS_RELEASE_FALLBACKS), turning the removed static pin into
dynamic discovery. Addresses review feedback on the macOS upstream fallback.
* Studio: pin transformers during MLX self-heal so it cannot break Studio
mlx-lm/mlx-vlm declare transformers>=5, but the single-env install pins
transformers==4.57.6. The self-heal used --upgrade with no constraint, so it
could upgrade transformers in the live venv and break the rest of Studio just to
make import mlx.core pass. Pin transformers to the installed version via a
constraint file: the resolver either finds an mlx build compatible with it or
fails (we stay chat-only), never upgrading transformers underneath Studio.
Addresses review feedback on the MLX repair install.
* Studio: harden MLX self-heal against an unsupported mlx-vlm
Pinning transformers alone made uv backtrack mlx-vlm to 0.3.9 (below unsloth-zoo's
mlx-vlm>=0.4.4), which imports but breaks VLM Train/Export -- so the self-heal
could clear chat-only onto a broken stack. Mirror the main installer: set
UV_OVERRIDE=overrides-darwin-arm64.txt so a current mlx-vlm coexists with the
transformers pin, require the same minimum versions unsloth-zoo declares, and
gate/validate on a full mlx_stack_available() check (not a bare import) so an
old or partial stack stays chat-only. Addresses PR review.
* Studio: filter Blackwell-incapable CUDA in resolve_upstream_asset_choice
resolve_upstream_asset_choice returned the first windows-cuda choice unfiltered,
so a Blackwell host could be handed an sm_120-incapable cuda-12.4 build while the
sibling planners drop it. Apply _drop_blackwell_incapable_windows_cuda here too
and fall through to the CPU bundle on a Blackwell host with no capable GPU asset.
Addresses PR review.
* Studio: re-poll health so MLX self-heal reaches an open UI
The sidebar cached the initial /api/health, so a successful background MLX
self-heal (chat_only flips false) did not re-enable Train/Export until a manual
reload. While chat-only for the recoverable mlx_unavailable reason, re-poll
/api/health and stop once Train/Export become available. Addresses PR review.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: make the disabled Train/Export tooltip reachable
The greyed Train/Export items pass a tooltip explaining why (e.g. MLX missing),
but a disabled <button> fires no pointer events and SidebarMenuButton only showed
tooltips while collapsed, so the explanation never appeared. Wrap a disabled
button in a focusable span and show its tooltip while expanded too; enabled items
keep the collapsed-only behavior. Addresses PR review.
* Studio: gate Train/Export on the full MLX stack, not bare mlx.core
detect_hardware enabled MLX training whenever `import mlx.core` worked, but the
MLX self-heal (utils/mlx_repair) treats a stack without mlx-lm/mlx-vlm at the
versions unsloth-zoo requires as inadequate. That asymmetry let the UI enable
Train/Export on exactly the partial/backtracked stack the self-heal is trying to
repair (greyed-in-but-broken VLM export). Gate on the same mlx_stack_available()
criterion so a partial stack stays chat-only (reason mlx_unavailable) and the
background repair restores it. Addresses PR review.
* Fix MLX repair and health auth for PR #6494
* Fix macOS upstream prebuilt fallback for PR #6494
* Fix MLX stack validation for PR #6494
* Fix MLX self-heal validation for PR #6494
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Review fixes: isolate hardware-state test, robust transformers pin
- test_chat_only_reason.py: detect_hardware() assigns module globals directly,
which monkeypatch does not revert; the autouse fixture now saves and restores
DEVICE/CHAT_ONLY/CHAT_ONLY_REASON/IS_ROCM so a chat-only verdict here cannot
leak into other backend tests (e.g. test_utils.py) on a GPU host.
- mlx_repair.py: read the transformers version from importlib.metadata instead of
importing transformers, so the install pin is not silently dropped when
transformers has valid metadata but fails to import.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix CI: model full MLX stack in dispatch tests, keep selection test offline
dispatch (macOS) job:
- detect_hardware now gates MLX on the full stack (mlx_stack_available imports
mlx_lm/mlx_vlm and checks dist versions), so faking only mlx.core makes the
apple_silicon_mlx profile resolve to CPU. The dispatch tests assert the routing
decision when the stack IS usable, so model a complete stack:
test_hardware_dispatch_matrix patches utils.mlx_repair.mlx_stack_available and
test_is_mlx_dispatch_gate patches hardware._has_usable_mlx_stack. The stack
predicate's own internals stay covered by test_mlx_repair.py.
Repo tests (CPU) job:
- test_no_cuda_attempt_on_published_path_for_13_1 fell through to a live
github_release_assets() upstream fetch after the Blackwell filter dropped every
published attempt, which the offline security scanner blocks. Stub that fetch so
the walk-back deterministically finds no usable CUDA build and raises
PrebuiltFallback without network.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Harden MLX self-heal: prepare transformers constraint inside the try
attempt_mlx_repair runs on a daemon thread, but _transformers_constraint_args was
called before the try. A failure there (e.g. tempfile.mkstemp on a full disk or a
bad TMPDIR) would propagate unhandled and silently kill the self-heal thread.
Move the call inside the try and initialize constraint_path so any such failure
is caught and leaves Studio chat-only instead of crashing the thread.
---------
Co-authored-by: Daniel Han <michaelhan2050@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
* Use uppercase -GGUF suffix for GGUF export default names
Match the HF/Unsloth GGUF repo convention (Model-GGUF). Default export save dir
'<model>-gguf' -> '<model>-GGUF', the local-path sibling dir token '_gguf' -> '_GGUF',
and the model-name placeholder 'my-model-gguf' -> 'my-model-GGUF'. Pre-filled defaults
only; no backend/path logic change.
* Keep local GGUF sibling dir lowercase to match backend cleanup
The uppercase change to siblingGgufDirectory diverged from the backend's
hard-coded intermediate '<checkpoint>_gguf' dir (core/export/export.py), which
the export relocates GGUFs out of and then deletes. With the user's save dir
defaulting to '_GGUF', that no-longer-equal lowercase sibling would be relocated
and removed, which can delete an existing export. Revert the sibling default to
'_gguf'; the user-facing GGUF export name (buildRelativeSaveDirectory) keeps the
uppercase -GGUF token.
* Tighten GGUF sibling-dir comment
* Trim comments to be more succinct
---------
Co-authored-by: Daniel Han <michaelhan2050@gmail.com>
* Show model provider/org in the trust_remote_code consent dialog
The consent dialog showed only the trailing model name (modelName.split('/').pop()),
dropping the owner. The HF org/owner is the 'who do I trust' signal the prompt is
asking about, so render it: 'NVIDIA-Nemotron-3-Nano-4B from "unsloth"'. A null
provider for local paths and bare names leaves those renders unchanged. Applies to
the enable/blocked/malware variants (shared description block).
* Only show consent provider tag for a confident single Hub repo
Tighten parseModelDisplay so the 'from "<provider>"' tag is shown only for a
canonical owner/repo Hub id (exactly one slash, both segments non-empty) that is
not a local path and not part of a multi-repo scan. This avoids misattributing a
relative local directory name (models/llama/7b) or a LoRA base/external repo's
finding to the wrong publisher in a trust decision. Extract a ProviderSuffix
component so both description branches render the clause identically via &&.
* Tighten consent provider-tag comments
* Source the consent provider tag from the backend
The dialog inferred the provider client-side from the model id, using
scanCreatedRepos (a cleanup-only list) to detect multi-repo scope and a
regex that missed bare relative paths like a local owner/model dir. Both
could attribute the scanned code to the wrong publisher.
Move the decision to the backend, where locality and scan scope are
known: _consent_provider returns the owner only for a single, non-local,
canonical owner/repo Hub id, and the route returns it as payload[provider].
The frontend now renders scan.provider directly.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Suppress consent provider tag when external auto_map code is scanned
A single Hub repo can declare an auto_map that loads code from another repo
(owner/other--module.Class). The scanner fingerprints that external repo's
Python, but security_targets still held only the primary, so the dialog
attributed the custom code to the primary publisher. Pass the external refs
collected during the scan to _consent_provider and return no provider when any
are present, so attribution is shown only for genuinely self-contained repos.
* Trim comments to be more succinct
---------
Co-authored-by: Daniel Han <michaelhan2050@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Scan auto_map for GGUF-only repo ids in the consent gate
The trust_remote_code consent gate treated any repo classified GGUF-only
(ships .gguf, no transformers-loadable weight) as having no remote code,
so _config_has_auto_map returned False even when a config declared an
auto_map and the repo shipped the referenced .py. The evaluator then
skipped the scan/fingerprint for that target entirely.
GGUF-inertness is a property of the loader, not the repo. A GGUF
selection loads via llama.cpp, which never reads config.json/auto_map,
and that case is already short-circuited upstream by the caller's
is_gguf check (the inference route skips the remote-code preflight for a
GGUF load). Every path that reaches this helper (export, training,
non-GGUF inference) loads through transformers/Unsloth from_pretrained,
which DOES import auto_map even for a repo that only ships .gguf weights:
the custom module runs before from_pretrained fails on the missing
transformers weights. The export path has no is_gguf guard and passes the
source straight to FastLanguageModel.from_pretrained(trust_remote_code=True),
so the in-helper GGUF skip let a repo with config.json (auto_map) +
modeling_x.py + only a .gguf run unreviewed code during export.
Drop the redundant repo-level GGUF short-circuit (and the now-unused
_is_gguf_repo helper). A direct .gguf file reference stays inert via
_is_direct_gguf_file_ref because that genuinely is a single-file llama.cpp
load; repo ids are always scanned. A GGUF repo whose auto_map ships no .py
still allows via the existing empty-code path, so legitimate GGUF loads
are unaffected (and GGUF inference never reaches this helper at all). Only
a repo that actually contains a .gguf can change behavior here; non-GGUF
repos (safetensors, MLX) are byte-identical before and after.
Update the GGUF auto_map test to expect a scan, and add two regression
tests: a GGUF-only repo shipping auto_map Python is scanned and blocked,
and a transformers-style repo (safetensors / MLX .npz) with auto_map stays
scanned and blocked.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Remove trust_remote_code config defaults; consent dialog is the only enabler
trust_remote_code is a per-load decision that must go through the remote-code
consent dialog, which scans the auto_map code and pins the exact version. Two
pre-set paths could still enable it without the user reviewing any code, and the
GGUF consent bypass rode one of them into the export flow:
- 4 model_defaults YAMLs shipped trust_remote_code: true (GLM-4.7-Flash,
Nemotron-3-Nano-30B-A3B, PaddleOCR-VL, ERNIE-4.5-VL).
- The frontend consent hook silently enabled trust_remote_code on a clean scan
whenever the caller flagged the model as needing it.
Remove every trust_remote_code key from the model_defaults YAMLs (the loaders
already default to False when the key is absent) and delete the frontend silent
auto-enable, so trust_remote_code is only turned on after the user approves the
scanned code in the dialog.
The three models that genuinely run custom code ship auto_map, which the consent
gate detects on its own via _config_has_auto_map, so the dialog still fires for
them in inference, training, and export (Nemotron is also re-granted by the
trusted-org auto-enable in the workers). GLM-4.7-Flash has no auto_map:
glm4_moe_lite is native in transformers 5.0+ and it loads with
trust_remote_code=False, so its YAML flag was a no-op.
Adds test_yaml_trust_remote_code_removed.py.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Drop YAML sections emptied by trust_remote_code removal
Removing trust_remote_code from a model YAML whose section had no other key
left a bare `inference:` header, which PyYAML parses as None;
load_inference_config() then does `model_config.get("inference", {}).get(...)`
and crashes on the None. Drop those now-empty section headers (24 model
defaults, all the `inference:` section) so callers fall back to family/default
inference params, which is the same result those models had before (their only
inference override was trust_remote_code).
Strengthens test_yaml_trust_remote_code_removed.py to forbid any empty/None
top-level section and to load the affected models' inference config end to end.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add sweep asserting every model YAML loads via training + inference paths
Loads all model_defaults YAMLs through load_model_defaults (training) and
load_inference_config (inference) with the exact .get() access patterns the
routes use, so a malformed/None section that crashes either loader is caught.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Assert ex-TRC auto_map models still surface the consent dialog
Removing the trust_remote_code YAML default must not suppress the dialog for the
models that genuinely run custom code. The dialog is driven by the repo's auto_map
(via preflight_remote_code_consent_for_targets -> _config_has_auto_map), not the YAML
flag, so Nemotron/PaddleOCR-VL/ERNIE-4.5-VL still require consent; GLM-4.7-Flash (no
auto_map) takes no dialog and loads natively. Mocks only the Hub config + .py reader.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten comments in consent-gate changes
* Trim comments to be more succinct
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <michaelhan2050@gmail.com>
* Studio: self-heal unsloth namespace-package shadows in all subprocess workers
A directory named `unsloth` (or `unsloth_zoo`) without an __init__.py on
PYTHONPATH/sys.path, a stray source checkout or a polluted PYTHONPATH, makes
`import unsloth` resolve to an empty namespace package, so a worker's
`from unsloth import FastLanguageModel` dies with a cryptic
"cannot import name ... (unknown location)".
The LLM training path already recovered from this via `_ensure_real_packages`
in trainer.py (PR #6269), but the inference, export, and embedding-training
subprocesses imported Unsloth directly with no guard. Extract that helper into
a shared, dependency-free core/import_guards.py and call it before the Unsloth
import in every subprocess: it drops the offending sys.path entries, imports
the real packages (unsloth before unsloth_zoo so the pre-zoo GPU fixes run),
then restores sys.path. trainer.py now imports the shared helper instead of its
local copy.
Covers both unsloth and unsloth_zoo and both namespace origin forms (None and
"namespace"). The existing PR #6269 test now exercises the shared helper.
* Studio: distinguish a failed model load from no model in the attach gates
A failed load never sets the checkpoint, so the image and audio attach gates
fell through to "Load a model before adding images/audio", which reads as if
the user simply forgot to pick a model rather than that the load errored. Add a
dedicated lastModelLoadError to the chat runtime store, set only when an actual
load attempt fails (not on refresh, list, status, or unload errors, which keep
using modelsError) and cleared when the next load starts. The image gate (all
three call sites) and the audio gate now use it to report a failed load and
point at the server logs, while still blocking in exactly the same cases.
* Tighten namespace-shadow guard and load-error comments
* Studio: keep staged model's load settings visible while it loads
* Studio: pin staged load settings at click time, match loading pick by variant
* Studio: drop any stale staged pick after a successful load
* Studio: disable staged Load button while a different model loads
When a model is staged with "Load on selection" off and a different model (or a different GGUF variant of the same repo) is already loading, selectModel's in-flight-load guard matches on id + native path token only, so the click was silently deduped to a no-op. The staged Load button stayed enabled but did nothing, and the stale stage was then cleared once the other load finished.
Disable the staged Load button (showing "Another model loading...") whenever a different model is loading, so it is no longer an enabled no-op. The stage stays put and the user can retry once the in-flight load settles.
* Studio: refuse loading a different model while one is in flight
The in-flight-load guard in selectModel matched on id + native path token only, so a different GGUF variant of the same repo fell through and was silently deduped to a no-op. The earlier commit disabled the staged Load button for this case, but other entry points (e.g. selecting a different quant from the model picker with "Load on selection" on) still hit selectModel directly and no-op'd.
Make the guard variant-aware (id + GGUF variant + native path token) and, for a genuinely different model while a load is in flight, surface a "Another model is already loading" toast instead of silently returning. The load path has no clean supersession, so a second concurrent load is not started; the user is told to wait or cancel. Centralized in selectModel so every entry point is covered.
* Studio: lock staged model's load settings while it loads
The staged Load button snapshots context length, KV cache dtype, speculative
decoding, draft tokens, and tensor parallelism at click time, but the settings
sheet stays mounted during the load, so edits made while "Loading..." shows were
silently ignored and then overwritten by the load response. Disable those
controls while the staged pick is loading so the visible state matches what the
run actually uses.
* Studio: refuse to stage a model while another load is in flight
With Load on selection off, selecting a model mid-load staged it into
pendingSelection and showed a disabled "Another model loading..." button,
implying the user could retry once the load settled. The post-load cleanup then
treated that queued pick as stale and silently cancelled and cleared it. Refuse
to stage while a load (or a cancel's background unload) is in flight: stageModel
no-ops in that state so every entry point (the chat selector and the Hub) is
covered, and stageOrLoad surfaces a toast on the common path. The immediate-load
path is unchanged; selectModel already rejects a concurrent load.
* Tighten staged-load guard comments
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Studio: fix llama.cpp update toast tag and reload hint
The post-update toast used the job's to_tag, which is the bare bNNNN build
number (same as installed_tag), so it showed e.g. "b9726" instead of the full
release tag. Use status.latest_tag (e.g. b9726-mix-<sha>) to match the tag the
banner already shows, falling back to to_tag and then a generic label.
Also drop "Reload your model to use it." when there is nothing to reload: only
append it when a local model is loaded, since external-provider models do not
use llama.cpp.
* Fix/adjust llama update toast for PR #6493
---------
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
Recognise the Gemma 4 separate-drafter MTP family, auto-download the drafter with retry, fall back to n-gram with a clear reason when it cannot be resolved, and retry the download on reload. Gemma 3n (ships no drafter) and embedded-MTP models (Qwen) are unaffected.
Fixes#6406
* Studio: Auto disables MTP for MLA models (GLM-5.2 et al.); UNSLOTH_MLA_MTP_ENABLED to re-enable
Studio's Auto speculative mode promotes any embedded-MTP model >=3B to
--spec-type draft-mtp. For MLA models (GLM-5.2/DeepSeek/Kimi) that is a
regression: llama.cpp's MLA/DSA MTP path keeps a duplicated full target-KV
context and recomputes the sparse-attention indexer every draft step, so it
runs ~2x slower than no speculation (GLM-5.2 UD-IQ1_S bench: 27 vs 45 tok/s,
flat across draft depth 1..6 and 96-100% acceptance, on both prose and code).
vLLM/SGLang get a speedup from the same model, so this is a llama.cpp
implementation gap, not a model property.
Auto now drops embedded MTP for MLA models and falls back to ngram-mod (or
spec-off when the binary lacks ngram-mod), mirroring the existing sub-3B
fallback. The metadata separator is kv_lora_rank: it is present on MLA models
and absent on non-MLA embedded-MTP models (Qwen3.x-MTP), whose MTP module is
structurally identical but fast, so a "full layer" heuristic cannot tell them
apart. Qwen MTP, separate drafters (Gemma, --model-draft), and non-MTP models
are unchanged.
Explicit overrides still engage the slower MTP route: choosing MTP / MTP+Ngram
in Settings, or passing --spec-type in extra args. UNSLOTH_MLA_MTP_ENABLED=1
re-enables Auto promotion for MLA once the upstream path is optimized.
A new spec_fallback_reason value "mla_mtp_disabled" surfaces this as an
Auto-mode policy downgrade (not a binary/update problem), with a settings
banner that points users at the MTP override. It is deliberately kept out of
the "Update llama.cpp" affordance since updating does not help.
Tests: resolver-matrix rows for MLA->ngram-mod / MLA-no-ngram->off /
non-MLA-Qwen->draft-mtp / MLA-separate-drafter->draft-mtp /
non-MTP-MLA->default / forced mtp|mtp+ngram on MLA->draft-mtp / env flag;
kv_lora_rank metadata fixtures; and reload-skip coverage (Auto ngram-mod is
idempotent, forced mtp bounces a reload).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: add an Open button to reveal the models folder in the file manager
* Studio: report models folder creation failures
---------
Co-authored-by: Wasim Yousef Said <wasimysdev@gmail.com>
* Fix clipped card shadows in Hub trending carousel
The carousel scroller only had vertical padding, so with overflow-x set
the first and last cards had their drop shadow clipped on the horizontal
edges. Add px-2 with a matching -mx-2 so the shadow has room while the
cards stay aligned with the section heading, and scroll-px-2 so snap-start
does not scroll the padding away on load.
* Align carousel edge fades with the scroll clip edge
The shadow fix gave the scroller an -mx-2 bleed, but the left/right fade
overlays stayed pinned to the wrapper edges, 8px inside the clip edge. That
left a thin strip where a card showed beside the fade, so the fade read as a
separate block instead of blending into the background. Offset both fades by
the same 8px so their opaque edge sits on the clip edge.
* Add click-and-drag panning to the Hub card carousel
The rows only scrolled by wheel or trackpad, and grabbing a card started
a native drag of its avatar image, so the cards could not be dragged to
move the row. Add mouse drag-to-scroll (touch and pen keep native
scrolling), swallow the click a drag would otherwise fire on a card, keep
plain clicks working, and block the avatar's native drag.
* Trim carousel edge fade width from 56px to 44px
* Smooth out carousel drag panning
Scroll snap was correcting the position on every drag frame, which made
the pan feel sticky. Disable snap while a drag is active and restore it on
release so the row follows the pointer and then settles on a card.
* Drop stale carousel drag when the button is released off-element
If a press ended outside the scroller before the drag threshold was
crossed, no pointerup reached us and the drag stayed armed, so a later
buttonless mousemove would scroll the row. Bail out and clear the drag
whenever the primary button is no longer held.
* Studio: fix Bypass Permissions menu freeze and show decimal GB for model sizes
Bypass Permissions freeze: the warning dialog lived inside the composer
"+"/More dropdown and kept the menu mounted via onSelect preventDefault,
so confirming or cancelling the dialog left both popovers frozen open.
Lift the dialog out of the menu into a store-driven
BypassPermissionsConfirmDialog mounted at a stable spot in the composer.
The menu item now closes normally on select and just toggles a new
bypassConfirmOpen store flag, so the popovers dismiss as expected.
Model search sizes: formatBytes divided bytes by 1024 but labelled the
result "GB", so unsloth/GLM-5.2-GGUF:UD-IQ1_S showed 201.8 GB where
Hugging Face reports 217 GB. Switch the search display to decimal
(base-1000) units to match what Hugging Face reports. The GPU-fit math
stays base-1024 since VRAM capacity is binary.
* Studio: address review feedback and add GLM-5.2 high/max/disabled thinking
Review feedback on the Bypass Permissions and size-format changes:
- Mount the Bypass Permissions warning dialog once at the chat-page root
instead of inside each Composer. It is driven by global store state, so
the per-composer mount meant Compare mode (multiple composers) rendered
duplicate dialogs and the shared-composer menu had none. A single root
mount fixes both.
- Defer opening the dialog past Radix's menu-close focus restoration with
setTimeout(0), so the dropdown does not steal focus back and break the
dialog's focus trap.
- Clamp the unit index in formatBytes so units[i] cannot go out of bounds
past TB (and to absorb log() float error at exact powers of 1000).
GLM-5.2 reasoning levels:
GLM-5.2's template gates thinking with enable_thinking and also reads a
reasoning_effort level ('high' or 'max'), so it needs high / max /
disabled rather than the binary toggle it got before (its style was
detected as enable_thinking, which made 'high' unreachable). Add a new
reasoning style 'enable_thinking_effort' that reuses the effort dropdown
but, unlike gpt-oss, can be fully disabled:
- detect_reasoning_flags classifies a template that has both
enable_thinking and reasoning_effort, extracting the discrete levels
from the quoted effort literals it branches on. Templates with only one
of the two (gpt-oss, Qwen3, DeepSeek, GLM-4.6) are unchanged.
- _request_reasoning_kwargs maps the new style to enable_thinking plus an
in-range reasoning_effort; disabling sends enable_thinking=false. The
gpt-oss reasoning_effort path is left untouched.
- The backend reports reasoning_effort_levels on the load/status response;
the frontend carries them through to the effort dropdown and sends
enable_thinking + reasoning_effort for this style.
Verified: backend reasoning kwargs render the real GLM-5.2 template to
"Reasoning Effort: High/Max" (thinking) and an empty <think></think>
(disabled); tsc, eslint, i18n parity and the production build all pass.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: address review feedback on reasoning effort and formatBytes
- chat-adapter localReasoningEffort: accept 'minimal' so a template that
branches on it (extracted into reasoning_effort_levels) is sent through
instead of being coerced to 'low' and then dropped by the backend.
- formatBytes: return '0 B' for non-finite / non-positive sizes (missing
metadata -> NaN, Infinity, negatives) and clamp the unit index lower
bound to 0, so sub-1-byte values can't produce a negative index.
* Studio: hybrid reasoning none gate and decimal GB in load progress
- _request_reasoning_kwargs: for enable_thinking_effort models, treat a
raw reasoning_effort='none' (OpenAI 'no reasoning' sentinel) as the
enable_thinking=false off gate, so a direct API caller can disable
thinking even without passing enable_thinking. The frontend already
sends enable_thinking=false; this only affects raw API callers.
- use-chat-model-runtime: the download / 'X of Y GB in memory' load
progress divided bytes by 1024**3 but labelled GB, so it disagreed with
the model picker and Hugging Face. Use decimal GB (1e9) to match.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: carry hybrid reasoning levels on all load paths and harden formatBytes
Review follow-ups on the enable_thinking_effort work:
- Every model-load path now copies reasoning_effort_levels and derives
supportsReasoningOff, via a shared reasoningCapsFromLoad() helper. The
shared/Compare composer load and the three chat-adapter auto-load paths
previously set only reasoningStyle, so a GLM-style hybrid model loaded
through Compare or first-chat auto-load fell back to the default
low|medium|high and lost its Max / Off controls.
- The local send path clamps the effort to the loaded model's advertised
levels (clampReasoningEffortToLevels) instead of a hard-coded list. A
stale "max" carried over from an external provider no longer reaches a
pure reasoning_effort (gpt-oss) model that only accepts none|low|medium|
high, where the backend would have dropped it.
- formatBytes divides iteratively instead of via Math.log, which has float
error at exact powers of 1000 (log(1e12)/log(1000) = 3.9999... would
label 1 TB as "1000 GB"). Keeps the non-finite/non-positive guard.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: restore sidebar roundness to its pre-#6349 state
* Studio: keep train recents as a rounded rectangle, not a pill
---------
Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local>
* Studio: free chat model VRAM at training start only when the GPU is tight
The training start route unconditionally tore down the transformers/MLX
inference subprocess before training, and never stopped the llama.cpp GGUF
server at all, so a loaded GGUF chat model kept holding VRAM for the whole
run. Conversely the HF model was always unloaded even when there was plenty
of room to keep it.
Make the unload VRAM aware and cover every inference backend:
- Add routes/training_vram.py with summarize_resident_chat(),
can_keep_chat_during_training() and free_chat_models_for_training(). The
keep/unload decision reuses the same estimator and live per device free
VRAM reader the training GPU selection already uses (auto_select_gpu_ids,
estimate_required_model_memory_gb, get_visible_gpu_utilization), so the
probe agrees with the placement computed later in start_training.
- When a chat model is resident and training fits alongside it with a
conservative margin (required_gb * 1.15 + 4 GB), keep it loaded so the
user can train and chat at the same time; on a multi GPU box training
lands on a different GPU and both coexist. Otherwise unload the HF/MLX
orchestrator and the llama.cpp GGUF server before training starts.
- The export subprocess shutdown stays unconditional and now runs first so
its freed VRAM is reflected in the decision.
Default deny: non CUDA backends, unestimable models, or any probe error
fall back to the previous always unload behavior.
Adds tests/test_training_vram_coexistence.py and updates two existing route
tests in test_gpu_selection.py.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: per-GPU floor for explicit GPU lists + don't unload chat on invalid gpu_ids
Address review feedback on the chat coexistence probe:
- Explicit gpu_ids mode now enforces a per-GPU floor in addition to the
aggregate free-VRAM check, mirroring auto_select_gpu_ids' min_per_gpu_N.
Without it, an uneven split such as free [45, 10] for a 40 GB job passed
the aggregate threshold and kept chat loaded even though the 10 GB GPU
could not hold its training shard, risking an OOM.
- Invalid explicit gpu_ids (ids outside the visible set, or a UUID/MIG
mask) make resolve_requested_gpu_ids raise. That request is rejected with
a 400 before training starts, so leave the resident chat model untouched
instead of unloading it.
- Tighten the target_modules / gpu_ids type hints to List[str] / List[int].
Adds tests for the per-GPU floor (uneven split unloads, even split keeps)
and for invalid gpu_ids keeping the chat model loaded.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: only free chat VRAM once training will start; handle in-flight and CPU-only chat
Address the second review pass on the chat-coexistence path:
- Run the chat/export VRAM teardown as a before_spawn hook inside
TrainingBackend.start_training, fired only after the start guards pass.
Previously the route freed chat VRAM before calling start_training, so a
refused start (e.g. a lingering pump thread) would tear down the resident
chat model even though no training job began.
- Treat an in-flight HF chat load (loading_models set, no active model yet)
as not safely sizeable: free it rather than risk both OOMing as the load
keeps allocating after training starts.
- Do not count or tear down a GGUF llama-server confirmed to run entirely on
CPU (_gpu_offload_active is False): it holds no VRAM, so killing it cannot
help training fit.
Adds tests for the before_spawn hook (runs on start, skipped when a
subprocess is alive or a pump thread will not die, survives a hook error),
the in-flight load flag, and the CPU-only GGUF exclusion in both the resident
summary and the unload path.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: treat any in-flight chat load (HF swap / mid-start GGUF) as unsafe to keep
Tighten the in-flight detection in summarize_resident_chat so the keep check
never sizes a load that is still allocating:
- Flag loading on ANY non-empty loading_models, not only when active_model_name
is empty. load_model adds the new model to loading_models before clearing the
old active_model_name, so a replacement load during a swap was previously
sized as a normal resident and could OOM as the new model finishes loading.
- Flag a GGUF server that is active but not yet healthy (is_loaded False) as
in-flight: it is still mmaping/offloading layers, so its final VRAM footprint
is unknown.
Consolidates the signal into a single resident["loading"] flag; the route frees
the chat model whenever it is set. Adds tests for the replacement HF load and
the mid-start GGUF cases.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: tighten comments in chat/training VRAM coexistence (comments only)
* Studio: run before_spawn VRAM hook only after GPU-selection validation
Reviewers found the before_spawn hook fired before prepare_gpu_selection
validated gpu_ids (and before config build), so a refused start (invalid
gpu_ids -> 400, or a bad grad-clip value) could still tear down chat/export
VRAM. Move the hook to immediately before proc.start(), once all synchronous
validation and process construction have passed. This also fixes the route's
in-flight-chat loading branch, since that teardown runs inside the same hook.
Add test_hook_skipped_when_gpu_selection_rejects.
* Studio: recompute GPU auto-selection after the before_spawn VRAM hook
Codex P2: with before_spawn moved after prepare_gpu_selection, placement was
frozen against the pre-teardown VRAM state while the hook freed export/chat
afterward. Auto-selection could pin training onto a GPU the hook then cleared
(or onto a kept chat model). Split validation from placement: explicit gpu_ids
are still validated before the hook (raise -> 400, no teardown; explicit
placement is VRAM-independent), but VRAM-dependent auto-selection now runs
after the hook so it sees the freed memory.
Add test_auto_placement_runs_after_hook and test_explicit_placement_validated_before_hook.
* Studio: allow chatting during training (lift sidebar gate + VRAM-aware load guard) (#6335)
* Studio: allow chatting during training (lift sidebar gate + VRAM-aware load guard)
The sidebar disabled New Chat, project, and home navigation while a training
run was active, so users could not chat during training even though the backend
serves inference fine alongside a run. This removes that gate and adds a backend
guard so the one genuinely risky operation, loading a new local chat model
mid-training, is refused with a clear 409 when it would not fit beside the run.
Frontend (app-sidebar.tsx): drop the chatDisabled = isTrainingRunning gate and
its consumers. Navigation triggers no model load on its own, so chat stays
usable during training.
Backend (routes/training_vram.py, routes/inference.py): add
can_load_chat_during_training plus a load/validate guard that sizes the same
effective load the loader performs (LoRA 4-bit to 16-bit resolved first, HF auto
placement via auto_select_gpu_ids, explicit multi-GPU per-GPU floor, GGUF sized
from on-disk shards and companions or the selected remote variant). It is a
no-op when training is inactive, never blocks external providers or
already-resident models, and default-denies only on a CUDA sizing failure so a
load can never OOM the run. Validate refuses early with the real settings so the
frontend does not unload the resident chat model for a load that would be
rejected.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: address review feedback for chat-during-training load guard
- Run the load/validate VRAM guard via asyncio.to_thread so the sync
nvidia-smi + HF metadata work never blocks the event loop.
- Size the GGUF KV cache at the requested context (_estimate_gguf_kv_gb)
and add it to the local GGUF estimate so large-context picks are not
under-counted.
- Keep the requested quantization when adapter_config.json is malformed
(not a JSON object) instead of raising in _effective_load_in_4bit.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: size the training load guard at the launcher's effective GGUF context
The GGUF KV-cache estimate used max_seq_length only, but the llama.cpp
launcher honors a user --ctx-size/-c in llama_extra_args. A load such as
max_seq_length=4096 with --ctx-size 131072 was sized against a 4k cache
while the server allocates 131k, so the guard could approve a long-context
GGUF load that then OOMs training. Size the guard's KV at the larger of
max_seq_length and the parsed --ctx-size (reusing the launcher's own
parse_ctx_override), keeping the conservative f16 cache so the estimate is
never smaller than what the server allocates.
The chat model picker also validated with the raw max_seq_length while
/load sizes with resolveLoadMaxSeqLength, so validate could pass, unload
the current model, then have /load reject the native-context load. Validate
now uses the same effective context; the load path is unchanged.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: size the GGUF training guard at the server parallel-slot count
The KV-cache estimate assumed a single slot, but llama-server allocates the
cache across --parallel slots (app.state.llama_parallel_slots). On a Studio
launched with --parallel N>1 the guard under-sized the cache N-fold and could
approve a GGUF chat load that then OOMs training. Thread the same slot count
the loader uses into the guard's KV estimate; default 1 leaves single-slot
setups unchanged.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Trim comments for chat-during-training guard
* Studio: keep chat generation alive across navigation; Train spinner + Return to Chat
Hoist the base chat runtime above the routed outlet so navigating to Train (or any tab) no longer aborts an in-flight generation; only an explicit Stop cancels. Add a Train sidebar spinner and swap New Chat to Return to Chat while a run is active, with a lightweight completion watch so the spinner clears from any tab. Also respawn a chat llama-server killed mid-session and guard unreadable HF cache dirs that 500'd the hub model list.
* Studio: show Return to Chat on the Train tab whenever a chat is live
Previously the top sidebar item only swapped to Return to Chat while training was running; on the Train tab with an idle/just-finished run it stayed New Chat, which started a fresh thread and cancelled an in-flight generation. Show Return to Chat (and navigate back, preserving the run) whenever a generation is running or its thread is still active, or training is in progress.
* Studio: keep a running chat alive when starting a New Chat
Starting a New Chat (or switching threads) while a generation was in flight
remounted the single-chat runtime provider, which detached the in-flight run
and cut the previous chat off (it showed up frozen / empty when reopened).
Key the single-chat view by project instead of by thread or new-chat nonce so
the provider stays mounted and assistant-ui switches to a fresh thread in place.
The previous generation keeps streaming in the background and autosaves on
completion, and returning to that thread reattaches the live run instead of
reloading a half-saved one.
Also:
- "Return to Chat" now lands on the thread that is still generating rather than
the empty new chat that became active after New Chat.
- Skip the explicit /inference/cancel POST when an abort comes from a runtime
detach (navigation / background switch) rather than an explicit Stop, so a
backgrounded generation is never cancelled behind the scenes.
* Studio: make model export non-blocking and inline
The Export tab opened a full-screen modal that trapped focus, could not be
closed or cancelled while running, and showed no progress. It also stopped
training and unloaded the chat model before loading, so export could not run
alongside them.
Export now mirrors the training runtime pattern:
- Inline panel embedded where the Export Model button was, with no modal or
backdrop, so the rest of the UI stays usable during an export.
- Global export runtime store plus an app-root lifecycle hook, so a run keeps
going and streaming across navigation and is reflected on the Export nav item
from any tab.
- The worker log stream now stays connected across the load to export phase
boundary instead of stranding on "Waiting for worker output".
- Progress bar driven by phase and quant index (quant N of M for GGUF), with
elapsed time and a working Cancel.
- load-checkpoint no longer stops training or unloads inference; export loads in
its own subprocess in parallel and surfaces out-of-memory as a clear error.
- Add POST /api/export/cancel and is_export_active on /api/export/status.
* Studio: show Return to Chat on the Export tab too
Extend the New Chat to Return to Chat swap to the Export route so leaving a
running chat for Export offers a way back to the live generation, matching the
Train tab.
* Studio: smooth out Export animations and polish the panel
- Drop the height-based reveal animations (source switch, run panel, quant
picker, hub fields) that caused flashing and reflow; use instant swaps and
quick opacity fades instead.
- Method and quant cards now transition colors only, with no transition-all or
hover lift, so selecting a method or quant is crisp instead of jumpy.
- Auto-scroll the export panel into view when it opens and add a scroll-to-bottom
button when its output is below the fold, like Chat.
- Show Return to Chat on the Export tab while an export is running, matching how
training drives it on the Train tab.
- Surface the current phase or stage in the live output before the first worker
line arrives so the panel never looks stuck while progress is advancing.
* Studio: show Return to Chat on every non-chat tab
Generalize the Return to Chat swap from just Train/Export to any non-chat route
(Recipes, Projects, Hub, ...) so a running or active chat is always one click
away, instead of showing New Chat there.
* Studio: stream export logs over the Cloudflare tunnel; drop janky export animations
Exporting over a --secure Cloudflare quick tunnel showed "connecting..." with no
logs while the progress bar advanced. Cloudflare buffers text/event-stream and
only flushes when the stream closes, so the SSE log stream never reached the
browser during the run (direct localhost is unaffected, which is why this only
showed up over the tunnel).
Add a tunnel-safe JSON poll fallback (GET /api/export/logs?since=) that the
runtime lifecycle hook polls while a run is active. Short JSON responses are not
buffered by the proxy, so logs show up in near real time over the tunnel. It
shares the orchestrator's monotonic seq cursor with the SSE stream and the store
de-dupes by seq, so the two transports run together (SSE on localhost, poll over
the tunnel) without double-printing. A successful poll marks the panel
"streaming" instead of leaving it stuck on "connecting...".
Also remove the framer-motion AnimatePresence reveals from the export config and
run panel (quant picker, hub fields, the inline run panel, and the live log
section). The expand/slide animations flashed and felt clunky; the sections now
render in place.
* Studio: recover export over the Cloudflare tunnel when the blocking POST times out (524)
A model export over a --secure Cloudflare quick tunnel showed "Request failed
(524)" even though the export succeeded on the backend (the GGUF was written).
Cloudflare returns 524 when a single request takes longer than ~100s to respond,
and a GGUF conversion routinely runs for minutes, so the blocking per-method
export POST is cut off while the backend keeps going.
Confirm completion via short status polls instead of relying on the long POST
response (the same approach that fixed log streaming):
- The orchestrator records each finished op's outcome (status / output_path /
error) with a monotonic seq, exposed on GET /api/export/status.
- parseJson now preserves the HTTP status; a 524/520/522/523/502/503 or a
status-less network drop is classified as a recoverable transport error.
- runExport wraps each phase (load, every export method, each GGUF quant): on a
recoverable failure it keeps the run alive (logs keep streaming, the panel
shows "reconnecting...") and polls status until the still-running op finishes,
then settles from the recorded result, recovering the output path for the
success banner. A real 4xx still fails immediately; localhost still uses the
fast POST response. applyBackendStatus also settles a reloaded run from the
last-op record.
Verified over the tunnel: a 3m14s gemma-4-E4B-it GGUF export now ends on the
success banner with the output path instead of 524.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: keep the export method + logs visible after navigating away mid-export
While an export was running, navigating to another tab and back to Export
remounted the page and reset the local form state (exportMethod, quant levels),
so the method card showed unselected and the run panel's log area was hidden
until the card was re-clicked. The run itself lives in the global store and was
unaffected.
Seed exportMethod / quantLevels from the active run's summary via lazy useState
initializers on (re)mount, and gate the panel's log area on the live run
(isExporting / logLines / the run's method) rather than only the local form
selection. The card stays selected and the logs/progress stay visible across
navigation; nothing changes when no run is active.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: address export/training review findings
- Export: guard Start against an empty GGUF quant selection so an inline-panel
run with no quant can't settle as success with no file produced.
- Export: thread the source HF token into the background load so gated/private
HF source exports (and gated bases) authenticate, matching the consent path.
- Export: only settle a recovered (non-owned) run as a finished export when the
last backend op was an export, not a standalone load_checkpoint.
- Training: free the export subprocess whenever an export is active, not only
once a checkpoint is loaded, so an in-flight export load can't race training
for VRAM (current_checkpoint is unset during the load phase).
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: polish Bypass permissions toggle and add it to chat menu settings
- Recolor the pill, menu item and settings caption from alarm red to a
bright yellow accent via a new --bypass token (light and dark).
- Use the shield icon on the composer pill (X on hover, like other pills).
- Expand the composer to two rows the moment Bypass permissions is on.
- Keep Bypass leftmost among tool pills; Compare still sits ahead of it.
- Add a Bypass permissions row to Settings > Chat menu so it can be pinned.
- Widen the "More" submenu so the label fits on one line.
* Studio: use shield-ban icon and lighten the Bypass permissions pill
- Swap the lucide shield-off icon for the Hugeicons shield-ban across the
pill, menu item and settings row.
- Lighten the pill background and deepen the yellow text a touch so the
label stays readable on a near-white tint.
* Studio: drop the Bypass permissions resting fill, keep it on hover
Show only the yellow icon and label at rest; the rounded hover pill picks
up the yellow accent like the other toggles.
* Studio: lighten the Bypass permissions hover fill
Drop the hover tint so the yellow label stays readable on hover.
---------
Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local>
Switch the sidebar profile button from rounded-full to rounded-[5px] so its hover background is a rounded rectangle instead of a pill. The collapsed icon-only state stays circular. Styling only.
* Studio: scale export GGUF size estimates from the real model size
The Export page showed hardcoded, model-independent GGUF quant size
labels (Q8_0 ~8.2 GB, BF16 ~14.2 GB, ...) calibrated for an ~8B model.
For a 35B MoE model like Qwen3.6-35B-A3B (67 GiB bf16, Q8 ~34 GiB) the
picker wrongly reported Q8 ~8.2 GB. Only the displayed estimate was
wrong; the actual export via save_pretrained_gguf was always correct.
Add GET /api/models/export-size, which returns a model's MoE-aware
fp16/bf16-equivalent size and total params using the existing
estimate_fp16_model_size_bytes (safetensors -> config -> local -> vllm).
The result is memoized and degrades to nulls so a size hint can never
break the Export page.
The Export picker now scales each quant from that size
(bytes ~= fp16_bytes * bits_per_weight / 16, GiB units to match the
model selector), and renders no size when it is unknown rather than a
misleading fixed number. The Est. size summary in the page and dialog
is restored now that the value comes from the backend.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio export-size: address review feedback
- Run the size estimate off the event loop with asyncio.to_thread so a slow
Hugging Face request cannot stall other API or SSE endpoints.
- Cache only successful estimates; a transient failure (offline, gated before
credentials) is no longer pinned as unavailable until restart.
- Forward the HF token so private and gated models can be sized, and refetch
when the token changes.
- Clamp the size formatter index so sub-1-byte values cannot pick an
out-of-range unit.
* Studio export-size: address second review pass
- Send the HF token in an X-HF-Token header instead of the query string, so
it never lands in URLs, logs, or browser history.
- Key the estimate cache by model id only (the fp16 size is token independent),
so HF tokens are never retained in the cache.
- Restrict local-path sizing to known Studio roots (outputs/exports/cache/home)
so an authenticated caller cannot trigger a scan of an arbitrary directory.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio export-size: fix CI (import-hoist + isolated-load test stubs)
- Import ExportSizeResponse from models.models in routes/models.py instead of
re-exporting it through models/__init__.py, so the import-hoist lint does not
flag a newly added but un-loaded re-export (models/__init__.py is unchanged).
- Add Header and ExportSizeResponse to the stubbed fastapi / models.models in
test_export_absolute_paths.py, which loads routes/models.py in isolation.
* Studio: validate export-size local path before filesystem access
CodeQL flagged the export-size local-path guard as path injection: the
user-provided model path was resolved and stat-ed before it was checked
for containment under a Studio data root. Decide containment by lexical
normalization (normpath/abspath/expanduser, no filesystem access) and
only touch the filesystem once the path is proven to sit under a trusted
root, so an unvalidated value never reaches a filesystem call. Add a
direct containment unit test (under-root, root itself, missing, /etc,
and '..' traversal).
* Studio: trim export-size comments to be more concise
Shorten docstrings and comments on the export-size endpoint, helpers, tests,
and frontend size utilities; drop comments that just restate the code. Verified
code-identical (comments only) via AST/TS-compiler check. No behavior change.
* Studio: harden export-size local-path handling
Address review feedback on the export-size endpoint's local sizing:
- Resolve symlinks and re-verify containment in _is_sizable_local_path so a
symlink inside a Studio root can't point the sizer outside it.
- Re-validate the resolved LoRA base before sizing, so a crafted adapter
whose base_model points outside the roots can't redirect the scan.
- Skip nested checkpoint-*/global_step* snapshots when summing local weight
sizes so a run dir's intermediate checkpoints don't inflate the estimate.
- Size the checkpoint directory for full fine-tune checkpoint exports (whose
base may be a local/custom path), keeping base-model sizing for adapters.
Adds tests for the adapter-base escape, symlink escape, and nested-checkpoint
exclusion.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
* Harden model fetching: consent gate for trust_remote_code
Add a load-path consent gate that scans a model's auto_map repository code
before it executes and blocks CRITICAL/HIGH findings unless the user pins
approval of that exact code version. Capability detection stays code-free,
reading raw config.json instead of AutoConfig.
- Scan config.json and tokenizer_config.json auto_map, nested local helpers,
and external owner/name--module repos; fail closed on partial downloads.
- Gate inference, training, and export workers, including the MLX path and a
LoRA's base model, and report requires_trust_remote_code from the raw config
so chat and auto-load surface the dialog.
- Verify trusted-org auto-enable against the Hub with the request token and key
the verdict cache by token; reject local-path and spoofed names.
- Add a consent dialog showing the flagged file, line, and surrounding code.
- Thread hf_token through the scan and load paths for gated repos.
* Address review: token handling, tokenizer/LoRA scan coverage, rollback
- Send the HF token for remote-code scans in the POST body, not the URL, so it
never lands in a log or browser history.
- Collect tokenizer_config.json auto_map files directly instead of relying only
on the repo file listing.
- Resolve a LoRA's base model for the validate flag and the scan endpoint so the
dialog scans the code the workers actually gate.
- Pass the request token to the training YAML trusted-org auto-enable.
- Resend a previously approved fingerprint when rolling back to a custom-code
model after a failed switch.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Consent UX: drop legacy chat toggle, fix decline copy, purge declined downloads
The per-model consent dialog is now the single approval path for custom
(auto_map) code in chat, so three leftovers from before it existed are removed:
- Remove the "Enable custom code" switch from Chat Settings and stop persisting
trust_remote_code, so a previously saved blanket-on cannot linger and load a
model without going through per-version review. The flag stays as an internal
YAML/preset default (e.g. first-party auto-enable); the load path still gates
every custom-code load on a fingerprint only the dialog produces.
- Reword the decline message and the auto-load toast to describe approving the
model's code from the dialog, not a missing settings toggle.
- On decline, purge the repo the scan downloaded so untrusted code is not left
on disk. A new /api/models/discard-remote-code endpoint deletes only a
metadata-only cache entry the scan created; it refuses local paths, loaded
models, and any repo with weight files cached, so a model the user already had
or pre-downloaded is always left untouched. The frontend only calls it when
the scan reported created_by_scan.
Adds discard-endpoint tests (delete metadata-only, refuse on weights/gguf,
refuse local, no-op when not cached) and a created_by_scan payload assertion.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Export: remove the user-facing trust remote code toggle
The Export page kept a "Trust remote code" switch (default on) next to the HF
token field. Like chat, custom (auto_map) code should be approved per model
through the load-time review dialog, not a persistent blanket switch, so the
toggle is removed. The export load path already routes through the same consent
dialog: an HF source now starts with trust_remote_code off and only enables it
when the user approves the scanned code in the dialog (a local checkpoint the
user exported stays trusted by default). With the dialog unreachable and no
approval, an HF source loads with trust_remote_code off, which fails closed
rather than running unreviewed code.
* Block loads of repos with unsafe files using Hugging Face's security scan
The trust_remote_code consent gate covers one load-time RCE vector (a repo's
auto_map Python). It does not cover the other: a malicious pickle inside a weight
file (pytorch_model.bin, *.pkl, *.dat) deserializes during from_pretrained even
with trust_remote_code False, so a repo with a normal config plus a poisoned
pickle slips past the existing gate.
Add a metadata-only malware gate that uses Hugging Face's own scan (picklescan +
ClamAV), read via model_info(securityStatus=True).security_repo_status. It never
downloads, opens, or unpickles the flagged files; it only reads the Hub's verdict
and surfaces the flagged file names. New evaluate_file_security runs
unconditionally (independent of trust_remote_code) in every load path (inference,
training SFT/MLX, export), blocking the load when a file is flagged
unsafe/suspicious/malicious. The /remote-code-scan preflight and the validate
endpoint also report the result so the consent dialog opens as a hard block (no
override) listing the flagged files, even for a repo with no custom code.
Policy: hard block with no user override; fail open when the scan is unavailable
(offline/unscanned) so legitimate loads are not broken; no first-party exemption
(a poisoned pickle in a compromised trusted repo still blocks); local paths and
GGUF are skipped (no Hub scan, non-pickle format). Blocking does not gate on
scansDone, since that is often false for clean repos and a file already flagged
unsafe is unsafe regardless.
Adds test_file_security.py covering the block/allow/fail-open/skip matrix.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: scan list-form tokenizer auto_map, gate unsafe files on all load paths
Fixes from a 10-reviewer pass on the model-fetching hardening:
- The remote-code scanner skipped tokenizer auto_map encoded as a [slow, fast]
list (transformers' standard tokenizer shape, e.g.
{"AutoTokenizer": ["owner/repo--tokenization_x.Slow", null]}). External
tokenizer code in that form was never fetched, scanned, or fingerprinted, so an
AutoTokenizer(trust_remote_code=True) load could run it. _auto_map_refs now
flattens string, list, and nested values. Adds a regression test.
- Compare-mode chat loads and background auto-load only gated on
requires_trust_remote_code, so a repo flagged unsafe by the Hub scan but with no
custom code skipped the hard-block dialog. Both now also gate on
requires_security_review, matching the main chat path.
- The /remote-code-scan and /validate routes collapsed a LoRA adapter to its base
before the malware scan, so unsafe files in the adapter repo itself were missed
in the pre-load review (the workers already scan both). Both routes now run the
file-security scan over the adapter and the base.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Require approval for all HIGH remote code, fail closed when unscannable
Tighten the load-time security gates based on review:
Consent gate
- HIGH-severity auto_map code now requires explicit, per-version approval for
every repo, including first-party unsloth/nvidia. The org is no longer a
blanket bypass: a compromised first-party repo with HIGH code still warrants
review. CRITICAL stays a hard block; clean code still loads after the consent
prompt.
- Fail closed when auto_map code is present but cannot be fully fetched or
listed to scan (gated, offline, transient, or a repo-listing failure that
could hide an imported helper). We cannot fingerprint code we cannot see, so
this is a non-approvable block, retryable once the repo is reachable.
- Scan auto_map from every config that can carry one (model, tokenizer, image
and feature processor, processor, video processor), not just config.json and
tokenizer_config.json, so a custom-processor model is not missed. The file
list is the single source of truth in remote_code_scan and is pinned to the
transformers filename constants by a guard test.
- Distinguish a genuine 404 (config truly absent) from a transient error: only
the latter forces a scan, so a repo with no config is correctly a no-op.
Malware gate
- Scan a remote repo even when its name ends in .gguf; only local paths skip the
Hub scan, so a repo cannot dodge the scan by naming itself "*.gguf".
- Correct the docstring: a file already flagged unsafe blocks regardless of
scansDone; the only fail-open path is an unavailable scan.
Coverage
- Resolve a remote LoRA adapter's base model (not just local directories) so the
base, where the code and weights actually execute, is scanned in validate,
the scan route, and the training and export workers.
- Gate the embedding training path (FastSentenceTransformer) with the malware
and consent checks, matching the other load paths.
Tests updated and added for each change.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Scope malware gate to the load-path vector; stop false-blocking first-party models
Follow-up hardening from a second review pass + a broad live model matrix
(unsloth/* , nvidia/* , third-party, and the eicar malware repo).
Malware / unsafe-file gate
- Scope the block to the actual RCE vector: a root-level file in a code-executing
format. from_pretrained deserializes weight files at the repo ROOT, so a flag is
only a load-path pickle vector there. Two exclusions, because neither is loaded:
inert formats (safetensors is tensor-only, gguf is non-pickle, configs/text/
images) and files in subdirectories. This keeps eicar blocked (its *.pkl/*.dat/
eicar_test_file sit at the repo root) while no longer false-blocking legitimate
first-party repos: nvidia/Nemotron-H-8B-Base-8K ships root safetensors plus NeMo
pickle checkpoints under nemo/ that the loader never touches, and the Hub flags
both; the gate previously hard-blocked it.
- Unknown / future non-"safe" levels now fail closed (block) instead of being
silently allowed, so Hub schema drift cannot introduce a bypass; in-progress
("pending"/"scanning"/"error") levels stay non-blocking to avoid false blocks.
Consent gate
- Ignore a STALE own-repo auto_map target that is absent from the repo listing (an
older config pointing at a file the repo no longer ships) instead of failing the
whole repo closed as unscannable. The present .py are still fully scanned, which
is the stronger coverage, and a file that is not there cannot execute. This
unblocks first-party models like unsloth/PaddleOCR-VL (its tokenizer_config.json
names processing_ppocrvl.py while the repo ships processing_paddleocr_vl.py). A
referenced .py that IS present but cannot be fetched, and a repo-listing failure,
still fail closed.
Remote LoRA base resolution
- Distinguish a genuine 404 (not a LoRA / repo absent -> None) from a transient
error: the transient case is retried once, then logged as a WARNING (a missed
base is scanned by neither gate) rather than silently skipped.
Discard endpoint
- Treat .onnx and .ckpt as weights so a repo whose only heavy artifact is one of
those is never eligible for the declined-download purge.
Tests added for each: load-path scoping (safetensors/subdir/Nemotron-H shapes,
unknown-level fail-closed, pending non-block), stale own-repo auto_map ref, remote
LoRA transient retry, and the empty-config-list (all-404 -> []) semantics.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Make LoRA-base transient-warning test robust to logging backend
Assert on the logger object directly instead of capsys, so the test does not
depend on whether the real structlog logger or the module-stub logger is active
(which varies with test collection order).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Allow a repo with auto_map but no executable code (e.g. GGUF) instead of blocking
A config can declare an auto_map yet the repo ship NO executable .py -- most
commonly a GGUF repo whose config.json carries an auto_map copied from the original
model (e.g. unsloth/Llama-3_1-Nemotron-Ultra-253B-v1-GGUF references
modeling_decilm.py, which the GGUF-only repo does not contain). A GGUF model loads
through llama.cpp, which never executes auto_map, and transformers cannot run a file
that is not present, so there is nothing to scan and trust_remote_code is a no-op.
The fail-closed change treated this empty result the same as "code is present but we
could not fetch it" and hard-blocked the load. Distinguish the two: repo_remote_code_files
now RAISES RemoteCodeUnscannable when code is present but cannot be fully fetched or
listed (offline / gated / transient / a present .py that 404s / a listing failure),
and returns an empty dict only when the listing succeeded and the repo genuinely ships
no executable .py. The consent gate blocks on the exception (fail closed) and allows the
empty case as a no-op. Real unscannable code still hard-blocks; eicar and CRITICAL/HIGH
custom code are unaffected.
Verified against all 37 unsloth/*Nemotron* models (two GGUF repos were false-blocked,
now load) and the existing matrix (eicar still blocks; DeepSeek-OCR / NVLM-D-72B still
prompt approvable consent). Tests updated to expect the raise for unscannable cases and
added for the no-executable-code no-op.
* Ignore vestigial auto_map in GGUF repos (llama.cpp never runs it)
A GGUF repo's config.json is often copied verbatim from the original
transformers model, auto_map and all, but a GGUF load goes through
llama.cpp which never executes auto_map, so the config is inert. Treat
a direct .gguf reference, and a repo that ships .gguf weights with no
.safetensors, as having no remote code so the consent flow is never
triggered. A mixed repo with both .gguf and .safetensors is still gated,
since the safetensors variant would load through transformers where
auto_map does run. The check sits behind the existing auto_map-present
gate so normal models pay no extra repo listing.
* Add scanner-result copy to the remote-code consent dialog
Make the consent dialog state the scan outcome in plain language for
every model. When the static scan finds nothing, reassure the user with
'Our automatic scanner did not flag any worrying files, but please
double check.' (shown only for the clean, approvable case). When the
scan flags custom code or unsafe files, label the list with 'Our
automatic scanner flagged issues including:'. The Hugging Face
attribution for unsafe files stays in the dialog description.
* Close GGUF-suffix consent bypass for repo ids ending in .gguf
The .gguf short-circuit in _config_has_auto_map skipped the scan for any
model name ending in .gguf, including a bare two-segment repo id like
'evil/model.gguf'. Such a repo can still ship safetensors plus auto_map
Python that transformers would execute, so skipping the scan was an
asymmetric bypass (file_security already scans those repos). Restrict the
short-circuit to genuine direct GGUF file references via
_is_direct_gguf_file_ref: a local .gguf path, or a remote repo_id plus
filename (three or more segments). A two-segment repo id named *.gguf now
falls through to the config scan and _is_gguf_repo file inspection, so it
only skips consent when it actually ships .gguf weights and no safetensors.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Align consent dialog body with the title and fix narrow-width overflow
The scan results (the 'Our automatic scanner...' label, finding/unsafe
cards, and the clean-scan reassurance) sat at the dialog's left padding
while the title and description were indented past the status icon, so
the body did not line up under the description. Move the title,
description and results into one column to the right of the icon so they
share a left edge, and let that column fill its width so the description
no longer wraps early.
Also stop a wide code snippet from pushing the dialog off-screen on
narrow viewports: AlertDialogHeader is a grid with place-items-center,
which sized the content row to its content; give the row w-full so it
fills the track, and add min-w-0 down the results chain so the snippet
scrolls inside its card instead of widening the dialog. Verified aligned
and contained from mobile portrait through ultrawide.
* Treat a repo as GGUF-only only when it ships no transformers weights
_is_gguf_repo excluded only .safetensors, so a repo with a .gguf and a
pytorch_model.bin (or .pt/.pth/.h5/.msgpack/.onnx/.ckpt) and no
safetensors was treated as GGUF-only and skipped the consent scan, even
though transformers can load that weight set and execute the repo's
auto_map code. Require the absence of ANY transformers-loadable weight
before treating the repo as a llama.cpp-only GGUF load. A genuine
GGUF-only repo (only .gguf) is still inert; a mixed repo with any pickle
or safetensors weight is gated. Adds a regression test across all the
non-safetensors weight formats.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Block flagged subdir weight shards referenced by a root index
The malware gate treated every subdirectory file as non-loadable, but
from_pretrained deserializes a subdir shard a root index references
(pytorch_model.bin.index.json -> shards/...-00001-of-00002.bin). Read the
root weight indexes and block a flagged subdir pickle the weight_map
points at; a flagged subdir pickle no index lists (NeMo nemo/*.distcp)
stays non-blocking, and an inconclusive index lookup fails closed.
* Pass hf_token to the export checkpoint load
ExportBackend.load_checkpoint scanned with hf_token in the worker but
loaded the weights unauthenticated, so a gated/private checkpoint passed
preflight then 401'd at from_pretrained. Add hf_token to load_checkpoint
and forward token to every from_pretrained branch; the worker passes the
command's hf_token.
* Scope created_by_scan to every HF cache the discard searches
created_by_scan used get_cache_path (active HF_HUB_CACHE only) while
/discard-remote-code deletes across active, legacy, and default caches. A
repo the user already had in a legacy/default cache was marked
scan-created and deleted on decline. Check all three caches for the repo
dir before declaring the scan created it.
* Scan the full .py closure of external auto_map repos
An auto_map cross-repo ref (owner/name--module.Class) only had its entry
file downloaded, but transformers also fetches that file's relative
imports from the same repo, so a dangerous helper.py was left outside the
scanned fingerprint. List each external repo's .py and scan the whole set
(plus the referenced entry files); fail closed if the repo cannot be
listed or fetched.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fail closed when a weight index cannot be fully read
_indexed_shard_paths treated a partial result as definitive: if one weight
index read cleanly but another failed transiently, it returned the shard
paths it did see. A flagged subdirectory pickle listed only by the index we
could not read would then be classed as "not a load input" and skipped,
re-opening the very fail-open this guard was added to close.
Return None whenever any index read is inconclusive, even if another read
cleanly, so the caller blocks the already-flagged subdir pickle. A repo that
ships no index files raises EntryNotFoundError for each (never inconclusive)
and still returns an empty set.
* Match cached repos case-insensitively in the created_by_scan guard
_repo_in_any_hf_cache resolved casing only against the active cache and then
probed every cache with an exact directory name. A case-variant already
present in a legacy or default cache (models--Unsloth--Foo for a scan of
unsloth/foo) was missed, so the repo was marked created_by_scan and deleted
on decline -- but discard_remote_code_download deletes case-insensitively,
so that delete would hit the user's pre-existing cache entry. Detect
case-insensitively too, mirroring the deletion path.
* Skip remote-code and security review for selected GGUF variants
validate_model ran the trust_remote_code and Hugging Face security-scan
preflight against the repo even when the selected artifact is a .gguf. A
GGUF loads through llama.cpp, which never executes the repo's auto_map
Python and never deserializes root pickle weights, so repo-level Transformers
artifacts (a config.json with auto_map, or an unsafe pytorch_model.bin next
to the .gguf in a mixed repo) are inert for that load. Gating the GGUF on
them is a false positive. Run both preflights only for non-GGUF loads.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Scope the malware gate to actual load roots and serialized files
Two fixes to evaluate_file_security so it neither misses a load-path pickle nor
false-blocks an inert file:
- Honor subdirectory load roots. Spark-TTS / BiCodec call from_pretrained on the
snapshot's LLM subdirectory, so a flagged pickle directly under it is a
root-level load artifact there. A new load_subdirs parameter (set from the
model's audio type via security_load_subdirs) reclassifies those files relative
to the load root and looks for weight indexes under it, so a flagged shard in
that subdir is no longer skipped as "not root-level".
- Exempt source files. A root .py is never deserialized by from_pretrained;
executable repo code runs only through auto_map, which the remote-code consent
gate scans. Flagging a Python helper here would false-block a repo that merely
ships a build or train script.
* Scan a LoRA adapter and base as one consent unit, and gate MEDIUM code
A LoRA load runs both the adapter's and the base's repo code. The consent gate
scanned them separately and pinned one fingerprint per repo, so an adapter that
shipped its own auto_map code was either never shown in the dialog (which only
saw the base) or impossible to approve with the base's fingerprint.
evaluate_remote_code_consent_for_targets now scans all of a load's repos as a
single combined unit and pins ONE fingerprint over the union of their code, so
approving the load approves every repo's code together. evaluate_remote_code_consent
becomes a thin single-target wrapper, and an unscannable target fails the whole
load closed.
Also gate MEDIUM findings: like HIGH they now block pending pinned approval, so a
direct API caller cannot run flagged code by setting trust_remote_code=True
without consenting. Only a clean scan loads without a fingerprint.
* Preflight a LoRA load's adapter and base as one combined consent scan
scan_model_remote_code rewrote a LoRA adapter to its base and scanned only the
base for remote code, so the dialog never surfaced an adapter's own auto_map
code. Scan the adapter and base together through
preflight_remote_code_consent_for_targets, which pins one combined fingerprint
the worker gate accepts. The malware preflight is also scoped to each target's
load subdirectories.
* Apply combined consent and subdir-aware malware scan in load workers
Each load worker (inference, export, training) evaluated remote-code consent
once per target with a single shared fingerprint, so a LoRA adapter that ships
its own auto_map code could not be approved by the base's fingerprint. They now
scan the adapter and base together via evaluate_remote_code_consent_for_targets,
which pins one combined fingerprint over the union of their code. The malware
scan in each worker is also scoped to the model's load subdirectories so a
flagged pickle under a from_pretrained load subdir is not missed.
* Report a consistent trust_remote_code requirement after a model loads
validate_model reports requires_trust_remote_code from the YAML default OR the
raw auto_map, but the load, already-loaded, and status responses reported only
the YAML default. A custom-code model approved and loaded via auto_map was then
reported as not requiring trust_remote_code, so the frontend stored false and a
later retry or rollback sent trust_remote_code=false and failed.
A shared resolver reports the same requirement for a loaded model (a value
stored at load time, else the trust_remote_code the load used, else the YAML
default, else the raw auto_map check), and the load response persists it so the
status and already-loaded paths stay consistent. The selected-GGUF security
review is also scoped to the model's load subdirectories.
* Run the consent gate on training resume and for YAML-only trust_remote_code
Three frontend gaps left a model loading without the trust_remote_code it needs:
- The shared consent helper returned early when the scan found no auto_map and no
unsafe files, dropping a requirement that comes from a model's Studio YAML
default (e.g. GLM-4.7-Flash). It now grants the caller's requirement with an
empty pin instead of sending trust_remote_code=false.
- Resume-from-history called startTraining directly with no consent gate, so a
resumed run whose model needs custom code (or an old run with no approved
fingerprint) hit the worker block with no dialog. It now runs the same gate as
a fresh start.
- HF export passed requiresTrustRemoteCode=false for every HF source, so a
YAML-only model could not flip the flag before export. It now signals the
requirement for HF sources.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Cover both LoRA repos in validate, report GGUF as inert, purge all declined repos
Three follow-on gaps from the combined adapter+base consent work:
- validate_model resolved requires_trust_remote_code from the base alone, so a
LoRA adapter that ships its OWN auto_map code (with a plain base) was reported
as not needing trust_remote_code and the consent dialog never opened. It now
checks the [adapter, base] target set, matching the scan route and the workers
(which already gate both) and the security review already running over both.
- The already-loaded, loaded, and status responses for a selected GGUF reported
requires_trust_remote_code from the model's YAML default. A GGUF loads through
llama.cpp, which never executes the repo's auto_map Python, so the requirement
is inert for that load. They now report False, matching validate_model (which
already skips both gates for GGUF) so a status refresh cannot flip the flag
back on.
- The remote-code scan downloads both the adapter's and the base's config, but
created_by_scan tracked only the primary, so a base the scan was first to pull
into the cache was left on disk when the user declined. The scan now reports
scan_created_repos (every repo it newly cached) and the decline cleanup purges
each; created_by_scan stays for older clients. The frontend falls back to the
primary flag when the list is absent.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Scan the repo the load fetches, purge external code on decline, harden consent pins
Six follow-on hardening fixes from a fresh review pass over the gate:
- The malware gate scanned the literal "Spark-TTS-0.5B/LLM" alias, but the trainer
downloads it as unsloth/Spark-TTS-0.5B and loads LLM/, so the alias 404'd and
failed open, missing a flagged LLM/ pickle. evaluate_file_security now resolves
the alias to the repo the loader fetches and scans LLM/ as a load root.
- security_load_subdirs relied only on tokenizer detection, which fails on an
unresolved alias or offline; it now also honors the Studio YAML audio_type
default, so a BiCodec LLM/ load root is not missed.
- The remote-code scan downloads external auto_map repos (owner/name--module.Class),
but the decline cleanup tracked only the model/adapter/base, leaving the external
untrusted code cached. The scan now enumerates external auto_map repos and reports
the ones it created in scan_created_repos, so a decline purges them too.
- External auto_map refs failed the whole load closed on a stale or mis-derived
dotted ref (sub.mod.py vs the real sub/mod.py) even though the actual file was
present and scanned. They now drop such refs when the repo listing is real, exactly
like the own-repo path; an empty/incomplete listing still fetches and fails closed.
- The combined consent fingerprint keyed code by the raw target string, so the scan
endpoint's canonicalized casing and a worker's raw user input produced different
pins for identical code, rejecting a valid approval. Hub repo ids are now folded to
lowercase in the key (local paths stay case-sensitive), so the pin tracks the code.
- Export threaded hf_token into the weight load but not into detect_audio_type /
is_vision_model, so a gated multimodal base 404'd in detection and fell through to
the text loader. Both probes now use the same token.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Thread the token through check-vision and guard the gate's parallel sites
The /check-vision endpoint classified a model without the hf_token, so a gated or
private vision model 404'd in the probe and was reported as a plain text model --
the same dropped-token shape as the export probes, at a sibling site. It now passes
the token like the neighboring /check-embedding endpoint.
Add deterministic consistency guards (tests/test_security_gate_consistency.py) that
enumerate the gate's parallel sites mechanically instead of relying on a review to
spot a missed sibling: every is_vision_model / is_embedding_model / detect_audio_type
caller under routes/ and core/ must thread the token, every GGUF response must report
trust_remote_code via the resolver or False (never the raw YAML default), and every
load worker that runs the malware or consent gate must resolve the LoRA base. A new
site that drops the token or mis-reports the requirement now fails CI directly.
* Narrow the LLM alias rewrite and make audio detection token-aware
Three fixes from the confirmatory review, one a regression from the previous round:
- _load_scan_target rewrote EVERY remote repo ending in "/LLM" to unsloth/<parent>,
so a real third-party repo named "<owner>/LLM" was scanned as unsloth/<owner>
while the loader still fetched the real repo -- a fail-open hole introduced when
the Spark-TTS alias handling was added. It now rewrites only a registry-known
bicodec alias; every other "/LLM" repo is scanned as itself.
- detect_audio_type cached results under the bare model name, so an unauthenticated
probe of a gated/private repo cached None and poisoned a later authenticated call
with the token. The cache is now keyed by (normalized_name, token_fingerprint),
matching the vision cache.
- The training fallback /check-vision call dropped the hf_token, misclassifying a
gated/private VLM when the config endpoint failed. It now passes the token, like
the getModelConfig call it falls back from; checkEmbeddingModel takes the token too.
Extend the consistency guards: every capability cache must be keyed by a tuple
including the token, so a cache re-declared as Dict[str, ...] fails CI.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Document the broad .py scan as deliberate and enforce it with a test
The remote-code scanner scans every .py in a repo once an auto_map exists, not
just the auto_map entry's static import closure. This is intentional: the entry
module can reach a sibling via an absolute import, importlib, or exec, none of
which a static relative-import closure follows, so closure-only scanning would be
a real bypass of a load-time RCE gate. The broad scan never under-scans; the cost
is that an unrelated benign script can over-block, which is the safe failure
direction (HIGH stays approvable; only CRITICAL hard-blocks).
Spell this out at both the local and remote scan sites so the choice reads as
deliberate, and add a test asserting an unrelated, never-imported .py is still
scanned -- so a future narrowing to the static closure fails CI.
* Purge a declined remote LoRA adapter the scan downloaded
scan_model_remote_code probed the created-by-scan state AFTER resolving the base,
but get_base_model_from_lora_identifier downloads a remote adapter's own
adapter_config.json, so the adapter looked already-cached and was dropped from
scan_created_repos. On decline the adapter -- including the auto_map .py the
preflight fetched -- was left on disk, defeating the "untrusted code is not left
on disk" guarantee for the adapter itself.
Snapshot the primary's cache state BEFORE base resolution and use it when marking
the adapter scan-created; on any probe error treat it as pre-existing so a decline
never deletes it. The base and external repos are unaffected (their configs are not
downloaded before their own probe). Add a test that models the mid-scan download
side effect, which the prior static-stub tests did not.
* Clear remote-code approval when the training model changes
Switching the training model from an approved custom-code model to a clean one
kept the previous model's trust_remote_code=true and approved fingerprint in the
store: setSelectedModel reset visionImageSize on a true switch but not the
remote-code approval. The clean model then trained with trust_remote_code=true,
which bypasses the compiler and disables fused cross-entropy.
Reset trustRemoteCode and approvedRemoteCodeFingerprint on a true model switch.
The new model's own YAML default is re-applied by loadAndApplyModelDefaults, and a
custom-code model still re-opens the consent dialog before training starts, so the
only change is that a clean model no longer inherits a stale approval.
* Trim verbose comments across the model-fetching hardening changes
Condense the explanatory comments and docstrings introduced across the
trust_remote_code consent gate, the malware/unsafe-file gate, the remote-code
scanner, the load workers, the model routes, and the security frontend into
fewer, tighter lines while preserving every security rationale (fail-open vs
fail-closed direction, the deliberate broad-scan anti-bypass note, the
empty-vs-unscannable distinction, stale-ref handling, and the alias-rewrite
spoof guard).
Comments and docstrings only. No code, logic, identifiers, or test behaviour
changed; verified comment-only via the AST/TypeScript checker (40/40), with the
backend test suite and frontend tsc green.
* Do not cache transient audio-detection failures
detect_audio_type cached _detect_audio_from_tokenizer's result
unconditionally, so a transient read failure (network error or 5xx,
returned as None) poisoned the cache and the later successful probe never
ran. Mirror the vision cache: _detect_audio_from_tokenizer now returns
(audio_type, definitive) and the caller caches only definitive results.
A read that succeeds with no audio tokens, or clean 404s for every
tokenizer path, stays a cacheable None; only a genuine transient failure
(connection error, timeout, 5xx, malformed body) skips the cache so the
next call retries.
---------
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio Hub: default downloads to Xet transport
Model and dataset downloads defaulted to HTTP; flip the default to Xet for faster parallel chunked transfers.
- Frontend: DEFAULT_TRANSPORT_MODE is now Xet, so a user with no saved preference starts on Xet. effectiveTransportMode() already downgrades to HTTP and warns when hf_xet is unavailable, so this degrades gracefully.
- Backend: DownloadModelRequest.use_xet and DownloadDatasetRequest.use_xet default to True, keeping the API in step with the UI. Set use_xet=False for sequential HTTP Range-resume.
- Align the internal _spawn_download_worker default so no caller silently falls back to HTTP.
Inference and training model loads were already Xet-first with an HTTP stall fallback, so this brings explicit downloads in line with the rest of Studio.
* Studio Hub: gracefully fall back to HTTP when Xet is unavailable
With Xet now the default, an omitted or explicit use_xet=True from a non-UI API caller would 400 on installs without hf_xet, since resolve_transport raises when the transport is unavailable.
Add resolve_effective_use_xet(), which downgrades a Xet request to HTTP (with a warning) when hf_xet is missing, mirroring the frontend's own downgrade. Both the model and dataset flows now derive a single effective use_xet and feed it to resolve_transport and spawn_worker, so the recorded transport and the worker env can never disagree. The UI is unaffected: it already resolves availability and passes use_xet explicitly.
* Add tests for resolve_effective_use_xet Xet to HTTP fallback
* Trim comments for PR #6433
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Studio Hub: show Google logo for diffusiongemma and future gemma derivatives
The provider-logo matcher only did a startsWith() prefix match, so qualifier-prefixed family names like diffusiongemma-* never matched Google's gemma- prefixes and fell back to the owner-initial (Unsloth U) tile.
Add an optional per-provider stems list: a case-insensitive substring fallback that runs only after every prefix misses, so prefix precedence (e.g. DeepSeek-R1-Distill- over Qwen) is preserved. Google gets stems: [gemma], which future-proofs the whole *gemma family (paligemma, codegemma, diffusiongemma, etc.) without enumerating each one.
* Studio Hub: match gemma stem only at a word boundary
Address review: a plain substring stem could over-match contrived names like gemmafy or gemman. Require the stem to end at a word boundary (next char not a letter), so gemma still catches diffusiongemma- and gemma-3n but never gemmafy. Splitting on delimiters would not work here, since the qualifier and family share one token (diffusiongemma).
* Trim comments for PR #6432
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Studio: make sidebar nav hover a fully rounded pill
The single-line sidebar rows used rounded-[14px], so the hover and active highlight was a soft rounded rectangle. Switch the main nav, the recents/projects rows, and the account row to rounded-full for a fully rounded pill. The multi-line training-history rows keep rounded-[14px], since a full pill on a two-line block rounds the corners too aggressively.
* Studio: drop now-redundant collapsed rounded-full on sidebar buttons
With the base class now rounded-full, the group-data-[collapsible=icon]:!rounded-full override resolves to the same radius, so remove it from the two nav buttons. The avatar keeps its modifier, since its base is not rounded-full.
* Studio: make toast text selectable
Toast messages could not be highlighted or copied. Sonner's per-toast onPointerDown calls setPointerCapture(), which steals the mouse drag and stops the browser from starting a text selection. dismissible:false stops the capture but also disables the close button (same flag gates its onClick).
Wrap the Toaster in a capture-phase pointerdown handler that swallows the event on toast text only, never on its buttons, so text becomes selectable while the close button and auto-dismiss keep working.
* Studio: harden toast pointerdown guard for non-HTML targets
Cast event.target to Element (closest() lives on Element, so SVG icon targets are covered too) and guard that closest is callable before using it.
* Studio: stop the llama.cpp update banner flickering and show the download size
The banner animated in and out with a motion opacity + scale + translate
transition. That transform/opacity transition promotes a GPU compositing
layer whose first and last frame can flash for a moment on real displays,
which reads as a flicker on appear and again on dismiss/snooze. Drop the
animation and render the banner as a plain conditional mount: it appears
and leaves cleanly with nothing to flash.
Also surface the download size. update-status now reports the size of the
prebuilt that Update would fetch (the latest-release asset matching this
host's bundle), and the banner shows it as whole MB next to the no-restart
note, so the cost of the update is clear before clicking.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Show llama.cpp update size for upstream and source-build installs
update_download_size_bytes now accepts the upstream ggml-org ubuntu-/win-
asset suffixes and falls back to the marker's binary_repo, so the size
resolves for CPU/ROCm prebuilts (the fork publish repo only carries the
app-* and macOS bundles). The source-build update path now populates
update_size_bytes from the resolved asset, matching the marker path.
Both fail open to null. Adds regression tests for the upstream and
source-build size lookups and the route field round-trip.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Set ThemeProvider defaultTheme to "system" and enable system preference so the app follows the OS theme. In the theme store, apply the resolved theme to the document immediately on mount so the store is the single source of truth and avoids an initial light flash on fresh origins (e.g. empty localStorage). Minor comment and formatting tweaks in setTheme were also made.
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
* Studio: add 'Load on selection' toggle to configure load options before loading
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: seed staged speculative decoding from the standing default
* Studio: address PR review for load-on-selection staging
* Studio: handle direct GGUF staging and stale-stage edge cases from load-on-selection review
* Studio: cancel replaced staged downloads and keep staged pick on load failure
* Studio: centralize staged-download cancel and guard staged-load restore
* fix: address staged GGUF load review
* fix: honor staged GGUF load metadata
* fix: clarify load-on-selection tooltip
Keep the load-on-selection hint visually anchored to the control and make the on/off behavior explicit without changing the broader deferred-load flow.
* Studio: reset orphaned staged knobs on abandon and cap Max Tokens to staged context
* Studio: remove dead code and cancel staged download when loading a different model
* fix: surface staged model in run settings before deferred load
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: imagineer99 <samleejackson0@gmail.com>
* fix(hub): stop demoting cached gguf variants on mmproj or filename mismatch
- bug: a quant with its bytes on disk was marked not fully downloaded when the API-preferred main filenames did not match or the mmproj adapter was absent
- fix: fall back to the on-disk quant byte signal, the same one inventory uses for on-device, so a present quant is no longer demoted
- broaden mmproj detection to accept any mmproj-looking cached file, not only the API-preferred name
* feat(hub): full-page redesign with trending feed, search, and persisted state
- convert the hub to a full-page view with a new layout, model cards, and a sortable models table
- add a trending/latest/finetune feed with model and section deep-link params validated on the hub route
- add recent searches and rework model and dataset search with pagination and infinite scroll
- persist feed and token state through a dedicated store and persist-storage layer
* fix(hub): browse sorting, deep-link presets, dataset URLs, and persistence
- sort dropdown: drive HF-wide browse across all repos by the chosen sort, respecting format and capability filters
- section deep-link: apply the section preset (format and sort) on refresh and deep-link, not only on click
- dataset detail URL: persist the resource kind so refresh and share resolve datasets correctly
- gguf card: show a Loading state for hub-cache dir-path repos via repo-id match
- active-model CTA: New Chat now actually opens a fresh chat
- persist-storage: fix throttle keying with a Map and dedupe a duplicated util
- transport-toggle: drop redundant controlled-tooltip state
* Studio: polish hub redesign UI and unify segmented tabs
Refinements on top of the hub full-page redesign:
- Unify every segmented control (Discover/On Device, models/datasets,
Unsloth/All, recent/name/size, settings tabs, train dataset source,
profile shape, theme, OS toggle) on one filled-pill design.
- Hub list now loads in larger batches with a shorter fetch interval so
results fill in fast instead of dripping one row at a time.
- Disable remote avatar fetches in list rows and brighten the colored
initial fallbacks so they read clearly without network calls.
- Add a split master-detail view for model lists and make it the default.
- Left-align the split "Showing GGUF models" header with the rows below.
- Round the "Load more" footer box and tidy On Device stats layout.
- Show recent trainings on Recipes and Export, falling back to recent
chats when there is no training history.
* Studio: address hub review comments (filter warning + scrollMargin)
- DiscoverFetchMoreFooter only shows the "results may be hidden by your
filters" note when a filter is actually active, instead of always.
- Use the destructured scrollMargin prop directly in the row transform
rather than reaching into virtualizer.options.scrollMargin.
* Fix/adjust Hub metadata and deep links for PR #6349
* Studio: drop avatar ring in hub split view
The split master-pane rows (discover + on device) added a ring-1 around
the owner avatar that read as a shadow. Remove it so split-view avatars
match the flat avatars elsewhere; grid cards and the full list keep theirs.
* Studio: hub sort + scope as dropdown pills beside view tabs
Recent/Name/Size and Unsloth/All were segmented controls that dropped to
their own row in the narrow split pane. Make each a compact dropdown pill
(HubOptionMenu) that sits in the header actions slot next to the view-mode
tabs in every layout, so split view no longer needs a separate row.
* Studio: align hub list header with the view tabs and rows
- Vertically center the "On device" / "Showing GGUF models" title with the
dropdown pill and view-mode tabs (items-center instead of items-end), so a
short title no longer sits low against the taller tab row.
- Nudge the back chevron 2px further left (-ml-2) so its glyph edge lines up
with the start of the row hover below it.
* Studio: align back chevron tip with the row hover edge
The arrow glyph is inset ~6px inside its centered icon box, so an
edge-aligned button left the visible chevron sitting in from the column.
Pull the button out (-ml-3.5) so the chevron tip lands on the row hover's
left edge instead of floating to its right.
* Studio: unify every bare tick on the shared check mark
Point all plain checkmarks at the canonical @/lib/tick-icon tick (the one
already used in the chat composer and menus), so there is a single tick
across the app:
- Hub: model-inspector, hub-option-menu, path-info-button were importing
the stock hugeicons Tick02Icon; switch them to the shared icon.
- Chat / assistant-ui: artifact-surface, prompt-storage-dialog, reasoning,
tool-ui-python, tool-ui-terminal, tool-ui-code-execution, and the
tool-fallback status map used lucide CheckIcon; render the shared tick
via HugeiconsIcon instead (tool-fallback wraps it to fit its icon map).
The circular CheckmarkCircle success badges are intentionally left as-is.
No bare CheckIcon/stock Tick02Icon references remain; verified the tick
renders in every converted spot via typecheck + build.
* Studio: nudge back chevron 2px right
-ml-3.5 pushed the chevron a touch too far left; -ml-3 sits it just
inside the row hover edge, aligned with the avatars below.
* Studio: search base-model chips across all publishers
Clicking a Base model chip searches the Hub for the upstream repo, which
lives under another publisher (google, meta, etc.). It left ownerScope at
the default "unsloth", so the search hard-restricted to the Unsloth org and
could never surface the base model. Switch the scope to "all" for this action.
* Studio: label the safetensors list header "Safetensors"
The focused list heading showed "Showing Checkpoint ... models" while the
format dropdown labels the same checkpoint filter value "Safetensors". Match
the dropdown so the header reads "Showing Safetensors ... models".
* Studio: simplify the focused list heading to "Models"
Drop the format/capability composition (e.g. "Showing Safetensors Reasoning
models") so the focused list heading just reads "Models" (or "Datasets").
Search keeps its "Results for ..." label.
* Studio: drop the header refresh button to the text baseline
The refresh button sat at the heading's vertical centre. Nudge it down so
it lines up with the bottom of the title text instead.
* Studio: hide redundant "Back to Hub" in the split detail pane
In split view on large screens the master list sits beside the detail, so
the back button is redundant. Hide it there (lg) and reclaim the top space.
It stays on the small-screen overlay and the full-page detail, where the
list is hidden and back is the only way out.
* Studio: match the readme scroll fade to the left column
The detail pane relied on the sticky back-bar's fade, which is now hidden in
split view. Add the same hub-scroll-fade overlay the master list uses so the
readme fades consistently at the top when scrolled. The back-bar, when shown,
sits above and covers it.
* Studio: align Hub refresh button to the heading text bottom
* Studio: nudge Hub refresh button up to the heading text
* Studio: optically centre the HF token shield in its circle
* Studio: preview the first visible on-device row in split view
* Studio: calm the on-device row colour and fix size tooltip contrast
* Studio: fix Hub reset tab and clear search when opening a section
* Studio: fix Hub feed defaults, filter sync, and GGUF vision download state
* Studio: condense Hub redesign code comments
---------
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: shimmyshimmer <info@unsloth.ai>
Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
* Studio: auto-render fenced HTML in chat replies as canvas cards
After an assistant reply finishes, append a clickable canvas card for each fenced html block in its text, with no render_html tool call and no extra message. Fragments and non-collapsed documents are covered; full documents already collapsed in place and blocks rendered by the render_html tool are skipped so nothing shows twice.
Hoists the fence helpers out of markdown-text.tsx into a shared module (html-fences.ts) and adds a line-based multi-fence scanner so several html blocks in one reply are all found.
* Studio: add HTML Code button to canvas cards and keep diffusion code visible
When Canvas mode is on or a diffusion model is loaded, the canvas card shows a Preview and an HTML Code button side by side; Code opens the panel source view. The requested view is threaded through openArtifact so the surface opens to preview or code.
Diffusion no longer collapses its full HTML answer, so the raw code stays in the message and the trailing canvas card is appended next to it.
* Studio: drop the Code button on diffusion cards since their code is already inline
* Studio: address review feedback on HTML canvas auto-cards
- Build the fence-body indent regex once per fence instead of per line.
- Only skip full-doc fences the in-place collapse can render (plain
unindented triple-backtick), so 4-backtick or indented docs still get a card.
- Scan each text part on its own so a fence cannot stitch across a tool,
source, or reasoning part.
- Exclude diffusion replies from the collapse/skip gates and the card Code
button, since diffusion keeps its HTML inline.
---------
Co-authored-by: Daniel Han <michaelhan2050@gmail.com>
* Add Studio API activity monitor
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Mark disconnected monitor streams cancelled
* Fix monitor lifecycle, parsing, and clock issues for PR #5558
Backend:
- Finalize the monitor entry as error on the chat-completions validation
reject paths (addresses the codex P2 comment around line 2337). Adds a
small _reject helper next to the api_monitor.start so each early-raise
in the GGUF tool-passthrough, non-empty-messages, GGUF vision/audio,
PIL image decode, and text-only model branches no longer leaves the
entry stuck running until eviction.
- _monitor_openai_chunk is now defensive about malformed shapes
(non-list choices, non-dict choice/delta) so a misbehaving provider
chunk does not raise into the streaming generator and abort the
user's response.
- _monitor_openai_sse_line accepts both data:value and data: value per
SSE spec; previously the single-space-only prefix silently dropped
tokens from compact emitters.
- openai_completions stream now keeps a residual buffer between
aiter_bytes chunks so a data: line whose newline lands in the next
TCP frame is reconstructed before parsing, instead of being dropped
by the per-chunk splitlines call.
api_monitor:
- duration_ms is derived from time.monotonic anchors and clamped at 0,
so NTP / manual clock steps no longer produce negative durations.
- finish() and fail() are idempotent: a second call (for example [DONE]
arriving after the generator's finally block already ran) no longer
moves finished_at or finished_monotonic.
- set_usage only derives total_tokens when no authoritative total has
been recorded, so a later partial-usage chunk does not clobber a
provider-reported total from an earlier chunk.
- _trim guards limit < 3 so the slice cannot underflow.
Tests:
- Adds regression coverage for the new idempotency, total preservation,
monotonic clock, and _trim guard behaviour.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Mark monitored stream failures correctly
* Fix completions monitor failures
* Fix responses stream monitor cleanup
* Fix audio input API monitor lifecycle
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix API monitor review follow-ups
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix completions monitor usage accounting
* Fix API monitor cancellation and usage gaps
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Move API monitor into settings
* Fix API monitor review followups
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Lazy load API monitor details
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix API monitor review issues
* Finalize passthrough monitor on clean EOF
* Finalize monitor on chat validation rejects
* Fix API monitor review follow-ups
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix API monitor follow-up review issues
* Monitor embeddings and tool-call replies
* Cover tool-call monitor replies and cancellations
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Scope API monitor entries by subject
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Finalize monitor entries on cancellation
* fix: address PR 5558 CI failures
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: update provider proxy test stub
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: imagineer99 <samleejackson0@gmail.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
* Chat search: match across all messages, not just title and preview
The Cmd/Ctrl+K chat search only matched the thread title and a 120
character preview of the newest message, so keywords anywhere else in a
conversation were unfindable. Build a per-thread haystack from the title
plus every message's text and match against that.
The haystack is lowercased once at index build time, so the keystroke
filter only normalizes the short query instead of re-lowercasing the
whole conversation per item.
* Chat search: search user messages first, expand to all only if no hit
User messages are short while assistant replies can be very long, so
matching the whole conversation on every keystroke scales with the
assistant text. Store a separate userSearchText (title plus user
messages) and match it first, expanding to the full per thread
searchText only when nothing matches user text anywhere. Row filtering
moves into selectVisibleChats with cmdk shouldFilter disabled so the
tier choice is deterministic.
* Chat search: include tool calls and sources in the full-text tier
extractText now also pulls reasoning/thinking, tool call name/args/result
and cited source title/url, so the expanded full-conversation tier finds
keywords that only appear in tool activity. Drop the now-unused preview
field since search no longer matches on it.
* Chat search: drop base64 image/audio payloads from the index
extractText stringified tool-call results wholesale, so an
image_generation result (image_b64) or audio payload would pour megabytes
of base64 into searchText and get lowercased on every rebuild. Add
searchableText, which keeps readable tool args/results (tool name, prompt,
text) but skips binary keys and strips data URLs, long base64 runs and the
__IMAGES__ suffix.
* chore: add Studio version info to General settings
* chore: cache Studio version fetch
* fix: clear cached Studio version request on failure
* fix: retry incomplete Studio version fetch
* fix: remove cached Studio version fetch
* chore: move connections below API
* fix: localize studio version rows
* feat: implement thread forking functionality with associated database updates and UI components
* fix(studio/chat): register fork-count listener even when thread unsaved
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Polish thread fork action menu
* fix-studio-fork-project-test-order
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: imagineer99 <samleejackson0@gmail.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
* feat(studio): expose provider_type selector in model provider dialog
Use the configured provider_type in the model provider payload instead of hardcoding openai. Add a provider type selector to the model provider dialog for the provider types supported by Data Designer.
* feat(studio): add MiniMax-M2.7 inference defaults
Add MiniMax-M2.7 model family entry to inference_defaults.json
with recommended parameters (temperature=1.0, top_p=0.95, top_k=40).
Pattern placed before minimax-m2.5 for correct longest-match-first
ordering.
* Fix/adjust provider type support for PR #4277
* Clarify provider type validation for PR #4277
* Fix provider type default validation for PR #4277
---------
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
Co-authored-by: PR Bot <pr-bot@minimaxi.com>
Daniel Han