* Show release notes in the update popup, sourced from CHANGELOG.md
The update banner only linked out to the online changelog, so there was no
way to see what an update contains before taking it.
Add CHANGELOG.md at the repo root as the source of release notes. Studio
reads it from the default branch, so editing the file updates the popup
without a release or rebuild, and falls back to the copy bundled in the
install when the repo is unreachable.
Notes are matched to one exact version. The popup asks for the version it is
offering and gets that section or nothing, so an older release's notes can
never appear next to a newer update. When there is no match the popup links
out to the online changelog instead.
The collapsed popup previews the top bullets with the leading sentence
highlighted; "Show release notes" expands the full notes in a scrollable
panel. Applies to both the browser and desktop banners, and the desktop
updater's own release body is used when CHANGELOG.md has no matching section.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: fence matching, nested bullets, BOM, updater notes field
Track the opening fence marker and length so a ``` sample inside a ````
block does not close it early and let the sample's heading be indexed as a
real release.
Preserve list indentation in the preview and take only top-level bullets, so
nested detail no longer consumes the four headline slots.
Strip a UTF-8 BOM before parsing. An editor on Windows can leave one on the
first line, which hid a section whose heading started the file.
Read `notes`/`pub_date` from latest.json in the manual Linux updater path,
with aliases for the older `body`/`date`. The workflow publishes Tauri's
field names, so the manual path's release body was always empty. Also loop
the preview tag strip until stable for CodeQL js/incomplete-multi-character
-sanitization; the value renders as text, so this is defence in depth.
* Address review: bare fence closers, HTML comments, underscores, notes URL
A closing fence must carry nothing after the delimiter, so a ```` line with
trailing text inside a ```` block is content rather than the end of it. Both
the parser and the preview extractor follow that rule now.
Skip headings inside HTML comments. A commented-out section is not rendered
by Markdown, so it must not be indexed as a release.
Strip only paired emphasis and park code spans first, so identifiers keep
their underscores: UNSLOTH_DISABLE_UPDATE_CHECK was previewing as
UNSLOTHDISABLEUPDATECHECK.
Prefer the caller's release URL over the API's generic changelog link, so the
desktop fallback points at the release page for the version being offered.
Look at the repo-root CHANGELOG.md before the packaging snapshot, and remove
the snapshot after build.sh, so an edited root file is never shadowed by a
stale copy.
Also nudge the notes container radius from 16px to 14px.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: comparison operators, hidden comments, remote failures
Require a name character after "<" when stripping tags. A bullet reading
"Support Python <3.15 and >3.9" previewed as "Support Python 3.9", because
the operators were consumed as if they were a tag.
Track HTML comments while collecting preview lines. A commented-out bullet
was previewed as a published change even though Markdown never renders it.
Report a remote lookup failure whenever nothing matched. The bundled
changelog cannot know a version newer than the install, so discarding the
error made an offline lookup read as "no notes were published". The hook now
treats a reported failure as its retryable error state.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: code-span delimiters, stale notes, retry past cached failures
Treat an HTML comment delimiter inside inline code as literal. A note reading
"Type `<!--` to begin a comment" put the parser into comment state, so every
release below it was swallowed into the entry above and became unfindable.
Applied to the preview extractor too.
Return no notes while the offered version differs from the fetched one. On
the render where the version changes, the hook still held the previous
release's notes, which the panel would show for a frame.
Let retry bypass a cached remote failure via a refresh flag on the endpoint.
Failures are cached for five minutes, so the visible Retry action could not
recover until the TTL expired. A cached success is still reused, so retries
cannot hammer the remote.
* Address review: CommonMark indentation, desktop release notes link
Allow up to three leading spaces on release headings and fences, and treat
four as indented code. An indented heading was unreachable and its notes were
appended to the release above, while an indented backtick line opened a fence
that swallowed later headings.
Link desktop release notes to the release page for the offered version on
every platform. The existing URL is built only in manual Linux package mode,
so in-app updates on macOS, Windows and AppImage fell back to the generic
changelog. The install button keeps using the manual URL.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: wrapped prose, autolinks, abbreviations in the preview
Accumulate contiguous prose lines into one preview item. A paragraph wrapped
across source lines renders as one block but previewed as three fragments,
which also ate the four-item limit.
Keep Markdown autolinks. <https://example.com/notes> was stripped as if it
were a tag, so "See <https://example.com/notes> for details" previewed as
"See for details".
Do not split the lead sentence at an abbreviation. "Supports several formats,
e.g. GGUF and Safetensors." highlighted only up to "e.g." and dimmed the
actual change; known abbreviations and single initials are skipped now.
* Address review: park code spans first, skip indented code blocks
Park code spans before any other inline transformation. Tags, links, images
and emphasis inside a span are literal, but the strips ran first, so "Use
`<button>` for actions" previewed as "Use for actions".
Skip lines inside an indented code block when collecting bullets. A "- pip
install ..." line in a four-space-indented block became the headline and
pushed out the real prose, though Markdown renders it as code. Continuation
lines of an open bullet are unaffected.
* Studio: skip raw HTML blocks when reading release notes
A <pre>, <script>, <style> or <textarea> block renders literally, so a
sample '## 9.9.9' heading inside one was indexed as a release and cut the
real section's body short. The preview had the same gap and listed sample
bullets as notes.
Both readers now track type 1 HTML blocks and skip their contents. Blocks
open only at the start of a line, so a tag named mid-sentence stays inline
text, and <details> is type 6 so its Markdown still parses.
* Studio: read HTML blocks the way CommonMark renders them
A fence inside a <pre> block was treated as a real fence, so the block's
closing tag was swallowed and every release below it disappeared. Raw HTML
state is now checked before fences, in both readers.
Type 6 and 7 blocks (<details>, <div>, a bare tag on its own line) run to
the next blank line, so a heading pressed against the opening tag is not a
release either. Type 7 cannot interrupt a paragraph, so prose followed by a
bare tag is unaffected.
Checked against a CommonMark reference: 20000 generated well-formed
changelogs now agree exactly on which headings are releases, and every
previewed note is text the renderer really shows.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: restore preview types dropped in the scanner refactor
The previous commit's refactor removed the Bullet and preview item
interfaces, so tsc -b failed and every job that builds the frontend
stopped there.
* Studio: fix release-notes preview and packaging review findings
Preview: a code span now closes on a run of the same length, so a note
containing backticks keeps them; thematic breaks no longer take a preview
slot; a quoted list is example output, so it stays out of the headline
bullets and is only used when a section has none of its own.
Popup: a failed lookup keeps the changelog link beside Retry, which the web
banner always offered before, and the desktop popup waits briefly for the
auto-auth token instead of recording a failure the user has to clear.
Packaging: the changelog snapshot is made by the build backend, so
python -m build, pip install . and sdist builds all ship the offline copy,
not only build.sh.
* Studio: scope the changelog fallback and hide staged sections
Installed, the levels above studio/ are site-packages, so a stray
CHANGELOG.md left there by another package outranked the bundled
snapshot. Those levels are now searched only when a checkout marker
(pyproject.toml or .git) is present, so a source checkout still serves
the editable file.
A section staged as only an HTML comment renders as nothing but was
reported as matched, leaving an empty notes surface. Notes that render
nothing now read as unpublished, so the popup links out instead.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: cover the remaining raw block forms and repository links
Parser and preview: processing instructions, declarations and CDATA are
literal like <pre>, so a sample heading or bullet inside one is no longer
read as a release. ATX headings now need a space or tab after the hashes,
matching CommonMark, so a pasted non-breaking space no longer truncates
the release above it.
Popup: the notes region follows the viewport and the card scrolls as a
backstop, so a window under about 430px high no longer pushes the title
and dismiss control off screen. Relative links in the notes resolve against
the repository instead of Studio's origin, where the renderer blocked them.
* Studio: reference-style images, empty previews and version queries
Reference definitions now resolve against the raw host when the label is
used as an image, so ![alt][arch] loads the file instead of its HTML page
on GitHub. Labels are matched the way CommonMark compares them, and a
reference written inside a fenced block does not count.
Notes that preview as nothing, such as a lone command block, no longer
leave an empty muted strip in the collapsed popup; expanding still shows
them. A version query that cannot parse is rejected up front rather than
looked up and reported as no notes.
* Studio: Markdown scanning fixes across the release notes path
Code spans are now scanned rather than matched by pattern, so a run of
backticks closes only on a run of the same length. The preview and the
link resolver share that scanner, so a link inside `a``b [x](y.md)`
stays literal in both.
Also: a closing fence may carry only spaces or tabs, so a delimiter with a
non-breaking space after it stays code in all three scanners; escaped
parentheses in a link target resolve to the literal path instead of being
mangled; the collapsed preview decodes entities the way the expanded view
renders them, while code spans stay literal; and release notes are fetched
through authFetch so an expired access token is refreshed and retried.
* Changelog: real 2026.7.5 notes, led by the AMD release
Fills the section the popup reads with the actual headline changes, so the
collapsed preview shows real content instead of placeholder notes. Leads with
AMD support and covers the 23 July update: RDNA2 and Gorgon Halo, Strix Halo
detection, RDNA4 and ROCm failure recovery, 2x faster unified memory loading,
whisper.cpp dictation, and rollback environment cleanup.
* Studio: fix release-notes text handling found by adversarial testing
Line endings are normalised first: a CRLF body from the desktop updater no
longer hides fences, so a code sample cannot become a headline bullet, and
lone CR text splits into bullets.
Preview: reference links and images render as their text, a definition line
renders as nothing, parentheses in a destination no longer truncate the
sentence, escaped punctuation stays literal, and a fence indented into a
list item is treated as the block it is.
Links: a badge resolves both its image and its outer link, indented code and
code spans that cross a line are left alone, a definition cannot interrupt a
paragraph, and image alt text no longer decides a label's host.
Also: an escaped backtick cannot open a code span, park sentinels in the
source cannot swap content, two in-flight requests for one version resolve
in order, and repeated bullets no longer share a React key.
Comment scanning no longer rescans code spans per delimiter and span lookup
is a binary search: the worst inputs measured drop from 96ms to 1ms at the
20k cap, and from 544ms to 15ms at 200k.
* Studio: parser and fetch fixes found by adversarial testing
A comment marker written in prose no longer swallows the rest of the file.
Only a comment that starts a line opens a block; one written mid-sentence is
inline HTML and hides its own line at most. This was the worst case found:
a single stray marker made every release below it unreachable and served
their notes under the newer version's heading.
Also in the parser: a closing delimiter takes its whole line, so a heading
glued after it is not a release; an exact heading is never shadowed by a
zero-padded one; setext headings are release boundaries; any heading, rule
or definition ends a paragraph; and the code-span guard is a linear scan
rather than a backtracking pattern, so 20k backticks parse in a millisecond
instead of over a minute.
Fetching: one deadline for the whole response with chunked reads, so a
trickling server cannot hold a worker for minutes, waiters give up instead
of queueing behind a stalled fetch, and identity encoding is requested so a
compressing proxy cannot produce mojibake notes. Truncated notes close an
open fence.
UI: images and the renderer's own link dialog are held inside the card,
which the shared preview's blanket max-width reset had let escape, and only
the notes region scrolls so the dismiss control stays reachable on a short
viewport.
The developer update override no longer beats the documented opt-out, and
its value has to parse as a version.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: CommonMark paragraph and block rules across the notes path
Setext detection now requires plain paragraph text above the underline. A
list item followed by --- is a list and a rule, not a heading: reading it as
one discarded the bullet and every note after it.
A backtick fence whose info string holds a backtick is not a fence, so such
a line no longer swallows the releases below it in the parser, the preview
and the link resolver.
Preview: only an ordered list starting at 1 interrupts a paragraph, an
unresolved reference keeps its brackets, a comment written mid-sentence
hides its own line at most instead of the rest of the document, a raw block
closer takes its whole line, and a code span closer after a backslash still
closes, since escapes do not apply inside a span.
Links: raw HTML blocks are literal, an escaped opener is not a link, and a
definition under a heading is a definition.
The overlay stack is capped to the viewport and both overlays can give up
height, so a long download list no longer pushes the update card's title and
dismiss control off screen.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: desktop notes by backend version, desktop stack cap, fetch budget
latest.json now publishes the backend release the desktop build pins, and
both desktop paths carry it: the manual metadata check through Rust, and the
in-app updater through the raw metadata it already exposes. The popup looks
release notes up by that version, so desktop stops asking CHANGELOG.md for
an app SemVer it never contains and falling back to the generic installer
text. Metadata without the field still parses and behaves as before.
The desktop overlay stack is capped to the viewport like the browser one,
since the download panel shares it and the card's own cap cannot see a
sibling.
The fetch budget now bounds each read, not just the gap between reads. Slow
headers followed by a slow body held a worker for 5.6s against a 3s budget;
it is 3.0s now, and a timeout is reported as one.
* Studio: keep list-nested headings out of the release index
A `## <version>` heading indented to a list item's content column is inside
that item in CommonMark, not a release boundary. Reading it as one truncated
the real release and indexed a version that does not exist.
parse_changelog now tracks the open list items by the column their content
starts at, and only counts a heading left of that column. Supporting rules,
each checked against markdown-it (commonmark preset): a marker needs
whitespace after it, so `2.0` stays a setext version; an item interrupts a
paragraph only when it has content, and an ordered one only when it starts at
1; an empty item takes one blank line; a dedented fence, break or heading
closes the item; and `- ## 2.0` is a heading inside the item.
* Studio: whole-paragraph setext headings, uppercase declarations, escaped marks
Three CommonMark conformance fixes on the notes path, each checked against
markdown-it (commonmark preset).
A setext heading is the whole paragraph above the underline, so a heading that
wraps kept its version only on the first line while the parser read the last:
`2026.7.5 - Release` over `July 25` left that release unindexed and its notes
unreachable. The parser now tracks every line of the open paragraph, including
lazy continuations, and stops at whatever really interrupts it: a quote marker,
a bullet, or an ordered marker starting at 1.
A type 4 HTML block needs an uppercase letter after `<!`, so prose mentioning
`<!note` was hiding every release below it until the next `>`.
In the link resolver, `\![alt][label]` renders as a link, so its definition
resolves to the file's page on GitHub rather than the raw-content host.
* Studio: the preview needs the uppercase declaration rule too
The backend parser stopped treating `<!note` as an HTML block, but the
collapsed preview still did, so prose mentioning one emptied the preview of
every bullet below it while the expanded notes rendered them. A shipped test
now pins the two to the same rule.
* Treat an empty HTML comment as closed and always release the changelog fetch flag
<!--> and <!---> are complete comments in CommonMark: the closer overlaps the
opener, so searching for --> past the opener never found it and the scanner
stayed in comment state for the rest of the file. An empty comment used as a
section marker hid every release below it, in both the backend parser and the
frontend preview.
get_remote_changelog cleared its single-flight flag only after except Exception,
so a BaseException stranded it and every later caller waited out the full
deadline for the life of the process. Move the release into a finally.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Compare resolved changelog paths instead of a hardcoded checkout name
The ordering assertion matched the string suffix /unsloth/CHANGELOG.md, so it
raised StopIteration in any checkout not literally named unsloth, and on
Windows the separator is a backslash so the suffix never matched there either.
Both are unrelated to the ordering under test. Verified failing on
ubuntu-24.04, macos-14-arm64 and windows-2025 alike, and passing after.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Scan backtick runs once instead of rescanning the suffix per opener
Every unmatched opener rescanned the rest of the line and the outer loop then
advanced by a single run, so a line of runs of 1, 2, 3 ... backticks was
quadratic: 321 KB took 7.688s, and release notes are reparsed on every popup
request, so one malformed remote changelog could tie up backend workers across
installed clients. Collect the runs in one pass and walk a cursor per run
length, since a length that runs out of partners stays out. Same 321 KB now
takes 0.013s and 5 MB takes 0.205s. Verified identical output against the old
implementation on 30000 randomized lines.
* Read type 6 and 7 HTML containers in the link resolver too
The resolver masked only type 1 blocks (pre, script, style, textarea), while
the backend parser and the collapsed preview already apply the type 6 and 7
rules, so the three disagreed on the same notes. A <details> or <div> with no
blank line inside is a type 6 block whose contents render verbatim, so two
things went wrong there: a relative link was rewritten into text the reader
sees literally, and a fence inside the block was taken for a real fence, which
silently stopped every link below it from resolving. A blank line, not the
closing tag, ends these blocks, so the common '<div align="center">' followed
by a blank line still holds Markdown and still resolves.
* Mask comments before fences, split only on Markdown line endings, stage the snapshot
Three separate reports, all confirmed against head.
The link resolver tracked no comment state, so a fence delimiter hidden inside
an HTML comment was read as a real fence. The fence then stayed open and every
visible line below was classified as code, so none of its links resolved: one
commented-out draft containing a stray backtick run silently broke the rest of
the notes. Comments are masked now, but only outside a fence, since fenced
content is literal and a comment opener in it is not one. Commented ranges join
the code spans, so a link the reader cannot see is not rewritten either.
Verified with 9 cases under node; 2 fail on the previous file.
str.splitlines also breaks on U+2028, U+2029, NEL, vertical tab and form feed,
none of which end a line in CommonMark. A separator sitting in prose ahead of
"## 9.9.9" made the parser index a release that renders nowhere and truncate
the notes above it: measured, the version list went from 2.0, 9.9.9, 1.0 to
2.0, 1.0 and the 2.0 body stopped being cut at the separator.
The build wrote the snapshot beside the checked-in sources, so a PEP 517 build
against an immutable checkout (Nix, Bazel, a read-only container mount) raised
PermissionError before build_py started and produced no wheel at all. The
source-tree copy is best effort now and the wheel takes its copy from the
staging directory. Reproduced both ways against a read-only package dir.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Use the backend's heading and quote marker rules in the preview
An ATX heading needs an ASCII space or tab after the marker, which is exactly
what _HEADING_PATTERN requires. The \s class also matches a non-breaking space,
so prose beginning "## Important change" with one was classified as a heading
and discarded by collectBullets, and a prose-only release then had no collapsed
preview at all rather than a wrong one.
A blockquote marker takes at most three leading spaces, like every other marker
in this file. Accepting any run let an indented code sample containing
"> - sample output" shed its indentation and enter the collector, so a release
with no real bullets showed code as its summary.
Both reproduced under node against the real module: the two cases fail on the
previous file and pass now, with a real heading, a real quoted bullet and an
ordinary bullet unchanged.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Collect preview reference labels only from lines that can be definitions
A definition-shaped line inside an indented code block or a deep fence is
literal text, so CommonMark leaves a later "[Beta] support" unresolved with its
brackets showing. The pre-scan ran over every line regardless, so the label was
recorded and toPlainText stripped the brackets: the collapsed preview claimed a
resolved reference the expanded notes do not have.
It now skips the same code the collector pass skips. A real definition takes at
most three spaces of indentation, so the indent test cannot reject one, which
the second case checks. Reproduced under node: the indented-code definition
resolved "Beta support" before and keeps its brackets now.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Let a document-level HTML block close an open list item
CommonMark HTML blocks of types 1 to 6 interrupt a paragraph, so a "<div>" to
the left of an open list item closes it and a following one-to-three-space
indented "## 2.0" is a real document heading. Two things stopped that: the block
opener was blanked before the list tracker saw it, so it read as a blank line,
and _may_be_lazy treated it as ordinary text that could continue the item's
paragraph. The item therefore stayed open and the release below the block was
swallowed entirely.
The opener's indentation is now taken before it is hidden, the way a fence
opener's already was, and an HTML block opener is no longer a candidate for lazy
continuation. Type 7 cannot interrupt a paragraph and is deliberately excluded,
since after_paragraph is the only state this helper is asked about.
Measured on the reported shape: the version list went from 3.0, 1.0 to
3.0, 2.0, 1.0. The test also pins the two cases that must not change, an
indented heading genuinely nested in an item and an ordinary lazy continuation,
both of which still suppress the heading.
* Let the download panel shrink inside the capped overlay stack
The bottom-right stack is capped to the viewport, but a flex item defaults to
min-height:auto, so the download panel's outer wrapper could not shrink below
its own content. min-h-0 had been added to the nested panel and not to this
wrapper, so on a short viewport the cap was absorbed by the update card, whose
header and actions are fixed, instead of by the download list, which scrolls.
Only the shared-stack branch takes it. Standalone is positioned fixed and is not
a flex item at all.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten release notes comments
Shorten the comments and docs added with the update popup release notes
so each explains its line in as few words as possible. Comments only, no
behaviour change.
* Measure release-notes indentation from the container
CommonMark measures a block's indentation from its container, not from the
left margin (spec 0.31.2 sections 4.4 and 5.2). The three changelog scanners
measured from the margin in different places, so they disagreed with the
renderer and with each other.
Under "- Details:" the content column is 2, so a four-space line is two
columns in: a paragraph holding a link. The link resolver read it as an
indented code block and left the destination relative, so it resolved against
Studio's own origin instead of the repository.
At document level the same four spaces really are code, and a top-level
bullet is not indented enough to continue the block. The preview promoted an
indented line that looked like a fence opener to a list-contained fence, so
with no later closer every bullet below it was skipped and the collapsed
popup lost its summary.
A fence is scoped to its container too: with no closing line it runs to the
end of the containing block, not the end of the document (section 4.5). A
dedented "## 2.0" closes the list item the fence sits in, so it is a real
release heading. Document-wide fence state kept the block open, so one
missing closing line hid every release below it.
Both frontend scanners now read their list columns from one module ported
from the backend's own tracker, which keeps the three in step.
Two smaller fixes ride along. A release body written as a GFM table rendered
as a grid but previewed as its raw "| Change | Detail | | --- | --- |"
delimiters, so table rows are now dropped from the collapsed summary the way
a code block already is. The comment scanner restarted its code-span search
at the first span for every opener, so a line of N spans and N openers cost N
squared: a 203 KiB line, well inside the 2 MiB the fetcher accepts, took 10.9s
and now takes 41ms.
Differential fuzzing against a CommonMark reference implementation puts the
parser's heading mismatches at 11 of 14275 documents, down from 617, and the
link resolver's at 147 of 6000, down from 217.
* Keep Retry reachable when the release notes fetch fails
The panel took fallbackMarkdown for every response that did not match, error
included, so markdown was always truthy on desktop and the error branch that
carries the Retry button was unreachable. The fallback there is the updater's
static install blurb, not this release's notes, so a transient failure showed
"Download the Apple Silicon .dmg" where the notes should be, with no way to ask
again until the cache expired.
The hook already separates the two: a reported failure is error and retryable,
"no section for this version" is ready and is not. The fallback now applies only
to the second, which is the case its prop documents.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Scope an unclosed comment to its block and end a release on a bare ##
Two CommonMark rules the changelog scanners read too strictly.
An HTML block only opens when the line itself begins with a comment marker
(spec 0.31.2 section 4.6, type 2). One written mid-sentence is inline raw HTML
and, unclosed, is ordinary text. The link resolver carried the open state to
every line below instead, so a note reading "- Type <!-- to begin a comment"
masked the relative links under it and they resolved against Studio's own
origin rather than the repository. maskComments now separates the block form
from the inline one and skips an opener sitting inside a code span, the way
_strip_comments and stripCommentSpans already do. The spans are scanned only
once an opener turns up, so a line without one costs what it did before.
An ATX heading's opening sequence may also be followed by the end of the line
(section 4.2), so a bare ## is an empty level-two heading. Both heading
patterns required whitespace after the hashes, so everything below such a line
stayed inside the release above it and the popup could show unrelated notes
under that version. An empty heading carries no version, so it ends the release
without indexing one of its own.
Differential runs against markdown-it-py: section bodies 7769 to 0 mismatches
over 36069 generated documents, comment-heavy link resolution 705 to 53 over
6000, and previews leaking a bare marker as headline text 22484 to 0 over
40000. The residual link cases are all one shape, a comment block opened inside
a list item that outlives the item, which the fence tracker scopes and the
comment tracker does not, in all three scanners alike.
* Give a hidden comment its own column and balance link destinations
A comment is an HTML block, so one written at the margin under a bullet is not
indented enough to continue that item and closes the list. All three scanners
blanked the line before list tracking saw it, which reads as a blank line and
leaves the item open, so a release heading below it looked like nested item
content and the new release merged into the one above. A hidden line now keeps
its own column through _hidden_structure and hiddenStructure, and only its
column, since the text a comment or a raw block hides is not Markdown and must
not open a list of its own. A line inside a block already open is that block's
content and still keeps nothing.
A link destination may hold parentheses while they balance, so [x]((draft).md)
points at (draft).md. The resolver stopped at the first paren, matched an empty
destination and left the markdown alone, so the link resolved against Studio's
own origin. The balanced form counts only while a closing paren or a title
still ends the link, so the stray paren in [x](a(b.md) stays the closer the way
CommonMark reads it rather than being swallowed into a link across lines.
* Scope paragraph state to the container a line is written in
Two lines the parser read as block starts are lazy paragraph text, so the
list they were written under closed early and the heading indented to the
item's content column was indexed as a release the renderer never shows.
A setext underline may never be a lazy continuation line (spec 0.31.2
section 4.3), so `===` written left of an open item is more of that item's
paragraph. Rejecting every underline-shaped line ended the list there. A row
of three dashes is still a thematic break, which does end it.
Lazy continuation runs the other way too: a marker written outside a
blockquote is not text of the quote's paragraph, so `2. item` under `> quote`
opens a list even though an ordered marker past 1 may not interrupt a
paragraph. Paragraph state is now scoped to its container: a quote line
leaves open only the quote's own paragraph, an underline needs one in its own
container, a definition ends one only when there is none to continue, and a
line four columns past its container is code, which may not interrupt.
The frontend pair reads the same tracker, so both scanners now carry the
quote state and a fence inside a list item ends with the item in the preview
the way it already did on the backend.
Measured against markdown-it-py (CommonMark 0.31.2) over 264k generated
documents: 3368 sections now match the renderer, none regressed, and every
list and quote corpus is exact.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Read a fence and an HTML block from the container it opens in
A block is measured from its container and not from the left margin (spec
0.31.2 sections 4.5 and 5.2), but the link resolver's fence, raw HTML and type
6 expressions all started at the margin, so a fence behind a quote marker and
one three columns under a nested bullet opened nothing. The sample inside was
then read as prose, and a relative link written in a code block or a details
body was rewritten into text the reader is shown verbatim. Matching runs of
backticks hid some of it by accident, since the code span scanner pairs them
across lines, but a tilde fence, a closer of a different length and every HTML
block went through. Each line is now read from the container it is written in,
which the list tracker already knew, and a block is scoped to that container
the way a fence inside an item already was: a line to the left of the item, or
outside the quote, ends the block along with it, and a bare quote marker is
the blank line that ends a type 6 block.
A destination holds parentheses while they balance, and a path may nest them,
so [x](((draft)).md) points at ((draft)).md. One nesting level was all the
expression allowed, so anything deeper fell through to the plain form, matched
an empty destination and left the link resolving against Studio's own origin.
The pairs are unrolled to the 32 levels cmark counts, and the balanced form is
still gated on a closer following it, so the stray paren in [x](a(b.md) stays
the closer the way CommonMark reads it rather than inventing a link across
lines.
Measured against markdown-it-py (CommonMark 0.31.2) over 66k generated
documents, comparing the rendered HTML rather than the destinations alone:
7286 documents in the parenthesis corpus and 313 in the container corpus now
match the renderer, and the link and definition corpora are unchanged. One
container document regresses, where closing the HTML block correctly exposes
an unrelated gap of its own: a link reference definition still leaves a
paragraph open, so the indented line below it reads as prose rather than as
code. The list tracker still matches the backend on every step, the repo's own
CHANGELOG resolves identically, and the pathological inputs measure the same.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Read a block from the item its marker opens, and let a comment reach its paragraph
Four things the three changelog scanners read differently from a renderer.
A fence written straight after a list marker is the item's own first content,
measured from the column that content starts, so "- ```md" opens one. All three
scanners matched the whole line and saw nothing, so the code sample below it was
prose: the resolver rewrote a destination the reader sees verbatim, and the
preview offered the info string as a headline bullet. A shared itemContent /
_item_content reads past a marker that really opens an item, capping the padding
the way the list tracker caps it so an over-indented line is still indented code.
An HTML block opener is read the same way, and its marker survives into the
structural line so the item it opens is still tracked.
An HTML block holds no lazy continuation line, so one opened on an item's
continuation line ends where the item does, exactly as a fence there already
did. The backend and the preview ended it only on a blank line, so it ran past
the item and swallowed the next release heading, which made those notes
unreachable and dropped every bullet below it from the collapsed popup. A raw
block inside an item ends on a blank line too, which is where cmark puts it.
A comment written mid-sentence is inline raw HTML belonging to the paragraph
around it, so its "-->" may arrive on a later line of that same paragraph. Ending
it at its own line left a backtick inside it pairing with a real one below, which
hid a following link from the resolver, and left the preview quoting text the
popup body does not show. A shared commentClosesBelow answers whether the closer
arrives before the paragraph breaks; where it does not, the opener stays the
ordinary text a renderer shows, so a note that merely mentions "<!--" still hides
nothing.
Only ASCII punctuation is escapable, so the backslash in "docs\alpha.md" is a
character of the path. Dropping every backslash rewrote it to a path that does
not exist, and a URL parser reads what survives as a separator, so a Windows or
namespaced path pointed at the wrong file either way. The destination expression
now escapes only punctuation, which also means a space still ends a destination:
"[x](a b.md)" and "[x](a(b.md)" are not links, so their paths are left alone
rather than half-rewritten. A destination that runs out of line still resolves,
since its closer is on the line below.
Fuzzed against markdown-it (CommonMark 0.31.2) over 20k-document corpora, with
the whole rewritten document rendered and compared, not just its destinations.
Release headings: 117 to 16 on containers, 88 to 10 on markers, 17 to 12,
nothing new anywhere. Link destinations: 8823 to 104 on markers, 114 to 98 on
comments, nothing new. Whole-document renders: 9271 to 220, 5116 to 245, 1265 to
671. The Python and TypeScript list trackers still agree over 26861 steps, and
itemContent and hiddenStructure agree over another 6335. 321 KB of unmatched
backticks still measures the same.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Let a definition follow a definition, and read a comment from the item it opens in
Three CommonMark conformance fixes in the changelog scanners.
A link reference definition is a block of its own that may not interrupt a
paragraph, so it opens none either: definitions are allowed to run
consecutively (spec 0.31.2 section 4.7). The link resolver counted one as
paragraph text, so every definition after the first fell outside the set of
lines a definition may start on and kept its relative destination, which then
resolved against Studio's own origin. The backend already read the line this
way.
The guard asking whether a `-->` is reachable from an opener read any line
whose first character was punctuation as the start of a new block. A `-->`
written on a line of its own is how a multiline comment is ordinarily closed,
and a wrapped line may open with emphasis, so neither counted as more of the
paragraph carrying the comment. The comment never closed and the collapsed
popup showed the author's internal note to the reader. It now tests for a
block that may actually interrupt a paragraph.
A comment is an HTML block too (section 4.6, type 2), so one written as a list
item's first content opens inside that item exactly as a fence written there
does. All three scanners looked for the opener at the margin of the line as
written, so a marker in front of it hid the block: the resolver rewrote a
destination inside raw HTML, which Streamdown then shows the reader as a
literal URL, and the preview quoted the hidden note back at them as though the
bullet were Markdown. The opener is now read from the item's content, the
marker survives into the structural line so the item it opens is still
tracked, and the block is scoped to that item the way a fence there is.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten the release notes comments without losing the reasons they record
---------
Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
* fix: add XPU device support and update hardcoded CUDA selections
* fix: add XPU device support for pytest CUDA skipped tests
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix device handling for PR #7401
- perplexity_eval.py: use DEVICE_TYPE_TORCH, not DEVICE_TYPE. The latter can
be "hip" or "mlx", which .to() rejects, so this regressed ROCm.
- test_batched_leftpad_generation_gpu.py: XPU diverges here today, so mark it
non-strict xfail on XPU instead of reverting to a CUDA-only guard. Keeps the
real XPU gap visible and turns green once it is fixed.
- Guard torch.xpu.is_available() with hasattr, matching device_type.py.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Re-enable the flash varlen attention test in CI for PR #7401
attention_dispatch.py now predefines flash_attn_func / flash_attn_varlen_func
as None, so test_run_attention_flash_varlen_receives_window_and_softcap no
longer needs flash_attn importable to be monkeypatched. Verified on a runner
shaped like the CPU-only one: the test fails against main's attention_dispatch
and passes at this head, so the deselect is now dead weight.
* Tighten comments for PR #7401
Drop the hasattr rationale: torch.xpu has existed since torch 2.3 and the
dependency floor is 2.4, so no supported build predates the namespace. The
guard stays as cheap defence, but the comment claimed something untrue.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
studio-tauri-smoke.yml only ever built the crate, so none of its ~100 unit
tests executed. Running them surfaced two that were broken on platforms CI
never exercised:
- non_utf8_import_name_preserves_csv_extension built a filename containing a
raw 0xFF byte. Linux stores that fine, macOS enforces UTF-8 on APFS/HFS+ and
refuses to create it, so the test panicked on the unwrap. Skip when the
filesystem rejects the name; the branch under test is only reachable where
such a file can exist.
- losing_a_studio_package_changes_the_fingerprint created the posix venv
layout unconditionally, but site_packages_dirs() only walks
lib/<pyver>/site-packages on unix and looks at Lib/site-packages on Windows.
The dist-info was therefore invisible to the fingerprint there, removing it
changed nothing and the assert_ne could never hold. Build the layout the
code actually reads for the target platform.
Add the cargo test step to the existing Tauri job, where the toolchain and
WebKit dev packages are already installed.
* Fix Windows no-torch setup
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix no-torch env normalization on Windows
* Accept on for Windows no-torch mode
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Keep no-torch mode across studio update on Windows
Guarding the direct torch/Triton install made `install.ps1 --no-torch`
actually produce a torch-free venv, which then broke the next
`unsloth studio update`. That path exports no UNSLOTH_NO_TORCH, so
$NoTorchMode was false, the stale-venv check read the missing torch as a
broken venv, and setup tried to delete the venv it was running out of:
[ERROR] Could not remove stale venv: Access to the path 'python.exe' is denied.
That teardown can never succeed there, because setup.ps1 runs via
unsloth.exe out of that same venv. The same gap also let the shared
dependency pass reinstall torch from PyPI, unpinned, into a GGUF-only
environment.
install_python_stack.py now records the mode in the install manifest and
setup.ps1 reads it back when no env var is exported, then re-exports a
canonical value for the dependency pass (setup.ps1 drops the manifest
before invoking it, so the child cannot repeat the lookup). The key is
additive and MANIFEST_SCHEMA is unchanged, so existing manifests stay
valid and a missing key keeps today's behaviour.
Also:
- read_manifest() caught only OSError, but UnicodeDecodeError is a
ValueError. That is now on the installer's import path, so a manifest
re-saved as ANSI or truncated mid-write would abort every install.
- The env predicate now trims surrounding whitespace, matching the
Python side.
- The Windows update smoke workflow asserts the update leaves the venv
GGUF-only, which is what would have caught this.
Known follow-up, pre-existing: an install killed between the manifest
drop and the dependency pass leaves no recorded mode, so a later update
still walks the stale-venv path. Closing that needs a marker the
installer never drops.
* Persist no-torch mode in a marker the dependency pass cannot drop
The install manifest alone was not enough. Both setup.ps1 and
install_python_stack.py remove it before every dependency pass, and it is
only rewritten on success, so a no-torch install interrupted in between
left nothing recording the mode. The next update then resolved no-torch
as false, read the expected missing torch as a stale venv, and tried to
delete the environment whose python.exe was running it, which leaves the
install unrepairable from the CLI.
Add .unsloth-no-torch next to the existing .unsloth-studio-owned marker,
written before the pass and cleared when torch is wanted. setup.ps1
writes it as soon as the mode resolves, so the window between the
manifest drop and its own torch install is covered too.
Read order stays manifest key first, then marker, so migrating out of
no-torch is never blocked by a marker an earlier run left behind. Neither
present still reads as "install torch", so nothing changes for installs
made before either existed.
Also adds the AGPL-3.0 header the new test file was missing.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <unslothai@gmail.com>
* Studio: preserve durations across reasoning blocks
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: keep a reasoning group's timer running when it reopens
A rendered reasoning group can be closed and then reopened: parseAssistantContent
coalesces adjacent reasoning parts, so a provider that emits each block as a
complete <think>...</think> chunk lands several blocks in one group. The tracker
wrote a group's duration once and never revisited it, so such a group froze at
its first close and displayed 0 seconds.
Measure from the first time an index becomes visible rather than from the last
startGroup, and reopen a closed group while its reasoning text is still growing.
Gating on growth is what stops the timer running on into the answer. A duration
supplied by the server is now recorded as authoritative so local timing cannot
overwrite it.
Also fill indices that a single delta skips. startGroup(n) could jump past
earlier indices and leave array holes, which JSON.stringify persists as null; a
skipped group became visible and closed inside the same chunk, so it gets a
measured zero instead.
Test discovery now globs tests/, so a second test file cannot be silently
skipped by CI, and tsconfig.test.json puts tests/ under typecheck for the first
time.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
* Studio: detect an interrupted dependency install instead of launching a backend that cannot import
An installer killed part-way leaves a venv with a working CLI but without
studio.txt's dependencies. Nothing recorded that, so three separate places all
reported it healthy:
- the desktop preflight probed only `unsloth -h` (typer + rich) and a hardcoded
desktop-capabilities dict, neither of which touches studio.backend, so it
returned ManagedReady and spawned a backend that died on `import structlog`;
- setup.sh's fast path compared the installed unsloth version against PyPI,
which matches on a half-built venv because unsloth is installed early, so
`unsloth studio update` printed "up to date" and repaired nothing;
- start_managed_repair calls that update and then re-checks with the same blind
probes, so Repair reported success without fixing anything.
install_python_stack.py now clears a completion manifest before the dependency
pass and writes it only after the final step. `unsloth studio verify-install`
and desktop-capabilities' new studio_install_ok field read it, the preflight
turns a false answer into ManagedStale so auto-repair runs, and setup.sh /
setup.ps1 gain an escape hatch next to the existing anyio one.
Separately, the wheel ships studio/ and studio.backend* but declared none of
their dependencies, so `unsloth train`, `export`, `chat`, `inference` and
`studio` all ended in a rich traceback after a plain pip install. structlog is
the only hard module-level import that chain reaches once starlette's
annotation-only import moves under TYPE_CHECKING, so it becomes a core
dependency and the rest of the server stack becomes a [studio] extra mirroring
studio.txt. The CLI import sites now report missing dependencies as a sentence
with two remedies.
Fixes#4701, #5260, #7147
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Match the trimmed comments merged on the pip branch
* Put the install manifest in the preflight fingerprint for PR #7492
The capability cache keyed the venv on pyvenv.cfg, uv.lock, requirements.txt,
the interpreter and site-packages/unsloth_cli/commands/studio.py, none of which
a repair touches when it only reinstalls studio.txt. So an entry cached while
the install was healthy stayed valid after the manifest was dropped, and the
probe returned Ready on exactly the half-built venv this is meant to catch.
* Address the review findings on PR #7492
Fail the install when the completion manifest cannot be written, instead of
exiting 0 without the record every later check requires, which is a repair
loop by construction.
Compare the version of the package the manifest names, so `studio update
--package X` does not read as a permanent version change.
Read the manifest from the venv that owns it when the CLI runs outside the
managed venv, and drop the dependency verdict in that case: the walk ran
against the wrong interpreter and says nothing about that venv.
Name the import that actually failed. `unsloth train` reaches torch through
the same guard, and the studio extra does not carry it, so recommending that
extra alone left the command failing in the same place.
* Declare click, which typer stopped providing, for PR #7492
unsloth_cli/commands/start.py imports click at module scope and
unsloth_cli/__init__.py imports that module, so every unsloth command needs
it. typer carried click through 0.19 and dropped it in 0.27, and the declared
floor is typer>=0.12.0, so a fresh resolve gets no click. On the published
wheel it still arrives because huggingface_hub requires click<9,>=8.4.2, which
is luck rather than a declaration. A wheel built from this branch's
dependency list has neither, and every command dies at import.
Verified: before, `unsloth --help` on a fresh venv raised ModuleNotFoundError
for click; after, it exits 0. The drift test now covers it.
* Keep a running backend from the previous app version manageable
The manageability bump gated two unrelated things through one constant. For
the managed CLI probe 2 is right: a CLI reporting 1 cannot answer
studio_install_ok. For a RUNNING backend it is wrong, because a process
already started cannot change what it reports, so bumping studio/backend/main.py
in lockstep does not help one the previous app version spawned.
That backend is proven ours by root id and ownership token, but
lifecycle_control_block_reason returned Unmanageable, and that branch never
calls adopt_verified_backend. has_owned_backend() stays false, so Repair falls
into block_external_conflict, which finds the same process and refuses: the app
could no longer stop a backend it owns the token for. The same regression in
backend.rs turned a terminal-launched same-root server from AttachedReady into
ExternalConflict.
Split the constant: DESKTOP_BACKEND_MANAGEABILITY_VERSION = 1 for the two
live-backend probes, DESKTOP_MANAGEABILITY_VERSION = 2 for the CLI probe. Every
real gate (protocol, auth, ownership, desktop-login, MIN_DESKTOP_BACKEND_VERSION)
is untouched, so an old backend still reaches OwnedStale, adopt, stop, repair.
Also stop the installer when the stale manifest cannot be removed. Windows
raises on a read-only or locked file, and the pass would then run behind a
marker that still names this version and these digests, so a run killed
part-way would verify as complete.
* Answer for the managed venv, not the one the CLI happens to run in
The guard matched ModuleNotFoundError.name, an import name, against
missing_requirements(), which returns distribution names. So a missing PyJWT
printed 'pip install jwt', and jwt, docx and fitz are each a real but unrelated
PyPI project (fitz is a neuroimaging workflow tool), so following the advice
installed the wrong package and left the backend just as broken. Map the import
to its distribution before deciding, and never offer the import itself.
install_state() verified the caller's own prefix. The wheel ships studio/, so a
CLI installed outside the managed venv always finds its own copy of the helper
first, and a healthy managed install reported studio_install_incomplete with a
missing list copied from the wrong venv. Selecting the root is not enough:
_installed_version() reads the running interpreter and req_root defaults to the
caller's studio.txt, so both checks still answered for the wrong venv. Hand
verify_install() that venv's own metadata, enumerated through
Distribution.discover(context = ...path), which does not fall back to sys.path.
The candidate order is untouched, so shadowed-tree detection is unchanged.
setup.ps1 replaces pip, torch and triton before install_python_stack.py runs,
so the manifest it drops is not dropped before the first mutation. A run killed
in between kept a marker that still verifies while torch was half-replaced;
drop it at the top of the dependency pass instead. setup.sh is unaffected, the
stack is the first thing its pass runs, and a test now pins both.
pip uninstall rewrites nothing that was fingerprinted, and cache_matches
re-reads the cached studio_install_ok rather than re-checking, so a venv that
lost a studio.txt package kept being served the healthy verdict. Fold a sorted
hash of the installed dist-info names into the marker hash.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* A missing manifest helper is a torn install, not an old one
studio/install_manifest.py ships in the same wheel as _studio_deps.py, so
nothing legitimately has one without the other: a CLI predating both never
reaches this code, and the desktop already calls such a CLI stale on
desktop_manageability_version.
Returning ok=true there reported a healthy install for a tree the package
update had half replaced, and the preflight then launched a backend whose
own run.py could be just as absent. Report it incomplete so repair runs.
* Tighten comments across the install-detection changes
* Validate Studio dependency readiness
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: Wasim Yousef Said <wasimysdev@gmail.com>
* fix: keep offline GGUF export off the Hub for VLM tokenizers (#7481)
Resolve cached snapshot directories before loading PreTrainedTokenizerFast
during VLM processor fallback so transformers does not call is_base_mistral()
-> model_info() when HF_HUB_OFFLINE is set. Also probe the local cache in
_has_tokenizer_model instead of model_info when offline.
Fixesunslothai/unsloth#7481
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* test: add real-cache offline GGUF integration checks for #7481
Download unsloth/gemma-3-270m-it-bnb-4bit (~430MB) and verify offline
snapshot resolution and tokenizer load with network blocked. Full unsloth
import tests remain GPU-gated.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: address Codex review on offline GGUF tokenizer paths (#7481)
- Only rewrite Hub repo ids to cached snapshot dirs when offline
- Copy tokenizer.model from cache offline in preserve_sentencepiece
- Do not cache negative offline tokenizer.model probe results
- Add regression tests for all three review items
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: probe HF cache before model_info for local-only GGUF saves (#7481)
Always resolve tokenizer.model from the local Hub cache before calling
model_info, and skip Hub metadata when the tokenizer was loaded with
local_files_only or offline env vars. Fixes Codex review on PR #7482.
* Fix lint blocker, false-green tests and offline defaults for PR #7482
Drop the two unused _env_says_offline imports that fail the Source lint
import-hoist check.
test_has_tokenizer_model_offline_skips_model_info and its local_files_only
twin set model_info.side_effect = AssertionError, but _has_tokenizer_model
wraps that call in "except Exception: return False", so the AssertionError
was swallowed and both passed on the merge base with the fix absent. Assert
model_info.call_count == 0 instead; both now fail on the base with
assert 1 == 0.
The real-cache integration tests called hf_hub_download and
PreTrainedTokenizerFast directly, so they exercised plain huggingface_hub and
passed identically on both trees. Route them through the resolver this PR
adds, and gate the file at module level since importing unsloth needs a GPU
host either way.
_resolve_hub_repo_local_dir and _resolve_hub_repo_cached_file defaulted to
local_files_only = False, so a helper named "resolve local dir" would
download with backoff retries when called without the flag. Every caller
already passes it explicitly, so default it closed.
Use tempfile.gettempdir() rather than a hardcoded /tmp, which silently
skipped both files on Windows, the platform in the bug report. Patch
socket.socket connect rather than replacing the class, which broke
isinstance checks.
Wire the unit tests into the Bucket-A CI list; Repo tests (CPU) ignores
tests/saving, so none of these ran anywhere.
* docs: note transformers 4.57.2-5.5.4 window for local tokenizer resolve
Name the version range where from_pretrained still probes model_info under
local_files_only, and point at the 5.6.0 upstream fix so the helper can be
removed once the supported floor moves past it.
* fix: enable real-cache suite in offline GGUF integration runner
Pass UNSLOTH_INTEGRATION_IMPORT=1 into the pytest subprocess so the
documented runner actually executes the real-cache tests instead of
reporting success after only the fake-cache unit file runs.
* docs: note integration runner enables UNSLOTH_INTEGRATION_IMPORT
Document that the runner sets the gate itself and still needs a host
that can import unsloth.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Keep an explicit local_files_only load local-only at save time
transformers takes local_files_only as an explicit from_pretrained parameter,
so it never lands in tokenizer.init_kwargs, and _offline_aware_load restores
HF_HUB_OFFLINE / TRANSFORMERS_OFFLINE as soon as the load window closes. A VLM
loaded with local_files_only = True but no offline env var therefore came back
with the Hub repo id in name_or_path and nothing recording the request, so
_tokenizer_wants_local_only returned False on the later save and
_has_tokenizer_model fell through to HfApi.model_info - and then
_preserve_sentencepiece_tokenizer_assets fetched tokenizer.model from the Hub
with local_files_only = False. On a disconnected host that is a network wait
before the export gives up.
Stamp the load's local-only mode onto the returned processor and its tokenizer
inside the forced-offline window, and honour that stamp in
_tokenizer_wants_local_only, so the save path inherits the load's contract.
Verified against a real hub-cache layout whose snapshot has tokenizer metadata
but no tokenizer.model: before, one model_info call plus an hf_hub_download with
local_files_only = False; after, zero model_info calls and cache probes only.
Two tests added to tests/saving/test_offline_gguf_vlm_tokenizer_7481.py; both
fail with the loader_utils hunk reverted and pass with it in place.
* Carry the load's cache_dir through to saving for PR #7482
The local-only stamp added in e7b7400de preserved only the boolean. Saving
still derived its cache from HF_HUB_CACHE or HF_HOME, which does not see a
caller-supplied cache_dir, and FastBaseModel.from_pretrained threads one all
the way down. So a local_files_only load against a custom cache missed on the
probe, and the stamp then stopped the Hub fallback that used to cover it, and
tokenizer.model was silently left out of the GGUF staging directory.
Stamp the cache_dir alongside the local-only marker and prefer it at both
sites in save.py that derive one from the environment. Reverting save.py
alone, with the helper still present, fails the new test on behaviour.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Merge main and drop an unused import for PR #7482
Brings the branch up to date with main, which clears the stale Source lint
blocker inherited from #7476 by taking studio/backend/utils/hardware/__init__.py
out of this PR's changed-file set.
pytest was imported in the new test file and never used, which the
import-hoist check flags in its own right.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* ROCm/AMD CI coverage: arch-table parity, native-Linux lib prepend, RDNA4 grouped_mm, discovery-based shell suite
Three merged ROCm fixes shipped without tests, and the CI wiring that
would have run them was gated on files the fixes do not touch.
Tests added (113):
tests/studio/install/test_rocm_arch_table_parity.py (27)
diffs the four duplicated gfx -> AMD pip-index tables across
install.sh, install.ps1, studio/setup.ps1 and install_python_stack.py,
plus the GPU-name -> arch tables and the torch 2.11 pin allowlist.
tests/studio/install/test_rocm_native_linux_lib_dirs.py (26)
covers #7233: system-ROCm lib dirs prepended ahead of bundled
libggml-hip, the /dev/kfd + not-WSL + libhsa gate, the opt-out env
var, root resolution order, and source parity between the two copies.
studio/backend/tests/test_grouped_mm_rdna4_fallback.py (46)
covers #7292: registration on the CUDA dispatch key, grouped and
ungrouped numerics, bias/dtype promotion, and the Linux HIP<7.13 +
RDNA4 name gate, executed from the shipped source rather than a copy.
tests/studio/test_ci_shell_suite_coverage.py (14)
fails if either shell runner goes back to a hardcoded list or skips
a file without a recorded reason.
CI wiring:
studio-backend-ci.yml: add install.sh / install.ps1 to the path filter
(the suites it runs assert against those two files, so install-only
changes -- the shape most AMD/ROCm routing fixes take -- skipped it),
and replace the 13-file hardcoded shell list with directory
discovery. That list had fallen seven files behind, including
test_strixhalo_wsl_reroute.sh, the only shell coverage of the ROCm
WSL reroute, which had never run on a PR.
tests/run_all.sh: same discovery loop so local and CI agree.
* Test review fixes: assert on outcomes, not on the code under test
Self-review of the previous commit found four tests that passed for the
wrong reason.
1. The arch-table parity test pinned expected gfx ids copied out of the
shipped tables, which enshrined three upstream inaccuracies as
correct: RX 9070 (non-XT) is gfx1201 not gfx1200, RX 7800 XT is
gfx1101 not gfx1100, and PRO V710 is gfx1101 not gfx1102 per AMD's
ROCm compatibility matrix. The expectation is now the AMD pip index
leaf -- the thing the tables exist to produce, and what a wrong
answer costs the user. The three known drifts are listed explicitly
with a test asserting they stay cosmetic, i.e. that the wrong and
right ids still map to the same wheel index. That test turns red the
day one of them starts routing users to the wrong wheel.
2. The RDNA4 device-name test extracted the regex from worker.py and
then matched with it, so it could not fail. Widening the pattern --
the dangerous edit, since it forces the slow Python mm fallback onto
RDNA3 users -- would have been silently accepted. It now reads the
live pattern and checks it against fixed cases, plus asserts the
name match stays guarded by `not _lin_arch` and that the name is
lowercased before matching.
3. The CI-coverage test matched a verbatim line of studio-backend-ci.yml,
so reindenting the step would fail the build while a real regression
to a hardcoded list could slip past a reformat. It now parses the
YAML, finds the step by name, and asserts on the glob plus the
absence of individual filenames. The path-filter test likewise reads
the parsed trigger instead of scanning raw text.
4. A set comprehension in the parity helper had a ternary whose branches
were identical.
Mutation-tested: widening the RDNA4 regex, desyncing one copy of the
name table, dropping install.sh from the path filter, and re-skipping
the ROCm WSL shell suite each fail at least two tests. Verified on
Linux (WSL Ubuntu 24.04) with CI's torch pin: 86 + 48 pass.
* Fix three wrong gfx ids in the GPU-name arch tables
The name -> gfx tables disagreed with AMD's ROCm compatibility matrix on
three entries. Corrected against the "Radeon GPU" list at
rocm.docs.amd.com/en/latest/compatibility/compatibility-matrix.html:
RX 9070, RX 9070 GRE gfx1200 -> gfx1201 (Navi 48, same die as the XT)
RX 7800 XT, RX 7700 XT gfx1100 -> gfx1101 (Navi 32, not Navi 31)
PRO W7700 gfx1100 -> gfx1101
PRO V710 gfx1102 -> gfx1101 (Navi 32, not Navi 33)
No wheel changes for anyone: gfx1200/gfx1201 both resolve to gfx120X-all
and gfx1100/gfx1101/gfx1102 all resolve to gfx110X-all, in all four copies
of the index-family map. That collapse is why the errors survived being
copied into six places -- the leaf-level tests could not see them.
It was not purely cosmetic, though. install.sh's second copy feeds
"Tip: set UNSLOTH_ROCM_GFX_ARCH=<arch>", so a 7800 XT user following the
printed advice exported gfx1100 and made a wrong id authoritative for
every later run. It would also have become a real misroute the moment AMD
split a family across index leaves, as they already do for gfx1151/gfx1150.
Fixed in all six places, which is two more than the table's own "kept in
sync with" comments claim exist:
install.sh _infer_amd_gfx_arch_from_gpu_name
install.sh case "$_gpu_disp_mkt" (banner + env tip; undocumented)
studio/setup.sh
install.ps1
studio/setup.ps1
studio/install_python_stack.py
Ordering is preserved: the gfx1102 arm still precedes gfx1101 in the shell
copies so "RX 7700S" cannot fall onto the "RX 7700" glob, and the
PowerShell copies keep the (?!S) lookahead.
Test changes:
- test_rocm_arch_table_parity.py gains _AMD_DOCUMENTED_ARCH, exact gfx
ids transcribed from AMD rather than from the tables. Agreement between
six copies proves nothing when all six were transcribed from the same
mistake, so the ground truth has to come from outside. Verified it
catches the bug: against the pre-fix tables it fails 6 tests.
- The parity check now covers all six copies. It had four; the two
install.sh copies were being treated as one, and
_WIN_GPU_NAME_ARCH_TABLE was not checked at all.
- test_rocm_support.py's TestGfxArchNameFallback pinned two of the wrong
ids as expected values; updated, and extended with a 9060 XT and a
7900 XTX case so each RDNA3/4 die is represented.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Guard against unregistered copies of the GPU-name arch table
Counting the copies by hand is what let them drift: the in-code "kept in
sync with" comments claimed four, the arch-id fix found six, and scanning
the tree turns up a seventh.
TestNoUnregisteredArchTable rediscovers the copies from the source tree
instead of trusting a hand-maintained list. A table line is one that names
a card and gives its arch; real tables score 9-17 such lines and the only
other hits in the repo are two single-line prose comments, so the
three-line threshold is not load-bearing. A companion test asserts the
scan still finds the known copies, so the heuristic cannot go blind and
pass by finding nothing.
The seventh copy is tests/_zoo_rocm_spoof.py, the fixture other ROCm tests
build their fake AMD host from. It states the mapping backwards (gfx ->
the name torch should report), which makes it an independent witness: it
had gfx1101 -> RX 7800 XT and gfx1201 -> RX 9070 XT right while all six
installer copies were wrong, and nothing compared the two. Now they are
round-tripped against each other.
RX 6700 XT is pinned as a known divergence rather than normalised. AMD's
compatibility matrix documents no consumer RX 6000 card and no gfx1031 at
all, the installer arm is commented "gfx103X family", and gfx1031 appears
only as an index-family key, never as a value a name table emits. With no
external source to correct against, changing shipped behaviour would be
guesswork. A test fails if the divergence ever disappears, so the
exemption cannot go stale.
Also adds the reverse of the AMD-matrix check: a documented card that
matches no arm anywhere is a silent CPU fallback rather than a wrong id.
This cannot detect hardware nobody transcribed, which would need a live
fetch of AMD's matrix and a non-hermetic suite; the docstring says so
rather than implying coverage that is not there.
Verified on Linux: 478 passed, plus all five new guards mutation-tested
to confirm each fails when its invariant is broken.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Docstring said six copies; the list under it now has seven
* tests: run discovered shell tests with bash, not sh
tests/run_all.sh discovered tests/sh/ instead of listing files, but still
invoked each one with sh. Every file there declares a bash shebang, and on
Debian/Ubuntu /bin/sh is dash: test_apt_distro_prompt.sh,
test_studio_home_node_dir.sh and test_with_llama_cpp_dir_link_behavior.sh
fail on bashisms under dash and pass under bash. The old hand-written list
happened to name only dash-clean files, so switching to discovery is what
surfaced it. Backend CI already used bash, so this was a local-only break.
Guarded by a new test asserting both runners invoke tests/sh/ with bash.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix Krackan Point (Radeon 860M/840M) routed to the gfx1150 wheel index
The GPU-name tables map 860M/840M and the Ryzen AI 7 350 / AI 5 340 CPU
strings to gfx1150, but Krackan Point is gfx1152. AMD's own lemonade table
(src/cpp/server/system_info.cpp) maps both Krackan iGPUs to gfx1152.
Unlike the three ids already fixed here, this one is not wheel-neutral:
repo.amd.com publishes gfx1150 and gfx1152 as separate index leaves with
separately built torch wheels, so these laptops were installing wheels
built for a different LLVM target. gfx1152 was absent from the codebase
entirely, so it needed the index-family maps, the torch 2.11 floor lists
(same _grouped_mm bug as gfx1150/1151), the Strix reroute set and the
Windows arch allowlist as well as the seven name tables.
The parity test added in this PR did not catch it because its AMD-matrix
expectations stopped at 890M/880M. Added the APU rows, so the case that
actually changes a wheel is now covered: reverting the tables fails 9
tests naming 860M, 840M and Krackan.
gfx1153 (Ryzen AI 5 430 era) is left alone; AMD publishes no gfx1153
wheel family, so there is nothing to route it to.
Verified: bash -n on both shell installers, PowerShell AST parse on both
.ps1 files, python ast.parse on all touched modules, install suite 1334
passed with no new failures against main, shell suite 20 files.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add gfx1152 to unified-memory classifiers, make parity allowlist set-based
Krackan Point (gfx1152, Radeon 860M/840M) is the third RDNA 3.5 APU and
shares one GPU/system-RAM pool exactly like Strix Point (gfx1150) and
Strix Halo (gfx1151), but only the installers knew about it. The two
runtime classifiers still had two-element arch sets, so a Krackan laptop
got the 0.90 discrete headroom factor on a shared pool and ran llama.cpp
without GGML_CUDA_ENABLE_UNIFIED_MEMORY.
- worker.py _rocm_classify_unified_memory: add gfx1152 to the arch set,
and 860m/840m to the device-name fallback. The NVIDIA GeForce 840M
cannot collide there: the function is only reached under _hw.IS_ROCM.
- llama_cpp.py _amd_apu_wants_unified_memory: add gfx1152 to the arch set.
- Tests for both, including the :sramecc-:xnack- suffix form.
TestGfx211AllowlistParity compared four hardcoded allowlist strings, so
adding gfx1152 to all four installers correctly turned three assertions
red without any installer actually disagreeing with another. Each test
now extracts the set its installer holds and compares it to one EXPECTED
constant. Order and spacing are free, membership is not, and the next
leaf is a one-line edit instead of four.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Roland Tannous <115670425+rolandtannous@users.noreply.github.com>
Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
* fix(studio): opt-in source-build GPU smoke validation (#5854)
Gap 1 (empty CUDA arch -> CPU) already landed in #6481. Wire gap 2: after a
GPU source build, optionally run the same staged llama-server smoke test as
the prebuilt path, then CPU-fallback on failure. Gated by
UNSLOTH_LLAMA_STAGED_VALIDATION (default off) to avoid Blackwell JIT stalls.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(install): normalize staged validation env in setup.sh (#7322)
Strip and lowercase UNSLOTH_LLAMA_STAGED_VALIDATION before the shell
gate so values like True and surrounding whitespace match the Python
staged_validation_enabled() helper.
* Rebuild visual server after staged-validation CPU fallback (#5854)
Mirror the primary source-build path by best-effort building
llama-diffusion-gemma-visual-server after smoke-failure CPU fallback.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: drive UI font size through a typography scale, not the root font size
Follow up to #7355. The preference now writes --ui-font-scale
(selected / 16) and a data-ui-font-size attribute on the root instead
of mutating the root font size, and the applier clears any stale inline
root font-size left by older builds. Because the rem base never moves,
every layout-only rem-to-px conversion from #7355 is reverted to its
original form: the spacing, radius and container tokens, sidebar and
thread widths, grid tracks, calc margins and hub.css dimensions match
pre-#7355 main again, which also restores rem-based accessibility
scaling for users with a larger browser default font size.
Typography scales through tokens in index.css, all exact at 16px:
- The named Tailwind sizes (--text-xs through --text-4xl) multiply
their defaults by the scale, so standard utilities scale
- One token per design px size (--text-ui-8 ... --text-ui-34) replaces
every arbitrary text-[Npx] class; leading-ui-* mirrors the exact
line heights and the numeric --leading-3..10 scale as well
- CSS font-size and line-height declarations multiply by the scale
- Chart labels scale through a .recharts-text rule; streamdown and
react-flow px text is re-based via scaled overrides; KaTeX's 1px
layout trick stays fixed by design
- The logo lockups keep their half-rate behavior via the scale var
- The explicit Code font size remains unmultiplied
Keeps the #7355 behavior fixes: color chip min width, voice select
min/max widths, and the select and dropdown menus scrolling an inner
viewport so their corners stay rounded. The whitespace-password and
IME rename guards that merged alongside are preserved.
* Studio: contract and Playwright coverage for the UI font size scale
test_ui_font_scale_contract.py pins the mechanism (scale var written,
root font size never mutated, tokens scaled, code font size not
multiplied, the Radix select viewport owning scroll state) and guards
against new raw pixel typography, with a documented allowlist for the
recharts fontSize props covered by the stylesheet override and the
offscreen clipboard textarea.
playwright_ui_font_scale.py drives the real appearance controls: root
font size fixed at 12/16/20, text and line height scale by size/16,
sidebar width invariant, explicit code font size stays fixed, an
overflowing dictation select scrolls its Radix viewport by keyboard
and wheel, and the default restores exactly. Wired into the UI smoke
workflow against the second studio boot.
The thinking-compact and descender contracts move back to the rem and
token forms now that layout values no longer need px pinning.
* refactor(studio): move chat model picker into features/model-picker
Relocate model-selector + its support files from components/assistant-ui
into a self-contained features/model-picker feature (own barrel), mirroring
the modular Hub layout. Pure move + import repoint; no behaviour change.
* feat(model-picker): add per-model config persistence layer
Superset PerModelConfig (customContextLength, kvCacheDtype, speculativeType,
specDraftNMax, tensorParallel, chatTemplateOverride, trustRemoteCode) persisted
to localStorage (unsloth_model_configs) with schema versioning + LRU budget.
KV-dtype and speculative value sets match main's sidebar (no q4_0/ngram-simple).
Reuses features/hub/lib/model-identity for normalization; adds storage-key layer
and applyPerModelConfigToRuntime (sets tensorParallel, which the old PR omitted).
* feat(picker): modular backend for chat-template validate + default fetch
New studio/backend/picker package (schemas/service/routes) mounted at /api/picker:
- POST /api/picker/validate-chat-template (Jinja syntax validation, no false positives)
- GET /api/picker/chat-template/{model_name} (default template from tokenizer_config.json,
reusing get_cache_path/resolve_cached_repo_id_case; graceful null, no model-code exec)
Frontend api/templates.ts client + hooks/use-model-defaults lazy cache. No backend
changes to the existing inference load route (per-model load fields already supported).
* feat(model-picker): bind picker on-device list to shared hub inventory
Picker now sources cached + local models from useHubInventory (the Hub's shared
store) via a thin adapter, replacing its own /api/models/* fetchers + module
caches. Hub, download manager, and picker now share one source of truth, so
completed downloads reflect in the picker automatically. Partial/live-download
rows are filtered from the cached lists (unchanged rendering). Local naming/search
preserved via additive LocalInventoryRow modelId/displayName. Variant expander,
scan-folder management, recommended-fit, search, external providers untouched.
Known minor: cached 'Downloaded date' sort tiebreak degrades to alphabetical
(hub cached rows carry no mtime); default 'recent' (load-time) sort preserved.
* feat(model-picker): per-model config step inside the picker
Picking a (non-external) model now opens an in-picker config view built from
main's current load controls (context length, KV cache dtype, speculative
decoding, draft tokens, tensor parallel) plus a chat-template editor backed by
the picker validate/default endpoints. 'Remember for this model' persists the
config per model+variant; Run forwards the config to the existing load flow via
meta.config. External models bypass the step. Two-view orchestration lives in
model-selector (single interception point); pickers.tsx call sites untouched.
trustRemoteCode dropped from PerModelConfig to preserve main's per-load consent.
* feat(chat): apply/persist per-model config through the load flow
handleCheckpointChange threads meta.config into the selection; stageOrLoad and
the autoload/Hub-run paths now apply the picker config (explicit pick or saved
remembered config) via applyPerModelConfigToRuntime before staging/loading, with
keepSpeculative set so a remembered speculative mode survives the model switch.
Replaces the old remembered-load-settings seeding (resolveInitialConfig now the
single source). SelectedModelInput carries config.
* refactor(chat): remove per-model load config from the right sidebar
The load knobs (context, KV cache, speculative, draft tokens, tensor parallel)
and the chat-template editor now live only in the picker config step. The sheet's
Model section keeps the staged Load/Cancel flow (config is applied at pick time);
sampling params, system prompt, and RAG are unchanged. Deletes the superseded
remembered-load-settings module + the store's applyRememberedLoadSettings action,
removes the now-dead sheet state/imports, and points the settings reset at
unsloth_model_configs. Delete-cleanup deferred (stale config is LRU-capped).
* fix(model-picker): remove leftover sidebar-staging cogwheel + empty Model section
The downloaded-variant gear (ModelLoadSettingsAction) staged a model straight
into the right-sidebar Run-settings flow -- the old 'configure before load' path
now fully replaced by the in-picker config step. Removed the gear + its component.
Also gate the sheet's 'Model' section to staged picks only (pendingSelection):
after the load-knob strip its content is staged-only, so it was rendering an
empty section header whenever a model was merely loaded.
* chore(chat): remove dead per-model-config setters + modelControlsDisabled
After the load-config UI moved into the picker, the store's per-model setters
(setKvCacheDtype/setSpeculativeType/setSpecDraftNMax/setTensorParallel/
setCustomContextLength/setChatTemplateOverride) had zero callers
(applyPerModelConfigToRuntime writes via setState), and the sheet's
modelControlsDisabled was unreferenced. Verified dead across the whole tree.
* fix(chat): config-step Load actually loads (ignore Load-on-selection)
Root cause: with Settings > Chat > 'Load on selection' turned OFF, the config
step's load went down the deferred-staging path -- opening the right sidebar with
'<model> is staged, not loaded yet / Choose Load model'. The in-picker config step
IS the deliberate load action, so its Load now loads immediately (or downloads +
auto-loads when not cached) regardless of the toggle. Renamed the button
'Run model' -> 'Load model' to match. Native/dropped picks still honor the toggle.
* refactor(chat,hub): retire 'Load on selection' — config step is the only load flow
The in-picker config step (and the Hub Run button) now fully supersede the old
stage-to-sidebar flow, so the Load-on-selection toggle is removed everywhere:
- chat stageOrLoad: every pick loads immediately, or downloads + auto-loads when
not cached (the previous default behaviour, now universal).
- hub Run: drops the stage branch; downloaded GGUFs load directly with their saved
per-model config (no collision with the chat config step — both end at selectModel).
- store: removed loadOnSelection field/setter/key/default; Settings>Chat toggle and
its settings-reset entry removed.
- staged sidebar section is now a download-progress view (auto-loads on completion).
No manual staging remains; stageModel is used only for background auto-load downloads.
* feat(model-picker): default chat template from GGUF + thread variant through config flow
Read the embedded tokenizer.chat_template from GGUF files (read_gguf_chat_template
in gguf_metadata) and use it as the per-model default. Plumb gguf_variant through
the picker service, /api/picker/chat-template route, frontend templates API, and
use-model-defaults so the right variant's template is fetched.
Also refine the picker config-page/model-selector wiring, drop the dead
ggufNativeContextLength runtime path, and add the per-model-config storage keys to
the settings prefs export.
* feat(model-picker): read safetensors chat template + hide editor where it has no effect
Resolve the default chat template for safetensors models: prefer the modern
chat_template.jinja, fall back to the tokenizer_config.json chat_template field,
then chat_template.json (multimodal processor), then the GGUF embedded template.
Applied to local dirs, the HF cache snapshot scan, and the HF remote fetch.
Hide the chat-template editor in the picker for safetensors models — the override
is only applied at load by the GGUF/llama.cpp backend, so editing it on safetensors
currently has no effect. GGUF keeps the editor. Nothing removed; the dialog stays
for when the safetensors apply path is wired up in a later branch.
* fix(model-picker): set legacy-migration flag only after the write succeeds
Set unsloth_model_configs_migrated only once writeMap confirms the migrated
map persisted, so a quota/storage failure no longer marks migration done and
silently drops the user's pre-existing remembered settings — the next load retries.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* MVP model picker fixes
* MVP picker config fix
* MVP safetensors config
* MVP max seq config
* MVP max seq fix
* Fix static max tokens cap ignoring model context
* Fix picker GGUF scan parity
* fix(studio): harden model picker config loading
Apply remembered per-model configs consistently from picker and Hub loads, keep default configs from overriding standing speculative settings, add config access for direct local GGUF files, and support saving or forgetting active model settings without a reload.
* Fix model picker config flow
* Fix model picker config loads
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Avoid recursive per-model config migration reads
* Apply the displayed context length when loading a GGUF
* Fix template validation, cached template lookup, and failed load rollback
- Validate chat templates with the loopcontrols extension so templates
that use break or continue tags pass the picker validator, matching the
inference renderer that already accepts them.
- Read the default chat template from the newest cache snapshot rather than
an arbitrary iterdir order, so an older cached revision no longer prefills
a stale template.
- Capture the runtime per-model config before a load and reapply it when the
load fails, so a failed switch leaves the active model context, KV cache,
template, and speculative settings as they were.
* Make chat template view only for safetensors models
Custom chat template overrides are applied at inference only for GGUF
models, which pass the template to llama-server. The safetensors backend
renders with the model built-in template and ignores the override, so
editing it would save a value that never loads. For safetensors the
config page now opens the template as a read-only preview with a note
that editing is not available yet. This can become editable once
inference support for custom safetensors templates lands in main.
* Fix model picker config edge cases
- Restore prior runtime config when a load no-ops for the active model
- Cap the picker validator request body via the protected prefixes
- Keep the GGUF context slider max above the loaded context
- Fetch subfolder chat templates for uncached Hub repos
- Show the compare side config when reopening the picker
* Keep saved GGUF context above the fallback ceiling
* Show the model config in the run settings sidebar
* Fix model config sidebar reset and context slider
- Stack the remember toggle and action buttons in the sidebar
- Reset the config to defaults instead of the loaded values
- Fetch the native context so the slider max is not the loaded value
* Fix model picker config and download regressions
- Run picker chat template routes off the event loop
- Depth and root guard local template directory scans
- Restore download manager flow for uncached hub picks
- Apply per model context length on reload
- Import model picker symbols from the feature barrel
* Fix model picker config and cached download sorting
- Restore load settings when a Hub run is rejected mid load
- Reuse one NumericValueInput instead of a duplicate copy
- Fix double decode of the model name in the template route
- Remove the unused reset-to-loaded settings action
- Fix cached model download sorting
* Fix model picker per-model config edge cases
Honor a saved or typed max seq length above the model's native context so
RoPE extended values are no longer clamped and silently overwritten. Allow
typing past native while the slider keeps native as a soft ceiling.
Guard the fetch success paths in use-model-defaults against an aborted
signal, and refetch when the HF token changes.
Hash the chat template content in the sidebar remount key instead of its
length. Enable reset for a GGUF whose native context is unknown, and floor
the context slider max so it can never fall below the min.
* Fix GGUF context auto-fit and gated model config token
Stop forcing a 32768 context when a GGUF native context is unknown so the backend auto-fits to VRAM again, while still honoring an explicit context edit.
Send the HF token as a query param so gated safetensors models resolve their max position embeddings.
Derive model default state during render to drop the set-state-in-effect calls.
* Fix native GGUF context ceiling and guard picker template reads
Restore the native context store field so the sidebar slider keeps the
full ceiling for drag and drop GGUFs. Limit local chat template reads to
the browse allowlist, skip malformed repo ids, and drop unused model
picker exports.
* Fix model picker lint boundaries
* Fix model picker review findings
Chat template editor never seeded its draft. Radix only calls onOpenChange
from internal events, so the seed in the nextOpen branch was dead and a model
with a saved override opened empty. Saving then cleared the override. Drop the
dead branch, treat draft as an untouched sentinel, and reset it on every close.
Uncached Hub picks could auto load a model after the user left the chat. Main
detached the staged pick on route exit and on chat context change. Carry the
context key on the pending pick and skip the load when it no longer matches.
Also clear configTarget when the picker closes, restore the onUpdated ref so
variant rows stop resubscribing on every parent render, skip the LRU write when
the entry is already most recent, import NumericValueInput relatively, and drop
the unused ModelUpdateAction barrel export.
* Preserve GGUF context on active reload
* Fix model picker per-model config regressions
- Stop reloading the already loaded model on re-pick
- Hide infra models from the chat picker
- Detect vision support on cached GGUF repos
- Honor saved maxSeqLength on auto load
- Restore default chat template for local GGUFs
- Warn on save failure and revert config on cancel
- Refetch picker inventory on open
- Persist read only per model config safely
* Fix stale model auto load
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix model picker numeric input sizing and constraints
Size value inputs to their content so long context lengths are not clipped,
restrict them to numeric characters, and stop the speculative decoding label
from truncating in the sidebar.
* Fix picker CI tests and harden chat template resolution for PR #6647
- tests: point the descender guard at the moved model-selector.tsx path
- tests: exclude the disabled Reload model button from the regenerate locator so .first targets the real Regenerate
- picker/service.py: reject symlinked template/gguf leaves that resolve outside the browse allowlist (HF cache reads unchanged)
- compare mode: resolve each pane's own remembered chat template instead of inheriting the other pane's from the store
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Protect future-schema per-model configs from deletion for PR #6647
savePerModelConfig already refuses to overwrite a stored config whose schema version is newer than this client understands, but deletePerModelConfig did not. Unchecking Remember on an older client therefore silently destroyed a newer client's saved config. Apply the same guard on delete and surface the blocked case through the existing saveFailed toast.
* Protect future-schema per-model configs from quota eviction for PR #6647
The save and delete guards already refuse to touch a stored config whose schema version is newer than this client understands, but the quota-eviction path did not, so a full store on an older client could still evict a newer client's config. Skip future-schema entries when evicting and fail the save if the budget cannot be met without them.
* Fix GGUF context persistence, compare context, and rollback settings for PR #6647
Persist a GGUF context override from the user's intent instead of collapsing it against the loaded context, which reintroduced the context-reset (f4838782cb reverted the native-baseline fix). model-config-page now collapses the saved value against native, and use-chat-model-runtime and chat-adapter retain the requested context on load so re-saving another setting keeps the override; a null request stays null so a VRAM auto-fit never becomes a stored override.
shared-composer: a compare pane with no explicit GGUF context now loads at native (0) like single-view, not the session maxSeqLength that silently shrank the shown context.
use-chat-model-runtime: restore the previous model's KV cache dtype and chat template on a failed-load rollback so it runs as it was, not with backend defaults.
* Preserve native path token when reloading the active model for PR #6647
handleReloadActiveModel rebuilt the selection without the store's activeNativePathToken, so reloading a file-picked GGUF after a settings change validated the display label as a repo/path and failed. Thread the active native token through the reload selection so native-loaded models reopen correctly.
* Make picker template validation resilient and accept HF generation tags for PR #6647
Import Jinja lazily inside validate_chat_template so a backend without the optional jinja2 package (GGUF-only installs) still starts instead of raising ModuleNotFoundError at import time. Register a no-op extension for the Transformers {% generation %} assistant-mask tag so pasting a valid HF chat template validates, matching the renderer, rather than being rejected as an unknown tag.
* Honor remembered compare config and parse processor chat_template.json for PR #6647
* Fix failed-load rollback context and processor template map fallback for PR #6647
* Restore speculative decoding config on failed-switch rollback
When a model switch fails after the previous model was unloaded, the
rollback reload restored tensor_parallel, KV cache dtype and the chat
template override, but omitted speculative_type and spec_draft_n_max and
cleared their loaded shadows to null. The previous model therefore came
back running at backend defaults (speculation off) while the UI still
showed it enabled, and the status resync confirmed the off state. Resend
the previous model's speculative settings in the rollback load and keep
the store's active and loaded speculative fields in sync with them.
* Reset max sequence length when a model has no saved config
applyPerModelConfigToRuntime reset every per-model field except
maxSeqLength, which it only wrote when the incoming config had one.
maxSeqLength is the sole field carried on store.params, so selecting a
model with no remembered config left the previous model's value in place
and later loaded the new model at that leaked length. Fall back to the
standing default so an unremembered model loads at its own default.
* Surface a message when a variant update cannot start
startManagedUpdate handled the conflict and error start outcomes but let
busy fall through as if the update began, so the confirm dialog closed
with no job created and the cached variant stayed stale. Show an info
message when the repo is busy with a sibling transfer so the click is
not silently dropped.
* Keep per-model speculative choices out of the global default
A staged load with a per-model or one-off config sets keepSpeculative,
which already skips reading the global speculative preference. The
matching save still ran unconditionally, so the model-specific choice was
written to the global unsloth_chat_speculative_type and a later model with
no saved config started from it instead of Auto. Skip saveSpeculativeType
when keepSpeculative so the per-model choice stays isolated.
* Seed non-active model settings from the app default max length
The Run settings page captured initialMaxSeqLength from the loaded
model's runtime params and fell back to it for a model with no saved
config. Opening settings for a different, unloaded model and clicking
Load then sent the active model's context (for example 64k) instead of
the 4096 default, risking validation failures or OOMs. Seed the default
for non-active models and keep the runtime value only for the active one.
* Prefer sidecar tokenizer chat template over the GGUF copy for variants
_chat_template_from_dir returned the embedded GGUF template first when a
variant was selected, reversing the tokenizer-first precedence of the
no-variant path. A model whose chat_template.jinja or tokenizer_config.json
supersedes a stale embedded template then got the wrong template on
variant selection. Keep tokenizer files first regardless of variant; the
variant only picks which GGUF is the fallback. Adds regression tests for
both the tokenizer-wins and gguf-fallback cases.
* Keep per-model speculative choices load-local in autoload and compare
The interactive load path treats a per-model speculative choice as
load-local and skips writing it to the global default. Autoload and
generalized compare still called saveSpeculativeType unconditionally, so a
remembered off or ngram setting leaked into unsloth_chat_speculative_type
and later models with no saved config inherited it. Persist the global
preference only when the value came from the global settings.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: record the compare pane's loaded context in runtime state so the active model's settings and any reload or save use it, not the previous context
* Studio: notify the user when a Hub autoload can't start because another download for the model is already running, instead of silently dropping it
* Studio: drop the merge's orphaned staged-model store helpers and unused alert imports
The main merge left isPendingGguf and pendingSelectionMatches referencing the
removed PendingModelSelection type, and the alert-dialog/alert imports unused
after the permission-mode dropdown replaced the bypass dialog, so tsc -b failed.
* Studio: cache a null default chat template so the viewer stops re-fetching it
A model with no sidecar or embedded template resolves to a terminal null, but
that result was never cached, so reopening the template viewer re-ran the
backend and Hugging Face lookup every time.
* Studio: detect direct-file GGUFs in run settings so Max Tokens uses their context
A GGUF loaded from a local file or custom folder has no variant label, so the
run-settings panel treated it as non-GGUF and clamped Max Tokens to the session
max_seq_length instead of the loaded GGUF context. Detect it via the reported
GGUF context and the .gguf checkpoint suffix, matching the chat page.
* Studio: prompt to re-select a local model file when its lease expired before reload
A file-picked GGUF is reachable only through a native path token that the
desktop host prunes after a TTL. Reloading reused that token blindly, so a
reload long after the initial load failed with an opaque error. Track the
token's expiry and, when it has passed, ask the user to re-select the file
instead of attempting a doomed reload.
* Fix descender-clipping test to tolerate sidebar layout utilities
The sidebar account-block div carries layout utilities (min-w-0, flex-1)
between 'flex' and 'flex-col', so the descender-clipping guard's regex,
which required 'flex' immediately followed by 'flex-col', no longer matched
and the test failed to locate the account-block div. Generalize the prefix
to allow intervening flex utilities while still capturing the leading-*
class before the collapsible visibility utility and asserting leading-tight,
so the guard against clipped glyph descenders is fully preserved.
* Harden picker chat-template resolution
Enforce the 64 KiB chat-template contract at the validate endpoint's request
model so a direct caller cannot submit a template far larger than the frontend
allows (MaxBodyMiddleware only bounds the whole request body, not this field);
oversized templates now return a clean 422.
Apply sidecar-over-GGUF template precedence globally across cached snapshots
instead of per snapshot. A repo with multiple cached revisions previously
returned the first snapshot's template, so a newer GGUF-only revision could
win over an older revision's maintained chat_template.jinja sidecar, which
contradicted the documented intent that sidecars supersede the embedded copy.
* Guard per-model config against future-schema and lossy migration
Two forward-compatibility gaps in the versioned per-model config store:
- The load/apply path returned and normalized a stored record without checking
its schema version, so a record written by a newer client was reinterpreted
under the current schema and applied to a live model load, even though save,
delete and eviction all refuse to touch future-schema records. Reject
future-schema records on load too.
- The one-time legacy migration enforced the storage budget without protecting
the entries it had just migrated and set the completion flag unconditionally.
When storage was already full of future-schema records (which are unevictable
by an older client), the migrated entries were the only evictable ones and
could be dropped while migration was still marked complete. Protect the
migrated keys during eviction and only mark migration complete when they
survive, so it retries once space frees up.
* Discard chat-template validation results after the dialog closes
Server-side template validation is async, but closing or cancelling the editor
did not abort it, so a late-arriving valid response still called onSave and
applied a template the user had already dismissed. Track a validation token
that is bumped on close and ignore any validation result whose token is stale.
* Record native lease expiry when loading a picked GGUF from the chip
The pending-native-model chip loaded via stageOrLoad directly, bypassing
loadNativeModelIntent, so activeNativePathExpiresAtMs was never recorded for a
chip-loaded file. A later reload then either skipped the lease-expiry guard
entirely (expiry left null) or compared against a previously loaded file's
stale expiry, so reload could reuse an already-pruned token or wrongly block a
still-valid one. Route the chip through loadNativeModelIntent, which builds the
same selection and records the expiry.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Prefer sidecar template for a directly selected local GGUF file
A direct .gguf file path read its embedded chat template without checking the
parent directory for a maintained sidecar (chat_template.jinja /
tokenizer_config.json), while directory and variant selections already prefer
the sidecar. That let the config editor preview or save a stale embedded
template for the same model depending on how it was selected. Check the parent
directory sidecars first, then fall back to the embedded copy, and cover both
paths with tests.
* Resolve cached chat template per revision, newest first
The earlier change searched every cached snapshot for a sidecar before
considering any snapshot's embedded GGUF template, which let an obsolete sidecar
from an older revision override the newest revision's template. Restore
per-snapshot resolution (newest first): a revision's sidecar still supersedes
its own embedded GGUF copy, but a newer revision is no longer overridden by an
older revision's sidecar.
* Preserve autoload transport conflicts and surface background busy downloads
- When a Hub autoload hits a transport conflict, keep pendingHubAutoLoad bound
instead of clearing it. Clearing it re-keyed the download surface and its
cleanup cancelled the conflict the toast tells the user to resolve, so the
Hub resume affordance was gone the moment it appeared. Return early on
conflict, mirroring the started branch, so resolving it from the Hub still
auto-loads on completion.
- The background-download branch handled started and conflict but silently
dropped a busy outcome, leaving the user with no feedback when a peer variant
of the same repo was already downloading. Surface the same busy toast the
autoload path uses.
* Fix context length, GGUF template, fetch state and lease expiry bugs
Keep explicit context length values instead of collapsing to null at
native. The collapse made the slider jump back at the native maximum
and made Reload load the previous context instead of the chosen one.
Prefer the first split when resolving a GGUF without a variant. Later
splits carry no chat template metadata, so picking the largest file
could return no template for a sharded model.
Clear stale fetch state when template and metadata lookups retry, so
a previous terminal error is not shown while a new fetch is running.
Record native path lease expiry together with the token when a load
commits. The expiry was written by only one load path and even when
the load did not start, so a reload could be blocked with an expired
file message for a still valid token.
* fix(model-picker): resolve review findings across config, inventory, and templates
- Apply remembered per-model config in the training-compare chat handoff so a
prior model's customContextLength no longer leaks into the next load
- Match GGUF variant labels with the inventory extractor too, so cached
no-quant-token files resolve their default chat template
- Show "Auto" instead of a fabricated 32768 when native context is unknown
- Reuse the identical staged auto-load object on same-pick so a re-pick during
download pre-flight no longer disarms auto-load via "busy"
- Union supports_vision when deduping cross-cache inventory rows
- Serve hidden-model needles from a new GET /api/hub/hidden-models endpoint and
merge them client-side, covering runtime-configured RAG embedders
- Clamp GET chat templates to MAX_CHAT_TEMPLATE_BYTES (route + jinja sidecar),
matching the validate endpoint's contract
- Lower-clamp stored customContextLength to shared CONTEXT_LENGTH_MIN
- Wipe unsloth_chat_load_on_selection in Settings "Reset all"
- Drop stale pendingHasContext comment describing deleted staging machinery
* Fix stale defaults cache, token in query string and rounded up context ceiling
Refresh cached chat template and max position data when a model update
completes. Send the HF token for model config requests in the dedicated
header instead of the URL. Snap the native sequence length ceiling down
to the nearest step so the slider cannot exceed the declared maximum.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix compare pane reverting active checkpoint on non-GGUF load
Re-read runtime params after setCheckpoint so the fresh checkpoint is
kept instead of being overwritten by the pre-setCheckpoint snapshot.
* Send the HF token via header for the vision and embedding checks
checkVisionModel and checkEmbeddingModel still passed the HuggingFace
token as a ?hf_token= query parameter, so it landed in server access
logs, proxy logs, and browser history. Move them to the
X-Unsloth-HF-Token header like getModelConfig already does, and accept
the header on the check-vision and check-embedding routes with the
existing query parameter kept as a fallback for older clients.
* Cap the chat template on the model load path
The load endpoint accepted an unbounded chat_template_override, so a
direct caller could hand llama.cpp an arbitrarily large Jinja template
even though the frontend, the validate endpoint, and the read paths all
enforce the 64 KiB limit. Reuse MAX_CHAT_TEMPLATE_BYTES in the
LoadRequest validator, rejecting oversized templates with a fast
character-count check before the exact UTF-8 byte check.
* Protect existing per-model configs during legacy migration
When the one-time legacy import pushes the store over budget, eviction
now protects the entries the user already has and drops only the
just-migrated legacy entries, so importing old load settings can never
discard a newer per-model config.
* Reset clears the context override instead of pinning the native value
Reset wrote the discovered native context into customContextLength for
GGUF models, but isDefaultConfig treats any non-null customContextLength
as an explicit pin, so Reset with Remember enabled persisted a fixed
context and future loads stopped using the native auto context. Reset
now restores the full default (customContextLength null); the native
value is still shown through the existing display fallback.
* Bound chat-template sidecar reads to a size limit
The chat_template.json, tokenizer_config.json, and Hub-downloaded sidecar
readers decoded and json-parsed the whole file before the extracted
template hit the 64 KiB response cap, so an oversized metadata file could
exhaust memory. Read them through a bounded reader (4 MiB envelope) that
returns None when the file is larger, matching the existing chat_template.jinja
size guard. Adds tests for oversized tokenizer_config.json and chat_template.json.
* Keep the native-path token and lease expiry in sync
Rollback after a failed reload restored the previous token but left the
failed load's expiry in the store, so a later reload could be falsely
blocked as expired (token A paired with load B's lease). Restore the
previous lease alongside the token, and clear the expiry wherever the
token is cleared on a non-GGUF transition, so the two never diverge.
* Clear the native file lease on compare-pane loads
* Studio: add regression tests for the model-picker per-model-config
Guard the specific regressions that reverted the predecessor change:
- backend pytest (studio/backend/tests/test_model_picker_regression.py):
infra-model hiding, HF token via header with query fallback, and the
chat-template byte caps.
- source contracts (tests/studio/test_model_picker_contracts.py): the token
stays out of the URL, the context ceiling is floored, the native lease is
cleared on compare-load and restored on rollback, the default caches key on
the inventory version, and the hidden needles stay present.
- Playwright E2E (tests/studio/playwright_model_config.py) wired into
studio-ui-smoke.yml on port 18898: Context Length persists across a reload,
Reset clears the stored override, and infra models are absent from the picker.
- optional GPU-gated inference smoke (tests/studio/test_gpu_inference_smoke.py)
that auto-skips on GPU-less CI and stays short on a GPU.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: model pinning, row menus, hub inference settings, and inventory filters
Pinning
- Add a pinned models store (localStorage) with repo and per-quant pins
- Pinned section in the model selector's On Device list and the hub inventory,
with newest pins first so Pin to top lands on top
- Deleting a repo drops its pins
Row menus
- Replace loose row icons with a shared 3-dots menu (pin, reveal in file
manager, copy identifier, copy path, delete) on picker rows, hub quant rows,
the hub run bar, and on-device inventory rows
- Menus only render for models actually on disk; platform-aware reveal labels
- Backend: cached-model-path and reveal-cached-model endpoints resolving
managed HF-cache repos only
Hub inference settings
- Gear in the GGUF run bar opens an Inference settings dialog reusing the chat
page's controls: model config (context length, KV cache, speculative
decoding, chat template), system prompt, reasoning, sampling, tools and
retrieval
Inventory
- Model-type filter (text, vision, embedding, STT, TTS, diffusion) beside the
sort pill, both with a sort icon, capped widths and truncation so the
On device heading never wraps
- Unsloth-owned repos without an upstream provider logo fall back to the
Unsloth mascot avatar
- Discover / On Device tabs widened; hub search bar narrowed to match
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: revert the Unsloth mascot avatar fallback
Unsloth-owned repos without an upstream provider match go back to the
colored-initial tile, and unslothai is no longer a relabeled owner.
* Studio: run-bar options on single models, and aligned type/capability filters
- Give single-model (non-GGUF) run bars the same 3-dots options menu and
settings gear as GGUF, at repo level
- Drop Pin to top from the run-bar menus; pinning stays in the On Device list
- Add an Image to text (diffusion) capability with detection, and surface it
in both the hub Discover capability filter and the On Device type filter
- Align the On Device type filter with the Discover capability options and
share the same detection so both dropdowns match
* Studio: apply hub inference config on reload, eject action, and run-bar polish
- Fix inference settings not applying: the hub dialog now writes the config to
the runtime before reload, matching the chat page (selectModel reads runtime
state, not the selection)
- Order the settings gear before the 3-dots menu in the run bars
- Replace the loaded-model run-bar action (New Chat) with Eject, wired through
the inspector to the hub's ejectModel
- Truncate the results heading so a long search query clips instead of
overlapping the header pills in split view
- Use a plain magnifying-glass icon for the no-results empty state
* Studio: fix GPU settings loss, load guards, pins, filters, and cached paths
Reloading a model from the chat sidebar or the hub gear dialog rebuilt the
per-model config without the GPU memory fields, so manual GPU layers, MoE
placement, and the GPU pick were reset on every reload and could be saved
over a remembered config. The active config now comes from a shared
useActiveModelConfig hook that carries the GPU fields for GGUF models, and
the sidebar remount signature tracks them through a shared gpuFieldsSignature
helper.
The in-flight load guard lived in a ref inside each useChatModelRuntime
instance, so the chat page, hub page, and gear dialog could not see each
other's loads. A load started from the gear dialog left the hub page free to
eject the model mid-reload or start a second concurrent load. The runtime
store now records the loading pick, selectModel checks it across instances,
and ejectModel refuses with a toast while any load is in flight.
The cached-model-path endpoint matched GGUF files by basename and excluded
only mmproj, so Copy path and Reveal could return an MTP drafter for a quant
and returned 404 for directory layouts like BF16/model-00001.gguf. Variant
files are now resolved from snapshot-relative paths with the same drafter,
mmproj, and big-endian exclusions as the load path, shared through a new
_main_variant_gguf_label helper.
Hub and picker fixes:
- rename the diffusion capability label from "Image to text" to
"Image generation", since it detects image generators
- validate pinned quants through the cached variant listing, keep the last
verified set while revalidating, and drop deleted quants immediately
- pass a measured scroll margin to the on-device virtual list so rows past
the overscan stay visible below the pinned block
- keep the delete menu for stopped partial safetensors downloads
- give the inventory type filter a reset in Clear filters, a truthful empty
state with a Show all types action, and hide it on the datasets view
- order picker pinned rows by pin recency, include pinned matches in the
empty-state check, and sync pins across browser tabs
- count only the visible rows in the On device list header
Tests: contract checks for each fix in test_model_picker_contracts.py and a
backend test for the variant label selection.
* Studio: reveal cached models in Windows Explorer under WSL
The reveal endpoint only branched on macOS, Windows, and generic Linux.
Under WSL the Linux branch spawned xdg-open, which is missing on a stock
distro without a Linux desktop, so the request failed with a 500 and the
UI showed a failed to open file manager error.
WSL is now detected with the existing helper and the path is converted
with wslpath before opening explorer.exe, selecting the file the same
way native Windows does. Directories open directly. When interop is
unavailable the old xdg-open fallback still runs. The macOS, native
Windows, and native Linux branches are unchanged, and the Tauri app is
covered since its hub reveal calls this same local endpoint.
Tests: platform guards for the WSL reveal, the interop fallback, and
the unchanged native Linux behavior in tests/studio/test_reveal_file_manager.py.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Adjust model picker row spacing and cogwheel hover consistency
* Studio: exact hidden model ids and newest revision cached paths
A custom RAG embedder repo was published to the frontend as a basename
substring needle, so a generic name like org/model could hide unrelated
models in the pickers. The hidden-models endpoint now sends full repo ids
that are matched exactly.
Copy path and Reveal picked a GGUF variant from an arbitrary cache
revision when the same file existed in more than one. The newest revision
now wins, matching the whole repo lookup.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix model picker GPU config, metadata, and cache selection
Load each compare model with its saved GPU memory mode, GPU layers, CPU MoE layers, and selected GPU IDs. Reconcile saved GPU IDs with the current hardware. Include the active native GGUF path token in metadata checks. Search all Hugging Face cache roots when resolving cached models and select the largest visible cache entry. Remove obsolete barrel exports and the staging-only GPU memory helper.
* Studio: hide hub inference settings gear for now
The cogwheel in the hub download cards is out of scope for this PR. The
dialog component stays in place and a TODO marks where the button
returns in a future PR.
* Refresh hidden model matchers
* Fix GGUF detection, compare context pin, and picker delete staleness
Treat any pick with a GGUF variant as GGUF in selectModel so the first
load after downloading an uncached quant validates and sizes with the
right GPU settings instead of unloading the current model on a wrong
preflight. Variant picks now also set isGguf on their selection meta.
Stop compare panes from inheriting the active model's context pin when
their own saved config says Auto. Null context in a remembered config
now means no pin, matching how the pane settings are shown.
Route picker deletes through the hub inventory client, which
invalidates the HF cache scan and the variants cache. The legacy
delete route left the scan cache warm, so deleted models reappeared
in the picker until the TTL expired. Removed the now unused legacy
delete client and updated the contract test to match.
* Studio: fix stale GGUF load-marker ordering test
The load-in-flight marker still precedes the hub-download guard and the
unload, but the llama_extra_args inheritance that used to sit between the
marker and the guard now runs ahead of the GGUF branch, so it is no
longer a landmark inside the sliced source. Drop it from the ordering
assertion and keep the marker -> guard -> unload invariant.
* Studio: fix per-model config edge cases in compare loads and saved defaults
- chat-settings-sheet: gate the MTP fallback note and context/VRAM warning on
the broader isGguf (variant, loaded gguf context, or .gguf suffix) instead of
isLoadedGguf, so direct-file and custom-folder GGUF loads still surface
those diagnostics.
- shared-composer: a compare pane's context now comes from its own config only
(a saved pin, else null for Auto/native). It no longer inherits the active
model's shared snapshot, which resolveFitMaxSeqLength treated as an explicit
pin and could load a pane at another model's context (VRAM/OOM), matching the
single-model load path.
- model-config-page: when an auto-fit GGUF is saved with fixed GPU layers
(Manual) and Remember, pin the displayed fitted context so a later fresh load
keeps the placement instead of sending native/0 and recreating the OOM.
- per-model-config: treat Auto GPU memory mode and Auto/default speculative type
as follow-global defaults; do not persist them as per-model overrides so later
global preference changes keep applying.
* Studio: gate vision capability on GGUF projectors and bound remote template downloads
- cache_inventory: only mark a cached repo vision-capable when it holds an actual
GGUF mmproj projector, not any file whose name merely contains "mmproj" (e.g.
mmproj_config.json), matching the runtime's GGUF-only projector detection.
- picker/service: pre-check the remote file size before downloading an uncached
repo's chat template / tokenizer config, so a maliciously large sidecar is
skipped instead of fetched and retained in full, mirroring the size gate the
local-file path already applies.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: add source-contract guards for the per-model-config edge-case fixes
Guard the four per-model-config fixes against silent regression in CI:
- local GGUF diagnostics gate on the broad isGguf, not the variant-only isLoadedGguf
- fixed-layer GGUF saves pin the displayed context
- Auto GPU mode and Auto/default speculative are not persisted as per-model overrides
- a compare pane's context comes from its own config, not the active model's snapshot
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: clear manual GPU knobs on Default and resolve local embedders before repo-id
- model-config-page: switching GPU Memory back to Default now clears the Manual-only
knobs (gpuLayers/nCpuMoe/selectedGpuIds); otherwise a remembered config kept stale
pins that a later load re-applied when the global GPU preference was Manual, despite
the page showing Default.
- routes/models hidden_model_matchers: resolve an existing local path before the repo-id
regex, mirroring is_hidden_model, so a local embedder shaped like "models/embedder" is
hidden by exact path instead of leaking as a chat model.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: add _is_mtp_drafter to the model_config stub in the export-paths test
routes/models.py imports _is_mtp_drafter from utils.models.model_config at module
load, but the lightweight stub in test_export_absolute_paths.py did not provide it,
so loading the module under the stub raised ImportError on Backend CI. Add the stub.
* Studio: read a picked GGUF's chat template through the native path lease
The picker chat-template GET has no native-path-lease plumbing, so a
desktop-picked (drag-drop) GGUF could not show its default chat template
in Run Settings until the model was loaded: the endpoint only receives
the display label, not the leased file path.
Read the embedded template through the existing lease-aware
/api/inference/validate probe instead. A new include_chat_template flag
resolves the granted canonical path and returns the GGUF's own embedded
template, never a sibling sidecar (the grant authorizes just that one
file); it skips the training guard like include_context_length and is
bounded by MAX_CHAT_TEMPLATE_BYTES. The frontend fetch mints a one-shot
validate-model lease when a native token is present and keeps the plain
GET path for HF and allowlisted local models.
Adds backend and source-contract regression tests.
* Studio: call worker.direct_wheel_url in the ROCm wheel-url test
The ROCm Mamba/SSM test referenced worker.py's private _direct_wheel_url,
but the worker imports the wheel helper under its public name
direct_wheel_url (utils.wheel_utils). When the worker module loads (its
imports resolve in CI), worker_mod._direct_wheel_url raised AttributeError;
the test only masked it by skipping when the worker could not be imported.
Call the name that actually exists so the assertion runs; it still returns
None for an empty cuda_major (ROCm).
* Studio: reset max sequence length to the app default, not the loaded value
For a non-GGUF active model, the per-model config seeds maxSeqLength from
the loaded runtime value so the panel opens showing the running context.
Reset set config.maxSeqLength to null, but the null fallback resolved back
to that captured runtime value, so the field kept showing the old custom
length and the config saved/reloaded it again. A remembered or active
max-length override therefore could not be cleared from Run settings.
Fall the null/default case back to the app default (clamped to the model's
native ceiling) instead of the active runtime snapshot, so Reset actually
clears the override. The initial view is unaffected: an active model's
config.maxSeqLength is already non-null, so it still shows the loaded value.
Adds a source-contract regression guard.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: persist default max length, refresh deleted quants, hide non-chat locals
Three follow-up fixes from review of the per-model-config picker:
- Max sequence length: the persisted per-model record now keeps config's
maxSeqLength (null after Reset) so isDefaultConfig can clear a remembered
override; the resolved app-default is substituted only into the load
request, never the saved record. Previously Reset saved the concrete
default and left the model pinned/remembered.
- GGUF variant expander: deleting a downloaded quant from a repo that still
has other cached quants now bumps the expander refresh key, so the removed
quant stops showing as downloaded and clickable (which would try to reload
the deleted file) until the repo is collapsed and reopened.
- Local picker rows: require capabilities.canChat before listing a local
models-folder / LM Studio row. A weightless folder (only config.json) is
classified non-chat, and toLocalModelInfo drops capabilities, so selecting
such a row would try to load a path the inventory already marked non-chat.
Adds source-contract regression guards for all three.
* Fix compare-pane and Reset context defaults in model picker
Two related per-model-config default regressions:
- A non-GGUF compare pane with no saved maxSeqLength fell back to the
active model's shared runtime snapshot, so comparing a saved 128K model
against an unconfigured pane loaded the latter at 128K and could OOM. It
now falls back to the shared app default (DEFAULT_MAX_SEQ_LENGTH), the
same fallback the single-model config path uses.
- contextAtDefault treated an explicit customContextLength equal to the
native ceiling as a default, which wedged the Reset button disabled for
a deliberate pin-to-native. It now counts as default only when there is
no override at all.
DEFAULT_MAX_SEQ_LENGTH becomes a single exported constant in
per-model-config.ts so the single-model config and the compare path share
one source of truth. Adds source-contract guards for both fixes.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Skip over-cap remote Jinja templates so the tokenizer template wins
The remote chat-template resolver bounded raw chat_template.jinja downloads
only by MAX_TEMPLATE_METADATA_BYTES (4 MiB), then returned the first
non-empty Jinja unconditionally. The picker route drops any template larger
than MAX_CHAT_TEMPLATE_BYTES (64 KiB), so an uncached repo whose
chat_template.jinja sits between 64 KiB and 4 MiB returned no template at
all, even when a valid smaller tokenizer_config.json template existed. The
local path already skips oversized .jinja files and falls through.
Gate the extracted Jinja on MAX_CHAT_TEMPLATE_BYTES and continue searching
when it exceeds the cap, matching _chat_template_from_jinja_file. The 4 MiB
download bound stays for JSON files that merely embed a small template. Adds
a regression test that a big Jinja plus a valid tokenizer config resolves to
the tokenizer template.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Guard legacy per-model-config migration idempotency
The v1->v2 localStorage migration (unsloth_load_settings ->
unsloth_model_configs) runs on every store read, so it must migrate exactly
once and never re-run, duplicate, or clobber a newer per-model config on a
reload or restart. That was covered only by a manual proof, so add durable
guards:
- Source-contract test pinning the three idempotency layers (the in-memory
legacyMigrationChecked guard, the persistent unsloth_model_configs_migrated
flag set in every terminal branch, and the non-overwriting Object.hasOwn
merge-skip) plus the readMap invocation. Reddens if any layer is dropped.
- Playwright model-config E2E: promote the legacy-migration step to a gating
check (soft_fail, which gates under the CI STUDIO_UI_STRICT=1) that the
migrated value is preserved and the flag is set, then reload again with a
fresh legacy seed present and assert the stored key set is unchanged, so a
second reload cannot re-migrate, duplicate, or clobber.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Note the migration E2E now gates idempotency under STUDIO_UI_STRICT
* Tighten model-picker per-model-config code comments
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
Co-authored-by: shimmyshimmer <shimmyshimmer@users.noreply.github.com>
Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local>
Co-authored-by: oobabooga <112222186+oobabooga@users.noreply.github.com>
* Raise the opencode invoke timeout in Local Agent Guides CI
The connection (opencode) cell flakes with a 600s timeout reported as guide drift, but it is not a hang: in a passing run the same opencode run finishes in ~482s (08:12:31 to 08:20:33), right against the shared AGENT_INVOKE_TIMEOUT of 600s, so about one run in six drifts past the cap.
opencode is the slow outlier. The print-mode agents (claude -p, codex exec) run one turn against a minimal injected system prompt, while opencode run runs its own full turn with opencode's large system prompt plus a separate small_model call to name the session (start.py pins small_model to the same 4B the server hosts). On a CPU-served gemma-4-E4B that is about 8 minutes, leaving no margin under 600s.
Double opencode's per-invoke timeout in agent-guides-drive.sh and keep the tight 600s cap for the fast agents, so a genuine headless-TTY hang still fails quickly. 1200s stays well under the 40-minute job budget.
* Normalize the agent invoke timeout before doubling it for opencode
Strip an optional trailing 's' from AGENT_INVOKE_TIMEOUT so the opencode
arithmetic, and the "${TIMEOUT}s" timeout message, stay valid if a
timeout(1)-style suffix is ever configured.
* Only double the opencode timeout for a bare-integer seconds value
Guard the arithmetic so a GNU timeout(1) duration suffix (s/m/h/d, including
floats like 0.5s) is passed through unchanged instead of breaking the
expansion; timeout(1) parses those directly. Bare seconds still double.
---------
Co-authored-by: danielhanchen <unslothai@gmail.com>
* Replace standalone Studio wording with Unsloth
Replace the single word Studio with Unsloth wherever it is used as
shorthand for Unsloth Studio in docs, CLI output, UI strings, i18n
locales, workflow display names, comments and docstrings.
Kept unchanged: the full name Unsloth Studio, third party product
names (LM Studio, Visual Studio, Mac Studio), feature names
(Recipe Studio, Fine-tuning Studio and its translations), and all
identifiers such as env vars, commands, paths and filenames.
* Address review feedback on the Studio wording rename
Use "an" before Unsloth where the rename left the article as "a".
Restore the split brand where Unsloth and Studio render as two halves
of the full product name: the onboarding sidebar subtitle and the
IPv6 localhost warning. Scope two messages to the full name Unsloth
Studio where plain Unsloth was misleading: the AMD README bullet and
the CLI studio setup error.
* fix(tokenizer): check for tokenizer.model after saving it, not before
`fix_sentencepiece_tokenizer` creates its temporary directory, then returns
early unless that directory already contains a tokenizer.model:
if not os.path.exists(temporary_location):
os.makedirs(temporary_location) # fresh, empty
if not os.path.isfile(f"{temporary_location}/tokenizer.model"):
return new_tokenizer # always true
old_tokenizer.save_pretrained(temporary_location) # writes that file
The file only appears on the line after the check, so the guard is always
true and the body never runs. Nothing else writes that path either --
`convert_to_fast_tokenizer` saves into a per-name subdirectory, not
`{temporary_location}/tokenizer.model`.
Both call sites are in `get_chat_template` and are commented "Must fix the
sentence piece tokenizer since there's no tokenizer.model file!" -- the
guard defeats the exact intent the caller states. The effect is silent: the
caller still gets a working `new_tokenizer`, but the sentencepiece piece
rename is skipped, so the mapped token (e.g. the eos token remapped to
`<|im_end|>`) is missing from tokenizer.model and GGUF/llama.cpp exports
carry the old piece.
`check_if_sentencepiece_model` in save.py does the same probe in the right
order -- makedirs, save_pretrained, then isfile. Match it.
Tests are added under tests/saving/ next to the existing sentencepiece
coverage, and to the two Bucket-A lists in consolidated-tests-ci.yml, since
Repo tests (CPU) --ignores tests/saving and these need protobuf.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Clear stale tokenizer.model before the sentencepiece guard
The guard now runs after old_tokenizer.save_pretrained, but the default
temporary_location is a fixed reusable directory. A fast-only tokenizer writes
no tokenizer.model, so a stale file from an earlier sentencepiece call could
pass the guard and patch the wrong model (e.g. mixing models in one process,
like a long-running server). Remove any existing tokenizer.model first, and add
a regression test.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Empty the reusable sentencepiece scratch directory each call
The final AutoTokenizer.from_pretrained reloads the whole temporary_location, so
removing only a stale tokenizer.model still let other artifacts from a previous
tokenizer (added_tokens.json, chat template, etc.) leak into the reload when the
default reusable directory is used across models in one process. Recreate the
directory instead, and add a regression test for the leaked-artifact case.
* Clear only top-level scratch files, keep subdirectories
Recreating the whole reusable directory deleted the {name} subtree that
convert_to_fast_tokenizer stores a converted tokenizer's source vocab in, so
old_tokenizer.save_pretrained could not copy tokenizer.model and the guard
returned the tokenizer unpatched for those legacy converted tokenizers. Remove
only stale top-level files (all the final reload reads) and leave subdirectories
intact. Add a regression test for the converted-source subdirectory.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Keep the current tokenizer's own source vocab when clearing
On a repeated get_chat_template(map_eos_token=True) call, the returned tokenizer's
vocab_file points back at the top-level tokenizer.model, and the cleanup deleted
that source before old_tokenizer.save_pretrained could re-emit it, so the guard
returned the tokenizer unpatched. Skip removing the old tokenizer's own source
vocab while still clearing stale files from a different tokenizer, and add a
regression test.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Use a per-call temporary directory for the sentencepiece fix
The scratch directory defaulted to a single shared path, so concurrent or repeated
get_chat_template(map_eos_token=True) calls could delete or overwrite each other's
tokenizer.model between save and reload (tripping the piece assertion or reloading
the wrong model), and stale files from an earlier tokenizer could leak into the
reload. Work in a unique per-call subdirectory instead: this isolates every call
without deleting anything the caller owns, and replaces the earlier per-file cleanup.
Tests updated to read the patched model from the reloaded directory and to cover
isolation and source-vocab preservation.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Pass only the applied token mappings into the sentencepiece fix
get_chat_template mirrors token remaps into tokenizer.model via
fix_sentencepiece_tokenizer, but two caller paths passed a mapping that did not
match what they wrote to the fast tokenizer JSON, so once the sentencepiece patch
runs the model and JSON disagree:
- the mapped-token path skipped entries whose target already existed but still
passed the full mapping, renaming a piece the JSON never changed;
- the EOS-swap path swapped both tokens in the JSON but passed only one direction,
leaving two stop_word pieces and no old EOS piece.
Pass the applied mapping (and both swap directions) instead. Add regression tests.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten sentencepiece guard comments
* Add SPDX license identifier to sentencepiece guard test
* Reclaim the per-call sentencepiece scratch directory
The per-call tempfile.mkdtemp fixed the shared-directory race but never cleaned
up, so a long-running process leaked one scratch dir per call. The dir cannot be
deleted eagerly for sentencepiece tokenizers because the returned tokenizer's
vocab_file points into it (a later save_pretrained copies the patched
tokenizer.model from there). Reclaim it correctly instead: remove the dir right
away on the fast-only path (the returned tokenizer never references it), and
attach a weakref.finalize so the sentencepiece dir is removed once its tokenizer
is garbage collected. Add regression tests for both.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Tighten the scratch-dir reclaim comment
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: danielhanchen <unslothshared@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
The permission-levels feature defaults an unset permission_mode to ask on
streaming requests, so the headless smoke probes hang at the approval
prompt until the job timeout. Declare permission_mode full in the tool
probe bodies; the gate itself is covered by unit tests.
* Single-pass GGUF export for direct outtypes + parallel multi-quant
save_to_gguf defaulted first_conversion to model_dtype before the block
that picks the optimal base conversion, leaving that block dead since it
landed (#3356). Every default export (fast_quantized -> q8_0) therefore
ran two passes: convert HF -> 16-bit GGUF, then llama-quantize -> q8_0,
writing a 2x-size intermediate that the cleanup step deletes again.
- Route single-output exports whose type convert_hf_to_gguf.py emits
directly (f32/f16/bf16/q8_0) through one conversion pass with no
16-bit intermediate. Measured on Qwen2.5-0.5B-Instruct (8-core CPU):
bytes written 1525 MB -> 531 MB (2.9x less), peak extra disk 994 MB
-> 0, wall time neutral on local NVMe (14.8s vs 15.4s). The dequantized
q8_0 tensors are bit-identical to the two-pass output (max diff 0 over
all 290 tensors, same quant-type table). On disk-capped runtimes
(Kaggle 20 GB, Colab) the removed intermediate is the difference
between an export that fits and one that dies - see the Kaggle error
text this file already carries. imatrix runs keep the two-pass route
since only llama-quantize can apply one; explicit first_conversion is
still honored.
- Run independent llama-quantize passes two at a time when several
quant methods are requested (thread budget split between workers,
outputs byte-identical, order preserved). Measured 1.38x wall-clock
on q4_k_m+q5_k_m+q6_k. Sequential under UNSLOTH_ENABLE_LOGGING=1 to
keep subprocess logs readable; kill switch
UNSLOTH_PARALLEL_GGUF_QUANTS=0. Duplicate methods now quantize once.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Guard parallel GGUF quant on Kaggle and make multi-quant failures atomic
Each llama-quantize pass loads the whole model into RAM, so running two at once on Kaggle can OOM a host that succeeded sequentially; skip the parallel path there. On a failed multi-quant export, stop launching queued passes and remove orphaned quant outputs so a failure leaves no partial GGUFs behind, keeping the 16-bit base for retry. Also accept 0/false/no/off/empty for UNSLOTH_PARALLEL_GGUF_QUANTS so a well-meant 'false' actually disables parallelism, and add tests/saving/test_gguf_single_pass_export.py to the CI saving bucket so the new tests run.
* Preserve pre-existing outputs for canceled quant passes on failure
The parallel cleanup unlinked every requested output name, so a failed rerun could delete a valid model.<METHOD>.gguf left by an earlier successful export for a method whose pass was canceled and never ran this session. Skip canceled futures and only remove outputs from passes that actually executed.
* Gate parallel GGUF quant on available memory and preserve prior outputs
Skip the two-worker path when RAM cannot hold two full-model quantizations at once (and on Colab as well as Kaggle), so a multi-quant export that fit sequentially no longer OOMs. On failure, remove only outputs this run newly created, tracked against a pre-launch snapshot, so a rerun into an existing _gguf directory never deletes a valid artifact from an earlier export.
---------
Co-authored-by: djs <dschroers2@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
* Studio CI: make tool-calling SSE probes resilient to transport stalls
* Studio CI: bound tool-probe SSE stalls to the job timeout and stop accepting partial tool events
* Studio CI: surface HTTP errors from the seed loop and bound the Mac best-effort probes
* Studio CI: keep completed tool results on a stall and cap seed reads by the remaining budget
* CI: retry transient HTTP timeouts in Studio smoke probes
The post() helper in the Studio inference smoke workflows does a single
urlopen with a 240s timeout against the local Studio server. On shared
runners this sporadically hits TimeoutError while the server is stalled,
failing the whole job for a transport hiccup; the same flake has recurred
across unrelated PRs on Linux and Windows (JSON/images and tool-calling
jobs) and passes on rerun.
Retry the probe up to 3 times on transport-level failures only
(TimeoutError, ConnectionError, non-HTTP URLError), 15s apart. HTTP
status errors still surface immediately, so genuine server failures are
unaffected. post_sse() is left unchanged: it has a 600s budget and has
not flaked.
* CI: retry only short probes so worst case fits the job budget
Some json-images calls pass timeout=600; three attempts there could spend
30 minutes in one step and hit the job's timeout-minutes instead of failing
with the Python error. Retry (3 attempts) only when timeout <= 300s, which
covers the observed flaky 180-240s probes; longer probes keep the pre-PR
single attempt.
* CI: give long smoke probes one capped retry
Round two of bounding the retries: timeout>300s probes previously got a
single attempt, so a transient stall in the 600s JSON-mode probes still
failed on first occurrence. Give them one retry with the attempt timeout
capped at 300s. Worst cases stay inside timeout-minutes: 240s probes
12.5 min, one 600s probe 15.25 min, the Windows JSON job's two long
probes 30.5 min against its 35 minute budget.
* scripts: refresh scan_packages allowlist baseline
Regenerate scripts/scan_packages_baseline.json against the current
resolved dependency set so the blocking pip scan-packages gate matches
what the scanner now finds. Refreshes evidence hashes for benign
findings whose code shifted lines (unsloth-zoo mlx loader, gguf/mlx
test /tmp fixtures) and adds two mainstream-library entries that were
newly surfaced (torch inductor codecache base64+subprocess compile
cache, torch testing common_utils socket import). Stale entries whose
matching code changed and no longer triggers are dropped.
All entries remain CRITICAL/HIGH findings manually judged benign;
matched on (package, file, check, evidence_hash).
* ci(security-audit): re-run scan when the allowlist baseline changes
The security-audit pull_request trigger listed the scanners but not
their allowlist baselines, so a baseline-only edit never re-ran the
scan that consumes it. A refreshed baseline could therefore merge
without CI confirming its evidence hashes match what the scanner finds.
Add scan_packages_baseline.json and scan_npm_packages_baseline.json to
the paths filter so baseline changes are validated on their own PR.
* unsloth start: add --resume to persist and reopen agent sessions
`unsloth start <agent>` launches a coding agent whose home is a throwaway
temp dir wiped on exit, so codex/openclaw/hermes/pi (which relocate their
whole home there) cannot resume a conversation after you quit. opencode and
claude keep their session data in a fixed user dir, so they already resume.
Add an opt-in --resume/--no-resume flag: it routes the launch to the stable
Unsloth agents dir (the same one --no-launch already uses) so the session
survives the exit, never touching the user's own ~/.<agent>. A bare --resume
also reopens the last conversation via the agent's native flag (codex
`resume --last`, opencode/claude/pi `--continue`). The default is unchanged:
a plain launch still uses a temp dir and persists nothing.
Add a dispatch-only `resume` job to the Local Agent Guides CI that drives the
real launch path and asserts the split: codex/pi are wiped without --resume
and persist with it, while opencode/claude persist either way.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* unsloth start: rename --resume to --persist
The session flag collided with agents' own resume flags. `unsloth start
claude --resume <id>` used to forward `--resume <id>` straight to Claude
(which keeps its history in ~/.claude regardless), so a boolean --resume on
unsloth start would have swallowed the session id and turned it into a stray
prompt. Name the persistence flag --persist instead, so every agent's native
resume flag (claude --resume <id>, codex resume, opencode --continue, ...)
still passes through untouched. Behavior is otherwise identical: --persist
keeps a launched agent's session under the Unsloth agents dir, and a bare
--persist reopens the last conversation.
Add a regression test that `--resume <id>` passes through verbatim, and in the
CI resume experiment skip the redundant second pass for opencode/claude (they
persist either way, and a second CPU turn only risks a timeout).
* unsloth start: correct --persist help and drop the buggy auto-resume
Reword the --persist help to be accurate: claude and opencode keep sessions in
the user's own stores and resume regardless, so --persist only stabilizes the
otherwise-ephemeral relocated home of codex/openclaw/hermes/pi. Drop the
bare-launch auto-append of native resume tokens: it errored on a first launch
with no prior session, and was inconsistent between launch and no-launch.
--persist now only keeps the session dir; resume via the agent's own command
(e.g. `unsloth start codex --persist resume`), which now finds it.
In the CI resume experiment, fail the pass when the launched turn exits
non-zero, so a write-then-error is not misread as PERSISTED.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Stabilize floating monitor drag
* Restore floating monitor exit animation
* Harden Windows Studio smoke checks
* Keep API menu badge removed
* Apply no-build-tools env overrides in-script
The runner does not apply step-level env keys containing parentheses,
so ProgramFiles(x86) kept its real value and Find-VsBuildTools still
detected VS through vswhere. Set the overrides inside each pwsh step
instead; child processes inherit them. The resolver step moves to pwsh
because bash cannot export a variable named ProgramFiles(x86).
* Reset chat UI session without a second browser context
macOS runs Chromium with --single-process, where closing the last
context tears down the whole browser, so the shutdown re-login died
with TargetClosedError on new_page. Clear cookies and swap pages
inside the same context instead, opening the replacement page before
closing the old one.
* Keep the no-build-tools Path filtered across session refreshes
install.ps1's Refresh-SessionPath and setup.ps1's Refresh-Environment
rebuild the session Path from the Machine and User registry scopes, so
the process-level filter could be undone mid-install and re-expose
CMake. Filter those scopes in the Prepare step with normalized dir
matching and restore them in cleanup.
* Drop stale localStorage auth tokens before re-login
Auth tokens live in localStorage, not cookies, and the login guest
guard redirects on their mere presence. Remove them during the session
reset so the /login navigation is deterministic instead of relying on
the tolerated redirect bounce.
* Studio: source CPU llama.cpp prebuilts from the unslothai fork
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: reject unknown Linux CPU arches and keep ROCm-tooling hosts off the CPU prebuilt
* Studio: extend the resolve-prebuilt ROCm-tooling guard to Windows
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: let ROCm-SDK-only CPU hosts take the fork CPU prebuilt
* Studio: accept windows-arm64 prebuilt kind and refresh stale fork-routing comments
* Studio: correct stale fork-routing comments and --resolve-prebuilt help
* Refresh stale ggml-org routing comments
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* fix: launch OpenClaw local TUI by default
* Fix/adjust OpenClaw launch paths for PR #6937
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Default OpenClaw to the local TUI only on a bare invocation
The first-arg startswith('-') branch rewrote passthrough globals into a broken
command: OpenClaw's grammar is openclaw [--dev] [--profile <name>] <command>, so
'unsloth start openclaw --profile test' became 'openclaw tui --local --profile
test', but tui does not accept --profile (or --dev), so the invocation failed.
A leading '--flag value' is ambiguous between a global (--profile test) and a tui
option (--message hi), so it cannot be reinterpreted safely. Default to the local
TUI only when no passthrough args are given, and forward everything else verbatim
so OpenClaw parses it under its own grammar. The bare-launch default (the point
of this change) is preserved; explicit subcommands and global flags pass through.
---------
Co-authored-by: wasimysaid <112766706+wasimysaid@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <danielhanchen@gmail.com>
Co-authored-by: Wasim Yousef Said <wasimysdev@gmail.com>
* version-compat CI: fake CPU training runs for SFT/GRPO/DPO
Adds a runtime layer on top of the patch-run canary: actually runs
trainer.train() for a couple of steps on a CPU-only runner under the CUDA
spoof, wrapping a plain tiny HF model in the Unsloth-patched trainer. Exercises
the real train() loop (collation, generation, the injected
_get_per_token_logps_and_entropies, loss, backward, optimizer) so a TRL or
transformers change that breaks the loop at runtime -- not just the source
structure -- surfaces here. No GPU, no meaningful numerics.
Needs a chain of small CPU shims (eager torch.compile, dynamo suppress, cuda
tensor-alloc redirect to CPU, model.for_training/for_inference equivalents)
documented inline. Does not exercise Unsloth's Triton/GPU kernels (CPU can't).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* cpu fake-train: force adamw_torch + disable dynamo for CPU runner
On a real CPU-build torch runner (GitHub CI) two things bit that a CUDA-build
torch with GPUs hidden masked locally:
- The default optimizer is adamw_8bit (bitsandbytes), whose is_on_gpu() check
dies on CPU tensors. Force optim=adamw_torch in all three configs.
- import unsloth reinstalls the real torch.compile over the eager passthrough,
so the GRPO hot path (chunked_selective_log_softmax) actually compiles and
inductor picks the spoofed CUDA device, crashing on device props
(gcnArchName). Re-apply the eager passthrough after import and flip
torch._dynamo.config.disable so every @torch.compile runs eager at call time.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* cpu fake-train: write checkpoints under pytest tmp_path
Use pytest's tmp_path for each trainer's output_dir instead of a hardcoded
relative temp/ci_* path, so a local pytest run does not leave untracked dirs in
the repo tree and the tests are CWD-independent.
* version-compat CI: disable dynamo at process level for the fake-run job
Set TORCHDYNAMO_DISABLE / TORCH_COMPILE_DISABLE in the fake-run step env so
dynamo/inductor is off before conftest.py's early import unsloth, not only via
the per-test runtime shim. Defense in depth on the GPU-less runner: the GRPO
hot path never compiles regardless of when its functions were decorated.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix PEFT replacement for TRL >= 1.7.0, add missing compute_aux_loss for TRL 1.7.0
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix GRPO for TRL >= 1.7.0: PEFT ref-adapter removal and return arity
rl.py: for trl >= 1.7.0, scope the PEFT removal regex to the ref-adapter
block only by anchoring the end on ref_param.data.copy_(param.data), so it
no longer also deletes the following gradient-checkpointing
enable_input_require_grads() block. Neutralize TRL 1.7.0's
`if _is_quantized_model:` bf16 cast the same way the existing
is_loaded_in_4bit cast is handled.
rl_replacements.py: initialize _extra_moe_kwargs before use (it was
referenced before assignment whenever compute_aux_loss was passed) and only
request output_router_logits when the aux loss is actually wanted.
rl_replacements.py: _get_per_token_logps_and_entropies now returns a 3-tuple
(logps, entropies, aux_loss) for trl >= 1.7.0 and a 2-tuple for older TRL,
matching how every TRL call site unpacks the result. Without this, TRL 1.7.x
_generate_and_score_completions unpacks 3 values from a 2-tuple and raises
"not enough values to unpack (expected 3, got 2)".
* Return zero aux_loss placeholder and drop inference-mode aux collection
* GRPO TRL >= 1.7.0: reject router aux-loss opt-in at init; drop zero aux placeholder
Unsloth's optimized GRPO forward cannot compute the MoE router auxiliary loss.
Previously an explicit opt-in (router_aux_loss_coef > 0) returned a fabricated
zero, silently training without the requested load-balancing penalty. Now reject
it at trainer init with a clear NotImplementedError, and return None (not zero)
for the aux slot of TRL's 3-tuple. Default stays off (coef 0), so the common
path is unaffected.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* GRPO hidden-states fallback: free ModelOutput before chunked log-softmax
The old/ref logprob fallback binds the full ModelOutput (which holds every
layer's hidden_states when output_hidden_states=True) and kept it alive across
chunked_hidden_states_selective_log_softmax, an avoidable OOM on large models.
Extract logits then del outputs in both the text and VLM branches.
* Version-compat CI: proactively catch TRL GRPO breakage
The existing TRL canary is a static symbol/source grep: it verifies symbols
exist but is blind to structural changes (TRL 1.7.0's 2->3-tuple per-token-logps
return arity and restructured PEFT ref-adapter block, which the fix in this PR
addresses, both slipped past it because the methods still existed).
Two additions:
- test_trl_grpo_pinned_symbols.py: extend TRL_TAGS to 1.5/1.6/1.7 and pin the
exact source-string contracts the rl.py / rl_replacements.py transforms depend
on for TRL >= 1.7.0 (PEFT elif ref-adapter block + enable_input_require_grads
survival, if _is_quantized_model, aux_loss_enabled anchor, compute_aux_loss
arity). A future TRL change fails on main a few days before the PyPI release.
- test_trl_grpo_fake_run.py + a version-compat-ci job: fake-CUDA run that drives
the real GRPO/SFT/DPO source-transform patchers against latest + main TRL on a
CPU-only runner (no training) and asserts the generated Unsloth trainer still
satisfies the transform contracts. Catches behavioral regressions the grep
cannot see.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fake-run test: use a normal Version import for the aux gate
* version-compat CI: fix fake-run job gate + torch-absent collection
- Drop the invalid job-level matrix if (matrix is not available in
jobs.<id>.if -> 'Unrecognized named-value: matrix' fails the whole
workflow). Use a single job that runs vs TRL latest always and re-runs
vs TRL main only on schedule/dispatch via a step-level github.event_name
guard. Validated with actionlint.
- Module-level skip the fake-run test when torch is absent so
daily-fresh-fetch (pytest-only, collects tests/version_compat/) does not
crash on the top-level spoof import.
* fake-run test: do not skip on import failure
unsloth/trl are installed in the grpo-fake-run job, so a failing import is the
import-time drift this canary must catch. Keep only the not-installed find_spec
skips; let a real import error fail the test.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* GRPO arity gate: regex downgrade + fail loud + CI coverage
The TRL < 1.7.0 per-token-logps return downgrade was an exact-string replace
anchored on the full return line incl. its comment, so a reformat (e.g.
pre-commit) could silently no-op it and ship a 3-tuple to older TRL. Switch to
a regex tolerant of comment/whitespace drift, and raise if the anchor stops
matching (re.subn count != 1) instead of failing silently. Add a monkeypatched
trl_version unit test asserting both arities, since CI only installs TRL >= 1.7.0
and never exercised the downgrade otherwise.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fake-run: give SFT/DPO a real contract, not just ast-parse
The SFT/DPO fake patch runs only checked the generated trainer parses. Also
assert the shared QLoRA _is_quantized_model bf16 cast is neutralized (TRL 1.7's
spelling, present in both sft_trainer and dpo_trainer), so a structural TRL
change to that block is caught for SFT/DPO too, not just GRPO.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* GRPO PEFT ref-adapter removal: lower gate to the TRL 1.4.0 floor
The elif is_peft_model(model) and args.beta != 0.0: ref-adapter block was
introduced in TRL 1.4.0 and is unchanged through 1.7.x, but the removal was
gated at >= 1.7.0, so for 1.4 <= TRL < 1.7 the transform fell through to the
0.27 branch (which matches the older if is_peft_available()... form) and
silently no-oped: a PEFT + beta != 0 GRPO run then computed the KL reference
from the copied ref adapter instead of the base model. Lower the gate to
1.4.0 and keep the 1.7.0-only router aux-loss fail-fast nested. Widen the
pinned-symbol contract test to run from 1.4.0 so the covered versions are
actually exercised.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Clear stale yolo approval state on no-launch reruns
The no-launch session config dir is deliberately reused across runs, but
the config writers only ever added the --yolo auto-approval settings and
never removed them. After one --yolo --no-launch run, every later run
without --yolo kept OpenClaw's tools.exec security=full/ask=off policy
plus exec-approvals.json, and OpenCode's permission allow block, so tool
execution stayed silently pre-approved.
Non-yolo runs now reset that state: OpenClaw drops the exec policy keys
and the yolo defaults in exec-approvals.json (approvals OpenClaw itself
recorded are kept; the file is removed when only the yolo payload is
left), and OpenCode drops the permission block. Launch mode is untouched
since it already uses an ephemeral temp dir.
* Strip only yolo-written values on non-yolo cleanup
Match each field against the exact value the yolo path writes before
removing it, so a stricter exec policy, approvals defaults set by the
user or the OpenClaw UI, and deny/ask OpenCode permission entries all
survive a plain no-launch rerun. An unparseable exec-approvals.json is
left in place, matching how an unparseable config is handled.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Write a prompting policy on non-yolo instead of deleting to a permissive default
OpenClaw and OpenCode both treat an omitted policy as permissive: OpenClaw's
effective exec policy for an unset tools.exec is security=full/ask=off on the
gateway host, and OpenCode defaults an unset permission to allow. So clearing
the yolo values on a non-yolo run did not restore prompting, it fell back to
those permissive defaults and left tool execution auto-approved.
A non-yolo run now writes an explicit prompting policy: OpenClaw gets
security=allowlist/ask=on-miss (verified to prompt even with the approvals file
removed, since the stricter of config and approvals wins), and OpenCode gets
edit/bash/webfetch=ask. Only a permissive/yolo value is tightened; a stricter
deny (or an ask the user set) is preserved, and the yolo approvals defaults are
still stripped. The file-edit CI path opts opencode/openclaw into --yolo, since
those agents now prompt by default and the headless test needs auto-approval.
* Respect existing exec mode, sandbox/node host, and global permission rules on non-yolo reset
The non-yolo reset for openclaw/opencode assumed an omitted policy was the
permissive yolo default and rewrote it, which corrupted or weakened stricter
setups it should have preserved:
- OpenClaw tools.exec.mode is the normalized policy knob and cannot be combined
with explicit security/ask (OpenClaw rejects the whole config), so writing
security+ask alongside a mode:deny/ask policy both broke the config and
relaxed it. Leave a mode-based policy untouched.
- host=sandbox defaults to security=deny and host=node routes to a paired node;
neither is written by --yolo (which only writes host=gateway). Treating the
missing security as full and popping host broadened those into gateway/auto
exec. Only rewrite a gateway-routed permissive policy, and never pop a
non-gateway host.
- OpenCode permission can be a string ("deny") or a {"*": ...} catch-all.
The old code dropped a string form and overrode a catch-all by writing
per-tool ask, weakening a stricter user rule. Now a string is left in place,
a catch-all governs absent tools, and only an effective allow is tightened.
- The non-yolo ask policy only lived in OPENCODE_CONFIG, which loads below
project opencode.json, so a project config allowing edit/bash/webfetch still
auto-approved. Carry the ask policy in OPENCODE_CONFIG_CONTENT (above project
config) too, symmetric to how yolo carries its allow.
Also harden the openclaw path against a malformed non-dict tools value.
Adds tests for mode/sandbox/node hosts, string and catch-all permissions, and
the inline ask policy over a project config.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Scope non-yolo resets to the exact yolo fingerprint and preserve granular denies
OpenClaw: reset only the exact host=gateway/security=full/ask=off policy --yolo
writes, so an omitted or host=auto/sandbox/node policy (which can resolve to a
sandbox security=deny default) is no longer broadened to allowlist/on-miss, and a
deliberate tools.exec.mode is left alone (OpenClaw never migrates our security/ask
write into a mode).
OpenCode: carry a granular object or a deny inline verbatim so a per-tool user rule
is not collapsed to a blanket ask, but floor any object that grants allow anywhere to
the string ask (which fully replaces a project object) so no inline allow pattern can
leak through into a silent auto-approve on a non-yolo session.
* Stop overriding project config on non-yolo; require full approvals fingerprint
The non-yolo OpenCode reset carried a session permission in
OPENCODE_CONFIG_CONTENT, which outranks the project opencode.json we
cannot read. That inline override could not correctly reflect the project:
it weakened a project deny to a prompt, mishandled global string rules,
leaked through a granular object's permissive default when no catch-all
was present, collapsed an object with an allow (losing its deny), and
missed per-agent permissions. All of these stem from forcing a value over
an unknown project config.
A non-yolo run now only undoes what --yolo wrote: it flips our own
explicit per-tool allow back to ask in our config file and carries no
permission inline, so the project's own permissions are honored as
written. Clearing our persisted yolo state is the actual fix; --yolo still
carries its allow inline so it works over a project config.
OpenClaw approvals cleanup now strips the yolo defaults only when the full
fingerprint (security=full, ask=off, askFallback=full) is present, so a
mixed user policy that merely shares askFallback=full (whose omitted
default is deny) is kept intact.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
The setup.ps1 unit-tests job intermittently fails on the windows-latest
runner with 'No repository with the name PSGallery was found.' when the
default PowerShell Gallery is not registered, so Set-PSRepository throws
before Pester can be installed. Register the default gallery first when it
is missing, then set its policy and install Pester as before.
* Studio: expose full compressed-tensors scheme set in an export formats dropdown
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: multi-select export formats, portable torchao FP8/INT8, GGUF LoRA, source parity
Export page overhaul on top of the formats dropdown:
- Unify merged precision into one sorted multi-select list (16-bit first, then
8-bit, then 4-bit). Drop "vLLM" from labels, add INT8 (W8A8), INT8 (W8A16),
INT4 (W4A16), MXFP4, MXFP8. Quick formats render as toggle pills; the rest live
in a multi-select "More formats" dropdown, so several formats export in one run.
- Add a portable torchao FP8/INT8 save path (Float8WeightOnlyConfig /
Int8WeightOnlyConfig) that needs no NVIDIA GPU to produce and loads in vLLM.
FP8 serializes to safetensors, INT8 to .bin. Wired into save_pretrained_merged
and push_to_hub_merged via a TORCHAO_EXPORT_SCHEMES registry and
_unsloth_save_torchao, parallel to the compressed-tensors path.
- Hide NVIDIA-only compressed-tensors formats when no NVIDIA GPU is present; keep
16-bit and portable FP8/INT8. The backend also rejects a compressed request on
non-NVIDIA hardware so it stays authoritative.
- Relax merged export to non-PEFT models so Local Model and Hugging Face sources
get the same 16-bit / compressed / portable options.
- GGUF: send the whole quant list in one call (merge once, quantize many).
- LoRA: add a GGUF adapter option (convert_lora_to_gguf.py) with an outtype
select (f16/bf16/f32/q8_0/auto), alongside the safetensors adapter.
- Thread the new fields through models, routes, orchestrator, and worker; extend
the export tests.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: gate export by accelerator with a torch-aware reason; fix export save dir naming
Export runs through Unsloth, which requires a compute accelerator (NVIDIA/AMD/Intel
GPU or Apple MLX) and has no CPU code path, so a bare-CPU host cannot export even
with PyTorch installed. Add export_capability() in utils/hardware that reports
export_supported plus a precise reason so the UI stops showing a generic "no GPU":
- pytorch_not_installed: a --no-torch install (even a physical GPU is unusable)
- no_accelerator: PyTorch present but no supported accelerator (bare CPU)
- mlx_unavailable: Apple Silicon where the MLX stack is missing or too old
Expose the fields on /api/system/hardware and /api/system, and guard the mutating
export routes (load-checkpoint, export/merged|base|gguf|lora) with HTTP 400 and the
reason, leaving read-only endpoints usable so the Export page still renders.
Make core/export/export.py import without PyTorch and without a usable accelerator
(the Unsloth import is caught) so the export worker degrades to a clear message
instead of crashing at import.
Frontend: keep /export reachable on chat-only hosts and gray out the method and
format options with the backend reason (Alert plus disabled MethodPicker) instead
of silently redirecting to /chat, so users see why export is unavailable.
Also fix the export save directory producing "model/null" for Local Model and
Hugging Face sources that have no run/checkpoint, naming the folder from the model id.
* CI: validate Studio export capability gating on Linux, Windows and macOS
Add a small pytest matrix that runs studio/backend/tests/test_export_capability.py
on ubuntu-latest, windows-latest and macos-latest. It confirms, on each real OS,
that hardware.export_capability() reports the right decision and reason
(pytorch_not_installed, no_accelerator, or mlx_unavailable) and that the export
backend imports without PyTorch and degrades to a clear message instead of crashing.
Hosted runners have no GPU/MLX, so this covers the "export unavailable, here is why"
path a Mac/Windows user without an accelerator sees; a real accelerator export is
validated separately. The job installs only a CPU PyTorch plus the backend import
deps (no unsloth, triton, or llama.cpp), so it runs in seconds with no GPU.
* Studio export: address Codex review (source-aware gating, GGUF LoRA token/MLX/guard)
Frontend (export-page):
- Gate LoRA and quantized-model restrictions on the active source. isAdapter /
isQuantized come from the selected checkpoint; in Local Model / Hugging Face
("model") source mode they were stale, so LoRA stayed wrongly enabled for a
direct base model (backend then rejects "No adapter to export") and a stale
"quantized" flag disabled every method for an unrelated, exportable model. Add
effectiveIsAdapter / effectiveIsQuantized (false outside checkpoint mode) and use
them in the method-reset effect and the MethodPicker disabled state.
- Hide the GGUF LoRA option on a macOS/MLX host (the backend rejects GGUF LoRA on
MLX), so users no longer pick it, wait through the load, and always fail. Disable
the "GGUF adapter" button on a Mac host and never send loraGguf there.
Backend (core/export/export.py):
- Pass the HF token into the GGUF LoRA conversion (save_pretrained_gguf), so a
gated/private base model's config fetch in convert_lora_to_gguf.py is
authenticated; without it the load can succeed but the conversion fails.
- Guard the save_pretrained_gguf capability check with getattr so an older Unsloth
model that lacks the method returns the clean "not supported" message instead of
an AttributeError that surfaces as a generic 500.
* Studio export: address 2nd Codex review (CI index, empty merged, test import)
- studio-export-capability-ci.yml: add --extra-index-url https://pypi.org/simple to
the torch install so torch's transitive deps still resolve; --index-url alone
replaces PyPI with only the CPU wheel index, which does not serve all of them.
- export-page handleStart: reject an empty merged selection (mirrors canExport), so
clicking the panel's Start button with every precision pill deselected no longer
submits mergedSelections: [] and launches an unintended default 16-bit export.
- test_export_imatrix_compressed: the torchao-registry test now reads unsloth/save.py
as text (like the other ast/string checks) instead of `import unsloth.save`, which
raised ModuleNotFoundError in the CPU studio-backend suite that has no unsloth
installed.
* Studio export: make comments succinct across the export changes
* Studio export: use load token for local GGUF LoRA export of gated bases
* Studio export: harden portable torchao path and gate multi-format Hub push
torchao (_unsloth_save_torchao):
- merge to an isolated temp staging dir so a co-selected 16-bit output at save_directory is not deleted
- narrow VLM detection to vision_config / ForVisionText2Text so T5/BART/Whisper are not misrouted
- forward trust_remote_code (from auto_map) to the reload so custom-code models export
Export UI:
- hide portable torchao formats on macOS/MLX (backend rejects quantized export there)
- restrict a Hub merged export to a single format (each writes to the repo root)
* Studio export: torchao tokenizer remote-code + XPU offload, scale GGUF timeout
torchao (_unsloth_save_torchao):
- honor auto_map in the staged tokenizer/processor configs (not just model.config) when
deriving trust_remote_code, so custom-code tokenizers reload after the merge
- offload single-device XPU models to CPU (and empty the XPU cache) before the reload, matching
the CUDA path, so an Intel GPU that fits the model once does not OOM on the second copy
Export orchestrator:
- scale the GGUF wait timeout by the number of requested quants so a multi-quant list export of a
large model does not time out at a flat 3600s
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio export: show portable torchao formats only on non-NVIDIA (CPU) hosts
Portable torchao FP8/INT8 is the fallback for hosts without the NVIDIA compressed-tensors path.
On an NVIDIA GPU the compressed-tensors FP8/FP4/INT formats are the intended path (llm-compressor
auto-installs), so hide the portable duplicates there; keep them on CPU / non-NVIDIA hosts and
continue hiding them on macOS/MLX.
* Studio export: report all output folders and the exported formats
- Multi-format merged export now collects every sibling output directory (one per selected
precision) instead of only the last; the success banner lists them all.
- Show the selected precision formats in the run summary (a Formats row, like GGUF Quantizations),
so the panel says what is being exported rather than just 'Merged Model'.
- Persist the selected formats in the run summary and seed them on mount, so navigating away and
back (or toggling the export method) restores the selection instead of resetting to 16-bit.
* Studio export: list all output formats, add GGUF LoRA target, default Q8_0, auto-select newest checkpoint
- Progress/summary panel now shows a Formats row with the selected merged
formats, and the success banner lists every output folder a multi-format
merged run creates (one line per format) instead of only the last one.
- Merged format selection is seeded from the active run, so navigating away
and back (or switching method cards) no longer resets it to 16-bit.
- GGUF / Llama.cpp now offers an Export target toggle (Full model or LoRA
adapter) for adapter checkpoints, reusing the LoRA GGUF export path.
- Removed the Auto GGUF LoRA output type and defaulted to Q8_0 in the UI,
the request model, and the backend defaults; the outtype list is now
Q8_0/F16/BF16/F32. Core save.py still accepts auto for external callers.
- When a finetune has no checkpoint selected, auto-select the newest one.
* Studio torchao export: robust reload class + optional VLM import
Two fixes to the portable torchao FP8/INT8 export reload, from review of the
narrowed VLM detection:
- Encoder-decoder seq2seq checkpoints (T5/BART/Whisper) are not causal LMs.
With the narrowed is_vlm test they now correctly skip the image-text class,
but fell through to AutoModelForCausalLM and failed to reload after the merge.
Reload them with their own architecture class from the config instead.
- AutoModelForImageTextToText was imported unconditionally at the top of the
torchao path, so on Transformers builds without that class the import aborted
every torchao export (even text-only). Import it lazily only for a VLM, with
the AutoModelForVision2Seq fallback used elsewhere in Unsloth.
* Studio: enable FP8/FP4 compressed export for newer-transformers models
The shipped llm-compressor 0.10.x pins transformers<=4.57.6, so FP8/FP4 export failed
for models needing a transformers 5.x sidecar (Qwen3.5, Gemma-4, Qwen3-Next): the
quantization subprocess crashed importing the removed TORCH_INIT_FUNCTIONS.
Run the quantization against a dedicated llm-compressor-main "shadow": a --target
package dir (transformers 5.10.2 + llm-compressor main + compressed-tensors) layered
over the existing torch. It installs --no-deps so torch is never touched (works on any
Studio torch build), is provisioned lazily and fingerprint-cached, and can be turned
off with UNSLOTH_DISABLE_LLMCOMPRESSOR_MAIN.
- transformers_version.py: provision + validate .venv_llmcompressor.
- export.py: route all compressed exports through the shadow when available; else keep
the workspace 0.10.x path and fail fast past its transformers ceiling.
- save.py: launch _compressed_quantize.py with a clean PYTHONPATH = shadow.
- _compressed_quantize.py: skip linear_attn / vision tower / MTP modules (matches the
RedHatAI and NVIDIA reference quants, and is required by the grouped schemes).
Verified all four schemes (fp8, w8a8, w4a16, mxfp4) on Qwen3.5-9B and Llama-3.2-1B, and
fp8 on Gemma-4, end to end through Studio.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix GGUF LoRA export tests
* Fix export CI expectations
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: wasimysaid <112766706+wasimysaid@users.noreply.github.com>
Co-authored-by: Wasim Yousef Said <wasimysdev@gmail.com>
* replaced connect with start
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix
* Studio: build the coding-agent command from the selected server
The API keys panel showed a hardcoded `unsloth start claude`. `unsloth start`
defaults to 127.0.0.1:8888 and only mints a key for a loopback server, so a
non-default port or a tunnel/remote base would target the wrong server or fail
to mint. Build the command from the panel base/key (and emit a key for
non-loopback), matching the other snippets in the panel.
* CLI: keep `unsloth connect` as a hidden alias for `unsloth start`
Avoids breaking existing scripts and docs that still call `unsloth connect`.
* Tests: stub _unstarted_cleanup in same-task disconnect test
The test builds _SameTaskStreamingResponse via __new__, so set the attribute
that __call__ now reads.
* Match coding-agent command loopback check to the CLI 127.0.0.0/8 rule (#6613)
* Keep unsloth_cli.commands.connect importable as a deprecated shim (#6613)
* Format the new coding-agents panel strings and import per biome (#6613)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Drop the unsloth connect alias and shim; unsloth start is the only command (#6613)
* Route unsloth connect to unsloth start as a hidden backward-compatible alias (#6613)
* Forward unsloth run model-load flags to unsloth start (gguf-variant, context-length, load-in-4bit, tensor-parallel) (#6613)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Session-scope coding agent config in unsloth start
Configure each agent for the current session instead of writing the Studio endpoint, key, and default model into the user's own config. Codex, OpenCode, OpenClaw, and Hermes get a private config relocated through their config-path env vars (CODEX_HOME, OPENCODE_CONFIG overlay, OPENCLAW_CONFIG_PATH plus OPENCLAW_STATE_DIR, HERMES_HOME). Claude Code suppresses the attribution header for the session via the CLAUDE_CODE_ATTRIBUTION_HEADER env var plus a --settings overlay, with no ~/.claude write. --launch uses an ephemeral temp dir removed after the agent exits; --no-launch uses a stable Unsloth-owned dir and prints the matching export lines.
* Read relocated agent session config in Local Agent Guides CI
The contract crosscheck and the openclaw/hermes patch helpers now read each agent's config from the relocated path printed by unsloth start --no-launch (CODEX_HOME, OPENCODE_CONFIG, OPENCLAW_CONFIG_PATH, HERMES_HOME) instead of fixed home paths. The Claude attribution A/B toggles the header for the session only (shipped-config HIT vs vanilla MISS) instead of editing ~/.claude/settings.json.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Skip the POSIX-only --no-launch parser test on Windows
test_no_launch_output_is_parseable mirrors the #6547 bash CI parser, which greps export/unset lines and only runs on Linux/macOS runners. On Windows --no-launch prints PowerShell ($env: / Remove-Item), so the export-line assertion does not apply there. Cross-OS staging CI surfaced this.
* Size Claude Code's auto-compact window to the loaded model's context
Claude Code auto-compacts against its native (~600k token) window, so against a smaller local model it overflows the server's context (silent truncation) long before it compacts. Set CLAUDE_CODE_AUTO_COMPACT_WINDOW to the loaded model's real context length (the value codex/openclaw already get via model_context_window / contextWindow). Omitted when the model reports no context length.
* Pin OpenCode/Hermes context window and set 90% compaction across agents
Feed every agent the server-determined sequence length (the value /v1/models reports from runtime_context_length) and a ~90% compaction threshold. OpenCode: a custom-provider model with no limit defaults to context 0, which silently disables auto-compaction, so set limit.context/output and scale the compaction buffer to 10% of the window. Hermes: pin model.context_length (it otherwise falls back to a 256k default when the server's /v1/models omits the field) and set compression.threshold 0.9. Claude: add CLAUDE_AUTOCOMPACT_PCT_OVERRIDE=90 alongside the window. Codex (model_context_window) and OpenClaw (contextWindow) already carried the window and auto-manage off it.
* Add `unsloth start pi` recipe
Pi was the only agent without a built-in recipe, so the agent-guides CI
hand-wrote ~/.pi/agent/models.json. Add a first-class `pi` command mirroring
the others:
- write_pi_config writes the session-scoped OpenAI-compatible provider config
(key in the config, like openclaw/opencode).
- pi() launches `pi --provider unsloth --model <id>` (Pi defaults to the google
provider, so the provider/model are pinned on the command line) with HOME
relocated for the session. Pi has no config-dir env var and resolves ~/.pi off
$HOME, so HOME-scoping keeps the user's ~/.pi untouched.
Migrate the agent-guides CI off the hand-written config onto the
`unsloth start pi --no-launch` path (connection + file-edit), with a crosscheck
for the provider api, so the documented recipe is exercised.
* Harden unsloth start for Windows and WSL agent launches
Address the Codex review on PR 6613:
- write_pi_config now pins the loaded contextWindow and a sane maxTokens so Pi
compacts instead of overflowing a small Studio context (it otherwise assumes
its 128000 default), matching the other agents.
- pi() sets USERPROFILE (and HOMEDRIVE/HOMEPATH when present) alongside HOME on
native Windows, where Node resolves ~/.pi via USERPROFILE rather than HOME, so
the session no longer reads or writes the user's real ~/.pi.
- The WSLENV bridge flags path-valued vars with /p so a Windows npm shim under
/mnt receives translated paths, while scalar vars (the numeric context window)
pass through untranslated. WSLENV is deduped on the bare name.
- _print_env prints the launch command with PowerShell-safe quoting so the inline
--settings JSON survives copy-paste on native Windows --no-launch.
Add tests for the WSLENV path flagging, PowerShell quoting, the Pi context
window, and the Pi USERPROFILE relocation.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Set CLAUDE_CODE_NO_FLICKER for the Claude session
A local server streams in bursts, so Claude Code's full-screen TUI redraw
flickers between tokens. Disable it for the session via CLAUDE_CODE_NO_FLICKER,
alongside the other CLAUDE_CODE_* session env knobs.
* Add a normalized --yolo flag routed to each agent's auto-approve mode
It is easy to forget which agent spells "run tools without prompting" which way,
so `unsloth start` now accepts all three spellings as one option (--yolo,
--dangerously-skip-permissions, --dangerously-bypass-approvals-and-sandbox) and
routes to the agent's own mechanism:
- claude: --dangerously-skip-permissions
- codex: --dangerously-bypass-approvals-and-sandbox
- hermes: --yolo
- pi: --approve (Pi's only approval gate is project trust)
- opencode: a permission allow block in opencode.json (no CLI flag exists)
- openclaw: tools.exec security=full / ask=off / host=gateway (no CLI flag exists)
Because the option is parsed by `unsloth start`, the "wrong" spelling for an
agent still routes correctly instead of leaking through to the agent and erroring.
IS_SANDBOX is deliberately left unset for Claude so its root/sandbox safety gate
still applies. Adds routing, cross-routing, and per-config tests.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix review findings: IPv6 loopback command, pi USERPROFILE under WSL, yolo guard
From a 10-reviewer pass over the PR:
- studio/frontend agent-command.ts: normalize bracketed IPv6 hosts. URL.hostname
returns "[::1]" for http://[::1]:8888, which never matched the "::1" loopback
checks, so the copied command embedded the placeholder API key for a local IPv6
server instead of the bare auto-minting command. Now [::1] is treated as loopback
like the CLI's is_loopback_url, so the command matches the CLI contract.
- pi(): also relocate USERPROFILE (and HOMEDRIVE/HOMEPATH) when running under WSL
against a /mnt Windows shim, not just on native Windows. Windows Node resolves
~/.pi via USERPROFILE, and the WSLENV bridge translates the path, so pi no longer
falls back to the user's real ~/.pi in that case.
- _yolo_command_flags: use .get so a config-based agent (or a typo) yields no flag
instead of a latent KeyError.
Adds tests for the WSL pi USERPROFILE relocation, the yolo unmapped-agent guard,
and that opencode/openclaw --yolo stays config-only (no argv flag).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix round-2 review findings: WSLENV /p upgrade, agent help text
- _merge_wslenv now upgrades a user's pre-existing unflagged WSLENV entry (e.g. a
bare HOME or USERPROFILE) to the path-translated form (HOME/p) instead of leaving
it as-is, so a Windows agent shim under WSL receives the translated session path
rather than the raw Linux path.
- Generalize the `unsloth start` registration help to list all six agents (was only
"Claude Code, Codex").
Adds a test for the WSLENV unflagged-entry upgrade.
* Fix round-3 review findings: complete openclaw --yolo, refresh stale copy
- openclaw --yolo now also writes the host approvals file (exec-approvals.json with
defaults security=full / ask=off / askFallback=full) alongside the tools.exec
config. OpenClaw gates tool execution on both layers (the stricter wins), so the
config alone could still leave it prompting or denying. Mirrors `openclaw
exec-policy preset yolo`. ask=off means nothing is ever prompted, so the runtime
socket block is unnecessary.
- Studio API panel copy: clarify that a local server auto-mints the key while a
remote one embeds it in the command, and add pi to the swap hint.
- Local Agent Guides CI: drop the stale "pi has no start.py recipe" note now that
all six agents are driven via `unsloth start <agent> --no-launch`.
Adds the openclaw approvals-file assertions and a no-yolo openclaw test.
* start: parse claude --version with a regex so a format change does not drop optimization flags
* start: offer to install a missing agent (prompt then run its install command)
* start: auto-start a Studio server for --model when none is running, and stop it on exit
* inference: surface an actionable message when llama-server cannot compile a tool grammar
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix review findings: kill the auto-started server tree on Windows; apply the tool-grammar message to the OpenAI passthrough too
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* start: split --model org/repo:variant so a running session is not evicted
`unsloth start <agent> --model org/repo:QUANT` failed against an already-running
Studio server and, worse, killed whatever model another session had loaded.
/v1/models lists a loaded GGUF under its bare repo id (e.g. unsloth/Qwen3-1.7B-GGUF),
so _resolve_model never matched the `:QUANT`-suffixed request. It then POSTed
/api/inference/load with model_path=org/repo:QUANT, which (a) Hugging Face rejects
("Repo id must use alphanumeric chars, '-', '_' or '.'") and (b) evicts the model the
other session was using, so a second 'unsloth start' in a new tmux/terminal tore down
the first. Re-running the command then attached to the now-empty server, which is why
it 'worked the second time'.
Mirror the org/repo:QUANT -> org/repo + --gguf-variant QUANT shorthand that
'unsloth run' and llama.cpp already accept, splitting it in _connect before we match or
serve. Matching now resolves against the loaded bare repo id (no spurious reload, no
eviction), and any real load uses a valid repo id plus gguf_variant. An explicit
--gguf-variant still wins; local paths and Windows drive letters pass through untouched.
The auto-serve path likewise spawns 'unsloth run --model org/repo --gguf-variant QUANT'.
* start: harden auth-key handling, codex teardown, and CI transcript redaction
Three review findings:
1. CI could leak a live key. agent-guides-drive.sh printed the raw
'unsloth start --no-launch' transcript (which carries export UNSLOTH_API_KEY /
ANTHROPIC_AUTH_TOKEN lines) to the Actions log on both the failure path and the
success path before redact() ran. Add cat_redacted() and use it for those two
prints, so the key is scrubbed on the way to the log while the on-disk file stays
intact for the env parsing that follows.
2. Outages masqueraded as bad keys. _key_accepted caught a broad Exception and
returned False, so a 5xx or timeout while checking a cached key looked like a
rejection: it discarded a good key and minted extra ones (local) or reported 'no
saved key' (remote). Only treat HTTP 401/403 as a rejection; let other errors
propagate so a real outage surfaces.
3. Codex preflight could leave the auto-started server up. _require_gguf_for_codex
runs after _connect may have auto-started Studio but before _run installs its
teardown finally, so a preflight rejection (e.g. a transformers-backend model) left
the server holding the port/GPU until the atexit backstop. Tear it down explicitly
at the point of failure.
Tests: a 5xx on a saved key surfaces without minting; a non-GGUF codex preflight
tears down the auto-served server.
* start: fix IPv6/portless studio URLs, Pi config-dir isolation, and Pi install recipe
Four review findings:
1. Pi ignored the session config when PI_CODING_AGENT_DIR was already set. Pi's
getAgentDir() reads process.env.PI_CODING_AGENT_DIR before falling back to
$HOME/.pi/agent, so a value inherited from the user's shell sent Pi to their real
config and skipped our provider/key (the HOME relocation alone was not enough). Pin
PI_CODING_AGENT_DIR at the session's .pi/agent dir; it is path-valued so the WSL
bridge translates it automatically.
2. Pi install hint dropped Pi's documented --ignore-scripts. Pi's README installs with
'npm install -g --ignore-scripts @earendil-works/pi-coding-agent' and notes it needs
no install scripts, so accepting the prompt now follows that safe recipe.
3. Auto-start ignored a portless UNSLOTH_STUDIO_URL. unsloth run binds to
'parsed.port or 8888', so http://127.0.0.1 launched the child on 8888 but the health
poll (and the returned base) still used port 80, stalling until the startup timeout.
Normalize the base to host:8888 (IPv6-safe) before starting and polling.
4. API-panel command mistook IPv6 loopback for the bare default. The bare 'unsloth
start' only probes 127.0.0.1:8888 on the IPv4 stack, so http://[::1]:8888 must carry
an explicit UNSLOTH_STUDIO_URL. Drop ::1 from the bare-default host set while keeping
it a loopback host (URL emitted, no key needed).
Tests: PI_CODING_AGENT_DIR is set to the session dir; _effective_base normalizes
portless/IPv6 bases; a portless UNSLOTH_STUDIO_URL auto-serves on :8888.
* start: apply fresh-review findings across CLI, CI, and the API-panel command
From a fresh multi-reviewer pass over the merged head plus the latest Codex bot review:
1. Load knobs now always consult the server. _resolve_model matched on model id alone,
so --gguf-variant / --context-length / --no-load-in-4bit / --tensor-parallel were
silently ignored whenever the id was already loaded (asking for UD-Q4_K_XL kept a
Q8_0 serving). With any explicit knob the CLI defers to /api/inference/load, whose
already-loaded dedup answers without reloading when variant and settings match, so a
second session running the same command still attaches without evicting the first.
2. OpenCode --yolo and the session model pin now ride in OPENCODE_CONFIG_CONTENT. A
project's own opencode.json outranks OPENCODE_CONFIG, so a repo config could silently
override the session model and the --yolo permission block; OPENCODE_CONFIG_CONTENT
outranks project config. The API key stays in the private file, never in printed env.
3. The --no-launch recipe's last line is a self-contained one-liner (inline VAR=value
assignments before the command, conflicting vars blanked). People copy just the last
line, and a bare codex/claude there ran against the user's real ~/.codex or Anthropic
credentials with zero isolation, e.g. inheriting a pre-existing damaged ~/.codex
state DB and blaming the recipe. The CI drive script scrubs the key from the one
'invoking:' echo this adds.
4. The auto-serve log is 0600 and the parent handle is closed. It sat world-readable in
the shared tempdir under a predictable name while carrying the minted sk-unsloth-
key from the unsloth run banner.
5. _key_accepted fails with a clean message on outages. Non-auth errors (5xx, network,
timeout) surfaced as a raw traceback; 401/403 still mean a rejected key.
6. _effective_base strips URL paths, and https loopback targets never auto-serve.
http://127.0.0.1:8888/studio polled /studio/api/health (404) and https://127.0.0.1
polled the wrong scheme, both spinning until the 15-minute startup timeout.
7. API-panel command: only literal 127.0.0.1:8888 earns the bare command. localhost can
resolve to ::1, which the bare CLI never probes, so it keeps UNSLOTH_STUDIO_URL.
8. CI artifact sweep covers redacted-configs/ and agent-workdir/, not just logs/.
Tests: 125 CLI tests pass (new coverage for each fix), 156 backend tests pass, ruff
clean. Adds an unsloth connect alias regression test.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* start: hand Pi a clean screen at launch
Pi paints inline from wherever the cursor sits: its first render assumes a
clean screen instead of clearing or entering the alternate screen itself
(current Pi never emits a clear at startup). Launched under unsloth start,
that left the session starting mid-scroll beneath the connection output.
Clear the screen (click.clear, cross-platform, no-op without a TTY) right
before the Studio banner so Pi opens exactly one line down on a clean
viewport. Launch path only: --no-launch recipes and piped output are never
wiped, and alternate-screen agents are left alone.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* start: auto-override hermes' 64K context floor for small model windows
Hermes refuses to initialize when the served model's context window is
under 64,000 tokens, and a second copy of the same check rejects the
compression model mid-session. write_hermes_config previously pinned the
real window, so any small local model (e.g. 40,960) failed at startup
with manual config.yaml instructions.
For windows below the floor the recipe now claims 65,536 in
model.context_length, scales compression.threshold so compaction still
fires at 90% of the real window, and sets
auxiliary.compression.context_length to cover the mid-session check.
Windows at or above the floor keep the exact previous behavior.
* ci: install pi with --ignore-scripts, matching the start.py hint
The pi cell predates the pi recipe in start.py and still installed the
package with lifecycle scripts enabled, so CI stopped exercising the
exact command users are prompted to run. npm_retry now passes extra
flags through, the pi branch mirrors the install hint verbatim, and the
stale no-recipe comment is refreshed.
* ci: fail loudly when a relocation var is missing from connect output
The empty-string guards ran after appending /config.toml or /config.yaml,
so they could never fire: crosscheck_contract silently skipped its
contract checks and patch_hermes_tools died on the root path with a bare
traceback. Check the raw variable first and guide_fail with the real
cause.
* staging: 6613 round 6 (https elision, no-launch home reuse, auto-start key fallback)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: shimmyshimmer <107991372+shimmyshimmer@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: Wasim Yousef Said <wasimysdev@gmail.com>
* Add --with-llama-cpp-dir flag to install.ps1 and install.sh
Users can now pass --with-llama-cpp-dir /path/to/llama.cpp to the
installer to skip downloading or building llama.cpp and use a local
directory instead. A junction (Windows) or symlink (Linux/macOS) is
created at the canonical install location, bypassing both the prebuilt
download (Phase 3) and source build (Phase 4) steps in setup.ps1/setup.sh.
The path is passed via UNSLOTH_LOCAL_LLAMA_CPP_DIR env var which
setup.ps1 and setup.sh read directly.
Ported from the idea in unslothai/unsloth#4384, reimplemented against
current Studio architecture.
* test: add static wiring test for --with-llama-cpp-dir flag
Cross-checks install.sh, install.ps1, studio/setup.sh and studio/setup.ps1
so the flag's contract (parse -> UNSLOTH_LOCAL_LLAMA_CPP_DIR env var -> link
local dir, skip prebuilt download and source build) can't silently regress.
Wired into studio-backend-ci.yml alongside the other tests/sh installer tests.
* Address review feedback on --with-llama-cpp-dir flag
- setup.ps1: delete an existing junction/symlink via DirectoryInfo.Delete()
instead of a recursive remove, which can traverse the link and wipe the
user's real llama.cpp directory on PowerShell 5.1.
- setup.ps1: short-circuit the build chain when a local dir is linked so CMake
never runs inside the user's checkout when it lacks a Windows-layout binary.
- install.sh / setup.sh: resolve paths with CDPATH= cd -P so a set CDPATH
cannot corrupt the resolved path.
- install.sh: seed _WITH_LLAMA_CPP_DIR from UNSLOTH_LOCAL_LLAMA_CPP_DIR so an
exported env var (piped-install style) is honored instead of being clobbered.
- setup.sh: create the root llama-quantize shim when linking a local source
build so GGUF export's check_llama_cpp() still finds it.
- setup.sh / setup.ps1: drop a stale link before the custom-home ownership
assert so re-runs with the flag stay idempotent.
- test: pin the new linked-dir build short-circuit.
* Harden --with-llama-cpp-dir against Codex/Gemini review findings
- install.sh: error when --with-llama-cpp-dir is the final arg with no path,
matching the existing --package/--python post-loop guards (was a silent
fallback to the normal prebuilt/source install).
- studio/setup.sh: canonicalize LLAMA_CPP_DIR before the self-link no-op
compare. _RESOLVED_LOCAL is fully resolved while LLAMA_CPP_DIR was textual,
so a symlinked $HOME made the guard miss and the rm -rf could wipe the
user's real llama.cpp tree.
- studio/setup.sh: make the llama-quantize shim non-fatal; it writes through
the link into the user's tree, which may be read-only (shared/CI cache),
and under set -e a failed ln aborted an otherwise-good reuse.
- studio/setup.ps1: detect a broken junction via Get-Item -Force instead of
Test-Path so a dangling link from a prior run is removed and mklink can
relink to a new valid directory.
- studio/setup.ps1: use Copy-Item -LiteralPath so a source path containing
[ ] isn't treated as a wildcard in the junction copy fallback.
- tests: update the wiring assertions for the LiteralPath copy and the
canonicalized compare.
* Validate/reuse local llama.cpp tree and guard the in-use case
Addresses the second Codex pass on the --with-llama-cpp-dir flag:
- Validate the linked tree before disabling installs (setup.sh + setup.ps1):
reusing a local dir skips BOTH the prebuilt download and the source build,
so the dir must already contain a runnable llama-server (build/bin on
Linux/macOS, build\bin\Release\llama-server.exe on Windows). Bail out with a
clear message instead of linking an unbuilt/wrong-platform checkout and
leaving Studio with no usable binary.
- Treat a canonical-path target as already linked when it holds a build
(setup.sh + setup.ps1): point the flag at ~/.unsloth/llama.cpp itself and an
existing build is reused (skip prebuilt + source) rather than clobbered by
the staged prebuilt installer (which uses os.replace()/replace). An empty
canonical dir still falls through to the normal in-place install.
- Abort when an in-use llama.cpp can't be removed on Windows (setup.ps1):
Remove-Item -ErrorAction SilentlyContinue can silently leave a locked tree
in place; detect that and stop with the same active-process message + exit 3
the prebuilt path uses, instead of junctioning over a half-present dir.
Left as follow-up (already tracked by the PR author as a non-blocker): the
in-app "Update llama.cpp" updater does not yet recognize a local-link install
as externally managed; that fix belongs in studio/backend/utils/llama_cpp_update.py.
* Accept all backend llama-server layouts in --with-llama-cpp-dir validation
The linked-tree validation only accepted build/bin[/Release]/llama-server, but
LlamaCppBackend._layout_candidates() resolves a root-level llama-server first,
then build/bin, then build/bin/Release on Windows. A `make` build or a flat
release extract (binary at the dir root) was therefore rejected with a hard
installer failure even though Studio would have run it.
Validate the same candidate set the backend uses in both setup scripts, and add
wiring-test assertions so the check can't silently narrow again.
* Treat --with-llama-cpp-dir local links as externally managed
A --with-llama-cpp-dir install junctions/symlinks the canonical llama.cpp dir to
the user's own checkout, but two backend paths still treated it as a Studio-owned
tree:
- The in-app updater (llama_cpp_update) offered and could apply an official
prebuilt over the link, writing through it into the user's checkout (or
failing) and silently dropping the link the flag created.
- Orphan cleanup (LlamaCppBackend._kill_orphaned_servers) resolved the linked
root into its kill allowlist, so a llama-server the user launched from the same
checkout was classified as ours and killed on startup.
Detect the canonical dir being a symlink/junction (reparse point) and treat the
install as unmanaged: get_update_status reports unsupported, start_update refuses
with reason "local_link", and the linked root is left out of the orphan
allowlist. Adds behavioral tests (link vs plain dir, updater refusal, and the
spared-vs-killed orphan control).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add behavioral shell test for --with-llama-cpp-dir linking
The existing tests/sh/test_with_llama_cpp_dir_flag.sh is a static grep of the
scripts. This adds a behavioral test that extracts the real link block from
studio/setup.sh (by content anchors, with a self-validating extraction) and runs
it against hermetic fake dirs, asserting the outcomes that matter:
- an external CMake build links and arms neither the prebuilt download nor the
source build
- a flat / make tree (root-level llama-server, no build/bin) is accepted too
- an unbuilt tree is rejected with a non-zero exit and no link left behind
- relinking over a stale link preserves the target's contents (no data loss)
- pointing at the canonical path is a no-op reuse, not a self-referential link
Symlink-identity checks run only where real symlinks exist (skipped on Windows
git-bash copy-mode); the link/skip/no-data-loss checks run everywhere. Wired into
studio-backend-ci.yml next to the static test.
* Install psutil in backend CI so orphan-cleanup tests run
The new orphan-cleanup tests import psutil for the process scan, but the Backend
CI deps step installed studio.txt plus a fixed extras list that omits it, so the
two tests failed with ModuleNotFoundError. Add psutil to both backend pytest dep
steps (kept in shared shape), and guard the import with pytest.importorskip so a
minimal env without psutil skips these tests instead of erroring.
---------
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Add FP8/FP4 compressed export to save_pretrained_merged
Adds compressed-tensors export (for vLLM) to save_pretrained_merged /
push_to_hub_merged via llm-compressor, alongside the existing lora /
merged_16bit / merged_4bit / gguf / torchao paths:
model.save_pretrained_merged("model", tokenizer, save_method="fp8")
Supported save_method values: fp8 (FP8_DYNAMIC), mxfp4, nvfp4 (W4A4) and
mxfp8. The LoRA is merged to 16bit at save_directory, then a quantized
checkpoint is written to save_directory + "-<fmt>". nvfp4 needs a small
calibration set (defaults to ultrachat, overridable via calibration_dataset).
Notes:
- llm-compressor is installed lazily on first use, pinning the current torch
and transformers via a constraints file so they are not upgraded (a plain
install pulls transformers>=5 and breaks Unsloth).
- Quantization runs in a separate process (unsloth/_compressed_quantize.py,
launched by file path) so Unsloth's transformers attention patches do not
interfere with the forward llm-compressor runs during calibration, mirroring
how GGUF export shells out to llama.cpp.
- mxfp8 needs a newer llm-compressor (transformers>=5); it is recognised and
raises a clear error until that stack is available.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Address review: main-process guard, calibration subsampling, tokenizer + dtype handling
- Route the 16bit merge through unsloth_generic_save for both LoRA and full
finetuned models, so non-PEFT models are written in 16bit consistently
instead of saving the original (possibly quantized) weights directly.
- Honor is_main_process: only the main process quantizes and writes the
compressed output, so distributed ranks do not race on the same dirs.
- Subsample an in-memory calibration Dataset before save_to_disk so large
training sets are not fully copied to a temp dir.
- Tolerate a missing tokenizer in the converter (data-free exports); still
require one for calibration based schemes.
- Open config.json via a context manager in both files.
- Drop the redundant nvfp4 entry from the unsupported-name check (fp4 covers it).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add direct LoRA to GGUF export and harden FP8/FP4 compressed export
- Run llm-compressor install and scheme check before the 16bit merge so
unsupported schemes (e.g. mxfp8) fail fast without writing a checkpoint
- Only the main process installs, merges, quantizes and uploads; isolate
hub pushes to a temp dir and clean all temp dirs in a finally
- Forward standard save kwargs (state_dict, max_shard_size, ...) to the merge
- Fall back to the first dataset split for Hub calibration ids
- Export LoRA adapters to GGUF via convert_lora_to_gguf.py: modernize
save_pretrained_ggml/push_to_hub_ggml and add save_method="lora" to
save_pretrained_gguf/push_to_hub_gguf; resolve base from the adapter config
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix LoRA GGUF shell-injection test and compressed export trailing-slash path
- Update tests/saving/test_save_shell_injection.py for the new delegation: the
LoRA to GGUF conversion now lives in _unsloth_save_lora_gguf, so assert it
passes argv as a list with no shell=True and that the legacy ggml wrappers
delegate to it instead of calling subprocess.Popen directly
- Normalize the local save_directory before building the "<dir>-<fmt>" sibling
so a trailing slash no longer nests the compressed output inside the 16bit dir
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Polish FP8/FP4 and LoRA GGUF export after review
- Warn (not silently downgrade) when an explicit quantization_method is not a
valid LoRA GGUF outtype; default stays f16
- Correct the inference hardware note: MXFP8 is 8-bit (cc >= 8.9), only FP4
needs Blackwell for full activation quantization
- Document that a local fp8/fp4 save keeps the 16bit merge at save_directory
and writes the quantized checkpoint to save_directory + "-<fmt>"
* Use sequential calibration pipeline and validate Hub access early
- nvfp4 calibration no longer forces the memory-hungry "basic" pipeline. The
quantization runs in a clean subprocess, so llm-compressor's default
sequential pipeline (layer-by-layer onloading) works and lets large models
that do not fit at once still calibrate; fall back to "basic" only if tracing
fails
- For push_to_hub compressed exports, create/validate the repo up front so a bad
token or denied repo fails before the merge and quantization instead of after
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Harden compressed export: explicit sequential pipeline, base-tokenizer calibration, GPU memory
- nvfp4 calibration now passes pipeline="sequential" explicitly (layer-by-layer
onloading) instead of relying on the inferred default, with a "basic" fallback
- Calibration datasets with a messages column no longer require a chat template:
base / non-chat tokenizers fall back to concatenating message contents
- Free the in-memory model's CUDA memory before the quantize subprocess loads its
own copy from disk (best-effort, single-device non-quantized only; restored
afterward), so a single GPU need not hold two copies at once
- Create the calibration temp dir in the system temp location instead of next to
the save directory, avoiding stray dirs in the workspace
* Free the failed calibration model before the basic-pipeline retry
In the sequential -> basic NVFP4 fallback, release the partially-processed model
and clear the CUDA cache before loading a fresh copy, so the retry does not
transiently hold two model copies on the GPU.
* Harden calibration data handling and compressed-export edge cases
- Calibration messages without a chat template now handle multimodal (list)
content, None content, and null message rows instead of crashing on join
- Raise a clear error when the calibration dataset is empty after subsampling
- Reset llm-compressor's global session before freeing the model in the
sequential -> basic NVFP4 fallback, so the old model is actually released
- LoRA GGUF export accepts a single-element list quantization_method
- Attach datasets metadata to the pushed repo on compressed hub exports
- Warn (instead of silently) if the model cannot be restored to its device
- Raise a clear error if the LoRA base model id cannot be determined
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Handle DatasetDict calibration, MoE routers, and MTP models in compressed export
- Reduce an in-memory DatasetDict calibration set to a single split before row
subsampling, so save_to_disk does not copy every split to the temp dir
- For MoE models, keep the router/gate unquantized and pass
moe_calibrate_all_experts so every expert is calibrated
- Warn when a model carries MTP / speculative-decoding tensors that the
compressed export does not include
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Support many more compressed-tensors schemes and address review
- Expand save_method to cover the full set of compressed-tensors preset schemes:
FP8 (dynamic/static/block), INT8, W8A8, W8A16, W4A16(+asym), W4A8, W4AFP8,
MXFP4(+A16), NVFP4(+A16), plus the gated MXFP8; calibration is used only for
the static-activation schemes (FP8 static, NVFP4)
- Broaden the near-miss save_method error to cover int/w-prefixed names
- MoE: also keep the Qwen shared-expert gate unquantized
- Strip non-model-input columns from already-tokenized calibration data so the
collator does not choke on a leftover messages column
- Forward the Hub token to the LoRA converter and the quantize subprocess so
gated/private base models and calibration datasets work without a global login
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Collapse compressed-tensors export help line so ruff-format converges
The print line in print_quantization_methods needed two ruff-format passes to
reach a fixpoint (merge implicit string concat, then collapse the single-arg
print). pre-commit.ci applies one pass per run, so it kept reformatting. Land
the converged single-line form directly.
* Add CPU-only regression tests for the export API
Cover all export paths without a GPU, for slow CPU-only CI:
- pure-function checks of the compressed-tensors scheme registry and save_method
normalization (aliases, calibration flags, near-miss errors)
- AST checks that every merged saver dispatches compressed export, the GGUF savers
expose the lora branch, torchao routes PTQ/QAT, the public methods stay attached,
and the export subprocesses remain shell-safe (argv list, sys.executable, no shell)
- monkeypatched dispatch checks that fp8/nvfp4/merged_16bit, the LoRA-GGUF outtype
resolution, and torchao PTQ/QAT reach the right helper with the right arguments
* Run the CPU-only export tests in consolidated CI
tests/saving is --ignored by the Repo tests (CPU) job, so the new GPU-free export
tests are added by path to consolidated-tests-ci.yml (collection sanity + Bucket-A run),
alongside the existing CPU saving tests, so they actually execute on CPU CI.
* Add GPU GGUF export + llama-cli inference smoke test
tests/saving/test_gguf_export_and_inference.py: skipif no CUDA. Trains a tiny
phrase-imprinting LoRA, exports a full-model q8_0 GGUF (merge -> convert_hf_to_gguf
-> llama-quantize), asserts a valid GGUF (magic + size), and - when a llama-cli
binary is available - runs one bounded generation (byte cap + watchdog kill) and
asserts the trained phrase round-trips through HF -> GGUF -> quantize -> inference.
The llama-cli step skips gracefully since the export only builds llama-quantize.
* Fix variant mismatch in compressed (FP8/FP4) export
save_pretrained_merged(..., save_method=fp8/nvfp4, variant=...) forwarded
the variant into the intermediate 16bit merge, so Transformers wrote
variant-named shards (model.<variant>.safetensors). The converter
subprocess then reloaded that directory with the default weight filenames,
so the compressed export failed after doing the merge.
Pop the variant out of the intermediate merge (internal staging that the
subprocess reloads with default names) and forward it via --variant so it
is applied to the final compressed checkpoint instead. Add a CPU AST guard
for the contract.
* Harden export paths from review
- install_llm_compressor: fall back to uv pip when this interpreter has no
pip seeded (uv-created/relocatable venvs), instead of failing with
No module named pip.
- LoRA GGUF export: if convert_lora_to_gguf.py is missing (a prebuilt or
reused CWD llama.cpp install carries binaries but not the converter
script), force a dedicated source checkout that ships it.
- push_to_hub_gguf(save_method=lora): return on non-main ranks, matching the
local save_pretrained_gguf lora branch, so only rank 0 converts/uploads.
- compressed export VLM detection: require a vision_config or a
ForVisionText2Text architecture; a bare *ForConditionalGeneration also
matches text seq2seq models (T5/BART/Whisper) and is no longer treated as
a VLM on its own.
- GGUF GPU smoke test: drop SFTConfig(max_length=1024), which raises under
newer TRL padding-free training; length enforcement is not needed here.
* Add imatrix option to GGUF export, enabling IQ low-bit quants
save_pretrained_gguf / push_to_hub_gguf gain imatrix_file:
None -> no imatrix (unchanged)
'/path' -> pass to llama-quantize --imatrix (a *.gguf_file is renamed to *.gguf)
True -> download the upstream unsloth/<base>-GGUF imatrix (imatrix_unsloth.dat or
.gguf_file), raising a clear error if none exists
An importance matrix unlocks the IQ low-bit quants (iq2_xxs, iq4_xs, ...), which were hard
disabled before. They are gated: requesting one without an imatrix raises a clear error.
- _resolve_imatrix_file resolves path/True (PEFT base first, normalized via get_model_name,
derives unsloth/<base>-GGUF, copies out of the HF cache before renaming *.gguf_file).
- IMATRIX_QUANTS registry replaces the old commented-out IQ entries; save_to_gguf accepts a
resolved imatrix and threads it into the quantize calls.
- The --imatrix flag is emitted by unsloth_zoo's quantize_gguf (companion change). save.py
fails fast with an upgrade hint if the installed unsloth_zoo lacks the imatrix kwarg.
Tests: tests/saving/test_imatrix_export.py (CPU: resolution, repo derivation, IQ gate,
--imatrix wiring) wired into CI; tests/saving/test_gguf_export_and_inference.py extended with
GPU iq2_xxs/iq4_xs export + inference. Verified end to end on Llama-3.2-1B: imatrix
auto-downloaded, iq2_xxs/iq4_xs exported and run via llama.cpp.
Note: requires the companion unsloth_zoo quantize_gguf imatrix change.
* Address imatrix/compressed review feedback: unsloth org GGUF repo, fail-fast, calibration split
- imatrix auto-resolve (imatrix_file=True): derive the upstream repo as unsloth/<base>-GGUF
instead of <org>/<base>-GGUF, so official bases (e.g. meta-llama/Llama-3.1-8B-Instruct) find
the matching Unsloth GGUF imatrix repo rather than failing on a nonexistent meta-llama/...-GGUF.
- Resolve/validate the imatrix before the 16-bit merge in save_pretrained_gguf, so a bad path or
an unavailable upstream imatrix fails fast instead of after a long, multi-GB merge.
- Compressed calibration: when a Hub dataset has no "train" split, resolve the first split name
and slice it, instead of materializing the whole dataset just to take num_samples rows. Keeps
the original materialize-then-subselect path as a last resort.
Tests: add unsloth/<base>-GGUF mapping for an official base id, and create the imatrix file in the
quantize_gguf flag test (quantize_gguf now validates the imatrix exists).
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: stop handing CI/user secrets to downloaded llama.cpp binaries
The macOS prebuilt path installs llama.cpp from the unslothai/llama.cpp
fork's latest (unpinned, mutable) release and then executes the
downloaded llama-server / llama-quantize binaries during install-time
validation. binary_env() built that child environment from a full
os.environ.copy(), so a compromised or tampered prebuilt would inherit
every secret in the process: HF_TOKEN and the workflow GitHub tokens in
CI, and HF / cloud credentials for end users running install.sh /
setup.sh.
We publish prebuilts daily, so pinning a release tag is not workable.
Instead, neutralise the impact: these binaries have no reason to read any
token, so strip secret-bearing variables (exact names plus
TOKEN/SECRET/PASSWORD/CREDENTIAL/PRIVATE_KEY/API_KEY markers) before
handing the env to a downloaded binary. The installer's own GitHub and
Hugging Face API calls read os.environ directly, so authentication and
release-API rate limiting are unaffected; PATH, LD_LIBRARY_PATH,
DYLD_LIBRARY_PATH and CUDA/ROCm vars are preserved. One change covers the
install-time validation path for all six macOS workflows and end users.
Follow-up (separate, sequenced): publish build-provenance attestations
from the fork's prebuilt workflows and verify them in CI, so a forged
release is rejected rather than merely starved of secrets.
* Strip KUBECONFIG, SSH_AUTH_SOCK, and PASSPHRASE-marked vars from binary env
Extend the deny-list per PR review: KUBECONFIG and SSH_AUTH_SOCK are
credential pointers/capabilities a downloaded binary never needs, and a
PASSPHRASE marker catches SSH_PASSPHRASE / GPG_PASSPHRASE. Tests updated.
* Studio: also scrub proxy/index env vars and URL-embedded credentials before running prebuilt binaries
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Scope mlx-ci secrets to the install + download commands for PR #6696
Drop the ambient step-level env block and pass GH/GITHUB/HF tokens only
on the installer and GGUF-download commands, so the directly invoked
llama-quantize / llama-server smoke runs see no secrets. The installer
still reads tokens from os.environ for the releases API and probe fetch.
* Trim verbose comments around the secret-env scrubber for PR #6696
Comment-only: condense the block comments added across this PR. Logic
unchanged (comment_tools.py check confirms code-only signature equal).
* Redirect HOME / cache pointers to an empty dir for prebuilt binaries (PR #6696)
Address Codex P2: stripping token env vars still let a tampered binary
read on-disk token stores (~/.cache/huggingface/token, ~/.aws/credentials,
~/.config/gh) through $HOME and the cache/config pointers. Point HOME plus
the HF / XDG / Windows home pointers at a single empty throwaway dir for
the downloaded-binary env. Defense in depth: a binary resolving the real
home via getpwuid is out of scope and needs OS sandboxing.
* Close residual credential-probe gaps for PR #6696
Address the latest Codex review:
- Strip token-only URL userinfo too (scheme://ghp_token@host), not just
the user:pass form.
- Redirect HOMEDRIVE/HOMEPATH alongside USERPROFILE so a Windows binary
cannot reconstruct the real profile from %HOMEDRIVE%%HOMEPATH%.
- Drop explicit credential-file pointers (NETRC, PIP_CONFIG_FILE,
DOCKER_CONFIG, GIT_CONFIG_GLOBAL) that live outside HOME.
- Probe ldd with a secret-free env: linux_runtime_dirs ran ldd on the
untrusted prebuilt with the inherited os.environ, and ldd may execute
the binary, so it could observe HF_TOKEN/GITHUB_TOKEN during the probe.
Factored the shared scrub into secret_free_environ().
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Separate token-bearing install from binary smoke; drop CI command files (PR #6696)
Address the two P1s in the latest review:
- mlx-ci: GitHub bakes secrets into the run-script text, so inline token
assignments in a step that later runs the prebuilt let a tampered binary
read them from the script. Split into a token-bearing install + download
step that never launches a binary, and a secret-free smoke step that runs
llama-quantize / llama-server.
- secret_free_environ now drops the GitHub Actions command files
(GITHUB_ENV, GITHUB_PATH, GITHUB_OUTPUT, GITHUB_STEP_SUMMARY, BASH_ENV) and
the smoke step unsets them, so a tampered prebuilt cannot inject PATH/env
into the later token-bearing MLX steps.
* Run the prebuilt smoke last, after all token-bearing steps (PR #6696)
Address the P1 workspace-poisoning vector: even with no secrets in its env,
a tampered prebuilt could edit the checkout or installed modules, and the
later HF_TOKEN MLX steps would then execute that poisoned code on push
builds. Move the prebuilt install + smoke to the end of the job so the
untrusted binary runs after every token-bearing step, leaving nothing for it
to corrupt. The MLX GGUF reload uses a source-built llama-cli, not this
prebuilt, so nothing depends on the earlier position.
* Trim comments around the secret-env scrubber and prebuilt CI steps (PR #6696)
Comment-only: condense the security-rationale block comments and merge the
duplicated prebuilt-step description in mlx-ci. Logic unchanged
(comment_tools.py check confirms the code-only signature is equal; install
suite still passes).
* Authenticate the GGUF export release-API lookup with the read-only GITHUB_TOKEN (PR #6696)
* Rename env scrubber off the secret-named identifier CodeQL flags as a clear-text sink (PR #6696)
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Two intermittent Studio CI failures, both runner-environment flakes unrelated
to test logic:
Windows 'Studio install + inference without Visual Studio': the 'Hide Visual
Studio + CMake' step renames C:\Program Files\Microsoft Visual Studio to
simulate a host with no build tools. A background handle on a Program Files
directory (Defender scan or an MSBuild node) makes Rename-Item intermittently
fail with 'Access is denied', and $ErrorActionPreference = Stop turns that into
a hard job failure. Wrap the VS and cmake renames in both Hide steps in a short
Rename-WithRetry (6 tries, 3s apart) to ride out the transient lock.
macOS 'Chat UI Tests': the re-login goto to /login can be interrupted by the
SPA auth guard redirecting to the same /login URL, which Playwright reports as
'Navigation to .../login is interrupted by another navigation to .../login'.
The goto already tolerated ERR_ABORTED; broaden it to also tolerate the same-URL
interrupt (the password-field wait right after confirms we landed on /login),
and add the same signature to the two Playwright flake-retry harnesses as a
safety net for any other navigation.
Validated: playwright_chat_ui.py parses + byte-compiles, both workflow YAMLs
parse, bash -n on the retry harnesses, PowerShell AST parse on all pwsh steps,
and a functional check of Rename-WithRetry (succeeds, and rethrows after
exhausting retries).
* Verify linuxdeploy AppImage digest before use in desktop release
The desktop release workflow downloaded linuxdeploy-x86_64.AppImage from a
GitHub release and ran chmod +x with no integrity check. Pinning the
versioned release path is reproducibility, not integrity: a release asset
can be replaced (or its delivery path compromised) after upload. The next
step builds the AppImage with the Tauri signing private key and a
contents:write GITHUB_TOKEN in scope, so a substituted linuxdeploy that
ran during packaging could exfiltrate signing material or tamper with
published release artifacts.
Pin the immutable SHA-256 of the asset and verify it with sha256sum -c
before chmod +x, so a mismatch fails the job closed before the binary is
ever executable. Extend the existing in-workflow guard to require both the
pinned digest and the verification step, so a future edit cannot silently
drop the check.
* Scope linuxdeploy guard to real step content, not its own text
The self-check searched every workflow line, so the digest assertion was
satisfied by the guard's own expectedLinuxdeployDigest line and the
verification assertion by a comment. Deleting the LINUXDEPLOY_SHA256 env
pin or the actual sha256sum -c command would still have passed.
Match the digest against the LINUXDEPLOY_SHA256 env line specifically and
require sha256sum -c on a non-comment line, so dropping either the pin or
the verification now fails the guard.
* Scope linuxdeploy guard to the Pin step block and check ordering
The previous predicate still scanned the whole workflow, so the literal
sha256sum -c in the guard's own code satisfied the verification check; a
deleted or post-chmod verification command would still pass.
Extract the 'Pin linuxdeploy for AppImage' step block and assert within it:
the LINUXDEPLOY_SHA256 env pins the expected digest, a non-comment line
runs sha256sum -c, and that verification precedes chmod +x.
* Withhold HF_TOKEN from pull_request runs of CI workflows
The pull_request-triggered CI workflows check out and execute PR-controlled
code (install.sh, .github/scripts/**, tests/**) with secrets.HF_TOKEN in the
step environment. For a same-repo PR, GitHub provides repository secrets to the
run, so a malicious or compromised branch could modify a checked-out script to
read and exfiltrate HF_TOKEN, including by writing it into the uploaded logs/
artifact. HF_TOKEN is an external Hugging Face credential of unknown scope, so
this is the high-value exposure.
Gate every HF_TOKEN reference in these workflows with
`github.event_name != 'pull_request' && secrets.HF_TOKEN || ''`, so the real
token flows only on the trusted schedule/push/workflow_dispatch runs and PR runs
see an empty string. All model repos used by these jobs are public
(unsloth/*-GGUF), so anonymous download still works on PRs; install_llama_prebuilt.py
only sends HF auth to Hugging Face hosts and tolerates an absent token.
GITHUB_TOKEN (passed as GH_TOKEN) is intentionally left in place: it is the
auto-provisioned, job-scoped, contents:read token that expires with the job and
gives a same-repo PR author nothing they do not already have, and
install_llama_prebuilt.py needs it to authenticate the GitHub releases API or
the prebuilt llama.cpp download hits the anonymous rate-limit bucket and 403s.
* Trim the HF_TOKEN gating comments to one line per site
Comment/whitespace-only: collapse the per-step rationale to a single line and
shorten the local-agent-guides header note. No workflow logic changes (verified
each file's parsed YAML is identical before/after).
* studio/setup.sh: guard empty CUDA arch detection in the source build
PR #5826 hardened setup.sh for fresh CUDA toolkits, but the source build
still set -DCMAKE_CUDA_ARCHITECTURES only when nvidia-smi reported a
compute capability. When that query returns nothing the build proceeded
with no explicit arch list, so llama.cpp built PTX only. On a driver older
than the toolkit that binary fails at runtime with "the provided PTX was
compiled with an unsupported toolchain" - the build succeeds, so neither
the build-time check nor the CPU fallback caught it (issue #5854).
Resolve the arch list before committing to a CUDA build. A new pure helper
_resolve_cuda_archs parses and de-duplicates the nvidia-smi compute_cap
output and honors an explicit UNSLOTH_LLAMA_CUDA_ARCHS override. When the
result is empty, build CPU llama.cpp instead of a PTX-only binary, with a
clear message pointing at the override - so the user still ends up with a
working llama-server. The override also lets advanced users force a native
build on hosts where nvidia-smi cannot report compute_cap.
No behavior change when an arch is detected: -DGGML_CUDA=ON plus the arch,
CUDA flags and NVCC_PREPEND_FLAGS are assembled exactly as before.
Adds tests/sh/test_resolve_cuda_archs.sh (single/multi/dedup/empty/garbage/
whitespace/override cases), wired into tests/run_all.sh and the
studio-backend-ci.yml shell-test loop.
* studio/setup.sh: resolve nvidia-smi via /usr/bin fallback for arch detection
Addresses review feedback on the empty-CUDA-arch guard: _setup_has_usable_nvidia_gpu
classifies a host as NVIDIA-usable using nvidia-smi on PATH OR /usr/bin/nvidia-smi,
but the new arch detection probed only `command -v nvidia-smi`. On a GPU host where
nvidia-smi is off PATH (reachable only at /usr/bin), arch detection returned empty
and the new empty-arch branch dropped the build to CPU, losing CUDA. Mirror the same
PATH-then-/usr/bin resolution so those hosts still get a native CUDA build.
Also scope _resolve_cuda_archs locals with `local` (no behavior change; it already
runs under command substitution).
* tests: update compute_cap-probe assertion for $_smi_bin resolution
The nvidia-smi /usr/bin fallback parameterized the binary in the compute_cap
probe (_setup_run_smi "$_smi_bin" ...), so the literal-string assertion in
test_compute_cap_probe_timeout_wrapped no longer matched. Assert the probe is
preceded by _setup_run_smi (timeout-wrapped) instead, scanning all occurrences
so the comment mention is ignored. Same intent, binary-agnostic.
* tests: ruff-format the compute_cap probe assertion (pre-commit)
Collapse the backslash-continued assert onto one line and normalize slice
spacing so the ruff-format pre-commit hook (0.6.9) is satisfied. Formatting
only; no behavior change.
* Tighten code comments (no logic change)
* studio(windows): build CPU when CUDA arch is undetectable (#5854)
The Windows source build added -DGGML_CUDA=ON unconditionally but only set
-DCMAKE_CUDA_ARCHITECTURES when $CudaArch was detected. With no detectable
compute capability that produced a PTX-only binary, the same hole the Linux
fix closed. Build CPU llama.cpp in that case, and honor UNSLOTH_LLAMA_CUDA_ARCHS
to force a CUDA build, matching setup.sh. Detected-arch builds are unchanged.
* test: anchor NVCC_PREPEND_FLAGS scope check on the final CPU branch
The undetectable-arch CPU fallback adds an earlier -DGGML_CUDA=OFF, so the
ordering check now anchors on -DGGML_CUDA=ON and the last -DGGML_CUDA=OFF
instead of the first.
---------
Co-authored-by: Daniel Han <michaelhan2050@gmail.com>
Michael Han