The web update banner copied both the macOS/Linux curl and the Windows irm
one-liner joined by a newline, so pasting into a single terminal ran the wrong
shell's command and errored. Detect the OS from navigator.userAgent and copy
only the matching command; the banner still displays both.
In the settings update panel, editable/local_repo checkouts were told to run
the remote curl/irm installer, which is the PyPI path (install.sh only installs
locally with --local) and would replace an editable checkout with a PyPI
package. Use the OS-specific local installer (./install.sh --local /
.\install.ps1 --local) as the checkout update command and drop the now
redundant duplicate block plus its unused localInstallerFallback locale key.
Update banners, the settings update panel, and the stale/MTP warnings now
point users at the install script one-liner (curl on macOS/Linux, irm on
Windows) rather than the `unsloth studio update` subcommand. Surfaces that
cannot detect the OS show both commands.
Internal callers, CI workflows, and the `unsloth studio update` subcommand
itself are unchanged.
Addresses review feedback on #5971. _reset_password_command() already
shell-quotes the launcher path on POSIX (shlex.quote), so wrapping the result in
another pair of single quotes in the error string produced a mangled hint for
installs / home dirs containing spaces, e.g.
Run ''/tmp/Unsloth Studio/.../unsloth' studio reset-password' in your terminal
which a shell mis-parses. Drop the outer quotes and put the command at the end of
the message so it is unambiguous and copy-pasteable in every case:
Incorrect password. To reset it, run this in your terminal: <cmd>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* studio: redesign chat composer
Reworks the new-chat composer and the compare composer into a single
rounded pill surface with a softer, lighter look.
- New welcome screen with a time-of-day sloth mascot and a lighter
heading.
- One rounded composer surface with a soft drop shadow. The input grows
inline as you type and collapses back to a single row when cleared.
- Tools and attachments live in a single plus menu; the thinking control
is a compact pill with a reasoning-effort submenu.
- Inlined glyphs for the thinking, send, and dictate controls, kept in
sync across the main and compare composers.
- Toast notifications match the composer surface: no border line, the
same drop shadow, and the same dark surface color, with a ring-less
close button.
- Dark mode: the side-menu shadow blends into the background, hovered
menu rows read clearly, and their roundness matches light mode.
- Composer styles use dedicated unsloth- prefixed classes so compare
mode keeps its own stacked layout.
* studio: sync compare-composer reasoning state and harden compare id
- Compare composer: keep "Preserve thinking" consistent with reasoning,
matching the main composer. Enabling it now turns reasoning on, and
disabling reasoning (the None option or the Thinking toggle) turns it
off, so the invalid "preserve on while thinking off" state can't occur.
- Guard crypto.randomUUID in the Compare action. It is undefined in
non-secure contexts (HTTP over a LAN IP) and would throw; fall back to
a timestamped random id, matching createNavigationNonce.
* studio: reflect pre-selected Search/Code tools when no model is loaded
The Search and Code pills only lit up when the tool was usable right now
(a model loaded and capable), so a tool turned on from the + menu showed
as off in the pill while the menu showed it on. toolsEnabled is persisted
and takes effect once a capable model loads, so the pill should reflect it.
The pills now disable only when a loaded model lacks the capability, and
otherwise reflect the selected state. Applied to the main and compare
composers.
* Studio: link MCP Servers heading to its PR and fix composer pill cursors
Make the "MCP Servers" heading in the chat Configuration sheet link to the
MCP PR, keeping the chevron as the toggle. The label and chevron are rendered
as siblings so we don't nest an <a> inside a <button>.
Also add cursor-pointer to the composer pills and the thinking pill so hovering
a clickable pill shows the hand cursor instead of the default arrow.
* Studio: refine chat composer and add compare-mode parity
- Composer expands to two rows only once the input wraps to a second line,
not on the first keystroke. Re-measure the autosize textarea on the width
swap so expanding no longer leaves a stray blank row.
- Light-mode composer shadow now matches Gemini's soft elevation.
- Plus menu: replace Canvas with a More submenu (Canvas, Compare chat, RAG)
and add Code above MCP. Active Web search/Code items use medium weight.
- Compare mode: the plus side menu, Search/Code toggles, and a Compare exit
pill now match single chat, with the thinking control on the right.
- Projects menu entries link to their tracking PR (#5725).
- Add cursor-pointer to the composer plus button.
* studio: refine composer controls and chat search shadow
- Active tool pills show an x on hover to signal click-to-disable
- Plus button rotates into an x when the tools menu opens
- Composer surface uses a 32px radius and a taller single-line height
- Even, ChatGPT-style spacing between the plus and tool pills in both the single and compare composers
- Send and mic circles resized and spaced, with the arrow centered in the circle
- Chat search box gets a borderless, soft Gemini-style shadow
* studio: size the pill hover x to match the icon it replaces
Cross-engine checks (Chromium, Firefox, WebKit) flagged the active-pill
hover x as a fixed 14px, so it popped smaller than the 19px Code icon.
Fill the glyph slot instead so the x tracks whatever icon it covers.
* studio: do not persist Kimi search/thinking mutual-exclusion in single composer
The single-chat composer flipped the other control off when toggling
search or thinking on Kimi, but without { persist: false }, so it
overwrote the user's saved preference. Match shared-composer and keep
the side effect session-only.
* studio: pointer cursor on model selector trigger and menu items
Add scoped marker classes so the model picker trigger and every
clickable element in its menu (tabs, model rows, delete, eject) show a
pointer cursor; disabled items stay not-allowed.
* studio: pass baseUrl when resolving reasoning caps in single composer
The docked composer omitted baseUrl, so a custom Gemini OpenAI-compat
gateway still advertised the native thinking ladder the backend cannot
honor. Pass selectedExternalProvider.baseUrl like the compare composer
so the resolver hides it.
* studio: grey side-menu hover, green pill hover, thinking hover x
- Plus side-menu items hover grey in light mode, not the green accent
- Thinking pill hovers green like the Search and Code pills
- The plain Thinking toggle shows an x on hover when active, matching
Search and Code; the effort dropdown trigger keeps its bulb
* studio: make the pill hover x a uniform size
The x filled the icon slot, so the wider Code chevron gave a bigger x
than Search and Compare. Pin it to a fixed 15px, centered, so every
pill's x matches.
* studio: broaden chat attachments, fix active hover color, gemini shadow
- Accept svg, source code and many text/config files as drag-and-drop
or picked attachments, matched by extension since their MIME is
unreliable; html keeps its own adapter
- Active (green) side-menu items keep their text and icon color on
hover instead of switching to the accent color
- Composer surface uses Gemini's soft centered shadow 0 0 20px rgba(0,0,0,0.04)
* studio: keep the thinking pill full height when icon-only
The inactive thinking pill has no label, so its flex row collapsed to
the icon height and the hover box looked short. Reserve one text line
(min-height: 1lh + padding) so it matches the Search and Code pills.
* studio: refine composer menu, drop overlay and greetings
- Open the MCP servers dialog directly from the composer plus menu
- Redesign the drag-and-drop affordance Gemini style, drop the badge and border, make the whole chat page a drop target
- Swap in Hugeicons for the RAG, attachment chip and new project icons
- Add time-based randomized welcome greetings, each matched to a fitting sloth
* studio: rename artifacts toggle to Canvas and make it opt-in
- Label the toggle Canvas everywhere, matching the plus menu
- Stop greying out the Canvas menu item; it toggles like the other items
- Only show the Canvas pill in the composer row once it is turned on, since it is less central than Search and Code
* studio: wire Canvas and MCP composer toggles, even out the pill row
- Open the MCP servers dialog from the menu, or toggle MCP on/off once a server is enabled
- Force MCP off when no server is enabled, so the toggle stays honest
- Show Canvas and MCP as opt-in pills that appear in the order they were toggled on
- Expand the composer and light up the pill when Canvas or MCP is on, like Search and Code
- Keep Compare directly after Code in the compare composer
- Use the same Code icon on both composers and give every pill an even icon slot
* studio: tidy composer toggle row and fix MCP enable/disable lifecycle
- Enable MCP automatically after a server is configured via the toggle flow
- Force MCP off everywhere once the last enabled server is removed
- Collapse the pill labels to icons only when more than 4 pills show, keeping Compare labelled
- Order Compare first in compare mode, before Search and Code
- Use the same Code icon and an even 19px icon slot across both composers
- Match the compare composer surface padding and send button inset to normal chat
* studio: revert compare composer padding change that cramped the input
Matching the surface padding to normal chat clipped the textarea text and
left a white strip on top. Restore the compare composer's own padding, which
gives proper top spacing. The send button inset fix stays.
* studio: center welcome greeting and soften composer scrollbar
Center the sloth and title together over the composer instead of
shifting the row left, which left the greeting sitting off to the side.
Keep the composer textarea scroll thumb faint by default and only darken
it when the thumb is hovered or dragged, so a tall draft no longer shows
a heavy dark rail.
* studio: match composer plus-menu tool gating to the pills
The new plus-menu tool entries did not carry the gating the visible pills
already enforce, so the menu and pills could disagree about a loaded
model's capabilities.
- Web search and Code menu items now disable when a loaded model lacks
the capability, while still allowing preselection with no model loaded.
- Enabling Web search from the menu on a Kimi model now flips thinking
off as a session-only change, since Kimi forbids search and thinking
together. This matches the Search pill.
- Added an Images menu item, shown only for image-generation models and
disabled until a model loads, so a short prompt has an entry point.
Applied to both the single-chat and compare composers.
* studio: round the active-pill hover x and even out pill padding
The hover x sat bare and the trailing label was tighter to the pill edge
than the leading icon, so the pill looked lopsided.
- Give the hover x a soft circular background that fills the icon slot,
matching the ChatGPT-style toggle and the icon it replaces.
- Add a little more trailing padding so the label and the leading icon
have even breathing room, and keep icon-only compact pills symmetric.
* studio: nudge the thinking bulb icon up by 0.5px
Bump the thinking lightbulb from 15px to 15.5px in the single-chat and
compare composers so it sits a touch larger next to the other controls.
* studio: drop the hover x circle on icon-only pills
When pills collapse to icon-only, the circle around the hover x is too
cramped in the small chip, so show a bare x there and keep the circle
only on the full-width labelled pills.
* studio: space the compare send button like normal chat
In compare mode the Thinking control sat right against the send button.
Match the normal composer's control spacing (gap-1.5 plus a send margin)
so Thinking has the same breathing room before send. The send button
keeps its 14px inset, so its position is unchanged.
* studio: make collapsed pill hover a circle, not a wide pill
Icon-only pills were wider than tall, so their rounded-full hover
highlight read as a fat rounded rectangle. Make the compact button a
square and center the glyph so the hover (and the x it reveals) sits in
a clean circle.
* studio: fix compare pane drops and audio picker lifetime
- Skip the page-level drop handler when the composer is hidden, so files
dropped on a compare pane are not swallowed by a hidden composer; the
shared compare composer keeps handling drops through its own dropzone.
- Build the audio file input on document.body instead of inside the plus
menu, so the menu closing on select no longer unmounts the input before
the OS picker returns and drops the file.
* studio/chat: stop projects list from white-screening on older backends
The projects list API returned data.projects directly, so a backend that
omits the field handed back undefined. useChatProjects cached that value,
then the next mount read undefined.length and crashed the whole chat page.
Default the projects and threads list APIs to an empty array and keep the
hook null-safe so a bad response can never poison the cache.
* studio/chat: align MCP dropdown with the + menu and add a chevron
Reuse the + menu surface (unsloth-plus-menu) for the MCP dropdown: rounded
corners, narrower width, neutral grey hover, and enabled rows shown as green
text with a right-aligned check instead of the emerald underlay. Add a
chevron to the MCP pill so it reads as openable, matching the Thinking pill.
* studio/chat: make MCP an opt-in pill and fix its dropdown placement
- MCP is back in the + menu as a toggle. The pill now only shows in the
composer when MCP is on, matching Canvas, instead of always sitting there.
- The dropdown follows the composer side like the + menu (opens down in the
welcome composer, up when docked) rather than always opening upward.
- Drop the dropdown caret when pills collapse so the icon is not squished.
- Stop force-syncing mcpEnabledForChat to the server count; the + menu owns it.
* studio/chat: MCP expands the composer, drop sidebar Compare, tidy scrollbars
- Toggling MCP now expands the composer and shows the tool pills, the same as
Canvas, instead of leaving the row collapsed.
- Remove the Compare item from the sidebar now that it lives in the + menu, and
point the compare tour step at the side-by-side view instead of the old button.
- Both sidebars only show their scrollbar on hover, and run settings reserves
the scrollbar gutter so the close button no longer shifts when it appears.
* studio/chat: tighten toggle gap, fix run-settings close button, collapsed Train
- Reduce the composer toggle gap by 2px (gap-1 to gap-0.5) in both composers.
- Move the run settings header out of the scroll area so the close button keeps
its position whether or not the scrollbar shows, and sits flush with the
topbar open button again instead of shifting left.
- Surface Train as an icon in the collapsed sidebar (it already has a labelled
section when expanded).
* studio/chat: tighten Thinking pill X padding, create projects inline
- The Thinking pill used px-2.5, so the hover X sat further in than the left
pills. Match their pl-2 so the X lines up.
- The + menu New project now opens a create dialog and jumps straight to the
new project, instead of routing to the projects list. Shared by both
composers via a small NewProjectDialog.
* studio/chat: soften account menu, hover scrollbars, show collapsed chevrons
- Account menu drops its border ring for the composer's soft shadow and opens
centered over its trigger.
- Settings and search reuse the hover-only scrollbar via a shared
hover-scrollbar class, matching the sidebars.
- Train and Recents keep their chevron visible while collapsed so it is clear
they can be expanded.
* studio/chat: roomier, more rounded account menu
Widen the account menu, add more left and right padding on the rows, bump the
row height and text a touch, and round the corners more, closer to the GPT
account menu.
* studio/chat: trim account menu width and nudge it up 2px
Pull the account menu in slightly on the left and right (narrower box, a touch
less row padding) and lift it 2px higher above the trigger.
* studio/settings: drop outline ring, circular close hover, pointer cursors
- Remove the settings dialog outline ring, keeping just the soft shadow.
- The close button hover is now a circle instead of a rounded rectangle.
- Every clickable control in the settings dialog uses a pointer cursor.
* studio/chat: bump MCP pill icon to 14.5px
Nudge the MCP icon up 0.5px so it sits even with the other pill glyphs.
* studio/chat: bump MCP pill icon to 15px
Nudge the MCP icon up another 0.5px.
* studio/settings: add a Settings title above the tabs
Put a Settings heading at the top of the sidebar so the tabs sit below it,
matching the Claude settings layout. Hidden on mobile where the nav is a row.
* studio/settings: rounder tab hover, bigger title, less-round search dialog
* studio/sidebar: round nav row hover boxes 2px more (10px to 12px)
* studio: drop settings dark shadow + divider, add tab left padding, tune hover roundness
* studio/model-selector: roomier padding, borderless box, rounder hover rows; settings divider light-only
* studio/search: match chat box shadow (soft light, none dark)
* studio/sidebar: borderless chat context menus, rename submenu to Projects with folder-export icon
* studio/model-selector: match light corner radius in dark, drop dark shadow, more visible dark hover
* studio/sidebar: chat context menu matches + side menu styling; relabel submenu Move to project
* studio: borderless message export menu (no dark shadow), match dark corner radius to light on export menu and settings
* studio/sidebar: open chat options menu GPT-style (down-right) and widen so Move to project fits one line
* studio/chat: message export menu uses the chatbox shadow in light mode
* studio/sidebar: narrow chat options menu slightly (w-60 to w-56)
* studio: unify all download icons to Hugeicons download-01; round profile button hover 1px more
* studio/run-settings: bump header to 16px
* studio/sidebar: trim chat options menu width slightly (w-56 to 216px)
* studio/sidebar: trim chat options menu width to w-52
* studio/profile: camera-01 Hugeicons glyph and chatbox shadow on avatar button
* studio: match dark-mode corner radius to light globally (single --radius token)
* studio/recipes: borderless New Recipe menu with chatbox shadow in light, none in dark
* studio: borderless dropdowns globally, chatbox shadow in light, none in dark
* studio: extend borderless + chatbox/none shadow to select, combobox and popover overlays
* studio/mcp: nudge MCP dropdown radius to 20px so its wider box reads as round as the + menu
* studio: restore dark dropdown shadow to avoid same-color merge; greet name ~1/3 of lines; bigger sloth + more gap
* studio/train: active tab is a borderless pill (no underline), roomier padding, more tab gap and bottom spacing
* studio/chat: nudge welcome up ~5px (still vh-based) and trim sloth image to 44px
* studio/train: active tab pill is white with chatbox shadow in light, taller padding
* studio/chat: welcome offset to calc(30vh - 10px)
* studio/chat: welcome offset to 28vh (drop the -10px)
* studio/chat: tighten sloth-to-text gap by 1px (16px to 15px)
* studio/train: revert light active pill to grey fill, drop white bg + shadow
* studio: app-wide hand cursor on every clickable control (disabled excluded)
* studio/chat: welcome offset to 26vh
* studio/chat: welcome offset to 28vh
* studio/chat: harden project and thread list guards against non-array payloads
* studio/sidebar: give the profile row more height and breathing room
* studio/sidebar: trim the profile row top and bottom padding slightly
* studio/sidebar: reduce Train and Recents section label size slightly
* studio/sidebar: trim the profile row top and bottom padding a touch more
* studio/sidebar: enlarge the profile hover area top and bottom
* studio/sidebar: increase profile hover roundness by 1px
* studio/sidebar: trim the profile row top and bottom padding slightly
* studio/sidebar: trim the profile row top and bottom padding slightly
* studio/chat: cache composer line metrics so wrap detection runs once, not per keystroke
* studio/chat: restore the prior view when exiting compare opened from the + menu
* studio/tests: drive Compare from the composer + menu after it moved out of the sidebar
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio/tests: open Compare from the composer + menu in the extra UI suite too
* studio: fix chat dictation microphone access
* studio: snappier plus-to-x spin and steady composer expand gap
Speed up the composer plus icon morph from 480ms to 300ms.
Add row-gap on the expanded composer line so the space between the text
and the controls row stays the same whether the box expanded from
wrapped text or from a toggle being on. The gap sits on the line, not the
input, so the placeholder max-height clamp never crops it.
* studio: only show composer tool pills once a model is loaded
Persisted Search/Code/Canvas/MCP toggles were surfacing the composer pill row on a fresh page load before any model was selected, so an empty composer looked different from the clean just-ejected state. Gate the composerExpanded tool checks on modelLoaded so a model-less composer stays collapsed, while saved preferences still apply the moment a model loads.
* studio: hide RAG composer menu item temporarily
Hide the placeholder RAG entry from the composer plus menu in both single chat and compare until the feature is ready, and drop the now-unused DatabaseIcon import.
* studio: let composer tools pre-select before a model loads
Selecting Web search, Code, Canvas or MCP from the + menu with no model
loaded did nothing visible: the toggle turned on but the composer never
expanded, so the pill stayed hidden. Drop the model-loaded gate from the
expand check so an active tool always surfaces its pill.
Align MCP with the Search/Code pattern too: grey it out only when a loaded
model lacks tool support, so MCP stays toggleable and the pill stays
clickable before a model is loaded instead of looking disabled.
---------
Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: Daniel Han <23090290+danielhanchen@users.noreply.github.com>
* Update Install Scripts
Add SPDX AGPL-3.0 headers to the installer scripts and let the piped web
installs take their common options from the environment.
- install.sh / install.ps1: read UNSLOTH_NO_TORCH (and UNSLOTH_PYTHON for
install.sh) so a piped install needs no positional flags. Flags and the
pipe forms still work; an explicit flag wins.
- Fix the UNSLOTH_STUDIO_HOME example so the variable sits after the pipe
and reaches sh instead of curl.
- Add SPDX headers to install.sh, install.ps1, the uninstall scripts, and
the MLX install scripts.
- Drop the internal test package names from the studio install comments.
* Mirror UNSLOTH_PYTHON env var to install.ps1
install.ps1 now reads UNSLOTH_PYTHON to pin the Python version, matching
install.sh, and lists all three env vars (UNSLOTH_NO_TORCH, UNSLOTH_PYTHON,
UNSLOTH_STUDIO_HOME) in the header examples. The requested version is
preferred during detection and used as the winget install target; behavior
is unchanged when the variable is unset.
The Studio login error rewrote the backend's PATH-based command into a relative Windows path (.\unsloth_studio\Scripts\unsloth.exe ...) that only resolves from inside the Studio home dir and fails with CommandNotFoundException elsewhere. Removes the Windows-only rewrite and the now-unused usePlatformStore import so the backend's unsloth studio reset-password command is shown as-is on all platforms.
Clears the safe set of Dependabot advisories.
Frontend (overrides, all transitive; npm audit now 0):
mermaid 11.14.0->11.15.0, hono 4.12.17->4.12.18, qs 6.15.1->6.15.2,
ip-address 10.1.0->10.1.1 (also clears express-rate-limit), and
brace-expansion 5.0.5->5.0.6 (scoped, the 1.1.14 line is untouched).
Desktop: tauri 2.10.3->2.11.1, pulling the runtime crates it requires
(tao, wry, tauri-runtime, tauri-build, tray-icon).
Backend (test-only): pytest <9.0 -> >=9.0.3,<10, and the pinned
pytest-rerunfailures==15.1 -> >=16.2,<17 (16.2 is the first release with
pytest 9 support). Verified pytest 9.0.3 + rerunfailures 16.3 +
pytest-json-report + pytest-xdist run reruns, fixtures, json reports and
xdist together.
Left out intentionally: transformers (stays 4.x for unsloth compat; patch
is 5.0.0rc3), sqlfluff (major 3->4), glib/rand (stack-coupled transitive).
Fixes#5572. Replaces cmdk's default fuzzy filter with a strict substring-token filter so threads that do not match the query are hidden and the empty state shows. Each item uses its unique thread id as the cmdk value, with the searchable title and preview supplied via keywords.
Adds studio/backend/utils/datasets/dataset_none_detect.py, a standalone scanner that reports None/empty content turns in alpaca, chatml, sharegpt, and gptoss datasets without modifying data, plus generator and runner scripts under tests/utils. Depends only on the datasets library and is not wired into the package init, so it stays import-light.
* Studio: persist Tauri window size and maximized state across launches
* Studio: keep window state under the app home dir, not ~/.config
* Undo an unnecessary change
* Address Gemini's feedback
* Revert to tauri-plugin-window-state implementation
* fix(Studio): restore saved window size before default layout
* Fix cross-platform window-state restore
* fix(Studio): avoid clobbering saved window size
---------
Co-authored-by: Wasim Yousef Said <wasimysdev@gmail.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
* studio: cap training dataset uploads
* studio: clean up failed dataset uploads
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: raise upload limits to 500MB
* studio: make upload limit configurable
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: stream upload routes
* studio: split recipe upload caps
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: tighten upload limit handling
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: import settings router directly
* studio: polish upload cap setting control
* studio: cap settings request bodies
* studio: stub settings route in desktop auth test
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix: toast cancel and style
* fix: align model load toast Cancel and dismiss on the right
* fix: show short cased model name in loaded toast and removed prefix org
* revert: chat load toast refactor to visual-only changes
* fix: align model load toast close button
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: guard empty toast label and dedupe toast padding CSS
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Roland Tannous <rolandtannous@gravityq.ai>
Co-authored-by: Roland Tannous <115670425+rolandtannous@users.noreply.github.com>
* feat: align project sidebar UX with ChatGPT
* feat: align project sidebar UX with ChatGPT
* feat(chat): load stored project list
* feat(chat): add project sidebar workflows
* fix: stabilize project page navigation
* fix: projects chat loading
* fix: show project chat thread
* style: sidebar project spacing and hover clipping
* style: add expandable project chat history and move-to-project submenu
* feat: polish project sidebar
* feat: persist project sandbox paths
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: only create sandbox project workspace dir
* feat: add optional project workspace deletion from delete dialog
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: stabilize chat projects CI failures
* fix: polish project chat navigation
* Studio: manage chat history with projects
Group chats into projects with a dedicated projects page and route.
Sidebar shows recents with per-row actions and a vertical more-vertical
menu, and the sidebar scrollbar stays hidden so rows never shift on
hover. Includes chat settings and composer refinements.
* Studio: projects sidebar and breadcrumb polish
Sidebar:
- Remove the Compare nav item.
- Widen the sidebar to match the projects layout.
- Replace the scroll-gated bottom fade with a static fade pinned above
the profile box, so it no longer attaches to Recents or lags the
collapse and expand animation.
Topbar breadcrumb (chat-page):
- On a project landing show "Projects" linking to the projects list.
- Inside a project chat show the project name and chat title, with the
project name linking back to that specific project page.
- Drop the divider between the model selector and the breadcrumb.
* Studio: make project workspace delete test cross-platform
test_chat_project_delete_files_removes_workspace rooted the project under
pytest tmp_path, which resolves to /private/tmp on macOS. The workspace
delete guard refuses paths under the system denylist by design, so the
test passed on Linux CI but failed on macOS.
Add a workspace_projects_home fixture that keeps tmp_path on Linux and
Windows (CI unchanged) and falls back to a home subdir only when the temp
root is on the platform denylist. Derive the workspace path from the
created project so it tracks the projects home.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: satisfy import-hoist check for new path re-exports
documents_root and project_workspaces_root are re-exported from
utils.paths but only referenced as __all__ string literals, which the
import-hoist safety net does not count as a use. It flagged the two newly
added re-exports as unused imports and failed Source lint.
Name-load both via a module-level _REEXPORTED tuple so the check sees
them used. No behaviour change; consumers still import them from
utils.paths.
* fix: avoid projects empty-state flash
* fix: batch chat search indexing
* Studio: polish chat sidebar, run settings, and search
- Use the native OS scrollbar for the chat sidebar, Run settings panel, and chat search list instead of a custom scrollbar
- Highlight the active run in the sidebar and keep chat search available during training
- Stop the training log view from replaying when navigating back to a run
- Rename the chat settings panel to Run settings and align its toggle icon and position
- Tighten heading and sidebar letter spacing and lighten the Train and Recents labels
- Match the search dialog corner style across light and dark and drop the stray border
- Make the MCP Servers section header plain text instead of a link
- Remove a stray .orig backup file
* studio/frontend: restore Compare entry point in the sidebar
The chat-projects sidebar redesign dropped the Compare nav item and moved
it to thread-sidebar.tsx, which is not imported or rendered anywhere. That
left no way for a user to start a new model comparison (enterCompare only
fired from the guided tour and the training handoff), and broke the
Compare/Recipes/Export UI smoke test that clicks [data-tour="chat-compare"].
Re-add the Compare NavItem to the New Chat / Search group, carrying
data-tour="chat-compare" and the same new-comparison navigation as before.
* studio/frontend: use Unsloth green for the fallback profile avatar
Switch the initials-avatar background from blue to #14b789 so the sidebar
and edit-profile avatar match the Unsloth brand colour.
* studio/frontend: turn project breadcrumb into a project switcher dropdown
* studio/frontend: stop project card kebab clicks from opening the project
* studio/frontend: hide project switcher outside projects
* studio/frontend: stabilize project switcher loading
* style: project switcher alignment
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: shimmyshimmer <107991372+shimmyshimmer@users.noreply.github.com>
Co-authored-by: Unsloth <michaelhan@Michaels-MacBook-Pro.local>
Co-authored-by: Roland Tannous <rolandtannous@gravityq.ai>
Co-authored-by: Roland Tannous <115670425+rolandtannous@users.noreply.github.com>
* fix: support connected models in compare mode
* fix: support connected models in compare mode
* studio/frontend: wait for compare prompt append
---------
Co-authored-by: Roland Tannous <115670425+rolandtannous@users.noreply.github.com>
Improve the rejection message when an MCP server address is not an http(s) URL. It now points to the expected http(s):// form with an example, and only mentions that local commands are disabled when the value contains whitespace (a reliable command signal), since a lone token may just be a scheme-less URL. Wording is host-scoped rather than desktop-only because self-hosted hosts can opt in via an env var. Backend only, with tests; accepted input is unchanged.
Move MCP from the chat settings sidebar to a composer pill next to Artifacts. The dropdown lists keyless presets (Context7, Exa, Hugging Face), all off by default, created on first enable and deduped by URL. Add custom MCP opens the existing manage dialog. Enabled rows show a green underlay and a tick that becomes an X on hover to remove; enabling Exa turns off the built-in Web Search and shows a hover tooltip. Reconciles the per-chat MCP flag on load so already-enabled servers activate the pill. No backend changes.
* Studio: cover B300 (sm_103) with the Linux prebuilt bundles
sm_103 (B300 / GB300 Blackwell Ultra) was in no bundle's supported_sms,
so those hosts fell through to a slow source compile. The newer and
portable bundles already ship base compute_100 PTX, which the driver
JIT-compiles forward to sm_103, so list sm_103 alongside sm_100 in those
bundles and let B300 install the prebuilt.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
setup.ps1 already forwards --has-rocm whenever AMD is detected regardless
of whether gfx resolved. setup.sh only forwarded --rocm-gfx, so a Linux
host where AMD was detected but gfx resolution failed would forward nothing,
leaving the installer with has_rocm=False and falling back to a source build.
Add the elif branch to forward --has-rocm when _setup_amd_detected is true
but _setup_gfx is empty, matching setup.ps1 parity. Add a source-level test
to verify both flags are present in setup.sh.
setup.sh and setup.ps1 resolve the AMD gfx target (rocminfo/hipinfo/amd-smi,
name inference, or UNSLOTH_ROCM_GFX_ARCH) but never passed it to
install_llama_prebuilt.py. The installer re-probed on its own and, on hosts where
hipinfo/amd-smi cannot report the arch (amd-smi-only Linux, HIP-runtime-only
Strix Halo, name-inferred GPUs), left rocm_gfx_target unset. The lemonade HIP
prebuilt selection needs that arch, so those hosts got no GPU prebuilt and fell
back to a source build.
Add a --rocm-gfx argument (defaulting to UNSLOTH_ROCM_GFX_ARCH) and fold it into
the host profile via _apply_host_overrides: a forwarded gfx is authoritative
(setup already applied visible-device selection) and implies has_rocm. setup.sh
and setup.ps1 now forward their resolved arch.
Add unit tests for _normalize_forwarded_gfx and _apply_host_overrides, plus
source checks that both setup scripts forward --rocm-gfx.
* Studio: fall back to source build for arm64 Linux GPU hosts
setup.sh routes a Linux host with any GPU tool to the unslothai/llama.cpp fork,
which publishes only linux-x64 bundles. On an arm64 host with a GPU (GH200,
GB200, DGX Spark) the resolver then selected an x64 CUDA bundle, which cannot run
on aarch64. Routing those hosts to ggml-org instead would install a CPU-only
arm64 build, silently losing the GPU.
Guard resolve_simple_install_release_plans so an arm64 Linux host on the fork
raises PrebuiltFallback before any release is fetched, letting setup.sh do a
source build that actually targets the GPU. x86_64 hosts and arm64 CPU hosts
(which route to ggml-org) are unaffected.
Add tests covering the arm64 fork raise, the x86_64 pass-through, and the arm64
CPU ggml-org path.
* Studio: install ggml-org arm64 CPU prebuilt when the arm64 GPU source build fails
Per review of #5924: arm64 Linux GPU hosts have no CUDA prebuilt anywhere (the
unslothai fork is x64 only, ggml-org ships no Linux CUDA build), so they source
build for the GPU. If that build produces no binary, the host was left without
llama.cpp.
Add a --cpu-fallback flag to install_llama_prebuilt.py that drops the host GPU
attributes so the CPU prebuilt for the host arch is selected (a GPU host cannot
otherwise pick the CPU bundle). setup.sh calls it against ggml-org as a last
resort for arm64 Linux when the source build degraded, installing the
ubuntu-arm64 CPU build instead of leaving the host with no llama.cpp.
Add tests: force_cpu drops GPU attrs before planning, a CPU-forced arm64 host
selects the ggml-org ubuntu-arm64 bundle, and setup.sh wires the fallback.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
The pinned b9360 cuda-13.1 Windows fallback for Blackwell (sm_120) only fired
when the driver advertised CUDA >= 13.1. The mainstream Blackwell branch ships
the r580 driver that reports CUDA 13.0, so those hosts missed the pin, were
gated off the in-release 13.3 build, and dropped to the CPU-only cuda-12.4
build.
b9360's binary is native sm_120a SASS (no PTX, so no JIT) and its bundled
runtime is cuda-13.1 cudart; both run on a 13.0 r580+ driver under CUDA
minor-version compatibility. Lower _PINNED_BLACKWELL_DRIVER_FLOOR to (13, 0) so
the pin covers the 13.0 branch too. The generic published-runtime gate stays
conservative (an unverified in-release 13.1 build is still gated off 13.0); only
the specific, hash-pinned, SASS-verified b9360 build is extended.
Refs #5887.
* Studio: match the Linux llama.cpp prebuilt to the runtime cudart major
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: defer the Windows CUDA Toolkit check so prebuilt users are not blocked
setup.ps1 section 1e validated the installed CUDA Toolkit against the driver in
Phase 1 and hard-exited when the toolkit was newer than the driver supports.
That ran before the prebuilt llama.cpp install (Phase 3.4), which is
self-contained and needs no local toolkit. A Blackwell host (sm_120) on a driver
that advertises CUDA 13.2 with CUDA Toolkit 13.3 installed was blocked entirely,
even though the prebuilt selector would have given it a working GPU build.
The toolkit (nvcc) is only consumed by the Phase 4 source build. Move section 1e
into a Resolve-CudaToolkit function and call it lazily at the source-compile
branch, gated on $HasNvidiaSmi. The prebuilt path no longer touches the toolkit;
forced/source builds keep the identical requirement, winget auto-install, and
error text via -RequireOrExit.
Add a pwsh unit test that extracts the real function and runs it against a
spoofed Blackwell sm_120 / driver 13.2 / toolkit 13.3 host: the prebuilt path
defers, the forced build still fails fast.
Refs #5879.
* Studio: correct a stale Phase-1 prerequisites comment (CUDA Toolkit is now resolved lazily)
* Studio: retrigger Windows GGUF CI (artifact upload flake)
* Studio: pin the last pre-macOS-26 llama.cpp prebuilt instead of walking back
ggml-org moved their macOS build runner to macOS 26 (Tahoe) at b9428, so
b9428 and every newer upstream prebuilt is stamped minos 26 and fails to
dyld-load on macOS 14 / 15. #5883 handled this by walking back release by
release at install time.
Replace that with a deterministic pin: a host below macOS 26 selects b9415
directly (the last upstream build stamped below 26: arm64 minos 14, x64
minos 13.3), so it loads on macOS 13.3 / 14 / 15 / 26. Hosts on macOS 26+
and unknown-version hosts keep latest selection unchanged.
Only the ggml-org upstream path is pinned; the unslothai/llama.cpp fork
ships its own minos-13.3 prebuilts (#5893), so the pin is a no-op there and
goes dormant once macOS routes to the fork. The Mach-O minos preflight from
#5883 stays as a post-download backstop.
Refs #5883, #5893.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* docs: fork ships arm64 minos 14 / x64 minos 13.3, not uniform 13.3
The per-slice fork producer pins arm64 to 14.0 and x64 to 13.3; update the
pinned_macos_release_tag docstring to match. No behavior change.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
The SourcesGroup measurement container renders every citation badge
off-screen with `invisible absolute` so we can measure how many fit in
two rows. Absolute descendants still contribute to the parent's
scrollable overflow region, so long source lists added hundreds of
pixels of phantom scroll space below the assistant message that the
user could scroll into.
Wrap the measurement container in an absolute, `h-0`, `overflow-hidden`
box so the off-screen pills are clipped out of the scrollable overflow
region. Measurement still works because offsetTop is read relative to
the positioned wrapper.
* Studio: harden stdio MCP gating and fix transport edge cases
- Gate the Data Recipe stdio path behind UNSLOTH_STUDIO_ALLOW_STDIO_MCP so a hosted deployment cannot spawn local processes through recipes
- Enforce the gate inside _client() so the transport sink cannot spawn when disabled
- keep_alive=False so stdio probes/calls do not leave orphan subprocesses
- Force OAuth off for stdio servers on create and update
- Drop stored headers when a server switches transport type
- Reject a command whose first token is a URL scheme
- Add MCP gate and improvement tests
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* tests: skip Data Recipe stdio tests when data_designer is absent
The data_designer plugin is only installed in the Studio test job, so guard
the two build_mcp_providers tests with importorskip so the core matrix skips
them instead of failing on ModuleNotFoundError.
---------
Co-authored-by: Daniel Han <michaelhan2050@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Classify llama-server startup failures so diffusion/image GGUFs (FLUX, Qwen-Image, LTX, ERNIE-Image, Z-Image, ...) point users to the Images page instead of a misleading out-of-memory error. Other unknown architectures get a precise unsupported message; Ollama and OOM fallbacks are preserved.
Architecture is matched exactly against general.architecture, covering the arches Unsloth ships as GGUF: flux, qwen_image, ltxv, wan, lumina2.
Fixes#5842.
Make the NVIDIA prebuilt selection version-dynamic and driver-gated on both the
Windows (ggml-org) and Linux (unslothai/llama.cpp) paths, so CUDA version bumps
are handled with no code change while staying safe across driver versions.
- Derive candidate CUDA runtime lines from the driver major; pick the highest
upstream-published minor the driver can actually run, so a sub-13.3 driver is
never handed an unguaranteed 13.3 build.
- Pin b9360's cuda-13.1 build (immutable, hash-verified) as a GPU fallback for a
Blackwell host on a 13.1/13.2 driver that the in-release 13.3 build gates off,
on both the simple and published install paths. Dormant for every other host
and self-disabling once upstream ships a driver-runnable build again.
- Seed the published-path ordering from the release's real published minors so a
future CUDA major is selectable with no code change.
Refs #5861, #5817, #5807, #5858, #5854, #5826, #5887.
Make macOS llama.cpp prebuilt selection host-OS-version aware: skip a prebuilt whose minimum-OS exceeds the host and walk back to the newest release that loads (macOS 26 keeps latest; 14/15 land on a compatible older release). Source-build fallback pins CMAKE_OSX_DEPLOYMENT_TARGET=13.3. CI: binary-load assertion plus a macos-14/15/26 install matrix. No change to Linux/Windows or CUDA selection.
Harden local GGUF detection so a .gguf path is not misrouted to the transformers backend during the brief Windows lock window after llama-server is killed. Catch OSError from stat() and treat the path as the file, while a directory named *.gguf still falls through to the directory scan. Adds regression tests for the lock-window and directory cases.
Follow-up cleanups to the merged AMD ROCm support PR #5301:
1. De-duplicate the torchao Windows-ROCm import stub into a single shared
module (studio/backend/core/_torchao_stub.py); both workers call one
install_torchao_windows_rocm_stub() entrypoint.
2. Align the gfx name/arch comment columns in setup.sh and setup.ps1.
3. Isolate the float16 dtype fallback to AMD without native bf16; NVIDIA
keeps dtype=None so unsloth's own bf16/fp16/FORCE_FLOAT32 detection is
honored.
4. Hoist unconditional stdlib imports (gc, glob, re, subprocess, copy,
types, sys, importlib.metadata) from function bodies to module top
across the PR #5301-touched files; heavy/optional/relative imports stay
lazy.
5. bitsandbytes Windows-ROCm install now uses plain pip (force_pip=True)
instead of UV_SKIP_WHEEL_FILENAME_CHECK, per the AMD hackathon docs.
Also adds scripts/verify_import_hoist.py (a scope-aware LEGB AST resolver
that catches dangling-alias and rename-clash bugs in import-hoist
refactors) and wires it into the Lint CI source-lint job as a self-test
plus a pull_request compare gate.
* feat(studio): use lemonade-sdk/llamacpp-rocm per-GPU prebuilts for ROCm hosts
For AMD GPUs that rocminfo/hipinfo reports a recognised gfx target
(gfx103X / gfx110X / gfx1150 / gfx1151 / gfx120X), resolve_lemonade_rocm_choice()
now fetches the latest lemonade-sdk/llamacpp-rocm release and returns the
matching per-architecture zip, bundling all required ROCm runtime libs.
This runs before the existing upstream ggml-org combined-ROCm tarball fallback
on Linux and before the upstream HIP zip on Windows, so both platforms benefit
from the more targeted build when available.
Changes:
- Add LEMONADE_ROCM_REPO / LEMONADE_ROCM_RELEASES_API constants
- Add HostInfo.rocm_gfx_target populated from rocminfo (Linux) / hipinfo (Windows)
- Add _LEMONADE_GFX_FAMILIES prefix map and _lemonade_gfx_family() helper
- Add resolve_lemonade_rocm_choice() that fetches latest lemonade release and
constructs the llama-{tag}-{os}-rocm-{gfxFamily}-x64.zip asset URL
- Wire into resolve_upstream_asset_choice() for both Linux (ubuntu) and Windows paths
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Honor pinned llama.cpp tag in lemonade ROCm resolver
The resolver always fetched lemonade-sdk/llamacpp-rocm's /releases/latest,
ignoring the upstream llama.cpp tag the caller had pinned. On a reproducible
install where the user requested 'b1260' that meant we would silently pick
up whatever lemonade had published as latest at install time, with no way
to roll back to the matching tag.
Lemonade tags llama.cpp upstream tags 1:1, so:
- When llama_tag is unset or 'latest', keep hitting /releases/latest.
- When llama_tag is pinned (e.g. 'b1260'), hit /releases/tags/b1260.
- When the pinned tag is not published by lemonade (404), skip silently
and let the caller fall through to the upstream tarball -- this keeps
pinned installs reproducible instead of drifting.
resolve_lemonade_rocm_choice now takes llama_tag (default 'latest' for
backward compatibility) and both call sites in resolve_upstream_asset_choice
forward the upstream llama_tag to it.
Note: this PR still has open integration concerns flagged in review --
the simple-policy planner and approved-checksum manifest don't yet route
or accept lemonade assets. Those are larger changes and not in scope for
this commit; addressing the pinned-tag drift independently because it is
small, localized, and self-contained.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add mock test for lemonade ROCm prebuilt asset resolution
Validates GPU family mapping and that resolve_lemonade_rocm_choice
returns real lemonade release URLs for all supported gfx targets on
both Linux and Windows, without requiring AMD hardware.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Wire lemonade ROCm prebuilts into the simple-policy install path
setup.sh invokes install_llama_prebuilt.py with --simple-policy, which
dispatches through resolve_simple_install_release_plans -> direct_linux_release_plan
(or direct_upstream_release_plan on Windows). Those planners only
handled CUDA + CPU attempts, so ROCm-only hosts (e.g. gfx1151 Strix
Halo) had no compatible prebuilt asset and silently fell through to
source build, even though resolve_lemonade_rocm_choice already knew
how to fetch a per-GPU lemonade-sdk binary.
Add a lemonade ROCm/HIP attempt to both simple-policy planners for
ROCm-only hosts, and add regression tests that drive the dispatchers
end-to-end so this can't be skipped silently again.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Document that lemonade ROCm prebuilts work on any glibc Linux
The lemonade-sdk asset filename uses "ubuntu" as a label, but the binary
is a manylinux-style glibc build with no Ubuntu-specific dependencies.
It runs on Arch, Fedora, openSUSE, Debian, etc. as long as the host
glibc is recent enough.
No behavior change -- the dispatch already runs for any Linux ROCm
host. This commit only clarifies the comment, docstring, and log
message so users on non-Ubuntu distros (e.g. Strix Halo on Arch) don't
mistake the asset name for distro gating.
* Pattern matching fix for libggml-cpu*.so*
* fix(lemonade): pass resolved tag to lemonade resolver; add upstream HIP fallback; stub API in tests
- direct_linux_release_plan: pass bundle.upstream_tag (not requested_tag)
to resolve_lemonade_rocm_choice so a "latest" request doesn't mix a
newer lemonade binary with an older planned unsloth release (Codex P2)
- direct_upstream_release_plan: same fix on the Windows path (release_tag
instead of requested_tag); also add the upstream HIP asset
(llama-<tag>-bin-win-hip-radeon-x64.zip) as a fallback between lemonade
and CPU so unsupported GPUs or transient lemonade failures don't silently
downgrade to CPU when an upstream ROCm prebuilt exists (Codex P2)
- test file: stub fetch_json with a synthetic lemonade release payload so
the suite is hermetic and not subject to GitHub API rate limits (Codex P1);
add test_simple_policy_windows_hip_falls_back_to_upstream_when_lemonade_unavailable
to cover the new HIP fallback path
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(lemonade): revert bad tag fix; keep upstream HIP fallback + hermetic tests
The previous commit wrongly passed bundle.upstream_tag / release_tag to
resolve_lemonade_rocm_choice. Lemonade uses its own versioning (b1262,
b1264, …) completely independent of unslothai's tags (b9186, …), so
passing a resolved unslothai tag caused a 404 and silently skipped the
lemonade binary entirely. Revert both call sites to requested_tag.
Keep the two valid fixes from the prior commit:
- Upstream HIP fallback (llama-<tag>-bin-win-hip-radeon-x64.zip) between
lemonade and CPU in direct_upstream_release_plan, so unsupported GPUs
or transient lemonade failures don't silently downgrade to CPU (Codex P2)
- Stub fetch_json in tests so the suite is hermetic (Codex P1)
* fix(studio/rocm): respect HIP_VISIBLE_DEVICES when picking lemonade gfx target
The rocminfo / hipinfo regex took the first gfx match in the agent listing.
On mixed APU + dGPU hosts (e.g. Strix Halo gfx1151 + discrete RX 7900 gfx1100)
this picked whichever GPU appeared first in the tool's stdout, not the one
HIP actually runs on. The downloaded lemonade asset could then be a binary
for a different arch than the active device.
Extracted a module-level _pick_rocm_gfx_target() helper that:
- collects every gfx token in order via re.findall (skips gfx000 / generic ISAs)
- if HIP_VISIBLE_DEVICES or ROCR_VISIBLE_DEVICES is set, parses the first
comma-separated entry as an integer index into that list
- falls back to the first GPU for non-integer (UUID-style) or out-of-range
values, matching the previous default behaviour
Both Linux (rocminfo) and Windows (hipinfo) branches use the helper.
Existing 18 lemonade tests pass; no behavioural change for single-GPU hosts.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio/rocm): dedup rocminfo gfx tokens and honor disabled visibility
Follow-up to 8793aef0. rocminfo / hipinfo emit each gfx target multiple
times per GPU (Name, ISA triple, marketing name), so the prior re.findall
indexing returned the wrong device when HIP_VISIBLE_DEVICES picked GPU 1
on a mixed-arch host -- the helper picked the second occurrence of GPU 0
instead. Collapse to unique tokens (insertion-ordered) before indexing.
Also handle HIP_VISIBLE_DEVICES / ROCR_VISIBLE_DEVICES values of '' and
'-1' as "no AMD visible" (matches the rest of Studio's visibility code),
returning None so the planner does not pick a Lemonade asset for a
hidden GPU.
* fix(studio/rocm): memoise lemonade release lookup + HIP_PATH hipinfo fallback
Robustness pass on top of 25d4ab63:
1. resolve_lemonade_rocm_choice() is called twice per install (planner +
resolve_upstream_asset_choice), so every install previously hit
api.github.com twice with identical args -- doubling the 403/rate-limit
failure surface on busy CI runners. Extract the fetch into a
functools.lru_cache(maxsize=8) helper keyed on (api_url, llama_tag).
The cached helper also owns the error-path logging the resolver was
doing inline. Tests that need to vary fetch_json output across
invocations should call _fetch_lemonade_release_cached.cache_clear().
2. Windows detect_host probe for hipinfo / amd-smi only used
shutil.which(), so HIP SDK installs that set HIP_PATH but do not put
%HIP_PATH%\bin on system PATH classified the host as non-ROCm. The
PowerShell installer and studio/install_python_stack.py already
resolve HIP_PATH\bin\hipinfo.exe as a fallback; mirror that here so
the install planner agrees with the rest of Studio on what counts as
a ROCm host.
Tests: 18 passed (shipped); 52 extra sim cases pass (added 2 for the
lru_cache deduplication path).
* fix(studio/rocm): lemonade URL trust pinning, runtime overlay covers HIP libs, opt-out env
Robustness pass driven by 5 parallel reviewers of head c08b15e6:
1. URL trust pinning. AssetChoice.url comes from the GitHub API response's
browser_download_url field. Lemonade attempts are not in the approved-hash
manifest, so a compromised API response could redirect the download to an
attacker-chosen host without the integrity gate catching it. New
_is_trusted_github_release_url() validates https + github.com/<expected_repo>
release path OR objects.githubusercontent.com (GitHub's CDN). Resolver
refuses to download otherwise.
2. UNSLOTH_DISABLE_LEMONADE_ROCM opt-out. Users who prefer the upstream HIP
build path can set this env var to skip lemonade outright (e.g. for
air-gapped installs or stricter trust requirements). The install log
already prints a NOTE explaining that lemonade lacks approved-hash
coverage so users know the trust model.
3. linux-rocm runtime overlay patterns extended to cover lemonade's bundled
HIP/ROCm runtime libs (libamdhip64.so*, libhsa-runtime64.so*, libhipblas*,
librocblas*, librocsolver*, librocsparse*, librocrand*, libMIOpen*,
libmagma*). The upstream tarball does not ship these (links against
system /opt/rocm), so the new glob entries are no-op for upstream and
load-bearing for lemonade. Without this, install_from_archives would
drop the bundled runtime libs from the lemonade ZIP and llama-server's
RPATH would fail to load amdhip64 at first inference.
4. _lemonade_release_api_for now URL-encodes llama_tag with quote(safe="").
Defence in depth: a tag containing /, ?, #, or whitespace cannot reshape
the request URL. Tags come from internal resolution today, but this
removes the risk if a future caller passes user-controlled input.
5. Empty browser_download_url skipped explicitly in the resolver. The
release_asset_map helper defaults missing URLs to "". Previously this
would have been passed to download_file("") which raises a less obvious
error than the new clean log + return None.
6. Docstring on _lemonade_release_api_for clarified: lemonade tags match
ggml-org/llama.cpp tags 1:1, NOT unslothai/llama.cpp fork tags. The
earlier P1 review report misread this and re-asked for the tag-drift
fix that the author intentionally reverted in f256cea950.
7. Autouse pytest fixture in test_lemonade_llamacpp_rocm_bins_mock.py
clears _fetch_lemonade_release_cached between tests. Today's tests all
mock the same payload so no pollution surfaces, but the lru_cache
becomes a footgun the moment any future test parametrises return values.
Tests: 28 passed (was 18, added 10 covering URL pinning, opt-out env,
pinned-tag helper, URL encoding, empty URL, runtime patterns).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio/rocm): complete lemonade runtime overlay + honor CUDA_VISIBLE_DEVICES
runtime_patterns_for_choice("linux-rocm") was missing libamd_comgr.so*,
librocm_kpack.so*, and librocm_sysdeps_*.so*, which are direct NEEDED
entries of libamdhip64.so.7 in every lemonade bundle. install_from_archives
did not copy them, so the preflight ldd walk failed on llama-server and every
lemonade attempt fell back to source build. Confirmed on gfx1151 b1272 by
h34v3nzc0dex; manually staging the full bundle with the three missing
patterns restored the install and preserved bench parity.
Also extend _pick_rocm_gfx_target to check CUDA_VISIBLE_DEVICES after
HIP_VISIBLE_DEVICES and ROCR_VISIBLE_DEVICES: AMD's HIP runtime honours
all three with identical semantics, so mixed-arch hosts where users set
only CUDA_VISIBLE_DEVICES were still picking the first gfx token instead
of the user-selected device.
Tests: 30 passed (was 28); added 2 for CUDA_VISIBLE_DEVICES (multi-GPU
pick + -1 opt-out) and extended the runtime-patterns test to assert the
three newly added lib globs are present.
* fix: tighten CDN trust check and fix multi-GPU arch selection
- _is_trusted_github_release_url: require /github-production-release-asset-
path prefix so only real GitHub release CDN URLs are accepted
- _pick_rocm_gfx_target: parse rocminfo Agent N section boundaries to build
a per-physical-GPU arch list; same arch across multiple GPUs no longer
collapses to a single token, so HIP_VISIBLE_DEVICES indexing works correctly
- tests: update CDN test to use realistic path prefix, add rejection test for
arbitrary CDN path, add regression test for same-arch multi-GPU case
* fix: use broad lib*.so* glob for linux-rocm runtime overlay
Replace the explicit lib allowlist in runtime_patterns_for_choice with
lib*.so* for the linux-rocm path.
The lemonade ROCm ZIPs carry a full HIP/ROCm runtime including transitive
deps like libLLVM.so.23.0git and libclang-cpp.so.23.0git (pulled in by
libamd_comgr.so.3). These names change across ROCm releases and were not
in the allowlist, so preflight would see them as unresolved NEEDED entries
and fall back to a source build. The broad glob catches everything in the
bundle now and in future releases without needing to enumerate each library.
* fix: show lemonade binary tag in install summary log line
Store binary_repo and binary_release_tag in UNSLOTH_PREBUILT_INFO.json
so the setup.sh summary can distinguish the source tree (unslothai/llama.cpp)
from the actual binary origin (lemonade-sdk/llamacpp-rocm).
Before: 'installed release: unslothai/llama.cpp@b9334'
After: 'installed release: unslothai/llama.cpp@b9334 + lemonade@b1280'
Upstream installs are unchanged (binary_repo == published_repo).
* fix(merge-compat): align Windows ROCm guard and helper name with strix branch
- elif host.has_rocm instead of if...not to match fix/rocm-strix-halo-unified-memory
- _resolve_exe instead of _resolve_amd_exe with identical body/docstring
Eliminates the two conflict hunks that would otherwise block a clean bot merge
of feature/lemonade-rocm-prebuilts on top of fix/rocm-strix-halo-unified-memory.
* fix: three PR review corrections for lemonade ROCm prebuilt integration
- _lemonade_release_api_for: fix docstring claiming lemonade tags match
ggml-org 1:1 -- lemonade may be several builds behind ggml-org (noted
by oobabooga, confirmed: lemonade b1281 vs ggml-org b9370)
- direct_linux_release_plan: move cpu_choice into else-branch so ROCm-only
hosts never get a CPU fallback in the attempts list -- a failed lemonade
binary now raises PrebuiltFallback and triggers the HIP source build
instead of silently installing a CPU-only binary
- apply_approved_hashes: pass lemonade attempts through without requiring
a manifest entry; lemonade is explicitly documented as relying on
functional validation only, so rejecting it here caused PrebuiltFallback
on the non-simple-policy path before the upstream ROCm/HIP fallback
could be considered
* fix: copy hipblaslt/rocblas library subdirs from lemonade ROCm archives
copy_globs matches filenames only and copies flat, so it cannot
preserve the hipblaslt/library/<gfx>/ and rocblas/library/<gfx>/
Tensile kernel catalog trees that lemonade ROCm zips ship alongside
the .so files. Add runtime_subdirs_for_choice() and call shutil.copytree
for each named subdir after the copy_globs pass in install_from_archives.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Jaeic Lee <jaeiclee@users.noreply.github.com>
* fix(studio): set HIP_VISIBLE_DEVICES in apply_gpu_ids for ROCm training workers
Training workers are spawned via multiprocessing spawn before detect_hardware()
runs, so IS_ROCM is still False. If the user never set HIP_VISIBLE_DEVICES in
their shell, _inherits_rocm_visibility is also False, leaving the worker with
only CUDA_VISIBLE_DEVICES set. On ROCm hosts the HIP runtime honors
HIP_VISIBLE_DEVICES over CUDA_VISIBLE_DEVICES, so the worker saw the full
device list and torch raised "no usable HIP accelerator" on some setups.
Fall back to probing torch.version.hip (a build-time attribute, safe to read
before GPU init) to detect ROCm when neither IS_ROCM nor inherited env vars
are available. Mirrors the existing fix in llama_cpp.py for llama-server
subprocess GPU pinning.
Fixes https://github.com/unslothai/unsloth/issues/5180
* test: tighten apply_gpu_ids ROCm fallback assertions
Replace loose OR chain with exact string matches, split into three
focused tests, and add a guard check for the try/except wrapper.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: detect ROCm unified memory (Strix Halo / AMD iGPU) via torch fallback
amd-smi on iGPUs with shared/unified memory (e.g. Radeon 8060S on Strix
Halo) reports only the dedicated VRAM slice (~512 MB) in its metric output,
so get_visible_gpu_utilization() was returning usable_gb ≈ 0.35 GB instead
of the full GTT pool (~128 GB). torch.cuda.mem_get_info() already surfaces
the correct unified-pool size.
Add _reconcile_rocm_unified_memory(): after amd-smi returns a valid result
on a ROCm device, cross-check each device's vram_total_gb against
torch.cuda.mem_get_info(). When torch reports a larger total, replace the
amd-smi VRAM fields in-place. No-op for discrete AMD GPUs where the two
sources agree.
Fixes: "Falling back to all visible GPUs -- model may not fit" on AMD iGPU
machines even when 100+ GB of unified memory is available.
* Apply unified-memory reconciliation in get_gpu_utilization too
The visible-GPU path was already corrected for AMD iGPUs with unified memory
(Strix Halo / Radeon 8060S), but get_gpu_utilization was still returning the
raw 512 MB amd-smi VRAM slice. Studio's /api/train/hardware endpoint and the
live GPU monitor read from this primary path, so users continued seeing the
wrong total even after auto_select_gpu_ids picked the right device.
Refactor to share the per-device correction:
* _apply_unified_memory_correction(metrics, torch_info) -- the actual
replacement logic, in-place on a single metrics dict.
* _reconcile_rocm_unified_memory(...) -- multi-device,
iterates utilization["devices"] (visible-GPU path).
* _reconcile_primary_rocm_unified_memory(...) -- single flat
metrics dict (primary-GPU path), uses parent_visible_spec to pick the
primary index, falls back to ordinal 0 when no visibility env is set.
get_gpu_utilization now calls the primary reconciler under IS_ROCM, so both
endpoints surface the real unified-memory pool on iGPUs while leaving
discrete AMD GPUs untouched (torch_total <= smi_total -> no replace).
* Use 'is not None' and log debug on torch.version.hip probe failures
Two small follow-ups to the apply_gpu_ids ROCm fallback:
1. Match detect_hardware()'s 'getattr(torch.version, "hip", None) is not None'
form so the entire codebase has one canonical 'this torch was built with
HIP' check. On every shipping torch wheel hip is either None or a non-empty
version string, so the new form agrees with the old bool() form on every
real install.
2. Log the probe failure at debug level instead of swallowing it silently.
The broad 'except Exception' is intentional (we never want apply_gpu_ids
to crash a worker over a probe), but the silent pass made it impossible
to tell whether the fallback was firing or being skipped.
* fix(studio): honour HIP_VISIBLE_DEVICES in _get_parent_visible_gpu_spec before IS_ROCM is set
When a user has HIP_VISIBLE_DEVICES set in their shell (e.g. "1" to select
GPU 1) but detect_hardware() has not yet run in the Studio parent process,
IS_ROCM is still False. _get_parent_visible_gpu_spec() was gated on IS_ROCM
so it fell through to CUDA_VISIBLE_DEVICES (unset), saw all physical GPUs,
and auto-selected index 0. apply_gpu_ids then overwrote HIP_VISIBLE_DEVICES
with "0", making the intended GPU invisible to ROCm torch in the worker,
which triggered the "no usable HIP accelerator" error (issue #5180).
Apply the same _inherits_rocm_visibility pattern already used in
apply_gpu_ids: check for HIP_VISIBLE_DEVICES / ROCR_VISIBLE_DEVICES in the
environment regardless of IS_ROCM so the correct GPU index is preserved.
* fix(install): harden AMD ROCm GPU detection for multi-GPU and env-filtered setups
The previous rocminfo awk pattern could miss discrete GPUs on machines
where HIP_VISIBLE_DEVICES/ROCR_VISIBLE_DEVICES is used to mask an
integrated GPU — the env vars filter rocminfo output but may not
propagate into the install script subprocess, causing detection to
fail entirely.
Two changes:
- Tighten rocminfo pattern from /gfx[0-9]/ && !/gfx000/ to
/gfx[1-9][0-9]/ — simpler and correctly excludes the CPU agent
(gfx000) without a negative lookahead
- Add sysfs KFD topology fallback: reads
/sys/class/kfd/kfd/topology/nodes/*/gpu_id which is a kernel-level
view unaffected by HIP_VISIBLE_DEVICES or ROCR_VISIBLE_DEVICES
Fixes detection failure reported in Discord by Chains (gfx1201 + iGPU
machine where env var exclusion of the iGPU caused rocminfo to return
no usable device).
* Fix KFD sysfs awk fallback to read properties file
The fallback added by this PR reads /sys/class/kfd/kfd/topology/nodes/*/gpu_id
files but matches the literal token 'gpu_id' against their content. Those
files contain only a single decimal value (e.g. '0' for CPU agents, '50432'
for GPU agents), so the regex never matches and 'found' stays 0, making the
fallback a no-op on every host. The properties file in the same directory
contains key/value lines like 'gpu_id 50432' which is what the existing awk
pattern expects.
Reproduced with a synthetic sysfs layout: against gpu_id files awk exits 1;
against properties files awk exits 0 when any node reports gpu_id > 0.
* fix(setup.ps1): detect AMD ROCm GPU on Windows, bring to parity with setup.sh
setup.ps1 only checked nvidia-smi and fell straight to "gpu: none" on AMD
machines. setup.sh already probed rocminfo/amd-smi/hipconfig/hipinfo.
Add three-tier detection mirroring install_llama_prebuilt.py's detect_host():
1. hipinfo: gcnArchName in output confirms a real HIP GPU (not just SDK)
2. amd-smi list: "GPU: <digit>" data rows as fallback
3. WMI Win32_VideoController: last resort -- detects AMD GPU even without
HIP SDK, then guides user to install it rather than silently going CPU
Also corrects the "none" message to mention AMD ROCm alongside NVIDIA so
users with AMD hardware understand the requirement.
Fixes: rohit-style install where Strix Halo (Radeon 8060S) showed
"gpu: none" even with the HIP SDK present.
* fix(install.ps1): detect AMD ROCm GPU on Windows, bring to parity with setup.ps1
install.ps1 had the same nvidia-smi-only GPU detection as setup.ps1 before
the setup.ps1 fix. Applies the same three-tier AMD detection:
1. hipinfo: gcnArchName confirms real HIP GPU
2. amd-smi list: GPU data rows as fallback
3. WMI Win32_VideoController: detects AMD GPU without HIP SDK and guides
user to install it
Fixes: install.ps1 showing "gpu: none" while setup.ps1 correctly showed
"AMD GPU detected" on the same machine (reported by rohit, RX 7600 XT).
* fix(install.ps1): suppress 'No NVIDIA GPU detected' when AMD GPU is present
* feat: add Windows AMD ROCm PyTorch wheel installation
install_python_stack.py:
- Add _ROCM_WINDOWS_WHEEL_BASE and _ROCM_WINDOWS_RELEASES constants
pointing to AMD repo.radeon.com (ROCm 7.2 -> torch 2.9.1+rocm7.2.1)
- Extend _ensure_rocm_torch() with a Windows branch: detects ROCm via
_has_rocm_gpu() / _detect_rocm_version(), requires Python 3.12 (cp312
is the only ABI AMD publishes for Windows), installs the direct wheel
URL from repo.radeon.com
install.ps1:
- Capture ROCmVersion during AMD detection via hipconfig --version /
amd-smi version (needed for wheel URL selection)
- After Get-TorchIndexUrl, add an AMD wheel override block: when HasROCm
and Python 3.12 detected, set ROCmTorchWheelUrl to AMD wheel URL
- Expand torch install branch to handle ROCmTorchWheelUrl with
uv pip install --force-reinstall --no-cache-dir
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: also install torchvision and torchaudio from AMD Windows repo
AMD publishes matching torchvision-0.24.1+rocm7.2.1 and
torchaudio-2.9.1+rocm7.2.1 cp312 wheels at the same repo.radeon.com
release folder. Install all three in both install.ps1 and
install_python_stack.py Windows ROCm path.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* feat: add ROCm 7.1.1 Windows wheel mapping
AMD uses a different version string for 7.1.1 wheels:
2.9.0+rocmsdk20251116 (date-tagged) instead of +rocm7.1.1.
Adds the 7.1.1 release folder to both install.ps1 and
install_python_stack.py so users with ROCm 7.1 get ROCm
torch instead of falling back to CPU.
* fix: install rocm_sdk_core and rocm_sdk_libraries_custom alongside torch
The AMD Windows torch wheels declare rocm[libraries]==<ver> as a hard
dependency. Without installing rocm_sdk_core and rocm_sdk_libraries_custom
from the same AMD release folder, uv cannot resolve the dependency and
fails with 'No solution found'. Include all 5 wheels in one install call.
* fix: expand ROCm wheel array to scalars for Invoke-InstallCommand
@array splatting inside a scriptblock only works when the native command
is prefixed with '&'. Invoke-InstallCommand uses '& $Command' to run the
block, so @ROCmAllWheelUrls was not being expanded. Extract to scalar
variables $rw0-$rw4 which are captured correctly by the closure.
* fix: use --no-deps for AMD Windows torch wheel install
uv's resolver looks up rocm[libraries]==0.1.dev0 on PyPI during
dependency resolution before downloading any wheels, and fails because
the package doesn't exist on PyPI. --no-deps skips resolution entirely
and installs all 5 AMD wheels directly. The GPU runtime dependency is
satisfied by the HIP SDK, not a Python package.
* fix: setup.ps1 and install_python_stack.py now install ROCm torch on Windows
setup.ps1 was always setting CuTag='cpu' for non-NVIDIA hosts and installing
cpu-only PyTorch, overwriting the ROCm torch installed by install.ps1.
Adds the same AMD wheel selection logic (ROCm version detection, Python 3.12
check, 5-wheel install with --no-deps) to setup.ps1's torch install block.
install_python_stack.py: remove IS_WINDOWS guard from _ensure_rocm_torch()
call site so the Windows path in _ensure_rocm_torch() is reachable during
'unsloth studio update' as well.
* fix: suppress manual-install warning when ROCm torch already present; fix progress counter
- Gate the 'must be installed manually' warning on torch.version.hip being empty
so it doesn't fire when our ROCm torch install succeeded
- Update _TOTAL counter to include the 3 ROCm steps on Windows now that
_ensure_rocm_torch() is called there (fixes 10/9 display)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* feat: add rocm step display in setup.ps1; fix warning and progress counter
- Add 'rocm' step after 'cuda' in setup.ps1 showing ROCm version or HIP SDK missing
- Move ROCm version detection up to GPU detection block so it's available early
- Suppress 'must be installed manually' warning when torch.version.hip is set
- Fix _TOTAL counter to include ROCm steps on Windows (fixes 10/9 display)
* fix: detect AMD SDK ROCm torch via __version__ when torch.version.hip is unset
AMD's repo.radeon.com wheels (e.g. 2.9.0+rocmsdk20251116) do not set
torch.version.hip, leaving it None. All three probes that relied solely on
torch.version.hip now also check for 'rocm' in torch.__version__.lower():
- hardware.py detect_hardware(): IS_ROCM was never set, causing the studio
to report 'Hardware detected: CPU' even after AMD wheels were installed
and HIP DLLs were on PATH.
- install_python_stack.py _ensure_rocm_torch(): skip-if-already-installed
probe would always reinstall on subsequent runs.
- install_python_stack.py Windows AMD warning: suppression check always
failed, so the 'must be installed manually' note kept appearing after
a successful AMD wheel install.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* perf: drop --no-cache-dir from AMD ROCm torch wheel installs
uv caches downloaded wheels by default; passing --no-cache-dir forced a
full redownload of the ~2 GB torch wheel on every install run. CUDA installs
never had this flag -- AMD was the only path affected.
* fix: use install-state flag instead of subprocess probe for AMD Windows warning
Replace the subprocess torch probe in the post-install warning block with a
module-level _rocm_windows_torch_installed flag set by _ensure_rocm_torch().
Subprocess re-import of torch is unnecessary and fragile -- the install
function already knows whether it succeeded.
* fix: hoist global declaration to top of _ensure_rocm_torch
Python requires the global statement to appear before any assignment
to the variable within a function. Moving it to the function top fixes
the SyntaxError on line 354.
* fix: pass AMD torch install status via env var to suppress false warning
setup.ps1 now sets UNSLOTH_ROCM_TORCH_INSTALLED=1 after a successful AMD
wheel install. install_python_stack.py reads this at the top of
_ensure_rocm_torch() to skip both the subprocess probe and the warning --
no re-import of torch needed, and the warning message now correctly says
'could not be auto-installed' rather than 'must be installed manually'.
* fix: register ROCm DLL directory before torch import on Windows
Python 3.8+ ignores PATH for extension DLL loading on Windows; amdhip64.dll
and other HIP runtime DLLs must be registered via os.add_dll_directory().
Without this, torch.cuda.is_available() always returns False on AMD ROCm
Windows even when HIP_PATH is correctly set in system environment variables.
Reads HIP_PATH / ROCM_PATH env vars first, then falls back to scanning
common ROCm install roots (C:\Program Files\AMD\ROCm, F:\ROCm, C:\ROCm).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: remove hardcoded non-standard ROCm paths from DLL directory scan
Only use HIP_PATH/ROCM_PATH (set by AMD installer) and the standard
C:\Program Files\AMD\ROCm\<version>\bin location. Custom drive paths
like F:\ROCm are user-specific and should not be hardcoded.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: prevent torchao overrides step from overwriting AMD ROCm torch
torchao==0.14.0 in overrides.txt declares torch as a dependency. Without
--no-deps, uv resolves torch from PyPI and installs 2.11.0+cpu on top of
the AMD ROCm wheels (2.9.0+rocmsdk20251116). This was the root cause of
'Hardware detected: CPU' -- the AMD wheels were installed but then
immediately overwritten by the overrides step.
When _rocm_windows_torch_installed is True, add --no-deps to the overrides
pip_install call so torchao is installed without pulling in CPU torch.
* fix: add rocm_sdk namespace tarball to Windows ROCm wheel installs
torch/_rocm_init.py calls `import rocm_sdk` at startup, which requires
the rocm namespace tarball (rocm-*.tar.gz) in addition to the SDK wheel
packages. This tarball was missing from both install.ps1 and setup.ps1,
causing ModuleNotFoundError on first torch import.
- Add rocm-0.1.dev0.tar.gz to ROCm 7.1.1 install (provides rocm_sdk namespace)
- Add rocm-7.2.1.tar.gz + rocm_sdk_devel to ROCm 7.2.1 install
- Install tarball in a dedicated step before main SDK/torch wheels
- Switch to @array splatting in install.ps1 scriptblock for dynamic wheel count
- Remove --no-cache-dir from Python-side ROCm wheel install (prevents ~2GB redownload)
* feat: enable ROCm 7.2 torch install + warn on gfx1151 with ROCm < 7.2
Chigoma333 (AMD Radeon 8060S / gfx1151, Strix Halo) confirmed that ROCm
7.1 segfaults when tensors are moved to GPU, but ROCm 7.2 + torch
2.11.0+rocm7.2 works fully including training.
Changes:
- Uncomment (7,2): "rocm7.2" in _ROCM_TORCH_INDEX (was blocked by <2.11.0)
- Add _ROCM_TORCH_PKG_SPECS dict with per-tag version bounds:
rocm7.2 → torch>=2.11.0,<2.12.0; all older tags → <2.11.0
- Add _detect_amd_gfx_codes() helper that parses rocminfo output
- Warn on gfx1151/gfx1150 (Strix Halo) when ROCm < 7.2 is installed,
pointing users at the known segfault and recommending upgrade
- install.sh get_torch_index_url(): enable rocm7.2 case (previously capped
to rocm7.1), cap unknown future tags to rocm7.2
- install.sh: override TORCH_CONSTRAINT to >=2.11.0,<2.12.0 when rocm7.2
index is selected, so pip can actually resolve torch 2.11.0
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: prefer Python 3.12 for AMD ROCm users when 3.13 is also installed
After GPU detection, if ROCm HIP SDK is found and the selected Python
is not 3.12, run a second pass to locate a 3.12 install via py.exe and
PATH (catches uv-managed installs). Switch $DetectedPython to 3.12 so
the venv is created with a compatible interpreter for the cp312-only AMD
Windows torch wheels.
NVIDIA and Intel GPU paths are unaffected -- the re-detection block only
runs when $HasROCm is true.
Fixes: #5301
* fix: also check uv-managed Python 3.12 for AMD ROCm #5301
* fix: hide amd-smi console popups on Windows, guard torch.distributed.is_initialized for ROCm #5301
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: suppress remaining console popups on Windows, patch torch.distributed.is_initialized for ROCm #5301
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: stub all missing torch.distributed attrs for ROCm Windows wheel #5301
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: inject torch.distributed stub when C backend missing in ROCm Windows wheel #5301
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(rocm/windows): pre-stub torch._C._distributed_c10d + raise amd-smi timeout
Two fixes for Windows ROCm regressions reported by electroglyph on #5301:
1. worker.py — torch.distributed stub now fires unconditionally on Windows
The previous stub only injected sys.modules in the except branch, meaning
it was silently skipped when `import torch.distributed` happened to succeed
(the C backend is lazily resolved). The crash then hit later when
transformers/trl triggered the lazy load. Fix: on win32 we pre-populate
sys.modules['torch._C._distributed_c10d'] AND set the attribute on the
torch._C extension module *before* attempting the import, covering both
the early-ImportError and lazy-load failure modes.
2. amd.py — increase amd-smi timeout from 5 s to 30 s on Windows (10 s Linux)
amd-smi on Windows must cold-init the ROCm runtime on first invocation;
5 s was consistently too short, producing repeated 'Command timed out'
warnings in the server log. 30 s gives enough headroom without blocking
indefinitely on broken installs.
3. install.ps1 — widen Python 3.12 enforcement to ROCmGpuLabel (WMI-only path)
Users whose HIP SDK is not on PATH were detected via WMI but not switched
to Python 3.12 before the install started, causing a second pass. Guard
now fires on (HasROCm -or ROCmGpuLabel).
* fix(rocm): guard c10d stub, fix TorchIndexFamily for 7.1, clean dead code + comments
- worker.py: wrap c10d stub injection in `if _c10d_key not in sys.modules` so
Windows NVIDIA users with a real torch.distributed are never affected
- install.ps1: fix Get-TauriTorchIndexFamily receiving hardcoded "rocm7.2"
even when ROCm 7.1 wheels are installed; now branches on $ROCmVersion
- main.py: remove dead `import ctypes as _ctypes` (ctypes is never called)
- hardware.py, install_python_stack.py, worker.py, install.ps1: shorten
verbose multi-line comment blocks throughout
- tests: update 4 stale assertions that expected rocm7.2 to be absent/capped
* fix(tests): match windows AMD warning assertion to actual source string
* chore: trim verbose comment blocks across all ROCm-related files
* fix: guard reconcile call against None numeric_ids; add torchvision lower bounds
* fix(install.ps1): recreate venv with Python 3.12 after ROCm switch
Venv was created with 3.13 before GPU detection ran; switching
$DetectedPython to 3.12 had no effect since $VenvPython still
pointed to the 3.13 interpreter inside the already-created venv.
* ux: detect AMD GPU before Python selection to avoid double venv creation
- Early hipinfo + WMI probe runs before Find-CompatiblePython so Python
3.12 is selected upfront when AMD is detected; venv is now created
exactly once instead of 3.13 then immediately 3.12.
- Post-venv recreation block replaced with a simple warning for the rare
case where AMD was missed by the early probe.
- setup.ps1: show venv's actual Python version (e.g. 3.12) instead of
the system Python found by the pre-activation search (was showing 3.13).
* fix(rocm/win): auto-stub all _distributed_c10d symbols via PEP-562 __getattr__
The bare ModuleType stub caused ImportError when torch._dynamo was imported
(triggered by trainer.py accessing torch._dynamo.config at load time).
torch._dynamo pulls in torch.distributed.fsdp._flat_param which does:
from torch._C._distributed_c10d import FakeProcessGroup
and potentially other symbols. Adding module __getattr__ auto-creates a
stub class for any missing symbol so all such imports succeed without
enumerating every individual symbol. Applied to both the primary stub
and the fallback stub in the except branch.
* chore: trim c10d stub comment
* fix(rocm/win): auto-stub missing torch.distributed attrs (Store, ProcessGroup, …)
* fix(rocm/win): pre-stub fsdp submodules in sys.modules; fix __getattr__ subpackage clash
* feat(rocm/win): arch-aware wheel selector always picks newest ROCm release
Replace HIP-SDK-version-gated wheel selection with GPU arch-based logic.
Select-ROCmWheelRelease (PS) and _select_windows_rocm_release (Python) map
gcnArchName → minimum ROCm version, then pick the newest available release
that satisfies it (currently always rocm-rel-7.2.1 for any supported GPU).
Wheels bundle their own ROCm runtime so the installed HIP SDK 7.1 does not
prevent using 7.2.1 wheels on gfx1200 (RX 9060 XT) and similar RDNA 4 GPUs.
Also installs the bitsandbytes Windows ROCm continuous-release wheel and sets
BNB_ROCM_VERSION=72 in worker.py before ML imports so bnb loads the
libbitsandbytes_rocm72.dll that ships in that wheel.
* fix(rocm/win): stub class metaclass for ProcessGroup.BackendType; amd-smi circuit breaker
torchao.float8.inference accesses ProcessGroup.BackendType as a class-level
attribute. Plain type() stubs have no __getattr__ on the metaclass so this
raises AttributeError. Introduce _StubClassMeta whose __getattr__ returns
child stub classes, fixing the torchao import chain.
Add an amd-smi circuit breaker in amd.py: after 3 consecutive failures the
module stops spawning the process, eliminating the repeated Windows UAC /
DiskPart elevation prompts caused by polling a non-functional amd-smi.
Also guard BNB_ROCM_VERSION=72 behind a DLL existence check so bitsandbytes
fails with its own detection message rather than a harder "DLL not found" when
the Windows ROCm bnb wheel is not yet installed.
* fix: stub __members__ so torchao float8 enum check doesn't crash on ROCm Windows
torchao.float8.inference accesses ProcessGroup.BackendType.__members__
expecting a Python Enum registry dict. _StubClassMeta.__getattr__ was
blocking all dunder attributes, causing AttributeError. Return {} for
__members__ specifically so the isinstance/iteration checks pass cleanly.
* fix: stub distributed tensor/functional_collectives to prevent missing C++ op crash on ROCm Windows
torch._dynamo.trace_rules eagerly loads torch.distributed.tensor at import
time, which pulls in _functional_collectives.py. That file registers Meta
kernels for _c10d_functional C++ ops, but those ops are only registered
by torch._C._distributed_c10d — a C extension absent from ROCm Windows
wheels. Pre-stubbing the affected modules in sys.modules prevents the real
import chain from running and avoids the "operator does not exist" crash.
* fix: give mod stubs __path__ and pre-stub _tensor to fix 'not a package' import error
_make_mod_stub now sets __path__=[] so Python treats stub modules as
packages. Without it, any import of a submodule raises "is not a package".
Also pre-stub torch.distributed._tensor and its submodules so that
_tensor/__init__.py (which re-exports from torch.distributed.tensor) never
runs and torchao's `from torch.distributed._tensor import DTensor` gets a
harmless stub instead of crashing.
* fix: stub torch.ops._c10d_functional namespace with hashable op sentinels
torchao.dtypes.nf4tensor uses _c10d_functional ops as dict keys at import
time (all_gather_into_tensor.default, wait_tensor.default) and
torch.ops.c10d.scatter_.default. None of these ops are registered on ROCm
Windows because torch._C._distributed_c10d (the C extension) doesn't ship.
Replace the whole _c10d_functional namespace with a custom stub whose ops
return hashable .default objects, so dict-key construction doesn't crash.
Also inject a scatter_ stub into torch.ops.c10d if it's missing.
* fix: stub entire torchao package on ROCm Windows instead of individual ops
torchao is not supported on ROCm Windows and its import chain transitively
requires torch._C._distributed_c10d (absent from the ROCm Windows wheel).
Rather than stub each missing op one by one, stub the whole torchao package
upfront. Unsloth uses bitsandbytes for quantization, not torchao, so this
has no functional impact. transformers gracefully handles an importable-but-
empty torchao by disabling TorchAoHfQuantizer.
* fix: set __spec__ on mod stubs so importlib.util.find_spec doesn't raise
Manually-injected sys.modules entries have __spec__=None by default.
importlib.util.find_spec() raises ValueError when it finds a module in
sys.modules with __spec__=None (transformers.utils.import_utils hits this
when checking if torchao is available). Give every stub a minimal
ModuleSpec(name, loader=None, is_package=True) to satisfy find_spec.
* fix: add meta path finder to auto-stub subpackages of stub modules
`import torchao.prototype` goes through the import machinery, not
__getattr__, so an empty __path__ means ModuleNotFoundError. Rather than
list every submodule explicitly, register a MetaPathFinder that intercepts
any import whose parent is one of our stubs (detected by loader=None in the
parent's ModuleSpec). Real installed packages always have a SourceFileLoader
so they are never intercepted. Also register child stubs in sys.modules
from __getattr__ as a belt-and-suspenders measure.
* fix: use _unsloth_stub sentinel instead of loader=None for stub detection
The import machinery overwrites module.__spec__ with the spec returned by
find_spec (which has loader=_StubSubpackageLoader, not None), so the
loader=None check broke for second-level subpackages. Switch to a custom
_unsloth_stub object identity sentinel set directly on each stub module --
it survives __spec__ being replaced and correctly identifies stubs at any
depth (torchao.prototype.safetensors, etc.).
* refactor(rocm/win): switch to repo.amd.com arch-aware index, remove stubs
AMD recommends repo.amd.com/rocm/whl/{arch}/ as the Windows ROCm wheel
source. These wheels bundle their own ROCm runtime, support all Python
versions (not just cp312), and include the full torch._C extension set
(including _distributed_c10d) that the old repo.radeon.com wheel omitted.
Changes:
- install.ps1: remove Select-ROCmWheelRelease + hardcoded cp312 wheel
URLs; remove Python 3.12 forced-preference logic; install via
--index-url repo.amd.com/rocm/whl/{arch-family}/
- studio/setup.ps1: same -- remove Select-ROCmWheelRelease, switch to
repo.amd.com arch-aware index URL
- studio/install_python_stack.py: replace _ROCM_WINDOWS_RELEASES /
_select_windows_rocm_release with _windows_rocm_index_url() using the
_GFX_TO_AMD_INDEX_ARCH map; drop Python 3.12 restriction
- studio/backend/core/training/worker.py: remove all stub machinery
(_make_mod_stub, _StubSubpackageFinder, _StubSubpackageLoader,
_StubClassMeta, torchao/fsdp/dtensor stubs, _c10d_functional ops
stubs, BNB DLL detection) -- no longer needed with new wheel source
* fix(rocm/win): restore _distributed_c10d + torchao stubs; fix BNB install
repo.amd.com torch wheels also omit torch._C._distributed_c10d on Windows
(RCCL is not shipped on Windows). torch/distributed/__init__.py imports
from it unconditionally at module level, so the stub must land in
sys.modules before any torch.distributed import.
torchao (pulled in by transformers.quantizers) walks
torchao.float8.distributed_utils -> torch.distributed._functional_collectives
-> distributed_c10d at import time. Stubbing torchao up-front short-circuits
that chain.
worker.py:
- Restore _make_mod_stub / _StubSubpackageFinder / _StubSubpackageLoader
- Restore _StubClassMeta for ProcessGroup.BackendType attribute access
- Restore _distributed_c10d stub with __getattr__ (Windows only)
- Restore torchao stubs (5 modules, Windows only)
install_python_stack.py:
- BNB AMD wheel install was inside the early-return branch that fires when
torch is already a ROCm build (installed by install.ps1). Move BNB install
outside that branch so it always runs on Windows ROCm — the PyPI
bitsandbytes has only CUDA DLLs and fails to load on ROCm.
* worker: remove _distributed_c10d stub; stub only torchao
The installed torch/distributed/__init__.py from repo.amd.com
(torch==2.10.0+rocm7.12.0) is now properly guarded with
`if is_available():`, so `import torch.distributed` alone is safe.
The crash only comes via torchao's import chain:
torchao.float8.distributed_utils
→ torch.distributed._functional_collectives (unguarded import)
→ torch.distributed.distributed_c10d
→ torch._C._distributed_c10d ← absent on Windows ROCm
Stubbing torchao short-circuits the chain entirely. No need to stub
_distributed_c10d. Remove _StubClassMeta and the _c10d stub block;
keep only _make_mod_stub + _StubSubpackageFinder + torchao seeds.
* fix: BNB AMD wheel skipped + torch.compile segfault on Windows ROCm
install_python_stack.py: the UNSLOTH_ROCM_TORCH_INSTALLED=1 early-return
path (set by setup.ps1 when it installed torch itself) returned before
ever reaching the AMD BNB prerelease wheel install. The PyPI
bitsandbytes==0.49.x ships only CUDA DLLs, so loading it on ROCm fails
with "libbitsandbytes_rocm72.dll not found". Now installs the AMD
Windows BNB wheel before returning on that path too.
worker.py: torch._grouped_mm crashes on gfx1200 (null HIP kernel pointer,
0xC0000005) when torch.compile's JitDecomp system dispatches it during
the first forward pass. Detect Windows ROCm via torch.version.hip
(already in sys.modules from section 1e) and set TORCHDYNAMO_DISABLE=1
to bypass the broken kernel dispatch.
* fix: BNB AMD wheel install fails uv wheel filename check
The bitsandbytes continuous-release wheel is intentionally mismatched:
filename encodes 1.33.7.preview (= 1.33.7rc0 in PEP 440) but wheel
metadata reports 0.50.0.dev0. uv rejects this by default.
Introduce _install_bnb_windows_rocm() helper that sets
UV_SKIP_WHEEL_FILENAME_CHECK=1 only for this specific install, then
restores the previous env value. Both BNB install call sites (the
UNSLOTH_ROCM_TORCH_INSTALLED early-return path and the normal Windows
ROCm path) now use this helper.
* worker: patch _grouped_mm CUDA dispatch on Windows ROCm (gfx1200 null kernel)
TORCHDYNAMO_DISABLE=1 stopped the compiler frontend but not the autograd
JitDecomp system, which also dispatches _grouped_mm and hits the same
null HIP kernel crash (0xC0000005).
Verified that torch.library.Library("aten","IMPL").impl("_grouped_mm", fn,
"CUDA") successfully overrides the broken HIP kernel with a Python mm
fallback on torch==2.10.0+rocm7.12.0.
Schema: _grouped_mm(Tensor self, Tensor mat2, Tensor? offs=None,
Tensor? bias=None, ScalarType? out_dtype=None) -> Tensor
The fallback handles both the simple case (offs=None → torch.mm) and the
grouped case (offs provided → split self by offsets, multiply each group
against the corresponding slice of mat2, then cat results).
Keep _WINDOWS_ROCM_GROUPED_MM_LIB alive at function scope to prevent the
C++ dispatch registration from being freed by GC.
* worker: fix torchao stub — return stub classes not modules for isinstance()
peft/tuners/lora/torchao.py does:
from torchao.dtypes import AffineQuantizedTensor, LinearActivationQuantizedTensor
isinstance(weight, (AffineQuantizedTensor, LinearActivationQuantizedTensor))
The stub __getattr__ was returning stub modules, which isinstance() rejects
with "arg 2 must be a type, a tuple of types, or a union".
Add _StubTypeMeta metaclass whose __instancecheck__ always returns False,
and _make_stub_type() to create stub classes via it. Change _make_mod_stub
__getattr__ to return stub classes instead of stub modules for leaf
attribute access, so isinstance() gets a valid type and returns False.
_StubSubpackageFinder still handles import-style subpackage creation
(those still need module objects in sys.modules); __getattr__ only fires
for from-import or direct attribute access, which are the isinstance paths.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* tests: add coverage for Windows ROCm install paths and worker patches
Add conftest.py to fix pre-existing sys.path issue that prevented
test_rocm_support.py from running at all (install_python_stack.py
imports from backend.utils.wheel_utils which needs studio/ on sys.path).
New test classes cover everything added in this session:
- TestWindowsRocmIndexUrl: arch → AMD pip index URL mapping (gfx120X-all,
gfx1151, gfx1150, gfx110X-all, unknown → None, trailing slash)
- TestDetectWindowsGfxArch: hipinfo output parsing, missing/timeout/bad
returncode/no-gcnArchName paths
- TestInstallBnbWindowsRocm: UV_SKIP_WHEEL_FILENAME_CHECK set+restored,
env restored on exception, no-op when URL missing
- TestRocmTorchInstalledEnvVar: UNSLOTH_ROCM_TORCH_INSTALLED=1 skips
pip_install, calls _install_bnb_windows_rocm, sets flag
- TestWorkerWindowsRocmPatches: _grouped_mm CUDA dispatch override,
offs/grouped variant handling, GC-prevention sentinel,
_StubTypeMeta __instancecheck__, _StubSubpackageFinder registration,
torchao key submodule pre-stubbing, TORCHDYNAMO_DISABLE guard
- TestRocmTorchPkgSpecs: rocm7.2 torch 2.11.x spec, default <2.11 cap,
3-tuple shape, _GFX_TO_AMD_INDEX_ARCH RDNA4/3.5/3 coverage
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* tests: fix encoding, IS_WINDOWS patching, and wrong assertion
- Add encoding="utf-8" to all read_text() calls (54 occurrences) so
tests pass on Windows where the default codec is cp1252 and source
files contain UTF-8 emoji (e.g. ⚠️ in install_python_stack.py)
- Add @patch.object(stack_mod, "IS_WINDOWS", False) to Linux-path
TestEnsureRocmTorch tests so they reach the Linux code path when run
on a Windows machine instead of short-circuiting into the Windows branch
- Fix test_grouped_mm_patch_guarded_by_windows_and_hip_check: the source
uses getattr(_torch_for_rocm, "version", None) not torch.version, so
check for '"version"' and '"hip"' substrings instead
137 passed, 2 skipped
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: pin BNB_ROCM_VERSION=72 for torch==2.11.0+rocm7.13.0 compatibility
AMD's pip index now ships torch==2.11.0+rocm7.13.0 (ROCm 7.13).
bitsandbytes auto-detects HIP 7.13 from torch.version.hip and looks for
libbitsandbytes_rocm713.dll, which the AMD Windows prerelease wheel does
not ship (it only ships rocm72.dll), causing a load error at training start.
Fix:
- worker.py section 1f: set BNB_ROCM_VERSION=72 (via setdefault) before
section 2 ML imports, so bitsandbytes always loads rocm72.dll on Windows ROCm
- install_python_stack.py: set BNB_ROCM_VERSION=72 in _install_bnb_windows_rocm()
for any post-install imports; update comment to document root cause
- tests: 4 new assertions covering the fix (141 passed, 2 skipped)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: detect BNB ROCm DLL suffix dynamically instead of hardcoding '72'
BNB_ROCM_VERSION was pinned to '72' which works today (AMD wheel ships
rocm72.dll) but would break again if AMD ships a future wheel with a
different DLL suffix (e.g. rocm713.dll).
Add _detect_bnb_rocm_dll_ver() to install_python_stack.py: scans the
installed bitsandbytes package dir for libbitsandbytes_rocm{VER}.dll
using importlib.util.find_spec (no BNB import needed) and returns the
suffix. '72' remains the fallback when detection fails.
Apply the same detection inline in worker.py section 1f. Both paths
still respect a pre-set BNB_ROCM_VERSION (caller override wins).
Tests: +8 cases covering detection logic and fallback (147 passed, 2 skipped).
* fix: patch torch.distributed stubs in server process for Windows ROCm
On Windows ROCm, torch.distributed ships without process-group helpers
(is_initialized, is_available, get_rank, get_world_size). The worker
subprocess already patches these in section 1e, but the main server
process calls _determine_attention_impl_for_gpu_estimate() which calls
unsloth's resolve_attention_implementation() → is_initialized(), causing:
"Could not resolve attention implementation for '...':
module 'torch.distributed' has no attribute 'is_initialized'"
Fix: patch the missing attrs onto torch.distributed at the top of
_determine_attention_impl_for_gpu_estimate, matching the same stubs
already applied in worker.py section 1e. No-ops on Linux/CUDA where
torch.distributed is fully populated.
* fix: gate _grouped_mm dispatch patch on HIP < 7.13
AMD fixed the gfx1200 null HIP kernel in ROCm 7.13 (torch 2.11+).
Users on the new wheel now get the real GPU _grouped_mm kernel for
MoE workloads instead of the Python mm fallback.
Changes:
- worker.py: add _hip_ver_at_least() helper; wrap full _grouped_mm
patch in `if not _hip_ver_at_least(7, 13):` with else branch that
logs the skip reason; update section-1f comment to document the fix
- test_rocm_support.py: add 5 tests covering the helper definition,
the (7, 13) gate expression, the else branch, the skip log message,
and the AMD-format version string parsing (.split(".")[:2])
Verified: torch==2.11.0+rocm7.13.0 — 3D batch and grouped (offs)
variants both succeed; null crash only present on rocm7.12 and earlier.
* fix: stub is_torchelastic_launched on torch.distributed for Windows ROCm
resolve_attention_implementation calls is_torchelastic_launched() which
does not exist in the incomplete torch.distributed shipped with the
Windows ROCm wheel, causing a warning on every model config load in the
server process. Add it to the stub table alongside the four helpers
already patched in _determine_attention_impl_for_gpu_estimate.
Also adds two tests: one confirming the new stub and one confirming all
five core distributed helpers are covered.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: explicit warnings on AMD ROCm arch/version fallbacks + Fast-Install arg order
setup.ps1:
- Fix Fast-Install argument order: packages before flags, consistent with
all other Fast-Install calls in the file
(was: Fast-Install --force-reinstall --index-url $url torch ...)
(now: Fast-Install torch torchvision torchaudio --force-reinstall --index-url $url)
- Add explicit [WARN] substep when $HasROCm is true but arch mapping fails:
- GPU arch detected but not in supported wheel list → names the arch and
lists supported families so user knows exactly what to report
- HIP SDK present (amd-smi path) but gcnArchName unreadable → instructs
user to re-install the HIP SDK; previously fell back silently to CPU
install.sh:
- Add [WARN] to stderr before silent CPU fallback when AMD GPU is confirmed
(rocminfo/amd-smi) but ROCm version cannot be read from any source
(amd-smi, /opt/rocm/.info/version, hipconfig, dpkg, rpm)
- Add [WARN] to stderr when ROCm version is too old (< 6.0) with upgrade link
install.ps1 and setup.sh: no changes needed (already handle these paths correctly)
* fix: robust gfx arch detection for Strix Halo / HIP-runtime-only installs
Covers users who have the HIP runtime (amd-smi available) but not the
full HIP SDK (no hipinfo), which is common on Strix Halo iGPU systems.
Without this, $ROCmGfxArch stays null and the installer silently falls
back to CPU-only PyTorch despite a working GPU.
Detection waterfall (setup.ps1 + install.ps1):
1. hipinfo gcnArchName -- full HIP SDK (existing, unchanged)
2. amd-smi list gfx pattern -- newer amd-smi versions embed arch
3. amd-smi static --asic -- ROCm 6+ ASIC details with GFX target
4. UNSLOTH_ROCM_GFX_ARCH env -- manual override escape hatch
5. GPU name → arch table -- best-effort from marketing name:
890M / Strix Halo → gfx1151 (RDNA 3.5 iGPU, Strix Halo)
880M / Strix Point → gfx1150 (RDNA 3.5 iGPU, Strix Point)
780M / Phoenix → gfx1103 (RDNA 3 iGPU)
RX 7900/7800/7700 → gfx1100 (RDNA 3 desktop)
RX 9070 XT / 9080 → gfx1201 (RDNA 4)
RX 9070 / 9060 XT → gfx1200 (RDNA 4)
When arch is inferred from name, a Cyan substep tells the user to set
UNSLOTH_ROCM_GFX_ARCH to skip inference on future installs.
WMI block intentionally does not set $HasROCm (no runtime confirmation).
Tests: 11 new tests in TestStrixHaloGfxArchDetection covering all five
detection levels, WMI safety, and gfx regex in both ps1 files.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: resolve hipinfo/hipconfig via HIP_PATH/ROCM_PATH when not on PATH
AMD HIP SDK sets HIP_PATH on Windows but does not always add the bin
directory to PATH. Get-Command hipinfo therefore silently fails and
detection falls through to WMI, which cannot provide a gfx arch, leaving
the user with a CPU-only PyTorch install and no warning.
Changes:
- setup.ps1 / install.ps1: before falling through to amd-smi, attempt to
locate hipinfo.exe and hipconfig.exe under $env:HIP_PATH\bin (then
$env:ROCM_PATH\bin) when Get-Command returns nothing
- Emit a [WARN] with the resolved path and a one-liner to permanently fix
PATH via SetEnvironmentVariable
- Emit a [WARN] when HIP_PATH/ROCM_PATH is set but the exe is still not
found (incomplete SDK install)
- Emit a [WARN] with the first hipinfo output line when hipinfo runs but
returns a non-zero exit code (e.g. "no ROCm-capable device detected")
- 18 new tests in TestHipSdkEnvPathResolution; total 183 passed, 2 skipped
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* feat: print HIP SDK path and full hipconfig version in terminal on AMD detection
Both install.ps1 and setup.ps1 now emit substeps under the gpu step when
AMD ROCm is detected:
gpu AMD ROCm (gfx1200)
HIP SDK: C:\Program Files\AMD\ROCm\7.1
hipconfig: 7.1.51803-d3a86bd04
Previously only the gpu label (e.g. "AMD ROCm (gfx1200)") was shown with
no indication of where the SDK was found or which exact build was active.
The full hipconfig build string (e.g. 7.1.51803-d3a86bd04 instead of just
7.1) is now stored in ROCmVersionFull and also used in setup.ps1's
'rocm' step label.
9 new tests in TestHipSdkDetectedSubstep; total 192 passed, 2 skipped
* fix: Strix rocm7.1 segfault bypass + Ubuntu 24.04 HIP gcc-install-dir
Issue 1 (install.sh): gfx1151/gfx1150 + ROCm 7.1 causes a segfault in
torch._grouped_mm (moe_utils.py:167). The Radeon repo now ships cp313
wheels for rocm-rel-7.1, so _amd_gpu_radeon=true silently lands on the
broken combo. When Strix Halo/Point is detected and TORCH_INDEX_URL is
rocm7.1, override to rocm7.2 PyTorch index, update TORCH_CONSTRAINT, and
set _amd_gpu_radeon=false to bypass the Radeon repo entirely. Emits a
clear [WARN] explaining the segfault and linking to the ROCm upgrade docs.
Issue 2 (setup.sh): ROCm 7.x ships clang-20 which on Ubuntu 24.04+ picks
/usr/lib/gcc/x86_64-linux-gnu/14/ (runtime dir, no C++ headers), causing
'cstdlib file not found' and a failed llama.cpp HIP build. Iterate gcc
versions 14→11 to find the first install dir that has both runtime and
/usr/include/c++/<ver> headers, then pass --gcc-install-dir to clang via
CMAKE_HIP_FLAGS. Fix confirmed by h34v3nzc0dex (llama.cpp 417/417 clean).
11 new tests across TestStrixRocm71Override and TestSetupShGccInstallDir;
total 203 passed, 2 skipped
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: BNB_ROCM_VERSION in server process + torch._C._distributed_c10d stubs
Two errors visible in training logs on Windows ROCm:
1. Server process bitsandbytes crash:
"Configured ROCm binary not found at libbitsandbytes_rocm713.dll"
The installed BNB wheel ships rocm72.dll (not rocm713.dll). The
training worker already sets BNB_ROCM_VERSION=72 via DLL detection
but the server process (main.py) imported bitsandbytes before that
ran. Fix: add the same DLL-scan + BNB_ROCM_VERSION assignment to
main.py inside the existing win32 guard, before any downstream
import can pull in bitsandbytes.
2. torch.distributed import failure:
"No module named 'torch._C._distributed_c10d'; torch._C is not a package"
torch._C is a C extension on Windows ROCm — Python cannot do
submodule imports from it, so torch.distributed fails to import
before our attribute stubs could ever run. Fix: inject empty
ModuleType stubs for _distributed_c10d, _distributed_autograd and
_distributed_rpc into sys.modules inside the win32 guard in
hardware.py BEFORE importing torch.distributed, so the import
succeeds and our attribute stubs take effect.
9 new tests in TestServerStartupRocmFixes; total 212 passed, 2 skipped
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(win32): populate distributed c10d stub with dummy symbols
torch.distributed tries to `from torch._C._distributed_c10d import
FakeProcessGroup` (and ProcessGroup, Work, Store, etc.). The previous
empty ModuleType stub caused an AttributeError on those names.
Populate every stub with a _Dummy class for each known symbol so the
import chain completes silently on Windows ROCm where torch._C is a
compiled extension and its _distributed_c10d submodule doesn't exist.
Adds four new tests in TestServerStartupRocmFixes covering FakeProcessGroup,
ProcessGroup, setattr population, and all three _distributed_* siblings.
* fix(win32): distinguish HIP SDK installed vs GPU not ROCm-accessible
Previously, when hipinfo was found but exited non-zero (e.g. "no
ROCm-capable device detected"), both install.ps1 and setup.ps1 fell
through to the WMI-label-only branch and printed "AMD GPU detected --
HIP SDK not found" -- factually wrong since the SDK binary is present.
Add $HipSdkInstalled flag (set true when hipinfo binary is found,
regardless of exit code). When HipSdkInstalled && !HasROCm:
- Show "AMD GPU detected -- not ROCm-accessible (HIP <ver>)" instead
- Explain this is a driver issue, not an SDK issue, with a link
- Still run hipconfig version capture so version shows in output
- CPU-only hint now says "GPU not ROCm-accessible" not "require HIP SDK"
Also applies to setup.ps1 (same detection block, same branches).
Adds TestHipSdkInstalledButDeviceInaccessible (11 tests).
* fix(win32): scope ROCm workarounds to AMD hosts only
Three Codex-flagged issues where Windows ROCm workarounds incorrectly
applied to Windows CUDA (NVIDIA) machines:
main.py (P1): BNB_ROCM_VERSION was set unconditionally on all win32
hosts. On NVIDIA, bitsandbytes sees BNB_ROCM_VERSION and looks for a
ROCm DLL that doesn't exist, breaking bitsandbytes initialisation.
Fix: gate the block on HIP_PATH/ROCM_PATH being present (ROCm hosts only).
worker.py (P2): torchao stubs were seeded for all win32 runs, shadowing
real torchao on Windows CUDA and silently disabling torchao quantization
for NVIDIA users. Fix: gate on HIP_PATH/ROCM_PATH (win32 ROCm only).
install_python_stack.py (P1): _detect_windows_gfx_arch() only checked
shutil.which("hipinfo"), skipping the HIP_PATH/ROCM_PATH fallback that
the PowerShell installers use. On installs where the HIP SDK bin dir is
not on PATH, _ensure_rocm_torch() returned early without installing
ROCm wheels or bitsandbytes. Fix: mirror the env-var fallback.
* fix(linux): route Strix + ROCm 7.1 to AMD arch-specific index
Instead of falling back to pytorch.org/rocm7.2, the Strix override now
routes to repo.amd.com/rocm/whl/gfx1151/ (or gfx1150/) which serves
torch 2.11.0+rocm7.13.0 -- AMD's build containing the actual _grouped_mm
kernel fix, verified on real gfx1151 hardware by h34v3nzc0dex.
This exercises the real GPU kernel path rather than the rocm7.2 workaround.
UNSLOTH_AMD_ROCM_MIRROR can override the base URL for air-gapped installs.
Also teaches _tauri_torch_index_family to recognise AMD arch-specific URLs
(repo.amd.com/rocm/whl/gfx*) and return the rocm7.13 family label so
_tauri_gpu_branch correctly classifies these installs as rocm.
Suggested by h34v3nzc0dex based on hardware-verified probe results.
* fix(studio/rocm): gate ROCm-only side-effects on active torch runtime
Address five edge cases flagged during PR review:
1. studio/backend/main.py: BNB_ROCM_VERSION was set whenever HIP_PATH or
ROCM_PATH was present in the environment. A Windows CUDA user who once
installed the HIP SDK and reverted to a CUDA torch wheel still has those
env vars set, so bitsandbytes would try to load libbitsandbytes_rocm72.dll
against a CUDA torch and crash. Now probe torch.version.hip inside the
env-var guard (worker.py already does this).
2. studio/backend/main.py: os.add_dll_directory returned handles were
discarded. Per CPython docs, the directory leaves the DLL search list when
the handle is garbage collected. Retain handles in module-level
_ROCM_DLL_HANDLES list so they survive process lifetime.
3. studio/install_python_stack.py: _install_bnb_windows_rocm() returned None
regardless of pip_install_try outcome, and the caller flipped
_rocm_windows_torch_installed to True unconditionally. On a failed BNB
install the post-install "manual install may be required" warning was
suppressed and the user was misled. Helper now returns bool; caller gates
on it.
4. studio/install_python_stack.py: _detect_windows_gfx_arch returned the raw
capture group, so mixed-case hipinfo output ("Gfx1151") missed the
lowercase keys in _GFX_TO_AMD_INDEX_ARCH and silently fell back to CPU
torch. Lowercase the token.
5. studio/install_python_stack.py: UNSLOTH_ROCM_TORCH_INSTALLED=1 early-
return trusted the env var even when the venv was wiped between runs.
Subprocess-probe torch importability first; fall through to the full
install path if the probe fails.
Tests: 231 passed, 1 skipped in tests/studio/install/test_rocm_support.py
(adds one new test for case 5 fall-through).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio/rocm): worker.py parity + don't roll back ROCm torch on bnb failure
Addresses findings from a 10x reviewer pass on the prior fix commit:
1. studio/backend/core/training/worker.py (parity with main.py):
- Gate the torchao stub block on torch.version.hip / 'rocm' in
torch.__version__ instead of HIP_PATH / ROCM_PATH env-var presence.
Same root cause as main.py: HIP SDK env vars stick around on CUDA hosts.
- Add module-level Windows ROCm DLL registration block. Worker subprocesses
inherit env vars but not the parent's add_dll_directory handles, so the
first `import torch` in the worker could fail to find amdhip64.dll when
HIP_PATH\bin is not on PATH. Mirrors main.py setup. Handles retained at
module scope via _ROCM_DLL_HANDLES.
- Promote _WINDOWS_ROCM_GROUPED_MM_LIB to module scope with `global` in
run_training_process so the torch.library.Library registration survives
past function return / mid-run garbage collection.
- Harden _torch_has_hip() to also accept 'rocm' in torch.__version__
(AMD SDK / Radeon wheels may not set torch.version.hip).
2. studio/install_python_stack.py:
- Don't roll back ROCm torch when bitsandbytes install fails. The prior
commit gated _rocm_windows_torch_installed on _install_bnb_windows_rocm()
returning True; if torch installed successfully but bnb failed, the flag
stayed False and later install steps could overwrite ROCm torch with the
generic CPU torch wheel. Set the flag after torch install; surface bnb
failure as a separate warning instead.
- _detect_windows_gfx_arch now probes in three tiers: UNSLOTH_ROCM_GFX_ARCH
env-var override (matches the PowerShell installer), then hipinfo (PATH
or HIP_PATH\bin), then amd-smi (`static --asic`, `list`). Without the
amd-smi fallback, runtime-only Radeon installs without hipinfo on PATH
made `studio update` return early and leave the venv on CPU torch.
- Linux torch-already-rocm probe in _ensure_rocm_torch now matches the
Windows probe shape: accepts torch.version.hip OR 'rocm' in
torch.__version__ to cover AMD SDK / Radeon Linux wheels.
3. studio/backend/utils/hardware/hardware.py:
- apply_gpu_ids() final-fallback torch probe accepts 'rocm' in
torch.__version__ in addition to torch.version.hip, matching
detect_hardware(). AMD SDK wheels could otherwise leak through with
CUDA-only visibility masks on a spawned ROCm worker.
Tests: 231 passed, 1 skipped in tests/studio/install/test_rocm_support.py
(no test changes needed; the probe shape that prints the hip version (or
'rocm' sentinel) preserves the existing non-empty-string contract).
Not addressed in this commit (deferred or out of scope):
- Tag drift / lemonade checksum (PR 5303 surface, not this PR).
- install.sh rocm7.2.1 URL: small fix, separate.
- install.ps1 / setup.ps1 'Radeon 8060S' marketing-name fallback table.
- Strix Halo + ROCm 7.1 routing asymmetry in Python update path.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio/rocm): robustness pass - rocm tag normalisation, Strix routing parity, hardened detection
Robustness pass on top of 76137b2d. Four targeted fixes:
1. install.sh ROCm-tag routing normalisation.
`rocm7.2.1` would route to https://download.pytorch.org/whl/rocm7.2.1
which does not exist (PyTorch publishes major.minor URLs only). Same
for any future patch-level tag. Normalise every rocm{maj.min}* pattern
to the bare {maj.min} index URL.
2. install.ps1 + studio/setup.ps1 marketing-name fallback.
The gfx1151 row matched 890M / Strix Halo / HX 37x / HX 38x / AI 9 HX
but not the actual retail name 'AMD Radeon 8060S Graphics' shipped by
OEMs (Ryzen AI MAX+ 395). Add '8060S' to the regex.
3. install_python_stack.py Strix + ROCm 7.1 routing parity with install.sh.
The shell installer reroutes Strix Halo / Point + ROCm 7.1 to
repo.amd.com/rocm/whl/{gfx}/ (which serves torch 2.11.0+rocm7.13.0
with the upstream _grouped_mm fix). The Python `studio update` path
only warned and still installed the broken generic rocm7.1 wheel.
Mirror the override: detect gfx1151/gfx1150 on ROCm 7.1, route to
the AMD per-gfx index, honour UNSLOTH_AMD_ROCM_MIRROR override.
4. _detect_windows_gfx_arch amd-smi parsing tightened.
The amd-smi fallback added in the prior commit used a bare
`\bgfx[1-9][0-9a-z]{2,3}\b` match against the lowercased stdout,
which could pick up stray gfx references in warnings / device-name
strings. Anchor on labelled lines first (Target_Graphics_Version,
ASIC, Arch, gfx) and fall back to the bare match only when no
labelled line is present.
Tests: 231 passed, 1 skipped in tests/studio/install/test_rocm_support.py;
sim_5301 23 cases pass (6 new sims for the Strix override + amd-smi parsing).
* fix(studio/rocm): multi-GPU selection, Strix sibling handling, defensive cleanups
Round 4 robustness pass based on 5 parallel Opus reviewers of head 21773215.
Seven items from across regression / edge-case / error-paths / architecture
reviews:
1. studio/backend/main.py BNB gate: aligned with the broad ROCm check used
everywhere else in this PR (torch.version.hip OR 'rocm' in __version__).
AMD SDK / Radeon Linux wheels do not always populate torch.version.hip;
without this, main.py would silently skip BNB_ROCM_VERSION while worker.py
set it.
2. studio/install_python_stack.py _install_bnb_windows_rocm: init _ok = False
before the try block. Without this, if pip_install_try itself raises
(e.g. OSError on uv binary missing), the finally block restored env vars
correctly but the subsequent `if not _ok:` raised UnboundLocalError,
masking the original exception.
3. studio/install_python_stack.py _detect_windows_gfx_arch:
- Rewrote to use re.findall (not re.search) on both hipinfo and amd-smi
output, dedup tokens preserving order, and select via new
_pick_visible_index() helper.
- HIP_VISIBLE_DEVICES / ROCR_VISIBLE_DEVICES (first comma entry, integer)
now picks the right GPU on multi-AMD-GPU hosts. Out-of-range or non-int
values fall back to the first GPU (matches detect_host behaviour in
install_llama_prebuilt.py).
4. studio/install_python_stack.py Strix override now consults the runtime
target before flipping:
- Previous behaviour intersected gfx_codes with {gfx1151, gfx1150} and
picked the first Strix arch, ignoring whether HIP_VISIBLE_DEVICES
selected a non-Strix sibling (e.g. discrete RX 7900 in a mixed APU+dGPU
box). Could install Strix-specific wheels onto a gfx1100 dGPU.
- Now resolves the runtime gfx via _pick_visible_index() and only
overrides when that runtime target is in the Strix set.
5. studio/backend/main.py + studio/backend/core/training/worker.py: ROCm
version dir scan no longer sorts lexically. Previous sort placed "10.0"
before "7.0" alphabetically, which would mis-prioritise ROCm 10.x bin
dirs once AMD ships them. New _ver_key() splits on "." and sorts
numerically with a string fallback.
6. install.sh Strix override URL: replaced ${var%/} (strips one trailing
slash) with a while-loop that strips all trailing slashes, matching
Python's .rstrip("/"). A user setting UNSLOTH_AMD_ROCM_MIRROR with
"http://corp/whl///" no longer ends up with "http://corp/whl///gfx1151/"
which strict pip proxies (artifactory, sonatype) 404 on.
7. studio/install_python_stack.py: bumped torch import probe timeout from
30s to 90s. PyTorch's lazy .so loading can take 60-90s on cold NFS or
USB-backed venvs. The shorter timeout was producing a false "torch
missing" classification and reinstalling a working ROCm torch.
Tests: 231 passed, 1 skipped. sim_5301 30 cases pass (added 7 new sims for
multi-GPU detection, Strix sibling handling, and _ok-init regression).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio/rocm): worker BNB/grouped_mm broad gate, install.sh Strix visibility, runtime-only ROCm detection
Round-5 robustness pass based on 20 parallel reviewers of head 96b9e465.
1. studio/backend/core/training/worker.py - BNB version pin / dynamo disable
/ _grouped_mm fallback block was still gated on torch.version.hip alone
despite the torchao stub block above already using the broad check. AMD
SDK / Radeon Windows wheels (torch.__version__ contains "rocm" but
torch.version.hip is None) silently skipped the Windows ROCm runtime
patches. Aligned to the same broad check (8/20 reviewers).
2. studio/backend/core/training/worker.py - _hip_ver_at_least() now also
parses the ROCm version out of torch.__version__ (e.g. "2.11.0+rocm7.13.0")
when torch.version.hip is missing, so the kernel-fix gate is correct for
SDK / Radeon wheels too.
3. studio/backend/core/training/worker.py - _grouped_mm_safe_impl with
offs=None now picks torch.bmm/matmul for 3-D inputs instead of always
calling torch.mm. The real _grouped_mm accepts 3-D batched matmul; the
prior fallback raised "self must be a matrix" on MoE workloads (2/20).
4. studio/backend/main.py - dropped the HIP_PATH / ROCM_PATH env-var gate
from the BNB block; probe torch directly. Runtime-only Radeon / AMD SDK
Windows installs do not set those SDK env vars but still ship ROCm torch
(5/20 reviewers).
5. install.sh - Strix override now collects every gfx token from
rocminfo / amd-smi (in enumeration order), then indexes by
HIP_VISIBLE_DEVICES / ROCR_VISIBLE_DEVICES so a mixed Strix iGPU + non-
Strix dGPU host where the user selected the dGPU does NOT get rerouted
to the Strix per-gfx index. Mirrors the Python update path (5/20 reviewers).
6. install.sh - Strix detection chain now also probes `amd-smi static --asic`,
matching the PowerShell installer (1/20). Closes the gap on runtime-only
Strix hosts where `amd-smi list` does not surface a gfx token.
7. studio/install_python_stack.py - _has_rocm_gpu() now has the sysfs KFD
topology fallback (/sys/class/kfd/kfd/topology/nodes/*/gpu_id), matching
install.sh. On minimal package-managed installs without rocminfo /
amd-smi GUI tools, `studio update` can now detect the GPU and repair the
venv instead of returning early (2/20).
8. studio/install_python_stack.py - _detect_amd_gfx_codes() now falls back
to `amd-smi list` and `amd-smi static --asic` when rocminfo is missing
(2/20). Strix routing on runtime-only Radeon hosts now matches what
install.sh has done for a while.
9. studio/install_python_stack.py - Strix override now applies even when
has_hip_torch is True. The whole point of the override is to repair an
existing broken torch.version.hip == "7.1" install; skipping the
reinstall left users on the known _grouped_mm segfaulting stack (3/20).
Tests: 231 passed, 1 skipped. sim_5301 30 cases pass. sim_cross 12 pass.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio/rocm): code review hardening pass
- main.py: numeric DLL sort (string sort picked rocm72 over rocm713);
add basename() to regex; log warning on detection failure; log info
when BNB_ROCM_VERSION is set (mirrors worker.py)
- worker.py: explicit len-guard in _hip_ver_at_least() with warning
logs instead of silent IndexError/ValueError swallow
- hardware.py: isinstance(result, dict) guard before result.get() in
_smi_query() to prevent AttributeError on non-dict backend returns
- amd.py: round() before int() on parsed GPU IDs; log warning when
truncation occurs (defensive against malformed amd-smi output)
- setup.sh: quote --gcc-install-dir value in CMAKE_HIP_FLAGS so paths
with spaces do not break the CMake argument
- install.ps1, setup.ps1: apply colon-split + ToLower() to hipinfo
gcnArchName match (consistent with each other and with setup.sh)
- install.sh: tighten ROCm tag case patterns to explicit
rocmX.Y|rocmX.Y.* to avoid unintended prefix matches
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio/training): GPU OOM guard to prevent system freeze on VRAM exhaustion
On RDNA 4 (gfx1200/gfx1201) and other ROCm GPUs, exhausting VRAM can
cause a HIP driver hang that freezes the entire system rather than
raising a recoverable Python exception.
Two-part fix:
- set_per_process_memory_fraction(0.90) caps the HIP/CUDA allocator at
90% of VRAM so PyTorch raises OutOfMemoryError before hitting the
hardware limit, keeping the driver alive and the system responsive
- top-level exception handler detects OOM errors by type and message
and surfaces a clear actionable message to the UI (reduce
max_seq_length, enable gradient_checkpointing, lower batch size)
instead of the raw CUDA/HIP error string
* fix(studio/rocm): OOM guard ROCm-only + unified memory, multi-GPU arch selection
OOM guard (worker.py):
- Scope to _hw.IS_ROCM only -- NVIDIA CUDA has a graceful OOM path and
does not need the allocator cap
- Detect unified memory by comparing torch VRAM against psutil system RAM;
use 0.80 on unified-memory APUs (gfx1151 Strix Halo) where the GPU pool
is carved from host RAM, 0.90 on discrete cards
Multi-GPU arch selection:
- install.ps1 / setup.ps1: replace -match (first hit only) with
[regex]::Matches() to collect all gcnArchName entries, then index by
HIP_VISIBLE_DEVICES / ROCR_VISIBLE_DEVICES
- install_python_stack.py: index into full token list before dedup so
HIP_VISIBLE_DEVICES=2 on [gfx1100, gfx1100, gfx1151] resolves gfx1151
- install.sh: remove awk dedup from gfx token collection for same reason
GCC multiarch (setup.sh):
- Only append -linux-gnu when gcc -print-multiarch does not already return
the full triple, fixing double-suffix on Ubuntu 24.04
* fix(tests): update ROCm version cap expectations from rocm7.1 to rocm7.2
Daniel's normalisation commit updated the cap from rocm7.1 to rocm7.2
since PyTorch now publishes that index and rocm7.2 ships torch 2.11.0.
Test expectations were stale.
* fix(tests): correct MLX smoke test losses_per_step assertion
logging_steps=1 with max_steps=30 produces 30 loss entries, not 7.
The assertion was stale from a previous config.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio/worker): detect unified-memory APU by GPU name not VRAM/RAM ratio
The previous heuristic (VRAM > 50 % of system RAM) false-positived on discrete
cards in low-RAM systems — e.g. RX 9060 XT 16 GB on a 16 GB or 24 GB machine
would trip the unified-memory path and log "unified memory host" when it should
say "discrete".
AMD iGPUs (gfx1150/gfx1151 Strix Halo, Strix Point, etc.) expose names with a
digit+M suffix ("AMD Radeon 890M"), while discrete cards use "RX NNNN [XT|XTX]"
naming. Matching that suffix is reliable across all current ROCm-capable AMD
consumer GPUs and does not require psutil.
Also includes the device name in the log line to ease future debugging.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(install/setup.ps1): force array on hipinfo gcnArchName parse to fix single-GPU arch truncation
When [regex]::Matches() finds exactly one match, PowerShell's pipeline
unwraps the result to a scalar string. Indexing a scalar string with [0]
returns the first *character*, so a one-GPU system would parse
gcnArchName "gfx1200" as "g", which is not in the supported arch map
and triggers the CPU-only fallback.
Wrapping with @() forces the result to remain an array regardless of
match count. On a single-GPU machine the arch is now correctly read as
"gfx1200" (or whatever the full name is) so the ROCm wheel index is
selected.
Reproducer: hipinfo exits 0 and outputs exactly one gcnArchName line.
Without @(), $_hipAllArches = "gfx1200" (String); $_hipAllArches[0] = 'g'.
With @(), $_hipAllArches = @("gfx1200") (Object[]); $_hipAllArches[0] = "gfx1200".
* fix(studio/rocm): classify unified-memory APU via VRAM/RAM ratio, not arch list
Replace the gcnArchName allowlist {gfx1150, gfx1151} with a
psutil-based heuristic: unified APUs expose the entire system RAM
as the HIP pool (ratio ≥ 0.90), discrete cards are well below that.
No arch name required — future APUs classify correctly without code changes.
Also removes the stale import re / \d[Mm]\b device-name regex that
5d84704 left behind, and logs vram/sys GiB for easier on-hardware
verification.
Addresses h34v3nzc0dex review: Radeon 8060S (gfx1151, 128 GiB
unified) now correctly gets 0.80 cap instead of 0.90.
* fix(studio/rocm): revert to gcnArchName for unified-memory APU classification
VRAM/RAM ratio >= 0.90 false-positives on machines where discrete VRAM
equals system RAM (e.g. RX 9060 XT 16 GB + 16 GB system RAM → ratio 1.0,
incorrectly classified as unified → wrong 0.80 cap applied).
gcnArchName is the correct signal: naming-independent, stable within a
product family, and already parsed throughout this PR. Unified set is
{gfx1150, gfx1151} (Strix Point + Strix Halo).
* fix(studio/llama-prebuilt): resolve hipinfo via HIP_PATH/ROCM_PATH on Windows
shutil.which("hipinfo") returns None when the HIP SDK bin dir is not on
PATH -- the HIP SDK installer sets HIP_PATH/ROCM_PATH but does not always
add the bin dir to PATH. This caused has_rocm=False in the prebuilt asset
selector, so AMD ROCm machines got the CPU llama.cpp zip instead of the
HIP one, silently running all chat inference on CPU.
Add _resolve_exe() that falls back to %HIP_PATH%\bin and %ROCM_PATH%\bin
when shutil.which() finds nothing, mirroring the same fallback already
present in setup.ps1.
* fix(studio/llama-prebuilt): pass --has-rocm from setup.ps1 to skip re-detection
The Python prebuilt installer re-detects ROCm independently via
shutil.which("hipinfo"), which fails when hipinfo is not on PATH
(HIP SDK sets HIP_PATH but doesn't always add the bin dir to PATH).
This caused has_rocm=False and downloaded the CPU llama.cpp zip even
on confirmed AMD ROCm machines.
setup.ps1 already performs reliable ROCm detection with its own
HIP_PATH/ROCM_PATH fallback. Add --has-rocm flag to
install_llama_prebuilt.py so setup.ps1 can forward its result directly,
and pass it whenever $HasROCm is true. The Python script then overrides
has_rocm=True in the HostInfo without re-probing.
* fix(studio/llama-prebuilt): add HIP asset to simple-policy Windows path
direct_upstream_release_plan (used by --simple-policy, which setup.ps1
always passes) only checked has_usable_nvidia on Windows and fell
straight to CPU for AMD ROCm machines, ignoring has_rocm entirely.
The --has-rocm override had no effect because the simple-policy code
path never reached resolve_asset_choice where has_rocm was checked.
Add an elif branch for has_rocm that tries the upstream HIP asset
(llama-TAG-bin-win-hip-radeon-x64.zip) before falling through to the
CPU fallback, consistent with the non-simple-policy path.
* fix(studio/setup.ps1): auto-remove mismatched llama.cpp install kind
When an existing llama.cpp install is the wrong kind for the current
GPU (e.g. windows-cpu on an AMD ROCm machine that should have
windows-hip), the prebuilt installer skips on tag match and never
upgrades. Read install_kind from UNSLOTH_PREBUILT_INFO.json before
invoking the installer and remove the directory if the kind doesn't
match, forcing a fresh download of the correct variant.
* fix(studio/setup.ps1): show live PyTorch install output in verbose mode for ROCm
The ROCm torch reinstall (setup.ps1 phase) always silently captured
output, so in --verbose mode the torch downgrade mid-install
(2.11.0+rocm → 2.10.0 → 2.11.0+rocm) looked like the final state was
2.10.0. Match the CPU/CUDA blocks which show live uv output when
$script:UnslothVerbose is set.
* fix(rocm/windows): set ROCBLAS_TENSILE_LIBPATH for bundled rocblas.dll
The llama.cpp ROCm prebuilt bundles rocblas.dll next to the binary but
not the Tensile kernel library files it depends on at runtime
(rocblas/library/TensileLibrary*.dat + *.hsaco). The bundled DLL
searches for these files relative to its own location by default, i.e.
<binary_dir>/rocblas/library/, which does not exist in the prebuilt
install tree. This causes a silent crash on the very first GEMM
(prefill) with no output from llama-server, seen by the caller as
WinError 10054 / 10061. Model load and the single-token warmup pass
because they use simpler code paths that do not trigger rocBLAS GEMM.
Fix: set ROCBLAS_TENSILE_LIBPATH in the subprocess env to
<HIP_PATH>/bin/rocblas/library so the bundled DLL finds the kernel
files from the system ROCm installation. Uses setdefault so a user-
supplied env var is never overwritten. No-ops on CUDA and CPU (no
HIP_PATH) and on Linux (win32 branch only).
Reproducer log:
rocBLAS error: Cannot read .../Release/rocblas/library/TensileLibrary.dat
rocBLAS error: Could not initialize Tensile host:
directory_iterator: The system cannot find the path specified.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(install.sh): restore gfx token dedup in Strix multi-GPU awk indexer
536a54df removed the per-source `| awk '!seen[$0]++'` dedup from the
_gfx_all collection step but left the indexer awk as bare NF, so on a
mixed-arch host (e.g. dGPU gfx1100 + Strix iGPU gfx1151) where
rocminfo emits each gfx token twice (Name: field + ISA triple),
HIP_VISIBLE_DEVICES=1 indexed vals[1] = the second gfx1100 occurrence
instead of gfx1151, triggering the Strix routing on the wrong GPU.
Add !seen[$0]++ to the indexer awk so duplicate tokens from the same
GPU collapse to one entry before the HIP_VISIBLE_DEVICES index is
applied -- matching exactly what the Python side does with dict.fromkeys()
in _detect_amd_gfx_codes(). The comment above the block ("skip
duplicates") already documented this as the intended behaviour.
* fix(studio/install): correct _TOTAL progress count on Windows
base_total += 3 fired for all non-macOS platforms including Windows,
but flash-attn (line 1620) and ROCm torch final (line 1705) are both
guarded by 'not IS_WINDOWS and not IS_MACOS', so on Windows with torch
enabled _TOTAL was 13 while only 11 _progress() calls actually execute.
Split into +1 for the ROCm torch check (all non-macOS) and +2 for the
two Linux-only steps, so Windows gets _TOTAL=11 and Linux gets 14.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(install.ps1): enforce torch>=2.11.0 for gfx120X and Strix on Windows
The AMD arch-specific index (repo.amd.com/rocm/whl/gfx120X-all/ and
gfx1151/) publishes torch wheels from 2.7.1 through 2.11.0. Without a
version floor pip can resolve to torch 2.10.0+rocm7.12 on RDNA 4
(gfx120X) or torch 2.10.0+rocm7.1 on Strix (gfx1151/gfx1150), both of
which have a null-pointer crash in torch._C._grouped_mm (TheRock
issues #5284 / #3284). torch 2.11.0+rocm7.13 contains the fix.
Add $ROCmTorchFloor alongside $ROCmIndexUrl: set to torch>=2.11.0 for
the two affected arch families, null for all others. Wire it into the
uv pip install call so the broken wheels are never selected.
* fix(rocm/windows): address Codex nits - deterministic DLL suffix, CUDA llama.cpp kind, HIP_VISIBLE_DEVICES arch indexing
- install_python_stack.py / worker.py: _detect_bnb_rocm_dll_ver() and the
inline worker probe now collect ALL libbitsandbytes_rocm*.dll suffixes and
return max() by numeric value instead of stopping at the first glob hit.
Filesystem glob order is not guaranteed; this ensures '713' always wins
over '72' when both variants are present in the wheel.
- setup.ps1 (expectedKind): add 'windows-cuda' branch so NVIDIA hosts are
not treated as 'windows-cpu'. Previously an existing windows-cuda prebuilt
was always considered a mismatch on non-ROCm machines, forcing an
unnecessary re-download on every update.
- setup.ps1 (amd-smi gfx arch): collect ALL gfx tokens from amd-smi list
output in GPU order and honour HIP_VISIBLE_DEVICES / ROCR_VISIBLE_DEVICES
when selecting which arch to use. On mixed-arch AMD systems where the
visible GPU is not the first enumerated one, this prevents installing an
incompatible wheel index. Falls back to index 0 (same as before) when the
visibility var is unset or is a comma-separated list.
- test_rocm_support.py: add test_picks_highest_suffix_when_multiple_dlls to
cover the multi-DLL case that was previously untested.
* fix(rocm): misleading amd-smi log, BNB spec consistency, torch ceiling for AMD index
amd.py: split 'returncode != 0 or not stdout' into two separate branches.
Previously, exit-0 with empty output logged 'amd-smi returned code 0' (which
reads as success, not a warning) and incorrectly incremented the circuit-breaker
counter. Now: non-zero exit logs the code and counts toward the limit as before;
empty stdout on exit 0 logs at DEBUG level and does not penalise the counter
(amd-smi --json always emits at least [] on exit 0, so this branch is rare and
is not a tool failure).
main.py: replace spec.origin / os.path.dirname() with
spec.submodule_search_locations to match install_python_stack.py and worker.py.
For normal wheel installs both approaches reach the same directory, but using
submodule_search_locations is the canonical way and handles editable bitsandbytes
installs correctly. Also use max() by numeric suffix (same as the other two sites)
instead of a sort-then-break loop.
install.ps1: add <2.12.0 ceiling to the torch constraint for gfx120X (RDNA 4)
and gfx1151/gfx1150 (Strix). AMD actively publishes new versions on their
per-arch index; without a ceiling, a future 2.12.0+rocmX.Y wheel would be
pulled in automatically before being validated on these architectures. The
ceiling matches the existing Linux install_python_stack.py constraint for the
same arches. Bump both when 2.12.x is confirmed working.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(rocm): torch floor in setup.ps1, torchvision pin for Strix, rocmsdk in _hip_ver_at_least
setup.ps1: add \ (mirrors install.ps1) and derive \
from it. Previously the AMD index install called 'Fast-Install torch torchvision
torchaudio --force-reinstall --index-url \' with no version
constraint, so pip could resolve torch 2.10.0+rocm7.12 for gfx1151/gfx1200 --
the exact broken wheel the PR is meant to avoid. Now gfx120X and Strix enforce
'torch>=2.11.0,<2.12.0', matching install.ps1 and the Linux constraint.
install_python_stack.py: pin torchvision and torchaudio in _strix_override_pkgs.
The Strix Linux override uses --index-url (exclusive, no PyPI fallback); bare
unversioned 'torchvision' and 'torchaudio' could resolve a build from AMD's
index targeting a different torch major, causing ABI/version mismatches at
runtime. Now pinned to '>=0.26.0,<0.27.0' and '>=2.11.0,<2.12.0' respectively,
matching _ROCM_TORCH_CONSTRAINT['rocm7.2'].
worker.py: extend _hip_ver_at_least to handle AMD SDK wheel version strings.
The fallback regex r'rocm(\d+)\.(\d+)' cannot match '2.9.0+rocmsdk20251116'
(no rocmX.Y component), so the function always returned False on SDK/Radeon
wheels -- installing the Python _grouped_mm workaround on wheels that already
have the working HIP kernel. Added a second check: if the version string
contains '+rocmsdk', assume >= 7.13 (the rocmsdk format post-dates the
gfx120X null-kernel fix) and skip the fallback.
* fix(rocm): warn on OOB HIP_VISIBLE_DEVICES, bail on empty numeric_ids mask
- setup.ps1: when HIP/ROCR_VISIBLE_DEVICES names an index beyond the
detected GPU count, emit a yellow warning and fall back to GPU 0
instead of silently reading allGfxArches[-1] (wrong arch)
- hardware.py _reconcile_primary_rocm_unified_memory: distinguish
numeric_ids=None (no env var, use torch ordinal 0) from numeric_ids=[]
(empty mask / HIP_VISIBLE_DEVICES=-1, no GPU visible); bail out early
in the empty case to avoid querying torch.device(0) incorrectly
* fix(rocm): gate StubSubpackageFinder on win32 ROCm, add gcnArchName fallbacks
- worker.py _StubSubpackageFinder: the meta_path append was running on
every platform on every call to run_training_process; moved it inside
the if _is_win32_rocm: block since stubs are only seeded there and the
finder is a pure accumulation on Linux/Windows CUDA
- worker.py OOM guard: AMD SDK / Radeon wheels may not populate
gcnArchName, causing Strix Halo to be misclassified as discrete and
get the 0.90 cap (12.8 GB OS headroom) instead of 0.80 (25.6 GB);
now tries gcn_arch_name / arch_name / gfx_arch_name variants first,
then falls back to device-name matching (890M -> Strix Halo,
880M -> Strix Point) with a debug log when the fallback fires
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(rocm): pin torchvision/torchaudio in setup.ps1, remove -Unique from arch array
- setup.ps1 ROCm torch install: torchvision and torchaudio were passed
bare alongside pinned torch>=2.11.0,<2.12.0 for gfx1151/gfx1200 arches.
AMD publishes packages independently so a future torchvision 0.27 (for
torch 2.12) on the same arch index would cause pip ResolutionImpossible
or an ABI-incompatible install. Added torchvisionFloorMap and
torchaudioFloorMap mirroring install_python_stack.py's strix override
(torchvision>=0.26.0,<0.27.0, torchaudio>=2.11.0,<2.12.0) and derived
ROCmVisionSpec/ROCmAudioSpec used in all three Fast-Install call sites.
- setup.ps1 amd-smi arch detection: Select-Object -Unique was collapsing
same-arch multi-GPU arrays (e.g. two gfx1151 APUs -> 1-element array)
causing HIP_VISIBLE_DEVICES=1 to trigger a false out-of-range warning
and fall back to GPU 0 even though the correct GPU would have been at
index 1. Removed -Unique; added comment noting the positional-index
assumption and its non-contiguous-GPU limitation.
* fix(rocm): add 8060s/8050s to OOM guard device-name fallback, extract classifier helper
Path 3 of the OOM guard device-name fallback only checked for 890m/880m
(gfx1150 Strix Point SKU names). Strix Halo (gfx1151) ships as Radeon 8060S
(Ryzen AI MAX+ 395) and Radeon 8050S (cut-down SKU) -- neither matches, so
the fallback returned is_unified=False and applied the 0.90 fraction instead
of 0.80, leaving ~12.8 GiB OS headroom on a 128 GiB pool instead of ~25.6 GiB.
Fix: add 8060s and 8050s to the name-match set. Also correct the comment that
mislabelled 890M as a Strix Halo name (it is Strix Point).
Refactor: extract the three-path classifier into _rocm_classify_unified_memory()
so it can be unit-tested directly. Add 31 test cases in test_rocm_oom_guard.py
covering all three paths and the regression case (Radeon 8060S Graphics).
Reported-by: h34v3nzc0dex
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(rocm): pass explicit dtype on bf16-unsupported hardware (RDNA2)
dtype=None lets unsloth auto-detect the model dtype. On RDNA2 (gfx103x,
e.g. RX 6600) is_bfloat16_supported() incorrectly returns True, so unsloth
picks bf16 and the first bf16 kernel dispatch triggers:
LLVM ERROR: Cannot select: intrinsic %llvm.amdgcn.fdot2.bf16.bf16
Replace every dtype=None in load_model() with _auto_dtype which resolves
to None when bf16 is supported (all modern NVIDIA + RDNA3+) and
torch.float16 otherwise. This gives RDNA2 users a working float16
training path without touching NVIDIA behaviour at all.
Fixes: https://github.com/unslothai/unsloth/issues/5337
* fix: reduce log noise for expected non-issues on Windows ROCm
Three log lines fired at warning/error level for conditions that are
completely expected on a Windows HIP SDK-only setup:
amd.py
- amd-smi WinError 2 (FileNotFoundError): downgrade warning -> debug.
amd-smi ships with Adrenalin, not the HIP SDK; absence is normal.
- 'disabling' message: downgrade warning -> info with clearer text
'not available (not installed; expected on HIP SDK-only systems);
GPU VRAM polling disabled'
hardware.py
- torch.distributed.Store missing: downgrade warning -> debug.
The distributed stub added in this PR intentionally omits Store; the
attention-impl fallback to eager is expected and non-actionable.
worker.py
- causal-conv1d: add early Windows exit (info) in both
_ensure_causal_conv1d_fast_path and _causal_conv1d_install hook;
no cp313/win_amd64 wheel exists, so the install always fails.
- FLA: add early Windows exit (info) in
_ensure_flash_linear_attention_unconditional; triton dependency has
no cp313/win_amd64 wheel.
- Defense-in-depth: _install_package_wheel_first non-HIP PyPI failure
logs info+debug on Windows instead of error; FLA failure logs
info+debug on Windows instead of warning.
* [AMD] FIx installation of bitsandbytes when it's from .dev and skip rebuilding llama.cpp if we build it manually.
* fix: use force_pip for Windows ROCm bitsandbytes prebuilt wheel install
uv rejects the bnb continuous-release wheel due to filename/metadata
version mismatch (1.33.7.preview vs 0.50.0.dev0). Switch to force_pip=True
(pip bypass) instead of the UV_SKIP_WHEEL_FILENAME_CHECK env var workaround
-- cleaner and consistent with how the Linux path handles it.
BNB_ROCM_VERSION is still set post-install to the detected DLL suffix so
the worker subprocess loads the correct libbitsandbytes_rocm{VER}.dll even
when torch.version.hip reports a newer HIP version than the wheel ships.
* fix: three small correctness fixes found in PR review
- _install_bnb_windows_rocm: use UV_SKIP_WHEEL_FILENAME_CHECK=1 with
try/finally instead of force_pip=True so the env var is always
restored and the failing CI test passes
- _determine_attention_impl_for_gpu_estimate: gate torch._C distributed
stubs on IS_ROCM so Windows CUDA users keep the real extension
- install.ps1 amd-smi fallback: collect all gfx tokens and index by
HIP_VISIBLE_DEVICES, matching the hipinfo path on multi-GPU hosts
* fix: stub torchao in export subprocess on Windows ROCm
On Windows, the ROCm build of PyTorch ships without the distributed
C extension (torch._C._distributed_c10d). torchao, which is pulled in
transitively by transformers.quantizers at import time, walks into
torch.distributed._functional_collectives -> distributed_c10d and
crashes with:
No module named 'torch._C._distributed_c10d'; 'torch._C' is not a package
This only affected the export subprocess because the training subprocess
already applied an identical torchao stub (introduced separately to fix
the same root cause). The export subprocess had no such guard and died
during 'Importing Unsloth...' before any model loading could happen.
Fix: apply the same _StubSubpackageFinder / torchao stub pattern to the
export subprocess entry point, gated on Windows ROCm detection, before
any import of transformers or unsloth_zoo.
Root cause tracked in ROCm/TheRock#3284 (libuv / torch.distributed
missing on Windows ROCm builds).
Ref: https://github.com/ROCm/TheRock/issues/3284
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* install.sh, setup.sh: add GPU arch step logging to match PS1 scripts
Both shell scripts were missing the step "gpu" terminal log block that
install.ps1 and setup.ps1 emit. This adds equivalent output: GPU label
with gfx arch (e.g. "AMD ROCm (gfx1151)"), ROCm root path, hipconfig
version, and marketing name substep. Includes the same gfx arch detection
chain (rocminfo → amd-smi list → amd-smi static --asic), UNSLOTH_ROCM_GFX_ARCH
env override, and name-based arch inference table (Strix Halo/Point, RDNA 3/4)
as the PS1 versions. install.sh also replaces bare echo blocks for the AMD
ROCm and CPU-only cases with formatted substep output.
* Fix BNB_ROCM_VERSION gate, ROCm GPU mask preference, APU unified memory and Release build for PR #5301
- main.py: gate BNB_ROCM_VERSION on the rocm bnb DLL or HIP_PATH/ROCM_PATH instead of importing torch on every Windows host
- hardware.py: prefer HIP/ROCR visible-device masks only on ROCm hosts so a stale mask cannot override CUDA_VISIBLE_DEVICES on NVIDIA
- llama_cpp.py: set GGML_CUDA_ENABLE_UNIFIED_MEMORY=1 only for unified-memory APUs (gfx1150/gfx1151)
- setup.sh: pass -DCMAKE_BUILD_TYPE=Release for the HIP source build
- add test_amd_apu_unified_memory.py
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: guard recompile_limit + fix AMD VRAM monitor fallback
trainer.py: torch._dynamo.config.recompile_limit does not exist in
some ROCm torch builds (e.g. pytorch.org/whl/rocm6.2 wheels). Guard
the assignment so training doesn't crash on RDNA2/RDNA3.
hardware.py: when amd-smi/nvidia-smi is unavailable or returns no
usable data (HIP SDK-only Windows, Docker, unexpected JSON format),
the existing fallback used torch.cuda.memory_allocated() which is
process-specific and reads near-zero even with a fully loaded model.
Switch to torch.cuda.mem_get_info() via _torch_get_per_device_info()
which reports system-wide VRAM occupancy so the GPU monitor shows
real usage on all AMD systems without requiring amd-smi.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: Windows VRAM monitor via Performance Counter API
When amd-smi/nvidia-smi is unavailable on Windows, query dedicated GPU
VRAM via Windows Performance Counters (same source as Task Manager).
This gives system-wide cross-process usage, fixing the near-zero reading
caused by torch.cuda.mem_get_info only seeing the Studio server process.
Linux fallback path unchanged (mem_get_info is system-wide on ROCm).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: rename to _rocm_windows_perf_counter_vram_gb, scope to IS_ROCM
Function is AMD ROCm specific — amd-smi absent on Windows when only the
HIP SDK is installed. Scoped to IS_ROCM so NVIDIA Windows path is
untouched (nvidia-smi handles that case).
* fix: AMD VRAM monitor — Linux DRM sysfs + Windows perf counter
Linux: read /sys/class/drm/card*/device/mem_info_vram_used|total for
system-wide GPU memory across all processes. No tools required, always
present on Linux AMD systems.
Windows: Windows Performance Counter API (already added).
Both paths are gated on IS_ROCM and only fire when amd-smi is absent.
torch mem_get_info remains as last resort (process-local).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: AMD GPU monitor — utilization, temperature, and power for Windows and Linux fallback paths
- Windows: GPU utilization via \GPU Engine(*engtype_3D*)\Utilization Percentage perf counter
- Windows: temperature and power via ADL (atiadlxx.dll, ships with Adrenalin)
- Linux: GPU utilization via DRM sysfs gpu_busy_percent
- Linux: temperature via hwmon temp1_input (millidegrees C)
- Linux: power via hwmon power1_average / power1_input (microwatts)
All paths are no-op fallbacks (None) when the source is unavailable.
Mirrors what nvidia-smi provides on the CUDA path.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: remove ADL ctypes — does not support AMD iGPU (Strix Halo)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
Co-authored-by: Erland366 <erland.pg366@gmail.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
* Studio setup.sh: cope with fresh CUDA toolkits like 13.3
CUDA 13.3 shipped today. Three loose ends in studio/setup.sh surfaced
during the llama.cpp build path:
1. setup.ps1 already aborts cleanly when the CUDA toolkit is below
llama.cpp's minimum (12.4) via #4517, but setup.sh still hit the
generic cmake failure described in #4437. Added a min-version check
that downgrades to a CPU build for nvcc < 12.4 with a clear message
pointing to the toolkit archive.
2. The first day a new CUDA toolkit ships, its host-compiler whitelist
lags whatever gcc/clang the distro is on, so nvcc rejects the host
compiler with a wall of "#error -- unsupported GNU version" before
any real compile runs. NVCC_PREPEND_FLAGS now carries
-allow-unsupported-compiler so the build moves on instead.
3. The Linux CUDA/ROCm configure failure path had no symmetry with the
macOS Metal fallback: a single nvcc failure left BUILD_OK=false and
no llama.cpp at all. Generalised the existing Metal -> CPU fallback
to cover any GPU_BACKEND, so a CUDA configure or build failure now
transparently retries with the CPU args and the user still ends up
with a working llama-server.
Pulled the version probe out into _nvcc_meets_llama_minimum so it can
be unit-tested. Added tests/sh/test_nvcc_meets_llama_minimum.sh and two
extra cases in tests/sh/test_get_torch_index_url.sh covering the legacy
"CUDA Version: 13.3" header (driver-reported) and the future 13.7
case. Wired the new test into tests/run_all.sh and the studio-backend
CI workflow.
* tests: relax pr4562 regression to allow generic GPU fallback label
* studio tests: assert setup.sh exports NVCC_PREPEND_FLAGS=-allow-unsupported-compiler
The -allow-unsupported-compiler flag is the core of the fresh-CUDA-toolkit fix
(it lets nvcc accept a host gcc/clang newer than its release-time whitelist, so
CUDA 13.3 day-one builds do not abort on '#error -- unsupported GNU version'),
but it had no automated coverage. Add a source-pattern test asserting the flag
is present, delivered via NVCC_PREPEND_FLAGS so it also covers cmake's CUDA
compiler-id probe, and kept out of CMAKE_ARGS for bash word-splitting safety.
* studio/setup.ps1: allow unsupported host compiler for CUDA build (Windows parity)
Mirror the Linux setup.sh headline fix from this PR on Windows. A freshly
released CUDA toolkit ships with a host-compiler whitelist that lags the
installed toolchain, so nvcc can reject the host with
"#error -- unsupported Microsoft Visual Studio version!" before any real
compile runs (the MSVC analogue of the gcc wall the Linux side hit on
CUDA 13.3). Set NVCC_PREPEND_FLAGS=-allow-unsupported-compiler in the CUDA
build branch so both cmake's configure-time CUDA compiler-id probe and the
cmake --build step proceed. The flag disables the host version check only and
is a no-op when the compiler is already supported.
Set via the process environment (not the $CmakeArgs array), after the
Refresh-Environment calls that re-sanitize CUDA env vars, and appended
idempotently to any value the user already set.
Validated with PowerShell 7.6.2: full setup.ps1 AST parse is clean and the
snippet is idempotent (empty -> set, existing -> append once, no duplicate).
Needs real Windows + CUDA CI to exercise the actual nvcc/MSVC build.
Adds test_setup_ps1_exports_allow_unsupported_compiler asserting the flag is
present, env-delivered, kept out of $CmakeArgs, and scoped to the CUDA-on branch.
* studio: tighten code comments added in this PR
Shorten the verbose multi-line comments and test docstrings introduced by
this PR (setup.sh, setup.ps1, and the shell/python tests) to be succinct
while preserving the rationale. No code or test-assertion changes.
* Studio: keep web search/code pills off on model load if user disabled them
* Studio: avoid redundant localStorage reads when resolving tool pills on load
* fix(studio/colab): merge iframe+keepalive into start(), add proxy_headers to uvicorn
- Move serve_kernel_port_as_iframe and keepalive loop into colab.start()
so both run in the same cell execution context, eliminating the race
where the proxy URL was shown before the iframe cell had a chance to run
- Add a 2s sleep after run_server() before show_link() to give Colab's
proxy infrastructure time to register the bound port
- Add proxy_headers=True and forwarded_allow_ips="*" to uvicorn Config
so X-Forwarded-Proto/Host from Colab's reverse proxy are trusted
- Simplify notebook start cell (no more separate iframe cell needed)
* fix(studio/colab): fix iframe blocking and server thread crash in Colab
Two root causes for the long-standing proxy/iframe breakage:
1. SecurityHeadersMiddleware set X-Frame-Options: DENY and
frame-ancestors 'none' unconditionally, blocking
serve_kernel_port_as_iframe regardless of server health.
Fix: detect Colab via COLAB_BACKEND_URL/COLAB_GPU env vars,
relax frame-ancestors to *.prod.colab.dev and omit X-Frame-Options.
2. asyncio.run() in the daemon thread conflicted with nest_asyncio's
global patches applied on the main thread, causing the server to
crash silently after ready_event fired.
Fix: use explicit new_event_loop() + run_until_complete() in the
daemon thread to bypass nest_asyncio's asyncio.run patch.
Also replace blind time.sleep(2) with a health endpoint poll so the
link and iframe are only shown once the server is truly reachable.
* fix(studio/colab): use reliable /content + google.colab path for Colab detection
COLAB_BACKEND_URL and COLAB_GPU env vars aren't consistently set across
all Colab runtime versions. Use /content dir + google.colab package path
as a more reliable signal, computed once at module load.
* fix(studio/colab): fix port mismatch, health-check silence, and CSP framing
Four bugs causing the iframe and URL button to always fail:
1. Port not propagated back: run_server auto-increments when 8888 is taken,
but start() kept using the original port for show_link() and
serve_kernel_port_as_iframe() — now reads app.state.server_port.
2. Silent health-check failure: the poll loop never checked whether any
attempt succeeded; on all-fail it continued and showed a dead link —
now exits early with a clear error message.
3. CSP frame-ancestors too narrow: '*.prod.colab.dev' only matches one
subdomain level; actual Colab proxy URLs are two levels deep
(e.g. foo.region.prod.colab.dev), and the parent frame may also be
colab.research.google.com or a sandboxed null-origin output iframe —
changed to '*' in Colab mode (single-user sandbox, no security loss).
4. _IS_COLAB detection hardcoded python3.10/3.11 paths: Python 3.12+
Colab runtimes wouldn't match when env vars aren't set — replaced with
a glob over python3.*/dist-packages/google/colab.
* fix(studio/colab): harden Colab startup against every known failure mode
colab.py:
- get_colab_url: retry eval_js up to 3x (10s timeout each), validate that
result is a real https:// URL containing the port before accepting it;
log a clear warning when falling back to localhost
- show_link: safe short_url truncation (try/except around str.index so an
unexpected URL shape never blocks the link card from rendering); also
emit the URL via logger so it's visible in cell text output even if
HTML display is suppressed
- start: detect "already running" at entry — on cell re-run Studio is
still healthy on port 8888; skip re-launch and go straight to
show+iframe so the user never ends up with mismatched port state
- start: wrap run_server in try/except (SystemExit + Exception) so
startup errors surface as readable messages rather than cell crashes
- start: check frontend_path/index.html exists, not just the directory
- start: remove unused `import sys`
- start / keepalive: catch KeyboardInterrupt so interrupting the cell
prints a clean "stopped" message instead of a raw traceback
- extract _is_studio_healthy() and _show_and_embed() helpers to
deduplicate the fast-path and normal-path logic
main.py:
- _build_csp: in Colab mode, extend script-src to include
*.prod.colab.dev and *.googleusercontent.com (Colab injects scripts
from these origins into the output iframe scaffolding)
- _build_csp: in Colab mode, extend connect-src with blob:, data:,
wss://*.prod.colab.dev, and wss://*.googleusercontent.com so
WebSocket streams and Colab kernel traffic are not blocked by CSP
* fix(studio/colab): fix iframe width responsiveness and height sizing
Replace serve_kernel_port_as_iframe with a raw CSS iframe for two
reasons:
1. Width responsiveness: serve_kernel_port_as_iframe sets the width as
an HTML attribute (width="100%") which Colab's output machinery can
bake into a fixed pixel value on first render, causing the Studio to
stop following the notebook panel width when it opens/closes or the
window resizes. A CSS style property (style="width:100%") participates
in normal reflow and always tracks the parent container width.
2. Height sizing: the hardcoded height=1200 was too tall on short monitors
(forced outer-page scroll) and wasted space on tall ones. A small JS
snippet reads screen.availHeight and sets height to ~82% of the screen,
clamped to [600, 1100]px, with a resize listener that re-fits on zoom
changes and panel open/close events.
Also eliminate the double eval_js call: _show_and_embed now fetches the
Colab proxy URL once and passes it to show_link via the new _url kwarg,
so google.colab.kernel.proxyPort is only called once per invocation.
Falls back to serve_kernel_port_as_iframe if IPython.display.HTML is
unavailable for any reason.
* fix(studio/colab): fix link button + add fullscreen hover button to iframe
Link button: target="_blank" is blocked by Colab's output sandbox.
Switch to onclick="window.open(url,'_blank')" which the sandbox allows.
Fullscreen: add a small button that appears on hover in the top-right
corner of the iframe. Clicking it calls requestFullscreen() on the
wrapper div and stretches the iframe to 100vh/100vw. Exits back to
normal on fullscreen change.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* revert(studio/colab): remove fullscreen button
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio/colab): address review feedback
- Wrap both urlopen calls in with statements to prevent socket/fd leaks
- Replace JS resize listener with CSS height:82vh — simpler, responsive,
and no risk of leaked window listeners on cell re-runs
- Use importlib.util.find_spec("google.colab") instead of a glob path
to detect Colab; more robust across Python versions and venv layouts
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio/colab): fall back to href navigation when window.open is blocked
window.open from a cross-origin sandboxed Colab output iframe can be
silently blocked by the browser (returns null, no exception). The old
code returned false unconditionally, so a blocked popup left the button
doing nothing. Now: if window.open succeeds the new tab opens and the
href is suppressed; if it returns null the browser follows the href,
navigating the output cell to Studio — always does something useful.
* fix(studio/colab): remove button, give iframe a branded header bar
The "Open Unsloth Studio" button was unreliable in Colab's sandboxed
output context regardless of how window.open was called. Since the
iframe already loads Studio inline, the button added no value and
confused users with a URL that 404s outside the output cell.
Replace the separate link card + bare iframe with a single block:
a slim black header bar (Unsloth logo + truncated URL) flush on top
of the full-height responsive iframe. Cleaner and removes the broken
button entirely.
* studio: gate uvicorn proxy_headers/forwarded_allow_ips behind _IS_COLAB
forwarded_allow_ips="*" was applied unconditionally, so every Studio
deployment trusted X-Forwarded-* headers from any client. Only Colab needs
that, because its reverse proxy fronts the kernel. For a normal
local/standalone Studio this is an unwanted relaxation, especially when bound
to 0.0.0.0.
Now proxy_headers/forwarded_allow_ips are only set when _IS_COLAB. Standalone
runs fall back to uvicorn's defaults (proxy_headers honored from loopback
only), restoring the prior security posture, while Colab keeps the wide trust
its proxy requires.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* fix: honor --ctx-size and other forwarded args from `unsloth studio run` in Studio's context-fit logic
* refactor: extract resolve_requested_ctx as single source of truth
The test helper was reimplementing the two-line
'ctx_override = parse_ctx_override(...); requested_ctx = ctx_override
if ctx_override is not None else n_ctx' pattern locally, so the test
asserted against its own reimplementation rather than production logic.
Extract the conditional into resolve_requested_ctx and have both the
production caller and the test use it.
* fix(studio): honor pass-through cache type flags in KV VRAM estimate
Studio's KV cache VRAM estimate computed from the first-class
cache_type_kv even when the user passed -ctk/--cache-type-k/-ctv/
--cache-type-v via extras. Those flags reached llama-server fine
(last-wins on the CLI) but the pre-launch estimate kept using the
default f16 bytes-per-element, so GPU placement decisions could be
off when the user lowered cache precision via pass-through.
Adds parse_cache_override + resolve_cache_type_kv in llama_server_args.py
(mirroring parse_ctx_override / resolve_requested_ctx), wires both into
load_model alongside the existing ctx resolution, and adds focused
unit tests for the parser + resolver.
Follow-up to @rolandtannous review on #5815.
---------
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
* style: remove dark mode upload circle
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Daniel Han