* feat(studio): add Tauri native GGUF intake
* feat(studio): polish native GGUF intake
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio): load backend helpers during local setup
* fix(studio): acquire native load lease before unload
* Studio: harden native path lease verification and Tauri intake
- Wrap path.resolve(strict=True) and Path.stat() in NativePathLeaseError so a deleted or unmounted GGUF returns 400 instead of leaking the full filesystem path through the generic load_model/validate_model handler.
- Re-apply _reject_network_or_device_path to the resolved canonical path for defense in depth after symlink resolution.
- Replace try/except ValueError pattern in the device-path guard with Path.is_relative_to; the previous shape silently swallowed NativePathLeaseError (which subclasses ValueError) so /dev,/proc,/sys were never actually rejected.
- Broaden the lease redaction regex and dict-key check (Python and Rust diagnostics) to cover both native_path_lease and nativePathLease so the camelCase form emitted by Tauri/frontend payloads is also redacted.
- Hoist the redact_native_paths import to module top in loggers/handlers; the recursive filter no longer pays a per-record import lookup.
- Persist activeNativePathToken in the chat runtime store so the rollback branch can mint a fresh lease and reload the previous native GGUF when a new load fails after unload; clear it in clearCheckpoint and overwrite it on each successful load.
- use-native-drop: read options through a ref so the Tauri onDragDropEvent listener is registered once and stays attached across option changes; reject ambiguous multi-file drops up front instead of silently registering only the first GGUF.
- pick_native_model: use an async pick_file with a tokio oneshot channel instead of blocking_pick_file so the Tokio worker is not held for the duration of the OS dialog.
- registerNativeModelPath: drop the duplicate sourceKind argument; the Rust command parameter is source_kind.
- install_python_stack: insert the script directory (studio/) on sys.path; the previous insert pointed at studio/backend/ which does not satisfy `from backend.utils.wheel_utils import ...`.
* install_python_stack: keep _BACKEND_DIR on sys.path
Restore the studio/backend insertion. Although the immediately following `from backend.utils.wheel_utils import (...)` is satisfied by studio/ already being on sys.path[0] when invoked as `python studio/install_python_stack.py`, wheel_utils itself runs `from utils.native_path_leases import ...`, which requires studio/backend/ to be importable. Without the backend insertion, the existing tests/python/test_install_python_stack.py collection fails with ModuleNotFoundError: No module named 'utils'.
* Studio: tighten native path lease lifecycle and Tauri intake IPC
- register_native_model_path now hardcodes NativePathSourceKind::Drop on the Rust side and the frontend stops sending source_kind. The previous JS payload (source_kind only) never reached the Rust deserializer because Tauri's default ArgumentCase::Camel maps the Rust parameter source_kind to the JS key sourceKind, so drag/drop registration silently failed. Hardcoding the source kind also keeps audit metadata trustworthy on this command.
- Add native_path_secret_removed_for_child_start context manager and wrap multiprocessing.Process.start() at the inference, export, training, and data-recipe job spawn sites. The previous wrapper-only scrub left UNSLOTH_STUDIO_NATIVE_PATH_LEASE_SECRET visible to spawn-platform import-time worker code. The wrapper run_without_native_path_secret stays as defense-in-depth inside the child.
- Stop passing exc_info=True from the native-grant load/validate error logs in routes/inference.py. The structlog filter_sensitive_data processor runs before the renderer, so ConsoleRenderer formatted tracebacks bypassed redaction; the redacted str(e) preserves the message text.
- Replace the os.path.normcase string equality on the resolved canonical path with Path.samefile (with a normcase fallback) so Windows leases that differ only in extended-length \\?\ prefix or short-name spelling are accepted.
- Wrap consumeNativePathToken in its own try/catch in the chat runtime rollback. If the previous native-model token has aged out of TOKEN_TTL we now surface a clear modelsError instead of silently swallowing the rollback inside the outer catch.
- Reject non-ASCII lease strings in _split_lease and convert UnicodeEncodeError / binascii.Error / ValueError raised by _b64decode into NativePathLeaseError so verify_native_path_lease never escapes raw exceptions to the route handler.
- Tighten dropStateForPaths to mark multi-file payloads invalid so the overlay matches the post-fix drop handler that rejects the same payload.
- Replace the one-shot fetch in useNativePathLeasesSupported with a delayed-retry loop so the picker/drop becomes available once the backend is up rather than staying disabled for the rest of the session after a transient failure.
- Drop the unused setActiveNativePathToken setter; the value is set via setState directly in use-chat-model-runtime.
- Add a toast on auto-load failure in use-native-drop so a collapsed model selector does not hide the error.
- Burn the lease nonce before _validate_current_stat so a stat-failed lease is single-use even if a later state change happens to match the original size/mtime.
* Studio: cache lease secret, harden native path stat checks, polish intake UX
- Cache the decoded UNSLOTH_STUDIO_NATIVE_PATH_LEASE_SECRET on first verify and validate that it is base64-decodable and at least 32 bytes. Subsequent _decode_secret calls return from the cache and never touch os.environ, so concurrent /api/inference/load and /api/health requests no longer race with native_path_secret_removed_for_child_start scrubbing the env. native_path_leases_supported now wraps _decode_secret so the health flag matches what verify_native_path_lease actually accepts.
- Replace path.is_file()/is_dir() + path.stat() with os.lstat() in _validate_current_stat and explicitly reject S_ISLNK; size and mtime checks now refer to the link itself, closing the same-size+same-mtime symlink-swap window that the prior follow-symlink stat() left open.
- Add an issued_at_ms < expires_at_ms sanity check in _validate_payload to reject internally inconsistent (HMAC-protected) lease payloads.
- Sort _NATIVE_PATH_REDACTIONS by length (descending) before iterating in redact_native_paths so a longer registered path is replaced before a shorter prefix path; otherwise logs containing /foo/X.gguf.bak after only /foo/X.gguf was registered would leak the .bak suffix.
- classify_existing_path now re-checks the canonical path with symlink_metadata after canonicalize, so a regular file that is replaced with a symlink in the small canonicalize window is rejected at registration.
- ModelSelector renders the local file picker as its own block (not in the eject ternary), so a user with an active model can still replace it via the picker rather than only via drag/drop.
- useNativePathLeasesSupported caps the readiness probe at MAX_READINESS_POLLS (60 = ~5 minutes) and aborts the in-flight fetch on unmount via AbortController, so a permanently-disabled backend stops generating sustained traffic and hot-reload no longer leaks open connections.
- useChooseNativeModel returns a stable useCallback closure and guards the OS dialog with a useRef so rapid double-clicks cannot open multiple dialogs and orphan Rust tokens.
- Branch the multi-file drop toast: if no GGUF was present we say "Only .gguf model files can be dropped here." and otherwise "Drop a single .gguf model file." so users dropping non-GGUF attachments get an accurate explanation.
* native_path_leases: lstat the signed canonical path before resolving
The earlier change to lstat inside _validate_current_stat operates on grant.canonical_path, which is the post-resolve target. If the user atomically replaces the originally-signed file with a symlink to a different file of identical size and mtime, path.resolve(strict=True) follows the symlink, samefile returns True (both ends share the new inode), and the lstat in _validate_current_stat sees the regular target file rather than the symlink, so the swap goes undetected.
Add an os.lstat on the signed canonical path before path.resolve(strict=True), and reject S_ISLNK there. The lstat in _validate_current_stat stays as defense-in-depth for swaps that occur strictly between resolve and stat.
* Studio: scrub native lease secret before mp.Queue spawn and tighten lease lifecycle
- Move _CTX.Queue / _CTX.Event / _CTX.Process construction inside native_path_secret_removed_for_child_start at the inference, export, training and data-recipe spawn sites. The first Queue creation lazily spawns Python's multiprocessing.resource_tracker child, so when it ran outside the scrub context the tracker process inherited the lease secret. Reproduced via the proc filesystem environ entry; the wrapped order keeps the tracker clean.
- native_path_secret_removed_for_child_start now refcounts entries: the env var is popped on the first entry and restored only when the last context exits. Concurrent training/inference/export starts no longer serialize on the env lock across the entire proc.start yield, while still guaranteeing the env stays empty for the duration of every overlapping spawn.
- run_without_native_path_secret now also nulls the module-level cached lease secret. With the existing spawn-only multiprocessing context the cache is irrelevant in practice, but a future fork caller would otherwise inherit the in-memory secret even though the env var was scrubbed.
- filter_sensitive_data now applies the native lease key check on the top-level event_dict, not only on nested dicts, so a logger call that includes a lease value as a top-level keyword field actually redacts it (the bare value does not match the prefix-anchored regex).
- chat-page loadNativeModelIntent now passes intent.id to clearModelIntent so a second drag-drop during an in-flight first auto-load is not wiped from the chip area when the first resolves.
- Bump useNativePathLeasesSupported's MAX_READINESS_POLLS from 60 to 720 so first-run installs that compile llama.cpp from source or download large CUDA wheels (well past 5 minutes) don't permanently disable the native picker.
* native_path_leases: serialize first-decode against scrub context
_decode_secret used a separate _SECRET_INIT_LOCK from the env scrub's _NATIVE_PATH_ENV_LOCK, so the very first decode (before the cache is populated) could race a concurrent native_path_secret_removed_for_child_start and read os.environ during the env-empty window, raising "Native path grants require the managed desktop backend." Subsequent calls hit the cache and were already safe.
Acquire _NATIVE_PATH_ENV_LOCK around the env read inside _SECRET_INIT_LOCK and fall back to _SCRUB_SAVED_SECRET when the scrub has temporarily popped the env var. Lock ordering (init then env) is consistent with no other caller, so no deadlock.
* Studio: surface native model load errors and harden native path label cache
- Native model load and validate now bubble up the actual exception (with
paths redacted) and apply the same friendly-error rewrite the non-native
path uses, so users see "CUDA OOM", "trust_remote_code required", etc.
instead of a generic "Failed to load native model: <label>".
- run_without_native_path_secret now also nulls _SCRUB_SAVED_SECRET so a
forked grandchild that imports native_path_leases cannot recover the
secret via the scrub-aware fallback in _decode_secret.
- _NATIVE_PATH_LABELS now has its own 10000-entry cap independent of the
100-entry redaction list, so display_label_for_native_path no longer
falls back to returning the raw canonical path after 101 native paths
in one session. Redaction list keeps the 100-entry cap for log-scan
performance.
- _validate_payload now also rejects null bytes in display_label, which
is echoed back in HTTP responses and log lines.
* Studio: harden native path lease validation and chained native rollback
- child_env_without_native_path_secret now copies os.environ under
_NATIVE_PATH_ENV_LOCK so a concurrent scrub-context env pop cannot
raise RuntimeError: dictionary changed size during iteration in a
background hardware scan or other env reader.
- _validate_payload and grant construction route every signed numeric
field (version, issued_at_ms, expires_at_ms, size_bytes, modified_ms)
through new _required_int / _optional_int helpers that wrap raw int()
ValueError into NativePathLeaseError. The single upstream catcher
produces 400 instead of 500 for malformed signed payloads.
- verify_native_path_lease now runs _validate_current_stat before
_consume_nonce, so a transient stat error on the canonical path no
longer permanently burns the nonce. Concurrent verifies still
serialize through _consume_nonce, so single-use is preserved.
- Chained native model rollback now restores activeNativePathToken in
the chat runtime store after a successful rollback loadModel. Without
this, a second consecutive failed switch could not re-roll-back
because the store token had been overwritten by the failed attempt.
- validate_model now applies the same not_supported_hints friendly
rewrite to native model errors that load_model already does, so a
native .gguf that fails validation with an upstream "is not supported"
message gets the same actionable wording as the non-native branch.
* Studio: harden native path log redaction, status disclosure, and chip lifecycle
- structlog processor chain now runs format_exc_info before
filter_sensitive_data so traceback strings are produced (and then
redacted) rather than passed through as untouched (type, value, tb)
tuples that the JSON or console renderer formats after the redaction
filter has already finished.
- native_path_secret_removed_for_child_start clears _CACHED_LEASE_SECRET
in addition to popping the env var, so a fork during the scrub window
cannot inherit the cached bytes via the parent's heap. Parent verify
calls during the window keep working through the existing scrub-aware
fallback in _decode_secret.
- load_model's except ValueError handler now redacts native paths and
uses the native model log label when native_grant_backed is true.
Previously a ValueError raised after lease verification (e.g. from
ModelConfig.from_identifier or downstream GGUF parsing) returned the
raw exception string in the HTTP response body.
- llama_cpp_backend now records the native display label at GGUF load
time, and /api/inference/status prefers it over the redaction store.
After a Python backend restart the redaction store is empty; the
attribute keeps the friendly label, and an absolute model_identifier
with no other label source falls back to the basename so the canonical
path no longer appears in active_model.
- reveal_path_token uses native "reveal and select" commands on macOS
(open -R) and Windows (explorer /select,) so the file is highlighted
in the file manager. Linux keeps the existing parent-directory open.
- Native model rollback that fails because the previous token cannot be
consumed now throws a rollback-specific Error, and the outer empty
catch was replaced with one that re-throws the rollback error. The
rollback-specific message now reaches the user instead of being
overwritten by the original load error message.
- NativeModelChip tracks the Rust token's expiresAtMs on a single
setTimeout, disables the Load button at expiry, and relabels it
"Select again" with an explanatory tooltip so users do not click into
a guaranteed-failure path after the 15-minute TTL elapses.
* Studio: tighten native artifact policy, mmproj sibling check, and intake UX
- is_open_safe_artifact no longer grants Open for directories. Reveal
already handles directory navigation, so the change closes the
attack surface where a macOS .app artifact could be launched via
open_path_token + open::that_detached.
- Display labels are sanitized in classify_existing_path. Control
characters in filenames (newlines, tabs, NUL et al.) are replaced
with spaces and the label is trimmed and capped, so a file named
with embedded newlines cannot inject forged log lines or scramble
the UI status panel.
- validate_entry_path skips the size_bytes/modified_ms equality check
when the operation is Reveal or Open. Cloud-sync agents (Dropbox,
iCloud Drive, OneDrive) routinely rewrite extended-attribute
metadata which bumps mtime, and the user expects Reveal/Open to
remain available for files in synced folders.
- llama_cpp_backend gains a _native_grant_backed flag at GGUF load
success. /api/inference/status only applies the absolute-path
basename fallback when that flag is true, so a non-native absolute
local GGUF still reports its canonical model_identifier and unload
by identifier keeps working.
- Native vision GGUFs now run through _validate_native_mmproj_companion
before llama-server starts: the companion mmproj must be a regular
file, not a symlink, and must live in the same resolved directory as
the granted GGUF. This stops a hostile sibling or symlinked mmproj
from being loaded under a single-file lease.
- Chained native rollback restructured: the rollback loadModel + state
+ refresh runs inside its own try/catch that swallows so the outer
throw error surfaces the ORIGINAL load failure. The native-token
consume-failure case still throws the rollback-specific message
early, before the inner block runs, so its actionable guidance is
preserved.
- Loading-model state and the duplicate-load guard in the chat runtime
hook now compare both the model id and the native path token. Two
drops or picks with the same basename in different folders no longer
silently dedup; the second token is honored.
- chat-page loadNativeModelIntent awaits selectModel before clearing
the pending intent. If selectModel returns early via dedup or
throws, the chip and its token stay so the user can retry instead
of losing the selection.
- NativeModelChip's Reveal button is disabled when the lease has
expired (Rust would reject it anyway), and the Load button label
reads "Expired" instead of "Select again" so the disabled element
no longer promises an action it cannot perform.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Add ROCm detection to install.sh and expand shell tests
Add AMD ROCm GPU detection to get_torch_index_url() in install.sh.
When nvidia-smi is not found, probe for ROCm via amd-smi, /opt/rocm
version file, hipconfig, dpkg-query, and rpm.
Includes validation guard for malformed _rocm_tag, Debian epoch prefix
stripping, ROCm 7.2+ cap to rocm7.1 index, bitsandbytes AMD install,
and status messaging. Shell tests expanded to 23 cases.
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Add ROCm torch reinstall support to install_python_stack.py
Add _detect_rocm_version() and _ensure_rocm_torch() to detect when a
Linux host has ROCm but the venv received CPU-only torch, and reinstall
with the correct ROCm wheels. Covers ROCm 6.0 through 7.1 with a
30-second timeout on the torch GPU probe subprocess.
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Add ROCm support to llama.cpp prebuilt installer
Add has_rocm field to HostInfo, extend detect_host() to probe for ROCm
via hipcc/amd-smi/rocm-smi/ROCM_PATH, and route ROCm hosts to upstream
prebuilts (Linux ROCm 7.2 prebuilt with source fallback, Windows HIP
prebuilt with CPU fallback). Add linux-rocm and windows-hip install
kinds to runtime_patterns_for_choice().
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Add IS_ROCM hardware flag and fix AMD error message
Add IS_ROCM flag to hardware.py detect_hardware() (set when
torch.version.hip is present, DeviceType stays CUDA). Export IS_ROCM
from __init__.py. Add "rocm" key to get_package_versions().
Replace "We do not support AMD" error in tokenizer_utils.py with a
helpful message pointing to ROCm installation docs.
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Add comprehensive ROCm support test suite (68 tests)
Add tests/studio/install/test_rocm_support.py covering all ROCm code
paths across install_llama_prebuilt.py, install_python_stack.py,
hardware.py, tokenizer_utils.py, and install.sh. All tests use mocks
and run without AMD hardware.
Covers: asset selection (11), runtime patterns (5), HostInfo (4),
ROCm version detection (9), torch reinstall (9), index mapping (8),
hardware flag (8), tokenizer message (2), install.sh structure (10),
and live regression (1).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Harden ROCm support: probe error handling, version cap, validation
Address review findings from 8 independent reviewers:
- Wrap _ensure_rocm_torch() torch probe in try/except for
TimeoutExpired and OSError so a hung or broken torch import does not
crash the installer (8/8 reviewers flagged this)
- Add torch>=2.4,<2.11.0 version cap to the ROCm reinstall path to
prevent installing unsupported torch 2.11.0 from the rocm7.1 index
- Use with-statement for file reads in _detect_rocm_version() to avoid
resource leaks
- Handle ROCM_PATH="" correctly (use `or "/opt/rocm"` instead of
default parameter to avoid relative path resolution)
- Strengthen shell validation guard from rocm[0-9] to rocm[1-9] to
reject rocm0.x tags that would produce nonexistent PyTorch index URLs
- Switch shell version cap from blocklist to allowlist (rocm6.*|rocm7.0*
|rocm7.1* pass through, everything else caps to rocm7.1) so future
ROCm 10+ does not fall through to a nonexistent index
- Add sorted() to _ROCM_TORCH_INDEX lookup for defensive ordering
- Fix test_probe_timeout_handled: replace zero-assertion test with
proper assertions verifying reinstall proceeds after timeout
* Clean up rocm_paths list construction in detect_host()
Filter None from the ROCM_PATH env var lookup at list construction time
instead of relying on the inline `if p` guard in the any() call.
* Require actual AMD GPU presence before selecting ROCm paths
All 8 reviewers across 2 cycles independently flagged that ROCm
detection used toolkit/filesystem hints (hipcc, /opt/rocm, rocm-core)
as a proxy for GPU presence, which would misroute CPU-only or NVIDIA
hosts that happen to have ROCm tools installed.
Now all 3 detection points (install.sh, install_python_stack.py,
install_llama_prebuilt.py) probe for an actual AMD GPU before
entering the ROCm path:
- install.sh: check rocminfo for gfx* GPU names, or amd-smi list
for device rows, before version detection
- install_python_stack.py: new _has_rocm_gpu() function probes
rocminfo and amd-smi list before _ensure_rocm_torch() proceeds
- install_llama_prebuilt.py: detect_host() probes rocminfo/amd-smi
list instead of just checking tool existence or directory paths
Also:
- Shell test mock amd-smi now handles "list" subcommand
- Python tests updated to mock _has_rocm_gpu where needed
- Added test_no_gpu_with_rocm_tools_skips to verify the new guard
- Test index lookups now use sorted() to match production code
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Harden hipconfig version parsing and torch probe compatibility
- Add parts[1].isdigit() check in hipconfig version parsing to handle
versions like "6.3-HIP" where the minor component has non-numeric
suffix (strip "-" prefix before int() conversion)
- Use getattr() in torch probe subprocess to safely handle old or
custom torch builds that may lack torch.version.hip/cuda attributes
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Strengthen AMD GPU detection and add NVIDIA precedence guard
- Change amd-smi list detection from any-non-empty-output to requiring
"gpu" marker in output, matching the shell-side NR>1 check. Prevents
false positives from header-only amd-smi list output.
- Add nvidia-smi check at the top of _ensure_rocm_torch() so mixed
AMD+NVIDIA hosts preserve NVIDIA precedence (matching install.sh and
install_llama_prebuilt.py behavior).
- Apply the same amd-smi marker fix to install_llama_prebuilt.py
detect_host() for consistency.
* Add Windows-specific ROCm/HIP detection in detect_host()
The previous detect_host() ROCm check used rocminfo and amd-smi list
which are Linux-only tools. On Windows, has_rocm would always be False,
making the Windows HIP prebuilt path at line 1794 unreachable.
Now detect_host() uses platform-specific detection:
- Linux: rocminfo (check for gfx GPU names) or amd-smi list
- Windows: hipinfo.exe, amd-smi, or amdhip64.dll on PATH
This allows Windows AMD users to get the HIP prebuilt binary instead
of silently falling through to the CPU prebuilt.
* Add AMD ROCm gaps: Mamba/SSM source builds, GPU monitoring, Windows messaging, RDNA expansion
- worker.py: Add HIP detection to causal-conv1d/mamba-ssm probe, check
for hipcc before ROCm source builds, improve status messages and error
reporting, add timeout and uv support for the source build fallback
- amd.py: New AMD GPU monitoring module via amd-smi metric --json,
mirroring nvidia.py structure (utilization, temperature, power, VRAM)
- hardware.py: Branch to amd.py when IS_ROCM is True for GPU utilization,
visible GPU queries, and physical GPU count
- install_python_stack.py: Detect AMD GPUs on Windows and warn that
ROCm-enabled PyTorch must be installed manually
- kernels/utils.py: Expand is_rdna() to cover RDNA2 (gfx1030-1032),
RDNA3 (gfx1102-1103), RDNA3.5 (gfx1150-1152) alongside existing entries
- tests: Add 32 new tests covering all changes (95/95 pass)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Harden ROCm detection, fix VRAM heuristic, and expand RDNA2 coverage
- Windows ROCm detection: validate actual GPU presence via hipinfo/amd-smi
output markers instead of just checking tool existence on PATH
- _ensure_rocm_torch: validate nvidia-smi actually reports a GPU before
giving NVIDIA precedence (fixes AMD-only hosts with stale NVIDIA tools)
- amd.py _parse_numeric: handle dict-shaped metric objects from newer
amd-smi versions ({"value": 10, "unit": "W"}) and strip MiB/GiB units
- amd.py VRAM heuristic: raise threshold from 100k to 10M to correctly
handle MI300X (192 GB = 196608 MB) and other high-VRAM GPUs
- amd.py visible GPU: use AMD-reported GPU IDs instead of enumerate index
so non-dense sets like CUDA_VISIBLE_DEVICES=1,3 report correctly
- install.sh: add ROCm <6.0 minimum version guard (no PyTorch wheels
exist for older versions); fix rocm7.1* glob to not match rocm7.10+
- is_rdna: add gfx1033-1036 for RDNA2 mobile GPUs (RX 6600M etc.)
- worker.py: increase ROCm source build timeout from 600s to 1800s;
fix success log message for ROCm source builds
- Tests: update mocks for _has_usable_nvidia_gpu, add RDNA2 target asserts
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add HIP_VISIBLE_DEVICES support, unit-aware VRAM parsing, Windows GPU validation
- hardware.py: check HIP_VISIBLE_DEVICES and ROCR_VISIBLE_DEVICES on ROCm
before falling back to CUDA_VISIBLE_DEVICES, so multi-GPU AMD setups with
HIP-specific env vars report the correct visible device set
- amd.py: add _parse_memory_mb() that reads "unit" from dict-shaped amd-smi
JSON (e.g. {"value": 192, "unit": "GiB"}) and converts to MB correctly;
fixes MI300X VRAM misreported as 0.19 GB instead of 192 GB
- install_python_stack.py: Windows AMD warning now validates actual GPU
presence via hipinfo/amd-smi output markers before printing
- install_llama_prebuilt.py: restore amdhip64.dll fallback for Windows HIP
detection after tool-based checks, so Windows HIP installs without CLI
tools on PATH are still detected
- hardware.py: fix IS_ROCM comment to accurately describe its role
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix HIP_VISIBLE_DEVICES empty-string handling in GPU visibility spec
Use explicit None checks instead of Python `or` operator when reading
HIP_VISIBLE_DEVICES / ROCR_VISIBLE_DEVICES, so that an empty string
("") is correctly honored as "no visible GPUs" rather than silently
falling through to CUDA_VISIBLE_DEVICES on mixed ROCm+CUDA systems.
* Fix IS_ROCM test assertion for multi-line formatting
* Cap torchvision/torchaudio versions, remove amdhip64.dll fallback, fix visible GPU count
- Cap torchvision<0.26.0 and torchaudio<2.11.0 alongside torch<2.11.0 in
both install.sh and install_python_stack.py to prevent resolver from
selecting incompatible companion packages from ROCm wheel index
- Remove amdhip64.dll fallback in Windows ROCm detection (DLL presence
without hipinfo/amd-smi is not proof of GPU existence)
- Fix get_visible_gpu_count() to use _get_parent_visible_gpu_spec() which
respects HIP_VISIBLE_DEVICES/ROCR_VISIBLE_DEVICES on ROCm hosts
* Attribute is_rdna() RDNA2/3/3.5/4 expansion to PR #4428
The is_rdna() expansion to cover RDNA2 (gfx1030-1036), RDNA3
(gfx1100-1103), RDNA3.5 (gfx1150-1152), and RDNA4 (gfx1200-1201)
architectures is based on the original work from PR #4428.
Co-authored-by: GoldenGrapeGentleman <yueyuan@amd.com>
Co-authored-by: billishyahao <bill.he@amd.com>
* Support AMD Radeon for studio (#4770)
Co-authored-by: Iswarya Alex <iswarya.alex@amd.com>
* Remove ROCm test files from main PR
Move test_rocm_support.py and shell test additions to a separate PR
to keep the main ROCm support PR focused on implementation changes.
* Fix installer and hardware detection issues for PR #4720
- Fix empty _tri_arg passed to uv pip install in Radeon path (causes
"Empty field is not allowed for PEP508" error)
- Fix Radeon fallback: use ROCm index instead of CPU-only when
repo.radeon.com is unreachable (TORCH_INDEX_URL already has ROCm)
- Use $TORCH_CONSTRAINT in fallback paths instead of hardcoded strings
- Fix _pick_radeon_wheel: relax suffix to match manylinux_2_28_x86_64
wheels (AMD Radeon repo does not use bare linux_x86_64 platform tag)
- Fix IS_ROCM export: use __getattr__ so callers always see the live
value after detect_hardware() runs
- Fix apply_gpu_ids: set HIP_VISIBLE_DEVICES and ROCR_VISIBLE_DEVICES
on ROCm so _get_parent_visible_gpu_spec picks up narrowed GPU set
- Fix _parse_memory_mb: distinguish GB (1000 MB) from GiB (1024 MiB)
- Add amd-smi version as a fallback in _detect_rocm_version
- Fix trailing whitespace and missing newline at EOF in install.sh
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix GPU detection false positives and add missing health groups
- Fix _has_rocm_gpu() false positive: require "GPU: <number>" data rows
from amd-smi list, not just header containing "gpu"
- Apply same fix in detect_host() in install_llama_prebuilt.py
- Add runtime_payload_health_groups for linux-rocm and windows-hip so
partial/corrupt ROCm/HIP prebuilt installs are properly detected
- Add bitsandbytes install to Radeon fallback paths (was only in the
success path, skipped when repo.radeon.com was unreachable)
- Keep DEVICE/CHAT_ONLY as direct imports in __init__.py (matching main)
and only use __getattr__ for IS_ROCM
* Fix _ensure_rocm_torch and Windows AMD warning false positives
- _ensure_rocm_torch: only skip when HIP is already present, not for
CUDA builds (which are unusable on AMD-only hosts). Fixes the case
where a venv has a stale CUDA wheel and the repair step is skipped.
- Windows AMD warning: use GPU data row check (same as Linux fix) to
avoid false positives from amd-smi list header-only output.
* Fix amd-smi GPU detection for GPU[N] output format
Older amd-smi versions output "GPU[0] : Card series: ..." instead of
"GPU: 0". The regex now matches both "GPU: <digit>" and "GPU[<digit>"
formats to detect actual GPU data rows.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Harden AMD GPU detection against false positives
- install.sh: replace weak amd-smi list check (awk 'NR>1 && NF') with
strict pattern matching GPU data rows (/^GPU[[:space:]]*[:\[]/)
- All files: reject rocminfo gfx000 (CPU HSA agent) by requiring
gfx[1-9] instead of gfx[0-9] in the rocminfo GPU probe
- Fixes false positives on hosts with ROCm tools but no AMD GPU
* Remove duplicate comment from pre-commit merge
* Refactor: deduplicate AMD detection, consolidate bitsandbytes, clean up imports
- Extract _has_amd_rocm_gpu() shell function to avoid duplicating the
rocminfo/amd-smi GPU detection logic in get_torch_index_url and
the Radeon auto-detect block
- Consolidate bitsandbytes install into a single case block after torch
install (was duplicated 4 times across Radeon success/fallback paths)
- Move math and re imports to top of amd.py (were inline in functions)
- Add _smi_query() helper in hardware.py to centralize IS_ROCM backend
selection for get_gpu_utilization and get_visible_gpu_utilization
Addresses Gemini code review suggestions.
* Fix VRAM parsing for string values and GB/GiB consistency
- Extract unit from string-valued VRAM fields (e.g. "192 GiB") so
_parse_memory_mb correctly applies the unit multiplier instead of
treating the value as bare MB
- Treat GB and GiB identically (both as binary x1024) since GPU tools
including amd-smi use binary units even when labeling them "GB"
- Fixes incorrect VRAM reporting on MI300-class cards (was showing
~0.19 GB instead of 192 GB for string-valued outputs)
* Add --no-cache to uv for ROCm HIP source builds
Avoid stale cache artifacts from partial HIP source builds when
uv is used for causal-conv1d/mamba-ssm compilation on ROCm.
The pip path already uses --no-cache-dir; this adds the uv equivalent
(--no-cache) only when is_hip is True.
* Fix critical: initialize _amd_gpu_radeon before case block
_amd_gpu_radeon was only set inside the */rocm*) case arm, so on
NVIDIA/CPU/macOS paths where TORCH_INDEX_URL does not contain "rocm",
the variable was unbound. With set -u (nounset) enabled, this crashes
the installer for every non-AMD user.
Move initialization to before the case block so it is always defined.
* Fix Windows AMD: route has_rocm hosts to HIP prebuilt path
resolve_release_asset_choice was selecting windows-cpu for all Windows
x86_64 hosts including those with has_rocm=True. Windows AMD users
should fall through to resolve_upstream_asset_choice which tries the
HIP prebuilt first. Add "not host.has_rocm" guard to the published
windows-cpu selection.
* Harden ROCm detection, Radeon wheel fallback, and HIP visibility
Addresses review findings from parallel reviewers on PR #4720:
- install.sh: add _has_usable_nvidia_gpu() helper requiring nvidia-smi -L
to actually list a GPU before treating the host as NVIDIA. Fixes the
stale-nvidia-smi-on-PATH regression where AMD-only hosts fell into the
CUDA branch.
- install.sh: fix hipconfig awk blocks to propagate a non-zero exit code
when the output is not a recognisable version string, so the ||-chain
continues to dpkg-query / rpm instead of terminating early.
- install.sh: fail-closed on Radeon wheel fallback. When torch,
torchvision or torchaudio is missing from the Radeon repo for the
active Python tag, fall back to the standard ROCm index instead of
silently mixing Radeon wheels with PyPI defaults. Quote all wheel
arguments individually so wheel filenames cannot be word-split or
glob-expanded.
- install_llama_prebuilt.py: detect_host() now requires nvidia-smi -L to
list a GPU before setting has_physical_nvidia. Routes AMD ROCm hosts
with a broken leftover nvidia-smi to the ROCm path instead of
misclassifying them as NVIDIA.
- install_llama_prebuilt.py: scan upstream assets for any rocm-<version>
prebuilt instead of hard-coding rocm-7.2, so ROCm 6.x / 7.0 / 7.1 / 7.3+
users pick up a matching upstream prebuilt when one exists.
- install_llama_prebuilt.py: validate_server() adds --n-gpu-layers 1 for
linux-rocm and windows-hip hosts, so new HIP prebuilts are preflighted
on the GPU path instead of passing validation on CPU only.
- install_llama_prebuilt.py: restore the published windows-cpu fallback
for AMD Windows hosts without a HIP prebuilt so hash-approved bundles
are still preferred over the raw upstream CPU asset.
- install_python_stack.py: drop the /opt/rocm / hipcc gate in
_ensure_rocm_torch() and rely on _has_rocm_gpu(). Runtime-only ROCm
installs (package-managed minimal installs, Radeon software) that ship
amd-smi / rocminfo without hipcc can now repair a CPU-only venv via
"unsloth studio update". Adds an explicit IS_WINDOWS / IS_MACOS guard.
- studio/backend/utils/hardware/amd.py: honour HIP_VISIBLE_DEVICES /
ROCR_VISIBLE_DEVICES / CUDA_VISIBLE_DEVICES in
get_primary_gpu_utilization(). A process restricted to GPU 2 now
reports metrics for GPU 2 instead of physical GPU 0. Tighten the plain
bytes unit detection to an explicit allowlist.
- studio/backend/utils/hardware/hardware.py: route
get_backend_visible_gpu_info()'s backend_cuda_visible_devices field
through a helper that reads HIP_VISIBLE_DEVICES on ROCm. Drop the
unconditional "(rocm=False)" suffix in apply_gpu_ids() logs.
* Fix round 2 regressions: ROCm validate_server and Windows HIP routing
Follow-up to 810b833b addressing review findings on the first round of
hardening commits:
- install_llama_prebuilt.py validate_server: gate --n-gpu-layers on the
resolved install_kind instead of host.has_rocm. AMD Windows hosts
without a HIP prebuilt fall back to windows-cpu and must not be
validated with GPU layers; thread install_kind through from the
caller.
- install_llama_prebuilt.py resolve_release_asset_choice: reinstate the
"not has_rocm" guard on the published windows-cpu bundle so AMD
Windows hosts reach resolve_upstream_asset_choice() where the new
HIP prebuilt path lives. Prefer a published windows-hip bundle first
when one exists, fall through to upstream HIP + upstream CPU
otherwise.
- install_llama_prebuilt.py detect_host: also set has_physical_nvidia
when the secondary --query-gpu block confirms a working NVIDIA GPU,
so older nvidia-smi versions without -L support do not silently skip
the Linux diagnostics that key off has_physical_nvidia.
- install_llama_prebuilt.py: drop redundant "import re as _re" /
"import re as _re_rocm" local aliases in favour of the existing
top-level "import re".
- install_python_stack.py _ensure_rocm_torch: run the AMD
bitsandbytes install unconditionally after the HIP-torch probe so
"unsloth studio update" on venvs that already have ROCm torch still
gains the AMD bitsandbytes build.
- install.sh: add a non-x86_64 early-exit to get_torch_index_url() so
aarch64 / arm64 Linux hosts do not hit the ROCm wheel index
(PyTorch only publishes ROCm wheels for linux_x86_64).
- install.sh: add bitsandbytes install to the migrated-environment
branch so upgrades pick it up for ROCm hosts instead of only the
fresh-install path.
- install.sh: in the Radeon wheel path, pass version constraints +
--no-index --find-links to uv instead of explicit wheel URLs so a
version-compatible torch / torchvision / torchaudio triple is
resolved, rather than picking the highest-version wheel for each
package independently.
- studio/backend/utils/hardware/amd.py _first_visible_amd_gpu_id: fall
through to lower-priority visibility env vars when the first entry
is malformed (leading comma, all-whitespace first token) instead of
silently returning GPU 0.
* Fix round 3 findings: x86_64 guard, ROCm version clip, Radeon deps
Address issues surfaced by the round 3 reviewers on top of 8636fa63:
- install_python_stack.py _ensure_rocm_torch: add the same `x86_64`
guard that install.sh already has. Linux aarch64 / arm64 ROCm hosts
must skip the repair path entirely; PyTorch only publishes ROCm
wheels for linux_x86_64, and without this guard
`unsloth studio update` aborts with a missing-wheel error on non
x86_64 hosts.
- install_llama_prebuilt.py resolve_upstream_asset_choice: add a
best-effort _detect_host_rocm_version() helper (reading
/opt/rocm/.info/version, amd-smi version, hipconfig --version) and
filter rocm_candidates to entries whose major.minor is <= host
version. Falls back to the newest candidate only when no compatible
one exists, so a ROCm 6.4 host downloads rocm-6.4 instead of being
handed the numerically newest rocm-7.2 bundle (which fails preflight
and forces a source build).
- install.sh: remove the round 2 --no-index switch from the Radeon
wheel branch. --no-index forced uv to ignore PyPI entirely, which
broke transitive dependency resolution (filelock, sympy, networkx,
jinja2, fsspec, setuptools, typing-extensions, ...) on a fresh venv.
Restore the round 1 explicit wheel URL invocation but add a
torch / torchvision / torchaudio version-pair sanity check so a
mismatched trio (e.g. torch 2.9.1 + torchvision 0.23.0 + torchaudio
2.9.0) falls back to the standard ROCm index instead of installing a
broken combination.
- install_python_stack.py _ensure_rocm_torch: restructure the
"tag is None" path so it no longer short-circuits the bitsandbytes
install. On a ROCm runtime older than anything in
_ROCM_TORCH_INDEX, print the "no wheel" warning but still run the
AMD bitsandbytes install.
- studio/backend/core/training/worker.py: restore the pre-PR
"no timeout" behaviour for non-HIP causal-conv1d / mamba-ssm source
builds. The round 2 "timeout = 1800 if is_hip else 300" cap aborts
slow non-HIP builds (Linux aarch64, unsupported torch/CUDA combos)
after 5 minutes; omit timeout for the non-HIP branch so the cap
only applies to ROCm source builds.
* Fix round 4 findings: apply_gpu_ids env inheritance, Radeon X.Y, bitsandbytes gate
Address remaining issues surfaced by the round 4 reviewers:
- studio/backend/utils/hardware/hardware.py apply_gpu_ids: mirror the
selection into HIP_VISIBLE_DEVICES / ROCR_VISIBLE_DEVICES whenever
the caller already had a ROCm visibility env var set, not only when
IS_ROCM has already been set by detect_hardware(). Training and
inference workers call apply_gpu_ids() before detect_hardware()
runs, so the old guard would leave a forked ROCm worker with a
stale HIP_VISIBLE_DEVICES mask that no longer matched the
narrowed CUDA_VISIBLE_DEVICES selection.
- install.sh get_radeon_wheel_url: accept X.Y ROCm versions in
addition to X.Y.Z. The `/opt/rocm/.info/version` file and some
hipconfig versions report only two components, and the Radeon
repository publishes both rocm-rel-X.Y.Z/ and rocm-rel-X.Y/
directories, so treating X.Y as invalid caused Radeon hosts to fall
back to the generic ROCm index even when a matching AMD wheel set
existed.
- install_python_stack.py _ensure_rocm_torch: only install the AMD
bitsandbytes build when the venv actually has a ROCm-compatible
torch (either already present or just installed by this function).
Previously the bitsandbytes install ran unconditionally, which
could leave an AMD bitsandbytes layered on top of a CPU/CUDA torch
on hosts where the ROCm runtime is older than any entry in
_ROCM_TORCH_INDEX. Also add --force-reinstall so an existing
CPU/CUDA bitsandbytes is replaced by the AMD build during upgrades.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix gemini findings: amd-smi metric envelope validation and dict-wrapped GPU id
Two medium-severity defensive fixes from the gemini-code-assist review on
the AMD monitoring backend:
1. _extract_gpu_metrics may return a dict where every value is None when
amd-smi succeeds (zero exit) but the JSON envelope contains no usable
fields (error response, unsupported card). The new _has_real_metrics
helper lets get_primary_gpu_utilization surface available:False and
lets get_visible_gpu_utilization skip ghost device rows so the UI
does not render placeholder cards with empty numbers.
2. Newer amd-smi versions wrap scalar fields as {"value": 0, "unit":
"none"}, including the per-GPU id. The previous int(raw_id) call
silently fell back to the enumeration index in that case, losing the
real GPU id. Routing raw_id through the existing _parse_numeric
helper handles bare ints, floats, strings, and the dict shape
uniformly, with a debug log on parse failure.
* Fix gemini round 2 findings: explicit length guard on ROCm version file parser
Both _detect_rocm_version (install_python_stack.py) and
_detect_host_rocm_version (install_llama_prebuilt.py) read /opt/rocm/.info/version
or $ROCM_PATH/lib/rocm_version, split on "." and unconditionally accessed
parts[1]. The surrounding broad `except Exception: pass` already swallowed
the resulting IndexError, so a one-component file like "6\n" did fall
through to the next detection source -- but the control flow relied on
exception handling instead of an explicit check.
Add `if len(parts) >= 2:` guards in both helpers so the loop falls through
on its own without raising. Behaviour is unchanged for the common multi-
component case; the previously-silent IndexError path becomes an explicit
no-op.
* Fix gemini round 3: include has_rocm in validate_server fallback path
When validate_server is called without an explicit install_kind (older
call sites that have not been updated), the fallback was only enabling
--n-gpu-layers for NVIDIA and macOS arm64 hosts. AMD ROCm Linux hosts
fell through to the CPU validation path even though the prebuilt being
exercised was a HIP binary.
Add host.has_rocm to the fallback expression so the GPU offload flag is
applied consistently with the install_kind=='linux-rocm' / 'windows-hip'
branches above.
* Fix gemini round 4: remove risky bytes-vs-MB heuristic in _parse_memory_mb
The previous heuristic divided any bare number above 10_000_000 by
1024*1024 on the assumption that large unit-less values were bytes.
This misclassified small VRAM allocations: 5 MB of used VRAM reported
as 5_242_880 bytes without a unit would be taken at face value and
render as 5_242_880 MB (~5 TB) in the monitoring UI.
Modern amd-smi always provides explicit units (MiB/GiB dict form),
and legacy amd-smi returns bare numbers in MB -- the heuristic never
had a real workload to handle. Drop it and default to MB for bare
numeric input, keeping the existing unit-aware branches for dict /
string inputs unchanged.
The unrelated gemini suggestion to "default minor to 0" in the
amd-smi version awk parser was intentionally NOT applied: rocm7.0
and rocm7.1 ship different wheel sets, so silently substituting 0
for a missing minor could install the wrong wheels. The existing
reject-and-fall-through behaviour is safer.
* Fix gemini round 5: POSIX compliance and leading-comma visibility parsing
Three medium findings from gemini-code-assist addressed in this commit:
1. _pick_radeon_wheel used grep -o and sort -V, both GNU extensions
that are not in POSIX and break on BSD/BusyBox coreutils. install.sh
has a #!/bin/sh shebang so the whole pipeline was rewritten as a
single awk script that extracts all href="..." hits on each line,
filters to wheels matching the package prefix and python tag, and
picks the newest version via zero-padded lexical comparison. No
external sort or grep is needed.
2. _first_visible_amd_gpu_id in the AMD monitoring backend treated a
leading comma (e.g. HIP_VISIBLE_DEVICES=",1") as "fall through to
the next env var", which is surprising given the clear intent to
narrow to device 1. Filter empty tokens after the split and return
the first real one. An all-commas value ("," / ",,,") still falls
through because no real tokens exist; the empty-string and "-1"
explicit-zero cases are unchanged.
The unrelated amd-smi version awk parser suggestion was not applied
(see round 4 commit message for rationale: defaulting a missing minor
to 0 could silently install the wrong ROCm wheel set).
* Fix 20-reviewer.py findings: base drift, Radeon %2B, dpkg/rpm fallback, bnb, backend label
Consolidated fix batch from a 20-parallel reviewer.py run on the current
head. Each fix is drawn from a high-consensus finding and addresses a
real bug or feature gap, not a stylistic preference.
1. install.sh: bump `unsloth>=2026.4.2` -> `unsloth>=2026.4.4` at five
call sites so this branch no longer regresses main's version floor
(main bumped to 2026.4.4 in #4876). Without this, merging 4720 would
silently downgrade the minimum version pin for fresh installs.
2. install.sh: URL-decode Radeon wheel names before extracting the
torch / torchvision / torchaudio version strings. Real wheel URLs
from repo.radeon.com are percent-encoded ("torch-2.10.0%2Brocm7.2.0...")
so the previous `[+-]` terminator in the sed regex never matched,
`_torch_ver` stayed empty, `_radeon_versions_match` stayed false,
and every Radeon consumer install silently fell back to the generic
ROCm index. Now decode %2B -> + first, then extract, then validate.
3. install.sh: the two AMD bitsandbytes install lines were running
`uv pip install "bitsandbytes>=0.49.1"` without `--force-reinstall`,
so upgrades where the venv already has a CPU/CUDA bitsandbytes
satisfying the constraint would keep the stale non-AMD wheel. Add
`--force-reinstall --no-cache-dir` to both call sites, matching the
pattern already used in install_python_stack.py::_ensure_rocm_torch.
4. install_python_stack.py and install_llama_prebuilt.py: add
`dpkg-query -W rocm-core` and `rpm -q rocm-core` fallbacks to the
Python-side ROCm version detectors so they match the chain in
install.sh::get_torch_index_url. Package-managed ROCm installs
(Debian/Ubuntu/RHEL/Fedora distro packages) can expose GPUs via
rocminfo/amd-smi but still lack /opt/rocm/.info/version, hipconfig,
or amd-smi `version` output -- without these fallbacks, `unsloth
studio update` on such hosts returned None and skipped the ROCm
torch repair. Also strip the dpkg epoch prefix ("1:6.3.0-1") before
parsing so epoch-annotated packages parse correctly.
5. hardware.py: add a `_backend_label(device)` helper that returns
"rocm" when IS_ROCM is set and the device is DeviceType.CUDA, and
use it for every `"backend": ...` emission in JSON responses served
to the Studio frontend. Internally we still represent ROCm hosts as
DeviceType.CUDA (ROCm torch reuses the whole torch.cuda.* API
surface), but the user-facing API now correctly reports "rocm" on
AMD boxes instead of labeling them as "cuda".
All 250 simulation scenarios pass (was 233 before this batch: added 17
new regression tests covering the version pin, %2B decoding, bnb
force-reinstall flags, dpkg/rpm fallback presence, and the
_backend_label helper's four-way truth table).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix gemini round 6 + URL audit: amd.py defensive checks, rocm6.5+ clip to 6.4
Two rounds of fixes in one commit, plus a full URL audit of every PyPI /
download.pytorch.org / repo.radeon.com reference the PR introduces.
amd.py (4 medium gemini findings on commit b3627bc2):
1. _extract_gpu_metrics used `and vram_total_mb` as part of the vram_util
gate. The follow-up `vram_total_mb > 0` already handles the division
guard, but the truthiness check was redundant and slightly surprising
for a 0.0 valid value. Replace with explicit `is not None and > 0`
for both vram_util and power_util.
2. get_physical_gpu_count called `data.get("gpu", ...)` without guarding
for non-dict envelopes. A scalar / string JSON response from amd-smi
would raise AttributeError. Add an isinstance(data, dict) check and
return None for unexpected shapes.
3. get_visible_gpu_utilization had the same .get() exposure on the outer
envelope. Rewrite the gpu_list extraction as an explicit
list/dict/else cascade so a malformed scalar envelope produces
gpu_list=[data] and continues without raising.
4. The same function's per-entry loop also called gpu_data.get() on
whatever was inside gpu_list. If a scalar ever leaks into the list
(directly or via the previous fix's fallback), _extract_gpu_metrics
would raise on the first .get() inside the helper. Skip non-dict
entries in the loop before extracting metrics.
install.sh (URL audit finding, previously flagged by 20-reviewer as #13):
5. get_torch_index_url used `rocm6.*` in the rocm tag case statement,
which matched rocm6.5 and rocm6.6 and emitted
download.pytorch.org/whl/rocm6.5 -- which returns HTTP 403 because
PyTorch only publishes rocm 5.7, 6.0-6.4, 7.0-7.2. Enumerate the
supported 6.x minors explicitly and add a rocm6.* fallback branch
that clips to rocm6.4 (the last supported 6.x wheel set).
URL audit results (all URLs PR 4720 references):
- 14/14 download.pytorch.org/whl/{cpu,cu118,cu124,cu126,cu128,cu130,
rocm6.0..6.4,rocm7.0..7.2} return HTTP 200.
- 9/9 repo.radeon.com/rocm/manylinux/rocm-rel-{5.7,6.0,6.1,6.2,6.3,
6.4,7.0,7.1,7.2}/ return HTTP 200.
- X.Y.Z patch directories exist for 7.0.2, 7.1.1, 7.2.1 but NOT for
6.3.0, 6.4.0, 6.2.1 -- install.sh already handles this via the X.Y.Z
-> X.Y fallback sed in the Radeon wheel install block.
- Docs links (rocm.docs.amd.com, docs.unsloth.ai AMD guide) and the
llama.cpp GitHub releases API endpoint all return 200.
Test suite: 255 -> 258. New regression coverage:
- U17: get_physical_gpu_count tolerates scalar amd-smi envelope
- U18: get_visible_gpu_utilization tolerates scalar envelope
- U19a-c: vram_util / power_util return None on zero total, but
vram_total_gb still echoes 0.0 (not None)
- A_rocm{6.5,6.6,6.9}_clips_to_rocm64: install.sh clips unsupported
6.x minors to rocm6.4 instead of producing a 403 index URL
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix reviewer.py round 2: tokenizer AMD multi-GPU, --no-torch bnb, main.py backend label
Three high-confidence findings from a second 20-parallel reviewer.py run
on commit 7effb3ae. Triaged 15 total findings and applied the three that
were confirmed as real bugs; the rest were either false positives (e.g.
"migrated AMD venv not repaired" -- _ensure_rocm_torch runs downstream
via setup.sh regardless), design decisions (e.g. visibility mask env
vars not consulted in installer detection), or edge cases the existing
fallback logic already handles.
1. unsloth/tokenizer_utils.py [6/20]: the multi-GPU guard's shell probe
runs `nvidia-smi --query-gpu=memory.used`, catches the failure, then
only raises if `torch.cuda.is_available()` is False. On ROCm torch,
torch.cuda.is_available() returns True (ROCm reuses the torch.cuda.*
API), so the guard becomes dead code on AMD hosts and multi-GPU AMD
setups slip through even though unsloth does not support them yet.
Add a torch.cuda.device_count() > 1 fallback inside the except so
AMD multi-visible-device setups are flagged consistently with the
original CUDA memory check.
2. install.sh [1/20]: the fresh-install bitsandbytes block for AMD ROCm
ran unconditionally when TORCH_INDEX_URL matched `*/rocm*`, even when
SKIP_TORCH=true (from --no-torch or Intel Mac auto-detect). A user
running `install.sh --no-torch` on an AMD host would still pull in
bitsandbytes despite explicitly asking for GGUF-only mode. Wrap the
case block in an outer `[ "$SKIP_TORCH" = false ]` guard.
3. studio/backend/main.py [3/20]: the /api/system endpoint returned
`"device_backend": get_device().value`, which is "cuda" on ROCm
hosts (because ROCm torch piggybacks on torch.cuda). Other endpoints
(hardware.py) already use the _backend_label helper which swaps
"cuda" -> "rocm" when IS_ROCM. Route /api/system through the same
helper so the Studio UI reports the backend consistently across all
endpoints.
4. studio/backend/tests/test_utils.py: update test_backend_matches_device
to call _backend_label(get_device()) instead of raw get_device().value
so the test matches the new contract and still passes on CUDA hosts.
Tests: 258 -> 261. New regression coverage:
- X08 main.py /api/system uses _backend_label
- X09 tokenizer multi-GPU guard has device_count() fallback
- X10 fresh-install bnb case block gated on SKIP_TORCH=false
* fix: prevent bitsandbytes from overwriting ROCm torch with CUDA wheels
During install, bitsandbytes was installed without --no-deps, causing
uv to resolve torch from PyPI (CUDA build) and silently overwrite the
ROCm wheels that were just installed in the previous step.
This happened in three places:
- install.sh: bitsandbytes install in both migrated and fresh paths
- install_python_stack.py: bitsandbytes install inside _ensure_rocm_torch()
Additionally, multiple install steps in install_python_stack.py (extras,
overrides, studio deps) can pull in CUDA torch via transitive
dependencies. A final _ensure_rocm_torch() call at the end of the
install sequence ensures ROCm torch is always in place at runtime.
All changes are gated behind ROCm-specific conditions and do not affect
NVIDIA, CPU-only, macOS, or Windows install paths.
Tested on AMD Instinct MI300X VF with ROCm 7.2.0 -- confirms
torch==2.10.0+rocm7.1 with HIP 7.1.25424 after install.
* fix: ROCm inference fallback -- skip Unsloth patching and bnb 4-bit on HIP
On AMD ROCm (HIP), two issues prevent the normal Unsloth inference path:
1. Unsloth's global monkey-patching of transformers model classes
(LlamaRotaryEmbedding, attention modules) triggers
_assert_async_cuda_kernel crashes on HIP during generation.
Training uses different code paths and works fine.
2. bitsandbytes 4-bit matmul kernels also trigger HIP assertion
failures on MI300X (CDNA3 / gfx942), even without Unsloth patching.
This commit adds a ROCm-specific inference fallback that:
- Skips importing Unsloth at module level (prevents global patching)
- Loads models in 16-bit with plain transformers + PEFT instead
- Resolves pre-quantized model names (e.g. "xxx-bnb-4bit" -> "xxx")
since pre-quantized HF repos still trigger bnb codepaths
- Guards get_chat_template calls (unavailable without Unsloth import)
- Fixes max_seq_length=0 being passed to from_pretrained (GGUF
semantics don't apply to transformers path)
The NVIDIA path is completely unchanged -- Unsloth import and
for_inference() optimization remain active. GGUF inference (via
llama-server/HIP) is unaffected since it never imports Python model
classes. AMD GPUs typically have large VRAM (e.g. 192GB on MI300X)
so 16-bit loading is practical for inference.
Tested on AMD Instinct MI300X VF (ROCm 7.2, HIP 7.1.25424):
- Simple generation: PASS
- Compare mode (base vs finetuned): PASS
- GGUF inference + tool calling: PASS (unaffected by this change)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: guard audio/vision inference on ROCm, remove unused import
- Add clear RuntimeError for audio/vision model inference on ROCm
(these paths use Unsloth's FastModel/FastVisionModel which would
crash on HIP; GGUF inference is the supported path on AMD)
- Remove unused `import os as _os` from the ROCm changes
* fix: amd-smi parsing for newer output format (gpu_data wrapper, mem_usage, temperature)
amd-smi on recent ROCm versions (7.x) wraps metric output in a
{"gpu_data": [...]} envelope instead of returning a raw list. This
caused get_primary_gpu_utilization() and get_visible_gpu_utilization()
to fail silently (returning available=False) because the GPU data
dict was never unwrapped.
Additionally:
- VRAM data moved from "vram" to "mem_usage" with "total_vram" /
"used_vram" keys. Added fallback key lookup.
- Temperature "edge" sensor returns "N/A" on MI300X VF; the previous
dict.get() chain returned the "N/A" string instead of falling
through to "hotspot". Changed to a loop that checks each key until
a parseable value is found.
Tested on AMD Instinct MI300X VF (ROCm 7.2, amd-smi 24.x):
- GPU utilization: 0% (idle), up to 100% during training
- Temperature: 40-44C (from hotspot sensor)
- VRAM: 0.28/191.69 GB (idle)
- Power: 158-211W draw
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Bug fix detecting radeon (#4940)
* Bug fix detecting radeon
* Expanding GPU target for gfx1100*
* Generalize gfx family-prefix filter to cover gfx10/gfx12 as well
rocminfo on ROCm 6.1+ emits LLVM generic-family ISA lines alongside the
specific GPU (e.g. gfx11-generic next to gfx1100). The outer grep captures
the bare family prefix from the generic line, and passing that to
-DGPU_TARGETS breaks the HIP build because clang only accepts specific
gfxNNN ids.
The previous filter only special-cased gfx11. Generalize it so any bare
2-digit family prefix (gfx10, gfx11, gfx12, ...) is dropped whenever a
specific sibling target is present in the same list. No real AMD GPU has
a 2-digit gfx id, so the filter can only ever drop family prefixes and
never a real target.
Covers the existing gfx11 cases unchanged, and extends the same fix to
gfx10-1-generic / gfx10-3-generic (RDNA1/2) and gfx12-generic (RDNA4),
which would otherwise hit the same build failure on newer rocminfo.
---------
Co-authored-by: Iswarya Alex <iswarya.alex@amd.com>
Co-authored-by: Daniel Han <danielhanchen@users.noreply.github.com>
---------
Co-authored-by: Eda Z <eda.zhou@amd.com>
Co-authored-by: GoldenGrapeGentleman <yueyuan@amd.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: billishyahao <bill.he@amd.com>
Co-authored-by: Iswarya Alex <47045679+iswaryaalex@users.noreply.github.com>
Co-authored-by: Iswarya Alex <iswarya.alex@amd.com>
Co-authored-by: Daniel Han <danielhanchen@users.noreply.github.com>
Wasim Yousef Said