GitHub macOS runners (and any headless or virtualized Mac) have no usable
Metal, so the macos-arm64 prebuilt loads the validation model on CPU. The new
offload check rejected it and forced a source build that also runs on CPU and
then failed to launch, breaking 'Install + load (macos-*)' and 'Studio API &
Auth Tests'.
A CPU-only Metal load is an unfixable environment limitation, not a fixable
binary fault like a missing cudart/cublas DLL or a PTX-only CUDA build, so a
rebuild gives the same result. Split the GPU kinds into _GPU_INSTALL_KINDS
(launched with --n-gpu-layers, so a broken libggml-metal.dylib still surfaces)
and _GPU_OFFLOAD_REQUIRED_KINDS (CUDA/ROCm/HIP only, where a CPU-only load is
rejected). validate_server, the existing-install re-validation, the fast-path
reuse gate, the smoke-test require-signal, and the success message all key on
the offload-required set, so macOS accepts its prebuilt while CUDA/ROCm
rejection is unchanged.
run_smoke_spoof drives the rejection contract with a CUDA kind on every runner
(install_kind is explicit, so it is OS-independent) and adds a macOS-only case
asserting a CPU-only Metal load is accepted. New unit tests cover the split.
- direct_upstream_release_plan: Linux x86_64 and arm64 CPU branches now also
require not host.has_rocm, so a ROCm host whose HIP prebuilt is missing or
rejected never silently takes a CPU upstream tarball as a 'success'.
- setup.ps1: a Windows ROCm host with no usable HIP prebuilt and no HIP source
build path is now marked LlamaCppDegraded so the CPU last-resort installs a
clearly labelled CPU build instead of a 'built' install that runs on CPU.
- validate_server gains require_gpu_signal; smoke_test_server_binary sets it for
GPU install kinds so the '0 = offload confirmed' CLI contract treats a
no-GPU-signal log as inconclusive (EXIT_ERROR) rather than a silent pass.
- run_smoke_spoof + unit test cover the new no-signal-is-inconclusive contract.
A metadata match used to short-circuit straight to reuse, so a previously
installed CPU-only 'GPU' binary survived every rerun and restart -- the exact
'picks CPU forever' report. install_prebuilt now smoke-tests a matching GPU
install and reinstalls if it loads on CPU; setup.sh/setup.ps1 smoke-test a reused
source build on a GPU host and rebuild if it ran on CPU. Non-GPU installs keep
the fast path.
The post-build smoke test relied on the installer re-detecting the GPU, but on
amd-smi-only or name-inferred ROCm hosts that probe can miss the GPU and resolve
a CPU kind, skipping the offload gate. setup.sh/setup.ps1 now pass --install-kind
from the backend they just built, and the --smoke-test CLI applies --has-rocm /
--rocm-gfx host overrides like the install path does.
Reviewers found a real CPU-only log shape that passed: an 'offloading 0
repeating layers to GPU' planning line, or a GPU KV/compute buffer, could read
as GPU before the definitive 'offloaded 0/33' was seen. Check the explicit
counted offload first (any N>0 wins, all zero is CPU-only), restrict the
buffer-size signal to GPU model buffers (KV/compute on GPU with weights on CPU
is still CPU inference), and add HIP/MUSA/CANN to the model-buffer markers so an
older log naming those backends is not misread as CPU. Same in both classifiers.
Exclude host-pinned buffers (CUDA_Host etc.) from the GPU buffer-size signal so a
binary that pins host memory but loads weights on CPU is not misread as GPU
offload. Scan every 'offloaded N/M layers to GPU' line and accept if any N>0 so a
speculative draft model logging 0/k before the main model's 33/33 is not flagged
CPU-only. Same fix in the installer and runtime classifiers.
Adds a GPU-less smoke that drives install_llama_prebuilt.py --smoke-test against
a fake llama-server emitting CPU-only vs GPU device_info logs, asserting a
CPU-only binary tagged as a GPU install is rejected (exit 2) and a GPU one is
accepted (exit 0), plus the selection and classifier unit tests, on all three
OSes. This is the coverage gap that let the silent CPU-only path ship.
setup.sh/setup.ps1 now run install_llama_prebuilt.py --smoke-test on a freshly
source-built GPU binary and retry a CPU build if it loaded on CPU only. When a
GPU host's source build produces no binary, both scripts fall back to the CPU
prebuilt (--cpu-fallback) as a labelled last resort instead of leaving the host
without llama.cpp. setup.ps1 also guards an empty CUDA arch (no PTX-only binary,
#5854). The POSIX smoke-test exit code is captured set -e safe. Adds a fake
llama-server and an end-to-end spoof test that runs the real validate_server
against it with no GPU.
A llama-server whose GPU backend fails to initialize still serves HTTP 200 from
CPU, so install validation accepted it and Studio ran 'GPU' inference on CPU
(#5807/#5106/#5830). Add a log classifier (buffer-size, offloaded-layers, and
device_info signals) and reject a GPU-intended binary that loaded on CPU, so the
resolver advances. Crucially, a GPU host no longer gets a CPU prebuilt appended
to its simple-policy attempts, so when no GPU bundle offloads the installer falls
through to a source build for the native arch instead of silently installing CPU.
Add a --smoke-test CLI for setup scripts and align the runtime classifier with
the same signals so its CPU-only warning fires on current llama.cpp.
Improve the rejection message when an MCP server address is not an http(s) URL. It now points to the expected http(s):// form with an example, and only mentions that local commands are disabled when the value contains whitespace (a reliable command signal), since a lone token may just be a scheme-less URL. Wording is host-scoped rather than desktop-only because self-hosted hosts can opt in via an env var. Backend only, with tests; accepted input is unchanged.
Move MCP from the chat settings sidebar to a composer pill next to Artifacts. The dropdown lists keyless presets (Context7, Exa, Hugging Face), all off by default, created on first enable and deduped by URL. Add custom MCP opens the existing manage dialog. Enabled rows show a green underlay and a tick that becomes an X on hover to remove; enabling Exa turns off the built-in Web Search and shows a hover tooltip. Reconciles the per-chat MCP flag on load so already-enabled servers activate the pill. No backend changes.
* Studio: cover B300 (sm_103) with the Linux prebuilt bundles
sm_103 (B300 / GB300 Blackwell Ultra) was in no bundle's supported_sms,
so those hosts fell through to a slow source compile. The newer and
portable bundles already ship base compute_100 PTX, which the driver
JIT-compiles forward to sm_103, so list sm_103 alongside sm_100 in those
bundles and let B300 install the prebuilt.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
setup.ps1 already forwards --has-rocm whenever AMD is detected regardless
of whether gfx resolved. setup.sh only forwarded --rocm-gfx, so a Linux
host where AMD was detected but gfx resolution failed would forward nothing,
leaving the installer with has_rocm=False and falling back to a source build.
Add the elif branch to forward --has-rocm when _setup_amd_detected is true
but _setup_gfx is empty, matching setup.ps1 parity. Add a source-level test
to verify both flags are present in setup.sh.
The 'Collect llama-server logs' diagnostic step copies llama-server stdout into
the workspace for debugging. On the Windows runners it intermittently dies with
exit code -1073741502 (0xC0000142, STATUS_DLL_INIT_FAILED) when spawning the
copy, which failed the whole Tool calling Tests job even though the tests had
already run.
It is a diagnostic-only step, so mark it continue-on-error in all three jobs, the
same treatment #5913 gave the artifact-upload steps. A transient log-collection
crash no longer turns a green job red.
Refs #5913.
setup.sh and setup.ps1 resolve the AMD gfx target (rocminfo/hipinfo/amd-smi,
name inference, or UNSLOTH_ROCM_GFX_ARCH) but never passed it to
install_llama_prebuilt.py. The installer re-probed on its own and, on hosts where
hipinfo/amd-smi cannot report the arch (amd-smi-only Linux, HIP-runtime-only
Strix Halo, name-inferred GPUs), left rocm_gfx_target unset. The lemonade HIP
prebuilt selection needs that arch, so those hosts got no GPU prebuilt and fell
back to a source build.
Add a --rocm-gfx argument (defaulting to UNSLOTH_ROCM_GFX_ARCH) and fold it into
the host profile via _apply_host_overrides: a forwarded gfx is authoritative
(setup already applied visible-device selection) and implies has_rocm. setup.sh
and setup.ps1 now forward their resolved arch.
Add unit tests for _normalize_forwarded_gfx and _apply_host_overrides, plus
source checks that both setup scripts forward --rocm-gfx.
The deferred Windows CUDA Toolkit check (Resolve-CudaToolkit in setup.ps1) has a
pwsh unit test but nothing ran it. Rather than add a separate workflow, fold a
fast GPU-free gate into the existing Windows GGUF CI: parse setup.ps1 and run
tests/studio/test_resolve_cuda_toolkit.ps1 right after checkout, before the heavy
GGUF smoke, so a setup.ps1 regression fails fast on the runner that is already
spun up.
Refs #5912.
* Studio: fall back to source build for arm64 Linux GPU hosts
setup.sh routes a Linux host with any GPU tool to the unslothai/llama.cpp fork,
which publishes only linux-x64 bundles. On an arm64 host with a GPU (GH200,
GB200, DGX Spark) the resolver then selected an x64 CUDA bundle, which cannot run
on aarch64. Routing those hosts to ggml-org instead would install a CPU-only
arm64 build, silently losing the GPU.
Guard resolve_simple_install_release_plans so an arm64 Linux host on the fork
raises PrebuiltFallback before any release is fetched, letting setup.sh do a
source build that actually targets the GPU. x86_64 hosts and arm64 CPU hosts
(which route to ggml-org) are unaffected.
Add tests covering the arm64 fork raise, the x86_64 pass-through, and the arm64
CPU ggml-org path.
* Studio: install ggml-org arm64 CPU prebuilt when the arm64 GPU source build fails
Per review of #5924: arm64 Linux GPU hosts have no CUDA prebuilt anywhere (the
unslothai fork is x64 only, ggml-org ships no Linux CUDA build), so they source
build for the GPU. If that build produces no binary, the host was left without
llama.cpp.
Add a --cpu-fallback flag to install_llama_prebuilt.py that drops the host GPU
attributes so the CPU prebuilt for the host arch is selected (a GPU host cannot
otherwise pick the CPU bundle). setup.sh calls it against ggml-org as a last
resort for arm64 Linux when the source build degraded, installing the
ubuntu-arm64 CPU build instead of leaving the host with no llama.cpp.
Add tests: force_cpu drops GPU attrs before planning, a CPU-forced arm64 host
selects the ggml-org ubuntu-arm64 bundle, and setup.sh wires the fallback.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
The pinned b9360 cuda-13.1 Windows fallback for Blackwell (sm_120) only fired
when the driver advertised CUDA >= 13.1. The mainstream Blackwell branch ships
the r580 driver that reports CUDA 13.0, so those hosts missed the pin, were
gated off the in-release 13.3 build, and dropped to the CPU-only cuda-12.4
build.
b9360's binary is native sm_120a SASS (no PTX, so no JIT) and its bundled
runtime is cuda-13.1 cudart; both run on a 13.0 r580+ driver under CUDA
minor-version compatibility. Lower _PINNED_BLACKWELL_DRIVER_FLOOR to (13, 0) so
the pin covers the 13.0 branch too. The generic published-runtime gate stays
conservative (an unverified in-release 13.1 build is still gated off 13.0); only
the specific, hash-pinned, SASS-verified b9360 build is extended.
Refs #5887.
* Studio: match the Linux llama.cpp prebuilt to the runtime cudart major
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Studio: defer the Windows CUDA Toolkit check so prebuilt users are not blocked
setup.ps1 section 1e validated the installed CUDA Toolkit against the driver in
Phase 1 and hard-exited when the toolkit was newer than the driver supports.
That ran before the prebuilt llama.cpp install (Phase 3.4), which is
self-contained and needs no local toolkit. A Blackwell host (sm_120) on a driver
that advertises CUDA 13.2 with CUDA Toolkit 13.3 installed was blocked entirely,
even though the prebuilt selector would have given it a working GPU build.
The toolkit (nvcc) is only consumed by the Phase 4 source build. Move section 1e
into a Resolve-CudaToolkit function and call it lazily at the source-compile
branch, gated on $HasNvidiaSmi. The prebuilt path no longer touches the toolkit;
forced/source builds keep the identical requirement, winget auto-install, and
error text via -RequireOrExit.
Add a pwsh unit test that extracts the real function and runs it against a
spoofed Blackwell sm_120 / driver 13.2 / toolkit 13.3 host: the prebuilt path
defers, the forced build still fails fast.
Refs #5879.
* Studio: correct a stale Phase-1 prerequisites comment (CUDA Toolkit is now resolved lazily)
* Studio: retrigger Windows GGUF CI (artifact upload flake)
Extracts the real guard block from install.sh and asserts: clean arm64 venv
untouched, x86_64 venv rebuilt as arm64, the x86_64-then-3.13.8 corner case,
arm64 3.13.8 downgrade preserved, --python skip, and Intel/Rosetta no-op.
Co-authored-by: Ramakrishna Bachu <ramankrishna10@gmail.com>
On Apple Silicon, uv can create the venv from a cached x86_64 (Rosetta)
Python, so the venv reports x86_64 to wheel resolvers and the torch install
never resolves (the CPU index ships no macOS wheels). Detect an x86_64 venv
on an arm64 host and recreate it with an arch-explicit arm64 CPython,
re-inspecting before the existing 3.13.8 check so both invariants hold.
Skipped when --python is set; non-macOS, Intel, and healthy arm64 venvs are
unaffected.
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Studio: pin the last pre-macOS-26 llama.cpp prebuilt instead of walking back
ggml-org moved their macOS build runner to macOS 26 (Tahoe) at b9428, so
b9428 and every newer upstream prebuilt is stamped minos 26 and fails to
dyld-load on macOS 14 / 15. #5883 handled this by walking back release by
release at install time.
Replace that with a deterministic pin: a host below macOS 26 selects b9415
directly (the last upstream build stamped below 26: arm64 minos 14, x64
minos 13.3), so it loads on macOS 13.3 / 14 / 15 / 26. Hosts on macOS 26+
and unknown-version hosts keep latest selection unchanged.
Only the ggml-org upstream path is pinned; the unslothai/llama.cpp fork
ships its own minos-13.3 prebuilts (#5893), so the pin is a no-op there and
goes dormant once macOS routes to the fork. The Mach-O minos preflight from
#5883 stays as a post-download backstop.
Refs #5883, #5893.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* docs: fork ships arm64 minos 14 / x64 minos 13.3, not uniform 13.3
The per-slice fork producer pins arm64 to 14.0 and x64 to 13.3; update the
pinned_macos_release_tag docstring to match. No behavior change.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
The SourcesGroup measurement container renders every citation badge
off-screen with `invisible absolute` so we can measure how many fit in
two rows. Absolute descendants still contribute to the parent's
scrollable overflow region, so long source lists added hundreds of
pixels of phantom scroll space below the assistant message that the
user could scroll into.
Wrap the measurement container in an absolute, `h-0`, `overflow-hidden`
box so the off-screen pills are clipped out of the scrollable overflow
region. Measurement still works because offsetTop is read relative to
the positioned wrapper.
* Studio: harden stdio MCP gating and fix transport edge cases
- Gate the Data Recipe stdio path behind UNSLOTH_STUDIO_ALLOW_STDIO_MCP so a hosted deployment cannot spawn local processes through recipes
- Enforce the gate inside _client() so the transport sink cannot spawn when disabled
- keep_alive=False so stdio probes/calls do not leave orphan subprocesses
- Force OAuth off for stdio servers on create and update
- Drop stored headers when a server switches transport type
- Reject a command whose first token is a URL scheme
- Add MCP gate and improvement tests
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* tests: skip Data Recipe stdio tests when data_designer is absent
The data_designer plugin is only installed in the Studio test job, so guard
the two build_mcp_providers tests with importorskip so the core matrix skips
them instead of failing on ModuleNotFoundError.
---------
Co-authored-by: Daniel Han <michaelhan2050@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Classify llama-server startup failures so diffusion/image GGUFs (FLUX, Qwen-Image, LTX, ERNIE-Image, Z-Image, ...) point users to the Images page instead of a misleading out-of-memory error. Other unknown architectures get a precise unsupported message; Ollama and OOM fallbacks are preserved.
Architecture is matched exactly against general.architecture, covering the arches Unsloth ships as GGUF: flux, qwen_image, ltxv, wan, lumina2.
Fixes#5842.
Make the network-touching install and download steps in the security-audit workflow resilient to transient failures without relaxing any integrity check.
- Add top-level retry and backoff env knobs for pip, cargo, and npm.
- Wrap the pip-audit + cargo install and npm ci steps in an exponential-backoff retry helper, preserving --locked and --ignore-scripts.
- Re-pin swatinem/rust-cache to the v2.9.1 commit so the SHA matches its comment.
- Split the OSV-Scanner download and SHA-256 verification into a hard-gated step: a checksum mismatch fails the job, while a transient download failure skips the scan; the advisory scan stays non-blocking.
Make the NVIDIA prebuilt selection version-dynamic and driver-gated on both the
Windows (ggml-org) and Linux (unslothai/llama.cpp) paths, so CUDA version bumps
are handled with no code change while staying safe across driver versions.
- Derive candidate CUDA runtime lines from the driver major; pick the highest
upstream-published minor the driver can actually run, so a sub-13.3 driver is
never handed an unguaranteed 13.3 build.
- Pin b9360's cuda-13.1 build (immutable, hash-verified) as a GPU fallback for a
Blackwell host on a 13.1/13.2 driver that the in-release 13.3 build gates off,
on both the simple and published install paths. Dormant for every other host
and self-disabling once upstream ships a driver-runnable build again.
- Seed the published-path ordering from the release's real published minors so a
future CUDA major is selectable with no code change.
Refs #5861, #5817, #5807, #5858, #5854, #5826, #5887.
Make macOS llama.cpp prebuilt selection host-OS-version aware: skip a prebuilt whose minimum-OS exceeds the host and walk back to the newest release that loads (macOS 26 keeps latest; 14/15 land on a compatible older release). Source-build fallback pins CMAKE_OSX_DEPLOYMENT_TARGET=13.3. CI: binary-load assertion plus a macos-14/15/26 install matrix. No change to Linux/Windows or CUDA selection.
Harden local GGUF detection so a .gguf path is not misrouted to the transformers backend during the brief Windows lock window after llama-server is killed. Catch OSError from stat() and treat the path as the file, while a directory named *.gguf still falls through to the directory scan. Adds regression tests for the lock-window and directory cases.
Follow-up cleanups to the merged AMD ROCm support PR #5301:
1. De-duplicate the torchao Windows-ROCm import stub into a single shared
module (studio/backend/core/_torchao_stub.py); both workers call one
install_torchao_windows_rocm_stub() entrypoint.
2. Align the gfx name/arch comment columns in setup.sh and setup.ps1.
3. Isolate the float16 dtype fallback to AMD without native bf16; NVIDIA
keeps dtype=None so unsloth's own bf16/fp16/FORCE_FLOAT32 detection is
honored.
4. Hoist unconditional stdlib imports (gc, glob, re, subprocess, copy,
types, sys, importlib.metadata) from function bodies to module top
across the PR #5301-touched files; heavy/optional/relative imports stay
lazy.
5. bitsandbytes Windows-ROCm install now uses plain pip (force_pip=True)
instead of UV_SKIP_WHEEL_FILENAME_CHECK, per the AMD hackathon docs.
Also adds scripts/verify_import_hoist.py (a scope-aware LEGB AST resolver
that catches dangling-alias and rename-clash bugs in import-hoist
refactors) and wires it into the Lint CI source-lint job as a self-test
plus a pull_request compare gate.
* feat(studio): use lemonade-sdk/llamacpp-rocm per-GPU prebuilts for ROCm hosts
For AMD GPUs that rocminfo/hipinfo reports a recognised gfx target
(gfx103X / gfx110X / gfx1150 / gfx1151 / gfx120X), resolve_lemonade_rocm_choice()
now fetches the latest lemonade-sdk/llamacpp-rocm release and returns the
matching per-architecture zip, bundling all required ROCm runtime libs.
This runs before the existing upstream ggml-org combined-ROCm tarball fallback
on Linux and before the upstream HIP zip on Windows, so both platforms benefit
from the more targeted build when available.
Changes:
- Add LEMONADE_ROCM_REPO / LEMONADE_ROCM_RELEASES_API constants
- Add HostInfo.rocm_gfx_target populated from rocminfo (Linux) / hipinfo (Windows)
- Add _LEMONADE_GFX_FAMILIES prefix map and _lemonade_gfx_family() helper
- Add resolve_lemonade_rocm_choice() that fetches latest lemonade release and
constructs the llama-{tag}-{os}-rocm-{gfxFamily}-x64.zip asset URL
- Wire into resolve_upstream_asset_choice() for both Linux (ubuntu) and Windows paths
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Honor pinned llama.cpp tag in lemonade ROCm resolver
The resolver always fetched lemonade-sdk/llamacpp-rocm's /releases/latest,
ignoring the upstream llama.cpp tag the caller had pinned. On a reproducible
install where the user requested 'b1260' that meant we would silently pick
up whatever lemonade had published as latest at install time, with no way
to roll back to the matching tag.
Lemonade tags llama.cpp upstream tags 1:1, so:
- When llama_tag is unset or 'latest', keep hitting /releases/latest.
- When llama_tag is pinned (e.g. 'b1260'), hit /releases/tags/b1260.
- When the pinned tag is not published by lemonade (404), skip silently
and let the caller fall through to the upstream tarball -- this keeps
pinned installs reproducible instead of drifting.
resolve_lemonade_rocm_choice now takes llama_tag (default 'latest' for
backward compatibility) and both call sites in resolve_upstream_asset_choice
forward the upstream llama_tag to it.
Note: this PR still has open integration concerns flagged in review --
the simple-policy planner and approved-checksum manifest don't yet route
or accept lemonade assets. Those are larger changes and not in scope for
this commit; addressing the pinned-tag drift independently because it is
small, localized, and self-contained.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add mock test for lemonade ROCm prebuilt asset resolution
Validates GPU family mapping and that resolve_lemonade_rocm_choice
returns real lemonade release URLs for all supported gfx targets on
both Linux and Windows, without requiring AMD hardware.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Wire lemonade ROCm prebuilts into the simple-policy install path
setup.sh invokes install_llama_prebuilt.py with --simple-policy, which
dispatches through resolve_simple_install_release_plans -> direct_linux_release_plan
(or direct_upstream_release_plan on Windows). Those planners only
handled CUDA + CPU attempts, so ROCm-only hosts (e.g. gfx1151 Strix
Halo) had no compatible prebuilt asset and silently fell through to
source build, even though resolve_lemonade_rocm_choice already knew
how to fetch a per-GPU lemonade-sdk binary.
Add a lemonade ROCm/HIP attempt to both simple-policy planners for
ROCm-only hosts, and add regression tests that drive the dispatchers
end-to-end so this can't be skipped silently again.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Document that lemonade ROCm prebuilts work on any glibc Linux
The lemonade-sdk asset filename uses "ubuntu" as a label, but the binary
is a manylinux-style glibc build with no Ubuntu-specific dependencies.
It runs on Arch, Fedora, openSUSE, Debian, etc. as long as the host
glibc is recent enough.
No behavior change -- the dispatch already runs for any Linux ROCm
host. This commit only clarifies the comment, docstring, and log
message so users on non-Ubuntu distros (e.g. Strix Halo on Arch) don't
mistake the asset name for distro gating.
* Pattern matching fix for libggml-cpu*.so*
* fix(lemonade): pass resolved tag to lemonade resolver; add upstream HIP fallback; stub API in tests
- direct_linux_release_plan: pass bundle.upstream_tag (not requested_tag)
to resolve_lemonade_rocm_choice so a "latest" request doesn't mix a
newer lemonade binary with an older planned unsloth release (Codex P2)
- direct_upstream_release_plan: same fix on the Windows path (release_tag
instead of requested_tag); also add the upstream HIP asset
(llama-<tag>-bin-win-hip-radeon-x64.zip) as a fallback between lemonade
and CPU so unsupported GPUs or transient lemonade failures don't silently
downgrade to CPU when an upstream ROCm prebuilt exists (Codex P2)
- test file: stub fetch_json with a synthetic lemonade release payload so
the suite is hermetic and not subject to GitHub API rate limits (Codex P1);
add test_simple_policy_windows_hip_falls_back_to_upstream_when_lemonade_unavailable
to cover the new HIP fallback path
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(lemonade): revert bad tag fix; keep upstream HIP fallback + hermetic tests
The previous commit wrongly passed bundle.upstream_tag / release_tag to
resolve_lemonade_rocm_choice. Lemonade uses its own versioning (b1262,
b1264, …) completely independent of unslothai's tags (b9186, …), so
passing a resolved unslothai tag caused a 404 and silently skipped the
lemonade binary entirely. Revert both call sites to requested_tag.
Keep the two valid fixes from the prior commit:
- Upstream HIP fallback (llama-<tag>-bin-win-hip-radeon-x64.zip) between
lemonade and CPU in direct_upstream_release_plan, so unsupported GPUs
or transient lemonade failures don't silently downgrade to CPU (Codex P2)
- Stub fetch_json in tests so the suite is hermetic (Codex P1)
* fix(studio/rocm): respect HIP_VISIBLE_DEVICES when picking lemonade gfx target
The rocminfo / hipinfo regex took the first gfx match in the agent listing.
On mixed APU + dGPU hosts (e.g. Strix Halo gfx1151 + discrete RX 7900 gfx1100)
this picked whichever GPU appeared first in the tool's stdout, not the one
HIP actually runs on. The downloaded lemonade asset could then be a binary
for a different arch than the active device.
Extracted a module-level _pick_rocm_gfx_target() helper that:
- collects every gfx token in order via re.findall (skips gfx000 / generic ISAs)
- if HIP_VISIBLE_DEVICES or ROCR_VISIBLE_DEVICES is set, parses the first
comma-separated entry as an integer index into that list
- falls back to the first GPU for non-integer (UUID-style) or out-of-range
values, matching the previous default behaviour
Both Linux (rocminfo) and Windows (hipinfo) branches use the helper.
Existing 18 lemonade tests pass; no behavioural change for single-GPU hosts.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio/rocm): dedup rocminfo gfx tokens and honor disabled visibility
Follow-up to 8793aef0. rocminfo / hipinfo emit each gfx target multiple
times per GPU (Name, ISA triple, marketing name), so the prior re.findall
indexing returned the wrong device when HIP_VISIBLE_DEVICES picked GPU 1
on a mixed-arch host -- the helper picked the second occurrence of GPU 0
instead. Collapse to unique tokens (insertion-ordered) before indexing.
Also handle HIP_VISIBLE_DEVICES / ROCR_VISIBLE_DEVICES values of '' and
'-1' as "no AMD visible" (matches the rest of Studio's visibility code),
returning None so the planner does not pick a Lemonade asset for a
hidden GPU.
* fix(studio/rocm): memoise lemonade release lookup + HIP_PATH hipinfo fallback
Robustness pass on top of 25d4ab63:
1. resolve_lemonade_rocm_choice() is called twice per install (planner +
resolve_upstream_asset_choice), so every install previously hit
api.github.com twice with identical args -- doubling the 403/rate-limit
failure surface on busy CI runners. Extract the fetch into a
functools.lru_cache(maxsize=8) helper keyed on (api_url, llama_tag).
The cached helper also owns the error-path logging the resolver was
doing inline. Tests that need to vary fetch_json output across
invocations should call _fetch_lemonade_release_cached.cache_clear().
2. Windows detect_host probe for hipinfo / amd-smi only used
shutil.which(), so HIP SDK installs that set HIP_PATH but do not put
%HIP_PATH%\bin on system PATH classified the host as non-ROCm. The
PowerShell installer and studio/install_python_stack.py already
resolve HIP_PATH\bin\hipinfo.exe as a fallback; mirror that here so
the install planner agrees with the rest of Studio on what counts as
a ROCm host.
Tests: 18 passed (shipped); 52 extra sim cases pass (added 2 for the
lru_cache deduplication path).
* fix(studio/rocm): lemonade URL trust pinning, runtime overlay covers HIP libs, opt-out env
Robustness pass driven by 5 parallel reviewers of head c08b15e6:
1. URL trust pinning. AssetChoice.url comes from the GitHub API response's
browser_download_url field. Lemonade attempts are not in the approved-hash
manifest, so a compromised API response could redirect the download to an
attacker-chosen host without the integrity gate catching it. New
_is_trusted_github_release_url() validates https + github.com/<expected_repo>
release path OR objects.githubusercontent.com (GitHub's CDN). Resolver
refuses to download otherwise.
2. UNSLOTH_DISABLE_LEMONADE_ROCM opt-out. Users who prefer the upstream HIP
build path can set this env var to skip lemonade outright (e.g. for
air-gapped installs or stricter trust requirements). The install log
already prints a NOTE explaining that lemonade lacks approved-hash
coverage so users know the trust model.
3. linux-rocm runtime overlay patterns extended to cover lemonade's bundled
HIP/ROCm runtime libs (libamdhip64.so*, libhsa-runtime64.so*, libhipblas*,
librocblas*, librocsolver*, librocsparse*, librocrand*, libMIOpen*,
libmagma*). The upstream tarball does not ship these (links against
system /opt/rocm), so the new glob entries are no-op for upstream and
load-bearing for lemonade. Without this, install_from_archives would
drop the bundled runtime libs from the lemonade ZIP and llama-server's
RPATH would fail to load amdhip64 at first inference.
4. _lemonade_release_api_for now URL-encodes llama_tag with quote(safe="").
Defence in depth: a tag containing /, ?, #, or whitespace cannot reshape
the request URL. Tags come from internal resolution today, but this
removes the risk if a future caller passes user-controlled input.
5. Empty browser_download_url skipped explicitly in the resolver. The
release_asset_map helper defaults missing URLs to "". Previously this
would have been passed to download_file("") which raises a less obvious
error than the new clean log + return None.
6. Docstring on _lemonade_release_api_for clarified: lemonade tags match
ggml-org/llama.cpp tags 1:1, NOT unslothai/llama.cpp fork tags. The
earlier P1 review report misread this and re-asked for the tag-drift
fix that the author intentionally reverted in f256cea950.
7. Autouse pytest fixture in test_lemonade_llamacpp_rocm_bins_mock.py
clears _fetch_lemonade_release_cached between tests. Today's tests all
mock the same payload so no pollution surfaces, but the lru_cache
becomes a footgun the moment any future test parametrises return values.
Tests: 28 passed (was 18, added 10 covering URL pinning, opt-out env,
pinned-tag helper, URL encoding, empty URL, runtime patterns).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio/rocm): complete lemonade runtime overlay + honor CUDA_VISIBLE_DEVICES
runtime_patterns_for_choice("linux-rocm") was missing libamd_comgr.so*,
librocm_kpack.so*, and librocm_sysdeps_*.so*, which are direct NEEDED
entries of libamdhip64.so.7 in every lemonade bundle. install_from_archives
did not copy them, so the preflight ldd walk failed on llama-server and every
lemonade attempt fell back to source build. Confirmed on gfx1151 b1272 by
h34v3nzc0dex; manually staging the full bundle with the three missing
patterns restored the install and preserved bench parity.
Also extend _pick_rocm_gfx_target to check CUDA_VISIBLE_DEVICES after
HIP_VISIBLE_DEVICES and ROCR_VISIBLE_DEVICES: AMD's HIP runtime honours
all three with identical semantics, so mixed-arch hosts where users set
only CUDA_VISIBLE_DEVICES were still picking the first gfx token instead
of the user-selected device.
Tests: 30 passed (was 28); added 2 for CUDA_VISIBLE_DEVICES (multi-GPU
pick + -1 opt-out) and extended the runtime-patterns test to assert the
three newly added lib globs are present.
* fix: tighten CDN trust check and fix multi-GPU arch selection
- _is_trusted_github_release_url: require /github-production-release-asset-
path prefix so only real GitHub release CDN URLs are accepted
- _pick_rocm_gfx_target: parse rocminfo Agent N section boundaries to build
a per-physical-GPU arch list; same arch across multiple GPUs no longer
collapses to a single token, so HIP_VISIBLE_DEVICES indexing works correctly
- tests: update CDN test to use realistic path prefix, add rejection test for
arbitrary CDN path, add regression test for same-arch multi-GPU case
* fix: use broad lib*.so* glob for linux-rocm runtime overlay
Replace the explicit lib allowlist in runtime_patterns_for_choice with
lib*.so* for the linux-rocm path.
The lemonade ROCm ZIPs carry a full HIP/ROCm runtime including transitive
deps like libLLVM.so.23.0git and libclang-cpp.so.23.0git (pulled in by
libamd_comgr.so.3). These names change across ROCm releases and were not
in the allowlist, so preflight would see them as unresolved NEEDED entries
and fall back to a source build. The broad glob catches everything in the
bundle now and in future releases without needing to enumerate each library.
* fix: show lemonade binary tag in install summary log line
Store binary_repo and binary_release_tag in UNSLOTH_PREBUILT_INFO.json
so the setup.sh summary can distinguish the source tree (unslothai/llama.cpp)
from the actual binary origin (lemonade-sdk/llamacpp-rocm).
Before: 'installed release: unslothai/llama.cpp@b9334'
After: 'installed release: unslothai/llama.cpp@b9334 + lemonade@b1280'
Upstream installs are unchanged (binary_repo == published_repo).
* fix(merge-compat): align Windows ROCm guard and helper name with strix branch
- elif host.has_rocm instead of if...not to match fix/rocm-strix-halo-unified-memory
- _resolve_exe instead of _resolve_amd_exe with identical body/docstring
Eliminates the two conflict hunks that would otherwise block a clean bot merge
of feature/lemonade-rocm-prebuilts on top of fix/rocm-strix-halo-unified-memory.
* fix: three PR review corrections for lemonade ROCm prebuilt integration
- _lemonade_release_api_for: fix docstring claiming lemonade tags match
ggml-org 1:1 -- lemonade may be several builds behind ggml-org (noted
by oobabooga, confirmed: lemonade b1281 vs ggml-org b9370)
- direct_linux_release_plan: move cpu_choice into else-branch so ROCm-only
hosts never get a CPU fallback in the attempts list -- a failed lemonade
binary now raises PrebuiltFallback and triggers the HIP source build
instead of silently installing a CPU-only binary
- apply_approved_hashes: pass lemonade attempts through without requiring
a manifest entry; lemonade is explicitly documented as relying on
functional validation only, so rejecting it here caused PrebuiltFallback
on the non-simple-policy path before the upstream ROCm/HIP fallback
could be considered
* fix: copy hipblaslt/rocblas library subdirs from lemonade ROCm archives
copy_globs matches filenames only and copies flat, so it cannot
preserve the hipblaslt/library/<gfx>/ and rocblas/library/<gfx>/
Tensile kernel catalog trees that lemonade ROCm zips ship alongside
the .so files. Add runtime_subdirs_for_choice() and call shutil.copytree
for each named subdir after the copy_globs pass in install_from_archives.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Jaeic Lee <jaeiclee@users.noreply.github.com>
