ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Fix gemini round 3: include has_rocm in validate_server fallback path

Browse source 
When validate_server is called without an explicit install_kind (older
call sites that have not been updated), the fallback was only enabling
--n-gpu-layers for NVIDIA and macOS arm64 hosts. AMD ROCm Linux hosts
fell through to the CPU validation path even though the prebuilt being
exercised was a HIP binary.

Add host.has_rocm to the fallback expression so the GPU offload flag is
applied consistently with the install_kind=='linux-rocm' / 'windows-hip'
branches above.
This commit is contained in:
Daniel Han 2026-04-08 11:59:34 +00:00
commit c12e8b7052
1 changed files with 8 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

10
studio/install_llama_prebuilt.py
View file

@ -4380,8 +4380,14 @@ def validate_server(
if install_kind is not None:
_enable_gpu_layers = install_kind in _gpu_kinds
else:
_enable_gpu_layers = host.has_usable_nvidia or (
host.is_macos and host.is_arm64
# Older call sites that don't pass install_kind: keep ROCm
# hosts in the GPU-validation path so an AMD-only Linux host
# is exercised against the actual hardware rather than the
# CPU fallback. NVIDIA and macOS-arm64 are already covered.
_enable_gpu_layers = (
host.has_usable_nvidia
or host.has_rocm
or (host.is_macos and host.is_arm64)
)
if _enable_gpu_layers:
command.extend(["--n-gpu-layers", "1"])