ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Studio: sort GPU probe result + honor explicitly empty ROCm masks

Browse source 
Two reviewer-flagged correctness nits on top of eff55fb8.

1) Gemini medium: the torch fallback returned an unsorted list when the
   visibility mask was non-sequential (e.g. CUDA_VISIBLE_DEVICES=5,2,9),
   diverging from the docstring guarantee and the nvidia-smi path. Now
   sorted by physical id.

2) Codex P2: an explicitly empty HIP_VISIBLE_DEVICES="" should mean
   "no GPUs" per the codebase convention in
   utils/hardware/hardware.py::_get_parent_visible_gpu_spec. The previous
   `or` chain treated empty string as falsy and silently fell through to
   ROCR / CUDA, producing wrong physical IDs. Switch to `is not None`
   checks to match.

Verified via sim_5172_rocm_precedence.py (9/9 cases pass) including the
two new R8 (sort) and R9 (empty-HIP honored) cases.
This commit is contained in:
Daniel Han 2026-04-24 16:07:38 +00:00
commit 6b1cccd637
1 changed files with 16 additions and 5 deletions
Download patch file Download diff file Expand all files Collapse all files

21
studio/backend/core/inference/llama_cpp.py
View file

@ -618,15 +618,25 @@ class LlamaCppBackend:
# with ``CUDA_VISIBLE_DEVICES=2,3`` would get rewritten to
# ``CUDA_VISIBLE_DEVICES=0,1`` and target the wrong GPUs.
physical_ids: Optional[list[int]] = None
# Match the codebase convention in
# ``utils/hardware/hardware.py::_get_parent_visible_gpu_spec``:
# treat an explicitly empty mask (``HIP_VISIBLE_DEVICES=""``)
# as "set to no GPUs" rather than falling through to the next
# var. ``or`` would coerce empty string to falsy and silently
# promote the wrong source.
if getattr(torch.version, "hip", None) is not None:
hip_v = os.environ.get("HIP_VISIBLE_DEVICES")
rocr_v = os.environ.get("ROCR_VISIBLE_DEVICES")
cvd = (
os.environ.get("HIP_VISIBLE_DEVICES")
or os.environ.get("ROCR_VISIBLE_DEVICES")
or os.environ.get("CUDA_VISIBLE_DEVICES")
hip_v
if hip_v is not None
else rocr_v
if rocr_v is not None
else os.environ.get("CUDA_VISIBLE_DEVICES")
)
else:
cvd = os.environ.get("CUDA_VISIBLE_DEVICES")
if cvd and cvd.strip():
if cvd is not None and cvd.strip():
try:
physical_ids = [int(x.strip()) for x in cvd.split(",") if x.strip()]
except ValueError:
@ -640,7 +650,8 @@ class LlamaCppBackend:
else ordinal
)
gpus.append((idx, free_bytes // (1024 * 1024)))
return gpus
# Match the nvidia-smi path's docstring guarantee of sorted-by-id.
return sorted(gpus, key = lambda g: g[0])
except Exception as e:
logger.debug(f"torch GPU probe failed: {e}")
return []