Use 'is not None' and log debug on torch.version.hip probe failures

Two small follow-ups to the apply_gpu_ids ROCm fallback:

1. Match detect_hardware()'s 'getattr(torch.version, "hip", None) is not None'
   form so the entire codebase has one canonical 'this torch was built with
   HIP' check. On every shipping torch wheel hip is either None or a non-empty
   version string, so the new form agrees with the old bool() form on every
   real install.

2. Log the probe failure at debug level instead of swallowing it silently.
   The broad 'except Exception' is intentional (we never want apply_gpu_ids
   to crash a worker over a probe), but the silent pass made it impossible
   to tell whether the fallback was firing or being skipped.
This commit is contained in:
Daniel Han 2026-05-06 12:29:20 +00:00
commit cb0edfc56c

View file

@ -1400,12 +1400,24 @@ def apply_gpu_ids(gpu_ids) -> None:
)
_is_rocm = IS_ROCM or _inherits_rocm_visibility
if not _is_rocm:
# Use ``is not None`` here to match the detect_hardware() check at
# module top -- torch ships HIP version as a non-empty string on
# ROCm builds and None on CUDA builds, so the two forms agree on
# every shipping torch wheel; the ``is not None`` form is the one
# the rest of the codebase reads for "this torch was built with
# HIP". Keep the broad ``except`` as a safety net (we never want
# apply_gpu_ids to crash a worker over a probe failure) but log at
# debug level so the skip is observable when needed.
try:
import torch as _torch
_is_rocm = bool(getattr(_torch.version, "hip", None))
except Exception:
pass
_is_rocm = getattr(_torch.version, "hip", None) is not None
except Exception as e:
logger.debug(
"apply_gpu_ids: torch.version.hip probe skipped (%s: %s)",
type(e).__name__,
e,
)
if _is_rocm:
os.environ["HIP_VISIBLE_DEVICES"] = value
os.environ["ROCR_VISIBLE_DEVICES"] = value