Import bitsandbytes before the hardware spoof rewrites torch (#7471)
tests/studio/install/test_rocm_rdna_routing.py errors out on CPU-only CI,
taking Repo tests (CPU) with it, all 12 cases with
OSError: libhipblas.so.2: cannot open shared object file
AttributeError: module 'torch._C' has no attribute '_cuda_getCurrentRawStream'
The spoof presents torch as a Radeon card, which flips
torch.cuda.is_available() to True and sets torch.version.hip. bitsandbytes
gates its backend on exactly that:
if torch.cuda.is_available():
from .backends.cuda import ops as cuda_ops
so a bitsandbytes imported afterwards walks into the CUDA/ROCm path against a
CPU-only wheel and dies reading torch._C._cuda_getCurrentRawStream. It reaches
the test because unsloth_zoo imports it eagerly, guarded by except ImportError,
which neither OSError nor AttributeError satisfies.
Import it in the spoof instead, while is_available() is still False, so the CPU
path is cached in sys.modules before torch is rewritten. Placed in the shared
apply(), ahead of the first mutation and inside the idempotence guard, so the
ROCm spoof that layers on top gets it too.
Co-authored-by: danielhanchen <unslothai@gmail.com>
This commit is contained in:
parent
dc24bba43e
commit
0c1c9f71db
1 changed files with 14 additions and 0 deletions
|
|
@ -22,6 +22,20 @@ def apply() -> None:
|
|||
if getattr(torch.cuda, "_unsloth_consolidated_spoof", False):
|
||||
return
|
||||
|
||||
# Settle bitsandbytes against the real torch first. Its __init__ does
|
||||
# `if torch.cuda.is_available(): from .backends.cuda import ops`, and that
|
||||
# module reads torch._C._cuda_getCurrentRawStream at import. On a CPU-only
|
||||
# wheel that attribute is absent, so a bitsandbytes imported AFTER this
|
||||
# spoof raises AttributeError (or OSError hunting libhipblas for the ROCm
|
||||
# spoof) rather than ImportError, which slips past the `except ImportError`
|
||||
# guards its importers use. Importing it here, while is_available() is
|
||||
# still False, caches the CPU path in sys.modules for everything that
|
||||
# follows.
|
||||
try:
|
||||
import bitsandbytes # noqa: F401
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# Device probes (cheap, value-returning)
|
||||
torch.cuda.is_available = lambda: True
|
||||
torch.cuda.device_count = lambda: 1
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue