ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

add UNSLOTH_ALLOW_CPU=1 path for CPU-only CI (#5429)

Browse source 
Lets `import unsloth.trainer` succeed on hosts without a CUDA/XPU/HIP
accelerator (typical of zoo's source-inspection test matrix). The env
var is read exactly once per process via @functools.cache on
`get_device_type()`, so production hosts pay no runtime cost.

Three edits beyond the device_type fallback:

* `_gpu_init.py:212/247` -- the bf16 + libcuda/bnb setup blocks call
  `torch.cuda.get_device_capability()` and `libcuda_dirs()`/`bnb.functional.lib.*`
  unconditionally when DEVICE_TYPE == "cuda". Guard with
  `and torch.cuda.is_available()` so the new CPU-CI sentinel doesn't
  fault those.
* `_gpu_init.py:353` -- gate `_patch_trl_trainer()` (the
  `_backwards_compatible_trainer.__init__` wrapper). Under
  UNSLOTH_ALLOW_CPU we want pristine upstream TRL classes for
  downstream `inspect.getsource(SFTTrainer)` drift detectors.
* `models/_utils.py:1196` -- same `and torch.cuda.is_available()` guard
  for `get_device_capability()` at import time.
* `models/rl.py:PatchFastRL` -- early-return under UNSLOTH_ALLOW_CPU=1
  so the heavier `patch_trl_rl_trainers()` (which replaces
  `trl.SFTTrainer` with the compiled `UnslothSFTTrainer` class)
  doesn't fire either. Without this gate the drift detectors that
  do `inspect.getsource(SFTTrainer)` see the wrapper source and
  spurious fail.

Local sanity: `UNSLOTH_ALLOW_CPU=1 python -c "import unsloth.trainer"`
succeeds on a CPU-only venv, `trl.SFTTrainer.__init__.__qualname__`
stays `SFTTrainer.__init__` (not `UnslothSFTTrainer.__init__`), and
`inspect.getsource(SFTTrainer)` still contains `self._signature_columns`.
Without the env var on a CUDA host, TRL is still patched normally
(verified `UnslothSFTTrainer.__init__`).
This commit is contained in:
Daniel Han 2026-05-14 20:27:14 -07:00 • committed by GitHub
commit cb15a7a5b6
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
4 changed files with 27 additions and 5 deletions
Download patch file Download diff file Expand all files Collapse all files

19
unsloth/_gpu_init.py
View file

@ -209,7 +209,7 @@ del fix_peft_transformers_weight_conversion_import
del patch_peft_weight_converter_compatibility
# Torch 2.4 has including_emulation
if DEVICE_TYPE == "cuda":
if DEVICE_TYPE == "cuda" and torch.cuda.is_available():
major_version, minor_version = torch.cuda.get_device_capability()
SUPPORTS_BFLOAT16 = major_version >= 8
@ -233,12 +233,18 @@ elif DEVICE_TYPE == "xpu":
# torch.xpu.is_bf16_supported() does not have including_emulation
# set SUPPORTS_BFLOAT16 as torch.xpu.is_bf16_supported()
SUPPORTS_BFLOAT16 = torch.xpu.is_bf16_supported()
else:
# CPU-only CI under UNSLOTH_ALLOW_CPU=1. We can't probe device
# capability, so assume no bf16 -- training won't run on this host
# anyway, this branch only exists to let `import unsloth.trainer`
# succeed for source-inspection tests.
SUPPORTS_BFLOAT16 = False
# For Gradio HF Spaces?
# if "SPACE_AUTHOR_NAME" not in os.environ and "SPACE_REPO_NAME" not in os.environ:
import triton
if DEVICE_TYPE == "cuda":
if DEVICE_TYPE == "cuda" and torch.cuda.is_available():
libcuda_dirs = lambda: None
if Version(triton.__version__) >= Version("3.0.0"):
try:
@ -349,5 +355,10 @@ from unsloth_zoo.rl_environments import (
launch_openenv,
)
# Patch TRL trainers for backwards compatibility
_patch_trl_trainer()
# Patch TRL trainers for backwards compatibility.
# Skipped under UNSLOTH_ALLOW_CPU=1 (CPU-only CI) because rebinding
# trl.SFTTrainer.__init__ to a generic wrapper changes
# inspect.getsource(SFTTrainer.__init__) and corrupts downstream
# drift detectors that anchor on the pristine upstream source.
if os.environ.get("UNSLOTH_ALLOW_CPU", "0") != "1":
_patch_trl_trainer()