ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Call patch_compiling_bitsandbytes on the FastLanguageModel path (#6144)

Browse source 
* Call patch_compiling_bitsandbytes on the FastLanguageModel dispatch path

* Guard the FastLanguageModel patch call so an old unsloth_zoo cannot crash loads
This commit is contained in:
Daniel Han 2026-06-10 08:12:54 -07:00 • committed by GitHub
commit 8129994061
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 8 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

8
unsloth/models/loader.py
View file

@ -722,6 +722,14 @@ class FastLanguageModel(FastLlamaModel):
load_in_4bit_kwargs = False
load_in_8bit_kwargs = False
# Mirror FastModel: bitsandbytes < 0.46.0 needs dynamo disabled.
# Best effort: never crash the load (old unsloth_zoo without the
# zoo #710 fix raises NameError here on Python 3.13).
try:
patch_compiling_bitsandbytes()
except Exception as e:
print(f"Unsloth: Could not patch bitsandbytes for torch.compile - {e}")
model, tokenizer = dispatch_model.from_pretrained(
model_name = model_name,
max_seq_length = max_seq_length,