ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Handle canonical inputs_embeds kwarg in unsloth_base_fast_generate (#3082) (#6015)

Browse source 
The kwarg-dispatch in unsloth_base_fast_generate recognized input_ids,
input, input_features and the misspelled input_embeds, but not HF's
canonical inputs_embeds. So generate(inputs_embeds=...) fell through to the
'first kwarg' fallback, which picks whatever kwarg happens to come first
(e.g. attention_mask) and uses it as input_ids -- giving the wrong tensor /
batch size, or the KeyError reported in #3082 on older versions.

Add an explicit inputs_embeds branch so embedding inputs (e.g. multimodal
audio+text) are routed correctly regardless of kwarg order.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Etherll <61019402+Etherll@users.noreply.github.com>
This commit is contained in:
Prathamesh Jadhav 2026-06-11 18:00:54 +05:30 • committed by GitHub
commit ab9689c034
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 3 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

3
unsloth/models/vision.py
View file

@ -283,6 +283,9 @@ def unsloth_base_fast_generate(self, *args, **kwargs):
input_ids = kwargs["input"]
elif "input_features" in kwargs:
input_ids = kwargs["input_features"]
elif "inputs_embeds" in kwargs:
# canonical HF name for embedding inputs (e.g. multimodal generate)
input_ids = kwargs["inputs_embeds"]
elif "input_embeds" in kwargs:
input_ids = kwargs["input_embeds"]
elif "inputs" in kwargs: