ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

the pixtral vision notebook fails during inferenc with unused kwargs token_type_ids. This fixes the error

Browse source
This commit is contained in:
Mathew Mathew 2025-05-02 21:40:15 -05:00
commit bbe5e2a221
1 changed files with 5 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

5
unsloth/models/vision.py
View file

@ -112,6 +112,11 @@ def unsloth_base_fast_generate(
arch = self.config.architectures[0]
# Remove token_type_ids - WRONG for Gemma 3 since bidirectional attention
if hasattr(self, "generate") and hasattr(self, "forward"):
# did not combine with below since self might not have model
keys = inspect.signature(self.forward).parameters.keys()
if "token_type_ids" not in keys:
kwargs.pop("token_type_ids", None)
# kwargs.pop("token_type_ids", None)
# VLMs do not allow logits_to_keep