ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Merge pull request #2466 from mmathew23/fix_pop_token_type_ids

Browse source 
the pixtral vision notebook fails during inference
This commit is contained in:
Michael Han 2025-05-09 14:57:31 -07:00 • committed by GitHub
commit e3f6c5eff4
1 changed files with 5 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

5
unsloth/models/vision.py
View file

@ -112,6 +112,11 @@ def unsloth_base_fast_generate(
arch = self.config.architectures[0]
# Remove token_type_ids - WRONG for Gemma 3 since bidirectional attention
if hasattr(self, "generate") and hasattr(self, "forward"):
# did not combine with below since self might not have model
keys = inspect.signature(self.forward).parameters.keys()
if "token_type_ids" not in keys:
kwargs.pop("token_type_ids", None)
# kwargs.pop("token_type_ids", None)
# VLMs do not allow logits_to_keep