ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Detach logits before returning from function (#3554)

Browse source
This commit is contained in:
pluesclues 2025-11-04 10:29:27 -05:00 • committed by GitHub
commit 91db850488
1 changed files with 1 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

2
unsloth/models/rl_replacements.py
View file

@ -510,7 +510,7 @@ def grpo_trainer__get_per_token_logps_and_entropies(function_name, function):
os.environ["UNSLOTH_RETURN_HIDDEN_STATES"] = "0"
# logits = logits[:, :-1, :] # (B, L-1, V), exclude the last logit: it corresponds to the next token pred
return logits, entropies # logps, entropies
return logits.detach(), entropies # logps, entropies
# input_ids = input_ids[:, -logits_to_keep:]
# For transformers<=4.48, logits_to_keep argument isn't supported, so here we drop logits ourselves.
# See https://github.com/huggingface/trl/issues/2770