Fix Mistral DPO/preference training crash on non-xformers platforms (e.g. Intel XPU) (#4889)
* Fix Mistral training crash when xformers is unavailable * Fix/adjust Mistral DPO training crash fix for PR #4889 - Clarify comment in MistralForCausalLM_fast_forward: the DPO embed-masking block runs BEFORE attention_mask is nulled out, and it is the consumer that requires a 2D mask. - Add defensive attention_mask.ndim == 2 guard to the LlamaModel_fast_forward DPO embed-masking block so it self-protects if a 4D mask ever reaches it. --------- Co-authored-by: Daniel Han <danielhanchen@gmail.com>
This commit is contained in:
parent
da2ef6dce6
commit
7aa442289b
2 changed files with 11 additions and 0 deletions
|
|
@ -1030,6 +1030,7 @@ def LlamaModel_fast_forward(
|
|||
if (
|
||||
getattr(self, "_has_no_labels", False) is True
|
||||
and (attention_mask is not None)
|
||||
and attention_mask.ndim == 2
|
||||
and (past_key_values is None)
|
||||
and (not train_embed_tokens)
|
||||
and self.training
|
||||
|
|
|
|||
|
|
@ -188,6 +188,16 @@ def MistralForCausalLM_fast_forward(
|
|||
|
||||
# If attention_mask exists, it will be handled in the attention forward
|
||||
|
||||
elif self.training:
|
||||
# During training, LlamaModel_fast_forward's DPO embed-masking
|
||||
# block requires a 2D attention_mask (it does
|
||||
# inputs_embeds *= attention_mask.unsqueeze(0).transpose(0, 1).transpose(1, 2)).
|
||||
# Afterwards, LlamaModel_fast_forward sets attention_mask=None
|
||||
# before the attention layers anyway, so leaving the 2D mask
|
||||
# untouched here is safe and avoids converting to 4D (which would
|
||||
# crash the DPO block).
|
||||
pass
|
||||
|
||||
else:
|
||||
# Not using xformers - need to create attention masks
|
||||
if (
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue