ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Studio: fix misleading "increase max_seq_length" message for train-on-completions (#6664)

Browse source 
The post-filter safety net for 'Train on completions' fires when
train_on_responses_only() masks every token in too many rows. Its trigger is
a row-drop ratio, not a token-length check, but the message hardcoded
"max_seq_length is too short, try increasing (e.g. 8192)" -- advice that
fires identically at any max_seq_length and can recommend a value below the
user's current setting (telling someone already at 16384 to use 8192).

The dominant real cause is that the model's response template is not found in
the formatted samples: the dataset is already formatted, or its structure
doesn't match the model's chat template, so every token gets masked and the
rows are dropped. Reword the error (and the comment above it) to lead with
that cause and the actionable fix (turn off 'Train on completions'), and
mention max_seq_length only as a secondary possibility without a hardcoded
recommendation.
This commit is contained in:
Daniel Han 2026-06-25 03:30:12 -07:00 • committed by GitHub
commit e1698e05c7
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 18 additions and 10 deletions
Download patch file Download diff file Expand all files Collapse all files

28
studio/backend/core/training/trainer.py
View file

@ -3543,9 +3543,12 @@ class UnslothTrainer:
# ── Safety net: check if all samples were filtered out ──
# train_on_responses_only masks non-response tokens with -100;
# if max_seq_length is too short the response is truncated away,
# every sample becomes all -100, and Unsloth drops them, leaving
# 0 usable samples. Skip this len()-based check for streaming.
# a row becomes all -100 (and Unsloth drops it) when the response
# template is not found in the formatted text. That is usually a
# dataset/template mismatch (already-formatted data, or 'Train on
# completions' applied to data that doesn't match the model's chat
# template), and only sometimes max_seq_length truncating the
# response away. Skip this len()-based check for streaming.
if detect_streaming_dataset(self.trainer.train_dataset):
logger.info("Skipping post-filter length check for streaming dataset\n")
else:
@ -3560,13 +3563,18 @@ class UnslothTrainer:
if filtered_len == 0 or drop_pct > 30:
max_seq = training_args.get("max_seq_length", 2048)
error_msg = (
f"{dropped}/{original_len} samples ({drop_pct}%) "
f"were dropped after applying 'train on responses "
f"only' — only {filtered_len} remain. This usually "
f"means max_seq_length ({max_seq}) is too short "
f"and the response portion is being truncated "
f"away. Try increasing max_seq_length (e.g. 8192) "
f"or disabling 'Train on completions'."
f"{dropped}/{original_len} samples ({drop_pct}%) were "
f"dropped after applying 'Train on completions': after "
f"masking, those rows had no trainable response tokens "
f"left. The usual cause is that this model's response "
f"template was not found in the formatted samples, so "
f"every token was masked out. That typically means the "
f"dataset is already formatted, or its structure does "
f"not match the model's chat template, so 'Train on "
f"completions' should be turned off for this dataset. "
f"Less commonly, a max_seq_length ({max_seq}) shorter "
f"than the prompt can truncate the response away; only "
f"raise it if your samples are actually longer than that."
)
logger.error(error_msg)
self._update_progress(error = error_msg, is_training = False)