Commit graph

1 commit

Author SHA1 Message Date
Daniel Han
48a3a78703
Studio: fail fast on an invalid first training batch (base VLM empty chat template) (#6358)
* Studio: fail fast on an invalid first training batch

Training a base vision-language model (e.g. Qwen/Qwen2-VL-7B or
unsloth/Qwen2-VL-7B) on a conversational image dataset crashed on the first
step with 'Expected ... Long, Int; but got torch.cuda.FloatTensor (embedding)'.
Root cause: the base model's chat template is a flat, media-only template that
renders to an empty string for role-based messages, so UnslothVisionDataCollator
hands the processor empty text, the processor returns empty input_ids, torch
defaults the empty tensor to float32, and the embedding lookup rejects it.

Add a preflight that runs one real batch through the trainer's own tokenization
and collation right before train(), and stops the run with an actionable message
when input_ids is empty or non-integer (pointing to the instruction-tuned variant
for the base-model case). Faithful across text, vision and audio-VLM paths, and
never blocks a run whose first batch is valid.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Trim comments in the training preflight

* Stub unsloth/trl in preflight test so backend CI collection passes

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-06-18 06:36:33 -07:00