48a3a78703* Studio: fail fast on an invalid first training batch Training a base vision-language model (e.g. Qwen/Qwen2-VL-7B or unsloth/Qwen2-VL-7B) on a conversational image dataset crashed on the first step with 'Expected ... Long, Int; but got torch.cuda.FloatTensor (embedding)'. Root cause: the base model's chat template is a flat, media-only template that renders to an empty string for role-based messages, so UnslothVisionDataCollator hands the processor empty text, the processor returns empty input_ids, torch defaults the empty tensor to float32, and the embedding lookup rejects it. Add a preflight that runs one real batch through the trainer's own tokenization and collation right before train(), and stops the run with an actionable message when input_ids is empty or non-integer (pointing to the instruction-tuned variant for the base-model case). Faithful across text, vision and audio-VLM paths, and never blocks a run whose first batch is valid. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Trim comments in the training preflight * Stub unsloth/trl in preflight test so backend CI collection passes --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| assets | ||
| auth | ||
| core | ||
| hub | ||
| loggers | ||
| models | ||
| plugins | ||
| requirements | ||
| routes | ||
| state | ||
| storage | ||
| tests | ||
| utils | ||
| __init__.py | ||
| _platform_compat.py | ||
| cloudflare_tunnel.py | ||
| colab.py | ||
| main.py | ||
| run.py | ||
| startup_banner.py | ||