48a3a78703* Studio: fail fast on an invalid first training batch Training a base vision-language model (e.g. Qwen/Qwen2-VL-7B or unsloth/Qwen2-VL-7B) on a conversational image dataset crashed on the first step with 'Expected ... Long, Int; but got torch.cuda.FloatTensor (embedding)'. Root cause: the base model's chat template is a flat, media-only template that renders to an empty string for role-based messages, so UnslothVisionDataCollator hands the processor empty text, the processor returns empty input_ids, torch defaults the empty tensor to float32, and the embedding lookup rejects it. Add a preflight that runs one real batch through the trainer's own tokenization and collation right before train(), and stops the run with an actionable message when input_ids is empty or non-integer (pointing to the instruction-tuned variant for the base-model case). Faithful across text, vision and audio-VLM paths, and never blocks a run whose first batch is valid. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Trim comments in the training preflight * Stub unsloth/trl in preflight test so backend CI collection passes --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> |
||
|---|---|---|
| .. | ||
| backend | ||
| frontend | ||
| src-tauri | ||
| __init__.py | ||
| install_llama_prebuilt.py | ||
| install_python_stack.py | ||
| LICENSE.AGPL-3.0 | ||
| package-lock.json | ||
| package.json | ||
| setup.bat | ||
| setup.ps1 | ||
| setup.sh | ||
| Unsloth_Studio_Colab.ipynb | ||