The start-route preflight caught the bf16-GPU and int8-torchao requirements but not the dense precisions' CUDA requirement: on a GPU-less host bf16_unsupported_reason exempts CPU-only, so a bf16/fp8 (or int8-with-torchao) DiT request passed the preflight, evicted resident workloads, then raised only in the trainer child. Add the dense-mode CUDA gate mirroring _resolve_base_precision so the doomed run is rejected up front. Also pin bf16_unsupported_reason in the two positive-path family-info tests so they are deterministic across GPU types (a non-bf16 CUDA box would otherwise empty every DiT family's advertised modes). |
||
|---|---|---|
| .. | ||
| __init__.py | ||
| diffusion_dit_trainer.py | ||
| diffusion_lora_trainer.py | ||
| diffusion_train_common.py | ||
| diffusion_training_service.py | ||
| resume.py | ||
| s3_dataset.py | ||
| trainer.py | ||
| training.py | ||
| worker.py | ||