Some users report that load_in_4bit=True is silently ignored on certain
checkpoints and the model is loaded in full precision. This adds a post-load
guardrail in FastBaseModel.from_pretrained and FastLlamaModel.from_pretrained
that detects two failure modes and emits a clear warning instead of letting
the user discover the issue via a confusing VRAM blow-up.
1. Total bypass: load_in_4bit=True requested, zero bitsandbytes Linear4bit /
Linear8bitLt modules in the loaded model. Usually a transformers / bnb
version mismatch or a backend-incompatible device_map.
2. Partial bypass: bnb quantized nn.Linear but a large fraction of weight
bytes live in non-nn.Linear Parameters that are not in the bnb skip list.
This catches the Gemma-4 MoE class where Gemma4TextExperts stores experts
as fused 3D nn.Parameter tensors for torch._grouped_mm; bnb's
replace_with_bnb_linear only swaps nn.Linear instances, so the fused
expert weights stay in BF16 and dominate the VRAM footprint. The
warning names the worst offenders so the user can correlate.
warnings.warn (not raise) so CPU / MLX / AMD-without-bnb backends that
legitimately have no Linear4bit modules are not broken.
Tests in tests/test_issue_5344_guardrail.py cover both branches plus the
full-finetuning, no-quant, and skip-list cases.
Refs #5344
e2a82b5a10