8ccb461570
The smallest-fitting-variant fallback now groups split GGUF shards by their variant prefix and sums all shard sizes per variant. For example, DeepSeek-V3.2 UD-Q4_K_XL has 9 shards totaling 379.8 GB. The previous code treated each shard as a separate "variant" and would have incorrectly selected a single 50 GB shard as fitting, ignoring the other 8 shards needed. Tested with unsloth/DeepSeek-V3.2-GGUF (237 GGUF files, 27 variants from 150 GB to 1.25 TB). Correctly groups and sorts all variants by total size. |
||
|---|---|---|
| .. | ||
| __init__.py | ||
| audio_codecs.py | ||
| inference.py | ||
| llama_cpp.py | ||
| orchestrator.py | ||
| worker.py | ||