* Single-pass GGUF export for direct outtypes + parallel multi-quant
save_to_gguf defaulted first_conversion to model_dtype before the block
that picks the optimal base conversion, leaving that block dead since it
landed (#3356). Every default export (fast_quantized -> q8_0) therefore
ran two passes: convert HF -> 16-bit GGUF, then llama-quantize -> q8_0,
writing a 2x-size intermediate that the cleanup step deletes again.
- Route single-output exports whose type convert_hf_to_gguf.py emits
directly (f32/f16/bf16/q8_0) through one conversion pass with no
16-bit intermediate. Measured on Qwen2.5-0.5B-Instruct (8-core CPU):
bytes written 1525 MB -> 531 MB (2.9x less), peak extra disk 994 MB
-> 0, wall time neutral on local NVMe (14.8s vs 15.4s). The dequantized
q8_0 tensors are bit-identical to the two-pass output (max diff 0 over
all 290 tensors, same quant-type table). On disk-capped runtimes
(Kaggle 20 GB, Colab) the removed intermediate is the difference
between an export that fits and one that dies - see the Kaggle error
text this file already carries. imatrix runs keep the two-pass route
since only llama-quantize can apply one; explicit first_conversion is
still honored.
- Run independent llama-quantize passes two at a time when several
quant methods are requested (thread budget split between workers,
outputs byte-identical, order preserved). Measured 1.38x wall-clock
on q4_k_m+q5_k_m+q6_k. Sequential under UNSLOTH_ENABLE_LOGGING=1 to
keep subprocess logs readable; kill switch
UNSLOTH_PARALLEL_GGUF_QUANTS=0. Duplicate methods now quantize once.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Guard parallel GGUF quant on Kaggle and make multi-quant failures atomic
Each llama-quantize pass loads the whole model into RAM, so running two at once on Kaggle can OOM a host that succeeded sequentially; skip the parallel path there. On a failed multi-quant export, stop launching queued passes and remove orphaned quant outputs so a failure leaves no partial GGUFs behind, keeping the 16-bit base for retry. Also accept 0/false/no/off/empty for UNSLOTH_PARALLEL_GGUF_QUANTS so a well-meant 'false' actually disables parallelism, and add tests/saving/test_gguf_single_pass_export.py to the CI saving bucket so the new tests run.
* Preserve pre-existing outputs for canceled quant passes on failure
The parallel cleanup unlinked every requested output name, so a failed rerun could delete a valid model.<METHOD>.gguf left by an earlier successful export for a method whose pass was canceled and never ran this session. Skip canceled futures and only remove outputs from passes that actually executed.
* Gate parallel GGUF quant on available memory and preserve prior outputs
Skip the two-worker path when RAM cannot hold two full-model quantizations at once (and on Colab as well as Kaggle), so a multi-quant export that fit sequentially no longer OOMs. On failure, remove only outputs this run newly created, tracked against a pre-launch snapshot, so a rerun into an existing _gguf directory never deletes a valid artifact from an earlier export.
---------
Co-authored-by: djs <dschroers2@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
8cfd1a2173