studio: reorder GGUF preference -- UD-Q4_K_XL first, all UD above standard

Reorder _GGUF_QUANT_PREFERENCE so all UD (Unsloth Dynamic) variants
come before standard quants. UD-Q4_K_XL is the default (best
size/quality tradeoff), followed by other UD quants in decreasing
preference order.

For repos without UD variants (e.g., bartowski), falls through to
standard quants starting with Q4_K_M.

Verified with:
  - unsloth/Qwen3.5-35B-A3B-GGUF -> UD-Q4_K_XL
  - bartowski/Qwen_Qwen3.5-35B-A3B-GGUF -> Q4_K_M
  - unsloth/DeepSeek-V3.2-GGUF -> UD-Q4_K_XL (9 shards)
  - unsloth/Llama-3.2-1B-Instruct-GGUF -> UD-Q4_K_XL
This commit is contained in:
Daniel Han 2026-03-14 09:03:49 +00:00
commit ab6fdccfb5

View file

@ -801,8 +801,29 @@ def detect_gguf_model(path: str) -> Optional[str]:
# Preferred GGUF quantization levels, in descending priority.
# Q4_K_M is a good default: small, fast, acceptable quality.
# UD (Unsloth Dynamic) variants are always preferred over standard quants
# because they provide better quality per bit. If the repo has no UD variants
# (e.g., bartowski repos), the standard quants are used as fallback.
# Ordered by best size/quality tradeoff, not raw quality.
_GGUF_QUANT_PREFERENCE = [
# UD variants (best quality per bit) -- Q4 is the sweet spot
"UD-Q4_K_XL",
"UD-Q4_K_L",
"UD-Q5_K_XL",
"UD-Q3_K_XL",
"UD-Q6_K_XL",
"UD-Q6_K_S",
"UD-Q8_K_XL",
"UD-Q2_K_XL",
"UD-IQ4_NL",
"UD-IQ4_XS",
"UD-IQ3_S",
"UD-IQ3_XXS",
"UD-IQ2_M",
"UD-IQ2_XXS",
"UD-IQ1_M",
"UD-IQ1_S",
# Standard quants (fallback for non-Unsloth repos)
"Q4_K_M",
"Q4_K_S",
"Q5_K_M",
@ -811,11 +832,15 @@ _GGUF_QUANT_PREFERENCE = [
"Q8_0",
"Q3_K_M",
"Q3_K_L",
"Q3_K_S",
"Q2_K",
"UD-Q2_K_XL",
"UD-IQ2_M",
"UD-IQ1_M",
"UD-IQ1_S",
"Q2_K_L",
"IQ4_NL",
"IQ4_XS",
"IQ3_M",
"IQ3_XXS",
"IQ2_M",
"IQ1_M",
"F16",
"BF16",
"F32",