From 7cbf428b0add57303fd98136502991a05dae956e Mon Sep 17 00:00:00 2001 From: Daniel Han Date: Wed, 10 Jun 2026 06:10:03 -0700 Subject: [PATCH] Fix JSON-mode smoke on Linux: bump quant, keep the hard Paris assert (#6138) At UD-IQ2_XXS the temp-0 greedy answer to the capital-of-France probe is hardware dependent: GitHub ubuntu runners deterministically answer city=France while other CPUs answer Paris, because the 2-bit argmax flips with the SIMD kernel path. Seeds do not rescue it: a staging-fork sweep on the affected runners measured 1/5 Paris at temp 0.7 and 1/5 at temp 1.0 (different winning seeds on different hardware), so a retry-across-seeds assert would still flake about a third of runs. The same sweep on UD-Q4_K_XL answered Paris 13/13 across temp 0, 0.7 and 1.0 with 5 seeds each, including 3x deterministic greedy. Bump the job's quant (roughly 570 MiB to 1.1 GiB, cache key already includes the variant) and leave the assertion exactly as it was. --- .github/workflows/studio-inference-smoke.yml | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/.github/workflows/studio-inference-smoke.yml b/.github/workflows/studio-inference-smoke.yml index 3a4c76a2bb..133dd601d1 100644 --- a/.github/workflows/studio-inference-smoke.yml +++ b/.github/workflows/studio-inference-smoke.yml @@ -20,7 +20,7 @@ # enable_tools / enabled_tools, and enable_thinking on/off. # # 3. JSON, images -# Qwen3-VL-2B-Instruct UD-IQ2_XXS (~570 MiB) + mmproj-F16 (~780 MiB). +# Qwen3-VL-2B-Instruct UD-Q4_K_XL (~1.1 GiB) + mmproj-F16 (~780 MiB). # response_format JSON-schema decoding and OpenAI image_url # (data URI) plus Anthropic source/base64 image inputs. # @@ -792,8 +792,14 @@ jobs: timeout-minutes: 30 env: GGUF_REPO: unsloth/Qwen3-VL-2B-Instruct-GGUF - GGUF_VARIANT: UD-IQ2_XXS - GGUF_FILE: Qwen3-VL-2B-Instruct-UD-IQ2_XXS.gguf + # UD-Q4_K_XL, not UD-IQ2_XXS: at 2-bit the temp-0 answer to the JSON + # step's capital-of-France probe flips with the host's SIMD kernels + # (GitHub runners deterministically answered France while other CPUs + # answer Paris; seeds do not rescue it, 1/5 Paris at temp 0.7). The + # Q4 quant answered Paris 13/13 across temps and seeds on the same + # runners, so the hard Paris assertion below stays reliable. + GGUF_VARIANT: UD-Q4_K_XL + GGUF_FILE: Qwen3-VL-2B-Instruct-UD-Q4_K_XL.gguf MMPROJ_FILE: mmproj-F16.gguf STUDIO_PORT: '18890' HF_HOME: ${{ github.workspace }}/hf-cache