ci(mac): switch Mac GGUF jobs to UD-Q4_K_XL + bump UI turn timeout

The IQ3_XXS quants the Linux smoke uses are pathological at
temperature=0 on Apple Silicon Metal:

  - Qwen3.5-2B IQ3_XXS emits 'The The The...' for tool-call prompts
    (no tool_calls in the response, hits max_tokens).
  - gemma-4-E2B IQ3_XXS emits '<unused5><unused5>...' for any prompt
    (model degenerates to padding tokens).

Both are inference-path-correct but quant-degenerate; the Linux CPU
backend hides the issue. Bump both to UD-Q4_K_XL, the smallest
published variant that generates real text + well-formed tool calls
on M1. Inference time goes up modestly (CI is cache-warm so download
cost is one-shot per HF release).

Also bump STUDIO_UI_TURN_TIMEOUT_MS to 540s for the Mac UI job:
the macos-14 free runner is 3-5x slower than ubuntu-latest at
gemma-3-270m CPU inference, and the existing 180s ceiling crowded
turn 4 ('say tree').
This commit is contained in:
Daniel Han 2026-05-07 05:58:58 +00:00
commit 4c70a9ed72
2 changed files with 20 additions and 4 deletions

View file

@ -292,8 +292,13 @@ jobs:
timeout-minutes: 25
env:
GGUF_REPO: unsloth/Qwen3.5-2B-GGUF
GGUF_VARIANT: UD-IQ3_XXS
GGUF_FILE: Qwen3.5-2B-UD-IQ3_XXS.gguf
# The Linux smoke uses UD-IQ3_XXS, but on Mac Metal that quant
# collapses for tool-call grammar at temperature=0 (model emits
# repeated padding tokens until max_tokens). UD-Q4_K_XL is the
# smallest published variant that produces well-formed tool
# calls + non-pathological text on M1.
GGUF_VARIANT: UD-Q4_K_XL
GGUF_FILE: Qwen3.5-2B-UD-Q4_K_XL.gguf
STUDIO_PORT: '18898'
HF_HOME: ${{ github.workspace }}/hf-cache
steps:
@ -607,8 +612,13 @@ jobs:
timeout-minutes: 30
env:
GGUF_REPO: unsloth/gemma-4-E2B-it-GGUF
GGUF_VARIANT: UD-IQ3_XXS
GGUF_FILE: gemma-4-E2B-it-UD-IQ3_XXS.gguf
# Linux smoke uses UD-IQ3_XXS, but on Mac Metal that gemma-4
# quant emits sentinel tokens (<unused5>) for any prompt at
# temperature=0 -- inference path is fine, the quant itself is
# broken on Metal. UD-Q4_K_XL is the smallest published variant
# that generates real text on M1.
GGUF_VARIANT: UD-Q4_K_XL
GGUF_FILE: gemma-4-E2B-it-UD-Q4_K_XL.gguf
MMPROJ_FILE: mmproj-F16.gguf
STUDIO_PORT: '18899'
HF_HOME: ${{ github.workspace }}/hf-cache

View file

@ -130,6 +130,10 @@ jobs:
BASE_URL: http://127.0.0.1:18896
PW_ART_DIR: logs/playwright
STUDIO_UI_STRICT: '1'
# macos-14 free runner is 3 vCPU / 7 GB / no Metal-accel
# available to llama.cpp from CI; gemma-3-270m turn latency
# has been observed to crowd the 180s default. Triple it.
STUDIO_UI_TURN_TIMEOUT_MS: '540000'
run: |
mkdir -p logs/playwright
python tests/studio/playwright_chat_ui.py
@ -174,6 +178,8 @@ jobs:
STUDIO_NEW_PW: ${{ env.STUDIO_EXTRA_NEW_PW }}
PW_ART_DIR: logs/playwright_extra
STUDIO_UI_STRICT: '1'
# See "Drive the chat UI" step.
STUDIO_UI_TURN_TIMEOUT_MS: '540000'
GGUF_REPO: ${{ env.GGUF_REPO }}
GGUF_VARIANT: ${{ env.GGUF_VARIANT }}
run: |