ci(mac): retry Playwright JSON crash + GGUF detect retry + MLX is_gguf guard

Two distinct Mac UI Chat failures captured in PR 5312's CI:

1. /api/inference/load 500 with FileNotFoundError on config.json for
   unsloth/gemma-3-270m-it-GGUF (a GGUF-only repo). Run 25487410091.
   Root cause: detect_gguf_model_remote in
   studio/backend/utils/models/model_config.py had a single
   hf_model_info call with no retry. On a transient HF Hub flake
   it returned None silently, the route at routes/inference.py:592
   treated the repo as non-GGUF, and dispatched to the MLX
   orchestrator. The orchestrator's _build_model_config re-ran
   from_identifier in the subprocess (this time succeeding,
   logging "Detected remote GGUF") but then handed an is_gguf=True
   ModelConfig to MLXInferenceBackend.load_model, which ignored
   is_gguf and called FastMLXModel.from_pretrained →
   mlx_lm.utils.load_model → opened a non-existent config.json on
   the GGUF-only repo. Fix:
     a) detect_gguf_model_remote retries up to 3 times with 1/2/4s
        backoff, bypassing retry on RepositoryNotFoundError /
        GatedRepoError / RevisionNotFoundError / EntryNotFoundError
        (those are permanent).
     b) MLXInferenceBackend.load_model now raises a clear
        RuntimeError if config.is_gguf=True, instead of letting
        mlx_lm surface a cryptic 'config.json does not exist'.

2. Playwright pipeTransport.js 'Unexpected end of JSON input' on
   macos-14 free runners. Runs 25489049059 + 25489429306. Chromium
   browser process dies mid-test → driver Node process can't parse
   the truncated JSON-RPC line and exits. Hits ~50% of runs (well
   above acceptable flake). Fix: retry the chat-UI step up to 3
   times, FULLY resetting Studio (kill, reset-password, reboot,
   /api/health wait, re-export STUDIO_OLD/NEW/NEW2_PW) between
   attempts so the change-password flow finds a fresh bootstrap on
   each retry. Same retry shape on the extra-UI step. Real
   assertion / timeout failures don't match the JSON-input pattern
   so they bypass retry and surface immediately. Updated the
   install-step comment to drop the now-incorrect '1.55-1.57 ship a
   Node 22 driver' claim — all 1.55-1.58 Mac drivers are Node 24,
   the racy crash is in pipeTransport itself.
This commit is contained in:
Daniel Han 2026-05-07 10:17:21 +00:00
commit d35bf6ac8e
3 changed files with 156 additions and 15 deletions

View file

@ -78,6 +78,28 @@ class MLXInferenceBackend:
model_name = config.identifier if hasattr(config, "identifier") else str(config)
is_vision = getattr(config, "is_vision", False)
# GGUF guard. GGUF models are served via llama-server in the
# parent process, NOT via mlx-lm in this MLX subprocess. The
# route at studio/backend/routes/inference.py:592 (`if config.
# is_gguf:`) is responsible for sending GGUF traffic to the
# llama-server backend before reaching the MLX orchestrator.
# If we end up here with is_gguf=True, the route's
# `detect_gguf_model_remote` returned None on its first call
# (transient HF Hub flake) but the subprocess re-detection
# succeeded. The subprocess cannot reach into the parent's
# llama-server, so all we can do is raise loudly so the caller
# gets a clear error instead of a cryptic
# "config.json does not exist" from mlx_lm.utils.load_model.
if getattr(config, "is_gguf", False):
raise RuntimeError(
f"MLXInferenceBackend cannot load GGUF model '{model_name}': "
f"GGUF models must be served by llama-server in the parent "
f"process. The /api/inference/load route should have "
f"detected this repo as GGUF before dispatching to the MLX "
f"orchestrator -- this fallback indicates a transient HF "
f"Hub failure during initial detection. Retry the request."
)
if hf_token:
import os

View file

@ -1327,16 +1327,45 @@ def detect_gguf_model_remote(
Check if a HuggingFace repo contains GGUF files.
Returns the filename of the best GGUF file in the repo, or None.
"""
try:
from huggingface_hub import model_info as hf_model_info
info = hf_model_info(repo_id, token = hf_token)
repo_files = [s.rfilename for s in info.siblings]
return _pick_best_gguf(repo_files)
except Exception as e:
logger.debug(f"Could not check GGUF files for '{repo_id}': {e}")
return None
Retries on transient HF Hub failures (network hiccups, 5xx, slow
cold-start of the API). Without retry, a single transient failure
here returns None silently and the caller treats the repo as
non-GGUF -- which on Apple Silicon (Mac UI route) means falling
through to the MLX backend, which then fails opening a non-existent
config.json on the GGUF-only repo. Three attempts with 1s/2s/4s
backoff covers the typical free-runner HF Hub flakiness.
"""
import time
from huggingface_hub import model_info as hf_model_info
last_err: Optional[Exception] = None
for attempt in range(3):
try:
info = hf_model_info(repo_id, token = hf_token)
repo_files = [s.rfilename for s in info.siblings]
return _pick_best_gguf(repo_files)
except Exception as e:
last_err = e
# 404 / RepoNotFound is permanent -- don't waste attempts.
err_name = type(e).__name__
if err_name in (
"RepositoryNotFoundError",
"GatedRepoError",
"RevisionNotFoundError",
"EntryNotFoundError",
):
logger.debug(
f"Could not check GGUF files for '{repo_id}': {e}"
)
return None
if attempt < 2:
time.sleep(2 ** attempt)
logger.warning(
f"Could not check GGUF files for '{repo_id}' after 3 attempts: "
f"{last_err}"
)
return None
def download_gguf_file(