ci(mac): retry Playwright JSON crash + GGUF detect retry + MLX is_gguf guard
Two distinct Mac UI Chat failures captured in PR 5312's CI:
1. /api/inference/load 500 with FileNotFoundError on config.json for
unsloth/gemma-3-270m-it-GGUF (a GGUF-only repo). Run 25487410091.
Root cause: detect_gguf_model_remote in
studio/backend/utils/models/model_config.py had a single
hf_model_info call with no retry. On a transient HF Hub flake
it returned None silently, the route at routes/inference.py:592
treated the repo as non-GGUF, and dispatched to the MLX
orchestrator. The orchestrator's _build_model_config re-ran
from_identifier in the subprocess (this time succeeding,
logging "Detected remote GGUF") but then handed an is_gguf=True
ModelConfig to MLXInferenceBackend.load_model, which ignored
is_gguf and called FastMLXModel.from_pretrained →
mlx_lm.utils.load_model → opened a non-existent config.json on
the GGUF-only repo. Fix:
a) detect_gguf_model_remote retries up to 3 times with 1/2/4s
backoff, bypassing retry on RepositoryNotFoundError /
GatedRepoError / RevisionNotFoundError / EntryNotFoundError
(those are permanent).
b) MLXInferenceBackend.load_model now raises a clear
RuntimeError if config.is_gguf=True, instead of letting
mlx_lm surface a cryptic 'config.json does not exist'.
2. Playwright pipeTransport.js 'Unexpected end of JSON input' on
macos-14 free runners. Runs 25489049059 + 25489429306. Chromium
browser process dies mid-test → driver Node process can't parse
the truncated JSON-RPC line and exits. Hits ~50% of runs (well
above acceptable flake). Fix: retry the chat-UI step up to 3
times, FULLY resetting Studio (kill, reset-password, reboot,
/api/health wait, re-export STUDIO_OLD/NEW/NEW2_PW) between
attempts so the change-password flow finds a fresh bootstrap on
each retry. Same retry shape on the extra-UI step. Real
assertion / timeout failures don't match the JSON-input pattern
so they bypass retry and surface immediately. Updated the
install-step comment to drop the now-incorrect '1.55-1.57 ship a
Node 22 driver' claim — all 1.55-1.58 Mac drivers are Node 24,
the racy crash is in pipeTransport itself.
This commit is contained in:
parent
8bc52ff1c5
commit
d35bf6ac8e
3 changed files with 156 additions and 15 deletions
|
|
@ -78,6 +78,28 @@ class MLXInferenceBackend:
|
|||
model_name = config.identifier if hasattr(config, "identifier") else str(config)
|
||||
is_vision = getattr(config, "is_vision", False)
|
||||
|
||||
# GGUF guard. GGUF models are served via llama-server in the
|
||||
# parent process, NOT via mlx-lm in this MLX subprocess. The
|
||||
# route at studio/backend/routes/inference.py:592 (`if config.
|
||||
# is_gguf:`) is responsible for sending GGUF traffic to the
|
||||
# llama-server backend before reaching the MLX orchestrator.
|
||||
# If we end up here with is_gguf=True, the route's
|
||||
# `detect_gguf_model_remote` returned None on its first call
|
||||
# (transient HF Hub flake) but the subprocess re-detection
|
||||
# succeeded. The subprocess cannot reach into the parent's
|
||||
# llama-server, so all we can do is raise loudly so the caller
|
||||
# gets a clear error instead of a cryptic
|
||||
# "config.json does not exist" from mlx_lm.utils.load_model.
|
||||
if getattr(config, "is_gguf", False):
|
||||
raise RuntimeError(
|
||||
f"MLXInferenceBackend cannot load GGUF model '{model_name}': "
|
||||
f"GGUF models must be served by llama-server in the parent "
|
||||
f"process. The /api/inference/load route should have "
|
||||
f"detected this repo as GGUF before dispatching to the MLX "
|
||||
f"orchestrator -- this fallback indicates a transient HF "
|
||||
f"Hub failure during initial detection. Retry the request."
|
||||
)
|
||||
|
||||
if hf_token:
|
||||
import os
|
||||
|
||||
|
|
|
|||
|
|
@ -1327,16 +1327,45 @@ def detect_gguf_model_remote(
|
|||
Check if a HuggingFace repo contains GGUF files.
|
||||
|
||||
Returns the filename of the best GGUF file in the repo, or None.
|
||||
"""
|
||||
try:
|
||||
from huggingface_hub import model_info as hf_model_info
|
||||
|
||||
info = hf_model_info(repo_id, token = hf_token)
|
||||
repo_files = [s.rfilename for s in info.siblings]
|
||||
return _pick_best_gguf(repo_files)
|
||||
except Exception as e:
|
||||
logger.debug(f"Could not check GGUF files for '{repo_id}': {e}")
|
||||
return None
|
||||
Retries on transient HF Hub failures (network hiccups, 5xx, slow
|
||||
cold-start of the API). Without retry, a single transient failure
|
||||
here returns None silently and the caller treats the repo as
|
||||
non-GGUF -- which on Apple Silicon (Mac UI route) means falling
|
||||
through to the MLX backend, which then fails opening a non-existent
|
||||
config.json on the GGUF-only repo. Three attempts with 1s/2s/4s
|
||||
backoff covers the typical free-runner HF Hub flakiness.
|
||||
"""
|
||||
import time
|
||||
from huggingface_hub import model_info as hf_model_info
|
||||
|
||||
last_err: Optional[Exception] = None
|
||||
for attempt in range(3):
|
||||
try:
|
||||
info = hf_model_info(repo_id, token = hf_token)
|
||||
repo_files = [s.rfilename for s in info.siblings]
|
||||
return _pick_best_gguf(repo_files)
|
||||
except Exception as e:
|
||||
last_err = e
|
||||
# 404 / RepoNotFound is permanent -- don't waste attempts.
|
||||
err_name = type(e).__name__
|
||||
if err_name in (
|
||||
"RepositoryNotFoundError",
|
||||
"GatedRepoError",
|
||||
"RevisionNotFoundError",
|
||||
"EntryNotFoundError",
|
||||
):
|
||||
logger.debug(
|
||||
f"Could not check GGUF files for '{repo_id}': {e}"
|
||||
)
|
||||
return None
|
||||
if attempt < 2:
|
||||
time.sleep(2 ** attempt)
|
||||
logger.warning(
|
||||
f"Could not check GGUF files for '{repo_id}' after 3 attempts: "
|
||||
f"{last_err}"
|
||||
)
|
||||
return None
|
||||
|
||||
|
||||
def download_gguf_file(
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue