unsloth/tests/studio
Daniel Han 9d47eb2e95
studio/tests: AbortSignal-bound in-page fetches and wall-clock watchdog for Playwright probes (#5391)
* studio/tests: AbortSignal-bound in-page fetches + wall-clock watchdog

Run 25696797934 / job 75446949358 on PR #5387 cancelled the
"Chat UI Tests" macos-14 job at 30 min: studio.log went idle after
the chat surface mounted, no further requests reached the server,
and Playwright silently sat on a `page.evaluate(async () => fetch(
/api/inference/load))` for 27+ minutes before the runner-level
timeout fired. The two other Chat UI Tests jobs on the same SHA
passed in 5-17 min, so this was a transient renderer wedge under
--single-process Chromium, not a regression from the security
bumps in that PR.

Root cause: Playwright's `page.evaluate(...)` has no `timeout=`
argument. If the JS body awaits a fetch whose promise never
settles (the renderer's network thread stalls behind the busy main
thread on the free macos-14 runner), the entire Python script
hangs until something external kills it.

Add two helpers in `_playwright_robust.py`:

  - `evaluate_fetch(page, url, *, method, headers, body, timeout_ms)`
    wraps `fetch()` in an `AbortController` so the JS resolves
    either with a real response or with `{status: 0, error:
    "AbortError..."}` after the budget elapses. Callers fail loud
    on a non-None `error` and the wedge surfaces as a one-line
    diagnostic instead of a 30-min cancel.

  - `install_wall_clock_watchdog(deadline_s)` starts a daemon
    Timer that hard-exits the process at the deadline. Belt-and-
    suspenders for any wedge inside the browser that the per-
    action timeouts cannot bound. Default 720s (12 min); healthy
    runs measure 5-9 min on macos-14 so the headroom is small
    without amplifying a wedge to the 30-min runner cap.

Wire both into `playwright_chat_ui.py` and `playwright_extra_ui.py`:

  - Replace every `page.evaluate(async () => fetch(...))` site
    with `evaluate_fetch(...)`: refresh-token exchange, defaults
    fetch, inference load, health probe, post-rotation refresh.
    Five sites in chat_ui, two in extra_ui.
  - Arm the watchdog at the top of `with sync_playwright()` and
    cancel it on clean exit.

Knobs (all default-safe, override only for slow runners):
  STUDIO_UI_WALL_TIMEOUT_S   (default 720s)
  STUDIO_UI_FETCH_TIMEOUT_MS (default 30000ms)
  STUDIO_UI_LOAD_TIMEOUT_MS  (default 180000ms)

Verified locally with `python -c "ast.parse(...)"` on all three
files and a unit smoke that confirms `evaluate_fetch`'s JS argument
shape and that `install_wall_clock_watchdog` returns a daemonised
Timer that responds to `.cancel()`.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-05-11 20:37:24 -07:00
..
install Studio: download paired cudart bundle on Windows CUDA installs (#5322) 2026-05-11 05:42:05 -07:00
_playwright_robust.py studio/tests: AbortSignal-bound in-page fetches and wall-clock watchdog for Playwright probes (#5391) 2026-05-11 20:37:24 -07:00
playwright_chat_ui.py studio/tests: AbortSignal-bound in-page fetches and wall-clock watchdog for Playwright probes (#5391) 2026-05-11 20:37:24 -07:00
playwright_extra_ui.py studio/tests: AbortSignal-bound in-page fetches and wall-clock watchdog for Playwright probes (#5391) 2026-05-11 20:37:24 -07:00
run_real_mlx_smoke.py CI: scope GITHUB_TOKEN permissions, add MLX CI, unblock ~60 skipped tests (#5312) 2026-05-11 03:19:13 -07:00
studio_api_smoke.py CI: scope GITHUB_TOKEN permissions, add MLX CI, unblock ~60 skipped tests (#5312) 2026-05-11 03:19:13 -07:00
test_cancel_atomicity.py Studio: make stop button actually stop generation (#5069) 2026-04-24 10:09:25 -07:00
test_cancel_id_wiring.py Studio: make stop button actually stop generation (#5069) 2026-04-24 10:09:25 -07:00
test_chat_preset_builtin_invariants.py Studio: Dark theme refactor, right sidebar redesign, and chat UI polish (#5150) 2026-05-07 14:33:31 +04:00
test_cli_repo_variant.py Studio: forward llama-server args from unsloth studio run , activate unsloth run , and allow passing model:quant to load models (#5271) 2026-05-04 17:08:04 +04:00
test_cli_run_alias.py Studio: forward llama-server args from unsloth studio run , activate unsloth run , and allow passing model:quant to load models (#5271) 2026-05-04 17:08:04 +04:00
test_cli_studio_defaults.py Default Studio host to 127.0.0.1 and prompt before auto-start (#5267) 2026-05-04 13:03:16 +04:00
test_export_output_path_contract.py feat(studio): MLX training tab on Apple Silicon (LoRA / full FT, VLM, export) (#5265) 2026-05-05 23:54:58 -07:00
test_hardware_dispatch_matrix.py CI: scope GITHUB_TOKEN permissions, add MLX CI, unblock ~60 skipped tests (#5312) 2026-05-11 03:19:13 -07:00
test_is_mlx_dispatch_gate.py feat(studio): MLX training tab on Apple Silicon (LoRA / full FT, VLM, export) (#5265) 2026-05-05 23:54:58 -07:00
test_llama_cpp_wall_clock_cap.py Studio: make stop button actually stop generation (#5069) 2026-04-24 10:09:25 -07:00
test_mlx_training_worker_behaviors.py feat(studio): MLX training tab on Apple Silicon (LoRA / full FT, VLM, export) (#5265) 2026-05-05 23:54:58 -07:00
test_stream_cancel_registration_timing.py Studio: make stop button actually stop generation (#5069) 2026-04-24 10:09:25 -07:00
test_studio_gguf_export_script_pin.py Pin Studio GGUF export to llama.cpp's local convert script (#5275) 2026-05-05 04:03:28 -07:00
test_studio_text_descender_clipping.py Studio: Fix clipped model selector text descenders (#5210) 2026-04-29 02:51:25 -07:00