unsloth/tests
Daniel Han db3393fbc8
studio/ci: harden three pre-existing CI flakes (#5627)
* studio/ci: harden three pre-existing CI flakes

Three independent fixes to flakes that have been failing on main for
multiple PRs in a row and obscuring real signal.

1. tests/studio/playwright_chat_ui.py:
   The theme-toggle x3 block called acct.click() then waited 3s for
   [role="menu"] to appear. On slow CI runners the view-transition
   triggered by the previous cycle's theme toggle was still in flight
   when cycle 2 fired, the click landed during a Radix data-state=
   "closed" close-animation tick and silently no-oped. Symptom:
   "theme cycle 2: account menu didn't open" at line 963.

   Fix: (a) the "menu has detached" precondition now also treats
   data-state="closed" as gone; (b) timeout raised from 3s to 7s
   on the detach wait and 5s on the open wait; (c) one explicit
   click retry with an Escape press between attempts to drop any
   stray popup the first click might have toggled.

2. .github/workflows/consolidated-tests-ci.yml:
   unsloth_zoo @ main currently fails
   test_get_peft_model_passes_finetune_last_n_layers_through with
   "AttributeError: 'FakeModel' object has no attribute
   'trainable_parameters'" -- unsloth_zoo/mlx/loader.py:2972 added a
   model.trainable_parameters() call that the test's fake model
   never stubbed. This blocks every unsloth PR's Core CI. Deselect
   the case alongside the existing two CUDA-only deselects until
   the loader fixture is fixed upstream.

3. .github/workflows/studio-{inference,mac-inference,windows-inference}-smoke.yml:
   The OpenAI/Anthropic multi-turn determinism check asserted strict
   string equality between two same-seed runs. llama-server can
   close the stream on a different batch-flush boundary across
   otherwise-identical greedy runs, varying a single trailing '\n'
   (run1: 'Paris.\n' vs run2: 'Paris.'). Generated tokens are the
   same; only trailing whitespace differs. Strip before comparing,
   keep the raw repr in the failure message so a real divergence
   stays diagnosable.

* studio/ci: fall back to scroll + JS-click for theme menuitem

PR #5627 fixed "account menu didn't open" but uncovered the next layer:
on small macOS arm64 CI viewports the Radix dropdown can render the
theme menuitem below the visible area, and force=True still requires
in-viewport for click to land:

    Locator.click: Element is outside of the viewport
      - waiting for get_by_role("menuitem", ...).first
      - attempting click action
        - scrolling into view if needed
        - done scrolling

The "done scrolling" line is misleading -- Playwright tries to scroll
the element into the viewport but Radix's positioning math keeps it
fixed off-screen, so the actionability gate fires.

Three-tier click fallback:
  1. force=True click with a 3s budget (current path).
  2. scroll_into_view_if_needed() then click.
  3. evaluate("el => el.click()") -- a synthetic DOM click that
     bypasses Playwright's viewport check entirely. Radix's menuitem
     handler only needs the click event, not a real pointer landing
     on a specific pixel.

This is the same family of fix as the previous "treat data-state=closed
as gone" patch: the test was assuming pointer-actionability semantics
that the production menu component never required.
2026-05-20 02:20:15 -07:00
..
notebooks CI: scope GITHUB_TOKEN permissions, add MLX CI, unblock ~60 skipped tests (#5312) 2026-05-11 03:19:13 -07:00
python Fix ORPO text-only tokenization with processors (#5501) 2026-05-18 00:40:30 -07:00
qlora Revert "[pre-commit.ci] auto fixes from pre-commit.com hooks" 2025-12-01 07:24:58 -08:00
saving fix: preserve tokenizer eos token on merged saves (#5451) 2026-05-17 06:44:23 -07:00
security ci: advisory lockfile supply-chain audit (no install-script changes) (#5604) 2026-05-19 05:56:56 -07:00
sh fix(tests/sh): accept pinned tokenizers line after #5359 (#5361) 2026-05-11 02:58:20 -07:00
studio studio/ci: harden three pre-existing CI flakes (#5627) 2026-05-20 02:20:15 -07:00
utils feat: Add cactus QAT scheme support (#4679) 2026-04-15 07:40:03 -07:00
version_compat tests: pinned-symbol canary for unsloth-zoo save_pretrained_merged guards (#5410) (#5433) 2026-05-17 01:35:28 -07:00
vllm_compat CI: scope GITHUB_TOKEN permissions, add MLX CI, unblock ~60 skipped tests (#5312) 2026-05-11 03:19:13 -07:00
__init__.py Qwen 3, Bug Fixes (#2445) 2025-04-30 22:38:39 -07:00
_zoo_aggressive_cuda_spoof.py CI: scope GITHUB_TOKEN permissions, add MLX CI, unblock ~60 skipped tests (#5312) 2026-05-11 03:19:13 -07:00
conftest.py tests: drift detector parity with unsloth-zoo (#5421) 2026-05-14 04:50:30 -07:00
run_all.sh fix: add tokenizers to no-torch deps and TORCH_CONSTRAINT for arm64 macOS py313+ (#4748) 2026-04-01 06:12:17 -07:00
test_callback_signature_drift.py tests + CI: callback signature drift detector (#5498) 2026-05-18 04:42:37 -07:00
test_cli_export_unpacking.py studio: stream export worker output into the export dialog (#4897) 2026-04-14 08:55:43 -07:00
test_finetune_last_n_layers.py fix(peft): expose finetune_last_n_layers for parity with mlx-lm CLI (#5564) 2026-05-19 05:58:44 -07:00
test_gemma4_chat_template.py update gema4 chat templates (#5116) 2026-04-22 09:04:08 -07:00
test_get_model_name.py feat: Add support for OLMo-3 model (#4678) 2026-04-15 07:39:11 -07:00
test_import_fixes_drift.py Fix loss function not patched for Qwen3.5 models (#5442) 2026-05-19 03:57:50 -07:00
test_loader_glob_skip.py Add unit tests for HfFileSystem glob skip guard (#4854) 2026-04-06 08:54:36 -07:00
test_model_registry.py Revert "[FIX] Vllm guided decoding params (#3662)" 2025-12-01 05:43:45 -08:00
test_multi_image_grpo_chunking.py Multi Image GRPO (#5197) 2026-05-13 04:27:49 -07:00
test_peft_weight_converter_compat.py Patch checkpoint reload init functions to strip unsupported args (#5167) 2026-04-29 02:50:49 -07:00
test_public_api_surface.py tests: public-api surface drift detector (companion to test_import_fixes_drift.py) (#5428) 2026-05-14 19:56:21 -07:00
test_raw_text.py Add Studio PR-time CI: pin enforcement, frontend, backend, wheel smoke (#5298) 2026-05-06 04:41:57 -07:00
test_resolve_model_class.py fix: guard resolve_model_class fallback against unresolvable transformers AutoModel entries (#5155) 2026-04-24 05:59:17 -07:00
test_studio_install_workspace_guard.py studio: security and hardening pass (auth rate-limit, sandbox, path containment, schema validation, headers) (#5375) 2026-05-13 06:12:18 -07:00
test_studio_root_resilience.py install: support STUDIO_HOME / UNSLOTH_STUDIO_HOME for custom install paths (#5190) 2026-05-05 23:17:40 -07:00