ci(mlx): consolidate to single Mac M1 job with robust no-mlx spoof
Previously the workflow ran the dispatch tests on two matrix legs (linux-cpu-spoof + macos-m1-real), which duplicated the spoofed hardware matrix (it works identically on any host) while only the Mac leg covered Apple-specific real-mlx checks. Drop the Linux leg, rename the workflow to "MLX CI on Mac M1", and rely on the Mac runner alone -- it now runs the SAME spoofed matrix PLUS the three real-Apple-Silicon checks (real `_IS_MLX = True`, real mlx wheel smoke imports, no spoof collisions with the live environment). Also fix the `apple_silicon_no_mlx` profile so the spoof works on a real Mac with mlx genuinely installed. Studio's `_has_mlx()` does literal `import mlx.core` and catches `ImportError`, which the previous spoof (delete `sys.modules["mlx"]` + patch `find_spec`) could not block when mlx was on disk -- Python would re-find and import the real package. The fix installs a `MetaPathFinder` for the duration of the spoof that raises `ImportError` for `mlx` / `mlx.*`, faithfully simulating "mlx not installed" regardless of whether the host has the wheel. No change to the dispatch logic in unsloth or studio; the Mac runner now exercises every profile end to end with the real wheels installed. Validated locally on .macsim_venv3 with a stand-in `mlx` package on disk at .fakemlx_pkg/ to mimic the macos-14 runner: 35 passed + 1 skipped.
This commit is contained in:
Daniel Han
parent
98d601bd45
commit
53ee9ce2ec
2 changed files with 82 additions and 111 deletions
166
.github/workflows/mlx-ci.yml
vendored
166
.github/workflows/mlx-ci.yml
vendored
|
|
@ -1,30 +1,35 @@
|
|||
# SPDX-License-Identifier: AGPL-3.0-only
|
||||
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved.
|
||||
|
||||
# Focused PR gate for the MLX dispatch surface. One job, two matrix
|
||||
# variants:
|
||||
# Focused PR gate for the MLX dispatch surface, running on a real
|
||||
# Apple Silicon runner.
|
||||
#
|
||||
# - linux-cpu-spoof ubuntu-latest with hardware probes spoofed via
|
||||
# monkeypatch (no Apple Silicon, no real GPU, no
|
||||
# real MLX install required).
|
||||
# - macos-m1-real macos-14 (M1, 3 vCPU / 7 GB / Apple Silicon
|
||||
# standard runner -- FREE for public repositories
|
||||
# per the GitHub Actions billing reference, larger
|
||||
# variants like macos-14-large/-xlarge are paid
|
||||
# so we deliberately avoid those). Adds two
|
||||
# Mac-only steps before the dispatch tests:
|
||||
# 1. verify unsloth._IS_MLX flips True on real
|
||||
# Apple Silicon (no spoof);
|
||||
# 2. smoke-import every PR-A MLX-only module
|
||||
# (mlx_loader, mlx_trainer, mlx_compile,
|
||||
# mlx_utils, mlx_cce, gated_delta_vjp). Each
|
||||
# does `import mlx.core as mx` at module top
|
||||
# level, so this catches a future change that
|
||||
# breaks the real `mlx` PyPI wheel without
|
||||
# needing a Mac developer in the loop.
|
||||
# Runner: macos-14 (M1, 3 vCPU / 7 GB / Apple Silicon standard runner
|
||||
# -- FREE for public repositories per the GitHub Actions billing
|
||||
# reference; larger variants like macos-14-large/-xlarge are paid so
|
||||
# we deliberately avoid those).
|
||||
#
|
||||
# Both variants then run the SAME three dispatch test files documented
|
||||
# in tests/studio/README.md:
|
||||
# Why a single Mac job (no Linux+spoof leg): the dispatch tests are
|
||||
# 100% spoofed monkeypatches and run identically on any host, so the
|
||||
# Linux leg was duplicating the matrix tests already covered on Mac
|
||||
# while missing everything Apple-specific. The Mac job runs the SAME
|
||||
# spoofed matrix PLUS three things only a real Apple Silicon host
|
||||
# can prove:
|
||||
#
|
||||
# 1. unsloth._IS_MLX flips True on Darwin+arm64 with mlx genuinely
|
||||
# installed (no spoof).
|
||||
# 2. Every PR-A MLX-only unsloth_zoo module (mlx_loader, mlx_trainer,
|
||||
# mlx_compile, mlx_utils, mlx_cce, gated_delta_vjp) imports
|
||||
# against the real `mlx` + `mlx-lm` + `mlx-vlm` PyPI wheels --
|
||||
# each does `import mlx.core as mx` at module top level, so this
|
||||
# catches a future change that breaks the real wheels without
|
||||
# needing a Mac developer in the loop.
|
||||
# 3. The hardware-dispatch spoofs do not collide with the real
|
||||
# environment (the test fixture installs a MetaPathFinder that
|
||||
# blocks `import mlx.core` for "no-mlx" profiles, faithfully
|
||||
# simulating a Mac without mlx even when mlx IS installed).
|
||||
#
|
||||
# Three dispatch test files documented in tests/studio/README.md:
|
||||
# - test_hardware_dispatch_matrix.py parametrized 7-profile matrix
|
||||
# + 2 dispatch-priority canaries
|
||||
# - test_is_mlx_dispatch_gate.py AST + runtime guard on
|
||||
|
|
@ -32,9 +37,7 @@
|
|||
# - test_mlx_training_worker_behaviors.py AST contract checks on
|
||||
# studio/backend/core/training/worker.py
|
||||
#
|
||||
# Surfaces two PR checks ("MLX dispatch (linux-cpu-spoof)" and
|
||||
# "MLX dispatch (macos-m1-real)") sharing one workflow definition so
|
||||
# adding new dispatch tests applies to both runners automatically.
|
||||
# Surfaces a single PR check ("MLX CI on Mac M1 / dispatch").
|
||||
#
|
||||
# Security audit footprint: every package this workflow installs is
|
||||
# already covered by .github/workflows/security-audit.yml -- the deps
|
||||
|
|
@ -44,7 +47,7 @@
|
|||
# git URL through PyPI metadata; the audit comment in security-audit.yml
|
||||
# documents this). No new package is introduced solely by MLX CI.
|
||||
|
||||
name: MLX CI
|
||||
name: MLX CI on Mac M1
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
|
|
@ -71,21 +74,9 @@ permissions:
|
|||
|
||||
jobs:
|
||||
dispatch:
|
||||
name: MLX dispatch (${{ matrix.label }})
|
||||
runs-on: ${{ matrix.os }}
|
||||
timeout-minutes: ${{ matrix.timeout }}
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
include:
|
||||
- os: ubuntu-latest
|
||||
label: linux-cpu-spoof
|
||||
real_mlx: false
|
||||
timeout: 10
|
||||
- os: macos-14
|
||||
label: macos-m1-real
|
||||
real_mlx: true
|
||||
timeout: 15
|
||||
name: dispatch
|
||||
runs-on: macos-14
|
||||
timeout-minutes: 15
|
||||
steps:
|
||||
- uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4.3.1
|
||||
|
||||
|
|
@ -94,64 +85,24 @@ jobs:
|
|||
python-version: '3.12'
|
||||
cache: 'pip'
|
||||
|
||||
# Linux variant: spoofed hardware. Mirrors the Repo tests (CPU)
|
||||
# dep set from studio-backend-ci.yml so unsloth's import chain
|
||||
# succeeds on a runner without any GPU. CPU torch from the
|
||||
# PyTorch index, transformers + datasets + bitsandbytes from
|
||||
# the standard PyPI index, unsloth-zoo from PyPI.
|
||||
# All explicit pip installs are version-pinned to a single
|
||||
# released version. unsloth-zoo on Linux is the latest PyPI
|
||||
# release; on macOS it is sourced from git main (PR-A is not
|
||||
# yet on PyPI). The pin set was the latest as of 2026-05-07
|
||||
# within each project's existing constraint ranges; bump
|
||||
# alongside the rest of the security audit when a new release
|
||||
# of any of these lands.
|
||||
- name: Install deps (Linux+CPU spoof)
|
||||
if: matrix.real_mlx == false
|
||||
run: |
|
||||
python -m pip install --upgrade pip
|
||||
pip install -r studio/backend/requirements/studio.txt
|
||||
pip install \
|
||||
'python-multipart==0.0.27' \
|
||||
'aiofiles==25.1.0' \
|
||||
'sqlalchemy==2.0.49' \
|
||||
'cryptography==48.0.0' \
|
||||
'pyyaml==6.0.3' \
|
||||
'jinja2==3.1.6' \
|
||||
'mammoth==1.12.0' \
|
||||
'unpdf==1.0.0' \
|
||||
'requests==2.33.1' \
|
||||
'typer==0.25.1' \
|
||||
'numpy==2.4.4' \
|
||||
'pytest==9.0.3' \
|
||||
'pytest-asyncio==1.3.0' \
|
||||
'httpx==0.28.1'
|
||||
pip install --index-url https://download.pytorch.org/whl/cpu \
|
||||
'torch==2.10.0' 'torchvision==0.25.0'
|
||||
pip install 'transformers==5.5.0'
|
||||
pip install 'bitsandbytes==0.49.2'
|
||||
pip install 'unsloth_zoo==2026.5.1'
|
||||
pip install -e . --no-deps
|
||||
|
||||
# macOS variant: real Apple Silicon. Install ladder validated
|
||||
# locally against a Linux mac-sim venv (platform spoofed +
|
||||
# mlx_simulation shim + real datasets/transformers/structlog).
|
||||
# macOS install ladder, validated locally against a Linux
|
||||
# mac-sim venv (platform spoofed + mlx_simulation shim + real
|
||||
# datasets/transformers/structlog).
|
||||
#
|
||||
# 1. studio/backend/requirements/studio.txt brings structlog,
|
||||
# fastapi, etc. The hardware probe imports structlog at
|
||||
# module top level.
|
||||
# 2. Same pytest / numpy / httpx stack as the Linux variant.
|
||||
# 2. Same pytest / numpy / httpx stack the rest of the repo CI
|
||||
# uses.
|
||||
# 3. torch is explicitly installed: unsloth-zoo's pyproject
|
||||
# deliberately excludes torch on darwin+arm64 (mlx replaces
|
||||
# it for runtime use), but the dispatch tests spoof
|
||||
# torch.cuda / torch.xpu / torch.backends.mps via monkeypatch
|
||||
# and so the test process needs torch importable. We pull
|
||||
# from the PyTorch CPU index for both Linux and macOS so
|
||||
# Apple Silicon gets the explicit cpu+MPS arm64 wheel rather
|
||||
# than something the default PyPI resolver might pick up.
|
||||
# https://download.pytorch.org/whl/cpu hosts
|
||||
# torch-x.y.z-cp312-...-macosx_*_arm64.whl alongside the
|
||||
# Linux x86_64 wheels.
|
||||
# from the PyTorch CPU index so Apple Silicon gets the
|
||||
# explicit cpu+MPS arm64 wheel rather than something the
|
||||
# default PyPI resolver might pick up. The CPU index hosts
|
||||
# macosx_*_arm64 wheels alongside the Linux x86_64 ones.
|
||||
# 4. unsloth-zoo from git main (NOT PyPI), WITH deps. PR-A's
|
||||
# MLX support landed after the most recent unsloth-zoo PyPI
|
||||
# release; the wheel still raises NotImplementedError on
|
||||
|
|
@ -165,13 +116,12 @@ jobs:
|
|||
# dataprep/raw_text.py.
|
||||
# 5. unsloth -e . --no-deps so the editable install does not
|
||||
# fight the unsloth-zoo dep set.
|
||||
# See the comment on the Linux variant -- same pin set, with
|
||||
# two macOS-specific adjustments: no torchvision (zoo's mlx-vlm
|
||||
# replaces it on Apple Silicon), no bitsandbytes (CUDA-only),
|
||||
# no transformers explicit pin (zoo's deps already constrain
|
||||
# it), and unsloth-zoo from git main rather than PyPI.
|
||||
- name: Install deps (macOS real Apple Silicon)
|
||||
if: matrix.real_mlx
|
||||
#
|
||||
# All explicit pip installs are version-pinned to a single
|
||||
# released version (the latest as of 2026-05-07 within each
|
||||
# project's existing constraint range). bump alongside the rest
|
||||
# of the security audit when a new release lands.
|
||||
- name: Install deps
|
||||
run: |
|
||||
python -m pip install --upgrade pip
|
||||
pip install -r studio/backend/requirements/studio.txt
|
||||
|
|
@ -195,10 +145,9 @@ jobs:
|
|||
pip install "unsloth_zoo @ git+https://github.com/unslothai/unsloth-zoo"
|
||||
pip install -e . --no-deps
|
||||
|
||||
# Mac-only sanity: confirm _IS_MLX activates on real Apple
|
||||
# Silicon hardware with no platform spoof.
|
||||
# Real Apple Silicon sanity: confirm _IS_MLX activates on real
|
||||
# hardware with no platform spoof.
|
||||
- name: Verify _IS_MLX flips True on real Apple Silicon
|
||||
if: matrix.real_mlx
|
||||
run: |
|
||||
python -c "
|
||||
import platform
|
||||
|
|
@ -209,10 +158,9 @@ jobs:
|
|||
print('OK: _IS_MLX activated on real Apple Silicon')
|
||||
"
|
||||
|
||||
# Mac-only sanity: confirm every PR-A MLX-only module loads
|
||||
# against real mlx + mlx-lm + mlx-vlm wheels.
|
||||
# Real Apple Silicon sanity: confirm every PR-A MLX-only module
|
||||
# loads against real mlx + mlx-lm + mlx-vlm wheels.
|
||||
- name: Smoke-import every MLX-only unsloth_zoo module
|
||||
if: matrix.real_mlx
|
||||
run: |
|
||||
python -c "
|
||||
import importlib
|
||||
|
|
@ -232,13 +180,11 @@ jobs:
|
|||
print('OK: FastMLXModel + MLXTrainer surface present')
|
||||
"
|
||||
|
||||
# Both variants run the same dispatch tests. The monkeypatch
|
||||
# spoofs in test_hardware_dispatch_matrix.py override
|
||||
# platform.system / platform.machine / torch.cuda /
|
||||
# torch.xpu / torch.backends.mps for each test, so they
|
||||
# behave identically on Linux and on real Apple Silicon. The
|
||||
# macOS variant additionally proves the spoofs do not collide
|
||||
# with the real environment.
|
||||
# Spoofed dispatch matrix. Runs on the real Mac too -- the
|
||||
# test fixture installs a MetaPathFinder that blocks
|
||||
# `import mlx.core` for "no-mlx" profiles, so the spoofs
|
||||
# faithfully simulate every supported hardware combo regardless
|
||||
# of whether mlx is installed for real.
|
||||
- name: MLX dispatch tests (3 files, 36 tests)
|
||||
env:
|
||||
PYTHONPATH: ${{ github.workspace }}/studio
|
||||
|
|
|
|||
|
|
@ -263,17 +263,42 @@ def spoof_hardware(monkeypatch):
|
|||
monkeypatch.setitem(sys.modules, "mlx", fake_mlx)
|
||||
monkeypatch.setitem(sys.modules, "mlx.core", fake_mlx_core)
|
||||
else:
|
||||
# Drop any cached mlx modules and patch find_spec so the
|
||||
# unsloth gate (which uses importlib.util.find_spec) sees
|
||||
# mlx as absent.
|
||||
monkeypatch.delitem(sys.modules, "mlx", raising = False)
|
||||
monkeypatch.delitem(sys.modules, "mlx.core", raising = False)
|
||||
real_find_spec = importlib.util.find_spec
|
||||
|
||||
def _no_mlx(name, *args, **kwargs):
|
||||
if name == "mlx":
|
||||
if name == "mlx" or name.startswith("mlx."):
|
||||
return None
|
||||
return real_find_spec(name, *args, **kwargs)
|
||||
|
||||
monkeypatch.setattr(importlib.util, "find_spec", _no_mlx)
|
||||
|
||||
# Studio's _has_mlx() literally does `import mlx.core`, not
|
||||
# find_spec, so on a real Apple Silicon host with mlx
|
||||
# genuinely installed the import would still succeed. Block
|
||||
# it via a meta_path finder that raises ImportError for any
|
||||
# `mlx` / `mlx.*` import while this profile is active.
|
||||
class _BlockMLXFinder:
|
||||
def find_spec(self_inner, name, path = None, target = None):
|
||||
if name == "mlx" or name.startswith("mlx."):
|
||||
raise ImportError(
|
||||
f"mlx import blocked by spoof_hardware "
|
||||
f"(profile={profile.name})"
|
||||
)
|
||||
return None
|
||||
|
||||
blocker = _BlockMLXFinder()
|
||||
# Replace meta_path with a NEW list so monkeypatch can fully
|
||||
# restore the original on teardown (mutating the list in
|
||||
# place would survive the test).
|
||||
monkeypatch.setattr(
|
||||
sys, "meta_path", [blocker, *sys.meta_path],
|
||||
)
|
||||
|
||||
return _apply
|
||||
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue