ci(mlx): expand macOS install ladder to match the Linux dep set
The first attempt installed only mlx + mlx-lm + pytest +
unsloth_zoo with --no-deps + unsloth -e --no-deps. That ladder
under-specifies what the MLX import branch in unsloth/__init__.py
actually needs:
- The studio backend hardware module imports structlog at module
top level. Without it tests/studio/test_hardware_dispatch_matrix.py
fails at the very first `from utils.hardware import hardware as hw`
with ModuleNotFoundError.
- unsloth/__init__.py loads dataprep/raw_text.py via
spec_from_file_location, which `from datasets import Dataset`. With
--no-deps on unsloth-zoo neither datasets nor transformers nor any
other shared dep got pulled in.
Mirror the Linux job's working ladder, with two MAC-specific
adjustments:
- Drop bitsandbytes (CUDA-only).
- Drop CPU torch (mlx replaces it on Apple Silicon, and unsloth-zoo
already gates torch on `sys_platform != darwin or platform_machine != arm64`).
- Install unsloth_zoo from git main WITH deps so pip resolves
mlx + mlx-lm + mlx-vlm (gated on darwin+arm64 in the zoo's
pyproject) plus the shared deps (datasets, transformers,
sentencepiece, ...).
Validated locally against a Linux mac-sim venv (platform spoofed to
Darwin/arm64 via mlx_simulation, real datasets/transformers/structlog
installed via the same ladder, fake mlx via the shim):
- Step 1 _IS_MLX activation: OK
- Step 2 import each of unsloth_zoo.mlx_{loader,trainer,compile,utils,cce}
+ unsloth_zoo.gated_delta_vjp + FastMLXModel + MLXTrainer surface: OK
- Step 3 36 tests across the three dispatch files: 36 passed in 0.43s
The Linux job (mlx-dispatch) is unchanged.
This commit is contained in:
parent
c4c2b2aefb
commit
4c85b8259e
1 changed files with 30 additions and 10 deletions
40
.github/workflows/mlx-ci.yml
vendored
40
.github/workflows/mlx-ci.yml
vendored
|
|
@ -115,18 +115,38 @@ jobs:
|
|||
# checks for. Once mlx is on the path the gate flips True and the
|
||||
# MLX import branch in unsloth/__init__.py runs.
|
||||
#
|
||||
# Install unsloth_zoo from git main rather than PyPI: the PR-A
|
||||
# MLX support (#620) merged AFTER the most recent unsloth_zoo
|
||||
# release, so the PyPI wheel still raises NotImplementedError on
|
||||
# Apple Silicon at `device_type.get_device_type()`. Studio's own
|
||||
# install.sh overlays unsloth-zoo from git main for the same
|
||||
# reason; we mirror that here.
|
||||
- name: Install deps (real mlx + unsloth + unsloth-zoo from git)
|
||||
# Dep install ladder, validated locally against a Linux mac-sim
|
||||
# venv:
|
||||
#
|
||||
# 1. Studio backend requirements -- structlog, fastapi, etc. The
|
||||
# hardware probe imports structlog at module top-level, so the
|
||||
# import chain fails without it.
|
||||
#
|
||||
# 2. Other Studio deps the Linux job also installs (numpy<3,
|
||||
# httpx, pytest stack, ...).
|
||||
#
|
||||
# 3. unsloth_zoo from git main, WITH deps. PR-A's MLX support
|
||||
# merged after the latest unsloth_zoo PyPI release, so the
|
||||
# wheel still raises NotImplementedError on Apple Silicon
|
||||
# when device_type.get_device_type() runs unguarded. Studio's
|
||||
# own install.sh overlays unsloth-zoo from git main for the
|
||||
# same reason. Pulling deps lets pip resolve the platform-
|
||||
# conditional MLX-only wheels (mlx, mlx-lm, mlx-vlm, gated
|
||||
# on darwin+arm64 in unsloth-zoo's pyproject) AND the shared
|
||||
# deps (datasets, transformers, sentencepiece, ...) the
|
||||
# unsloth __init__ MLX branch loads via dataprep/raw_text.py.
|
||||
#
|
||||
# 4. unsloth -e . --no-deps so the editable install does not
|
||||
# fight unsloth-zoo's already-resolved pin set.
|
||||
- name: Install deps (Studio backend + real mlx + unsloth + unsloth-zoo from git)
|
||||
run: |
|
||||
python -m pip install --upgrade pip
|
||||
pip install mlx mlx-lm
|
||||
pip install pytest pytest-asyncio
|
||||
pip install --no-deps "unsloth_zoo @ git+https://github.com/unslothai/unsloth-zoo"
|
||||
pip install -r studio/backend/requirements/studio.txt
|
||||
pip install \
|
||||
python-multipart aiofiles sqlalchemy cryptography \
|
||||
pyyaml jinja2 mammoth unpdf requests typer \
|
||||
'numpy<3' pytest pytest-asyncio httpx
|
||||
pip install "unsloth_zoo @ git+https://github.com/unslothai/unsloth-zoo"
|
||||
pip install -e . --no-deps
|
||||
|
||||
- name: Verify _IS_MLX flips True on real Apple Silicon
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue