378e33c8a5* Studio: defer llama.cpp update probes and self-heal MLX on macOS Two macOS startup problems shared one root area in the FastAPI lifespan: - The llama.cpp capability + freshness probes ran inline before the server yielded, so a cold/slow/flaky network on the GitHub freshness check blocked 'Application startup complete' (~34s on CI, longer in the field). Move both probes to a daemon thread; app.state stays None until ready (status routes already re-probe at request time). Opt out with UNSLOTH_DISABLE_UPDATE_CHECK=1. - Train and Export were greyed out because mlx/mlx-lm/mlx-vlm arrive only transitively and a resolver backtrack silently drops them, so CHAT_ONLY stayed true. Add utils/mlx_repair.py: when Apple Silicon is detected without MLX, reinstall mlx/mlx-lm/mlx-vlm by name on a daemon thread and re-run hardware detection (opt out UNSLOTH_DISABLE_MLX_AUTOREPAIR=1). Surface a chat_only_reason in /api/health plus a sidebar tooltip so a greyed Train/Export explains itself instead of failing silently. * Studio: guard model defaults against a None model name load_model_defaults(None) called model_name.lower() with no guard, raising 'Error loading model defaults for None' before any model is selected. Return an empty dict for a falsy/non-str name. * Studio: drop obsolete upstream macOS + Windows Blackwell prebuilt pins Both pins worked around gaps in ggml-org upstream prebuilts, but Studio now routes every GPU host and all of macOS to the unslothai/llama.cpp fork (published_repo_for_host), which ships the needed bundles, so both pins are dead code on the default install path: - macOS b9415: macOS always routes to the fork (its own macOS bundles), and host_supports_macos_minos() is the backstop. The pin only fired under an explicit --published-repo ggml-org override. - Windows Blackwell b9360: Windows-NVIDIA routes to the fork, whose windows-x64-cuda13 bundle covers Blackwell (manifest max_sm 120, toolkit 13.3), so the pin's self-disable check makes it dormant on every default install; it could only activate under the same upstream override on a 13.0-13.2 driver. Remove the pin constants, functions, and call sites. Keep the Blackwell capability detection (_drop_blackwell_incapable_windows_cuda, _host_is_blackwell, _windows_cuda_attempt_covers_blackwell) that still drops a non-sm_120 cuda-12.4 build on a Blackwell host. After this, an explicit --published-repo ggml-org override on a Blackwell 13.0-13.2 host loses its GPU fallback and lands on CPU; the default fork path is unaffected. Update the install selection-logic and macOS-compat unit tests for the new no-pin behavior. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: walk back deeper on the macOS upstream prebuilt path After removing the b9415 macOS pin, the explicit --published-repo ggml-org upstream path still used the default 2-release fallback, so a pre-macOS-26 host behind a run of macOS-26-only builds would exhaust two too-new plans (minos is only checked post-download) and drop to a source build before reaching a loadable older release. Walk back as deep as the fork macOS path (DEFAULT_MAX_MACOS_RELEASE_FALLBACKS), turning the removed static pin into dynamic discovery. Addresses review feedback on the macOS upstream fallback. * Studio: pin transformers during MLX self-heal so it cannot break Studio mlx-lm/mlx-vlm declare transformers>=5, but the single-env install pins transformers==4.57.6. The self-heal used --upgrade with no constraint, so it could upgrade transformers in the live venv and break the rest of Studio just to make import mlx.core pass. Pin transformers to the installed version via a constraint file: the resolver either finds an mlx build compatible with it or fails (we stay chat-only), never upgrading transformers underneath Studio. Addresses review feedback on the MLX repair install. * Studio: harden MLX self-heal against an unsupported mlx-vlm Pinning transformers alone made uv backtrack mlx-vlm to 0.3.9 (below unsloth-zoo's mlx-vlm>=0.4.4), which imports but breaks VLM Train/Export -- so the self-heal could clear chat-only onto a broken stack. Mirror the main installer: set UV_OVERRIDE=overrides-darwin-arm64.txt so a current mlx-vlm coexists with the transformers pin, require the same minimum versions unsloth-zoo declares, and gate/validate on a full mlx_stack_available() check (not a bare import) so an old or partial stack stays chat-only. Addresses PR review. * Studio: filter Blackwell-incapable CUDA in resolve_upstream_asset_choice resolve_upstream_asset_choice returned the first windows-cuda choice unfiltered, so a Blackwell host could be handed an sm_120-incapable cuda-12.4 build while the sibling planners drop it. Apply _drop_blackwell_incapable_windows_cuda here too and fall through to the CPU bundle on a Blackwell host with no capable GPU asset. Addresses PR review. * Studio: re-poll health so MLX self-heal reaches an open UI The sidebar cached the initial /api/health, so a successful background MLX self-heal (chat_only flips false) did not re-enable Train/Export until a manual reload. While chat-only for the recoverable mlx_unavailable reason, re-poll /api/health and stop once Train/Export become available. Addresses PR review. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: make the disabled Train/Export tooltip reachable The greyed Train/Export items pass a tooltip explaining why (e.g. MLX missing), but a disabled <button> fires no pointer events and SidebarMenuButton only showed tooltips while collapsed, so the explanation never appeared. Wrap a disabled button in a focusable span and show its tooltip while expanded too; enabled items keep the collapsed-only behavior. Addresses PR review. * Studio: gate Train/Export on the full MLX stack, not bare mlx.core detect_hardware enabled MLX training whenever `import mlx.core` worked, but the MLX self-heal (utils/mlx_repair) treats a stack without mlx-lm/mlx-vlm at the versions unsloth-zoo requires as inadequate. That asymmetry let the UI enable Train/Export on exactly the partial/backtracked stack the self-heal is trying to repair (greyed-in-but-broken VLM export). Gate on the same mlx_stack_available() criterion so a partial stack stays chat-only (reason mlx_unavailable) and the background repair restores it. Addresses PR review. * Fix MLX repair and health auth for PR #6494 * Fix macOS upstream prebuilt fallback for PR #6494 * Fix MLX stack validation for PR #6494 * Fix MLX self-heal validation for PR #6494 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Review fixes: isolate hardware-state test, robust transformers pin - test_chat_only_reason.py: detect_hardware() assigns module globals directly, which monkeypatch does not revert; the autouse fixture now saves and restores DEVICE/CHAT_ONLY/CHAT_ONLY_REASON/IS_ROCM so a chat-only verdict here cannot leak into other backend tests (e.g. test_utils.py) on a GPU host. - mlx_repair.py: read the transformers version from importlib.metadata instead of importing transformers, so the install pin is not silently dropped when transformers has valid metadata but fails to import. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix CI: model full MLX stack in dispatch tests, keep selection test offline dispatch (macOS) job: - detect_hardware now gates MLX on the full stack (mlx_stack_available imports mlx_lm/mlx_vlm and checks dist versions), so faking only mlx.core makes the apple_silicon_mlx profile resolve to CPU. The dispatch tests assert the routing decision when the stack IS usable, so model a complete stack: test_hardware_dispatch_matrix patches utils.mlx_repair.mlx_stack_available and test_is_mlx_dispatch_gate patches hardware._has_usable_mlx_stack. The stack predicate's own internals stay covered by test_mlx_repair.py. Repo tests (CPU) job: - test_no_cuda_attempt_on_published_path_for_13_1 fell through to a live github_release_assets() upstream fetch after the Blackwell filter dropped every published attempt, which the offline security scanner blocks. Stub that fetch so the walk-back deterministically finds no usable CUDA build and raises PrebuiltFallback without network. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Harden MLX self-heal: prepare transformers constraint inside the try attempt_mlx_repair runs on a daemon thread, but _transformers_constraint_args was called before the try. A failure there (e.g. tempfile.mkstemp on a full disk or a bad TMPDIR) would propagate unhandled and silently kill the self-heal thread. Move the call inside the try and initialize constraint_path so any such failure is caught and leaves Studio chat-only instead of crashing the thread. --------- Co-authored-by: Daniel Han <michaelhan2050@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: wasimysaid <wasimysdev@gmail.com>
83 lines
3.2 KiB
Python
83 lines
3.2 KiB
Python
# SPDX-License-Identifier: AGPL-3.0-only
|
|
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
|
|
|
"""detect_hardware() records WHY a host is chat-only so the UI can explain the
|
|
greyed-out Train/Export instead of disabling them silently.
|
|
|
|
The key case is Apple Silicon without an importable MLX -> "mlx_unavailable",
|
|
which is the usual cause of "Train and Export greyed out" on Macs after a
|
|
reinstall/update dropped MLX.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import sys
|
|
from pathlib import Path
|
|
|
|
import pytest
|
|
|
|
_BACKEND = Path(__file__).resolve().parent.parent
|
|
if str(_BACKEND) not in sys.path:
|
|
sys.path.insert(0, str(_BACKEND))
|
|
|
|
import utils.hardware.hardware as hw # noqa: E402
|
|
|
|
|
|
@pytest.fixture(autouse = True)
|
|
def _no_torch(monkeypatch):
|
|
# Force the non-CUDA/XPU path regardless of the test host's real GPUs.
|
|
monkeypatch.setattr(hw, "_has_torch", lambda: False)
|
|
# detect_hardware() assigns these module globals directly (not via monkeypatch),
|
|
# so save and restore them; otherwise a chat-only verdict here leaks into other
|
|
# backend tests (e.g. test_utils.py) when they share a process on a GPU host.
|
|
saved = (hw.DEVICE, hw.CHAT_ONLY, hw.CHAT_ONLY_REASON, hw.IS_ROCM)
|
|
try:
|
|
yield
|
|
finally:
|
|
hw.DEVICE, hw.CHAT_ONLY, hw.CHAT_ONLY_REASON, hw.IS_ROCM = saved
|
|
|
|
|
|
def test_apple_silicon_without_mlx_is_chat_only_with_reason(monkeypatch):
|
|
monkeypatch.setattr(hw, "is_apple_silicon", lambda: True)
|
|
monkeypatch.setattr(hw, "_has_usable_mlx_stack", lambda: False)
|
|
hw.detect_hardware()
|
|
assert hw.CHAT_ONLY is True
|
|
assert hw.CHAT_ONLY_REASON == "mlx_unavailable"
|
|
|
|
|
|
def test_apple_silicon_with_mlx_enables_training(monkeypatch):
|
|
monkeypatch.setattr(hw, "is_apple_silicon", lambda: True)
|
|
monkeypatch.setattr(hw, "_has_usable_mlx_stack", lambda: True)
|
|
hw.detect_hardware()
|
|
assert hw.CHAT_ONLY is False
|
|
assert hw.CHAT_ONLY_REASON is None
|
|
|
|
|
|
def test_apple_silicon_with_incomplete_mlx_stack_stays_chat_only(monkeypatch):
|
|
# Bare `import mlx.core` works but the full mlx/mlx-lm/mlx-vlm stack does not
|
|
# (e.g. a backtracked/old mlx-vlm). The training gate must match the self-heal
|
|
# validator and stay chat-only so the UI does not enable a broken Train/Export.
|
|
monkeypatch.setattr(hw, "is_apple_silicon", lambda: True)
|
|
monkeypatch.setattr(hw, "_has_mlx", lambda: True)
|
|
monkeypatch.setattr(hw, "_has_usable_mlx_stack", lambda: False)
|
|
assert hw.detect_hardware() == hw.DeviceType.CPU
|
|
assert hw.CHAT_ONLY is True
|
|
assert hw.CHAT_ONLY_REASON == "mlx_unavailable"
|
|
|
|
|
|
def test_intel_mac_reason(monkeypatch):
|
|
monkeypatch.setattr(hw, "is_apple_silicon", lambda: False)
|
|
monkeypatch.setattr(hw, "_has_mlx", lambda: False)
|
|
monkeypatch.setattr(hw.platform, "system", lambda: "Darwin")
|
|
hw.detect_hardware()
|
|
assert hw.CHAT_ONLY is True
|
|
assert hw.CHAT_ONLY_REASON == "intel_mac"
|
|
|
|
|
|
def test_cpu_only_non_mac_reason(monkeypatch):
|
|
monkeypatch.setattr(hw, "is_apple_silicon", lambda: False)
|
|
monkeypatch.setattr(hw, "_has_mlx", lambda: False)
|
|
monkeypatch.setattr(hw.platform, "system", lambda: "Linux")
|
|
hw.detect_hardware()
|
|
assert hw.CHAT_ONLY is True
|
|
assert hw.CHAT_ONLY_REASON == "no_gpu"
|