Studio macOS: faster startup, MLX self-heal, drop obsolete prebuilt pins (#6494)
* Studio: defer llama.cpp update probes and self-heal MLX on macOS Two macOS startup problems shared one root area in the FastAPI lifespan: - The llama.cpp capability + freshness probes ran inline before the server yielded, so a cold/slow/flaky network on the GitHub freshness check blocked 'Application startup complete' (~34s on CI, longer in the field). Move both probes to a daemon thread; app.state stays None until ready (status routes already re-probe at request time). Opt out with UNSLOTH_DISABLE_UPDATE_CHECK=1. - Train and Export were greyed out because mlx/mlx-lm/mlx-vlm arrive only transitively and a resolver backtrack silently drops them, so CHAT_ONLY stayed true. Add utils/mlx_repair.py: when Apple Silicon is detected without MLX, reinstall mlx/mlx-lm/mlx-vlm by name on a daemon thread and re-run hardware detection (opt out UNSLOTH_DISABLE_MLX_AUTOREPAIR=1). Surface a chat_only_reason in /api/health plus a sidebar tooltip so a greyed Train/Export explains itself instead of failing silently. * Studio: guard model defaults against a None model name load_model_defaults(None) called model_name.lower() with no guard, raising 'Error loading model defaults for None' before any model is selected. Return an empty dict for a falsy/non-str name. * Studio: drop obsolete upstream macOS + Windows Blackwell prebuilt pins Both pins worked around gaps in ggml-org upstream prebuilts, but Studio now routes every GPU host and all of macOS to the unslothai/llama.cpp fork (published_repo_for_host), which ships the needed bundles, so both pins are dead code on the default install path: - macOS b9415: macOS always routes to the fork (its own macOS bundles), and host_supports_macos_minos() is the backstop. The pin only fired under an explicit --published-repo ggml-org override. - Windows Blackwell b9360: Windows-NVIDIA routes to the fork, whose windows-x64-cuda13 bundle covers Blackwell (manifest max_sm 120, toolkit 13.3), so the pin's self-disable check makes it dormant on every default install; it could only activate under the same upstream override on a 13.0-13.2 driver. Remove the pin constants, functions, and call sites. Keep the Blackwell capability detection (_drop_blackwell_incapable_windows_cuda, _host_is_blackwell, _windows_cuda_attempt_covers_blackwell) that still drops a non-sm_120 cuda-12.4 build on a Blackwell host. After this, an explicit --published-repo ggml-org override on a Blackwell 13.0-13.2 host loses its GPU fallback and lands on CPU; the default fork path is unaffected. Update the install selection-logic and macOS-compat unit tests for the new no-pin behavior. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: walk back deeper on the macOS upstream prebuilt path After removing the b9415 macOS pin, the explicit --published-repo ggml-org upstream path still used the default 2-release fallback, so a pre-macOS-26 host behind a run of macOS-26-only builds would exhaust two too-new plans (minos is only checked post-download) and drop to a source build before reaching a loadable older release. Walk back as deep as the fork macOS path (DEFAULT_MAX_MACOS_RELEASE_FALLBACKS), turning the removed static pin into dynamic discovery. Addresses review feedback on the macOS upstream fallback. * Studio: pin transformers during MLX self-heal so it cannot break Studio mlx-lm/mlx-vlm declare transformers>=5, but the single-env install pins transformers==4.57.6. The self-heal used --upgrade with no constraint, so it could upgrade transformers in the live venv and break the rest of Studio just to make import mlx.core pass. Pin transformers to the installed version via a constraint file: the resolver either finds an mlx build compatible with it or fails (we stay chat-only), never upgrading transformers underneath Studio. Addresses review feedback on the MLX repair install. * Studio: harden MLX self-heal against an unsupported mlx-vlm Pinning transformers alone made uv backtrack mlx-vlm to 0.3.9 (below unsloth-zoo's mlx-vlm>=0.4.4), which imports but breaks VLM Train/Export -- so the self-heal could clear chat-only onto a broken stack. Mirror the main installer: set UV_OVERRIDE=overrides-darwin-arm64.txt so a current mlx-vlm coexists with the transformers pin, require the same minimum versions unsloth-zoo declares, and gate/validate on a full mlx_stack_available() check (not a bare import) so an old or partial stack stays chat-only. Addresses PR review. * Studio: filter Blackwell-incapable CUDA in resolve_upstream_asset_choice resolve_upstream_asset_choice returned the first windows-cuda choice unfiltered, so a Blackwell host could be handed an sm_120-incapable cuda-12.4 build while the sibling planners drop it. Apply _drop_blackwell_incapable_windows_cuda here too and fall through to the CPU bundle on a Blackwell host with no capable GPU asset. Addresses PR review. * Studio: re-poll health so MLX self-heal reaches an open UI The sidebar cached the initial /api/health, so a successful background MLX self-heal (chat_only flips false) did not re-enable Train/Export until a manual reload. While chat-only for the recoverable mlx_unavailable reason, re-poll /api/health and stop once Train/Export become available. Addresses PR review. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: make the disabled Train/Export tooltip reachable The greyed Train/Export items pass a tooltip explaining why (e.g. MLX missing), but a disabled <button> fires no pointer events and SidebarMenuButton only showed tooltips while collapsed, so the explanation never appeared. Wrap a disabled button in a focusable span and show its tooltip while expanded too; enabled items keep the collapsed-only behavior. Addresses PR review. * Studio: gate Train/Export on the full MLX stack, not bare mlx.core detect_hardware enabled MLX training whenever `import mlx.core` worked, but the MLX self-heal (utils/mlx_repair) treats a stack without mlx-lm/mlx-vlm at the versions unsloth-zoo requires as inadequate. That asymmetry let the UI enable Train/Export on exactly the partial/backtracked stack the self-heal is trying to repair (greyed-in-but-broken VLM export). Gate on the same mlx_stack_available() criterion so a partial stack stays chat-only (reason mlx_unavailable) and the background repair restores it. Addresses PR review. * Fix MLX repair and health auth for PR #6494 * Fix macOS upstream prebuilt fallback for PR #6494 * Fix MLX stack validation for PR #6494 * Fix MLX self-heal validation for PR #6494 * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Review fixes: isolate hardware-state test, robust transformers pin - test_chat_only_reason.py: detect_hardware() assigns module globals directly, which monkeypatch does not revert; the autouse fixture now saves and restores DEVICE/CHAT_ONLY/CHAT_ONLY_REASON/IS_ROCM so a chat-only verdict here cannot leak into other backend tests (e.g. test_utils.py) on a GPU host. - mlx_repair.py: read the transformers version from importlib.metadata instead of importing transformers, so the install pin is not silently dropped when transformers has valid metadata but fails to import. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Fix CI: model full MLX stack in dispatch tests, keep selection test offline dispatch (macOS) job: - detect_hardware now gates MLX on the full stack (mlx_stack_available imports mlx_lm/mlx_vlm and checks dist versions), so faking only mlx.core makes the apple_silicon_mlx profile resolve to CPU. The dispatch tests assert the routing decision when the stack IS usable, so model a complete stack: test_hardware_dispatch_matrix patches utils.mlx_repair.mlx_stack_available and test_is_mlx_dispatch_gate patches hardware._has_usable_mlx_stack. The stack predicate's own internals stay covered by test_mlx_repair.py. Repo tests (CPU) job: - test_no_cuda_attempt_on_published_path_for_13_1 fell through to a live github_release_assets() upstream fetch after the Blackwell filter dropped every published attempt, which the offline security scanner blocks. Stub that fetch so the walk-back deterministically finds no usable CUDA build and raises PrebuiltFallback without network. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Harden MLX self-heal: prepare transformers constraint inside the try attempt_mlx_repair runs on a daemon thread, but _transformers_constraint_args was called before the try. A failure there (e.g. tempfile.mkstemp on a full disk or a bad TMPDIR) would propagate unhandled and silently kill the self-heal thread. Move the call inside the try and initialize constraint_path so any such failure is caught and leaves Studio chat-only instead of crashing the thread. --------- Co-authored-by: Daniel Han <michaelhan2050@gmail.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: wasimysaid <wasimysdev@gmail.com>
This commit is contained in:
Daniel Han
GitHub
parent
f89c829fcf
commit
378e33c8a5
19 changed files with 1088 additions and 299 deletions
|
|
@ -379,24 +379,18 @@ def _start_helper_precache_if_enabled() -> None:
|
|||
threading.Thread(target = _precache, daemon = True, name = "helper-gguf-precache").start()
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(app: FastAPI):
|
||||
"""Startup: detect hardware, seed default admin if needed. Shutdown: clean up compiled cache."""
|
||||
clear_unsloth_compiled_cache()
|
||||
def _run_llama_cpp_startup_probes(app: FastAPI) -> None:
|
||||
"""llama.cpp capability (MTP support) + freshness (release age) probes.
|
||||
|
||||
# Remove stale .venv_overlay from old versions; switching now uses .venv_t5/.
|
||||
overlay_dir = Path(__file__).resolve().parent.parent.parent / ".venv_overlay"
|
||||
if overlay_dir.is_dir():
|
||||
shutil.rmtree(overlay_dir, ignore_errors = True)
|
||||
|
||||
# Detect hardware first — sets the DEVICE global used everywhere.
|
||||
detect_hardware()
|
||||
|
||||
# Reap download workers orphaned by a previous crash before new downloads start.
|
||||
reap_hub_orphan_workers()
|
||||
|
||||
# llama.cpp probes: capability (MTP support) + freshness (release age).
|
||||
# Both cached; freshness has a 24h disk TTL.
|
||||
Runs OFF the startup critical path (see _start_llama_cpp_probes_if_enabled).
|
||||
Both are cached and freshness has a 24h disk TTL, but on a cold/expired cache
|
||||
the freshness check makes a blocking GitHub request, and on macOS the first
|
||||
`llama-server --help` exec can stall on Gatekeeper verification -- neither must
|
||||
ever gate `Application startup complete`. Writes app.state only; nothing reads
|
||||
those values synchronously at startup (the status routes call
|
||||
check_prebuilt_freshness directly at request time), so populating them late is
|
||||
safe.
|
||||
"""
|
||||
try:
|
||||
from core.inference.llama_cpp import LlamaCppBackend
|
||||
from utils.llama_cpp_freshness import (
|
||||
|
|
@ -429,6 +423,61 @@ async def lifespan(app: FastAPI):
|
|||
import structlog as _structlog
|
||||
_structlog.get_logger(__name__).debug("llama.cpp startup probes failed: %s", _probe_exc)
|
||||
|
||||
|
||||
def _start_llama_cpp_probes_if_enabled(app: FastAPI) -> None:
|
||||
"""Run the llama.cpp startup probes on a daemon thread, off the startup
|
||||
critical path so they never delay `Application startup complete`. Skipped
|
||||
entirely when update checks are disabled, so a fully offline boot makes no
|
||||
background network calls."""
|
||||
if os.environ.get("UNSLOTH_DISABLE_UPDATE_CHECK") == "1":
|
||||
return
|
||||
|
||||
threading.Thread(
|
||||
target = _run_llama_cpp_startup_probes,
|
||||
args = (app,),
|
||||
daemon = True,
|
||||
name = "llama-cpp-startup-probe",
|
||||
).start()
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(app: FastAPI):
|
||||
"""Startup: detect hardware, seed default admin if needed. Shutdown: clean up compiled cache."""
|
||||
clear_unsloth_compiled_cache()
|
||||
|
||||
# Remove stale .venv_overlay from old versions; switching now uses .venv_t5/.
|
||||
overlay_dir = Path(__file__).resolve().parent.parent.parent / ".venv_overlay"
|
||||
if overlay_dir.is_dir():
|
||||
shutil.rmtree(overlay_dir, ignore_errors = True)
|
||||
|
||||
# Detect hardware first — sets the DEVICE global used everywhere.
|
||||
detect_hardware()
|
||||
|
||||
# Apple Silicon with MLX missing => Train/Export are greyed out (chat-only).
|
||||
# Reinstall mlx by name on a background thread (off the critical path) and
|
||||
# re-detect, so a reinstall/update that dropped mlx self-heals. No-op
|
||||
# elsewhere; opt out with UNSLOTH_DISABLE_MLX_AUTOREPAIR=1.
|
||||
try:
|
||||
from utils.mlx_repair import start_mlx_autorepair_if_needed
|
||||
start_mlx_autorepair_if_needed()
|
||||
except Exception as _mlx_exc:
|
||||
import structlog as _structlog
|
||||
_structlog.get_logger(__name__).debug("mlx autorepair skipped: %s", _mlx_exc)
|
||||
|
||||
# Reap download workers orphaned by a previous crash before new downloads start.
|
||||
reap_hub_orphan_workers()
|
||||
|
||||
# llama.cpp probes: capability (MTP support) + freshness (release age).
|
||||
# These used to run inline here and could block `Application startup complete`
|
||||
# for tens of seconds on macOS (cold GitHub freshness cache / slow network, and
|
||||
# Gatekeeper verifying the unsigned binary on first `--help` exec). They only
|
||||
# write app.state and nothing reads it synchronously at startup, so run them on
|
||||
# a daemon thread off the startup critical path (mirrors the helper-precache and
|
||||
# RAG-warm threads). Default to None until the thread populates them.
|
||||
app.state.llama_cpp_capabilities = None
|
||||
app.state.llama_cpp_freshness = None
|
||||
_start_llama_cpp_probes_if_enabled(app)
|
||||
|
||||
from storage.studio_db import cleanup_orphaned_runs
|
||||
|
||||
try:
|
||||
|
|
@ -909,6 +958,8 @@ async def health_check(request: Request):
|
|||
device_type = platform_map.get(sys.platform, sys.platform)
|
||||
return {
|
||||
**base,
|
||||
# Why chat_only is set. This fingerprints the host, so keep it authed.
|
||||
"chat_only_reason": getattr(_hw_module, "CHAT_ONLY_REASON", None),
|
||||
"version": UNSLOTH_VERSION,
|
||||
"studio_version": STUDIO_VERSION,
|
||||
"device_type": device_type,
|
||||
|
|
|
|||
83
studio/backend/tests/test_chat_only_reason.py
Normal file
83
studio/backend/tests/test_chat_only_reason.py
Normal file
|
|
@ -0,0 +1,83 @@
|
|||
# SPDX-License-Identifier: AGPL-3.0-only
|
||||
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
"""detect_hardware() records WHY a host is chat-only so the UI can explain the
|
||||
greyed-out Train/Export instead of disabling them silently.
|
||||
|
||||
The key case is Apple Silicon without an importable MLX -> "mlx_unavailable",
|
||||
which is the usual cause of "Train and Export greyed out" on Macs after a
|
||||
reinstall/update dropped MLX.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
_BACKEND = Path(__file__).resolve().parent.parent
|
||||
if str(_BACKEND) not in sys.path:
|
||||
sys.path.insert(0, str(_BACKEND))
|
||||
|
||||
import utils.hardware.hardware as hw # noqa: E402
|
||||
|
||||
|
||||
@pytest.fixture(autouse = True)
|
||||
def _no_torch(monkeypatch):
|
||||
# Force the non-CUDA/XPU path regardless of the test host's real GPUs.
|
||||
monkeypatch.setattr(hw, "_has_torch", lambda: False)
|
||||
# detect_hardware() assigns these module globals directly (not via monkeypatch),
|
||||
# so save and restore them; otherwise a chat-only verdict here leaks into other
|
||||
# backend tests (e.g. test_utils.py) when they share a process on a GPU host.
|
||||
saved = (hw.DEVICE, hw.CHAT_ONLY, hw.CHAT_ONLY_REASON, hw.IS_ROCM)
|
||||
try:
|
||||
yield
|
||||
finally:
|
||||
hw.DEVICE, hw.CHAT_ONLY, hw.CHAT_ONLY_REASON, hw.IS_ROCM = saved
|
||||
|
||||
|
||||
def test_apple_silicon_without_mlx_is_chat_only_with_reason(monkeypatch):
|
||||
monkeypatch.setattr(hw, "is_apple_silicon", lambda: True)
|
||||
monkeypatch.setattr(hw, "_has_usable_mlx_stack", lambda: False)
|
||||
hw.detect_hardware()
|
||||
assert hw.CHAT_ONLY is True
|
||||
assert hw.CHAT_ONLY_REASON == "mlx_unavailable"
|
||||
|
||||
|
||||
def test_apple_silicon_with_mlx_enables_training(monkeypatch):
|
||||
monkeypatch.setattr(hw, "is_apple_silicon", lambda: True)
|
||||
monkeypatch.setattr(hw, "_has_usable_mlx_stack", lambda: True)
|
||||
hw.detect_hardware()
|
||||
assert hw.CHAT_ONLY is False
|
||||
assert hw.CHAT_ONLY_REASON is None
|
||||
|
||||
|
||||
def test_apple_silicon_with_incomplete_mlx_stack_stays_chat_only(monkeypatch):
|
||||
# Bare `import mlx.core` works but the full mlx/mlx-lm/mlx-vlm stack does not
|
||||
# (e.g. a backtracked/old mlx-vlm). The training gate must match the self-heal
|
||||
# validator and stay chat-only so the UI does not enable a broken Train/Export.
|
||||
monkeypatch.setattr(hw, "is_apple_silicon", lambda: True)
|
||||
monkeypatch.setattr(hw, "_has_mlx", lambda: True)
|
||||
monkeypatch.setattr(hw, "_has_usable_mlx_stack", lambda: False)
|
||||
assert hw.detect_hardware() == hw.DeviceType.CPU
|
||||
assert hw.CHAT_ONLY is True
|
||||
assert hw.CHAT_ONLY_REASON == "mlx_unavailable"
|
||||
|
||||
|
||||
def test_intel_mac_reason(monkeypatch):
|
||||
monkeypatch.setattr(hw, "is_apple_silicon", lambda: False)
|
||||
monkeypatch.setattr(hw, "_has_mlx", lambda: False)
|
||||
monkeypatch.setattr(hw.platform, "system", lambda: "Darwin")
|
||||
hw.detect_hardware()
|
||||
assert hw.CHAT_ONLY is True
|
||||
assert hw.CHAT_ONLY_REASON == "intel_mac"
|
||||
|
||||
|
||||
def test_cpu_only_non_mac_reason(monkeypatch):
|
||||
monkeypatch.setattr(hw, "is_apple_silicon", lambda: False)
|
||||
monkeypatch.setattr(hw, "_has_mlx", lambda: False)
|
||||
monkeypatch.setattr(hw.platform, "system", lambda: "Linux")
|
||||
hw.detect_hardware()
|
||||
assert hw.CHAT_ONLY is True
|
||||
assert hw.CHAT_ONLY_REASON == "no_gpu"
|
||||
|
|
@ -451,7 +451,8 @@ def test_health_response_reports_desktop_capability_fields(monkeypatch):
|
|||
|
||||
import studio.backend.main as backend_main
|
||||
|
||||
monkeypatch.setattr(backend_main._hw_module, "CHAT_ONLY", False)
|
||||
monkeypatch.setattr(backend_main._hw_module, "CHAT_ONLY", True)
|
||||
monkeypatch.setattr(backend_main._hw_module, "CHAT_ONLY_REASON", "mlx_unavailable")
|
||||
|
||||
seed_user()
|
||||
from auth.authentication import create_access_token
|
||||
|
|
@ -462,6 +463,12 @@ def test_health_response_reports_desktop_capability_fields(monkeypatch):
|
|||
app.add_api_route("/api/health", backend_main.health_check, methods = ["GET"])
|
||||
client = TestClient(app)
|
||||
|
||||
unauthenticated = client.get("/api/health")
|
||||
assert unauthenticated.status_code == 200
|
||||
unauthenticated_body = unauthenticated.json()
|
||||
assert unauthenticated_body["chat_only"] is True
|
||||
assert "chat_only_reason" not in unauthenticated_body
|
||||
|
||||
response = client.get(
|
||||
"/api/health",
|
||||
headers = {"Authorization": f"Bearer {token}"},
|
||||
|
|
@ -471,6 +478,7 @@ def test_health_response_reports_desktop_capability_fields(monkeypatch):
|
|||
|
||||
assert body["desktop_protocol_version"] == 1
|
||||
assert body["supports_desktop_auth"] is True
|
||||
assert body["chat_only_reason"] == "mlx_unavailable"
|
||||
|
||||
|
||||
def test_provision_desktop_auth_writes_secret_and_creates_db_without_backend_deps(
|
||||
|
|
|
|||
|
|
@ -105,6 +105,44 @@ def test_published_repo_for_host():
|
|||
)
|
||||
|
||||
|
||||
def test_macos_intel_and_arm_both_route_to_fork():
|
||||
# macOS uses the unslothai fork's own Mac prebuilts for BOTH arm64 and Intel;
|
||||
# there is no longer any upstream-on-macOS default path, so the obsolete
|
||||
# pre-macOS-26 pin (b9415) is gone.
|
||||
assert (
|
||||
ilp.published_repo_for_host(
|
||||
_host(system = "Darwin", is_macos = True, is_arm64 = True, machine = "arm64")
|
||||
)
|
||||
== FORK
|
||||
)
|
||||
assert (
|
||||
ilp.published_repo_for_host(
|
||||
_host(system = "Darwin", is_macos = True, is_x86_64 = True, machine = "x86_64")
|
||||
)
|
||||
== FORK
|
||||
)
|
||||
|
||||
|
||||
def test_macos_upstream_pin_only_for_explicit_pre26_upstream():
|
||||
pre26 = _host(
|
||||
system = "Darwin",
|
||||
is_macos = True,
|
||||
is_arm64 = True,
|
||||
machine = "arm64",
|
||||
macos_version = (15, 5),
|
||||
)
|
||||
assert ilp.pinned_macos_release_tag(pre26, UPSTREAM) == "b9415"
|
||||
assert ilp.pinned_macos_release_tag(pre26, FORK) is None
|
||||
tahoe = _host(
|
||||
system = "Darwin",
|
||||
is_macos = True,
|
||||
is_arm64 = True,
|
||||
machine = "arm64",
|
||||
macos_version = (26, 0),
|
||||
)
|
||||
assert ilp.pinned_macos_release_tag(tahoe, UPSTREAM) is None
|
||||
|
||||
|
||||
def _run_resolve(monkeypatch, capsys, plans_or_exc):
|
||||
monkeypatch.setattr(
|
||||
ilp,
|
||||
|
|
|
|||
278
studio/backend/tests/test_mlx_repair.py
Normal file
278
studio/backend/tests/test_mlx_repair.py
Normal file
|
|
@ -0,0 +1,278 @@
|
|||
# SPDX-License-Identifier: AGPL-3.0-only
|
||||
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
"""MLX self-heal: on Apple Silicon with MLX missing, reinstall it by name on a
|
||||
background thread (off the startup critical path). No-op elsewhere / when present
|
||||
/ when disabled. Models on core.training.worker's runtime backend self-heal.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
_BACKEND = Path(__file__).resolve().parent.parent
|
||||
if str(_BACKEND) not in sys.path:
|
||||
sys.path.insert(0, str(_BACKEND))
|
||||
|
||||
import utils.mlx_repair as mr # noqa: E402
|
||||
|
||||
|
||||
@pytest.fixture(autouse = True)
|
||||
def _reset_attempt_guard(monkeypatch):
|
||||
monkeypatch.setattr(mr, "_attempted", False)
|
||||
monkeypatch.delenv(mr.DISABLE_ENV_VAR, raising = False)
|
||||
|
||||
|
||||
def test_uv_cmd_targets_this_interpreter_with_mlx_packages(monkeypatch):
|
||||
monkeypatch.setattr(mr, "_uv_executable", lambda: "/usr/bin/uv")
|
||||
cmd = mr._uv_install_cmd("--upgrade", *mr.MLX_PACKAGES)
|
||||
assert cmd is not None
|
||||
assert cmd[:5] == ["/usr/bin/uv", "pip", "install", "--python", sys.executable]
|
||||
assert set(mr.MLX_PACKAGES) <= set(cmd)
|
||||
# Minimum versions are pinned so the resolver cannot backtrack to an old
|
||||
# mlx-vlm that imports but breaks VLM Train/Export.
|
||||
assert "mlx-vlm>=0.4.4" in cmd
|
||||
|
||||
|
||||
def test_uv_executable_finds_installer_location_when_path_is_minimal(monkeypatch, tmp_path):
|
||||
uv = tmp_path / ".local" / "bin" / "uv"
|
||||
uv.parent.mkdir(parents = True)
|
||||
uv.write_text("#!/bin/sh\n", encoding = "utf-8")
|
||||
uv.chmod(0o755)
|
||||
monkeypatch.setattr(mr.shutil, "which", lambda _x: None)
|
||||
monkeypatch.setattr(mr.Path, "home", lambda: tmp_path)
|
||||
assert mr._uv_executable() == str(uv)
|
||||
|
||||
|
||||
def test_no_uv_repair_stays_chat_only_without_pip(monkeypatch):
|
||||
monkeypatch.setattr(mr, "_uv_executable", lambda: None)
|
||||
monkeypatch.setattr(mr, "_transformers_constraint_args", lambda: ([], None))
|
||||
called = {"run": False}
|
||||
|
||||
def _fake_run(*_args, **_kwargs):
|
||||
called["run"] = True
|
||||
raise AssertionError("plain pip fallback must not run")
|
||||
|
||||
monkeypatch.setattr(mr.subprocess, "run", _fake_run)
|
||||
assert mr.attempt_mlx_repair() is False
|
||||
assert called["run"] is False
|
||||
|
||||
|
||||
def test_constraint_pins_installed_transformers(monkeypatch):
|
||||
transformers = pytest.importorskip("transformers")
|
||||
args, path = mr._transformers_constraint_args()
|
||||
try:
|
||||
assert args[:1] == ["--constraint"]
|
||||
assert args[1] == path
|
||||
assert Path(path).read_text().strip() == f"transformers=={transformers.__version__}"
|
||||
finally:
|
||||
if path:
|
||||
Path(path).unlink(missing_ok = True)
|
||||
|
||||
|
||||
def test_repair_install_pins_transformers_and_cleans_up(monkeypatch):
|
||||
pytest.importorskip("transformers")
|
||||
captured = {}
|
||||
created_paths = []
|
||||
real_args = mr._transformers_constraint_args
|
||||
|
||||
def _spy_args():
|
||||
args, path = real_args()
|
||||
if path:
|
||||
created_paths.append(path)
|
||||
return args, path
|
||||
|
||||
monkeypatch.setattr(mr, "_transformers_constraint_args", _spy_args)
|
||||
monkeypatch.setattr(mr, "_uv_executable", lambda: "/usr/bin/uv")
|
||||
|
||||
class _Result:
|
||||
returncode = 0
|
||||
stdout = ""
|
||||
|
||||
def _fake_run(cmd, **kwargs):
|
||||
captured["cmd"] = cmd
|
||||
captured["env"] = kwargs.get("env")
|
||||
return _Result()
|
||||
|
||||
monkeypatch.setattr(mr.subprocess, "run", _fake_run)
|
||||
monkeypatch.setattr(mr, "mlx_stack_available", lambda: True)
|
||||
|
||||
assert mr.attempt_mlx_repair() is True
|
||||
cmd = captured["cmd"]
|
||||
# transformers is pinned via a constraint file so the mlx install cannot
|
||||
# upgrade it underneath Studio, and the temp constraint file is cleaned up.
|
||||
assert "--constraint" in cmd
|
||||
assert "--upgrade" in cmd
|
||||
reinstall_pairs = set(zip(cmd, cmd[1:]))
|
||||
for name in mr._MLX_PACKAGE_NAMES:
|
||||
assert ("--reinstall-package", name) in reinstall_pairs
|
||||
for pkg in mr.MLX_PACKAGES:
|
||||
assert pkg in cmd
|
||||
assert created_paths and not Path(created_paths[0]).exists()
|
||||
# The install mirrors the main installer by relaxing the transformers pin via
|
||||
# UV_OVERRIDE so a current mlx-vlm can coexist with transformers==4.57.6.
|
||||
env = captured["env"]
|
||||
assert env is not None
|
||||
assert env.get("UV_OVERRIDE", "").endswith("overrides-darwin-arm64.txt")
|
||||
|
||||
|
||||
def test_repair_rejects_inadequate_stack(monkeypatch):
|
||||
# A successful uv run that still leaves an old/missing mlx-vlm must NOT clear
|
||||
# chat-only: attempt_mlx_repair returns False so Train/Export stay disabled.
|
||||
class _Result:
|
||||
returncode = 0
|
||||
stdout = ""
|
||||
|
||||
monkeypatch.setattr(mr.subprocess, "run", lambda *a, **k: _Result())
|
||||
monkeypatch.setattr(mr, "mlx_stack_available", lambda: False)
|
||||
assert mr.attempt_mlx_repair() is False
|
||||
|
||||
|
||||
def test_repair_invalidates_import_caches_before_stack_check(monkeypatch):
|
||||
events = []
|
||||
|
||||
class _Result:
|
||||
returncode = 0
|
||||
stdout = ""
|
||||
|
||||
def _stack_available():
|
||||
events.append("check")
|
||||
assert events == ["invalidate", "check"]
|
||||
return True
|
||||
|
||||
monkeypatch.setattr(mr.subprocess, "run", lambda *a, **k: _Result())
|
||||
monkeypatch.setattr(mr, "_uv_executable", lambda: "/usr/bin/uv")
|
||||
monkeypatch.setattr(mr, "_transformers_constraint_args", lambda: ([], None))
|
||||
monkeypatch.setattr(mr.importlib, "invalidate_caches", lambda: events.append("invalidate"))
|
||||
monkeypatch.setattr(mr, "mlx_stack_available", _stack_available)
|
||||
|
||||
assert mr.attempt_mlx_repair() is True
|
||||
assert events == ["invalidate", "check"]
|
||||
|
||||
|
||||
def test_stack_unavailable_without_mlx(monkeypatch):
|
||||
import importlib.metadata as metadata
|
||||
|
||||
def _missing(_name):
|
||||
raise metadata.PackageNotFoundError(_name)
|
||||
|
||||
monkeypatch.setattr(metadata, "version", _missing)
|
||||
assert mr.mlx_stack_available() is False
|
||||
|
||||
|
||||
def test_stack_unavailable_checks_versions_before_imports(monkeypatch):
|
||||
import importlib.metadata as metadata
|
||||
|
||||
def _version(name):
|
||||
if name == "mlx":
|
||||
return "0.21.0"
|
||||
return mr._MLX_MIN_VERSIONS[name]
|
||||
|
||||
def _import_module(_name):
|
||||
raise AssertionError("MLX modules must not import before versions pass")
|
||||
|
||||
monkeypatch.setattr(metadata, "version", _version)
|
||||
monkeypatch.setattr(mr.importlib, "import_module", _import_module)
|
||||
assert mr.mlx_stack_available() is False
|
||||
|
||||
|
||||
def test_stack_unavailable_when_companion_import_fails(monkeypatch):
|
||||
import importlib.metadata as metadata
|
||||
|
||||
monkeypatch.setattr(metadata, "version", lambda name: mr._MLX_MIN_VERSIONS[name])
|
||||
|
||||
def _import_module(name):
|
||||
if name == "mlx_vlm":
|
||||
raise ModuleNotFoundError(name)
|
||||
return object()
|
||||
|
||||
monkeypatch.setattr(mr.importlib, "import_module", _import_module)
|
||||
assert mr.mlx_stack_available() is False
|
||||
|
||||
|
||||
def test_stack_available_requires_runtime_imports_and_versions(monkeypatch):
|
||||
import importlib.metadata as metadata
|
||||
|
||||
imported = []
|
||||
|
||||
def _import_module(name):
|
||||
imported.append(name)
|
||||
return object()
|
||||
|
||||
monkeypatch.setattr(mr.importlib, "import_module", _import_module)
|
||||
monkeypatch.setattr(metadata, "version", lambda name: mr._MLX_MIN_VERSIONS[name])
|
||||
|
||||
assert mr.mlx_stack_available() is True
|
||||
assert imported == list(mr._MLX_RUNTIME_IMPORTS)
|
||||
|
||||
|
||||
def test_no_op_off_apple_silicon(monkeypatch):
|
||||
monkeypatch.setattr(mr, "is_apple_silicon", lambda: False)
|
||||
called = {"n": 0}
|
||||
monkeypatch.setattr(
|
||||
mr, "attempt_mlx_repair", lambda **_k: called.__setitem__("n", called["n"] + 1) or True
|
||||
)
|
||||
assert mr.start_mlx_autorepair_if_needed() is False
|
||||
assert called["n"] == 0
|
||||
|
||||
|
||||
def test_no_op_when_mlx_stack_present(monkeypatch):
|
||||
monkeypatch.setattr(mr, "is_apple_silicon", lambda: True)
|
||||
monkeypatch.setattr(mr, "mlx_stack_available", lambda: True)
|
||||
started = mr.start_mlx_autorepair_if_needed()
|
||||
assert started is False
|
||||
|
||||
|
||||
def test_disable_env_skips(monkeypatch):
|
||||
monkeypatch.setattr(mr, "is_apple_silicon", lambda: True)
|
||||
monkeypatch.setattr(mr, "mlx_stack_available", lambda: False)
|
||||
monkeypatch.setenv(mr.DISABLE_ENV_VAR, "1")
|
||||
assert mr.start_mlx_autorepair_if_needed() is False
|
||||
|
||||
|
||||
def test_apple_silicon_missing_mlx_starts_repair_and_redetects(monkeypatch):
|
||||
import threading
|
||||
|
||||
monkeypatch.setattr(mr, "is_apple_silicon", lambda: True)
|
||||
monkeypatch.setattr(mr, "mlx_stack_available", lambda: False)
|
||||
|
||||
repaired = {"called": False}
|
||||
|
||||
def _fake_repair(**_kw):
|
||||
repaired["called"] = True
|
||||
return True
|
||||
|
||||
redetected = {"called": False}
|
||||
|
||||
# _run_repair_and_redetect imports utils.hardware.hardware lazily; stub repair
|
||||
# and capture that re-detection is invoked on success.
|
||||
monkeypatch.setattr(mr, "attempt_mlx_repair", _fake_repair)
|
||||
|
||||
import utils.hardware.hardware as hw
|
||||
|
||||
monkeypatch.setattr(hw, "detect_hardware", lambda: redetected.__setitem__("called", True))
|
||||
|
||||
started = mr.start_mlx_autorepair_if_needed()
|
||||
assert started is True
|
||||
|
||||
# Join the daemon thread deterministically.
|
||||
for thread in threading.enumerate():
|
||||
if thread.name == "mlx-autorepair":
|
||||
thread.join(timeout = 5)
|
||||
|
||||
assert repaired["called"] is True
|
||||
assert redetected["called"] is True
|
||||
|
||||
|
||||
def test_attempts_only_once_per_process(monkeypatch):
|
||||
monkeypatch.setattr(mr, "is_apple_silicon", lambda: True)
|
||||
monkeypatch.setattr(mr, "mlx_stack_available", lambda: False)
|
||||
monkeypatch.setattr(mr, "attempt_mlx_repair", lambda **_k: False)
|
||||
|
||||
first = mr.start_mlx_autorepair_if_needed()
|
||||
second = mr.start_mlx_autorepair_if_needed()
|
||||
assert first is True
|
||||
assert second is False # guard prevents a second concurrent attempt
|
||||
35
studio/backend/tests/test_model_defaults_none_guard.py
Normal file
35
studio/backend/tests/test_model_defaults_none_guard.py
Normal file
|
|
@ -0,0 +1,35 @@
|
|||
# SPDX-License-Identifier: AGPL-3.0-only
|
||||
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
"""load_model_defaults must not raise on a None/empty model id.
|
||||
|
||||
Before the guard, calling it before a model is selected logged
|
||||
`Error loading model defaults for None: 'NoneType' object has no attribute 'lower'`.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
_BACKEND = Path(__file__).resolve().parent.parent
|
||||
if str(_BACKEND) not in sys.path:
|
||||
sys.path.insert(0, str(_BACKEND))
|
||||
|
||||
from utils.models.model_config import load_model_defaults # noqa: E402
|
||||
|
||||
|
||||
def test_none_and_empty_return_empty_without_error(caplog):
|
||||
with caplog.at_level(logging.ERROR):
|
||||
assert load_model_defaults(None) == {} # type: ignore[arg-type]
|
||||
assert load_model_defaults("") == {}
|
||||
assert "Error loading model defaults" not in caplog.text
|
||||
assert "NoneType" not in caplog.text
|
||||
|
||||
|
||||
def test_unknown_string_still_returns_defaults_dict():
|
||||
# A non-None unknown model name still resolves (falls back to default.yaml),
|
||||
# i.e. the guard only short-circuits None/empty, nothing else.
|
||||
result = load_model_defaults("definitely-not-a-real-model-xyz")
|
||||
assert isinstance(result, dict)
|
||||
|
|
@ -0,0 +1,98 @@
|
|||
# SPDX-License-Identifier: AGPL-3.0-only
|
||||
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
"""The llama.cpp startup probes must run OFF the FastAPI lifespan critical path.
|
||||
|
||||
Regression guard for the macOS slow-startup bug: the capability + freshness probes
|
||||
(added in #5528/#5529) used to run inline in `lifespan`, so a cold/slow GitHub
|
||||
freshness check blocked `Application startup complete` for tens of seconds. They now
|
||||
run on a daemon thread, and are skipped entirely when update checks are disabled.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
_BACKEND = Path(__file__).resolve().parent.parent
|
||||
if str(_BACKEND) not in sys.path:
|
||||
sys.path.insert(0, str(_BACKEND))
|
||||
|
||||
import main # noqa: E402
|
||||
import utils.llama_cpp_freshness as freshness # noqa: E402
|
||||
from core.inference.llama_cpp import LlamaCppBackend # noqa: E402
|
||||
|
||||
SLEEP = 5.0
|
||||
|
||||
|
||||
class _FakeApp:
|
||||
class _State:
|
||||
pass
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.state = _FakeApp._State()
|
||||
self.state.llama_cpp_capabilities = None
|
||||
self.state.llama_cpp_freshness = None
|
||||
|
||||
|
||||
@pytest.fixture(autouse = True)
|
||||
def _fast_capability_probe(monkeypatch):
|
||||
# Keep the (local) capability probe instant + offline so the freshness sleep
|
||||
# is the only slow thing under test.
|
||||
monkeypatch.setattr(
|
||||
LlamaCppBackend,
|
||||
"_find_llama_server_binary",
|
||||
staticmethod(lambda: "/no/such/llama-server"),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
LlamaCppBackend,
|
||||
"probe_server_capabilities",
|
||||
staticmethod(lambda _b: {"found": False}),
|
||||
)
|
||||
monkeypatch.delenv("UNSLOTH_DISABLE_UPDATE_CHECK", raising = False)
|
||||
|
||||
|
||||
def test_probe_does_not_block_startup(monkeypatch):
|
||||
"""`_start_llama_cpp_probes_if_enabled` returns immediately even though the
|
||||
freshness check sleeps for SLEEP seconds, then populates app.state later."""
|
||||
|
||||
def _slow_freshness(_bin, **_kw):
|
||||
time.sleep(SLEEP)
|
||||
return {"stale": False, "behind": False}
|
||||
|
||||
monkeypatch.setattr(freshness, "check_prebuilt_freshness", _slow_freshness)
|
||||
|
||||
app = _FakeApp()
|
||||
t0 = time.monotonic()
|
||||
main._start_llama_cpp_probes_if_enabled(app)
|
||||
elapsed = time.monotonic() - t0
|
||||
|
||||
assert elapsed < 0.5, f"startup probe blocked the caller for {elapsed:.2f}s"
|
||||
|
||||
# The daemon thread eventually populates app.state once the slow check returns.
|
||||
deadline = time.monotonic() + SLEEP + 5
|
||||
while app.state.llama_cpp_freshness is None and time.monotonic() < deadline:
|
||||
time.sleep(0.1)
|
||||
assert app.state.llama_cpp_freshness == {"stale": False, "behind": False}
|
||||
|
||||
|
||||
def test_disable_env_skips_probe_entirely(monkeypatch):
|
||||
"""UNSLOTH_DISABLE_UPDATE_CHECK=1 starts no probe thread and makes no call."""
|
||||
calls: list[int] = []
|
||||
|
||||
def _freshness(_bin, **_kw):
|
||||
calls.append(1)
|
||||
return {"stale": False}
|
||||
|
||||
monkeypatch.setattr(freshness, "check_prebuilt_freshness", _freshness)
|
||||
monkeypatch.setenv("UNSLOTH_DISABLE_UPDATE_CHECK", "1")
|
||||
|
||||
app = _FakeApp()
|
||||
main._start_llama_cpp_probes_if_enabled(app)
|
||||
time.sleep(0.5)
|
||||
|
||||
assert calls == [], "freshness check ran despite UNSLOTH_DISABLE_UPDATE_CHECK=1"
|
||||
assert app.state.llama_cpp_freshness is None
|
||||
|
|
@ -107,6 +107,7 @@ class TestGetDevice:
|
|||
patch("utils.hardware.hardware._has_torch", return_value = False),
|
||||
patch("utils.hardware.hardware.is_apple_silicon", return_value = True),
|
||||
patch("utils.hardware.hardware._has_mlx", return_value = True),
|
||||
patch("utils.hardware.hardware._has_usable_mlx_stack", return_value = True),
|
||||
):
|
||||
assert _reset_and_detect() == DeviceType.MLX
|
||||
|
||||
|
|
|
|||
|
|
@ -66,6 +66,11 @@ class DeviceType(str, Enum):
|
|||
|
||||
DEVICE: Optional[DeviceType] = None
|
||||
CHAT_ONLY: bool = True # No CUDA GPU -> GGUF chat only (Mac, CPU-only, etc.)
|
||||
# Why CHAT_ONLY is True (Train/Export disabled). None when training is enabled.
|
||||
# "mlx_unavailable": Apple Silicon but the MLX stack is missing, too old, or broken
|
||||
# (the usual cause of "Train/Export greyed out" on Macs after a reinstall dropped MLX);
|
||||
# "intel_mac": Intel Mac (no PyTorch/MLX); "no_gpu": CPU-only non-Mac host.
|
||||
CHAT_ONLY_REASON: Optional[str] = None
|
||||
IS_ROCM: bool = False # True when running on AMD ROCm (HIP) -- routes GPU monitoring to amd.py
|
||||
|
||||
|
||||
|
|
@ -106,6 +111,24 @@ def _has_mlx() -> bool:
|
|||
return False
|
||||
|
||||
|
||||
def _has_usable_mlx_stack() -> bool:
|
||||
"""True only when the FULL Studio MLX training/export stack is usable
|
||||
(mlx + mlx-lm + mlx-vlm at the minimum versions unsloth-zoo requires), not
|
||||
just a bare ``import mlx.core``. A backtracked/old mlx-vlm still imports but
|
||||
breaks VLM Train/Export, so the training gate must match the self-heal's own
|
||||
criterion (utils.mlx_repair.mlx_stack_available) -- otherwise detect_hardware
|
||||
would enable Train/Export on exactly the inadequate stack the MLX self-heal
|
||||
is trying to repair, leaving the user with greyed-in-but-broken buttons."""
|
||||
try:
|
||||
from utils.mlx_repair import mlx_stack_available
|
||||
return mlx_stack_available()
|
||||
except Exception as exc:
|
||||
# mlx_repair should always import; if it somehow cannot, fall back to the
|
||||
# bare import check rather than forcing a working host into chat-only.
|
||||
logger.debug("MLX stack availability check failed, using bare import: %s", exc)
|
||||
return _has_mlx()
|
||||
|
||||
|
||||
def _print_cuda_device_list(is_rocm: bool) -> None:
|
||||
"""List every visible CUDA/ROCm GPU with its index at startup.
|
||||
|
||||
|
|
@ -151,8 +174,9 @@ def detect_hardware() -> DeviceType:
|
|||
2. MLX (Apple Silicon via MLX framework)
|
||||
3. CPU (fallback)
|
||||
"""
|
||||
global DEVICE, CHAT_ONLY, IS_ROCM
|
||||
CHAT_ONLY = True # reset -- only CUDA/ROCm sets it to False
|
||||
global DEVICE, CHAT_ONLY, CHAT_ONLY_REASON, IS_ROCM
|
||||
CHAT_ONLY = True # reset -- only CUDA/ROCm/XPU/MLX sets it to False
|
||||
CHAT_ONLY_REASON = None
|
||||
IS_ROCM = False
|
||||
|
||||
# --- CUDA / ROCm: try PyTorch ---
|
||||
|
|
@ -190,7 +214,10 @@ def detect_hardware() -> DeviceType:
|
|||
return DEVICE
|
||||
|
||||
# --- MLX: Apple Silicon ---
|
||||
if is_apple_silicon() and _has_mlx():
|
||||
# Require the full mlx/mlx-lm/mlx-vlm stack (not a bare `import mlx.core`) so
|
||||
# the gate matches utils.mlx_repair: a partial/backtracked stack stays
|
||||
# chat-only (reason "mlx_unavailable") and the background self-heal repairs it.
|
||||
if is_apple_silicon() and _has_usable_mlx_stack():
|
||||
DEVICE = DeviceType.MLX
|
||||
CHAT_ONLY = False
|
||||
# Use platform.machine() ("arm64"); platform.processor() returns "i386"
|
||||
|
|
@ -201,6 +228,23 @@ def detect_hardware() -> DeviceType:
|
|||
|
||||
# --- Fallback ---
|
||||
DEVICE = DeviceType.CPU
|
||||
# CHAT_ONLY is still True here (every training-capable branch returned early),
|
||||
# so record WHY so the UI can explain the greyed-out Train/Export instead of
|
||||
# silently disabling them.
|
||||
if is_apple_silicon():
|
||||
# Reached the CPU fallback on Apple Silicon, so the MLX stack is missing,
|
||||
# too old, or broken. This is usually an environment problem recoverable
|
||||
# with `unsloth studio update`.
|
||||
CHAT_ONLY_REASON = "mlx_unavailable"
|
||||
logger.warning(
|
||||
"Apple Silicon detected but the MLX stack is incomplete or too old; "
|
||||
"Train/Export disabled (chat-only). Run `unsloth studio update` to "
|
||||
"restore MLX training."
|
||||
)
|
||||
elif platform.system() == "Darwin":
|
||||
CHAT_ONLY_REASON = "intel_mac" # Intel Mac: no PyTorch/MLX -> GGUF-only by design.
|
||||
else:
|
||||
CHAT_ONLY_REASON = "no_gpu"
|
||||
print("Hardware detected: CPU (no GPU backend available)")
|
||||
return DEVICE
|
||||
|
||||
|
|
|
|||
279
studio/backend/utils/mlx_repair.py
Normal file
279
studio/backend/utils/mlx_repair.py
Normal file
|
|
@ -0,0 +1,279 @@
|
|||
# SPDX-License-Identifier: AGPL-3.0-only
|
||||
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
"""Best-effort MLX self-heal for Apple Silicon.
|
||||
|
||||
On macOS, Studio enables Train/Export only when the MLX training/export stack is
|
||||
usable (see utils.hardware.hardware.detect_hardware -> CHAT_ONLY). MLX is pulled
|
||||
only transitively via unsloth-zoo, and a resolver backtrack (mlx-vlm ->
|
||||
transformers>=5 vs the single-env transformers pin) can silently drop it, leaving
|
||||
Train/Export greyed out after a reinstall/update. This reinstalls mlx by name on
|
||||
a background thread, then re-detects so the gate re-opens without a manual
|
||||
`unsloth studio update`.
|
||||
|
||||
The install mirrors the main Apple Silicon installer (install_python_stack.py):
|
||||
it points UV_OVERRIDE at overrides-darwin-arm64.txt so the resolver keeps the
|
||||
Studio transformers pin AND installs a current mlx-vlm, and it requires the same
|
||||
minimum versions unsloth-zoo declares so a backtracked old mlx-vlm (which still
|
||||
imports but breaks VLM Train/Export) is never accepted as healthy.
|
||||
|
||||
Mirrors the runtime backend self-heal already used for tilelang
|
||||
(core.training.worker._ensure_tilelang_backend_unconditional): default-on,
|
||||
best-effort, opt out with UNSLOTH_DISABLE_MLX_AUTOREPAIR=1.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib
|
||||
import os
|
||||
import platform
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import threading
|
||||
from pathlib import Path
|
||||
|
||||
import structlog
|
||||
|
||||
logger = structlog.get_logger(__name__)
|
||||
|
||||
DISABLE_ENV_VAR = "UNSLOTH_DISABLE_MLX_AUTOREPAIR"
|
||||
# Minimum versions unsloth-zoo requires on Apple Silicon (its pyproject darwin
|
||||
# deps). mlx-vlm especially must be >=0.4.4: an older one still imports but
|
||||
# breaks VLM Train/Export, so installing it would wrongly clear chat-only.
|
||||
_MLX_MIN_VERSIONS = {"mlx": "0.22.0", "mlx-lm": "0.22.0", "mlx-vlm": "0.4.4"}
|
||||
_MLX_PACKAGE_NAMES = tuple(_MLX_MIN_VERSIONS)
|
||||
_MLX_RUNTIME_IMPORTS = ("mlx.core", "mlx_lm", "mlx_lm.sample_utils", "mlx_vlm")
|
||||
MLX_PACKAGES = tuple(f"{name}>={version}" for name, version in _MLX_MIN_VERSIONS.items())
|
||||
_MLX_REINSTALL_ARGS = tuple(
|
||||
arg for name in _MLX_PACKAGE_NAMES for arg in ("--reinstall-package", name)
|
||||
)
|
||||
_REPAIR_TIMEOUT_S = 900
|
||||
|
||||
# Attempt at most once per process; success is sticky (mlx then imports and the
|
||||
# guard short-circuits on the next boot).
|
||||
_attempted = False
|
||||
_attempted_lock = threading.Lock()
|
||||
|
||||
|
||||
def is_apple_silicon() -> bool:
|
||||
return platform.system() == "Darwin" and platform.machine() == "arm64"
|
||||
|
||||
|
||||
def mlx_available() -> bool:
|
||||
try:
|
||||
import mlx.core # noqa: F401
|
||||
return True
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
def _mlx_runtime_imports_available() -> bool:
|
||||
for module in _MLX_RUNTIME_IMPORTS:
|
||||
try:
|
||||
importlib.import_module(module)
|
||||
except Exception:
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def _mlx_versions_satisfy_minimums() -> bool:
|
||||
try:
|
||||
from importlib.metadata import PackageNotFoundError
|
||||
from importlib.metadata import version as _dist_version
|
||||
|
||||
from packaging.version import Version
|
||||
except Exception:
|
||||
return False
|
||||
for name, minimum in _MLX_MIN_VERSIONS.items():
|
||||
try:
|
||||
if Version(_dist_version(name)) < Version(minimum):
|
||||
return False
|
||||
except PackageNotFoundError:
|
||||
return False
|
||||
except Exception:
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def mlx_stack_available() -> bool:
|
||||
"""`import mlx.core` works AND mlx/mlx-lm/mlx-vlm meet unsloth-zoo's minimums.
|
||||
|
||||
Check distribution versions before imports so a too-old but importable MLX
|
||||
module is not loaded into this process before repair can replace it."""
|
||||
if not _mlx_versions_satisfy_minimums():
|
||||
return False
|
||||
return _mlx_runtime_imports_available()
|
||||
|
||||
|
||||
def _uv_executable() -> str | None:
|
||||
"""Find uv even when macOS GUI launchers start with a minimal PATH."""
|
||||
found = shutil.which("uv")
|
||||
if found:
|
||||
return found
|
||||
for candidate in (
|
||||
Path.home() / ".local" / "bin" / "uv",
|
||||
Path.home() / ".cargo" / "bin" / "uv",
|
||||
Path("/opt/homebrew/bin/uv"),
|
||||
Path("/usr/local/bin/uv"),
|
||||
):
|
||||
try:
|
||||
if candidate.is_file() and os.access(candidate, os.X_OK):
|
||||
return str(candidate)
|
||||
except OSError:
|
||||
continue
|
||||
return None
|
||||
|
||||
|
||||
def _uv_install_cmd(*args: str) -> list[str] | None:
|
||||
uv = _uv_executable()
|
||||
if not uv:
|
||||
return None
|
||||
return [uv, "pip", "install", "--python", sys.executable, *args]
|
||||
|
||||
|
||||
def _mlx_install_env() -> dict[str, str]:
|
||||
"""Environment for the mlx install. Mirror the main installer
|
||||
(install_python_stack.py) by pointing UV_OVERRIDE at overrides-darwin-arm64.txt,
|
||||
which relaxes mlx-vlm/mlx-lm's transformers>=5 requirement to >=4.57.6. Without
|
||||
it, uv keeps the Studio transformers pin only by silently backtracking mlx-vlm
|
||||
to an old, unsupported version (uv honours UV_OVERRIDE; plain pip ignores it,
|
||||
so the transformers constraint below is the pip-path safety net)."""
|
||||
env = dict(os.environ)
|
||||
override = (
|
||||
Path(__file__).resolve().parents[1]
|
||||
/ "requirements"
|
||||
/ "single-env"
|
||||
/ "overrides-darwin-arm64.txt"
|
||||
)
|
||||
if override.is_file():
|
||||
env.setdefault("UV_OVERRIDE", str(override))
|
||||
return env
|
||||
|
||||
|
||||
def _transformers_constraint_args() -> tuple[list[str], str | None]:
|
||||
"""Pin transformers to the running version for the mlx install.
|
||||
|
||||
The install must never upgrade transformers underneath a running Studio
|
||||
(the single-env install pins transformers==4.57.6). With UV_OVERRIDE set this
|
||||
is belt-and-suspenders; on the plain-pip path (no UV_OVERRIDE support) it is
|
||||
the actual guard -- the resolver either finds an mlx build compatible with the
|
||||
pin or fails, leaving us chat-only rather than breaking Studio. Returns
|
||||
(pip args, temp file path to clean up).
|
||||
|
||||
Read the version from installed metadata rather than `import transformers`:
|
||||
transformers can have valid metadata yet fail to import (e.g. an incompatible
|
||||
huggingface_hub), and in that case we still want to pin it so the mlx install
|
||||
cannot quietly upgrade it out from under Studio."""
|
||||
from importlib.metadata import PackageNotFoundError, version as _dist_version
|
||||
|
||||
try:
|
||||
transformers_version = _dist_version("transformers")
|
||||
except PackageNotFoundError:
|
||||
return [], None
|
||||
except Exception:
|
||||
return [], None
|
||||
fd, path = tempfile.mkstemp(prefix = "mlx_repair_", suffix = ".txt")
|
||||
with os.fdopen(fd, "w") as fh:
|
||||
fh.write(f"transformers=={transformers_version}\n")
|
||||
return ["--constraint", path], path
|
||||
|
||||
|
||||
def attempt_mlx_repair(*, timeout: int = _REPAIR_TIMEOUT_S) -> bool:
|
||||
"""Install a usable mlx/mlx-lm/mlx-vlm stack by name into the running venv.
|
||||
Best-effort; returns True iff the resulting stack meets unsloth-zoo's minimums
|
||||
(so a backtracked old mlx-vlm is rejected, not accepted). transformers is held
|
||||
at its pinned version so the install can never upgrade it underneath Studio."""
|
||||
# Prepare the constraint inside the try: this runs on a daemon thread, so an
|
||||
# exception here (e.g. tempfile.mkstemp failing on a full disk or bad TMPDIR)
|
||||
# must leave Studio chat-only, not crash the background self-heal thread.
|
||||
constraint_path = None
|
||||
try:
|
||||
constraint_args, constraint_path = _transformers_constraint_args()
|
||||
cmd = _uv_install_cmd("--upgrade", *_MLX_REINSTALL_ARGS, *constraint_args, *MLX_PACKAGES)
|
||||
if cmd is None:
|
||||
logger.warning(
|
||||
"MLX self-heal requires uv so Studio can apply dependency overrides; "
|
||||
"staying chat-only. Run `unsloth studio update` to restore uv."
|
||||
)
|
||||
return False
|
||||
logger.info("MLX self-heal: installing %s", ", ".join(MLX_PACKAGES))
|
||||
result = subprocess.run(
|
||||
cmd,
|
||||
env = _mlx_install_env(),
|
||||
stdout = subprocess.PIPE,
|
||||
stderr = subprocess.STDOUT,
|
||||
text = True,
|
||||
timeout = timeout,
|
||||
)
|
||||
except subprocess.TimeoutExpired:
|
||||
logger.warning("MLX self-heal timed out after %ss; staying chat-only", timeout)
|
||||
return False
|
||||
except Exception as exc: # pragma: no cover - environment dependent
|
||||
logger.warning("MLX self-heal could not start: %s", exc)
|
||||
return False
|
||||
finally:
|
||||
if constraint_path and os.path.exists(constraint_path):
|
||||
try:
|
||||
os.remove(constraint_path)
|
||||
except OSError:
|
||||
pass
|
||||
if result.returncode != 0:
|
||||
tail = (result.stdout or "")[-2000:]
|
||||
logger.warning("MLX self-heal failed (staying chat-only):\n%s", tail)
|
||||
return False
|
||||
importlib.invalidate_caches()
|
||||
if not mlx_stack_available():
|
||||
logger.warning(
|
||||
"MLX self-heal produced an incomplete or too-old MLX stack "
|
||||
"(need %s); staying chat-only.",
|
||||
", ".join(f"{name}>={ver}" for name, ver in _MLX_MIN_VERSIONS.items()),
|
||||
)
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def _run_repair_and_redetect() -> None:
|
||||
if not attempt_mlx_repair():
|
||||
return
|
||||
try:
|
||||
from utils.hardware import hardware as hw
|
||||
hw.detect_hardware() # flips CHAT_ONLY / DEVICE now that mlx imports
|
||||
logger.info(
|
||||
"MLX self-heal succeeded; Train/Export enabled (reload the page). chat_only=%s",
|
||||
hw.CHAT_ONLY,
|
||||
)
|
||||
except Exception as exc: # pragma: no cover - defensive
|
||||
logger.warning("MLX installed but hardware re-detection failed: %s", exc)
|
||||
|
||||
|
||||
def start_mlx_autorepair_if_needed() -> bool:
|
||||
"""If this is an Apple Silicon host whose MLX stack is missing or too old,
|
||||
reinstall it on a daemon thread (off the startup critical path) and re-detect
|
||||
on success. Returns True iff a repair thread was started. No-op (returns False)
|
||||
off Apple Silicon, when the stack is already adequate, when already attempted
|
||||
this process, or when disabled via UNSLOTH_DISABLE_MLX_AUTOREPAIR=1."""
|
||||
global _attempted
|
||||
if os.environ.get(DISABLE_ENV_VAR) == "1":
|
||||
return False
|
||||
if not is_apple_silicon():
|
||||
return False
|
||||
if mlx_stack_available():
|
||||
return False
|
||||
with _attempted_lock:
|
||||
if _attempted:
|
||||
return False
|
||||
_attempted = True
|
||||
logger.warning(
|
||||
"Apple Silicon without a usable MLX stack; attempting a one-time background "
|
||||
"reinstall of mlx/mlx-lm/mlx-vlm to re-enable Train/Export. "
|
||||
"Set %s=1 to disable.",
|
||||
DISABLE_ENV_VAR,
|
||||
)
|
||||
threading.Thread(
|
||||
target = _run_repair_and_redetect,
|
||||
daemon = True,
|
||||
name = "mlx-autorepair",
|
||||
).start()
|
||||
return True
|
||||
|
|
@ -2354,6 +2354,12 @@ def load_model_defaults(model_name: str) -> Dict[str, Any]:
|
|||
MODEL_NAME_MAPPING aliases, else falls back to default.yaml. Returns the
|
||||
parameter dict, or {} if none found.
|
||||
"""
|
||||
# No model selected yet (or a non-string id): nothing to load. Guard before
|
||||
# the .lower() calls below so this doesn't raise and get logged as
|
||||
# "Error loading model defaults for None: 'NoneType' object has no attribute
|
||||
# 'lower'".
|
||||
if not isinstance(model_name, str) or not model_name:
|
||||
return {}
|
||||
try:
|
||||
script_dir = Path(__file__).parent.parent.parent
|
||||
defaults_dir = script_dir / "assets" / "configs" / "model_defaults"
|
||||
|
|
|
|||
|
|
@ -108,7 +108,7 @@ import {
|
|||
} from "@/features/chat";
|
||||
import { useSettingsDialogStore } from "@/features/settings";
|
||||
import { useEffectiveProfile, UserAvatar } from "@/features/profile";
|
||||
import { usePlatformStore } from "@/config/env";
|
||||
import { fetchDeviceType, usePlatformStore } from "@/config/env";
|
||||
import { clearAuthTokens, logout } from "@/features/auth";
|
||||
import { TOUR_OPEN_EVENT } from "@/features/tour";
|
||||
import {
|
||||
|
|
@ -211,6 +211,7 @@ function NavItem({
|
|||
dataTour,
|
||||
className,
|
||||
spinner,
|
||||
tooltip,
|
||||
}: {
|
||||
icon: typeof ZapIcon;
|
||||
label: string;
|
||||
|
|
@ -221,12 +222,15 @@ function NavItem({
|
|||
dataTour?: string;
|
||||
className?: string;
|
||||
spinner?: boolean;
|
||||
// Overrides the hover tooltip (defaults to `label`). Used to explain why a
|
||||
// disabled item (e.g. Train/Export on a chat-only host) is greyed out.
|
||||
tooltip?: string;
|
||||
}) {
|
||||
return (
|
||||
<SidebarMenuItem className={className}>
|
||||
<div className="relative">
|
||||
<SidebarMenuButton
|
||||
tooltip={label}
|
||||
tooltip={tooltip ?? label}
|
||||
disabled={disabled}
|
||||
onClick={onClick}
|
||||
isActive={active}
|
||||
|
|
@ -267,6 +271,33 @@ export function AppSidebar() {
|
|||
};
|
||||
|
||||
const chatOnly = usePlatformStore((s) => s.isChatOnly());
|
||||
const chatOnlyReason = usePlatformStore((s) => s.chatOnlyReason);
|
||||
// When Train/Export are greyed out (chat-only host), explain why on hover
|
||||
// instead of disabling them silently. mlx_unavailable is the common macOS case
|
||||
// after a reinstall/update dropped MLX and is recoverable via `unsloth studio update`.
|
||||
const trainExportDisabledHint: string | undefined = !chatOnly
|
||||
? undefined
|
||||
: chatOnlyReason === "mlx_unavailable"
|
||||
? "Training needs MLX. Run `unsloth studio update` to enable Train and Export."
|
||||
: chatOnlyReason === "intel_mac"
|
||||
? "Training needs Apple Silicon or a GPU. Intel Macs are chat-only."
|
||||
: chatOnlyReason === "no_gpu"
|
||||
? "Training needs an NVIDIA or AMD GPU."
|
||||
: undefined;
|
||||
|
||||
// The backend MLX self-heal (utils/mlx_repair) can reinstall MLX in the
|
||||
// background and flip chat_only false without a restart. The platform store
|
||||
// cached the initial /api/health, so re-poll while we are chat-only for the
|
||||
// recoverable mlx_unavailable case; the effect stops once Train/Export become
|
||||
// available (chatOnly flips false and this effect's guard returns early).
|
||||
useEffect(() => {
|
||||
if (!chatOnly || chatOnlyReason !== "mlx_unavailable") return;
|
||||
const id = window.setInterval(() => {
|
||||
void fetchDeviceType({ force: true }).catch(() => undefined);
|
||||
}, 15000);
|
||||
return () => window.clearInterval(id);
|
||||
}, [chatOnly, chatOnlyReason]);
|
||||
|
||||
const [shutdownOpen, setShutdownOpen] = useState(false);
|
||||
|
||||
const isChatRoute = pathname.startsWith("/chat");
|
||||
|
|
@ -1105,6 +1136,7 @@ export function AppSidebar() {
|
|||
pathname === "/studio" || pathname.startsWith("/studio/")
|
||||
}
|
||||
disabled={chatOnly}
|
||||
tooltip={trainExportDisabledHint}
|
||||
spinner={trainingInProgress}
|
||||
onClick={() => {
|
||||
if (chatOnly) return;
|
||||
|
|
@ -1133,6 +1165,7 @@ export function AppSidebar() {
|
|||
label={t("shell.navigation.train")}
|
||||
active={pathname === "/studio" || pathname.startsWith("/studio/")}
|
||||
disabled={chatOnly}
|
||||
tooltip={trainExportDisabledHint}
|
||||
spinner={trainingInProgress}
|
||||
onClick={() => {
|
||||
if (chatOnly) return;
|
||||
|
|
@ -1154,6 +1187,7 @@ export function AppSidebar() {
|
|||
label={t("shell.navigation.export")}
|
||||
active={pathname === "/export" || pathname.startsWith("/export/")}
|
||||
disabled={chatOnly}
|
||||
tooltip={trainExportDisabledHint}
|
||||
spinner={exportInProgress}
|
||||
onClick={() => {
|
||||
if (chatOnly) return;
|
||||
|
|
|
|||
|
|
@ -571,6 +571,7 @@ function SidebarMenuButton({
|
|||
} & VariantProps<typeof sidebarMenuButtonVariants>) {
|
||||
const Comp = asChild ? Slot.Root : "button"
|
||||
const { isMobile, state } = useSidebar()
|
||||
const isDisabled = Boolean(props.disabled || props["aria-disabled"])
|
||||
|
||||
const button = (
|
||||
<Comp
|
||||
|
|
@ -594,13 +595,26 @@ function SidebarMenuButton({
|
|||
}
|
||||
}
|
||||
|
||||
// A disabled <button> fires no pointer events (and is not focusable), so the
|
||||
// tooltip would never open. Wrap it in a focusable span with a hoverable box
|
||||
// so the explanation (e.g. why Train/Export are greyed out) is reachable.
|
||||
const trigger = isDisabled ? (
|
||||
<span tabIndex={0} className="flex w-full">
|
||||
{button}
|
||||
</span>
|
||||
) : (
|
||||
button
|
||||
)
|
||||
|
||||
return (
|
||||
<Tooltip>
|
||||
<TooltipTrigger asChild>{button}</TooltipTrigger>
|
||||
<TooltipTrigger asChild>{trigger}</TooltipTrigger>
|
||||
<TooltipContent
|
||||
side="right"
|
||||
align="center"
|
||||
hidden={state !== "collapsed" || isMobile}
|
||||
// Enabled items only show the tooltip when collapsed (icon labels);
|
||||
// a disabled item shows it while expanded too, since it explains why.
|
||||
hidden={isMobile || (!isDisabled && state !== "collapsed")}
|
||||
{...tooltip}
|
||||
/>
|
||||
</Tooltip>
|
||||
|
|
|
|||
|
|
@ -18,6 +18,10 @@ export type DeviceType = "mac" | "windows" | "linux" | string;
|
|||
interface PlatformState {
|
||||
deviceType: DeviceType;
|
||||
chatOnly: boolean;
|
||||
// Why chatOnly is set (null when training is enabled), from /api/health.
|
||||
// e.g. "mlx_unavailable" on Apple Silicon -> the UI explains the greyed-out
|
||||
// Train/Export instead of silently disabling them.
|
||||
chatOnlyReason: string | null;
|
||||
// From /api/health (authed): live tunnel URL, direct (non-tunnel) base, and
|
||||
// whether the server was launched with --secure.
|
||||
cloudflareUrl: string | null;
|
||||
|
|
@ -42,6 +46,7 @@ const localDeviceType = detectLocalPlatform();
|
|||
export const usePlatformStore = create<PlatformState>()((_, get) => ({
|
||||
deviceType: localDeviceType,
|
||||
chatOnly: localDeviceType === "mac",
|
||||
chatOnlyReason: null,
|
||||
cloudflareUrl: null,
|
||||
serverUrl: null,
|
||||
secure: false,
|
||||
|
|
@ -71,18 +76,21 @@ export async function fetchDeviceType(options?: {
|
|||
const data = (await res.json()) as {
|
||||
device_type?: string;
|
||||
chat_only?: boolean;
|
||||
chat_only_reason?: string | null;
|
||||
cloudflare_url?: string | null;
|
||||
server_url?: string | null;
|
||||
secure?: boolean;
|
||||
};
|
||||
const deviceType = data.device_type ?? detectLocalPlatform();
|
||||
const chatOnly = data.chat_only ?? false;
|
||||
const chatOnlyReason = data.chat_only_reason ?? null;
|
||||
// Cache only a server-reported platform. Unauthenticated responses fall
|
||||
// back to the browser platform, which can differ from the host (WSL,
|
||||
// SSH); keeping fetched=false retries once a token exists.
|
||||
usePlatformStore.setState({
|
||||
deviceType,
|
||||
chatOnly,
|
||||
chatOnlyReason,
|
||||
cloudflareUrl: data.cloudflare_url ?? null,
|
||||
serverUrl: data.server_url ?? null,
|
||||
secure: data.secure ?? false,
|
||||
|
|
|
|||
|
|
@ -179,9 +179,7 @@ DEFAULT_MAX_MACOS_RELEASE_FALLBACKS = env_int(
|
|||
16,
|
||||
minimum = 1,
|
||||
)
|
||||
# Deterministic macOS pin. At b9428 ggml-org's macOS runner moved to macOS 26
|
||||
# (Tahoe), so b9428+ prebuilts only load on macOS 26+. b9415 is the last build
|
||||
# stamped below 26 (arm64 minos 14, x64 minos 13.3); loads on macOS 13.3/14/15/26.
|
||||
# Last upstream macOS release before ggml-org moved macOS builds to Tahoe.
|
||||
_PINNED_MACOS_FALLBACK_TAG = "b9415"
|
||||
_PINNED_MACOS_LATEST_FLOOR = (26, 0)
|
||||
FORCE_COMPILE_DEFAULT_REF = os.environ.get("UNSLOTH_LLAMA_FORCE_COMPILE_REF", "master")
|
||||
|
|
@ -193,26 +191,12 @@ FORCE_COMPILE_DEFAULT_REF = os.environ.get("UNSLOTH_LLAMA_FORCE_COMPILE_REF", "m
|
|||
_MIN_CUDA_MAJOR = 12
|
||||
_MAX_PROBE_CUDA_MAJOR = 19
|
||||
|
||||
# Last ggml-org release whose Windows win-cuda-13 build is still sub-13.3
|
||||
# (cuda-13.1, b9360, 2026-05-27). Upstream bumped win-cuda-13 to 13.3 at b9365
|
||||
# and now ships only cuda-12.4 + cuda-13.3. cuda-12.4 predates Blackwell (ggml
|
||||
# compiles sm_120 only at toolkit >= 12.8), so a Blackwell host on a 13.0/13.1/13.2
|
||||
# driver is gated off 13.3 and would drop to a CPU-only 12.4 build. b9360 is
|
||||
# immutable, so we pin its cuda-13.1 build (plus paired cudart) as a GPU
|
||||
# fallback for exactly those hosts. See unslothai/unsloth#5887.
|
||||
_PINNED_BLACKWELL_FALLBACK_TAG = "b9360"
|
||||
_PINNED_BLACKWELL_FALLBACK_RUNTIME = "13.1"
|
||||
# Floor at 13.0: b9360 ships native sm_120a SASS (no PTX/JIT) and a bundled
|
||||
# cuda-13.1 cudart, both of which run on a CUDA 13.0 r580+ driver via CUDA
|
||||
# minor-version compatibility, so the mainstream 13.0 Blackwell branch is covered.
|
||||
_PINNED_BLACKWELL_DRIVER_FLOOR = (13, 0)
|
||||
# Blackwell sm_120 capability thresholds. A host is Blackwell when its highest
|
||||
# compute capability is at least sm_120; ggml compiles sm_120 only at toolkit
|
||||
# >= 12.8, so an in-release windows-cuda build at or above that already covers
|
||||
# Blackwell, while cuda-12.4 does not and is dropped on a Blackwell host.
|
||||
_BLACKWELL_MIN_SM = 120
|
||||
# ggml compiles Blackwell sm_120 only at toolkit >= 12.8, so an in-release
|
||||
# windows-cuda build at or above this already covers Blackwell and makes the
|
||||
# older pinned 13.1 fallback unnecessary (cuda-12.4 is below it).
|
||||
_BLACKWELL_MIN_TOOLKIT = (12, 8)
|
||||
_PINNED_BLACKWELL_LLAMA_SHA256 = "31ddb8b42d7ab4a47cab8c48c397519f580ca502df7e73f3ab396eacc16c8e8d"
|
||||
_PINNED_BLACKWELL_CUDART_SHA256 = "f96935e7e385e3b2d0189239077c10fe8fd7e95690fea4afec455b1b6c7e3f18"
|
||||
|
||||
|
||||
def _cuda_runtime_lines_for_major(major: int) -> list[str]:
|
||||
|
|
@ -1441,11 +1425,6 @@ def direct_upstream_release_plan(
|
|||
)
|
||||
)
|
||||
attempts[:] = _drop_blackwell_incapable_windows_cuda(host, attempts)
|
||||
# Blackwell on a 13.1/13.2 driver: prefer the pinned cuda-13.1 GPU
|
||||
# build over the CPU-only cuda-12.4 the in-release gating leaves.
|
||||
pinned = _pinned_windows_cuda_fallback(host, attempts)
|
||||
if pinned is not None:
|
||||
attempts.insert(0, pinned)
|
||||
elif host.has_rocm:
|
||||
hip_asset = f"llama-{release_tag}-bin-win-hip-radeon-x64.zip"
|
||||
hip_url = assets.get(hip_asset)
|
||||
|
|
@ -1571,19 +1550,9 @@ def direct_upstream_release_plan(
|
|||
|
||||
|
||||
def pinned_macos_release_tag(host: HostInfo, repo: str) -> str | None:
|
||||
"""Pin b9415 (the last upstream macOS build that loads below macOS 26) for a
|
||||
known pre-26 host on ggml-org upstream; return None to keep latest selection.
|
||||
The unslothai/llama.cpp fork ships its own prebuilts (arm64 minos 14, x64
|
||||
minos 13.3) and needs no pin, so this is a no-op there and for macOS 26+,
|
||||
unknown version, non-macOS."""
|
||||
if repo != UPSTREAM_REPO:
|
||||
if repo != UPSTREAM_REPO or not host.is_macos or host.macos_version is None:
|
||||
return None
|
||||
if not host.is_macos:
|
||||
return None
|
||||
version = host.macos_version
|
||||
if version is None:
|
||||
return None
|
||||
if version >= _PINNED_MACOS_LATEST_FLOOR:
|
||||
if host.macos_version >= _PINNED_MACOS_LATEST_FLOOR:
|
||||
return None
|
||||
return _PINNED_MACOS_FALLBACK_TAG
|
||||
|
||||
|
|
@ -1610,9 +1579,6 @@ def resolve_simple_install_release_plans(
|
|||
)
|
||||
requested_tag = normalized_requested_llama_tag(llama_tag)
|
||||
allow_older_release_fallback = requested_tag == "latest" and not published_release_tag
|
||||
# macOS: pin the last upstream build that loads on a pre-26 host instead of
|
||||
# fetching the latest (macOS 26 only) build and walking back release by
|
||||
# release. No-op on macOS 26+, unknown version, non-macOS, and the fork.
|
||||
if allow_older_release_fallback:
|
||||
pinned_macos = pinned_macos_release_tag(host, repo)
|
||||
if pinned_macos is not None:
|
||||
|
|
@ -3453,94 +3419,6 @@ def _drop_blackwell_incapable_windows_cuda(
|
|||
]
|
||||
|
||||
|
||||
def _pinned_windows_cuda_fallback(
|
||||
host: HostInfo, existing_cuda_attempts: list[AssetChoice]
|
||||
) -> AssetChoice | None:
|
||||
"""Pinned GPU fallback for a Blackwell host the in-release build gates off.
|
||||
Upstream stopped publishing a sub-13.3 Windows cuda13 build after b9360, and
|
||||
cuda-12.4 cannot offload sm_120, so a 13.1/13.2 driver would land on CPU.
|
||||
b9360's cuda-13.1 build is immutable and runs on those drivers. Returns None
|
||||
(dormant) whenever the in-release selection already offers a Blackwell-capable
|
||||
build (toolkit >= 12.8, e.g. a runnable cuda13/cuda14), so it self-disables
|
||||
once upstream ships a driver-runnable build again.
|
||||
|
||||
The b9360 binary reuses the current release's source tree and convert scripts
|
||||
and is recorded via binary_release_tag."""
|
||||
if not (host.is_windows and host.is_x86_64 and host.has_usable_nvidia):
|
||||
return None
|
||||
driver = host.driver_cuda_version
|
||||
if driver is None or driver < _PINNED_BLACKWELL_DRIVER_FLOOR:
|
||||
return None
|
||||
caps = normalize_compute_caps(host.compute_caps)
|
||||
if not caps or int(caps[-1]) < _BLACKWELL_MIN_SM:
|
||||
return None
|
||||
if any(_windows_cuda_attempt_covers_blackwell(attempt) for attempt in existing_cuda_attempts):
|
||||
return None
|
||||
tag = _PINNED_BLACKWELL_FALLBACK_TAG
|
||||
runtime = _PINNED_BLACKWELL_FALLBACK_RUNTIME
|
||||
base = (
|
||||
f"https://github.com/{UPSTREAM_REPO}/releases/download/"
|
||||
f"{urllib.parse.quote(tag, safe = '')}"
|
||||
)
|
||||
name = f"llama-{tag}-bin-win-cuda-{runtime}-x64.zip"
|
||||
cudart_name = f"cudart-llama-bin-win-cuda-{runtime}-x64.zip"
|
||||
return AssetChoice(
|
||||
repo = UPSTREAM_REPO,
|
||||
tag = tag,
|
||||
name = name,
|
||||
url = f"{base}/{name}",
|
||||
source_label = "upstream",
|
||||
install_kind = "windows-cuda",
|
||||
runtime_line = "cuda13",
|
||||
runtime_name = cudart_name,
|
||||
runtime_url = f"{base}/{cudart_name}",
|
||||
expected_sha256 = _PINNED_BLACKWELL_LLAMA_SHA256,
|
||||
runtime_sha256 = _PINNED_BLACKWELL_CUDART_SHA256,
|
||||
selection_log = [
|
||||
f"windows_cuda_selection: pinned {tag} cuda-{runtime} Blackwell GPU "
|
||||
f"fallback (in-release cuda13 gated off by driver "
|
||||
f"{driver[0]}.{driver[1]})"
|
||||
],
|
||||
)
|
||||
|
||||
|
||||
def _augment_checksums_with_pin(
|
||||
checksums: ApprovedReleaseChecksums, pin: AssetChoice
|
||||
) -> ApprovedReleaseChecksums:
|
||||
"""Add the pin's own verified hashes to a copy of the approved checksums so
|
||||
apply_approved_hashes keeps it on the published path (b9360 is not in the
|
||||
release manifest)."""
|
||||
artifacts = dict(checksums.artifacts)
|
||||
if pin.expected_sha256:
|
||||
artifacts[pin.name] = ApprovedArtifactHash(
|
||||
asset_name = pin.name,
|
||||
sha256 = pin.expected_sha256,
|
||||
repo = pin.repo,
|
||||
kind = "prebuilt",
|
||||
)
|
||||
if pin.runtime_name and pin.runtime_sha256:
|
||||
artifacts[pin.runtime_name] = ApprovedArtifactHash(
|
||||
asset_name = pin.runtime_name,
|
||||
sha256 = pin.runtime_sha256,
|
||||
repo = pin.repo,
|
||||
kind = "prebuilt",
|
||||
)
|
||||
return dataclasses_replace(checksums, artifacts = artifacts)
|
||||
|
||||
|
||||
def _with_pinned_windows_cuda_fallback(
|
||||
host: HostInfo, attempts: list[AssetChoice], checksums: ApprovedReleaseChecksums
|
||||
) -> tuple[list[AssetChoice], ApprovedReleaseChecksums]:
|
||||
"""Insert the Blackwell pin ahead of the Windows CUDA attempts and keep it
|
||||
through apply_approved_hashes, or return the inputs unchanged when dormant.
|
||||
Gives the published install path the same GPU fallback as the simple path."""
|
||||
attempts = _drop_blackwell_incapable_windows_cuda(host, attempts)
|
||||
pin = _pinned_windows_cuda_fallback(host, attempts)
|
||||
if pin is None:
|
||||
return attempts, checksums
|
||||
return [pin, *attempts], _augment_checksums_with_pin(checksums, pin)
|
||||
|
||||
|
||||
def published_windows_cuda_attempts(
|
||||
host: HostInfo,
|
||||
release: PublishedReleaseBundle,
|
||||
|
|
@ -3966,10 +3844,18 @@ def resolve_upstream_asset_choice(host: HostInfo, llama_tag: str) -> AssetChoice
|
|||
|
||||
if host.is_windows and host.is_x86_64:
|
||||
if host.has_usable_nvidia:
|
||||
attempts = resolve_windows_cuda_choices(host, llama_tag, upstream_assets)
|
||||
attempts = _drop_blackwell_incapable_windows_cuda(
|
||||
host, resolve_windows_cuda_choices(host, llama_tag, upstream_assets)
|
||||
)
|
||||
if attempts:
|
||||
return attempts[0]
|
||||
raise PrebuiltFallback("no compatible Windows CUDA asset was found")
|
||||
# A Blackwell host left with only an sm_120-incapable cuda-12.4 build
|
||||
# (upstream gated off 13.3) has no usable GPU prebuilt here; fall
|
||||
# through to the CPU bundle rather than returning a build it cannot
|
||||
# offload. A non-Blackwell NVIDIA host with no CUDA asset at all is
|
||||
# still a hard fallback.
|
||||
if not _host_is_blackwell(host):
|
||||
raise PrebuiltFallback("no compatible Windows CUDA asset was found")
|
||||
|
||||
# AMD ROCm on Windows: try upstream HIP prebuilt, then fall back to CPU
|
||||
if host.has_rocm:
|
||||
|
|
@ -4050,23 +3936,20 @@ def resolve_release_asset_choice(
|
|||
torch_preference.selection_log,
|
||||
)
|
||||
if published_attempts:
|
||||
pin_attempts, pin_checksums = _with_pinned_windows_cuda_fallback(
|
||||
host, published_attempts, checksums
|
||||
)
|
||||
pin_attempts = _drop_blackwell_incapable_windows_cuda(host, published_attempts)
|
||||
try:
|
||||
return apply_approved_hashes(pin_attempts, pin_checksums)
|
||||
return apply_approved_hashes(pin_attempts, checksums)
|
||||
except PrebuiltFallback as exc:
|
||||
log(
|
||||
"published Windows CUDA assets ignored for install planning: "
|
||||
f"{release.repo}@{release.release_tag} ({exc})"
|
||||
)
|
||||
upstream_assets = github_release_assets(UPSTREAM_REPO, llama_tag)
|
||||
upstream_attempts, upstream_checksums = _with_pinned_windows_cuda_fallback(
|
||||
upstream_attempts = _drop_blackwell_incapable_windows_cuda(
|
||||
host,
|
||||
resolve_windows_cuda_choices(host, llama_tag, upstream_assets),
|
||||
checksums,
|
||||
)
|
||||
return apply_approved_hashes(upstream_attempts, upstream_checksums)
|
||||
return apply_approved_hashes(upstream_attempts, checksums)
|
||||
|
||||
published_choice: AssetChoice | None = None
|
||||
if host.is_windows and host.is_x86_64:
|
||||
|
|
|
|||
|
|
@ -223,7 +223,7 @@ def _fake_macos_releases(tags):
|
|||
|
||||
|
||||
class TestMacosReleasePin:
|
||||
"""A pre-26 macOS host pins the last upstream release that loads on it (b9415); macOS 26+ and unknown-version hosts use normal latest selection."""
|
||||
"""Pre-26 upstream macOS pins the last loadable ggml-org release."""
|
||||
|
||||
TAGS = [f"b{n}" for n in range(9442, 9400, -1)] # newest-first, includes b9415
|
||||
|
||||
|
|
|
|||
|
|
@ -59,7 +59,6 @@ source_archive_logical_name = INSTALL_LLAMA_PREBUILT.source_archive_logical_name
|
|||
windows_cuda_upstream_asset_names = INSTALL_LLAMA_PREBUILT.windows_cuda_upstream_asset_names
|
||||
env_int = INSTALL_LLAMA_PREBUILT.env_int
|
||||
direct_upstream_release_plan = INSTALL_LLAMA_PREBUILT.direct_upstream_release_plan
|
||||
_pinned_windows_cuda_fallback = INSTALL_LLAMA_PREBUILT._pinned_windows_cuda_fallback
|
||||
CudaRuntimePreference = INSTALL_LLAMA_PREBUILT.CudaRuntimePreference
|
||||
published_windows_cuda_attempts = INSTALL_LLAMA_PREBUILT.published_windows_cuda_attempts
|
||||
_windows_cuda_attempt_covers_blackwell = (
|
||||
|
|
@ -2295,84 +2294,15 @@ class TestWindowsCudaAttempts:
|
|||
|
||||
|
||||
# ===========================================================================
|
||||
# N.1b. _pinned_windows_cuda_fallback -- pinned b9360 cuda-13.1 Blackwell fallback
|
||||
# N.1b. _windows_cuda_attempt_covers_blackwell -- Blackwell coverage classifier
|
||||
# ===========================================================================
|
||||
|
||||
|
||||
class TestPinnedBlackwellCudaFallback:
|
||||
"""Blackwell on 13.0/13.1/13.2, gated off in-release 13.3, gets the pinned b9360 cuda-13.1 GPU build; dormant otherwise."""
|
||||
class TestWindowsCudaAttemptCoversBlackwell:
|
||||
"""A windows-cuda attempt covers Blackwell only when its toolkit minor (from the asset name) is >= 12.8, or, for app-named bundles without a toolkit minor, its declared max_sm reaches sm_120."""
|
||||
|
||||
TAG = "b8508"
|
||||
|
||||
def _win_host(self, driver, caps):
|
||||
return make_host(
|
||||
system = "Windows",
|
||||
machine = "AMD64",
|
||||
driver_cuda_version = driver,
|
||||
compute_caps = caps,
|
||||
)
|
||||
|
||||
def test_pin_offered_for_driver_13_1_blackwell(self):
|
||||
pin = _pinned_windows_cuda_fallback(self._win_host((13, 1), ["120"]), [])
|
||||
assert pin is not None
|
||||
assert pin.tag == "b9360"
|
||||
assert pin.runtime_line == "cuda13"
|
||||
assert pin.name == "llama-b9360-bin-win-cuda-13.1-x64.zip"
|
||||
assert pin.runtime_name == "cudart-llama-bin-win-cuda-13.1-x64.zip"
|
||||
assert pin.url.endswith("/b9360/llama-b9360-bin-win-cuda-13.1-x64.zip")
|
||||
assert pin.runtime_url.endswith("/b9360/cudart-llama-bin-win-cuda-13.1-x64.zip")
|
||||
assert pin.install_kind == "windows-cuda"
|
||||
assert pin.expected_sha256 and len(pin.expected_sha256) == 64
|
||||
assert pin.runtime_sha256 and len(pin.runtime_sha256) == 64
|
||||
|
||||
def test_pin_offered_for_driver_13_2(self):
|
||||
assert _pinned_windows_cuda_fallback(self._win_host((13, 2), ["120"]), []) is not None
|
||||
|
||||
def test_pin_offered_for_sm121_variant(self):
|
||||
# sm_121 is Blackwell-family, also needs toolkit >= 12.8.
|
||||
assert _pinned_windows_cuda_fallback(self._win_host((13, 1), ["121"]), []) is not None
|
||||
|
||||
def test_pin_uses_max_of_multi_gpu_caps(self):
|
||||
assert _pinned_windows_cuda_fallback(self._win_host((13, 1), ["86", "120"]), []) is not None
|
||||
|
||||
@pytest.mark.parametrize("sm", ["89", "90", "100"])
|
||||
def test_pin_not_offered_to_non_blackwell(self, sm):
|
||||
# Ada/Hopper run cuda-12.4 fine; the pin must not fire.
|
||||
assert _pinned_windows_cuda_fallback(self._win_host((13, 1), [sm]), []) is None
|
||||
|
||||
def test_pin_offered_for_driver_13_0(self):
|
||||
# b9360 native sm_120a SASS + cuda-13.1 cudart run on a 13.0 r580+ driver via
|
||||
# minor-version compat; 13.0 is the mainstream Blackwell branch, so it must fire.
|
||||
assert _pinned_windows_cuda_fallback(self._win_host((13, 0), ["120"]), []) is not None
|
||||
|
||||
def test_pin_not_offered_below_floor(self):
|
||||
# 12.x predates Blackwell; the pin stays dormant below 13.0.
|
||||
assert _pinned_windows_cuda_fallback(self._win_host((12, 9), ["120"]), []) is None
|
||||
|
||||
def test_pin_not_offered_without_driver(self):
|
||||
assert _pinned_windows_cuda_fallback(self._win_host(None, ["120"]), []) is None
|
||||
|
||||
def test_pin_not_offered_on_linux(self):
|
||||
host = make_host(
|
||||
system = "Linux",
|
||||
machine = "x86_64",
|
||||
driver_cuda_version = (13, 1),
|
||||
compute_caps = ["120"],
|
||||
)
|
||||
assert _pinned_windows_cuda_fallback(host, []) is None
|
||||
|
||||
def test_pin_dormant_when_cuda13_attempt_present(self, monkeypatch):
|
||||
# A runnable in-release cuda13 build makes the pin unnecessary.
|
||||
mock_windows_runtime(monkeypatch, ["cuda13", "cuda12"])
|
||||
host = self._win_host((13, 1), ["120"])
|
||||
assets = {
|
||||
f"llama-{self.TAG}-bin-win-cuda-13.1-x64.zip": "https://example.com/13.1",
|
||||
f"llama-{self.TAG}-bin-win-cuda-12.4-x64.zip": "https://example.com/12.4",
|
||||
}
|
||||
existing = windows_cuda_attempts(host, self.TAG, assets, None)
|
||||
assert any(a.runtime_line == "cuda13" for a in existing)
|
||||
assert _pinned_windows_cuda_fallback(host, existing) is None
|
||||
|
||||
def _win_cuda_attempt(self, minor):
|
||||
major = minor.split(".")[0]
|
||||
return AssetChoice(
|
||||
|
|
@ -2385,30 +2315,6 @@ class TestPinnedBlackwellCudaFallback:
|
|||
runtime_line = f"cuda{major}",
|
||||
)
|
||||
|
||||
def test_pin_dormant_when_runnable_cuda14_present(self, monkeypatch):
|
||||
# An in-release cuda14 build must take precedence over the older b9360 13.1 pin.
|
||||
mock_windows_runtime(monkeypatch, ["cuda14", "cuda12"])
|
||||
host = self._win_host((14, 0), ["120"])
|
||||
assets = {
|
||||
f"llama-{self.TAG}-bin-win-cuda-14.0-x64.zip": "https://example.com/14.0",
|
||||
f"llama-{self.TAG}-bin-win-cuda-12.4-x64.zip": "https://example.com/12.4",
|
||||
}
|
||||
existing = windows_cuda_attempts(host, self.TAG, assets, None)
|
||||
assert any(a.runtime_line == "cuda14" for a in existing)
|
||||
assert _pinned_windows_cuda_fallback(host, existing) is None
|
||||
|
||||
def test_pin_dormant_when_runnable_cuda12_8_present(self):
|
||||
# A cuda-12.8 build also covers Blackwell, so the pin defers to it.
|
||||
host = self._win_host((13, 1), ["120"])
|
||||
existing = [self._win_cuda_attempt("12.8")]
|
||||
assert _pinned_windows_cuda_fallback(host, existing) is None
|
||||
|
||||
def test_pin_fires_when_only_cuda12_4_present(self):
|
||||
# cuda-12.4 does not cover Blackwell, so the pin still fires.
|
||||
host = self._win_host((13, 1), ["120"])
|
||||
existing = [self._win_cuda_attempt("12.4")]
|
||||
assert _pinned_windows_cuda_fallback(host, existing) is not None
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"minor, covers",
|
||||
[
|
||||
|
|
@ -2462,22 +2368,14 @@ class TestPinnedBlackwellCudaFallback:
|
|||
attempt = self._app_attempt(profile, runtime_line, max_sm)
|
||||
assert _windows_cuda_attempt_covers_blackwell(attempt) is covers
|
||||
|
||||
def test_pin_dormant_when_app_bundle_covers_blackwell(self):
|
||||
# Regression: the coverage check previously matched only legacy
|
||||
# -bin-win-cuda-X.Y-x64.zip names, so the pin never went dormant for an
|
||||
# app-named cuda13 bundle that covers Blackwell.
|
||||
host = self._win_host((13, 1), ["120"])
|
||||
existing = [self._app_attempt("newer", "cuda13", 120)]
|
||||
assert _pinned_windows_cuda_fallback(host, existing) is None
|
||||
|
||||
|
||||
# ===========================================================================
|
||||
# N.1c. direct_upstream_release_plan -- pinned Blackwell fallback ordering
|
||||
# N.1c. direct_upstream_release_plan -- Blackwell windows-cuda fallback ordering
|
||||
# ===========================================================================
|
||||
|
||||
|
||||
class TestDirectUpstreamBlackwellPin:
|
||||
"""The pin lands ahead of cuda-12.4 on the simple/upstream path; absent once a runnable in-release cuda13 build exists."""
|
||||
"""On the simple/upstream path a Blackwell host drops the sm_120-incapable cuda-12.4 attempt; with no pinned fallback it falls through to the windows-cpu build, while an in-release cuda-13.3 build is taken when present."""
|
||||
|
||||
TAG = "b9365"
|
||||
|
||||
|
|
@ -2503,7 +2401,7 @@ class TestDirectUpstreamBlackwellPin:
|
|||
lambda host: CudaRuntimePreference(runtime_line = None, selection_log = []),
|
||||
)
|
||||
|
||||
def test_blackwell_13_1_prepends_pin(self, monkeypatch):
|
||||
def test_blackwell_13_1_falls_to_cpu(self, monkeypatch):
|
||||
mock_windows_runtime(monkeypatch, ["cuda13", "cuda12"])
|
||||
self._no_torch(monkeypatch)
|
||||
host = make_host(
|
||||
|
|
@ -2514,10 +2412,10 @@ class TestDirectUpstreamBlackwellPin:
|
|||
)
|
||||
plan = direct_upstream_release_plan(self._release(), host, UPSTREAM_REPO, "latest")
|
||||
order = [(a.tag, a.runtime_line or a.install_kind) for a in plan.attempts]
|
||||
# cuda-12.4 (no sm_120) is dropped entirely on Blackwell rather than left as a
|
||||
# slow non-native fallback behind the pin.
|
||||
assert order == [("b9360", "cuda13"), (self.TAG, "windows-cpu")]
|
||||
assert plan.attempts[0].name == "llama-b9360-bin-win-cuda-13.1-x64.zip"
|
||||
# cuda-12.4 (no sm_120) is dropped entirely on Blackwell, and there is no pinned
|
||||
# b9360 fallback anymore, so the host falls through to the windows-cpu build.
|
||||
assert order == [(self.TAG, "windows-cpu")]
|
||||
assert "b9360" not in [a.tag for a in plan.attempts]
|
||||
# Direct/upstream path stays unverified-by-manifest (no approved hashes).
|
||||
assert plan.approved_checksums.artifacts == {}
|
||||
|
||||
|
|
@ -2819,7 +2717,7 @@ class TestPublishedWindowsCudaAttemptsDynamicMajor:
|
|||
|
||||
|
||||
class TestResolveReleaseAssetChoicePin:
|
||||
"""The manifest install path reaches the same b9360 Blackwell pin as the filename path, with its verified hash threaded."""
|
||||
"""The manifest install path drops sm_120-incapable windows-cuda attempts on Blackwell exactly like the filename path; with no pinned fallback a 13.1 host left with only cuda-12.4 has no usable CUDA attempt and walks back, while a 13.3 host keeps its in-release cuda-13.3 build."""
|
||||
|
||||
TAG = "b8508"
|
||||
|
||||
|
|
@ -2858,9 +2756,14 @@ class TestResolveReleaseAssetChoicePin:
|
|||
lambda host: CudaRuntimePreference(runtime_line = None, selection_log = []),
|
||||
)
|
||||
|
||||
def test_pin_applied_on_published_path_for_13_1(self, monkeypatch):
|
||||
def test_no_cuda_attempt_on_published_path_for_13_1(self, monkeypatch):
|
||||
mock_windows_runtime(monkeypatch, ["cuda13", "cuda12"])
|
||||
self._no_torch(monkeypatch)
|
||||
# After the published Blackwell filter drops every attempt, the resolver
|
||||
# walks back to the upstream release; stub that fetch so the unit test stays
|
||||
# offline (a live GitHub call is blocked by the security scanner) and the
|
||||
# walk-back deterministically finds no usable CUDA build -> PrebuiltFallback.
|
||||
monkeypatch.setattr(INSTALL_LLAMA_PREBUILT, "github_release_assets", lambda repo, tag: {})
|
||||
release = self._release([("13.3", "cuda13"), ("12.4", "cuda12")])
|
||||
checksums = self._checksums(["12.4"]) # 13.3 gated off for a 13.1 driver
|
||||
host = make_host(
|
||||
|
|
@ -2869,14 +2772,11 @@ class TestResolveReleaseAssetChoicePin:
|
|||
driver_cuda_version = (13, 1),
|
||||
compute_caps = ["120"],
|
||||
)
|
||||
result = resolve_release_asset_choice(host, self.TAG, release, checksums)
|
||||
assert result[0].tag == "b9360"
|
||||
assert result[0].name == "llama-b9360-bin-win-cuda-13.1-x64.zip"
|
||||
# The pin survives the approved-hash gate with its verified hash threaded.
|
||||
assert result[0].expected_sha256 and len(result[0].expected_sha256) == 64
|
||||
assert result[0].runtime_sha256 and len(result[0].runtime_sha256) == 64
|
||||
# The sm_120-incapable upstream cuda-12.4 zip is excluded on Blackwell.
|
||||
assert not any(a.runtime_line == "cuda12" for a in result)
|
||||
# The sm_120-incapable upstream cuda-12.4 zip is dropped on Blackwell, and there
|
||||
# is no pinned b9360 fallback anymore, so no usable windows-cuda attempt remains
|
||||
# and the manifest resolver walks back instead of selecting cuda-12.4.
|
||||
with pytest.raises(PrebuiltFallback):
|
||||
resolve_release_asset_choice(host, self.TAG, release, checksums)
|
||||
|
||||
def test_pin_dormant_on_published_path_for_13_3(self, monkeypatch):
|
||||
mock_windows_runtime(monkeypatch, ["cuda13", "cuda12"])
|
||||
|
|
@ -3299,6 +3199,29 @@ class TestResolveUpstreamAssetChoice:
|
|||
assert result.install_kind == "macos-arm64"
|
||||
assert result.name == name
|
||||
|
||||
def test_windows_blackwell_drops_incapable_cuda_to_cpu(self, monkeypatch):
|
||||
# A Blackwell host on a 13.1 driver gets cuda-13.3 gated off and only the
|
||||
# sm_120-incapable cuda-12.4 build left; it must fall through to the CPU
|
||||
# bundle rather than be handed a GPU build it cannot offload.
|
||||
names = [
|
||||
f"llama-{self.TAG}-bin-win-cuda-13.3-x64.zip",
|
||||
"cudart-llama-bin-win-cuda-13.3-x64.zip",
|
||||
f"llama-{self.TAG}-bin-win-cuda-12.4-x64.zip",
|
||||
"cudart-llama-bin-win-cuda-12.4-x64.zip",
|
||||
f"llama-{self.TAG}-bin-win-cpu-x64.zip",
|
||||
]
|
||||
self._mock_github_assets(monkeypatch, {n: f"https://x/{n}" for n in names})
|
||||
host = make_host(
|
||||
system = "Windows",
|
||||
machine = "AMD64",
|
||||
has_usable_nvidia = True,
|
||||
driver_cuda_version = (13, 1),
|
||||
compute_caps = ["120"],
|
||||
)
|
||||
result = resolve_upstream_asset_choice(host, self.TAG)
|
||||
assert result.install_kind == "windows-cpu"
|
||||
assert result.name == f"llama-{self.TAG}-bin-win-cpu-x64.zip"
|
||||
|
||||
def test_macos_arm64_missing(self, monkeypatch):
|
||||
self._mock_github_assets(monkeypatch, {})
|
||||
host = make_host(
|
||||
|
|
@ -3392,8 +3315,6 @@ def _macos_host(machine = "arm64", version = (15, 5)):
|
|||
|
||||
|
||||
class TestPinnedMacosReleaseTag:
|
||||
"""pinned_macos_release_tag: pin b9415 for ggml-org upstream macOS below 26; None (latest) for 26+, unknown version, the fork, non-macOS."""
|
||||
|
||||
def test_arm64_sequoia_pins_b9415(self):
|
||||
host = _macos_host("arm64", (15, 5))
|
||||
assert pinned_macos_release_tag(host, UPSTREAM_REPO) == "b9415"
|
||||
|
|
@ -3403,24 +3324,18 @@ class TestPinnedMacosReleaseTag:
|
|||
assert pinned_macos_release_tag(host, UPSTREAM_REPO) == "b9415"
|
||||
|
||||
def test_x64_ventura_13_3_pins_b9415(self):
|
||||
# b9415's Intel slice is minos 13.3, so 13.3 Intel hosts load it.
|
||||
host = _macos_host("x86_64", (13, 3))
|
||||
assert pinned_macos_release_tag(host, UPSTREAM_REPO) == "b9415"
|
||||
|
||||
def test_tahoe_26_0_takes_latest(self):
|
||||
def test_tahoe_takes_latest(self):
|
||||
host = _macos_host("arm64", (26, 0))
|
||||
assert pinned_macos_release_tag(host, UPSTREAM_REPO) is None
|
||||
|
||||
def test_tahoe_26_1_takes_latest(self):
|
||||
host = _macos_host("arm64", (26, 1))
|
||||
assert pinned_macos_release_tag(host, UPSTREAM_REPO) is None
|
||||
|
||||
def test_unknown_version_takes_latest(self):
|
||||
host = _macos_host("arm64", None)
|
||||
assert pinned_macos_release_tag(host, UPSTREAM_REPO) is None
|
||||
|
||||
def test_fork_repo_is_dormant(self):
|
||||
# The fork publishes its own minos-13.3 prebuilts.
|
||||
host = _macos_host("arm64", (15, 5))
|
||||
fork = INSTALL_LLAMA_PREBUILT.DEFAULT_PUBLISHED_REPO
|
||||
assert pinned_macos_release_tag(host, fork) is None
|
||||
|
|
@ -3431,7 +3346,7 @@ class TestPinnedMacosReleaseTag:
|
|||
|
||||
|
||||
class TestResolveSimpleMacosPin:
|
||||
"""Simple/upstream path: a pre-26 host resolves b9415 (no walk-back); a macOS 26 host takes the latest release."""
|
||||
"""Pre-26 upstream macOS resolves b9415; macOS 26 keeps latest."""
|
||||
|
||||
TAGS = ["b9442", "b9430", "b9428", "b9415"] # newest-first feed
|
||||
|
||||
|
|
@ -3478,9 +3393,7 @@ class TestResolveSimpleMacosPin:
|
|||
assert plans[0].llama_tag == "b9415"
|
||||
assert plans[0].attempts[0].install_kind == "macos-arm64"
|
||||
assert plans[0].attempts[0].name == "llama-b9415-bin-macos-arm64.tar.gz"
|
||||
# The pin overrode the requested tag before any release was fetched.
|
||||
assert calls[0][2] == "b9415"
|
||||
# Simple/upstream path stays unverified-by-manifest.
|
||||
assert plans[0].approved_checksums.artifacts == {}
|
||||
|
||||
def test_tahoe_host_takes_latest_release(self, monkeypatch):
|
||||
|
|
|
|||
|
|
@ -204,6 +204,16 @@ def spoof_hardware(monkeypatch):
|
|||
fake_mlx.core = fake_mlx_core
|
||||
monkeypatch.setitem(sys.modules, "mlx", fake_mlx)
|
||||
monkeypatch.setitem(sys.modules, "mlx.core", fake_mlx_core)
|
||||
# detect_hardware now gates MLX on the full stack via
|
||||
# utils.mlx_repair.mlx_stack_available() (it imports mlx_lm/mlx_vlm and
|
||||
# checks dist versions), which faking only mlx.core cannot satisfy. An
|
||||
# mlx profile means a complete, healthy stack, so model that here;
|
||||
# mlx_stack_available's own internals are covered by test_mlx_repair.py.
|
||||
if str(STUDIO_BACKEND) not in sys.path:
|
||||
sys.path.insert(0, str(STUDIO_BACKEND))
|
||||
import utils.mlx_repair as _mlx_repair # type: ignore
|
||||
|
||||
monkeypatch.setattr(_mlx_repair, "mlx_stack_available", lambda: True)
|
||||
else:
|
||||
# Drop cached mlx and patch find_spec so the unsloth gate sees mlx as absent.
|
||||
monkeypatch.delitem(sys.modules, "mlx", raising = False)
|
||||
|
|
|
|||
|
|
@ -178,6 +178,12 @@ def test_detect_hardware_picks_mlx_when_only_apple_silicon_available(monkeypatch
|
|||
monkeypatch.setitem(sys.modules, "mlx", fake_mlx)
|
||||
monkeypatch.setitem(sys.modules, "mlx.core", fake_mlx_core)
|
||||
|
||||
# detect_hardware now gates MLX on the full stack via _has_usable_mlx_stack()
|
||||
# (utils.mlx_repair.mlx_stack_available imports mlx_lm/mlx_vlm and checks
|
||||
# versions); faking mlx.core alone no longer satisfies it. This test asserts the
|
||||
# dispatch decision when the stack IS usable, so model that directly.
|
||||
monkeypatch.setattr(hw, "_has_usable_mlx_stack", lambda: True)
|
||||
|
||||
detected = hw.detect_hardware()
|
||||
assert detected == hw.DeviceType.MLX, f"expected MLX, got {detected!r}"
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue