From 378e33c8a589d1e006520f1b78f3ef083bab4314 Mon Sep 17 00:00:00 2001 From: Daniel Han Date: Mon, 22 Jun 2026 02:20:08 -0700 Subject: [PATCH] Studio macOS: faster startup, MLX self-heal, drop obsolete prebuilt pins (#6494) * Studio: defer llama.cpp update probes and self-heal MLX on macOS Two macOS startup problems shared one root area in the FastAPI lifespan: - The llama.cpp capability + freshness probes ran inline before the server yielded, so a cold/slow/flaky network on the GitHub freshness check blocked 'Application startup complete' (~34s on CI, longer in the field). Move both probes to a daemon thread; app.state stays None until ready (status routes already re-probe at request time). Opt out with UNSLOTH_DISABLE_UPDATE_CHECK=1. - Train and Export were greyed out because mlx/mlx-lm/mlx-vlm arrive only transitively and a resolver backtrack silently drops them, so CHAT_ONLY stayed true. Add utils/mlx_repair.py: when Apple Silicon is detected without MLX, reinstall mlx/mlx-lm/mlx-vlm by name on a daemon thread and re-run hardware detection (opt out UNSLOTH_DISABLE_MLX_AUTOREPAIR=1). Surface a chat_only_reason in /api/health plus a sidebar tooltip so a greyed Train/Export explains itself instead of failing silently. * Studio: guard model defaults against a None model name load_model_defaults(None) called model_name.lower() with no guard, raising 'Error loading model defaults for None' before any model is selected. Return an empty dict for a falsy/non-str name. * Studio: drop obsolete upstream macOS + Windows Blackwell prebuilt pins Both pins worked around gaps in ggml-org upstream prebuilts, but Studio now routes every GPU host and all of macOS to the unslothai/llama.cpp fork (published_repo_for_host), which ships the needed bundles, so both pins are dead code on the default install path: - macOS b9415: macOS always routes to the fork (its own macOS bundles), and host_supports_macos_minos() is the backstop. The pin only fired under an explicit --published-repo ggml-org override. - Windows Blackwell b9360: Windows-NVIDIA routes to the fork, whose windows-x64-cuda13 bundle covers Blackwell (manifest max_sm 120, toolkit 13.3), so the pin's self-disable check makes it dormant on every default install; it could only activate under the same upstream override on a 13.0-13.2 driver. Remove the pin constants, functions, and call sites. Keep the Blackwell capability detection (_drop_blackwell_incapable_windows_cuda, _host_is_blackwell, _windows_cuda_attempt_covers_blackwell) that still drops a non-sm_120 cuda-12.4 build on a Blackwell host. After this, an explicit --published-repo ggml-org override on a Blackwell 13.0-13.2 host loses its GPU fallback and lands on CPU; the default fork path is unaffected. Update the install selection-logic and macOS-compat unit tests for the new no-pin behavior. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: walk back deeper on the macOS upstream prebuilt path After removing the b9415 macOS pin, the explicit --published-repo ggml-org upstream path still used the default 2-release fallback, so a pre-macOS-26 host behind a run of macOS-26-only builds would exhaust two too-new plans (minos is only checked post-download) and drop to a source build before reaching a loadable older release. Walk back as deep as the fork macOS path (DEFAULT_MAX_MACOS_RELEASE_FALLBACKS), turning the removed static pin into dynamic discovery. Addresses review feedback on the macOS upstream fallback. * Studio: pin transformers during MLX self-heal so it cannot break Studio mlx-lm/mlx-vlm declare transformers>=5, but the single-env install pins transformers==4.57.6. The self-heal used --upgrade with no constraint, so it could upgrade transformers in the live venv and break the rest of Studio just to make import mlx.core pass. Pin transformers to the installed version via a constraint file: the resolver either finds an mlx build compatible with it or fails (we stay chat-only), never upgrading transformers underneath Studio. Addresses review feedback on the MLX repair install. * Studio: harden MLX self-heal against an unsupported mlx-vlm Pinning transformers alone made uv backtrack mlx-vlm to 0.3.9 (below unsloth-zoo's mlx-vlm>=0.4.4), which imports but breaks VLM Train/Export -- so the self-heal could clear chat-only onto a broken stack. Mirror the main installer: set UV_OVERRIDE=overrides-darwin-arm64.txt so a current mlx-vlm coexists with the transformers pin, require the same minimum versions unsloth-zoo declares, and gate/validate on a full mlx_stack_available() check (not a bare import) so an old or partial stack stays chat-only. Addresses PR review. * Studio: filter Blackwell-incapable CUDA in resolve_upstream_asset_choice resolve_upstream_asset_choice returned the first windows-cuda choice unfiltered, so a Blackwell host could be handed an sm_120-incapable cuda-12.4 build while the sibling planners drop it. Apply _drop_blackwell_incapable_windows_cuda here too and fall through to the CPU bundle on a Blackwell host with no capable GPU asset. Addresses PR review. * Studio: re-poll health so MLX self-heal reaches an open UI The sidebar cached the initial /api/health, so a successful background MLX self-heal (chat_only flips false) did not re-enable Train/Export until a manual reload. While chat-only for the recoverable mlx_unavailable reason, re-poll /api/health and stop once Train/Export become available. Addresses PR review. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Studio: make the disabled Train/Export tooltip reachable The greyed Train/Export items pass a tooltip explaining why (e.g. MLX missing), but a disabled