unsloth/studio/backend/tests/test_llama_route.py
Daniel Han 898d3dd0b5
Studio: offer the in-app llama.cpp update for source-build (markerless) installs (#6188)
* Studio: offer the in-app llama.cpp update for source-build (markerless) installs

Source-build installs have no UNSLOTH_PREBUILT_INFO.json marker, so freshness
reported supported=False and the Update button never showed (notably on macOS,
where the fork shipped no prebuilt before b9585 and setup fell back to a source
build). When an install has no marker but an official prebuilt now exists for
the host, surface the update and let one click swap it in place.

- install_llama_prebuilt.py: published_repo_for_host() (the setup.sh host->repo
  rule in Python) and a --resolve-prebuilt mode that reports whether a prebuilt
  exists for this host without downloading.
- llama_cpp_update.py: markerless branch in get_update_status/start_update,
  version-suppressed so source builds already newer than latest are not nagged;
  fail-open throughout.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: run llama update detection off the event loop, expose source_build

The markerless source-build check probes the host and reads GitHub, so run
get_update_status and start_update in a worker thread to keep the API
responsive. Expose source_build in the status response so the banner can label
the source-build switch.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Keep the llama-route auth stub out of sys.modules for the rest of the suite

test_llama_route.py replaced sys.modules['auth.authentication'] with a
bare stub at collection time and never restored it, so every later test
importing create_access_token got the stub: 17 failures across
test_desktop_auth, test_middleware, test_openai_tool_passthrough and
test_rag_preview on all four Backend CI Python versions. Import the
real module when its deps are available and only stub in minimal envs,
popping the stubs after the standalone route load either way.

* Studio: address review on the source-build update path

- published_repo_for_host: route CPU-only Windows to ggml-org too (mirrors
  setup.ps1; the fork ships no win-cpu bundle), macOS always the fork.
- markerless detection compares/display the upstream llama_tag, not a possible
  fork wrapper release_tag, so a source build is not wrongly judged newer.
- do not offer when there is no resolvable install root (a pinned
  LLAMA_SERVER_PATH outside a managed dir): an apply would not take effect.

* Ignore version probes in the update tests' subprocess capture

The status polls in these tests trigger the new source-build detection,
which shells out to llama-server --version through the same patched
subprocess.run. On slow runners that probe lands after the installer
call and clobbers the single captured argv, failing the flag
assertions (seen on the 3.10/3.11 Backend CI jobs). Skip probe calls
in all three fakes so only the installer invocation is captured.

* Skip markerless re-detection while the update job is swapping the tree

On a source-build install the frontend polls update-status every 3s
during an apply, and each poll ran _source_build_status, which execs
the very llama-server binary the job is concurrently replacing. On
Windows that exec can hold the exe long enough to fail the installer's
os.replace; everywhere it is a per-poll subprocess spawn for a status
the poller does not read (it only consumes job progress). Gate the
markerless branch on the job not running; the marked path is probe-free
and still returns the live job state.

* Studio: tighten source-build update root, repo routing, and downgrade guard

Only manage a markerless install when the active binary lives under a
resolvable llama.cpp root (marker dir, UNSLOTH_LLAMA_CPP_PATH it sits in,
or a llama.cpp ancestor); a pinned LLAMA_SERVER_PATH or a PATH/system
binary is left alone so an apply cannot install where it would not take
effect. Gate start_update on the same suppression as detection so a
direct POST cannot downgrade a source build newer than the latest
prebuilt. Route Linux hosts with AMD tooling (rocminfo/amd-smi/hipconfig/
hipinfo) to the fork in --resolve-prebuilt, matching setup.sh, so a HIP
source build is not offered an upstream CPU prebuilt.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* Studio: cover inactive env root and pinned llama.cpp checkout in update root tests

---------

Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-06-11 02:45:12 -07:00

111 lines
3.6 KiB
Python

# SPDX-License-Identifier: AGPL-3.0-only
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
"""routes/llama.py: the source_build field is exposed and the handlers run the
(now subprocess-touching) detection off the event loop via a worker thread.
The route file is loaded standalone with a stubbed auth dependency so the test
does not pull the whole routes package (matplotlib-heavy training router) and
works in a minimal env.
"""
from __future__ import annotations
import asyncio
import importlib.util
import sys
import threading
import types
from pathlib import Path
import pytest
_BACKEND = Path(__file__).resolve().parents[1]
if str(_BACKEND) not in sys.path:
sys.path.insert(0, str(_BACKEND))
pytest.importorskip("fastapi")
def _load_route():
# Prefer the real auth module; stub it only in minimal envs where its
# deps are absent. Stubs are popped after the load so they never leak
# into sys.modules for the rest of the suite.
stubbed = []
try:
import auth.authentication # noqa: F401
except Exception:
auth_pkg = types.ModuleType("auth")
auth_pkg.__path__ = []
auth_mod = types.ModuleType("auth.authentication")
auth_mod.get_current_subject = lambda: "test"
for name, stub in (("auth", auth_pkg), ("auth.authentication", auth_mod)):
if name not in sys.modules:
sys.modules[name] = stub
stubbed.append(name)
try:
spec = importlib.util.spec_from_file_location(
"llama_route_under_test", str(_BACKEND / "routes" / "llama.py")
)
mod = importlib.util.module_from_spec(spec)
sys.modules["llama_route_under_test"] = mod # so pydantic resolves forward refs
spec.loader.exec_module(mod)
return mod
finally:
for name in stubbed:
sys.modules.pop(name, None)
rl = _load_route()
def test_status_response_exposes_source_build():
payload = {
"supported": True,
"update_available": True,
"stale": False,
"installed_tag": None,
"latest_tag": "b9585",
"published_repo": "unslothai/llama.cpp",
"installed_at_utc": None,
"age_days": None,
"source_build": True,
"job": {"state": "idle"},
}
model = rl.LlamaUpdateStatusResponse(**payload)
assert model.model_dump()["source_build"] is True
# Extra/unknown keys must not crash the response model.
rl.LlamaUpdateStatusResponse(**{**payload, "unexpected": 1})
def test_status_handler_runs_off_event_loop(monkeypatch):
seen = {}
def fake_status(force_refresh = False):
seen["thread"] = threading.current_thread()
return {
"supported": True,
"update_available": True,
"source_build": True,
"latest_tag": "b9585",
"job": {"state": "idle"},
}
monkeypatch.setattr(rl, "get_update_status", fake_status)
out = asyncio.run(rl.llama_update_status(force_refresh = False, current_subject = "t"))
assert out.source_build is True
# Detection ran in a worker thread, not the event-loop thread.
assert seen["thread"] is not threading.main_thread()
def test_update_handler_runs_off_event_loop(monkeypatch):
seen = {}
def fake_start():
seen["thread"] = threading.current_thread()
return {"started": True, "reason": None, "job": {"state": "running"}}
monkeypatch.setattr(rl, "start_update", fake_start)
out = asyncio.run(rl.llama_update(current_subject = "t"))
assert out.started is True
assert seen["thread"] is not threading.main_thread()