Source llama.cpp prebuilts from unslothai/llama.cpp (CUDA, ROCm, macOS) (#5963)
* Studio: route arm64 Linux CUDA hosts to linux-arm64-cuda prebuilts
* Studio: SM-aware selection for windows-cuda app bundles
* Studio: select published ROCm bundles by gfx target (linux + windows)
* Studio: route macOS installs to the fork's prebuilt bundles
* Studio: fix windows cuda13 driver-13.0 gate and ROCm gfx prefix overreach
* Fix Blackwell Windows pin shadowing native app-bundle (b9360 over b9457)
* Match Windows cuda12 driver floor to Linux (12.x minor-version compat)
* Fix Windows app-bundle dropped when runtime DLLs come from torch/lib
* Fold the manifest resolver into the simple-path resolver (one entry, no dormant full path)
* Remove unused UNSLOTH_LLAMA_PUBLISHED_REPO override
* Route Windows GPU hosts to the fork prebuilts in setup.ps1
* Document sm_103 path divergence and mark --simple-policy as a no-op
* Note sm_103 coverage now comes from the producer manifest
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Remove the now-vestigial --simple-policy flag (one resolver handles all hosts)
* Unify the fork onto the manifest path; drop the linux-x64 filename path and hardcoded coverage tables
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Strip whitespace from manifest gfx_target/mapped_targets when parsing
* Windows CUDA: sort coverage-unknown bundles last so they can't outrank targeted ones
* Share the SM-coverage sort key between the linux and windows selectors via _sm_range
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: reject approved releases with an exact source archive but no source repo to clone from
* Studio: accept the fork's windows-rocm kind in the Windows reinstall check
* Studio: accept a manifest-bundle source repo in the exact-source release check
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: route Linux hosts to the fork only when a usable GPU is present
* Fix Windows AMD lemonade tag resolution for PR #5963
The fork release scan passes each scanned release's upstream tag
(b9518, ...) to the lemonade lookup, but lemonade publishes its own tag
series (b1292, ...) that never contains upstream tag numbers. On a
Windows AMD host every scanned release therefore 404s the lemonade
fetch twice, the upstream HIP zip is dropped by the approved-hash gate,
and the scan walks the whole release history until it dies on the
unauthenticated GitHub rate limit or falls to a HIP source build. The
Linux path already passes the requested tag ("latest") and works.
Thread the requested tag through resolve_release_asset_choice ->
resolve_asset_choice -> resolve_upstream_asset_choice as lemonade_tag,
used only by the lemonade lookups. Upstream asset names keep the
concrete per-release tag and all new parameters default to the old
behavior.
Verified on a gfx1151 box: before, the native Windows install scanned
b9518..b8811 and aborted on rate limit; after, it selects
llama-b1292-windows-rocm-gfx1151-x64.zip (lemonade) from fork release
b9518, passes staged validation, and the installed llama-server
enumerates ROCm0. WSL keeps selecting the matching ubuntu bundle.
Adds a regression test pinning that the Windows fork path resolves
lemonade via /releases/latest, never /releases/tags/<fork-tag>.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Plan lemonade for Linux ROCm hosts on the ggml-org direct path for PR #5963
Audit follow-up to 72f32364 across the other selection pathways. The
ggml-org direct planner kept its lemonade attempt for Windows ROCm
hosts but planned only the CPU tarball for Linux ROCm hosts, so an AMD
Linux box routed to ggml-org (for example a --published-repo override)
silently installed the CPU build. That lemonade planning used to live
in the --simple-policy dispatcher this PR removed.
Add the lemonade attempt ahead of the CPU tarball in the Linux x86_64
branch, mirroring the Windows branch, with the lookup keyed to the
requested tag. Adds a regression test asserting lemonade is the first
attempt for a Linux ROCm host on the direct path.
Also re-verified the other pathways on a gfx1151 box: the fork-routed
flows pass the requested tag everywhere, repeat runs over an existing
lemonade install correctly skip with "already matches selected release
b9518" on both native Windows and WSL, and macOS, CUDA and CPU
selection are untouched. Suites: 328 passed on Linux, Windows matches
the pre-existing baseline.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: danielhanchen <michaelhan2050@gmail.com>
This commit is contained in:
oobabooga
GitHub
parent
8bca7bcfc9
commit
cc1a724efc
8 changed files with 1311 additions and 1060 deletions
|
|
@ -1,10 +1,11 @@
|
|||
# SPDX-License-Identifier: AGPL-3.0-only
|
||||
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
||||
|
||||
"""Validates that the installer resolves lemonade ROCm prebuilt assets.
|
||||
"""Validates that the installer correctly resolves lemonade ROCm prebuilt assets.
|
||||
|
||||
Uses a faked HostInfo so no AMD GPU is needed. The lemonade GitHub API calls
|
||||
are stubbed so the suite runs offline and isn't subject to rate limits.
|
||||
Uses a faked HostInfo so no AMD GPU is needed. Network calls to the lemonade
|
||||
GitHub API are stubbed out so the suite runs without internet access and is
|
||||
not subject to rate limits.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
|
@ -32,7 +33,7 @@ if resolve_lemonade_rocm_choice is None or _LEMONADE_GFX_FAMILIES is None:
|
|||
@pytest.fixture(autouse = True)
|
||||
def _clear_lemonade_release_cache():
|
||||
"""Prevent cross-test pollution of the lemonade release lru_cache when
|
||||
tests vary the fetch_json mock return value."""
|
||||
future tests vary the fetch_json mock return value."""
|
||||
_cache = getattr(_mod, "_fetch_lemonade_release_cached", None)
|
||||
if _cache is not None and hasattr(_cache, "cache_clear"):
|
||||
_cache.cache_clear()
|
||||
|
|
@ -89,7 +90,9 @@ def _lookup_family(gfx: str) -> str | None:
|
|||
return None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# GPU family mapping
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
|
|
@ -111,7 +114,9 @@ def test_unknown_gpu_not_in_families():
|
|||
assert _lookup_family("gfx999") is None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Asset resolution - hits real lemonade GitHub API
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
|
|
@ -141,59 +146,71 @@ def test_unknown_gpu_falls_through_to_upstream():
|
|||
assert result is None
|
||||
|
||||
|
||||
# Simple-policy dispatcher must plan a lemonade ROCm attempt for AMD-only hosts.
|
||||
# This is the path setup.sh invokes (via --simple-policy), so the lemonade
|
||||
# integration is useless if it isn't wired in here.
|
||||
# ---------------------------------------------------------------------------
|
||||
# The Linux attempt builder must plan a lemonade ROCm attempt for AMD-only hosts.
|
||||
# This is the path setup.sh actually invokes (fork hosts now select from the
|
||||
# manifest), so the lemonade integration is useless if it isn't wired in here.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
direct_linux_release_plan = getattr(_mod, "direct_linux_release_plan", None)
|
||||
_linux_published_attempts = getattr(_mod, "_linux_published_attempts", None)
|
||||
direct_upstream_release_plan = getattr(_mod, "direct_upstream_release_plan", None)
|
||||
|
||||
PublishedLlamaArtifact = _mod.PublishedLlamaArtifact
|
||||
PublishedReleaseBundle = _mod.PublishedReleaseBundle
|
||||
|
||||
def _stub_unsloth_release(release_tag: str = "b9022") -> dict:
|
||||
# Minimal payload parse_direct_linux_release_bundle accepts. It needs at
|
||||
# least one `app-{label}-linux-x64*.tar.gz` asset to recognise the bundle;
|
||||
# we ship a bare CPU one so the planner has a baseline non-ROCm fallback.
|
||||
asset_name = f"app-{release_tag}-linux-x64.tar.gz"
|
||||
return {
|
||||
"tag_name": release_tag,
|
||||
"name": release_tag,
|
||||
"assets": [
|
||||
{
|
||||
"name": asset_name,
|
||||
"browser_download_url": f"https://example.invalid/{asset_name}",
|
||||
},
|
||||
],
|
||||
}
|
||||
|
||||
def _rocm_bundle(gfx_family: str, mapped_targets: list[str]) -> "PublishedReleaseBundle":
|
||||
"""A fork manifest bundle exposing a per-gfx linux-rocm artifact, so
|
||||
published_rocm_choice_for_host can match the host before the lemonade
|
||||
fallback is appended."""
|
||||
asset_name = f"app-b9457-linux-x64-rocm-{gfx_family}.tar.gz"
|
||||
artifact = PublishedLlamaArtifact(
|
||||
asset_name = asset_name,
|
||||
install_kind = "linux-rocm",
|
||||
runtime_line = None,
|
||||
coverage_class = None,
|
||||
supported_sms = [],
|
||||
min_sm = None,
|
||||
max_sm = None,
|
||||
bundle_profile = None,
|
||||
rank = 1000,
|
||||
gfx_target = gfx_family,
|
||||
mapped_targets = mapped_targets,
|
||||
)
|
||||
return PublishedReleaseBundle(
|
||||
repo = "unslothai/llama.cpp",
|
||||
release_tag = "v1.0",
|
||||
upstream_tag = "b9457",
|
||||
assets = {asset_name: f"https://example.invalid/{asset_name}"},
|
||||
artifacts = [artifact],
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.skipif(
|
||||
direct_linux_release_plan is None,
|
||||
reason = "simple-policy dispatcher not present on this branch",
|
||||
_linux_published_attempts is None,
|
||||
reason = "Linux attempt builder not present on this branch",
|
||||
)
|
||||
def test_simple_policy_plans_lemonade_for_rocm_host():
|
||||
def test_linux_attempts_include_fork_rocm_and_lemonade_for_rocm_host():
|
||||
host = _make_rocm_host("gfx1151")
|
||||
bundle = _rocm_bundle("gfx1151", ["gfx1151"])
|
||||
with patch.object(_mod, "fetch_json", return_value = _stub_lemonade_release()):
|
||||
plan = direct_linux_release_plan(
|
||||
_stub_unsloth_release(),
|
||||
host,
|
||||
"unslothai/llama.cpp",
|
||||
"latest",
|
||||
)
|
||||
assert plan is not None, "ROCm host should not be skipped by simple-policy planner"
|
||||
kinds = [a.install_kind for a in plan.attempts]
|
||||
assert (
|
||||
"linux-rocm" in kinds
|
||||
), f"simple-policy planner did not include a lemonade ROCm attempt; got {kinds}"
|
||||
rocm_attempt = next(a for a in plan.attempts if a.install_kind == "linux-rocm")
|
||||
assert rocm_attempt.source_label == "lemonade"
|
||||
assert "gfx1151" in rocm_attempt.name
|
||||
attempts = _linux_published_attempts(host, bundle, "latest")
|
||||
kinds = [a.install_kind for a in attempts]
|
||||
assert "linux-rocm" in kinds, f"builder did not include any linux-rocm attempt; got {kinds}"
|
||||
sources = {a.source_label for a in attempts if a.install_kind == "linux-rocm"}
|
||||
# The fork's own per-gfx bundle is preferred, with the lemonade prebuilt as
|
||||
# the fallback -- both must be present for a covered ROCm host.
|
||||
assert "published" in sources, f"fork ROCm bundle missing; got {sources}"
|
||||
assert "lemonade" in sources, f"lemonade ROCm fallback missing; got {sources}"
|
||||
lemonade_attempt = next(a for a in attempts if a.source_label == "lemonade")
|
||||
assert "gfx1151" in lemonade_attempt.name
|
||||
|
||||
|
||||
@pytest.mark.skipif(
|
||||
direct_upstream_release_plan is None,
|
||||
reason = "simple-policy dispatcher not present on this branch",
|
||||
reason = "direct release planners not present on this branch",
|
||||
)
|
||||
def test_simple_policy_plans_lemonade_for_windows_hip_host():
|
||||
def test_direct_upstream_plan_includes_lemonade_for_windows_hip_host():
|
||||
host = _make_rocm_host("gfx1151", windows = True)
|
||||
release = {
|
||||
"tag_name": "b9022",
|
||||
|
|
@ -204,16 +221,14 @@ def test_simple_policy_plans_lemonade_for_windows_hip_host():
|
|||
plan = direct_upstream_release_plan(release, host, "ggml-org/llama.cpp", "latest")
|
||||
assert plan is not None, "Windows ROCm host should plan a lemonade HIP attempt"
|
||||
kinds = [a.install_kind for a in plan.attempts]
|
||||
assert (
|
||||
"windows-hip" in kinds
|
||||
), f"simple-policy planner did not include a lemonade HIP attempt; got {kinds}"
|
||||
assert "windows-hip" in kinds, f"planner did not include a lemonade HIP attempt; got {kinds}"
|
||||
|
||||
|
||||
@pytest.mark.skipif(
|
||||
direct_upstream_release_plan is None,
|
||||
reason = "simple-policy dispatcher not present on this branch",
|
||||
reason = "direct release planners not present on this branch",
|
||||
)
|
||||
def test_simple_policy_windows_hip_falls_back_to_upstream_when_lemonade_unavailable():
|
||||
def test_windows_hip_falls_back_to_upstream_when_lemonade_unavailable():
|
||||
"""If lemonade returns None (e.g. gfx999 or transient API failure), the planner
|
||||
must still include the upstream HIP asset rather than silently downgrading to CPU."""
|
||||
host = _make_rocm_host("gfx999", windows = True)
|
||||
|
|
@ -247,8 +262,9 @@ def test_lemonade_release_api_url_pinned_tag():
|
|||
|
||||
|
||||
def test_lemonade_release_api_url_encodes_tag():
|
||||
"""Slashes / hashes in the tag must be URL-encoded so the URL can't be
|
||||
reshaped (defence in depth -- tags should already be sanitised upstream)."""
|
||||
"""Unexpected slashes / hashes in the tag must be URL-encoded so the URL
|
||||
cannot be reshaped (defence in depth -- tags should already be sanitised
|
||||
upstream)."""
|
||||
url = _mod._lemonade_release_api_for("b1260/../latest")
|
||||
assert "/releases/tags/b1260%2F..%2Flatest" in url
|
||||
assert "//latest" not in url.split("/releases/tags/", 1)[1]
|
||||
|
|
@ -263,9 +279,9 @@ def test_lemonade_resolver_skipped_by_opt_out_env(monkeypatch):
|
|||
|
||||
|
||||
def test_lemonade_resolver_rejects_non_github_url(monkeypatch):
|
||||
"""If the GitHub API response contained an off-host download URL, the
|
||||
resolver must refuse it (lemonade assets aren't in the approved-hash
|
||||
manifest)."""
|
||||
"""If the GitHub API response somehow contained an off-host download URL,
|
||||
the resolver must refuse to use it (lemonade assets are not in the
|
||||
approved-hash manifest)."""
|
||||
bad_release = {
|
||||
"tag_name": _STUB_TAG,
|
||||
"assets": [
|
||||
|
|
@ -289,7 +305,7 @@ def test_lemonade_resolver_rejects_http_scheme():
|
|||
|
||||
|
||||
def test_lemonade_resolver_accepts_github_cdn():
|
||||
# Real GitHub release CDN URLs carry the /github-production-release-asset- prefix
|
||||
# Real GitHub release CDN URLs carry the /github-production-release-asset- prefix.
|
||||
assert _mod._is_trusted_github_release_url(
|
||||
"https://objects.githubusercontent.com/github-production-release-asset-abc123/456/789?token=x",
|
||||
"lemonade-sdk/llamacpp-rocm",
|
||||
|
|
@ -297,7 +313,7 @@ def test_lemonade_resolver_accepts_github_cdn():
|
|||
|
||||
|
||||
def test_lemonade_resolver_rejects_arbitrary_cdn_path():
|
||||
# A CDN URL without the release-asset path prefix must be rejected
|
||||
# A CDN URL without the release-asset path prefix must be rejected.
|
||||
assert not _mod._is_trusted_github_release_url(
|
||||
"https://objects.githubusercontent.com/abc/def",
|
||||
"lemonade-sdk/llamacpp-rocm",
|
||||
|
|
@ -339,7 +355,7 @@ def test_lemonade_runtime_patterns_include_hip_runtime():
|
|||
|
||||
Lemonade ZIPs carry transitive deps (libamd_comgr, libLLVM, libclang-cpp,
|
||||
...) whose names change across ROCm releases. A broad ``lib*.so*`` glob
|
||||
avoids enumerating every transitive dependency by name.
|
||||
avoids having to enumerate every transitive dependency by name.
|
||||
"""
|
||||
from install_llama_prebuilt import runtime_patterns_for_choice, AssetChoice
|
||||
|
||||
|
|
@ -353,7 +369,7 @@ def test_lemonade_runtime_patterns_include_hip_runtime():
|
|||
)
|
||||
pats = runtime_patterns_for_choice(choice)
|
||||
# The broad glob must be present so every .so in the lemonade bundle
|
||||
# (including future transitive deps) gets overlaid.
|
||||
# (including transitive deps added in future ROCm releases) gets overlaid.
|
||||
assert "lib*.so*" in pats, f"'lib*.so*' missing from linux-rocm patterns: {pats}"
|
||||
|
||||
|
||||
|
|
@ -365,9 +381,9 @@ _pick_rocm_gfx_target = getattr(_mod, "_pick_rocm_gfx_target", None)
|
|||
reason = "_pick_rocm_gfx_target not present on this branch",
|
||||
)
|
||||
def test_pick_rocm_gfx_target_honors_cuda_visible_devices(monkeypatch):
|
||||
"""AMD HIP honours CUDA_VISIBLE_DEVICES like HIP_VISIBLE_DEVICES; on a
|
||||
gfx1151 + gfx1100 mixed host, CUDA_VISIBLE_DEVICES=1 must select gfx1100."""
|
||||
# Two GPUs; rocminfo reports each token twice (as in real tool output).
|
||||
"""AMD HIP honours CUDA_VISIBLE_DEVICES identically to HIP_VISIBLE_DEVICES;
|
||||
on a gfx1151 + gfx1100 mixed host, CUDA_VISIBLE_DEVICES=1 must select gfx1100."""
|
||||
# Two GPUs; rocminfo reports each token twice (as in the real tool output).
|
||||
probe_out = "gfx1151\ngfx1151\ngfx1100\ngfx1100"
|
||||
monkeypatch.delenv("HIP_VISIBLE_DEVICES", raising = False)
|
||||
monkeypatch.delenv("ROCR_VISIBLE_DEVICES", raising = False)
|
||||
|
|
@ -396,7 +412,7 @@ def test_pick_rocm_gfx_target_same_arch_multi_gpu(monkeypatch):
|
|||
"""Regression: [gfx1100, gfx1100, gfx1151] with HIP_VISIBLE_DEVICES=2 must
|
||||
return gfx1151, not fall back to GPU 0 due to dict.fromkeys collapsing the
|
||||
two gfx1100 entries into one and making index 2 out of range."""
|
||||
# rocminfo output for 3 GPUs (2x gfx1100 dGPU + 1x gfx1151 APU).
|
||||
# Simulate rocminfo output for 3 GPUs (2x gfx1100 dGPU + 1x gfx1151 APU).
|
||||
# Each GPU gets its own Agent section with a few token mentions.
|
||||
probe_out = (
|
||||
"***\nAgent 1\n***\n gfx1100 some info\n gfx1100\n"
|
||||
|
|
@ -407,3 +423,96 @@ def test_pick_rocm_gfx_target_same_arch_multi_gpu(monkeypatch):
|
|||
monkeypatch.delenv("CUDA_VISIBLE_DEVICES", raising = False)
|
||||
monkeypatch.setenv("HIP_VISIBLE_DEVICES", "2")
|
||||
assert _pick_rocm_gfx_target(probe_out) == "gfx1151"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Fork release scan: Windows ROCm resolves lemonade by the requested tag
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_resolve_release_asset_choice = getattr(_mod, "resolve_release_asset_choice", None)
|
||||
_ApprovedReleaseChecksums = getattr(_mod, "ApprovedReleaseChecksums", None)
|
||||
|
||||
|
||||
@pytest.mark.skipif(
|
||||
_resolve_release_asset_choice is None or _ApprovedReleaseChecksums is None,
|
||||
reason = "fork release planner not present on this branch",
|
||||
)
|
||||
def test_fork_scan_windows_rocm_resolves_lemonade_by_requested_tag():
|
||||
"""The fork release scan pins llama_tag to per-release upstream tags
|
||||
(b9457, ...) that lemonade's own tag series never contains, so the
|
||||
lemonade lookup must use the requested tag ("latest") instead. Pinning
|
||||
lemonade to the per-release tag 404s on every scanned release and a
|
||||
Windows ROCm host ends in a rate-limited fatal instead of the lemonade
|
||||
prebuilt."""
|
||||
host = _make_rocm_host("gfx1151", windows = True)
|
||||
# No windows-rocm artifact in the bundle, matching current fork releases.
|
||||
bundle = _rocm_bundle("gfx1151", ["gfx1151"])
|
||||
checksums = _ApprovedReleaseChecksums(
|
||||
repo = "unslothai/llama.cpp",
|
||||
release_tag = "v1.0",
|
||||
upstream_tag = "b9457",
|
||||
artifacts = {},
|
||||
)
|
||||
seen_urls: list[str] = []
|
||||
|
||||
def _fake_fetch(api_url, *args, **kwargs):
|
||||
seen_urls.append(api_url)
|
||||
if "lemonade-sdk" in api_url:
|
||||
if api_url.endswith("/releases/latest"):
|
||||
return _stub_lemonade_release()
|
||||
raise RuntimeError(f"unexpected pinned lemonade fetch: {api_url}")
|
||||
# ggml-org asset listing for the upstream HIP/CPU filename fallbacks.
|
||||
return {"tag_name": "b9457", "assets": []}
|
||||
|
||||
with patch.object(_mod, "fetch_json", side_effect = _fake_fetch):
|
||||
attempts = _resolve_release_asset_choice(
|
||||
host,
|
||||
"b9457", # concrete per-release upstream tag from the scan loop
|
||||
bundle,
|
||||
checksums,
|
||||
requested_tag = "latest",
|
||||
)
|
||||
|
||||
lemonade = [a for a in attempts if a.source_label == "lemonade"]
|
||||
assert lemonade, f"lemonade attempt missing for Windows ROCm host; got {attempts}"
|
||||
assert "gfx1151" in lemonade[0].name
|
||||
assert any(
|
||||
u.endswith("/releases/latest") for u in seen_urls
|
||||
), f"lemonade was never resolved via /releases/latest; fetches: {seen_urls}"
|
||||
assert not any(
|
||||
"lemonade-sdk" in u and "/releases/tags/" in u for u in seen_urls
|
||||
), f"lemonade lookup was pinned to the fork release tag: {seen_urls}"
|
||||
|
||||
|
||||
@pytest.mark.skipif(
|
||||
direct_upstream_release_plan is None,
|
||||
reason = "direct release planners not present on this branch",
|
||||
)
|
||||
def test_direct_upstream_plan_includes_lemonade_for_linux_rocm_host():
|
||||
"""A Linux ROCm host on the ggml-org direct path (e.g. a --published-repo
|
||||
override) must plan lemonade before the CPU tarball, mirroring the Windows
|
||||
branch. The lemonade planning previously lived in the removed
|
||||
--simple-policy dispatcher, so without this leg such hosts silently
|
||||
install the CPU build."""
|
||||
host = _make_rocm_host("gfx1151")
|
||||
release = {
|
||||
"tag_name": "b9022",
|
||||
"name": "b9022",
|
||||
"assets": [
|
||||
{
|
||||
"name": "llama-b9022-bin-ubuntu-x64.tar.gz",
|
||||
"browser_download_url": (
|
||||
"https://github.com/ggml-org/llama.cpp/releases/download/"
|
||||
"b9022/llama-b9022-bin-ubuntu-x64.tar.gz"
|
||||
),
|
||||
}
|
||||
],
|
||||
}
|
||||
with patch.object(_mod, "fetch_json", return_value = _stub_lemonade_release()):
|
||||
plan = direct_upstream_release_plan(release, host, "ggml-org/llama.cpp", "latest")
|
||||
assert plan is not None, "Linux ROCm host should produce a direct plan"
|
||||
kinds = [a.install_kind for a in plan.attempts]
|
||||
sources = [a.source_label for a in plan.attempts]
|
||||
assert "linux-rocm" in kinds, f"lemonade ROCm attempt missing; got {kinds}"
|
||||
assert sources[0] == "lemonade", f"lemonade must be the first attempt; got {sources}"
|
||||
assert "gfx1151" in plan.attempts[0].name
|
||||
|
|
|
|||
File diff suppressed because it is too large
Load diff
|
|
@ -2556,7 +2556,9 @@ $LlamaCppDir = Join-Path $UnslothHome "llama.cpp"
|
|||
$NeedLlamaSourceBuild = $false
|
||||
$SkipPrebuiltInstall = $false
|
||||
$RequestedLlamaTag = if ($env:UNSLOTH_LLAMA_TAG) { $env:UNSLOTH_LLAMA_TAG } else { $DefaultLlamaTag }
|
||||
$HelperReleaseRepo = "ggml-org/llama.cpp"
|
||||
# GPU Windows (CUDA / ROCm) installs the fork's app-* prebuilts; CPU-only stays
|
||||
# on ggml-org (the fork ships no windows-cpu bundle). Mirrors setup.sh's routing.
|
||||
$HelperReleaseRepo = if ($HasNvidiaSmi -or $HasROCm) { "unslothai/llama.cpp" } else { "ggml-org/llama.cpp" }
|
||||
$LlamaPr = if ($env:UNSLOTH_LLAMA_PR) { $env:UNSLOTH_LLAMA_PR.Trim() } else { "" }
|
||||
|
||||
$LlamaPrForce = if ($env:UNSLOTH_LLAMA_PR_FORCE) { $env:UNSLOTH_LLAMA_PR_FORCE.Trim() } else { $DefaultLlamaPrForce }
|
||||
|
|
@ -2655,20 +2657,25 @@ if ($env:UNSLOTH_LLAMA_FORCE_COMPILE -eq "1") {
|
|||
if (Test-Path -LiteralPath $LlamaCppDir) {
|
||||
substep "Existing llama.cpp install detected -- validating staged prebuilt update before replacement"
|
||||
# If the existing install is the wrong kind (e.g. windows-cpu on a ROCm
|
||||
# machine that should have windows-hip), remove it so the installer is
|
||||
# machine that should have windows-rocm), remove it so the installer is
|
||||
# forced to download the correct variant rather than skipping on tag match.
|
||||
$existingMetaPath = Join-Path $LlamaCppDir "UNSLOTH_PREBUILT_INFO.json"
|
||||
if (Test-Path $existingMetaPath) {
|
||||
try {
|
||||
$existingMeta = Get-Content $existingMetaPath -Raw | ConvertFrom-Json
|
||||
$existingKind = $existingMeta.install_kind
|
||||
# A name-inferred gfx arch (Adrenalin-only, no confirmed runtime)
|
||||
# still wants the GPU (windows-hip) build -- the lemonade prebuilt
|
||||
# bundles its own runtime. Treat a known arch as ROCm-capable here,
|
||||
# mirroring the --rocm-gfx forward below.
|
||||
$expectedKind = if ($HasROCm -or $script:ROCmGfxArch) { "windows-hip" } elseif ($HasNvidiaSmi) { "windows-cuda" } else { "windows-cpu" }
|
||||
if ($existingKind -and $existingKind -ne $expectedKind) {
|
||||
substep "Removing mismatched llama.cpp install (found '$existingKind', need '$expectedKind')..."
|
||||
# A ROCm host may legitimately carry the fork's windows-rocm bundle
|
||||
# or the upstream windows-hip fallback, so accept either and never
|
||||
# treat a valid ROCm install as mismatched. A name-inferred gfx
|
||||
# arch (Adrenalin-only, no confirmed runtime) still counts as
|
||||
# ROCm-capable -- the lemonade prebuilt bundles its own runtime,
|
||||
# mirroring the --rocm-gfx forward below. NOTE: this block is
|
||||
# currently inert -- write_prebuilt_metadata does not persist an
|
||||
# install_kind key, so $existingKind is always null. If that changes,
|
||||
# add the remaining host kinds (e.g. windows-arm64) before relying on it.
|
||||
$expectedKinds = if ($HasROCm -or $script:ROCmGfxArch) { @("windows-rocm", "windows-hip") } elseif ($HasNvidiaSmi) { @("windows-cuda") } else { @("windows-cpu") }
|
||||
if ($existingKind -and ($existingKind -notin $expectedKinds)) {
|
||||
substep "Removing mismatched llama.cpp install (found '$existingKind', need one of: $($expectedKinds -join ', '))..."
|
||||
Remove-Item -Recurse -Force -LiteralPath $LlamaCppDir -ErrorAction SilentlyContinue
|
||||
}
|
||||
} catch {
|
||||
|
|
@ -2687,8 +2694,7 @@ if ($env:UNSLOTH_LLAMA_FORCE_COMPILE -eq "1") {
|
|||
"$PSScriptRoot\install_llama_prebuilt.py",
|
||||
"--install-dir", $LlamaCppDir,
|
||||
"--llama-tag", $RequestedLlamaTag,
|
||||
"--published-repo", $HelperReleaseRepo,
|
||||
"--simple-policy"
|
||||
"--published-repo", $HelperReleaseRepo
|
||||
)
|
||||
if ($HasROCm) {
|
||||
$prebuiltArgs += "--has-rocm"
|
||||
|
|
|
|||
|
|
@ -812,6 +812,7 @@ if ! command -v rocminfo >/dev/null 2>&1 && [ -x /opt/rocm/bin/rocminfo ]; then
|
|||
PATH="$PATH:/opt/rocm/bin"
|
||||
fi
|
||||
_setup_amd_detected=false
|
||||
_setup_nvidia_usable=false
|
||||
_setup_gfx_all=""
|
||||
_setup_mkt=""
|
||||
if command -v rocminfo >/dev/null 2>&1 && \
|
||||
|
|
@ -832,6 +833,7 @@ fi
|
|||
|
||||
if command -v nvidia-smi >/dev/null 2>&1 && \
|
||||
nvidia-smi -L 2>/dev/null | awk '/^GPU[[:space:]]+[0-9]+:/{found=1} END{exit !found}'; then
|
||||
_setup_nvidia_usable=true
|
||||
step "gpu" "NVIDIA GPU detected"
|
||||
elif [ "$_setup_amd_detected" = true ]; then
|
||||
_setup_vis="${HIP_VISIBLE_DEVICES:-${ROCR_VISIBLE_DEVICES:-}}"
|
||||
|
|
@ -910,34 +912,46 @@ _HOST_SYSTEM="$(uname -s 2>/dev/null || true)"
|
|||
_HOST_MACHINE="$(uname -m 2>/dev/null || true)"
|
||||
|
||||
# Pick the release repo install_llama_prebuilt.py plans against.
|
||||
# unslothai/llama.cpp ships only Linux CUDA bundles, so CPU-only Linux
|
||||
# x86_64 routes to ggml-org for bin-ubuntu-x64.tar.gz. Anything with a
|
||||
# GPU tool installed stays on unslothai (CUDA bundle / ROCm source build).
|
||||
# The fork ships CUDA (Linux x64/arm64, Windows), ROCm (Linux/Windows) and
|
||||
# macOS bundles. Only the plain CPU/Vulkan bundles still come from ggml-org, so
|
||||
# CPU-only Linux (x86_64 and arm64) routes there; GPU Linux, Windows and macOS
|
||||
# use unslothai.
|
||||
_LINUX_HAS_GPU=false
|
||||
for _GPU_TOOL in nvidia-smi rocminfo amd-smi hipconfig hipinfo; do
|
||||
if command -v "$_GPU_TOOL" >/dev/null 2>&1; then
|
||||
_LINUX_HAS_GPU=true
|
||||
break
|
||||
fi
|
||||
done
|
||||
# Route to the fork only for a usable GPU. NVIDIA counts only when a device is
|
||||
# actually enumerated (_setup_nvidia_usable, from the nvidia-smi -L probe above)
|
||||
# AND not hidden via CUDA_VISIBLE_DEVICES=-1 -- mirroring install_llama_prebuilt.py's
|
||||
# has_usable_nvidia. Mere nvidia-smi presence (CPU-only CUDA-toolkit containers,
|
||||
# broken drivers) or a hidden GPU therefore takes the ggml-org CPU prebuilt
|
||||
# instead of a slow source build. AMD is deliberately left on tooling presence,
|
||||
# not usability: an unusable NVIDIA host has a good CPU prebuilt to fall back to,
|
||||
# whereas tightening AMD would regress ROCm hosts exposing only hipconfig/hipinfo
|
||||
# into an unnecessary CPU build.
|
||||
if [ "$_setup_nvidia_usable" = true ] && [ "${CUDA_VISIBLE_DEVICES:-}" != "-1" ]; then
|
||||
_LINUX_HAS_GPU=true
|
||||
else
|
||||
for _GPU_TOOL in rocminfo amd-smi hipconfig hipinfo; do
|
||||
if command -v "$_GPU_TOOL" >/dev/null 2>&1; then
|
||||
_LINUX_HAS_GPU=true
|
||||
break
|
||||
fi
|
||||
done
|
||||
fi
|
||||
|
||||
if [ "$_HOST_SYSTEM" = "Darwin" ]; then
|
||||
_HELPER_RELEASE_REPO="ggml-org/llama.cpp"
|
||||
elif [ "$_HOST_SYSTEM" = "Linux" ] \
|
||||
if [ "$_HOST_SYSTEM" = "Linux" ] \
|
||||
&& [ "$_HOST_MACHINE" = "x86_64" ] \
|
||||
&& [ "$_LINUX_HAS_GPU" = false ]; then
|
||||
_HELPER_RELEASE_REPO="ggml-org/llama.cpp"
|
||||
elif [ "$_HOST_SYSTEM" = "Linux" ] \
|
||||
&& { [ "$_HOST_MACHINE" = "aarch64" ] || [ "$_HOST_MACHINE" = "arm64" ]; } \
|
||||
&& [ "$_LINUX_HAS_GPU" = false ]; then
|
||||
# Linux ARM64 (Ampere Altra, Raspberry Pi 5, GitHub `ubuntu-24.04-arm`,
|
||||
# CPU-only Jetson rescue mode, ...). unslothai/llama.cpp only ships
|
||||
# the Linux CUDA bundles, so without this branch the prebuilt
|
||||
# resolver returns 0 attempts on every release and the installer
|
||||
# falls all the way back to a source build. Upstream ggml-org ships
|
||||
# CPU-only Linux ARM64 (Ampere Altra, Raspberry Pi 5, GitHub
|
||||
# `ubuntu-24.04-arm`, CPU-only Jetson rescue mode, ...). The fork ships no
|
||||
# arm64 CPU bundle, so without this branch the prebuilt resolver returns 0
|
||||
# attempts and the installer falls back to a source build. ggml-org ships
|
||||
# llama-bNNNN-bin-ubuntu-arm64.tar.gz from at least b9072 onward.
|
||||
_HELPER_RELEASE_REPO="ggml-org/llama.cpp"
|
||||
else
|
||||
# GPU Linux (x64 CUDA/ROCm, arm64 CUDA), Windows (CUDA/ROCm), and macOS.
|
||||
_HELPER_RELEASE_REPO="unslothai/llama.cpp"
|
||||
fi
|
||||
unset _GPU_TOOL
|
||||
|
|
@ -1000,7 +1014,6 @@ else
|
|||
--install-dir "$LLAMA_CPP_DIR"
|
||||
--llama-tag "$_REQUESTED_LLAMA_TAG"
|
||||
--published-repo "$_HELPER_RELEASE_REPO"
|
||||
--simple-policy
|
||||
)
|
||||
if [ -n "${UNSLOTH_LLAMA_RELEASE_TAG:-}" ]; then
|
||||
_PREBUILT_CMD+=(--published-release-tag "$UNSLOTH_LLAMA_RELEASE_TAG")
|
||||
|
|
@ -1555,7 +1568,6 @@ if [ "$_LLAMA_CPP_DEGRADED" = true ] \
|
|||
--install-dir "$LLAMA_CPP_DIR"
|
||||
--llama-tag "$_REQUESTED_LLAMA_TAG"
|
||||
--published-repo "ggml-org/llama.cpp"
|
||||
--simple-policy
|
||||
--cpu-fallback
|
||||
)
|
||||
# Trust the installer's exit code: it validates the server before exiting 0,
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue