ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Studio: make GGUF disk-space preflight cache-aware (#5012)

Browse source 
* Studio: make GGUF disk-space preflight cache-aware

The pre-download disk check in LlamaCppBackend.load_model compared the
repo's total GGUF size against free disk without crediting bytes
already present in the Hugging Face cache. Re-loading a large cached
model (e.g. MiniMax-M2.7-GGUF at 131 GB) then failed cold with
"Not enough disk space to download any variant" whenever free disk
was below the full weight footprint, even though nothing actually
needed to be downloaded.

Subtract bytes already on disk via try_to_load_from_cache before
comparing against free space. A partial blob (interrupted download) is
not credited, so a second attempt still allocates room to finish the
download. The log line now also surfaces how much is already cached.

Adds tests/test_llama_cpp_cache_aware_disk_check.py covering the
fully-cached, partial-cache-insufficient-disk, partial-cache-enough-disk,
cold-cache, incomplete-blob, and zero-size-path-info cases. Sparse
tempfiles keep the GB-scale scenarios cheap to simulate.

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
This commit is contained in:
Daniel Han 2026-04-14 08:53:37 -07:00 • committed by GitHub
commit b2f80f210e
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
2 changed files with 272 additions and 3 deletions
Download patch file Download diff file Expand all files Collapse all files

32
studio/backend/core/inference/llama_cpp.py
View file

@ -904,10 +904,34 @@ class LlamaCppBackend:
try:
import os
from huggingface_hub import get_paths_info
from huggingface_hub import get_paths_info, try_to_load_from_cache
path_infos = list(get_paths_info(hf_repo, all_gguf_files, token = hf_token))
total_download_bytes = sum((p.size or 0) for p in path_infos)
total_bytes = sum((p.size or 0) for p in path_infos)
# Subtract bytes already present in the HF cache so we only
# preflight against what we actually have to download. Without
# this, re-loading a cached large model (e.g. MiniMax-M2.7-GGUF
# at 131 GB) fails cold whenever free disk is below the full
# weight footprint, even though nothing needs downloading.
already_cached_bytes = 0
for p in path_infos:
if not p.size:
continue
try:
cached_path = try_to_load_from_cache(hf_repo, p.path)
except Exception:
cached_path = None
if isinstance(cached_path, str) and os.path.exists(cached_path):
try:
on_disk = os.path.getsize(cached_path)
except OSError:
on_disk = 0
# Count as satisfied only when the full blob is present.
if on_disk >= p.size:
already_cached_bytes += p.size
total_download_bytes = max(0, total_bytes - already_cached_bytes)
if total_download_bytes > 0:
cache_dir = os.environ.get(
@ -919,9 +943,11 @@ class LlamaCppBackend:
total_gb = total_download_bytes / (1024**3)
free_gb = free_bytes / (1024**3)
cached_gb = already_cached_bytes / (1024**3)
logger.info(
f"GGUF download: {total_gb:.1f} GB needed, "
f"GGUF download: {total_gb:.1f} GB needed "
f"({cached_gb:.1f} GB already cached), "
f"{free_gb:.1f} GB free on disk"
)