unsloth/studio/backend/core
Daniel Han e73a89ff82
Studio: warn when a GPU model silently loaded on CPU (#6339)
* Studio: warn when a GPU model silently loaded on CPU

llama-server can serve HTTP 200 while running a model entirely on CPU when its GPU backend fails to init, so Studio could run a GGUF on CPU without saying so (#5807 / #5106 / #5830). The silent-CPU warning already exists but stopped firing on current llama.cpp because _classify_gpu_offload keyed only on the dropped 'model buffer size' lines. Add a shared classify_gpu_offload_lines (offloaded N/M counts, GPU model-buffer markers excluding _Host, device_info disconfirm-only) and delegate to it so the warning fires again. Log-only: no install or load behavior changes.

Pure classification of already-captured startup log lines, run once after load; no new subprocess, no slowdown.

* Studio: key the CPU-offload warning on the main model, not a draft

With MTP/speculative decoding llama-server logs 'offloaded N/M layers to GPU' twice: once for the main model and once for the small draft model. The old scan returned True on any non-zero count, so a drafter that fits on GPU while the main GGUF runs on CPU suppressed the warning (the Qwen3.6-27B-MTP case). Decide on the line with the most layers (the main model) instead, so a drafter cannot mask a main model on CPU.
2026-06-15 23:06:44 -07:00
..
data_recipe Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
export fix: respect absolute export paths to prevent cross-drive copy failures (WinError 112) (#6088) 2026-06-12 12:52:57 +02:00
inference Studio: warn when a GPU model silently loaded on CPU (#6339) 2026-06-15 23:06:44 -07:00
rag Studio: project sources backed by RAG (#6205) 2026-06-12 15:42:51 +02:00
training Studio: keep training from failing when a namespace-package shadows unsloth (#6269) 2026-06-15 06:06:48 -07:00
__init__.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
_torchao_stub.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
tool_healing.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00