unsloth/studio/backend/utils
Daniel Han d97777e74a Studio: reliably reap an orphaned llama-server so GPU loads are not stuck on CPU
Once a model overflowed VRAM and Studio later died in a way that bypasses the
graceful path (SIGHUP from a closed terminal, SIGKILL/OOM, or a direct-uvicorn
launch), the llama-server child was orphaned and kept holding GPU memory. GPU
placement is recomputed every load from a live free-VRAM probe, so the leftover
process made every subsequent load (even a tiny model, even after a restart)
spill to system RAM until it was killed by hand.

The main llama-server spawn now records its PID to a pidfile under the active
studio root, removed on _kill_process. The startup reaper kills that exact PID
first (path-independent, so it catches an orphan the install-root match misses),
verifying it is still a llama-server to guard against PID reuse; the pidfile
only ever names a Studio-spawned server, so unrelated user processes (vllm,
games) are never touched. The existing root-gated enumeration stays as a
fallback. A belt-and-suspenders kill is also wired into the FastAPI lifespan
shutdown (before hardware/cache teardown) to cover the direct-uvicorn path that
run.py's signal handler does not.

PR_SET_PDEATHSIG is intentionally not used on the main spawn: llama-server is
launched on a pooled asyncio.to_thread worker, and a thread-scoped death signal
could prematurely kill a healthy server. The reaper runs before any model loads
on the next start, so it fully covers the user-visible problem.

Adds tests for the pidfile reap (kills a recorded live server, skips a reused
non-llama PID, cleans a stale/missing pidfile, clears on kill) and for the
lifespan kill (runs first, errors swallowed).
2026-06-18 07:07:24 +00:00
..
datasets Studio: training survives a non-writable HF datasets cache (#6148) 2026-06-10 08:22:47 -07:00
hardware Studio: pin CUDA_DEVICE_ORDER=PCI_BUS_ID and list GPUs at startup (#6353) 2026-06-17 22:40:04 -07:00
inference Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
models Studio: add 'Load on selection' toggle to configure load options before loading (#6348) 2026-06-17 16:24:10 +01:00
paths Studio: fix training output dir escaping outputs root for models on another drive (#6293) 2026-06-13 04:06:17 -07:00
.gitkeep root studio folder 2026-02-02 09:13:49 +00:00
__init__.py Final cleanup 2026-03-12 18:28:04 +00:00
_studio_release_build.py Add Studio web update banner and release version display (#5308) 2026-05-11 18:24:01 +04:00
api_errors.py Studio: improve OpenAI- and Anthropic-compatible API spec compliance (#6010) 2026-06-09 17:13:25 +02:00
cache_cleanup.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
cpu_threads.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
downsample.py Formatting: ruff line-length 100, kwarg-spacing passes, drop blank after short local imports (#6079) 2026-06-08 04:24:13 -07:00
helper_precache_settings.py Studio: make Helper LLM startup pre-cache opt in (#6113) 2026-06-09 15:28:34 +02:00
hf_xet_fallback.py Studio: Xet-primary model downloads with automatic HTTP fallback on stall (#6372) 2026-06-16 06:17:54 -07:00
host_policy.py Studio: enable stdio MCP servers on a loopback bind (#6295) 2026-06-15 03:02:32 +01:00
lifespan_shutdown.py Studio: reliably reap an orphaned llama-server so GPU loads are not stuck on CPU 2026-06-18 07:07:24 +00:00
llama_cpp_freshness.py Studio: stop the llama.cpp update banner flickering and show the download size (#6338) 2026-06-17 21:28:54 -07:00
llama_cpp_update.py Studio: stop the llama.cpp update banner flickering and show the download size (#6338) 2026-06-17 21:28:54 -07:00
native_path_leases.py Formatting: ruff line-length 100, kwarg-spacing passes, drop blank after short local imports (#6079) 2026-06-08 04:24:13 -07:00
studio_version.py Studio: llama.cpp update banner redesign, About tab license info, UI polish (#6196) 2026-06-11 09:27:34 -07:00
subprocess_compat.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
transformers_version.py Studio: improve logging for dynamic transformers version switching (#6108) 2026-06-15 23:31:43 -07:00
update_status.py Studio: make code comments and docstrings more succinct (#6029) 2026-06-08 23:07:28 -07:00
upload_limits.py Formatting: ruff line-length 100, kwarg-spacing passes, drop blank after short local imports (#6079) 2026-06-08 04:24:13 -07:00
utils.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00
wheel_utils.py Reduce and tighten code comments and docstrings repo-wide (#6095) 2026-06-08 23:09:51 -07:00