ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Studio: in-app Update llama.cpp button to install the latest prebuilt (#6097)

Browse source 
Adds an in-app "Update llama.cpp" banner and button to Unsloth Studio. When the installed prebuilt is behind the latest published release, a non-invasive banner appears; clicking Update downloads the latest prebuilt for this host and swaps it in place in the background, with no restart.

Detection reuses the freshness check from #5529. The update re-runs install_llama_prebuilt.py the same way setup.sh and setup.ps1 do after #5963: it forwards the published repo and the AMD gfx target derived from the install marker, and does not pass the removed --simple-policy or the arm64-only --cpu-fallback.

While the installer swaps binaries the backend enters a maintenance state (flag set under the serial load lock, active server unloaded) so a concurrent load cannot start a server from a half-swapped binary; the next load uses the new build. The banner also handles refused responses and jobs started in another tab so it never sticks on "Updating...".

Verified end to end on an NVIDIA B200: installed b9493, detected the update, applied it, and confirmed the binary at the same path advanced to b9585 in the same process. Hermetic backend tests and the frontend type-check pass.
This commit is contained in:
Daniel Han 2026-06-10 10:04:26 -07:00 • committed by GitHub
commit dab0b77673
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
9 changed files with 1211 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

7
studio/backend/core/inference/llama_cpp.py
View file

@ -719,6 +719,9 @@ class LlamaCppBackend:
# Wraps load_model() end-to-end so concurrent loads serialise and never
# coexist as two llama-server processes (#5401).
self._serial_load_lock = threading.Lock()
# Set by the in-app updater while it swaps prebuilt binaries; load_model()
# rejects fast so no server starts from a half-swapped binary.
self._llama_update_in_progress = False
# Last extra_args / requested n_ctx, preserved across unload so the chat
# UI's /unload+/load Apply path can inherit them (#5401).
# ``_extra_args_source`` records the (model_identifier, hf_variant) the
@ -2806,6 +2809,10 @@ class LlamaCppBackend:
# Serialise the whole load so concurrent /load calls never leave two
# llama-server processes alive (#5401 / #5161). Doesn't block /unload.
with self._serial_load_lock:
# In-app update swapping binaries: refuse fast (set under this lock,
# so any in-flight load has drained) instead of using a half-swapped one.
if getattr(self, "_llama_update_in_progress", False):
raise RuntimeError("llama.cpp is updating; try again in a moment.")
# Duplicate /load that raced past the route check: do nothing if the
# live server already satisfies this request.
if self._already_in_target_state(