Stop an ignored-cancel sd-server, guard deletes during diffusion training, repair unusable managed binaries

sd-server does not interrupt an in-flight job, so when it ignores a cancel the
grace branch abandoned the poll and reported cancellation while the native job
kept a core (or the GPU) busy to completion and held the server's job slot. The
comment said the caller stops the server, but only unload does that
immediately: a superseding load stops it after its multi-gigabyte download, and
a load that then fails never gets there. Stop it here, as the deadline branch
already does.

DELETE /api/models/delete-finetuned checked only the LLM trainer, so it could
rmtree the output directory a live diffusion LoRA run was about to write its
adapter into. Consult the diffusion training service too, like the dataset
mutation and model-load routes.

find_sd_*_binary only checks is_file(), so an interrupted extraction (or a
prebuilt for the wrong CPU) left a present-but-unrunnable binary the installer
never retried: every load probed it, fell back to diffusers, and native
inference stayed off until the directory was deleted by hand. Probe it and
reinstall, but only for a copy under the installer-owned root -- SD_CLI_PATH,
UNSLOTH_SD_CPP_PATH, an in-tree build and anything on PATH are the user's.
This commit is contained in:
Daniel Han 2026-07-27 09:32:25 +00:00
commit a2342f80df
7 changed files with 162 additions and 14 deletions

View file

@ -366,6 +366,10 @@ def test_img_gen_abandons_when_cancel_not_honored(patched):
s = _server_with(popen, client)
with pytest.raises(SdCppCancelled):
s.img_gen({"prompt": "x"}, cancel_event = cancel, poll_interval = 0.01)
# And the process is stopped, not left running the abandoned job: sd-server does not interrupt
# an in-flight job, so a server that ignored the cancel would otherwise burn a core (or the GPU)
# to completion and hold its job slot against the next request.
assert not s.is_alive()
def test_img_gen_non_dict_submit_json_raises(patched):