Fix/adjust diffusion: round 13 P1+P2 batch for PR #5754

Round 13 reviewer aggregate (logs/review_round13_aggregate.md):

P1 fixes:
- routes/export.py load_checkpoint refuses (409) when an export job
  is currently active, mirroring the chat/diffusion/training handoff
  guards. ``is_export_active`` absence is tolerated for older / mocked
  backends.
- core/inference/diffusion.py local-path GGUF loader now accepts
  relative directories (Studio exports surface as ``exports/my-flux``)
  and confines ``gguf_filename`` to the chosen repo via
  ``_resolve_local_gguf_child``: absolute filenames, ``..`` segments,
  and Windows separators are rejected before any file is opened.
- core/inference/diffusion.py status() exposes ``active_gguf_filename``
  alongside the pending variant so delete guards can pair each owned
  repo with the GGUF variant it actually owns.
- routes/models.py cache delete + finetuned delete adopt a shared
  ``_diffusion_owned_targets`` + ``_variant_delete_is_safe_for_owned_gguf``
  helper. Per-variant deletes during a swap-in-flight cannot remove
  the active variant while the pending variant is loading.
- core/inference/llama_cpp.py publishes ``loading_model_identifier``
  before ``_download_gguf`` starts and clears it in ``finally``. Cache
  delete (routes/models.py) and the cross-workload release helpers
  (routes/inference.py::_release_llama_for and
  diffusion.py::_release_chat_backend_for_diffusion) consult it so a
  multi-GB HF download cannot be rmtree'd or be ignored by /images/load
  while still in flight.

P2 fixes:
- core/inference/diffusion.py adds
  ``generate_image_with_metadata`` + ``async_generate_with_metadata``;
  /images/generate uses it so the response model/family reflect the
  pipeline that actually produced the image even if an unload races
  the route.
- core/inference/diffusion.py: ``base_repo`` only applies when picking
  a GGUF quant. Filling Base diffusers repo while loading a full
  diffusers repo no longer silently swaps the load target.
- core/inference/diffusion.py: failed device placement / offload now
  drops pipe + transformer references explicitly before drain so
  partial allocations cannot keep VRAM around.
- core/inference/diffusion.py: torch/diffusers imports surface as a
  clear RuntimeError naming the missing dependency.
- core/inference/diffusion.py: _smart_base_repo splits on both POSIX
  and Windows separators so ``C:\\Users\\me\\base\\FLUX.2-klein-4B-GGUF``
  no longer picks the Base 4B variant via the parent dir.

Tests:
- 6 new regression cases (Windows leaf, traversal/backslash rejection,
  relative-dir local load, metadata snapshot, lock serialisation).
- All 59 diffusion backend + route tests pass.
This commit is contained in:
Daniel Han-Chen 2026-05-25 05:57:32 +00:00
commit ae41bfdbfd
7 changed files with 716 additions and 205 deletions

View file

@ -612,6 +612,13 @@ class LlamaCppBackend:
self._process: Optional[subprocess.Popen] = None
self._port: Optional[int] = None
self._model_identifier: Optional[str] = None
# Pending-load identifier: set BEFORE _download_gguf starts and
# cleared after the load finishes (success or failure). Delete
# guards and cross-workload handoff helpers read it via
# ``loading_model_identifier`` so a multi-GB HF download cannot
# have its cache rmtree'd or be ignored by /images/load,
# /training/start, /export/load while it is still resolving.
self._loading_model_identifier: Optional[str] = None
self._gguf_path: Optional[str] = None
self._hf_repo: Optional[str] = None
self._hf_variant: Optional[str] = None
@ -713,6 +720,19 @@ class LlamaCppBackend:
def model_identifier(self) -> Optional[str]:
return self._model_identifier
@property
def loading_model_identifier(self) -> Optional[str]:
"""Identifier of a load currently in progress, or None.
Populated while ``_download_gguf`` is fetching the GGUF for a
new ``load_model`` call. Cleared in the surrounding
``finally`` block, so a failed load leaves it None. Delete
guards in ``routes/models.py`` and handoff helpers in
``routes/inference.py`` consult this so a long HF download
cannot have its destination rmtree'd or be ignored by a
concurrent /images/load that thinks llama-server is idle."""
return self._loading_model_identifier
@property
def is_vision(self) -> bool:
return self._is_vision
@ -2673,25 +2693,44 @@ class LlamaCppBackend:
# Scope HF_HUB_OFFLINE to the download block only when DNS is
# dead; cleanup runs even on exception so a transient hiccup
# at the start of one load cannot quarantine future loads.
if hf_repo:
with _hf_offline_if_dns_dead():
model_path = self._download_gguf(
hf_repo = hf_repo,
hf_variant = hf_variant,
hf_token = hf_token,
)
# Auto-download mmproj for vision models
if is_vision and not mmproj_path:
mmproj_path = self._download_mmproj(
#
# Publish ``_loading_model_identifier`` BEFORE entering the
# download so /delete-cached and the cross-workload handoff
# helpers can see a multi-GB pending load: previously they
# only consulted ``model_identifier``, which the success
# path sets later (see "Set identifier early" below). That
# left a window where the user could rmtree the cache the
# download was still writing to, or start /images/load
# while llama-server was about to come up on the same GPU.
# Cleared in ``finally`` so failed / cancelled loads do not
# leak the pending state.
self._loading_model_identifier = model_identifier
try:
if hf_repo:
with _hf_offline_if_dns_dead():
model_path = self._download_gguf(
hf_repo = hf_repo,
hf_variant = hf_variant,
hf_token = hf_token,
)
elif gguf_path:
if not Path(gguf_path).is_file():
raise FileNotFoundError(f"GGUF file not found: {gguf_path}")
model_path = gguf_path
else:
raise ValueError("Either gguf_path or hf_repo must be provided")
# Auto-download mmproj for vision models
if is_vision and not mmproj_path:
mmproj_path = self._download_mmproj(
hf_repo = hf_repo,
hf_token = hf_token,
)
elif gguf_path:
if not Path(gguf_path).is_file():
raise FileNotFoundError(
f"GGUF file not found: {gguf_path}"
)
model_path = gguf_path
else:
raise ValueError(
"Either gguf_path or hf_repo must be provided"
)
finally:
self._loading_model_identifier = None
# Set identifier early so _read_gguf_metadata can use it for DeepSeek detection
self._model_identifier = model_identifier