Fix/adjust diffusion: round 25 P1 batch for PR #5754
Five P1 findings from round 25 reviewer aggregate. 1. routes/datasets.py: /download-progress now reuses the same identifier hardening that round 24 added to the model route. Token-shaped repo_ids like owner/hf_abcdefghij0123456789 used to pass the cheap _is_valid_repo_id regex and end up in warning logs. 2. routes/models.py: extend the llama.cpp cache-delete guard to path-ownership matching. A GGUF chat model loaded via a local HF snapshot path under models--owner--repo/snapshots used to slip past the owner/repo string compare and could be rmtree'd while llama-server still mmap'd it. Shares one fail-closed HF cache scan and an _owned_cache_path_matches helper with the safetensors and diffusion guards (round 25 also dedupes the diffusion-specific rescan). 3. routes/models.py: extend the safetensors cache-delete guard the same way for safetensors models loaded from local snapshot paths. 4. utils/datasets/llm_assist.py: _run_with_helper and _run_multi_pass_advisor now acquire the global llama backend via routes.inference.get_llama_cpp_backend instead of instantiating a private LlamaCppBackend. _gpu_workload_busy_for_helper already ensures the global backend is idle on entry, so this is safe, and it makes the helper/advisor load visible to the global delete guards (loading_model_identifier and friends). 5. requirements/studio.txt: bump huggingface-hub from 0.36.2 pin to 1.3.0,<2.0 floor and mirror the no-torch-runtime.txt transformers and tokenizers constraints. Fresh installs from studio.txt used to resolve transformers 5.x with hub 0.36.2, which crashed Flux2KleinPipeline import on missing is_offline_mode the first time the user hit /api/inference/images/load. Includes merge of origin/main (PR #5753 install pin bumps and the mlx export save_method fix from #5727) so the PR diff stops showing silent reverts of those landed changes. Tests: PYTHONPATH=studio/backend pytest test_diffusion_backend.py test_diffusion_routes.py test_cached_gguf_routes.py test_llama_cpp_cache_aware_disk_check.py test_inference_model_validation.py test_models_get_model_config_case_resolution.py ==> 105 passed locally. The 15 flash-attention test failures and the test_studio_api SDK suite errors reproduce on HEAD without these changes (pre-existing, unrelated infrastructure).
This commit is contained in:
parent
3df9386ac7
commit
7b5fe1cf10
4 changed files with 163 additions and 74 deletions
|
|
@ -245,9 +245,18 @@ def _run_with_helper(prompt: str, max_tokens: int = 256) -> Optional[str]:
|
|||
|
||||
backend = None
|
||||
try:
|
||||
from core.inference.llama_cpp import LlamaCppBackend
|
||||
# Round 25 P1 #4: use the GLOBAL llama backend instead of a
|
||||
# private ``LlamaCppBackend()`` instance. The private instance
|
||||
# was invisible to ``DELETE /api/models/delete-cached`` and the
|
||||
# other global delete guards because they inspect the singleton
|
||||
# returned by ``get_llama_cpp_backend()``. A concurrent cache
|
||||
# delete could rmtree the helper's mid-flight download or
|
||||
# mmap'd snapshot. ``_gpu_workload_busy_for_helper`` above
|
||||
# already ensures the global backend is idle before we reach
|
||||
# here, so taking it over is safe.
|
||||
from routes.inference import get_llama_cpp_backend
|
||||
|
||||
backend = LlamaCppBackend()
|
||||
backend = get_llama_cpp_backend()
|
||||
logger.info(f"Loading helper model: {repo} ({variant})")
|
||||
|
||||
ok = backend.load_model(
|
||||
|
|
@ -641,9 +650,15 @@ def _run_multi_pass_advisor(
|
|||
|
||||
backend = None
|
||||
try:
|
||||
from core.inference.llama_cpp import LlamaCppBackend
|
||||
# Round 25 P1 #4: mirror ``_run_with_helper`` and acquire the
|
||||
# GLOBAL llama backend so cache-delete and unload guards see
|
||||
# this advisor load via the singleton's
|
||||
# ``loading_model_identifier`` / ``model_identifier``. The
|
||||
# round 23/24 ``_gpu_workload_busy_for_helper`` already
|
||||
# blocks reach here unless the global llama backend is idle.
|
||||
from routes.inference import get_llama_cpp_backend
|
||||
|
||||
backend = LlamaCppBackend()
|
||||
backend = get_llama_cpp_backend()
|
||||
logger.info(f"Loading advisor model: {repo} ({variant})")
|
||||
t0 = time.monotonic()
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue