3a0bd55bdf
Add a lean, backend-agnostic memory policy that picks a CPU-offload policy and VAE tiling/slicing from measured free device memory vs the model's estimated resident footprint, then applies it to the built pipeline. auto stays resident when the model fits (byte-identical to the prior resident path), and falls to whole-module offload when tight; fast/balanced/low_vram are explicit overrides. Sequential submodule offload is unreliable for GGUF transformers on diffusers 0.38, so it falls back to whole-module offload and status reports the policy actually engaged. Verified on Z-Image-Turbo Q4_K_M (B200): auto reproduces the resident image with no VRAM/latency regression (PSNR inf); balanced/low_vram cut generation peak VRAM 47.9% (15951 -> 8318 MB) with byte-identical output, at the expected latency cost. 73 prior + 35 new CPU tests pass. |
||
|---|---|---|
| .. | ||
| .gitkeep | ||
| __init__.py | ||
| auth.py | ||
| data_recipe.py | ||
| datasets.py | ||
| export.py | ||
| inference.py | ||
| mcp_servers.py | ||
| models.py | ||
| providers.py | ||
| responses.py | ||
| training.py | ||
| users.py | ||