11612f6dc9
- Run GGUF load_model in asyncio.to_thread so the event loop stays free for progress polling during download (was blocking all requests). - Extract download phase out of the lock in LlamaCppBackend.load_model so unload_model/cancel can take effect immediately during download. - Fix "downloaded" badge for split GGUFs: check total cached bytes across all shards vs expected size, not just first shard existence. - Respect CUDA_VISIBLE_DEVICES in /api/system GPU reporting so the frontend GGUF fit estimation uses actual available VRAM. - Sort tight variants (need CPU offload) smallest-first instead of largest-first -- closer to GPU budget = faster inference. - Fix cancel: use refs instead of React state for abort controller and toast ID so both cancel buttons (text + toast) work reliably. Make cancel synchronous (fire-and-forget unload) for instant UI response. Check abortCtrl.signal.aborted after loadModel returns to prevent ghost model state. Skip rollback and suppress errors on cancel. - Dynamic top 4 GGUF models fetched from HF API sorted by downloads, prepended to the default recommended list. - Remove turnAnchor="top" for auto-scroll to bottom during generation. - Set default toast duration to 10s (was infinite for loading toasts). - Deduplicate cached GGUF repos using scan_cache_dir API (fixes Qwen/X-GGUF vs qwen/x-gguf duplicates from lowercased HF cache). - Pre-compile repo_id validation regex to silence CodeQL ReDoS warning. - Change welcome text and default suggestion text. |
||
|---|---|---|
| .. | ||
| assets | ||
| auth | ||
| core | ||
| loggers | ||
| models | ||
| plugins | ||
| requirements | ||
| routes | ||
| state | ||
| tests | ||
| utils | ||
| __init__.py | ||
| colab.py | ||
| main.py | ||
| run.py | ||