Training progress:
- Show row counts in status messages: "Loaded dataset from HuggingFace:
Open-Orca/OpenOrca (4,233,923 rows)" instead of just the dataset name
- Emit "Formatting dataset (N rows)..." and "Applying chat template
(N rows)..." status updates so users see progress during the
preprocessing stages that previously appeared stuck
Deferred llama.cpp compilation:
- Add LlamaCppBuilder that runs cmake build in a background thread
at server startup if the llama-server binary is missing
- Studio starts immediately and is usable for training/non-GGUF tasks
while llama.cpp compiles in the background
- GGUF model loads wait for the build to finish with a helpful message
- Add /api/inference/llama-cpp-status endpoint for build status
- Frontend shows "Waiting for llama.cpp to compile..." toast when
loading a GGUF while build is in progress
Previously only GGUF models showed download progress in Chat. Non-GGUF
models (safetensors, bnb quantized, etc.) showed a static message with
no progress indication. This adds progress tracking for all model types
and fixes several related issues.
Backend:
- Add /api/models/download-progress endpoint that checks the HF cache
blobs directory for completed and .incomplete files. Uses model_info()
(cached per repo) to determine expected total size for percentage.
- Add /api/models/cached-models endpoint that lists non-GGUF model repos
from the HF cache via scan_cache_dir().
- Fix progress stuck at 0.99: when no .incomplete files remain, report
1.0 immediately (blob deduplication can make byte totals mismatch).
Frontend:
- Remove the ggufVariant gate so download progress polling works for all
non-cached models, not just GGUFs.
- Use GGUF-specific endpoint when variant + expectedBytes available,
otherwise use the general download-progress endpoint.
- Fix toast stuck after load: check loadingModelRef.current before and
after the async poll to prevent overwriting the success toast.
- First poll at 500ms instead of waiting for the 2s interval.
- Show downloaded non-GGUF models in the Hub model picker "Downloaded"
section alongside GGUFs.
1. Progress endpoint now takes a variant parameter and only counts
.gguf files matching that variant (not all files in the repo cache,
which would include previously downloaded variants)
2. Tracks .incomplete files in HF blobs dir for in-progress single-shard
downloads, capping at 99% until the file is fully committed
3. Fixed loading text: "Loading model..." for cached, "Downloading
model..." for new downloads, with appropriate descriptions
4. Wording: "Downloading and loading model. Large models can take a
while." instead of "This may include downloading."
1. Loading text: shows "Loading model..." for cached models,
"Downloading model..." for new downloads. Toast description
adapts accordingly.
2. Download progress: polls /api/models/gguf-download-progress every
2s during downloads, updating the toast with percentage and GB
downloaded. Progress is estimated by checking the HF cache folder
size against the expected total bytes.
3. Passes isDownloaded and expectedBytes through the full chain from
variant click to selectModel for accurate UI state.
1. Interruptible downloads: load_model now checks a cancel event
between shard downloads. unload_model sets the event so cancel
stops the download at the next shard boundary.
2. /api/models/cached-gguf endpoint: scans the HF cache for
already-downloaded GGUF repos with their total size and cache path.
3. "Downloaded" section in Hub model picker: shows cached GGUF repos
at the top (before Recommended) so users can quickly re-load
previously downloaded models without re-downloading.
Replace Python-side GGUF download with llama-server's native -hf flag for
HuggingFace repos. Add frontend variant picker so users can choose
quantization (Q4_K_M, Q8_0, BF16, etc.) with file sizes. Fix vision
detection via mmproj files instead of hardcoding is_vision=False.
