When llama-server is built with shared libs (setup.sh default),
it needs libcudart.so.12 and other CUDA runtime libs. Add
/usr/local/cuda/lib64 and targets path to LD_LIBRARY_PATH so
the server starts correctly even when CUDA isn't on the system path.
Add granular status updates through the full preprocessing pipeline:
- "Downloading dataset: Open-Orca/OpenOrca..." before HF download
- "Downloaded Open-Orca/OpenOrca (4,233,923 rows)" after download
- "Formatting dataset (4,233,923 rows)..." before format step
- "Applying chat template to chatml_conversations (4,233,923 rows)..."
- "Dataset ready (4,233,923 samples, chatml_conversations format)"
Shows detected format name and row counts at each stage so users
can see progress through large dataset preprocessing instead of
a static "Loading and formatting dataset..." for minutes.
- Linux: CUDA detection via /usr/local/cuda*, Ninja preferred
- macOS: Metal backend auto-enabled (llama.cpp default), no CUDA
- Windows: CUDA via CUDA_PATH env and toolkit dirs, VS generator
fallback, binaries in build/bin/Release/, copy instead of symlink
- Reuse existing source (don't re-clone if CMakeLists.txt present)
- Both llama-server and llama-quantize verified and built
* fix: prefer existing CUDA_PATH toolkit to avoid version mismatch on multi-CUDA systems
* fix: validate GPU arch support before accepting CUDA toolkit (sm_120 + CUDA 12.4 fallback)
* debug: add temporary CUDA compatibility check print
* fix: auto-copy CUDA VS integration files when missing (No CUDA toolset found)
* fix: return false when nvcc --list-gpu-arch unavailable (reject old toolkit, scan for newer)
* fix: re-sanitize CUDA env vars before cmake build (survives Refresh-Environment)
* fix: use --list-gpu-code (sm_*) instead of --list-gpu-arch (compute_*) for arch probing
Training progress:
- Show row counts in status messages: "Loaded dataset from HuggingFace:
Open-Orca/OpenOrca (4,233,923 rows)" instead of just the dataset name
- Emit "Formatting dataset (N rows)..." and "Applying chat template
(N rows)..." status updates so users see progress during the
preprocessing stages that previously appeared stuck
Deferred llama.cpp compilation:
- Add LlamaCppBuilder that runs cmake build in a background thread
at server startup if the llama-server binary is missing
- Studio starts immediately and is usable for training/non-GGUF tasks
while llama.cpp compiles in the background
- GGUF model loads wait for the build to finish with a helpful message
- Add /api/inference/llama-cpp-status endpoint for build status
- Frontend shows "Waiting for llama.cpp to compile..." toast when
loading a GGUF while build is in progress
* studio: extract param count from model name as fallback
When HuggingFace API doesn't return totalParams for a model,
extract the param count from the model name (e.g. "Qwen3-0.6B"
-> "0.6B", "Llama-3.2-1B-Instruct" -> "1B"). Applied to both
the recommended list and HF search results.
* studio: read GGUF context_length via fast header parser, set max tokens
- Fast GGUF metadata reader (~30-55ms) parses only KV header, skips
tensor data and large arrays (tokenizer vocab etc)
- Extracts context_length and chat_template from GGUF metadata
- Returns context_length in LoadResponse for frontend to use
- Frontend sets maxTokens to actual context_length for GGUFs (e.g.
262144 for Qwen3.5-9B, 131072 for Qwen2.5-7B)
- Max Tokens slider shows "Max" and is locked for GGUFs
- Auto-load path also uses actual context_length from load response
- Toast auto-dismiss (5s) and close button for auto-load toast
* studio: GGUF TTS audio support (from PR #4318)
Add GGUF TTS audio generation via llama-server. When a GGUF model
loads, the backend probes its vocabulary to detect audio codecs
(SNAC/BiCodec/DAC/CSM/Whisper). If detected, the codec is pre-loaded
and the model is reported as audio to the frontend.
During chat, TTS models route to the audio generation path which sends
a per-codec prompt to llama-server's /completion endpoint, extracts
generated tokens/text, and decodes to WAV using AudioCodecManager.
Also strips base64 audio data from prior assistant messages to prevent
context overflow.
Co-authored-by: Manan Shah <mananshah511@gmail.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Remove package-lock.json from tracking
* studio: per-model inference defaults, GGUF max tokens fix, reasoning toggle
- Add inference_defaults.json with per-model-family sampling parameters
for ~50 families (Qwen3.5, Qwen3, Gemma-3, Llama-3, DeepSeek, etc.).
Values sourced from unslothai/docs and Ollama params blobs.
- Family-based lookup in inference_config.py: extracts model family from
identifier, matches against patterns (longest match first), merges with
priority: model-specific YAML > family JSON > default.yaml.
- Fix GGUF Max Tokens slider locked at "Max": store ggufContextLength
separately from maxTokens so the slider is adjustable (step=64).
- Fix Ministral YAML: top_p was literal string "default", now 0.95.
- Add reasoning toggle for thinking models (Qwen3.5, Qwen3, DeepSeek-R1,
DeepSeek-V3.1, etc.): detect enable_thinking support from GGUF chat
template metadata, pass --jinja to llama-server, send
chat_template_kwargs per-request. Frontend shows "Reasoning is ON/OFF"
pill button next to attachment button in composer.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: remove default system prompt injection
Backend was injecting "You are a helpful AI assistant." when no system
prompt was provided. Neither unslothai/docs nor Ollama specify a default
system prompt for most models. Now defaults to empty string, letting the
model's own chat template handle system behavior.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: use lightbulb icons and "Think" label for reasoning toggle
Lightbulb on when thinking enabled, lightbulb-off when disabled.
Label is just "Think" in both states; grayed out styling when off.
* studio: fix HTML file upload breaking chat
Replace SimpleTextAttachmentAdapter with custom TextAttachmentAdapter
(excludes text/html) and HtmlAttachmentAdapter that strips tags via
DOMParser, removing scripts/styles and extracting readable text content
instead of dumping raw HTML markup into the conversation.
* studio: show chat template in Configuration panel
Display the model's Jinja2 chat template in a new "Chat Template"
section under Settings (now open by default). For GGUFs, reads from
GGUF metadata; for safetensors, reads from tokenizer.chat_template.
Template is editable with a "Restore default chat template" button
that appears when modified. Section only shows when a model with a
chat template is loaded.
* studio: editable chat template with Apply & Reload
Chat template section now functional:
- Editing the template shows "Apply & Reload" (reloads model with
custom template) and "Revert changes" buttons
- For GGUFs: writes template to temp .jinja file, passes
--chat-template-file to llama-server on reload
- For non-GGUF: passes chat_template_override in load request
- Settings section now open by default
- selectModel supports forceReload to reload same model
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: fix DeepSeek reasoning detection and auto-load metadata
- Set _model_identifier before _read_gguf_metadata so DeepSeek
"thinking" template detection works (was always None before)
- Populate ggufContextLength, supportsReasoning, reasoningEnabled,
defaultChatTemplate in autoLoadSmallestModel GGUF path
* studio: add spacing before BETA badge in navbar
Add gap-1.5 on the logo Link container to space the BETA label
from the wordmark.
Co-authored-by: Imagineer99 <Imagineer99@users.noreply.github.com>
* studio: vertically center BETA badge with logo
---------
Co-authored-by: Manan Shah <mananshah511@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Imagineer99 <Imagineer99@users.noreply.github.com>
* Strip <think> blocks from LLM assist model output
* Add debug logging for raw LLM assist output
* Quiet llama-server logs, use structlog in llm_assist
* Fix think-tag stripping when response is inside tags
* Remove debug logging of raw model output
* Clarify GGUF download logs: show cache hit vs actual download
* Clarify heuristic-detected mapping in UI text
* Default helper model to Qwen3-4B-Instruct-2507 UD-Q4_K_XL
* Remove package-lock.json from tracking, add to .gitignore
* Auto-open mapping dialog on Start Training for custom_heuristic format
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Use last think block when extracting inner content (review feedback)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* Fix VLM GRPO matmul shape mismatch in _get_per_token_logps_and_entropies
VLM models (e.g. Qwen2.5-VL) can return logits [B*T, vocab_size] instead
of hidden states [B*T, hidden_dim] from their forward pass. When this
happens, chunked_hidden_states_selective_log_softmax tries to compute
logits @ lm_head.t() which fails with a shape mismatch.
Add a shape guard in the VLM branch of _get_per_token_logps_and_entropies:
check output.shape[-1] against lm_head.shape[1] (hidden_dim). When hidden
states are returned, the existing path is taken. When logits are returned,
scaling/softcapping/temperature are applied manually and
chunked_selective_log_softmax is used instead.
Also add chunked_selective_log_softmax to the import from unsloth_zoo.
The text-only branch (pixel_values is None) is unchanged.
Companion PR to unslothai/unsloth-zoo for grpo_accumulated_loss.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Remove redundant scaling in logits fallback path
When COMPILE_DISABLE=1 and the model returns logits directly, scaling
and softcapping are already applied by the model forward. Only
temperature (a GRPO training parameter) needs to be applied.
* Pass temperature to chunked_selective_log_softmax instead of manual cast
Use the new temperature parameter in chunked_selective_log_softmax
(added in companion zoo PR) to avoid casting the entire logits tensor
to float32 before the function call.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
The existing fix that removes use_reentrant=False from
gradient_checkpointing_kwargs was gated behind RLConfig_name ==
"GRPOConfig", so only GRPOConfig was protected. SFTConfig, DPOConfig,
KTOConfig, CPOConfig, ORPOConfig etc. were all still affected.
Remove the GRPOConfig guard so the fix applies to all compiled trainer
configs when TRL >= 0.27.0.
This is defense-in-depth alongside the unsloth_zoo fix that forces
use_reentrant=True in unsloth_checkpoint() itself.
- GGUF: use -c 0 for model's native context size (no 4096 cap)
- GGUF: hide Max Seq Length slider (irrelevant), set Max Tokens to Max
- Non-GGUF: default Max Tokens to 4096
- Max Tokens slider shows "Max" label when at ceiling for GGUFs
- Run non-GGUF load_model in asyncio.to_thread for progress polling
- Auto-load smallest downloaded model when chatting without selection
- Wait for in-progress model load before inference (modelLoading store flag)
- Recommended list: 4 GGUFs + 4 hub models after case-insensitive dedup
- Model selector waits for cached data before rendering
- Toast close button repositioned, Sampling section open by default
- Add logging to _get_repo_size_cached exception handler
- Use -c 0 for llama-server (model's native context size, no 4096 cap)
- Run non-GGUF backend.load_model in asyncio.to_thread for progress polling
- Auto-load smallest downloaded model when user chats without selecting one
- Wait for in-progress model load before inference (no "No model loaded" error)
- Add modelLoading flag to zustand store for cross-component coordination
- Dynamic top models: send 8 GGUFs + 8 hub models, frontend caps 4+4 after dedup
- Case-insensitive dedup: downloaded models correctly hide from recommended list
- Prevent duplicate toasts: guard against double selectModel calls
- Model selector waits for cached data before rendering (no empty flash)
- Toast close button positioned at top-right with proper spacing
- Sampling section expanded by default in chat settings
- Global toast close button styling fix
Change all repetition_penalty defaults from 1.1 (or 1.05/1.2 in
presets) to 1.0 across the entire backend and frontend. Most models
handle repetition well on their own and a non-1.0 penalty can degrade
output quality, especially for code, structured output, and creative
tasks.
Files changed:
- Backend: inference.py, llama_cpp.py, orchestrator.py, worker.py,
models/inference.py (Field defaults)
- Frontend: chat-settings-sheet.tsx (Creative/Precise presets),
runtime-provider.tsx (auto-title generation)
- maxTokens: 2048 -> 8192. The old 2048 limit caused generation to
stop mid-output for longer responses (e.g. reasoning/thinking models
that produce long chain-of-thought before the answer).
- repetitionPenalty: 1.1 -> 1.0 (disabled). Most models handle
repetition well on their own. A penalty of 1.1 can hurt quality
for creative tasks like code generation and ASCII art.
- Change welcome message from "Run LLMs or test your fine-tune" to
"Chat with your model".
Merge the toast UX refactor from PR #4304 (by @Shine1i):
- Toast duration 5s default with close button (X) for manual dismiss
- Inline progress bar component (ModelLoadInlineStatus) shown in the
header after toast is dismissed
- Model switch warning only for image compatibility (not generic)
- activeThreadId tracked in store via ActiveThreadSync
- Loading state cleanup via resetLoadingUi helper
- Toast uses Infinity duration during loading with onDismiss handler
Re-applied non-GGUF download progress additions on top:
- getDownloadProgress for all models (not just GGUF)
- hasShownProgress flag, loadingModelRef race condition checks
- First poll at 500ms, bytes-only fallback when expected size unknown
Don't show "Model changed for this chat" toast when the thread has
no messages. On a fresh page load with a stale thread from a previous
session, this warning is confusing. The warning is only useful
mid-conversation to alert about image compatibility with the new model.
When messages.length === 0, silently update the thread's modelId and
proceed with loading.
The _VISION_CHECK_SCRIPT subprocess used logger.info() but logger was
never defined in the subprocess context. This caused a NameError on
every vision check, making all transformers 5.x models (Qwen3.5,
GLM, etc.) fall back to text-only mode even when they support vision.
Replace logger.info() with print() since the parent process reads
the subprocess stdout via result.stdout.
- Add sloth emoji prefix to "Downloaded" and "Recommended" section
labels in the Hub model picker so they are visually distinct.
- Replace browser network errors ("NetworkError when attempting to
fetch resource" / "Failed to fetch") with a clearer message:
"Studio isn't running -- please relaunch it."
llama-server uses stb_image internally which does not support WebP,
TIFF, AVIF, and other formats that browsers accept for upload.
Uploading a WebP image to a vision GGUF model caused a 400 error:
"Failed to load image or audio file" / "failed to decode image bytes".
Convert all uploaded images to PNG via PIL before base64-encoding and
forwarding to llama-server. This handles WebP, TIFF, BMP, GIF, AVIF,
and any other format PIL supports. RGBA images are converted to RGB
first since PNG with alpha can cause issues in some vision pipelines.
GGUF repos with mmproj files (e.g. Qwen3.5-0.8B-GGUF) are already
detected as vision-capable by list_gguf_variants(), and is_vision is
set correctly in ModelConfig. However, the HF download path only
downloaded the main GGUF file without the mmproj projection file,
so llama-server started without --mmproj and rejected image uploads
with "text-only model" errors.
Add _download_mmproj() to LlamaCppBackend that:
- Lists repo files for mmproj*.gguf matches
- Prefers mmproj-F16.gguf (best quality), falls back to any mmproj
- Downloads via hf_hub_download (uses the same HF cache)
In load_model(), when is_vision=True and no explicit mmproj_path was
provided (HF mode), auto-download the mmproj after the main GGUF.
The downloaded path is passed to llama-server via --mmproj.
1. Backend: When a model fails with "No config file found" or similar
unsupported-model errors, wrap the message with "This model is not
supported yet. Try a different model." instead of showing the raw
Unsloth exception.
2. Frontend: Compute estimated download size from the HF search API's
safetensors.parameters dtype breakdown (BF16=2B/param, I32=4B/param,
F32=4B/param, etc.) and show it in the model picker instead of just
the param count. For example, Kimi-K2.5 now shows "~554 GB" instead
of "171B" (which was misleading since 171B params != 171GB download).
Three fixes on top of the download progress feature:
1. Backend: Replace broken "no .incomplete = done" completion check
with a 95% byte threshold. HF downloads files sequentially, so
between files there are briefly no .incomplete files even though
the download is far from done (e.g. Kimi-K2.5 reported "done"
after downloading 22KB of config files out of 595GB).
2. Frontend: Track hasShownProgress flag. Only show "Download
complete. Loading into memory..." if we actually displayed
download progress before. For already-cached models where the
first poll returns progress=1.0, this avoids the misleading
"Download complete" message.
3. Frontend: Deduplicate recommended vs downloaded -- filter out
models already in the "Downloaded" section. Cache the fetched
lists at module level so re-mounting the popover does not flash
an empty "Downloaded" section.
Previously only GGUF models showed download progress in Chat. Non-GGUF
models (safetensors, bnb quantized, etc.) showed a static message with
no progress indication. This adds progress tracking for all model types
and fixes several related issues.
Backend:
- Add /api/models/download-progress endpoint that checks the HF cache
blobs directory for completed and .incomplete files. Uses model_info()
(cached per repo) to determine expected total size for percentage.
- Add /api/models/cached-models endpoint that lists non-GGUF model repos
from the HF cache via scan_cache_dir().
- Fix progress stuck at 0.99: when no .incomplete files remain, report
1.0 immediately (blob deduplication can make byte totals mismatch).
Frontend:
- Remove the ggufVariant gate so download progress polling works for all
non-cached models, not just GGUFs.
- Use GGUF-specific endpoint when variant + expectedBytes available,
otherwise use the general download-progress endpoint.
- Fix toast stuck after load: check loadingModelRef.current before and
after the async poll to prevent overwriting the success toast.
- First poll at 500ms instead of waiting for the 2s interval.
- Show downloaded non-GGUF models in the Hub model picker "Downloaded"
section alongside GGUFs.
* fix: Ctrl+C not breaking out of backend on Linux
threading.Event.wait() without a timeout blocks at the C level on
Linux, preventing Python from delivering SIGINT. Use a 1-second
timeout loop so the interpreter can process pending signals.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* user can upload eval dataset, removed bugs
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* resolving merge conflicts
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* resolving gpt comments
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Roland Tannous <115670425+rolandtannous@users.noreply.github.com>
The `set -u` (nounset) flag in setup.sh causes `${_HIDDEN_GITIGNORES[@]}`
to fail with "unbound variable" when no parent .gitignore with `*` is
found (common on Mac where the install is not inside a Python venv).
Use the `${arr[@]+"${arr[@]}"}` idiom to safely expand empty arrays
under nounset mode.
Two issues caused the studio frontend to render without any styling
when installed via `pip install` (non-editable):
1. `pyproject.toml` package-data only included `frontend/dist/**/*`.
The `include-package-data = true` setting relies on `git ls-files`,
which fails in isolated builds (pip/uv copy source to a temp dir
without `.git`). This meant `frontend/src/`, `package.json`,
`vite.config.ts`, and other build files were missing from the
installed package. Tailwind had no source files to scan.
2. Python venvs auto-create a `.gitignore` with a bare `*` pattern.
Tailwind v4's oxide scanner walks parent directories and respects
`.gitignore` -- so even when source files are present, the venv's
`*` pattern causes the scanner to skip all `.tsx` files. The result
is a 34KB CSS skeleton with zero utility classes instead of the
expected 265KB.
Additionally, Vite adds `crossorigin` to script/link tags by default.
This forces CORS mode on font subresource loads, which Firefox
HTTPS-Only Mode does not exempt -- causing all @font-face downloads
to fail silently when Studio is served over HTTP.
Changes:
- pyproject.toml: Expand package-data to include frontend source,
config files, setup scripts, and backend requirements using glob
patterns (no node_modules)
- studio/setup.sh: Temporarily hide parent .gitignore files containing
a bare `*` during `npm run build`, with trap-based restoration
- studio/backend/main.py: Strip `crossorigin` attributes from HTML
at serve time so fonts load correctly on any protocol
* feat(studio): switch to password-only login and simplify first-time setup
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: align change-password button state with validation rules
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Roland Tannous <115670425+rolandtannous@users.noreply.github.com>
* fix: graceful shutdown on Windows (signal handlers for Ctrl+C)
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
* fix: update Colab notebook to use public unsloth repo and correct paths
* Update studio/Unsloth_Studio_Colab.ipynb
For efficiency, especially in environments like Colab, it's better to perform a shallow clone of the repository. This fetches only the latest commit from the specified branch, which is significantly faster and uses less disk space than cloning the entire project history.
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
* Update Unsloth_Studio_Colab.ipynb
* studio: add standard Unsloth header, news, section headings, and footer to Colab notebook
* studio: refine Colab notebook section headings and cell cleanup
---------
Co-authored-by: LeoBorcherding <LeoBorcherding@users.noreply.github.com>
* chat only with gguf for mac devices
* resolving gpt comments
* add change-password for chat only
* hide lora adaptors dropdown
* solving gpt comments
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* addressing the comment
* fixing auth flow
---------
Co-authored-by: Datta Nimmaturi <venkatadattasainimmaturi@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
- Run GGUF load_model in asyncio.to_thread so the event loop stays free
for progress polling during download (was blocking all requests).
- Extract download phase out of the lock in LlamaCppBackend.load_model
so unload_model/cancel can take effect immediately during download.
- Fix "downloaded" badge for split GGUFs: check total cached bytes
across all shards vs expected size, not just first shard existence.
- Respect CUDA_VISIBLE_DEVICES in /api/system GPU reporting so the
frontend GGUF fit estimation uses actual available VRAM.
- Sort tight variants (need CPU offload) smallest-first instead of
largest-first -- closer to GPU budget = faster inference.
- Fix cancel: use refs instead of React state for abort controller and
toast ID so both cancel buttons (text + toast) work reliably. Make
cancel synchronous (fire-and-forget unload) for instant UI response.
Check abortCtrl.signal.aborted after loadModel returns to prevent
ghost model state. Skip rollback and suppress errors on cancel.
- Dynamic top 4 GGUF models fetched from HF API sorted by downloads,
prepended to the default recommended list.
- Remove turnAnchor="top" for auto-scroll to bottom during generation.
- Set default toast duration to 10s (was infinite for loading toasts).
- Deduplicate cached GGUF repos using scan_cache_dir API (fixes
Qwen/X-GGUF vs qwen/x-gguf duplicates from lowercased HF cache).
- Pre-compile repo_id validation regex to silence CodeQL ReDoS warning.
- Change welcome text and default suggestion text.
_get_gpu_free_memory was filtering by CUDA_VISIBLE_DEVICES, so with
CUDA_VISIBLE_DEVICES='0' set by the training env, llama-server only
saw 1 GPU and used --fit for CPU offloading instead of spreading
across all 8 GPUs.
Since llama-server manages its own GPU allocation (the _select_gpus
method picks GPUs and sets CUDA_VISIBLE_DEVICES for the subprocess),
the query must see ALL physical GPUs to make the right decision.
1. Progress endpoint now takes a variant parameter and only counts
.gguf files matching that variant (not all files in the repo cache,
which would include previously downloaded variants)
2. Tracks .incomplete files in HF blobs dir for in-progress single-shard
downloads, capping at 99% until the file is fully committed
3. Fixed loading text: "Loading model..." for cached, "Downloading
model..." for new downloads, with appropriate descriptions
4. Wording: "Downloading and loading model. Large models can take a
while." instead of "This may include downloading."
