Adds a configurable Hugging Face model download cache location to Unsloth Studio, selectable from Settings, with per-cache download manifests, scoped deletion, and read-only inventory of previously selected caches.
* Replace standalone Studio wording with Unsloth
Replace the single word Studio with Unsloth wherever it is used as
shorthand for Unsloth Studio in docs, CLI output, UI strings, i18n
locales, workflow display names, comments and docstrings.
Kept unchanged: the full name Unsloth Studio, third party product
names (LM Studio, Visual Studio, Mac Studio), feature names
(Recipe Studio, Fine-tuning Studio and its translations), and all
identifiers such as env vars, commands, paths and filenames.
* Address review feedback on the Studio wording rename
Use "an" before Unsloth where the rename left the article as "a".
Restore the split brand where Unsloth and Studio render as two halves
of the full product name: the onboarding sidebar subtitle and the
IPv6 localhost warning. Scope two messages to the full name Unsloth
Studio where plain Unsloth was misleading: the AMD README bullet and
the CLI studio setup error.
* fix: handle case-variant GGUF cache hits for unsloth start
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* gguf cache: keep split shards co-located and isolate cache tests properly
When a cached main shard was reused from an older snapshot, the extra shards
were resolved independently and could come from a different snapshot dir (or a
fresh download into the current ref), leaving llama.cpp unable to load a
multi-shard GGUF whose pieces are split across directories. Only reuse a cached
main shard when every sibling shard sits in the same snapshot; otherwise fetch
the whole set together so they stay co-located.
Also patch huggingface_hub.constants.HF_HUB_CACHE (not just the HF_HUB_CACHE env
var) in the two cache tests that seeded a temp cache: the snapshot lookup reads
the module constant, so the env-only override let the real cache leak in and
skip an asserted download.
* Do not let a companion-only cache snapshot shadow real GGUF variants
When listing GGUF variants from the local HF cache, a newer snapshot may
contain only a companion file (for example a vision projector fetched on
demand) while the actual quant files live in an older snapshot. The prior
scan returned the first snapshot whose vision flag was set, yielding an
empty variant list and hiding the real quants. Keep scanning older
snapshots for actual variants and carry the vision flag across snapshots.
Also record the disk-space fallback variant's size in expected_sizes so
the later cache-reuse probe can size-verify the fallback main shard
instead of only checking for its existence.
* Propagate cached repo casing to companions and preflight split co-location
Two fixes to the case-variant GGUF cache reuse:
- Resolve the requested repo id to its cached canonical casing once in
load_model, up front, and pass it to the main GGUF and its companions
(mmproj / MTP drafter). Previously only _download_gguf resolved the
casing internally, so a case-variant request loaded the main file from
the canonical cache dir while the companions kept the requested casing
and missed the cached vision projector / drafter offline. Extracted the
resolution into a shared _resolve_repo_id_casing helper.
- Apply the split-shard co-location check in the disk-space preflight. When
a split GGUF's shards are cached across different snapshots the whole set
is refetched later, so counting them as cached made the preflight read 0
bytes to download, skip the smaller-variant fallback, and then fail the
full download on a low-disk machine.
* Reuse a co-located split GGUF snapshot and fix split fallback size probe
- When reusing a cached split GGUF, scan snapshots for one that holds the
whole set co-located instead of taking the newest snapshot's first shard.
A newer snapshot with only the first shard no longer shadows an older
complete snapshot, so an already-cached split model is reused rather than
refetched (which would fail offline).
- The disk-space fallback records its size in expected_sizes only for a
single-file fallback. _find_smallest_fitting_variant returns the whole
variant size, so using it as the first shard's expected size rejected a
valid cached first shard of a split fallback and forced a re-download.
* Scan for a complete split snapshot in the preflight; require a loaded catalog hit
- The disk-space preflight now uses the same co-located snapshot scan as the
download path (_cached_colocated_split_main) instead of the newest-snapshot
probe, so a newer snapshot holding only the first shard no longer masks an
older complete one and trips the smaller-variant fallback for a fully cached
split model.
- _resolve_model only attaches to a /v1/models entry that is actually loaded
(loaded != False). /v1/models also lists cached-but-unloaded catalog entries,
and matching one by case skipped /api/inference/load and left the agent
pointed at a model that is not resident.
* Restrict cross-snapshot GGUF cache reuse to offline
Reusing a same-name blob from an older or case-variant snapshot bypasses the
Hub revision/etag check, so a repo that updates a GGUF in place could serve
stale weights online. Gate the cross-snapshot and case-variant reuse (both the
disk-space preflight accounting and the download path) on HF_HUB_OFFLINE.
Online, hf_hub_download fetches the current revision and resumes a partial
download, so the reuse is unnecessary there; offline it remains the resilience
fallback. Marked the two reuse regression tests as the offline scenarios they
represent and added an online test asserting a fresh fetch.
* Harden offline cache reuse and hub-id detection
Three follow-ups on the case-variant GGUF cache path:
- Honor every truthy HF_HUB_OFFLINE spelling (1/true/yes/on), not just "1", when
gating the cross-snapshot and case-variant cache reuse. With HF_HUB_OFFLINE=true
the Hub calls are already offline, so the reuse must trigger or the cached GGUF
fails to load; route both the preflight accounting and the download path through
the same offline parse the rest of the backend uses.
- Resolve mmproj/MTP companions from the actual cached snapshot when offline.
resolve_cached_repo_id_case can keep a partial lower-case spelling when any dir
exists under the requested casing, so an hf_hub_download on that casing misses the
canonical companion; scan every case-variant snapshot and return the cached path.
- Restrict the case-insensitive model-id match to syntactically valid hub ids
(a single namespace/name over the HF charset). A server-side relative path such
as models/Llama/Foo.gguf is no longer treated as a hub id, so it cannot
casefold-match a differently cased path on a case-sensitive filesystem. This is
host independent, unlike the local-existence probe which cannot see a server path.
* Only casefold-match model ids against a loopback Studio
A two-segment string like Models/Foo is indistinguishable from a hub id, and the
local Path.exists() probe in _is_hub_model_id cannot see a path that exists only
on a remote Studio host. So against a remote server, casefolding could attach to
a distinct server-side path (Models/Foo vs models/foo) on a case-sensitive
filesystem. Gate the case-insensitive match on is_loopback_url(base): only a
local Studio, where the existence probe is authoritative, casefolds. For a remote
Studio the match is exact and a case-mismatched request falls through to
/api/inference/load, whose already-loaded dedup resolves it correctly.
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: Wasim Yousef Said <wasimysdev@gmail.com>
* fix(studio/llama_cpp): disable trust_env on the loopback health probe
_wait_for_health() polls http://127.0.0.1:<port>/health with the default
httpx trust_env=True, so an ambient HTTP(S)_PROXY in the environment is
applied to the loopback request. A proxy that returns 503 for 127.0.0.1
makes every probe fail, so the loop runs until timeout and Studio load
hangs (trust_env=False returns 200 immediately).
Pass trust_env=False so the local readiness probe never goes through a
proxy. This mirrors the existing trust_env=False handling in the sibling
llama_http / external_provider HTTP clients.
* test(offline_gguf_cache): accept trust_env kwarg in fake_get mock
_wait_for_health now calls httpx.get(..., trust_env=False); update the retry test's fake_get to accept the kwarg so it doesn't raise TypeError.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio/llama_cpp): bypass proxies for loopback clients
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix(studio/routes): bypass proxies for llama streams
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: wasimysaid <wasimysdev@gmail.com>
* Add Studio API activity monitor
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Mark disconnected monitor streams cancelled
* Fix monitor lifecycle, parsing, and clock issues for PR #5558
Backend:
- Finalize the monitor entry as error on the chat-completions validation
reject paths (addresses the codex P2 comment around line 2337). Adds a
small _reject helper next to the api_monitor.start so each early-raise
in the GGUF tool-passthrough, non-empty-messages, GGUF vision/audio,
PIL image decode, and text-only model branches no longer leaves the
entry stuck running until eviction.
- _monitor_openai_chunk is now defensive about malformed shapes
(non-list choices, non-dict choice/delta) so a misbehaving provider
chunk does not raise into the streaming generator and abort the
user's response.
- _monitor_openai_sse_line accepts both data:value and data: value per
SSE spec; previously the single-space-only prefix silently dropped
tokens from compact emitters.
- openai_completions stream now keeps a residual buffer between
aiter_bytes chunks so a data: line whose newline lands in the next
TCP frame is reconstructed before parsing, instead of being dropped
by the per-chunk splitlines call.
api_monitor:
- duration_ms is derived from time.monotonic anchors and clamped at 0,
so NTP / manual clock steps no longer produce negative durations.
- finish() and fail() are idempotent: a second call (for example [DONE]
arriving after the generator's finally block already ran) no longer
moves finished_at or finished_monotonic.
- set_usage only derives total_tokens when no authoritative total has
been recorded, so a later partial-usage chunk does not clobber a
provider-reported total from an earlier chunk.
- _trim guards limit < 3 so the slice cannot underflow.
Tests:
- Adds regression coverage for the new idempotency, total preservation,
monotonic clock, and _trim guard behaviour.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Mark monitored stream failures correctly
* Fix completions monitor failures
* Fix responses stream monitor cleanup
* Fix audio input API monitor lifecycle
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix API monitor review follow-ups
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix completions monitor usage accounting
* Fix API monitor cancellation and usage gaps
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Move API monitor into settings
* Fix API monitor review followups
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Lazy load API monitor details
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix API monitor review issues
* Finalize passthrough monitor on clean EOF
* Finalize monitor on chat validation rejects
* Fix API monitor review follow-ups
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix API monitor follow-up review issues
* Monitor embeddings and tool-call replies
* Cover tool-call monitor replies and cancellations
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Scope API monitor entries by subject
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Finalize monitor entries on cancellation
* fix: address PR 5558 CI failures
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix: update provider proxy test stub
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: imagineer99 <samleejackson0@gmail.com>
Co-authored-by: Lee Jackson <130007945+Imagineer99@users.noreply.github.com>
* Studio: add shared Xet-primary download helper with HTTP stall fallback
Xet is the fast default transport in huggingface_hub, but a stalled Xet
transfer hangs with no progress and no exception, and a blocked native thread
cannot be killed. The safetensors inference path already recovers (subprocess
watchdog + respawn with HF_HUB_DISABLE_XET=1); the GGUF and training paths do
not. Add a reusable helper that the in-process paths can adopt.
utils/hf_xet_fallback.py:
- DownloadStallError (moved here from core/inference/orchestrator.py, which now
imports it; behavior unchanged, still a RuntimeError subclass).
- get_hf_download_state / start_watchdog: a no-progress watchdog built on the
sparse-aware hub.utils.hf_cache_state helpers; fires only while a .incomplete
is present and the on-disk byte total is unchanged for stall_timeout.
- hf_hub_download_with_xet_fallback: cached files short-circuit; otherwise the
download runs in a spawn child (own process group) supervised by the watchdog.
On a stall it kills the child, makes the partial safe for HTTP via
prepare_cache_for_transport, and respawns once with HF_HUB_DISABLE_XET=1. Cancel
and deterministic errors (auth/missing/disk) propagate without a fallback.
Tests cover the watchdog state machine, the transport decision logic, and a
regression lock that HF_HUB_DISABLE_XET is honored in a fresh interpreter.
* Studio: route GGUF Chat-Mode downloads through the Xet->HTTP fallback
The GGUF load path (_download_gguf main+shards, _download_companion_gguf for
mmproj/MTP) called a bare blocking hf_hub_download with no recovery, so a Xet
stall hung the Chat-Mode load with no fallback. Route those three calls through
hf_hub_download_with_xet_fallback: Xet stays primary, HTTP is used only if Xet
stalls, per-file so finished shards stay cached. The existing _cancel_event is
threaded through, the Cancelled sentinel is preserved, and companions stay
best-effort (a terminal stall is swallowed to None). Cached files short-circuit
in the helper with no subprocess, so the fast path is unchanged.
The two offline mmproj tests are repointed from huggingface_hub.hf_hub_download
to the new call boundary (the helper) since the download now goes through it.
* Studio: recover a stalled training model-load via Xet->HTTP respawn
Training runs in a spawn subprocess and FastModel.from_pretrained downloads
internally, so the download cannot be wrapped per-file like GGUF. Instead the
worker now watches the HF cache during the model-load phase (emitting
model_load_started / model_load_completed and a stall event), and the parent
recovers a stall by terminating the worker and respawning it once with
HF_HUB_DISABLE_XET=1.
worker.py: set HF_HUB_DISABLE_XET=1 before any HF import when the parent passes
disable_xet (respawn), and wrap trainer.load_model with start_watchdog.
training.py: plumb disable_xet through the config; track the model-load window;
on a first-load stall arm a one-shot respawn (handled on the exiting pump thread,
so no pump self-join) that preserves the DB run row (history is not duplicated)
and re-runs the load over HTTP. A second stall, or a stall outside model-load,
surfaces as a normal error. W&B init happens after model-load, so a pre-load
respawn cannot duplicate it; the dataset is re-formatted in the new worker.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: add gated test-only fault-injection hook for the Xet stall path
UNSLOTH_HF_XET_FORCE_STALL=1 makes the Xet download attempt write a partial
blob and hang, so the no-progress watchdog and the HTTP fallback can be
exercised end to end against a real repo (never set in production). Used to
verify recovery on real models: a forced Xet stall on a 5.37GB Qwen3.5-35B-A3B
shard triggered the watchdog and the HTTP retry downloaded the correct file
(sha256 verified).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Studio: tighten Xet-fallback comments and consolidate its tests
Trim docstrings and inline comments across the Xet->HTTP fallback code to
the non-obvious why (spawn-not-thread, killpg-not-getpgid, the sparse-partial
HTTP-resume hazard); drop comments that merely restate the code. Verified
comment-only with an AST signature check.
Merge the three helper-level test files (watchdog, transport policy, and the
HF_HUB_DISABLE_XET regression lock) into tests/test_hf_xet_fallback.py, and
prefer the real structlog over a bare stub so test collection order cannot
leak an incomplete module to others that log at import.
Full backend suite: 3455 passed, 14 pre-existing flash-attn failures only.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Trim and tighten code comments and docstrings across the repository. Comment-only: every changed file verified code-identical to main via AST/token comparison.
Trim and tighten code comments and docstrings across studio/ Python. Comment-only: every changed file verified code-identical to main via AST/token comparison.
Raise ruff line-length to 100 and extend the local pre-commit format pipeline (def-signature magic-comma normalization, short multi-line assert collapse, kwarg '=' spacing, blank-line-after-short-import removal, adjacent string-literal / f-string+plain merge, redundant-pass pruning). Every transform re-checks the file AST and is dropped if it would differ; the whole-repo reformat is verified AST-identical per file and idempotent.
* studio: load cached GGUF models when fully offline
When huggingface.co is unreachable, GGUF model loads fail in three distinct
places even though the bits are already in ~/.cache/huggingface/hub. Each
failure has a different surface symptom:
1. list_gguf_variants() raises straight through HTTPException(500), so the
variant dropdown shows 'Failed to list GGUF variants'.
2. detect_gguf_model_remote() silently returns None after retries fail. The
caller then treats a GGUF-only repo as non-GGUF and routes it through the
transformers/MLX path. On Apple Silicon this surfaces as 'Unsloth currently
only works on NVIDIA, AMD and Intel GPUs.'
3. _download_gguf() loses list_repo_files() to the network and falls back to a
filename heuristic ('{repo}-{variant}.gguf'). When the repo name does not
echo the filenames (e.g. repo 'Qwen3.6-27B-MTP-GGUF' contains a file
'Qwen3.6-27B-UD-Q4_K_XL.gguf' with no MTP), hf_hub_download cannot find
that invented filename in the cache and aborts.
Fix in three layers:
- list_gguf_variants / detect_gguf_model_remote: honor HF_HUB_OFFLINE and
fall back to scanning the local HF cache snapshot when the API throws.
detect_gguf_model_remote still keeps its retry loop for transient flakes;
the cache fallback only kicks in after every attempt fails.
- _download_gguf: when list_repo_files() fails, look up variant -> real
filename inside the cached snapshot before resorting to the heuristic.
- llama_cpp.load_model / inference worker startup: when DNS for
huggingface.co fails (2s probe), set HF_HUB_OFFLINE=1 for the process so
every hf_hub_download call below resolves from cache instantly instead of
spending ~25s on five exponential retries.
Online behavior is unchanged: the API is tried first and only used to fail
over. The cache scan is a strict subset of what list_local_gguf_variants
already does today for local paths.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: tighten inline comments on offline GGUF fallback
* studio: address review feedback on offline GGUF fallback
Fixes from the review pass on #5505:
* ruff F823 (lint CI red): the late `import os` at the bottom of
LlamaCppBackend.load_model made `os` a function-local name, so my
new `os.environ` reference at the top of the same method was a
use-before-bind. Surfaces at runtime as
'cannot access local variable os where it is not associated with a value'
and is why the Mac/Windows Studio API jobs were failing too. The
env-var mutation has been moved into a module-level contextmanager,
so load_model no longer touches `os` directly.
* Codex P1: cache variant match now uses the relative path, not the
basename. Layouts like `BF16/foo.gguf` (variant token only in
parent dir) were silently skipped, falling through to the bogus
`{repo}-{variant}.gguf` heuristic and failing offline loads of
models stored under quant-named subdirs.
* Codex P1: HF_HUB_OFFLINE no longer persists past one model load.
llama_cpp.load_model now uses a contextmanager that probes DNS,
sets HF_HUB_OFFLINE/TRANSFORMERS_OFFLINE only when DNS is dead,
and pops them in finally (preserving any prior user setting of
TRANSFORMERS_OFFLINE). Pre-existing user-set HF_HUB_OFFLINE is
respected as a no-op. worker.py keeps the startup probe because the
orchestrator spawns a fresh worker per load -- comment updated to
make that lifecycle explicit, and a warning is now logged.
* Gemini: cache-dir lookup centralized in `_iter_hf_cache_snapshots`.
Three near-identical copies (in list/detect helpers and the
llama_cpp offline scan) now go through one helper.
* Gemini: `huggingface_hub.utils.is_offline_mode` does not exist in
1.x (verified locally); `huggingface_hub.constants.HF_HUB_OFFLINE`
is snapshot-at-import-time and does not reflect runtime mutations.
Manual env-var parsing kept.
* socket probe now saves and restores the prior default timeout
instead of unconditionally setting None on exit, so it composes
with caller code that already configured a timeout.
* worker.py probe now logs a warning when offline mode is auto-enabled
so debugging the case isn't blind.
* studio: regression tests for offline GGUF cache fallback
Lock in the offline fallback path from #5505 so future refactors can't
silently regress either bug. 26 tests, 0.55 s, no network/GPU/subprocess.
Covers:
* _iter_hf_cache_snapshots: missing cache, missing repo, missing
snapshots/, newest-mtime ordering, case-insensitive repo match.
* _list_gguf_variants_from_hf_cache and the list_gguf_variants
online/offline-env/API-exception/reraise paths.
* _detect_gguf_from_hf_cache and detect_gguf_model_remote 3x-fail
fallback. Pre-existing RepositoryNotFoundError early-return preserved.
* Codex P1 #1 regression: BF16/foo.gguf (quant only in subdir name)
must resolve via _detect_gguf_from_hf_cache, which now matches the
snapshot-relative path rather than the basename.
* _probe_dns_dead: returns True/False, restores prior socket timeout.
* Codex P1 #2 regression: _hf_offline_if_dns_dead sets env only inside
the block, restores on exit (including on exception), re-probes DNS
on the next call so a transient hiccup cannot lock the long-lived
LlamaCppBackend singleton offline. Honors a user-set HF_HUB_OFFLINE
as a no-op. Preserves a user-set TRANSFORMERS_OFFLINE across exit.
Follows the existing studio backend test stub pattern (loggers /
structlog / httpx stubs + backend dir on sys.path).
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* studio: extend offline cache fallback to _download_mmproj and quant label
Two follow-up fixes from the review pass on #5505:
* _download_mmproj() now mirrors _download_gguf()'s offline path:
when list_repo_files() fails, scan the local HF cache snapshot for
any GGUF whose basename starts with mmproj-. Without this, offline
vision GGUF loads succeed at the main weight (the existing PR fix)
but the mmproj returns None and llama-server starts without vision
support. Same _iter_hf_cache_snapshots helper, F16 preference and
fallback to the first match are preserved.
* _extract_quant_label() now considers parent directory segments when
the basename has no quant token. Layouts like BF16/foo.gguf are
already documented in this file and are returned by the new
snapshot-relative-path filter in _download_gguf; before this fix
their variant label collapsed to "foo" (the last hyphen segment of
the basename). Regex is the same; the search just walks parent
segments innermost-first if the basename misses.
Tests (studio/backend/tests/test_offline_gguf_cache_fallback.py):
* TestExtractQuantLabelSubdir: basename quant unchanged, quant-only-
in-parent, UD- prefix in parent, deeper nesting picks the
innermost matching segment.
* TestDownloadMmprojOfflineCacheFallback: cache fallback returns the
mmproj when list_repo_files fails, F16 preference holds when both
variants are in cache, no-mmproj cache returns None.
* httpx stub now prefers the real package when installed (the CI
install list already includes it) and falls back to the stub only
when httpx is genuinely missing. Newer huggingface_hub imports
HTTPError/Response/Request at module load, so the previous
fixed-set stub broke when those names were added upstream.
26 existing cases plus 7 new = 33 pass in 0.74s.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fix/adjust offline cache + DNS probe per PR #5505 review
Four review findings tightened, with regression tests:
- list_local_gguf_variants subdir collapse (P1 codex 10:08): pass the
snapshot-relative path to _extract_quant_label so BF16/foo.gguf and
Q4_K_M/foo.gguf produce distinct labels instead of folding to the same
basename pseudo-quant.
- list_gguf_variants cache fallback (P2 codex 12:10): surface
RepositoryNotFoundError / GatedRepoError / RevisionNotFoundError /
EntryNotFoundError to the caller instead of masking with stale cache,
matching detect_gguf_model_remote.
- _detect_gguf_from_hf_cache mmproj (P2 codex 12:10): exclude mmproj
files from the candidate list so a partial cache with only a vision
projector cannot route the projector as the main model.
- _probe_dns_dead global timeout (P2 codex 13:06): run the gethostbyname
on a daemon thread with join timeout so concurrent sockets in the same
interpreter never inherit a process-wide socket.setdefaulttimeout
mutation. Same shape applied in worker.py's startup probe.
* Make llama-server health check tolerant of warmup races
Two layered fixes for the Windows GGUF smoke CI Tool calling Tests
flake that exit-22'd on a single httpx.ReadError during llama-server
warmup. The 'windows-latest -> windows-2025-vs2026' image rollout is
hitting main with the identical symptom.
A. _wait_for_health: catch httpx.ReadError, RemoteProtocolError,
WriteError alongside ConnectError and TimeoutException. A TCP RST
mid-read while llama-server is still binding the port (WinError
10054) is a 'still warming up' signal, not fatal. The existing
_process.poll() check still wins for real crashes.
B. _drain_stdout + spawn: tee llama-server stdout/stderr to a
per-launch log file at ~/.unsloth/studio/logs/llama-server/
<port>.log. Any future subprocess crash leaves a forensic trace
on disk even when Studio's traceback only captures the symptom
(ReadError) and not the cause. Best-effort: a logging-side OSError
never blocks the load.
Regression coverage: TestWaitForHealthRetriesOnReadError pins the
retry behaviour for the three new exception types and verifies that a
real process exit still short-circuits the loop.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* ci(windows): retry inference/load + collect llama-server logs
Composite fix for the Tool calling Tests flake that exit-22'd on a
single httpx.ReadError during llama-server warm-up. The
windows-latest -> windows-2025-vs2026 runner image rollout has been
hitting main with the identical symptom.
- All three jobs (openai-anthropic, tool-calling, json-images) now
retry POST /api/inference/load up to 3 times with 10s backoff and
preserve the response body for post-mortem. One transient 500 no
longer fails the whole job.
- A new "Collect llama-server logs" step copies the per-launch
llama-server stdout teed by Studio under ~/.unsloth/studio/logs/
llama-server/ into the workspace, and the upload-artifact step
now includes logs/llama-server/*.log so any future subprocess
crash leaves a forensic trace.
---------
Co-authored-by: shimmyshimmer <shimmyshimmer@users.noreply.github.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
oobabooga