unsloth/studio/frontend/src/features/chat/api
Daniel Han f7e8a85d32
studio/frontend: cap auto-load cascade attempts (#5578)
* studio/frontend: cap auto-load cascade attempts

autoLoadSmallestModel walks every cached GGUF and safetensors repo with a
try/catch + continue, so a folder of broken caches (missing files, stale
llama.cpp prebuilt, GPU OOM) can fire dozens of failing POST /api/inference/load
calls in a row. Each call costs ~5 seconds (HF metadata probe + DNS guard
inside inference.py), so the user sees a runaway sequence of request_completed
log lines after sending one message that needed an auto-load.

Cap the total loadModel calls inside autoLoadSmallestModel at 3 (GGUF cascade
plus safetensors fallback share the same counter). Caching that fails three
times in a row is almost certainly an environment problem, not "we haven't
found the working one yet"; the default-Gemma download path still runs.

No behavior change on the happy path: success returns after the first hit
exactly like today, and the trust-remote-code skip path does not consume an
attempt slot.

* shorter comment on auto-load cap

* studio chat: extend autoload cap to default Gemma fallback

Cached cascade respected MAX_AUTO_LOAD_ATTEMPTS but the default-Gemma
download path skipped the budget, so a broken cache could still emit a
fourth /api/inference/load. Gate the fallback on the same cap (and bump
loadAttempts when we do call loadModel) so the total cross-path budget
is 3, matching the cap's intent.
2026-05-19 05:48:59 -07:00
..
chat-adapter.ts studio/frontend: cap auto-load cascade attempts (#5578) 2026-05-19 05:48:59 -07:00
chat-api.ts studio/frontend: wire logout, singleflight refresh, shared 422 helper, current-password input (#5490) 2026-05-18 00:03:03 -07:00
openai-containers.ts studio/frontend: wire logout, singleflight refresh, shared 422 helper, current-password input (#5490) 2026-05-18 00:03:03 -07:00
providers-api.ts studio/frontend: wire logout, singleflight refresh, shared 422 helper, current-password input (#5490) 2026-05-18 00:03:03 -07:00