8fb2fb9e2a
Auto-downloading a 1.1 GB cross-encoder at every studio start is
wrong for users who never use rerank — reranker is opt-in by design.
Move the precache from a startup daemon thread to an explicit
POST /api/rag/reranker/precache endpoint, and have the chat settings
sheet call it the moment the 'Use reranker' switch is flipped on.
- Backend: drop the startup _precache_reranker thread; add the
/api/rag/reranker/precache route that calls precache_reranker().
- Frontend: new precacheRagReranker() in rag-api, wired into the
Switch's onCheckedChange so the download runs synchronously
with a loading toast. On success: 'Reranker ready'. On failure:
error toast + auto-flip the switch back off so the next query
doesn't trigger another long hang.
First toggle-on pays the 1.1 GB download once; subsequent toggles
hit the HF cache and return ~instantly.
|
||
|---|---|---|
| .. | ||
| backend | ||
| frontend | ||
| src-tauri | ||
| __init__.py | ||
| install_llama_prebuilt.py | ||
| install_python_stack.py | ||
| LICENSE.AGPL-3.0 | ||
| package-lock.json | ||
| package.json | ||
| setup.bat | ||
| setup.ps1 | ||
| setup.sh | ||
| Unsloth_Studio_Colab.ipynb | ||