8fb2fb9e2a
Auto-downloading a 1.1 GB cross-encoder at every studio start is
wrong for users who never use rerank — reranker is opt-in by design.
Move the precache from a startup daemon thread to an explicit
POST /api/rag/reranker/precache endpoint, and have the chat settings
sheet call it the moment the 'Use reranker' switch is flipped on.
- Backend: drop the startup _precache_reranker thread; add the
/api/rag/reranker/precache route that calls precache_reranker().
- Frontend: new precacheRagReranker() in rag-api, wired into the
Switch's onCheckedChange so the download runs synchronously
with a loading toast. On success: 'Reranker ready'. On failure:
error toast + auto-flip the switch back off so the next query
doesn't trigger another long hang.
First toggle-on pays the 1.1 GB download once; subsequent toggles
hit the HF cache and return ~instantly.
|
||
|---|---|---|
| .. | ||
| data_recipe | ||
| .gitkeep | ||
| __init__.py | ||
| auth.py | ||
| chat_history.py | ||
| datasets.py | ||
| export.py | ||
| inference.py | ||
| models.py | ||
| providers.py | ||
| rag.py | ||
| training.py | ||
| training_history.py | ||