Matches #5910's text-only footprint. Removes image-vector embedding
(encode_images, _stream_image_chunks, the _BGEVLAdapter CLIP shim), the
multimodal `mode`/KBMode concept + VL embedders (single text embedder
now), the mode selector UI across the KB dialogs + thread settings, the
MM badges, the /images serving route, and the dead image rendering in
the search tool card. Captioning (figure text spliced into markdown)
stays — #5910 keeps it too. DB mode/image columns left dormant (no
migration). RagDefaultsSection dropped (no controls left).
Running the RAG suite surfaced three more test-only breakages (production
is correct):
- The scope tests patched `tool.__import__("core.rag.retrieval", ...)`, but a
module has no `__import__` attribute (AttributeError before the stub ran).
tool.py imports retrieval lazily, so patch core.rag.retrieval.retrieve_hybrid
at the source instead.
- test_format_hits_produces_fenced_chunks asserted a stale `score="..."`
attribute; the chunk tag emits chunk_index/tokens, not score.
- The execute_tool dispatch stub rejected the `mode` kwarg the handler passes.
Full RAG suite now: 101 passed, 5 skipped (server-gated model tests).
Backend:
- Deterministic SQLite connection cleanup. The RAG code used bare
`with get_connection() as conn:`, which commits but never closes, leaning
on GC to release handles (the rest of studio_db closes explicitly). Add a
closing_connection() context manager that commits/rolls back like sqlite3's
own manager and always closes, and route all 30 RAG call sites through it.
- filter_by_min_score no longer drops BM25-only and figure-ref hits. min_score
is a cosine floor, so it now gates only hits that carry a dense_score;
lexical and figure-ref hits (dense_score is None) pass through instead of
being silently discarded when the floor is raised.
- Fix two tests that could not pass against the production code: the RRF
fusion test asserted the wrong winner (c edges out b: 0.032266 vs 0.032258),
and two tool-handler scope tests stubbed retrieve_hybrid without accepting
the embedder_model kwarg the handler now passes (TypeError was swallowed,
leaving captured["scope"] unset).
Frontend:
- Removing an in-flight upload chip now routes through the teardown thunk
already registered for the aggregate-progress toast (abort, unsubscribe,
release the index slot, delete the backend doc with the correct kb/thread
scope key it closed over) and clears the toast entry. Deleting directly
leaked the concurrency slot and hardcoded the thread scope, mis-targeting
KB-scoped docs. Applied in both the composer hook and the compare-view
composer; drop the now-vestigial chip-scope-key tracking and unused
activeThreadId selectors. Add index-progress-store.remove(id).
Shorten and condense comments across the RAG backend, frontend, and
tests for readability. Comment text only; no code, strings, identifiers,
or logic changed. License headers and lint/type pragmas are preserved.
- chat-adapter pre-fetches retrieval on every turn when RAG is on,
regardless of provider. Users no longer have to phrase queries as
'the document I attached' for retrieval to fire. Local tool models
still get search_knowledge_base registered as a refinement path.
- New per-thread ragMinScore slider (Min relevance, 0..1) gates
retrieved hits by dense cosine similarity. Hits below the floor
(and BM25-only hits with no dense signal) are dropped server-side
so unrelated docs don't get injected when the user's query is
off-topic from what's indexed.
- Backend logs at search start (scope, top_k, min_score, query
preview), after retrieval (retrieved vs met_threshold counts),
and on return (final hit count) for both /api/rag/search and the
search_knowledge_base tool path.
- System-prompt nudge prepended when pre-fetch returns hits so the
model knows to cite [1], [2] rather than paraphrase silently.
Promotes RAG to a first-class composer toggle alongside Think / Web
Search / Code, with tool-use semantics on local models that support
tools and a pre-fetch fallback on external providers. The model
decides when to call `search_knowledge_base` on local inference; on
external providers retrieval still fires before each message (the
existing pre-fetch path), gated on the same button.
Backend
- core/rag/tool.py (new): search_knowledge_base handler + JSON-schema
tool spec. Resolves scope (kb_id wins over thread_id) from the
request's rag_scope, runs retrieve_hybrid + optional rerank, then
hydrates filename / page_number / text from sqlite and formats as
numbered Markdown citations ('[1] file.pdf (page 5): ...') for the
LLM to cite. Empty scope returns a user-facing hint; empty results
return a clear no-match message instead of an empty string.
- core/inference/tools.py: SEARCH_KNOWLEDGE_BASE_TOOL added to
ALL_TOOLS (lazy import keeps tools.py importable on inference
paths that never touch RAG). execute_tool() gains a tool_context
parameter that carries per-request extras the LLM doesn't see
(currently just rag_scope). The new 'search_knowledge_base' branch
dispatches to the handler with scope unpacked from tool_context.
- core/inference/llama_cpp.py + safetensors_agentic.py +
orchestrator.py: thread tool_context through generate_chat_completion_
with_tools / run_safetensors_tool_loop / execute_tool. Both local
backends (GGUF llama-server and safetensors agentic) carry the same
context object.
- models/inference.py: ChatCompletionRequest gains optional
rag_scope: dict ({kb_id?, thread_id?, enable_rerank?, default_top_k?,
reranker_model?}). Ignored unless 'search_knowledge_base' is in
enabled_tools.
- routes/inference.py: both the GGUF and safetensors call sites for
generate_chat_completion_with_tools forward payload.rag_scope into
tool_context.
Frontend
- chat-runtime-store.ts: global ragToolEnabled boolean + setter +
CHAT_RAG_TOOL_ENABLED_KEY localStorage, mirroring toolsEnabled /
codeToolsEnabled. Settings-hydration migration auto-flips
ragToolEnabled=true for pre-Phase-4 users who already had ragSource
set, so existing RAG users don't silently lose retrieval on upgrade.
- shared-composer.tsx: new 'RAG' pill button after Images (uses
lucide BookOpenIcon, composer-pill-btn style, data-active toggle).
Disabled when no model is loaded. Toggling on from ragSource='off'
auto-flips source to 'thread' so the sidebar lands ready-to-go.
- chat-adapter.ts:
* The existing pre-fetch block is now gated on ragToolEnabled AND
only fires when the tool path isn't viable (external provider OR
local model without tool-use support). Tool-capable local models
skip pre-fetch and let the LLM decide.
* The local-model body assembly adds 'search_knowledge_base' to
enabled_tools and packs ragSource + enableRerank + ragTopK into a
rag_scope object the backend tool handler consumes.
- chat-settings-sheet.tsx: entire Retrieval CollapsibleSection is
wrapped in {ragToolEnabled && ...} so it hides when the button is
off — the button is now the single on/off control. The 'Off'
option is removed from the Source dropdown (the button handles
that). Default open when shown so settings are one click away.
Tests
- test_rag_tool_handler.py: handler covers empty query, missing
scope, kb_id > thread_id precedence, thread-only path, citation
formatting (numbered + page numbers + unknown source); tool spec
shape (function/name/required); execute_tool dispatch with and
without tool_context; ALL_TOOLS includes the new spec without
dropping the existing ones.
Verification scope
- Local GGUF with tools: toggle button on, upload doc, ask about
doc content → assistant emits a search_knowledge_base tool call
card (rendered by the existing ToolFallback component since no
custom UI exists yet — that's a v2 nice-to-have).
- External provider (Anthropic / OpenAI / etc.): same button, same
UX, but uses the pre-fetch path under the hood.
- Migration: pre-existing ragSource != off → button initializes ON
so retrieval keeps working.
