Studio: RAG-as-tool composer button (Phase 4)
Promotes RAG to a first-class composer toggle alongside Think / Web
Search / Code, with tool-use semantics on local models that support
tools and a pre-fetch fallback on external providers. The model
decides when to call `search_knowledge_base` on local inference; on
external providers retrieval still fires before each message (the
existing pre-fetch path), gated on the same button.
Backend
- core/rag/tool.py (new): search_knowledge_base handler + JSON-schema
tool spec. Resolves scope (kb_id wins over thread_id) from the
request's rag_scope, runs retrieve_hybrid + optional rerank, then
hydrates filename / page_number / text from sqlite and formats as
numbered Markdown citations ('[1] file.pdf (page 5): ...') for the
LLM to cite. Empty scope returns a user-facing hint; empty results
return a clear no-match message instead of an empty string.
- core/inference/tools.py: SEARCH_KNOWLEDGE_BASE_TOOL added to
ALL_TOOLS (lazy import keeps tools.py importable on inference
paths that never touch RAG). execute_tool() gains a tool_context
parameter that carries per-request extras the LLM doesn't see
(currently just rag_scope). The new 'search_knowledge_base' branch
dispatches to the handler with scope unpacked from tool_context.
- core/inference/llama_cpp.py + safetensors_agentic.py +
orchestrator.py: thread tool_context through generate_chat_completion_
with_tools / run_safetensors_tool_loop / execute_tool. Both local
backends (GGUF llama-server and safetensors agentic) carry the same
context object.
- models/inference.py: ChatCompletionRequest gains optional
rag_scope: dict ({kb_id?, thread_id?, enable_rerank?, default_top_k?,
reranker_model?}). Ignored unless 'search_knowledge_base' is in
enabled_tools.
- routes/inference.py: both the GGUF and safetensors call sites for
generate_chat_completion_with_tools forward payload.rag_scope into
tool_context.
Frontend
- chat-runtime-store.ts: global ragToolEnabled boolean + setter +
CHAT_RAG_TOOL_ENABLED_KEY localStorage, mirroring toolsEnabled /
codeToolsEnabled. Settings-hydration migration auto-flips
ragToolEnabled=true for pre-Phase-4 users who already had ragSource
set, so existing RAG users don't silently lose retrieval on upgrade.
- shared-composer.tsx: new 'RAG' pill button after Images (uses
lucide BookOpenIcon, composer-pill-btn style, data-active toggle).
Disabled when no model is loaded. Toggling on from ragSource='off'
auto-flips source to 'thread' so the sidebar lands ready-to-go.
- chat-adapter.ts:
* The existing pre-fetch block is now gated on ragToolEnabled AND
only fires when the tool path isn't viable (external provider OR
local model without tool-use support). Tool-capable local models
skip pre-fetch and let the LLM decide.
* The local-model body assembly adds 'search_knowledge_base' to
enabled_tools and packs ragSource + enableRerank + ragTopK into a
rag_scope object the backend tool handler consumes.
- chat-settings-sheet.tsx: entire Retrieval CollapsibleSection is
wrapped in {ragToolEnabled && ...} so it hides when the button is
off — the button is now the single on/off control. The 'Off'
option is removed from the Source dropdown (the button handles
that). Default open when shown so settings are one click away.
Tests
- test_rag_tool_handler.py: handler covers empty query, missing
scope, kb_id > thread_id precedence, thread-only path, citation
formatting (numbered + page numbers + unknown source); tool spec
shape (function/name/required); execute_tool dispatch with and
without tool_context; ALL_TOOLS includes the new spec without
dropping the existing ones.
Verification scope
- Local GGUF with tools: toggle button on, upload doc, ask about
doc content → assistant emits a search_knowledge_base tool call
card (rendered by the existing ToolFallback component since no
custom UI exists yet — that's a v2 nice-to-have).
- External provider (Anthropic / OpenAI / etc.): same button, same
UX, but uses the pre-fetch path under the hood.
- Migration: pre-existing ragSource != off → button initializes ON
so retrieval keeps working.
This commit is contained in:
parent
7e069816a1
commit
c74fc13ebc
12 changed files with 531 additions and 10 deletions
193
tests/python/test_rag_tool_handler.py
Normal file
193
tests/python/test_rag_tool_handler.py
Normal file
|
|
@ -0,0 +1,193 @@
|
|||
"""Unit tests for the `search_knowledge_base` tool handler (Phase 4)."""
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
|
||||
REPO_ROOT = Path(__file__).resolve().parents[2]
|
||||
STUDIO_BACKEND = REPO_ROOT / "studio" / "backend"
|
||||
if str(STUDIO_BACKEND) not in sys.path:
|
||||
sys.path.insert(0, str(STUDIO_BACKEND))
|
||||
|
||||
|
||||
def _make_hit(chunk_id: str):
|
||||
"""Minimal stand-in for retrieval.Hit — just needs .chunk_id."""
|
||||
class _Hit:
|
||||
pass
|
||||
h = _Hit()
|
||||
h.chunk_id = chunk_id
|
||||
h.score = 1.0
|
||||
h.kind = "text"
|
||||
h.document_id = None
|
||||
h.chunk_index = 0
|
||||
return h
|
||||
|
||||
|
||||
def test_empty_query_returns_error():
|
||||
from core.rag.tool import search_knowledge_base
|
||||
|
||||
result = search_knowledge_base(query = "", scope_thread_id = "t-1")
|
||||
assert result.startswith("Error:")
|
||||
assert "empty" in result.lower()
|
||||
|
||||
|
||||
def test_missing_scope_returns_user_facing_hint():
|
||||
from core.rag.tool import search_knowledge_base
|
||||
|
||||
result = search_knowledge_base(
|
||||
query = "anything",
|
||||
scope_kb_id = None,
|
||||
scope_thread_id = None,
|
||||
)
|
||||
assert "No knowledge base" in result
|
||||
assert "thread documents" in result
|
||||
|
||||
|
||||
def test_kb_takes_precedence_over_thread():
|
||||
"""When both kb_id and thread_id are passed, kb_id wins."""
|
||||
from core.rag import tool
|
||||
|
||||
captured = {}
|
||||
|
||||
def _stub_retrieve(scope, query, k):
|
||||
captured["scope"] = scope
|
||||
return []
|
||||
|
||||
with patch.object(tool.__import__("core.rag.retrieval", fromlist = ["retrieve_hybrid"]),
|
||||
"retrieve_hybrid",
|
||||
_stub_retrieve):
|
||||
result = tool.search_knowledge_base(
|
||||
query = "x",
|
||||
scope_kb_id = "kb-abc",
|
||||
scope_thread_id = "thread-xyz",
|
||||
)
|
||||
|
||||
assert captured["scope"].startswith("kb_")
|
||||
assert "kb-abc" in captured["scope"]
|
||||
assert "thread" not in captured["scope"].split("kb_")[1]
|
||||
|
||||
|
||||
def test_thread_scope_when_only_thread_set():
|
||||
from core.rag import tool
|
||||
|
||||
captured = {}
|
||||
|
||||
def _stub_retrieve(scope, query, k):
|
||||
captured["scope"] = scope
|
||||
return []
|
||||
|
||||
with patch.object(tool.__import__("core.rag.retrieval", fromlist = ["retrieve_hybrid"]),
|
||||
"retrieve_hybrid",
|
||||
_stub_retrieve):
|
||||
tool.search_knowledge_base(
|
||||
query = "x",
|
||||
scope_thread_id = "thread-xyz",
|
||||
)
|
||||
|
||||
assert captured["scope"].startswith("thread_")
|
||||
|
||||
|
||||
def test_empty_results_message_is_user_facing():
|
||||
from core.rag.tool import _format_hits_for_llm
|
||||
|
||||
result = _format_hits_for_llm([])
|
||||
assert "No matching chunks" in result
|
||||
|
||||
|
||||
def test_format_hits_produces_numbered_citations():
|
||||
from core.rag.tool import _format_hits_for_llm
|
||||
|
||||
hits = [
|
||||
{"filename": "alpha.pdf", "page_number": 3, "text": "first body"},
|
||||
{"filename": "beta.md", "page_number": None, "text": "second body"},
|
||||
]
|
||||
result = _format_hits_for_llm(hits)
|
||||
assert "[1] alpha.pdf (page 3): first body" in result
|
||||
assert "[2] beta.md: second body" in result
|
||||
# Each hit on its own paragraph so the LLM can cite cleanly.
|
||||
assert "\n\n" in result
|
||||
|
||||
|
||||
def test_format_hits_handles_unknown_source():
|
||||
from core.rag.tool import _format_hits_for_llm
|
||||
|
||||
hits = [{"filename": None, "page_number": None, "text": "orphan"}]
|
||||
result = _format_hits_for_llm(hits)
|
||||
assert "[1] unknown source: orphan" in result
|
||||
|
||||
|
||||
def test_tool_spec_shape_is_openai_compatible():
|
||||
from core.rag.tool import SEARCH_KNOWLEDGE_BASE_TOOL
|
||||
|
||||
assert SEARCH_KNOWLEDGE_BASE_TOOL["type"] == "function"
|
||||
fn = SEARCH_KNOWLEDGE_BASE_TOOL["function"]
|
||||
assert fn["name"] == "search_knowledge_base"
|
||||
assert "query" in fn["parameters"]["required"]
|
||||
assert "top_k" in fn["parameters"]["properties"]
|
||||
# Description should hint at when to call so the LLM picks it up
|
||||
# appropriately. Don't lock the exact wording.
|
||||
assert "documents" in fn["description"].lower()
|
||||
|
||||
|
||||
def test_execute_tool_dispatches_to_search_knowledge_base():
|
||||
"""tools.execute_tool should route 'search_knowledge_base' correctly."""
|
||||
from core.inference import tools
|
||||
|
||||
called = {}
|
||||
|
||||
def _stub(*, query, top_k = None, scope_kb_id = None, scope_thread_id = None,
|
||||
enable_rerank = False, reranker_model = None, default_top_k = 5):
|
||||
called["query"] = query
|
||||
called["top_k"] = top_k
|
||||
called["scope_kb_id"] = scope_kb_id
|
||||
called["scope_thread_id"] = scope_thread_id
|
||||
called["enable_rerank"] = enable_rerank
|
||||
called["default_top_k"] = default_top_k
|
||||
return "stub-result"
|
||||
|
||||
with patch("core.rag.tool.search_knowledge_base", _stub):
|
||||
result = tools.execute_tool(
|
||||
"search_knowledge_base",
|
||||
{"query": "hello", "top_k": 7},
|
||||
tool_context = {
|
||||
"rag_scope": {
|
||||
"kb_id": "kb-1",
|
||||
"enable_rerank": True,
|
||||
"default_top_k": 3,
|
||||
}
|
||||
},
|
||||
)
|
||||
assert result == "stub-result"
|
||||
assert called["query"] == "hello"
|
||||
assert called["top_k"] == 7
|
||||
assert called["scope_kb_id"] == "kb-1"
|
||||
assert called["scope_thread_id"] is None
|
||||
assert called["enable_rerank"] is True
|
||||
assert called["default_top_k"] == 3
|
||||
|
||||
|
||||
def test_execute_tool_handles_missing_tool_context():
|
||||
"""tool_context=None should still dispatch without crashing."""
|
||||
from core.inference import tools
|
||||
|
||||
def _stub(*, query, **_kwargs):
|
||||
return f"got: {query}"
|
||||
|
||||
with patch("core.rag.tool.search_knowledge_base", _stub):
|
||||
result = tools.execute_tool(
|
||||
"search_knowledge_base",
|
||||
{"query": "ping"},
|
||||
tool_context = None,
|
||||
)
|
||||
assert result == "got: ping"
|
||||
|
||||
|
||||
def test_all_tools_includes_rag():
|
||||
from core.inference.tools import ALL_TOOLS
|
||||
|
||||
names = [t["function"]["name"] for t in ALL_TOOLS]
|
||||
assert "search_knowledge_base" in names
|
||||
assert "web_search" in names # regression — we shouldn't have removed the others
|
||||
assert "python" in names
|
||||
Loading…
Add table
Add a link
Reference in a new issue