2093fb1608
asg017/sqlite-vec is Apache-2.0 and OSI-approved. Replaces qdrant-client (~30 MB) with a small SQLite extension loaded into a dedicated rag.db file. Single file holds RAG vectors; bm25s indexes and chat-side studio.db are unaffected. - New core/rag/db.py owns the rag.db connection and sqlite-vec load. Extension load runs once at first open. Process-wide singleton protected by a lock; check_same_thread=False + WAL handles the FastAPI thread pool. - core/rag/vector_store.py keeps the same public API (ensure_collection / upsert_chunks / search / collection_exists / delete_scope / delete_document) so callers in routes/rag.py, core/rag/ingestion.py, core/rag/tool.py, and core/rag/retrieval.py don't change. ensure_collection is now a no-op; collection_exists returns True iff the scope has at least one indexed vector. - search uses sqlite-vec's vec_distance_cosine and converts distance to similarity in [0, 1] so the per-scope min_score threshold semantics stay identical. - Mixed-dim scopes coexist behind WHERE scope = ? — the per-scope embedder resolver guarantees one embedder per scope. - requirements/rag.txt swaps qdrant-client for sqlite-vec. - utils/paths/storage_roots.py drops rag_vectordb_root() (the old qdrant directory); rag.db lives directly under rag_root(). - Rewritten tests/python/test_rag_vector_store.py for the new semantics (collection_exists tracks populated scopes; new tests for filtered search and upsert conflict resolution). Python build requirement: connection.enable_load_extension(True) must be available. install.sh creates the venv via uv-managed python-build-standalone, which is compiled with --enable-loadable-sqlite-extensions, so this works on standard installs. core/rag/db.py raises an actionable error on the rare custom-interpreter case. |
||
|---|---|---|
| .. | ||
| data_recipe | ||
| .gitkeep | ||
| __init__.py | ||
| auth.py | ||
| chat_history.py | ||
| datasets.py | ||
| export.py | ||
| inference.py | ||
| models.py | ||
| providers.py | ||
| rag.py | ||
| training.py | ||
| training_history.py | ||