4f24b12cc9* Add customizable RAG embedding model setting and reorganize settings tabs Chat with files, project sources, and knowledge bases previously always embedded with unsloth/bge-small-en-v1.5. This adds a Settings option to pick any Hugging Face embedding model (or local path), with HF search autocomplete, server-side verification that the repo is actually an embedding model, and a save anyway escape hatch for offline or local models. The setting persists in app_settings and applies at runtime to both the sentence-transformers and llama-server GGUF embedder backends without a restart. Also reorganizes the General settings tab: Documents & RAG sits above Uploads, Helper LLM moved above the danger zone, and Model auto-switch (OpenAI API) moved to the bottom of the API tab. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Support local model paths on the GGUF embedder and normalize default saves Found by simulation testing of the embedding model setting: Local paths saved as the embedding model now work on the llama-server GGUF backend (the default backend on macOS and CPU). A path to a .gguf file is used directly and a directory is scanned for a variant-matching non-mmproj .gguf, with a clear error when none exists. Previously a local path was sent to the HF hub API and failed with a repo lookup error. Saving the default model explicitly no longer stores an override, so is_custom stays false and the UI does not show a reset button for the default value. * Address review: stale-vector handling, GGUF derivation, save-time guards Review follow-ups, each verified by new tests: Re-uploading a document after an embedding model change now re-indexes instead of deduping by content hash. Documents record the embedder that produced their vectors (lazy embedding_model column, NULL legacy rows keep deduping) and a mismatch replaces the old document. A vector width change no longer bricks the dense index. ensure_vec drops and recreates chunks_vec when the dim changes (old vectors are in a foreign space and only block inserts) and search_dense returns empty on a width mismatch instead of surfacing a vec0 error, so lexical search keeps working until documents are re-uploaded. Saving a local sentence-transformers folder with no .gguf now returns 409 with a clear message when the install embeds via llama-server, instead of failing at first index. force still saves. A custom RAG_EMBEDDING_MODEL env without RAG_EMBED_GGUF_REPO now derives the -GGUF companion repo instead of silently keeping the bge GGUF on CPU and macOS installs. The resolved GGUF path is tagged with the repo captured at entry, so a setting change during a download cannot mark the old model as current. GGUF repo detection matches gguf as a whole name segment rather than a substring, hf_token is trimmed before verification, and the settings combobox drops a redundant state mirror of its controlled value. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Shrink embedding model font to 11px in the input and dropdown The combobox wrapper applies className to the outer input group, so the size utility must target the inner input element; the previous text-xs never reached it and the field rendered at the browser default. * Show curated unsloth embedding models when the search field is empty The empty-query listing was the global top-downloads page, which holds no unsloth mirrors for the unsloth-first float to reorder, so the dropdown opened on third-party models. Match the model picker: curated unsloth listing when empty, whole-Hub search once a query is typed. * Address review: settings resilience and index consistency Keep the last known embedding model on settings store errors, remove the re-entrant dim lock in the llama-server backend, accept local GGUF saves and verify GGUF availability for HF repos on that backend, match local path embedders exactly in model list filters, drop same-width stale vectors from dense search, pin the embedder per ingestion job, and only replace completed documents after the re-index succeeds. * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Consolidate the GGUF repo derivation tests * Trim to a single core embedding-model test * Address review: GGUF repo saves and cache race Accept a GGUF-named HF repo on the llama-server backend by verifying GGUF availability instead of the sentence-transformers metadata gate, and guard the settings cache with a generation counter so a read overlapping a save cannot repopulate it with the pre-save value. --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
55 lines
2.1 KiB
Python
55 lines
2.1 KiB
Python
# SPDX-License-Identifier: AGPL-3.0-only
|
|
# Copyright 2026-present the Unsloth AI Inc. team. All rights reserved. See /studio/LICENSE.AGPL-3.0
|
|
|
|
"""Test for the customizable RAG embedding model: a saved override becomes the
|
|
effective model and derives its GGUF companion for the llama-server backend."""
|
|
|
|
from pathlib import Path
|
|
import sys
|
|
import types as _types
|
|
|
|
_BACKEND_DIR = str(Path(__file__).resolve().parent.parent)
|
|
if _BACKEND_DIR not in sys.path:
|
|
sys.path.insert(0, _BACKEND_DIR)
|
|
|
|
_loggers_stub = _types.ModuleType("loggers")
|
|
_loggers_stub.get_logger = lambda name: __import__("logging").getLogger(name)
|
|
sys.modules.setdefault("loggers", _loggers_stub)
|
|
|
|
import pytest
|
|
|
|
import utils.embedding_model_settings as ems
|
|
from core.rag import config as rag_config
|
|
|
|
|
|
@pytest.fixture
|
|
def settings_store(monkeypatch):
|
|
"""In-memory app_settings store patched under the module's lazy imports."""
|
|
import storage.studio_db as studio_db
|
|
|
|
store: dict = {}
|
|
monkeypatch.setattr(
|
|
studio_db, "get_app_setting", lambda key, fallback = None: store.get(key, fallback)
|
|
)
|
|
monkeypatch.setattr(
|
|
studio_db, "upsert_app_settings", lambda settings: store.update(settings) or store
|
|
)
|
|
ems._invalidate_cache()
|
|
yield store
|
|
ems._invalidate_cache()
|
|
|
|
|
|
def test_custom_model_overrides_default_and_derives_gguf(settings_store, monkeypatch):
|
|
"""The core contract: with nothing stored the default is in effect; a saved
|
|
custom model becomes the effective embedding model and derives its -GGUF
|
|
companion (what the llama-server backend loads); reset clears the override."""
|
|
monkeypatch.delenv("RAG_EMBED_GGUF_REPO", raising = False)
|
|
assert ems.get_rag_embedding_model() == rag_config.EMBEDDING_MODEL
|
|
assert rag_config.effective_gguf_repo() == rag_config.EMBED_GGUF_REPO
|
|
|
|
assert ems.set_rag_embedding_model(" org/my-embedder ") == "org/my-embedder"
|
|
assert rag_config.effective_embedding_model() == "org/my-embedder"
|
|
assert rag_config.effective_gguf_repo() == "org/my-embedder-GGUF"
|
|
|
|
assert ems.reset_rag_embedding_model() == rag_config.EMBEDDING_MODEL
|
|
assert ems.get_stored_embedding_model() is None
|