Built on Qwen3 (not CLIP), so the 77-token text cap that bit
BGE-VL-base is gone — long chunks embed losslessly. Loads via
vanilla SentenceTransformer with trust_remote_code, no custom
adapter needed. 2048-d shared text+image space.
Adds qwen-vl-utils>=0.0.14 to rag.txt for image preprocessing,
required by the Qwen3-VL embedder family.
BGE-VL's sentence-transformers shim (bge_vl_clip_transformer.py)
is coupled to specific ST internals and broke across both the 5.2
and 5.3 pin attempts. The canonical load path documented at
bge-model.com is transformers.AutoModel with trust_remote_code +
model.set_processor() + model.encode(text=/images=).
- Wrap that path in _BGEVLAdapter exposing the slice of
SentenceTransformer API the ingester uses (encode for text or PIL
images, get_sentence_embedding_dimension, best-effort tokenize).
- Restore BAAI/BGE-VL-base in RAG_EMBEDDER_MATRIX.
- Revert sentence_transformers pin to 5.2.0 — no longer relevant
since the multimodal path no longer touches ST.
BGE-VL-base ships a custom Transformer subclass that's tightly
coupled to a specific sentence-transformers internal API — it
imports a module path missing in older ST and overrides forward()
expecting a return shape that changed in newer ST. We can't pin ST
to BGE-VL's exact version without risking the training/inference
paths that use the same library, so swap the multimodal default to
the canonical clip-ViT-B-32 which sentence-transformers wraps
natively (no trust_remote_code, no shim, ST-version-independent).
Same 512-d shared text+image space.
Phase 3 lays two orthogonal per-KB knobs in the data and API layers so
follow-up commits (Phase 3B-late, Phase 3B-multimodal) only need to add
their code path and UI selector, not schema or types.
Schema (studio/backend/storage/studio_db.py)
- rag_knowledge_bases gains chunking_strategy ('standard'|'late', default
'standard') and mode ('text'|'multimodal', default 'text'). Both are
immutable after KB creation — changing either invalidates existing
chunks because they were ingested through a specific pipeline.
- rag_chunks gains kind ('text'|'image'|'caption', default 'text'),
image_path (NULLABLE), linked_chunk_id (NULLABLE) — used by Phase
3B-multimodal to pair image chunks with their captions.
- Idempotent ALTER TABLE additions for existing installs (mirrors the
chat_threads display_name / *_code_exec_container_id pattern earlier
in the file).
API (studio/backend/routes/rag.py)
- ChunkingStrategy + KBMode Literal aliases.
- CreateKBRequest accepts both fields with backward-compat defaults.
- KBResponse exposes both.
- _validate_mode_combo rejects (multimodal, late) with 400 — no public
open-weight embedder supports both at once. Surface the constraint
early rather than failing silently during ingestion.
Config (studio/backend/utils/rag/config.py)
- RAG_EMBEDDER_MATRIX dict keyed by (mode, strategy) → embedder name.
- resolve_embedder() helper falls back to RAG_EMBEDDING_MODEL for legacy
KBs that pre-date the columns.
- (multimodal, late) intentionally absent.
Frontend (studio/frontend/src/features/rag/)
- api/rag-api.ts: ChunkingStrategy + KBMode types; KnowledgeBase
interface and createKnowledgeBase request type updated.
- stores/rag-store.ts: createKB signature uses the shared request type.
No user-visible UI changes yet — the only currently-usable combination
is (text, standard), so adding one-option selectors would be UX noise.
Phase 3B-late and Phase 3B-multimodal each add the relevant selector
option as part of shipping the code path.
