Studio: swap multimodal RAG embedder to Qwen3-VL-Embedding-2B
Built on Qwen3 (not CLIP), so the 77-token text cap that bit BGE-VL-base is gone — long chunks embed losslessly. Loads via vanilla SentenceTransformer with trust_remote_code, no custom adapter needed. 2048-d shared text+image space. Adds qwen-vl-utils>=0.0.14 to rag.txt for image preprocessing, required by the Qwen3-VL embedder family.
This commit is contained in:
parent
810e3a80be
commit
8666ec9bd6
2 changed files with 10 additions and 6 deletions
|
|
@ -10,6 +10,10 @@
|
|||
qdrant-client>=1.12
|
||||
bm25s>=0.2
|
||||
|
||||
# Image preprocessing helpers required by Qwen3-VL-Embedding-2B (the
|
||||
# multimodal embedder). Not used in text-only mode.
|
||||
qwen-vl-utils>=0.0.14
|
||||
|
||||
# Layout-aware Markdown extraction (Phase 3A) so the chunker can split
|
||||
# on real headings instead of running paragraphs together. pymupdf4llm
|
||||
# preserves headings + pipe-tables; mammoth handles DOCX Heading styles;
|
||||
|
|
|
|||
|
|
@ -40,12 +40,12 @@ RAG_EMBEDDING_MODEL: str = (
|
|||
RAG_EMBEDDER_MATRIX: dict[tuple[str, str], str] = {
|
||||
("text", "standard"): "BAAI/bge-small-en-v1.5",
|
||||
("text", "late"): "nomic-ai/nomic-embed-text-v1.5",
|
||||
# BGE-VL-base is loaded via the canonical `transformers.AutoModel`
|
||||
# path (with trust_remote_code) — see `_BGEVLAdapter` in
|
||||
# core/rag/embeddings.py. This bypasses BGE-VL's sentence-transformers
|
||||
# shim entirely, sidestepping the ST-version coupling in the repo's
|
||||
# custom Transformer subclass. 512-d shared text+image space.
|
||||
("multimodal", "standard"): "BAAI/BGE-VL-base",
|
||||
# Qwen3-VL-Embedding-2B: 2B-param multimodal embedder built on
|
||||
# Qwen3 (not CLIP), so no 77-token text cap — long chunks embed
|
||||
# losslessly. Loads through vanilla SentenceTransformer with
|
||||
# trust_remote_code, no shim. 2048-d shared text/image space.
|
||||
# Trade-off: ~4 GB download / VRAM vs BGE-VL-base's ~600 MB.
|
||||
("multimodal", "standard"): "Qwen/Qwen3-VL-Embedding-2B",
|
||||
}
|
||||
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue