Studio: move RAG deps to dedicated rag.txt and install in normal path

no-torch-runtime.txt is only consumed in NO_TORCH (Intel Mac
GGUF-only) mode, so qdrant-client / bm25s / pymupdf etc. were never
installed in the normal install path. Split them into rag.txt and
add a step to install_python_stack.py that installs it after studio
deps, skipped only when NO_TORCH is set.
This commit is contained in:
Roland Tannous 2026-05-24 16:23:26 +04:00
commit 6c694d2ef8
3 changed files with 40 additions and 21 deletions

View file

@ -76,24 +76,3 @@ trl>=0.18.2,!=0.19.0,<=0.24.0
sentence-transformers
cut_cross_entropy
pillow
# RAG: vector store, lexical index, document parsers.
# qdrant-client supports a pure-Python local mode (QdrantClient(path=...))
# that we use to keep the studio install self-contained — no separate
# server. bm25s persists per-scope indices to disk.
qdrant-client>=1.12
bm25s>=0.2
# RAG parsers (Phase 3A): layout-aware Markdown extraction so the chunker
# can split on real headings instead of running paragraphs together.
# pymupdf4llm preserves headings + pipe-tables; mammoth handles DOCX
# Heading styles; markdownify converts HTML <h*>/<table>/<ul> faithfully.
pymupdf>=1.24
pymupdf4llm>=0.0.17
mammoth>=1.7
markdownify>=0.13
# pypdf is kept as a fallback for malformed PDFs that defeat pymupdf.
pypdf>=4.0
python-docx>=1.1
beautifulsoup4>=4.12
lxml>=5.0
chardet>=5.2

View file

@ -0,0 +1,27 @@
# Studio RAG dependencies.
# Installed by studio/install_python_stack.py in the normal (with-torch)
# path. Skipped in NO_TORCH (Intel Mac GGUF-only) mode because RAG
# embedding relies on sentence-transformers, which requires torch.
# Vector store + lexical index. qdrant-client supports a pure-Python
# local mode (QdrantClient(path=...)) that we use to keep the studio
# install self-contained — no separate server. bm25s persists per-scope
# indices to disk.
qdrant-client>=1.12
bm25s>=0.2
# Layout-aware Markdown extraction (Phase 3A) so the chunker can split
# on real headings instead of running paragraphs together. pymupdf4llm
# preserves headings + pipe-tables; mammoth handles DOCX Heading styles;
# markdownify converts HTML <h*>/<table>/<ul> faithfully.
pymupdf>=1.24
pymupdf4llm>=0.0.17
mammoth>=1.7
markdownify>=0.13
# pypdf is kept as a fallback for malformed PDFs that defeat pymupdf.
pypdf>=4.0
python-docx>=1.1
beautifulsoup4>=4.12
lxml>=5.0
chardet>=5.2

View file

@ -915,6 +915,8 @@ def install_python_stack() -> int:
base_total -= 1 # triton step is skipped on macOS
if not IS_WINDOWS and not IS_MACOS and not NO_TORCH:
base_total += 3
if not NO_TORCH:
base_total += 1 # studio RAG deps (rag.txt)
_TOTAL = (base_total - 1) if skip_base else base_total
# 1. Try to use uv for faster installs (must happen before pip upgrade
@ -1203,6 +1205,17 @@ def install_python_stack() -> int:
req = REQ_ROOT / "studio.txt",
)
# 8b. RAG dependencies (vector store + lexical index + layout-aware
# parsers). Skipped in no-torch mode because RAG embeddings go
# through sentence-transformers, which requires torch.
if not NO_TORCH:
_progress("rag deps")
pip_install(
"Installing RAG dependencies",
"--no-cache-dir",
req = REQ_ROOT / "rag.txt",
)
# 9. Data-designer dependencies
_progress("data designer deps")
pip_install(