unsloth/studio/backend/utils
Roland Tannous 4c1ab745d6 Studio: schema + API plumbing for per-KB chunking strategy + mode
Phase 3 lays two orthogonal per-KB knobs in the data and API layers so
follow-up commits (Phase 3B-late, Phase 3B-multimodal) only need to add
their code path and UI selector, not schema or types.

Schema (studio/backend/storage/studio_db.py)
- rag_knowledge_bases gains chunking_strategy ('standard'|'late', default
  'standard') and mode ('text'|'multimodal', default 'text'). Both are
  immutable after KB creation — changing either invalidates existing
  chunks because they were ingested through a specific pipeline.
- rag_chunks gains kind ('text'|'image'|'caption', default 'text'),
  image_path (NULLABLE), linked_chunk_id (NULLABLE) — used by Phase
  3B-multimodal to pair image chunks with their captions.
- Idempotent ALTER TABLE additions for existing installs (mirrors the
  chat_threads display_name / *_code_exec_container_id pattern earlier
  in the file).

API (studio/backend/routes/rag.py)
- ChunkingStrategy + KBMode Literal aliases.
- CreateKBRequest accepts both fields with backward-compat defaults.
- KBResponse exposes both.
- _validate_mode_combo rejects (multimodal, late) with 400 — no public
  open-weight embedder supports both at once. Surface the constraint
  early rather than failing silently during ingestion.

Config (studio/backend/utils/rag/config.py)
- RAG_EMBEDDER_MATRIX dict keyed by (mode, strategy) → embedder name.
- resolve_embedder() helper falls back to RAG_EMBEDDING_MODEL for legacy
  KBs that pre-date the columns.
- (multimodal, late) intentionally absent.

Frontend (studio/frontend/src/features/rag/)
- api/rag-api.ts: ChunkingStrategy + KBMode types; KnowledgeBase
  interface and createKnowledgeBase request type updated.
- stores/rag-store.ts: createKB signature uses the shared request type.

No user-visible UI changes yet — the only currently-usable combination
is (text, standard), so adding one-option selectors would be UX noise.
Phase 3B-late and Phase 3B-multimodal each add the relevant selector
option as part of shipping the code path.
2026-05-24 11:54:38 +04:00
..
datasets Studio: tools, thinking blocks, code execution and web search for safetensors (#5520) 2026-05-19 06:30:17 -07:00
hardware feat(studio): MLX training tab on Apple Silicon (LoRA / full FT, VLM, export) (#5265) 2026-05-05 23:54:58 -07:00
inference Fix Windows local GGUF model loading crash (#4730) 2026-04-01 01:38:09 -07:00
models studio: extend offline DNS auto-detect to inference parent + training (#5512) 2026-05-18 00:31:33 -07:00
paths Studio: add RAG with hybrid search, reranker, chat integration 2026-05-23 18:46:15 +04:00
rag Studio: schema + API plumbing for per-KB chunking strategy + mode 2026-05-24 11:54:38 +04:00
.gitkeep root studio folder 2026-02-02 09:13:49 +00:00
__init__.py Final cleanup 2026-03-12 18:28:04 +00:00
_studio_release_build.py Add Studio web update banner and release version display (#5308) 2026-05-11 18:24:01 +04:00
cache_cleanup.py fix(studio): prevent ModuleNotFoundError in dataset.map() on Windows (#4473) 2026-03-22 06:11:24 -07:00
downsample.py feat(studio): training history persistence and past runs viewer (#4501) 2026-03-25 00:58:55 -07:00
llama_cpp_freshness.py Studio: warn when llama.cpp prebuilt is at least 3 days behind (#5529) 2026-05-18 00:21:50 -07:00
native_path_leases.py Add native GGUF intake to Studio (#5246) 2026-05-04 11:46:18 +02:00
studio_version.py Add Studio web update banner and release version display (#5308) 2026-05-11 18:24:01 +04:00
subprocess_compat.py Add tauri (#5144) 2026-04-23 04:50:10 -07:00
transformers_version.py studio: extend offline DNS auto-detect to inference parent + training (#5512) 2026-05-18 00:31:33 -07:00
update_status.py Add Studio web update banner and release version display (#5308) 2026-05-11 18:24:01 +04:00
utils.py Final cleanup 2026-03-12 18:28:04 +00:00
wheel_utils.py studio: skip flash-attn install on Blackwell GPUs (sm_100+) (#5420) 2026-05-14 18:13:50 +04:00