unsloth/studio/backend/routes
danielhanchen 3851a543d8 Studio: contained chat document extraction with fast adaptive parsing
Adds document extraction to the chat composer with a small footprint that
reuses the existing RAG preview UI and is fast by default.

Backend
- Adaptive PDF extraction: born-digital pages produce layout-aware Markdown
  via pymupdf4llm and render no page images, so a text PDF issues no VLM
  calls. Only pages without a text layer are detected as scanned and
  rendered for OCR.
- Scanned pages are transcribed (not summarized) through the already loaded
  vision model over /v1/chat/completions. No dedicated OCR model is loaded
  and the chat model is never swapped out.
- Adaptive render DPI (120, env override) and bounded async caption
  concurrency (2 local, 3 gguf, env override).
- /chat/document-support and /chat/extract-document endpoints with NDJSON
  streaming progress, cancellation, multipart size guards, and token-budget
  truncation.

Frontend
- Reuses the existing RAG DocumentPreviewSheet and MarkdownPreview to render
  an extracted document inline (a new markdown preview target), instead of a
  separate preview panel.
- Extraction uses whatever model is loaded; there is no OCR model picker,
  cross-tab lock, or custom-code consent step.
- Compact document chips in the composer and transcript; image data is
  stripped from persisted attachments.
- Document settings expose a mode (fast text, auto, scanned), a caption
  toggle, a token budget, and concurrency. Unknown settings keys are ignored.

Adds backend tests for the adaptive path, the support probe, NDJSON
streaming, cancellation, error mapping, and the scanned-page dedup.
2026-06-25 12:28:33 +00:00
..
data_recipe Studio: improve OpenAI- and Anthropic-compatible API spec compliance (#6010) 2026-06-09 17:13:25 +02:00
.gitkeep root studio folder 2026-02-02 09:13:49 +00:00
__init__.py Studio: clean-room compact RAG (knowledge bases, hybrid search, fast indexing) (#5910) 2026-06-09 21:17:04 -07:00
auth.py CLI: stop unsloth connect from leaking Studio credentials to unverified servers (#6479) 2026-06-21 21:28:38 -07:00
chat_history.py feat: implement thread forking functionality with associated database… (#5810) 2026-06-15 14:57:39 +01:00
datasets.py Studio: training survives a non-writable HF datasets cache (#6148) 2026-06-10 08:22:47 -07:00
export.py Studio: persistent per-user trust_remote_code approval cache (#6551) 2026-06-22 05:12:49 -07:00
inference.py Studio: contained chat document extraction with fast adaptive parsing 2026-06-25 12:28:33 +00:00
llama.py Studio: fix llama.cpp update toast tag and reload hint (#6493) 2026-06-21 05:40:49 -07:00
mcp_servers.py fix validation error for stdio mcp (#6341) 2026-06-15 19:26:41 +01:00
models.py Studio: persistent per-user trust_remote_code approval cache (#6551) 2026-06-22 05:12:49 -07:00
preview.py Studio: shareable per-checkpoint preview links (#6486) 2026-06-24 06:31:53 -07:00
prompts.py Studio: clean-room compact RAG (knowledge bases, hybrid search, fast indexing) (#5910) 2026-06-09 21:17:04 -07:00
providers.py Studio: Add custom provider option to Connections (#6112) 2026-06-12 13:09:35 +02:00
rag.py Studio: project sources backed by RAG (#6205) 2026-06-12 15:42:51 +02:00
settings.py studio: persist personalization (profile, avatar, theme) server-side (#6516) 2026-06-22 04:09:48 -07:00
training.py Add Hugging Face dataset streaming mode to Studio (#4946) 2026-06-22 17:48:18 +03:00
training_history.py Studio: shareable per-checkpoint preview links (#6486) 2026-06-24 06:31:53 -07:00
training_vram.py Studio: free chat model VRAM at training start only when the GPU is tight (#6243) 2026-06-18 09:04:01 -07:00