unsloth/studio/backend/core
Roland Tannous 0be7ca39a4 Studio: render figure regions (vector + raster) for RAG captioning
page.get_images() only returns raster blobs embedded in the PDF's
resource dictionary, so vector schematics like Figure 1 — drawn purely
with paths/lines — were never extracted, and the VLM only ever saw
incidental embedded photos that happened to live near figures.

Replace the xref-based extraction with bbox rendering: union the
bounding rects of all vector drawings and raster image_info entries on
each page, expand a few points, and render the region with
get_pixmap(clip=bbox, matrix=2x). The captioner now receives the
actual figure — schematic arrows, box labels, legend text, and any
inset photos — and produces a caption that describes the figure as a
whole, not just one embedded sub-image.

Also sharpen the captioner prompt: explicitly tell the VLM the image
is a single figure cropped from a PDF page, and not to describe page
chrome or body paragraphs.
2026-05-27 16:36:54 +04:00
..
data_recipe chore(deps): bump the npm-oxc-validator group across 1 directory with 2 updates (#5667) 2026-05-22 04:46:20 -07:00
export feat(studio): MLX training tab on Apple Silicon (LoRA / full FT, VLM, export) (#5265) 2026-05-05 23:54:58 -07:00
inference Studio: don't kill chat-model llama-server when spawning helper backends 2026-05-27 12:44:44 +04:00
rag Studio: render figure regions (vector + raster) for RAG captioning 2026-05-27 16:36:54 +04:00
training studio: install flash-linear-attention and tilelang for Qwen3.5 family (#5434) 2026-05-18 03:49:06 -07:00
__init__.py [Studio] Show non exported models in chat UI (#4892) 2026-04-14 15:03:58 +04:00
tool_healing.py studio: extract tool-call XML parser into a reusable helper module (#5583) 2026-05-19 05:06:17 -07:00