Commit graph

10 commits

Author SHA1 Message Date
Roland Tannous
2ce77f8879 fix: probe image column candidates when multiple exist
When multiple image columns are found, probes them (HEAD for URLs,
os.path.exists for paths) and picks the first that works.
Skips probing when top candidate is PIL/dict (score >= 75).
2026-03-10 01:38:33 +00:00
Roland Tannous
b515ce6a7d fix: prefer URL image columns over bare filenames, add value-based fallback
find_image_column now scores candidates by resolvability (PIL > dict > URL > path)
and has a Pass 2 value-based fallback for columns not matching image keywords.
Fixes phiyodr/coco2017 picking file_name (unresolvable) over coco_url (resolvable).
2026-03-10 01:36:19 +00:00
Roland Tannous
698c9564ef fix: detect list-of-strings text columns and pick random element for VLM conversion
Handles datasets like phiyodr/coco2017 where captions is a list of strings.
2026-03-10 01:32:19 +00:00
Roland Tannous
095a051ee0 feat: add ShareGPT+image VLM format support and improve image column detection
- Detect and convert ShareGPT/ChatML conversations with <image> placeholders
- Add file_name/filename as image column keywords
- Detect image paths and URLs by value (string ending in .jpg/.png/etc)
2026-03-10 01:27:36 +00:00
Roland Tannous
87269a5c85 fix: use word-boundary matching for image/audio column detection
Substring matching caused false positives like 'pic' in 'topic',
leading to non-deterministic image column selection.
2026-03-10 00:38:02 +00:00
Roland Tannous
a0f03d3080 Add AGPL-3.0 SPDX headers to all source files 2026-03-09 20:17:45 +00:00
Manan17
6fd1dd2c0a variable changes and some cleanup 2026-03-03 09:35:11 +00:00
Manan17
90332924de Changes with audio training 2026-03-01 02:27:45 +00:00
Roland Tannous
11b3029dc6 Simplify dataset check to 2-tier, improve multimodal detection, auto-set trainOnCompletions, recheck dataset on reload 2026-02-19 11:25:54 +00:00
Roland Tannous
47ead076cf Refactor [dataset_utils.py](cci:7://file:///home/support/new-ui-prototype/studio/backend/utils/datasets/dataset_utils.py:0:0-0:0) into focused modules 2026-02-03 14:38:02 +00:00