Commit graph

16 commits

Author SHA1 Message Date
Wasim Yousef Said
33f4397b78
Studio fix recipe dataset preview (#6031)
* Studio: fix recipe dataset preview

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2026-06-09 14:02:00 +02:00
Daniel Han
187144d4e7
Reduce and tighten code comments and docstrings repo-wide (#6095)
Trim and tighten code comments and docstrings across the repository. Comment-only: every changed file verified code-identical to main via AST/token comparison.
2026-06-08 23:09:51 -07:00
Daniel Han
8292e699e4
Studio: make code comments and docstrings more succinct (#6029)
Trim and tighten code comments and docstrings across studio/ Python. Comment-only: every changed file verified code-identical to main via AST/token comparison.
2026-06-08 23:07:28 -07:00
Daniel Han
3ce187da02
Formatting: ruff line-length 100, kwarg-spacing passes, drop blank after short local imports (#6079)
Raise ruff line-length to 100 and extend the local pre-commit format pipeline (def-signature magic-comma normalization, short multi-line assert collapse, kwarg '=' spacing, blank-line-after-short-import removal, adjacent string-literal / f-string+plain merge, redundant-pass pruning). Every transform re-checks the file AST and is dropped if it would differ; the whole-repo reformat is verified AST-identical per file and idempotent.
2026-06-08 04:24:13 -07:00
Roland Tannous
47654cb91c Final cleanup 2026-03-12 18:28:04 +00:00
Roland Tannous
a2baf80511 Update license headers 2026-03-12 17:23:10 +00:00
Roland Tannous
dd6c38cc7b fix: probe image column candidates when multiple exist
When multiple image columns are found, probes them (HEAD for URLs,
os.path.exists for paths) and picks the first that works.
Skips probing when top candidate is PIL/dict (score >= 75).
2026-03-10 01:38:33 +00:00
Roland Tannous
81adc47b6e fix: prefer URL image columns over bare filenames, add value-based fallback
find_image_column now scores candidates by resolvability (PIL > dict > URL > path)
and has a Pass 2 value-based fallback for columns not matching image keywords.
Fixes phiyodr/coco2017 picking file_name (unresolvable) over coco_url (resolvable).
2026-03-10 01:36:19 +00:00
Roland Tannous
d6803de35a fix: detect list-of-strings text columns and pick random element for VLM conversion
Handles datasets like phiyodr/coco2017 where captions is a list of strings.
2026-03-10 01:32:19 +00:00
Roland Tannous
0b8325ab96 feat: add ShareGPT+image VLM format support and improve image column detection
- Detect and convert ShareGPT/ChatML conversations with <image> placeholders
- Add file_name/filename as image column keywords
- Detect image paths and URLs by value (string ending in .jpg/.png/etc)
2026-03-10 01:27:36 +00:00
Roland Tannous
56d02a3b57 fix: use word-boundary matching for image/audio column detection
Substring matching caused false positives like 'pic' in 'topic',
leading to non-deterministic image column selection.
2026-03-10 00:38:02 +00:00
Roland Tannous
d882678fe4 Add AGPL-3.0 SPDX headers to all source files 2026-03-09 20:17:45 +00:00
Manan17
f04c684d8a variable changes and some cleanup 2026-03-03 09:35:11 +00:00
Manan17
ab2ac39017 Changes with audio training 2026-03-01 02:27:45 +00:00
Roland Tannous
18c41c2b08 Simplify dataset check to 2-tier, improve multimodal detection, auto-set trainOnCompletions, recheck dataset on reload 2026-02-19 11:25:54 +00:00
Roland Tannous
62ddcfa019 Refactor [dataset_utils.py](cci:7://file:///home/support/new-ui-prototype/studio/backend/utils/datasets/dataset_utils.py:0:0-0:0) into focused modules 2026-02-03 14:38:02 +00:00