Roland Tannous
c171573a8f
fix: check for http(s) prefix instead of bare string type for URL detection
2026-03-05 06:10:10 +00:00
Roland Tannous
9ca45826d4
feat: parallel URL image probe with time estimate and progress reporting
...
- Add 200-sample parallel probe using ThreadPoolExecutor + safe_num_proc
to estimate download speed and failure rate before full conversion
- Abort with clear error if >=30% of probe images fail to download
- Show estimated download time in the training overlay modal
- Parallel batch conversion for URL-based datasets (vs sequential for local)
- Add warning field to /check-format response for URL-based image datasets
- Display URL warning in dataset preview dialog (amber banner)
- Thread progress_callback from trainer through format_and_template_dataset
to convert_to_vlm_format for real-time status updates
2026-03-04 23:40:38 +00:00
Roland Tannous
f59eaad212
feat: add tqdm progress bar to VLM conversion and download benchmark test
2026-03-04 23:29:43 +00:00
Roland Tannous
50885a7aa3
fix: add early probe to fail fast on datasets with too many broken image URLs
2026-03-04 23:29:43 +00:00
Roland Tannous
fdc23f4a43
fix: use fsspec for URL image downloads with per-sample error handling
2026-03-04 23:29:43 +00:00
Roland Tannous
929c3e9e1e
fix: cast URL image columns to HF Image() type in VLM conversion
2026-03-04 23:29:43 +00:00
Roland Tannous
91783c0fb2
Revert "Add index range dataset slicing to Studio training page"
2026-03-05 03:21:07 +04:00
Roland Tannous
e4ec16296e
feat: add tqdm progress bar to VLM conversion and download benchmark test
2026-03-04 13:30:27 +00:00
Roland Tannous
5ee9479e37
fix: add early probe to fail fast on datasets with too many broken image URLs
2026-03-04 08:05:40 +00:00
Roland Tannous
722744cf04
fix: use fsspec for URL image downloads with per-sample error handling
2026-03-04 07:50:55 +00:00
Roland Tannous
34fb9ec973
fix: cast URL image columns to HF Image() type in VLM conversion
2026-03-04 06:42:37 +00:00
Roland Tannous
d74174f7f5
Cap dataset.map num_proc on multi-GPU machines to prevent fork deadlocks
2026-02-23 14:25:31 +00:00
Roland Tannous
d69431fa57
Scale dataset num_proc dynamically to cpu_count//3 instead of hardcap 8
2026-02-18 08:38:53 +00:00
Roland Tannous
3d0d1c7020
fix: cap dataset.map() num_proc to 8 to prevent CUDA fork deadlocks
2026-02-17 23:12:45 +00:00
Roland Tannous
62ddcfa019
Refactor [dataset_utils.py](cci:7://file:///home/support/new-ui-prototype/studio/backend/utils/datasets/dataset_utils.py:0:0-0:0) into focused modules
2026-02-03 14:38:02 +00:00
