Roland Tannous
a1706c894f
fix: check for http(s) prefix instead of bare string type for URL detection
2026-03-05 06:10:10 +00:00
Roland Tannous
e04b9d53d6
feat: parallel URL image probe with time estimate and progress reporting
...
- Add 200-sample parallel probe using ThreadPoolExecutor + safe_num_proc
to estimate download speed and failure rate before full conversion
- Abort with clear error if >=30% of probe images fail to download
- Show estimated download time in the training overlay modal
- Parallel batch conversion for URL-based datasets (vs sequential for local)
- Add warning field to /check-format response for URL-based image datasets
- Display URL warning in dataset preview dialog (amber banner)
- Thread progress_callback from trainer through format_and_template_dataset
to convert_to_vlm_format for real-time status updates
2026-03-04 23:40:38 +00:00
Roland Tannous
1f03754c95
feat: add tqdm progress bar to VLM conversion and download benchmark test
2026-03-04 23:29:43 +00:00
Roland Tannous
63f723cc36
fix: add early probe to fail fast on datasets with too many broken image URLs
2026-03-04 23:29:43 +00:00
Roland Tannous
9487d17b94
fix: use fsspec for URL image downloads with per-sample error handling
2026-03-04 23:29:43 +00:00
Roland Tannous
bc244aeb23
fix: cast URL image columns to HF Image() type in VLM conversion
2026-03-04 23:29:43 +00:00
Roland Tannous
9333f99dd3
Revert "Add index range dataset slicing to Studio training page"
2026-03-05 03:21:07 +04:00
Roland Tannous
cc11f066b1
feat: add tqdm progress bar to VLM conversion and download benchmark test
2026-03-04 13:30:27 +00:00
Roland Tannous
7804a4db2e
fix: add early probe to fail fast on datasets with too many broken image URLs
2026-03-04 08:05:40 +00:00
Roland Tannous
9cbd3d44a7
fix: use fsspec for URL image downloads with per-sample error handling
2026-03-04 07:50:55 +00:00
Roland Tannous
3c4bf80cc2
fix: cast URL image columns to HF Image() type in VLM conversion
2026-03-04 06:42:37 +00:00
Roland Tannous
834013aae5
Cap dataset.map num_proc on multi-GPU machines to prevent fork deadlocks
2026-02-23 14:25:31 +00:00
Roland Tannous
29b25169c0
Scale dataset num_proc dynamically to cpu_count//3 instead of hardcap 8
2026-02-18 08:38:53 +00:00
Roland Tannous
c0f210bc2a
fix: cap dataset.map() num_proc to 8 to prevent CUDA fork deadlocks
2026-02-17 23:12:45 +00:00
Roland Tannous
47ead076cf
Refactor [dataset_utils.py](cci:7://file:///home/support/new-ui-prototype/studio/backend/utils/datasets/dataset_utils.py:0:0-0:0) into focused modules
2026-02-03 14:38:02 +00:00
