ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
920 commits 391 branches 58 tags 304 MiB
Commit graph

32 commits

Author SHA1 Message Date
Manan17
9909111982 resolved merge conflicts 2026-03-05 07:59:43 +00:00
Roland Tannous
c171573a8f fix: check for http(s) prefix instead of bare string type for URL detection 2026-03-05 06:10:10 +00:00
Roland Tannous
9ca45826d4 feat: parallel URL image probe with time estimate and progress reporting 
- Add 200-sample parallel probe using ThreadPoolExecutor + safe_num_proc
  to estimate download speed and failure rate before full conversion
- Abort with clear error if >=30% of probe images fail to download
- Show estimated download time in the training overlay modal
- Parallel batch conversion for URL-based datasets (vs sequential for local)
- Add warning field to /check-format response for URL-based image datasets
- Display URL warning in dataset preview dialog (amber banner)
- Thread progress_callback from trainer through format_and_template_dataset
  to convert_to_vlm_format for real-time status updates
 2026-03-04 23:40:38 +00:00
Roland Tannous
f59eaad212 feat: add tqdm progress bar to VLM conversion and download benchmark test 2026-03-04 23:29:43 +00:00
Roland Tannous
50885a7aa3 fix: add early probe to fail fast on datasets with too many broken image URLs 2026-03-04 23:29:43 +00:00
Roland Tannous
fdc23f4a43 fix: use fsspec for URL image downloads with per-sample error handling 2026-03-04 23:29:43 +00:00
Roland Tannous
929c3e9e1e fix: cast URL image columns to HF Image() type in VLM conversion 2026-03-04 23:29:43 +00:00
Roland Tannous
91783c0fb2 Revert "Add index range dataset slicing to Studio training page" 2026-03-05 03:21:07 +04:00
Roland Tannous
e4ec16296e feat: add tqdm progress bar to VLM conversion and download benchmark test 2026-03-04 13:30:27 +00:00
Roland Tannous
5ee9479e37 fix: add early probe to fail fast on datasets with too many broken image URLs 2026-03-04 08:05:40 +00:00
Roland Tannous
722744cf04 fix: use fsspec for URL image downloads with per-sample error handling 2026-03-04 07:50:55 +00:00
Roland Tannous
34fb9ec973 fix: cast URL image columns to HF Image() type in VLM conversion 2026-03-04 06:42:37 +00:00
Manan17
f04c684d8a variable changes and some cleanup 2026-03-03 09:35:11 +00:00
Manan17
ab2ac39017 Changes with audio training 2026-03-01 02:27:45 +00:00
Manan17
ac27edde35 merging with nightly 2026-03-01 02:27:45 +00:00
Manan17
6e8e70c987 fixing the chatml None error 2026-02-25 10:23:13 +00:00
Manan17
47fc79df6d My changes for dataset 2026-02-25 08:15:44 +00:00
Manan17
60912e45e6 adding custom mapping according to the chat templates 2026-02-25 07:56:30 +00:00
Roland Tannous
d74174f7f5 Cap dataset.map num_proc on multi-GPU machines to prevent fork deadlocks 2026-02-23 14:25:31 +00:00
Roland Tannous
fb1c321ad3 Add GLM, Qwen3 MoE, TinyQwen3 MoE, and Ministral 3 VL model defaults and GLM train_on_responses_only mapping 2026-02-23 05:51:43 +00:00
Roland Tannous
18c41c2b08 Simplify dataset check to 2-tier, improve multimodal detection, auto-set trainOnCompletions, recheck dataset on reload 2026-02-19 11:25:54 +00:00
Roland Tannous
d69431fa57 Scale dataset num_proc dynamically to cpu_count//3 instead of hardcap 8 2026-02-18 08:38:53 +00:00
Roland Tannous
3d0d1c7020 fix: cap dataset.map() num_proc to 8 to prevent CUDA fork deadlocks 2026-02-17 23:12:45 +00:00
Roland Tannous
b32ad350c5 feat: apply default chat template for base models without tokenizer chat_template 2026-02-16 15:56:06 +00:00
Roland Tannous
adf1ef5ea5 fix: auto-detect multimodal datasets in /check-format without requiring is_vlm flag 2026-02-13 17:29:39 +00:00
Roland Tannous
01fcb4f713 authentication refactor - added setup token and token refresh mechanism 2026-02-11 12:09:47 +00:00
Roland Tannous
75bb6c08a5 Add datasets check-format endpoint 2026-02-03 20:42:25 +00:00
Roland Tannous
9f9618980d fix custom_format_mapping flow for manual column mapping 2026-02-03 18:42:07 +00:00
Roland Tannous
8bd06e3e35 Add Flag for Dataset Detection 2026-02-03 18:21:05 +00:00
Roland Tannous
f57757231b remove duplicates from dataset_utils.py 2026-02-03 18:03:01 +00:00
Roland Tannous
62ddcfa019 Refactor [dataset_utils.py](cci:7://file:///home/support/new-ui-prototype/studio/backend/utils/datasets/dataset_utils.py:0:0-0:0) into focused modules 2026-02-03 14:38:02 +00:00
Roland Tannous
544d6944d1 root studio folder 2026-02-02 09:13:49 +00:00