ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
963 commits 391 branches 58 tags 304 MiB
Commit graph

33 commits

Author SHA1 Message Date
Roland Tannous
a0f03d3080 Add AGPL-3.0 SPDX headers to all source files 2026-03-09 20:17:45 +00:00
Manan17
8203637d89 resolved merge conflicts 2026-03-05 07:59:43 +00:00
Roland Tannous
a1706c894f fix: check for http(s) prefix instead of bare string type for URL detection 2026-03-05 06:10:10 +00:00
Roland Tannous
e04b9d53d6 feat: parallel URL image probe with time estimate and progress reporting 
- Add 200-sample parallel probe using ThreadPoolExecutor + safe_num_proc
  to estimate download speed and failure rate before full conversion
- Abort with clear error if >=30% of probe images fail to download
- Show estimated download time in the training overlay modal
- Parallel batch conversion for URL-based datasets (vs sequential for local)
- Add warning field to /check-format response for URL-based image datasets
- Display URL warning in dataset preview dialog (amber banner)
- Thread progress_callback from trainer through format_and_template_dataset
  to convert_to_vlm_format for real-time status updates
 2026-03-04 23:40:38 +00:00
Roland Tannous
1f03754c95 feat: add tqdm progress bar to VLM conversion and download benchmark test 2026-03-04 23:29:43 +00:00
Roland Tannous
63f723cc36 fix: add early probe to fail fast on datasets with too many broken image URLs 2026-03-04 23:29:43 +00:00
Roland Tannous
9487d17b94 fix: use fsspec for URL image downloads with per-sample error handling 2026-03-04 23:29:43 +00:00
Roland Tannous
bc244aeb23 fix: cast URL image columns to HF Image() type in VLM conversion 2026-03-04 23:29:43 +00:00
Roland Tannous
9333f99dd3 Revert "Add index range dataset slicing to Studio training page" 2026-03-05 03:21:07 +04:00
Roland Tannous
cc11f066b1 feat: add tqdm progress bar to VLM conversion and download benchmark test 2026-03-04 13:30:27 +00:00
Roland Tannous
7804a4db2e fix: add early probe to fail fast on datasets with too many broken image URLs 2026-03-04 08:05:40 +00:00
Roland Tannous
9cbd3d44a7 fix: use fsspec for URL image downloads with per-sample error handling 2026-03-04 07:50:55 +00:00
Roland Tannous
3c4bf80cc2 fix: cast URL image columns to HF Image() type in VLM conversion 2026-03-04 06:42:37 +00:00
Manan17
6fd1dd2c0a variable changes and some cleanup 2026-03-03 09:35:11 +00:00
Manan17
90332924de Changes with audio training 2026-03-01 02:27:45 +00:00
Manan17
2c5621dd8c merging with nightly 2026-03-01 02:27:45 +00:00
Manan17
5ce88f9aa1 fixing the chatml None error 2026-02-25 10:23:13 +00:00
Manan17
202dd082b7 My changes for dataset 2026-02-25 08:15:44 +00:00
Manan17
611febd2e3 adding custom mapping according to the chat templates 2026-02-25 07:56:30 +00:00
Roland Tannous
834013aae5 Cap dataset.map num_proc on multi-GPU machines to prevent fork deadlocks 2026-02-23 14:25:31 +00:00
Roland Tannous
62c260a109 Add GLM, Qwen3 MoE, TinyQwen3 MoE, and Ministral 3 VL model defaults and GLM train_on_responses_only mapping 2026-02-23 05:51:43 +00:00
Roland Tannous
11b3029dc6 Simplify dataset check to 2-tier, improve multimodal detection, auto-set trainOnCompletions, recheck dataset on reload 2026-02-19 11:25:54 +00:00
Roland Tannous
29b25169c0 Scale dataset num_proc dynamically to cpu_count//3 instead of hardcap 8 2026-02-18 08:38:53 +00:00
Roland Tannous
c0f210bc2a fix: cap dataset.map() num_proc to 8 to prevent CUDA fork deadlocks 2026-02-17 23:12:45 +00:00
Roland Tannous
ed6d4b2fb6 feat: apply default chat template for base models without tokenizer chat_template 2026-02-16 15:56:06 +00:00
Roland Tannous
4e4fc367b6 fix: auto-detect multimodal datasets in /check-format without requiring is_vlm flag 2026-02-13 17:29:39 +00:00
Roland Tannous
528d2e27f0 authentication refactor - added setup token and token refresh mechanism 2026-02-11 12:09:47 +00:00
Roland Tannous
996b16f9ee Add datasets check-format endpoint 2026-02-03 20:42:25 +00:00
Roland Tannous
4d89dd302c fix custom_format_mapping flow for manual column mapping 2026-02-03 18:42:07 +00:00
Roland Tannous
fa3724b82c Add Flag for Dataset Detection 2026-02-03 18:21:05 +00:00
Roland Tannous
842b2b3186 remove duplicates from dataset_utils.py 2026-02-03 18:03:01 +00:00
Roland Tannous
47ead076cf Refactor [dataset_utils.py](cci:7://file:///home/support/new-ui-prototype/studio/backend/utils/datasets/dataset_utils.py:0:0-0:0) into focused modules 2026-02-03 14:38:02 +00:00
Roland Tannous
75d8dcc824 root studio folder 2026-02-02 09:13:49 +00:00