ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
783 commits 391 branches 58 tags 304 MiB
Commit graph

65 commits

Author SHA1 Message Date
Manan17
8203637d89 resolved merge conflicts 2026-03-05 07:59:43 +00:00
Roland Tannous
a1706c894f fix: check for http(s) prefix instead of bare string type for URL detection 2026-03-05 06:10:10 +00:00
Roland Tannous
e04b9d53d6 feat: parallel URL image probe with time estimate and progress reporting 
- Add 200-sample parallel probe using ThreadPoolExecutor + safe_num_proc
  to estimate download speed and failure rate before full conversion
- Abort with clear error if >=30% of probe images fail to download
- Show estimated download time in the training overlay modal
- Parallel batch conversion for URL-based datasets (vs sequential for local)
- Add warning field to /check-format response for URL-based image datasets
- Display URL warning in dataset preview dialog (amber banner)
- Thread progress_callback from trainer through format_and_template_dataset
  to convert_to_vlm_format for real-time status updates
 2026-03-04 23:40:38 +00:00
Roland Tannous
1f03754c95 feat: add tqdm progress bar to VLM conversion and download benchmark test 2026-03-04 23:29:43 +00:00
Roland Tannous
63f723cc36 fix: add early probe to fail fast on datasets with too many broken image URLs 2026-03-04 23:29:43 +00:00
Roland Tannous
9487d17b94 fix: use fsspec for URL image downloads with per-sample error handling 2026-03-04 23:29:43 +00:00
Roland Tannous
bc244aeb23 fix: cast URL image columns to HF Image() type in VLM conversion 2026-03-04 23:29:43 +00:00
Roland Tannous
9333f99dd3 Revert "Add index range dataset slicing to Studio training page" 2026-03-05 03:21:07 +04:00
Roland Tannous
cc11f066b1 feat: add tqdm progress bar to VLM conversion and download benchmark test 2026-03-04 13:30:27 +00:00
Manan17
ee23628b0c dynamic detection of audio models and fixing autoencoder issues 2026-03-04 10:44:44 +00:00
Roland Tannous
7804a4db2e fix: add early probe to fail fast on datasets with too many broken image URLs 2026-03-04 08:05:40 +00:00
Roland Tannous
9cbd3d44a7 fix: use fsspec for URL image downloads with per-sample error handling 2026-03-04 07:50:55 +00:00
Roland Tannous
3c4bf80cc2 fix: cast URL image columns to HF Image() type in VLM conversion 2026-03-04 06:42:37 +00:00
Manan17
6fd1dd2c0a variable changes and some cleanup 2026-03-03 09:35:11 +00:00
Roland Tannous
da00f5ed1d Merge branch 'nightly' into feature/support-for-audio-models 2026-03-02 15:55:25 +04:00
Roland Tannous
c297d7aa84 Force num_proc=1 on Windows to avoid slow spawn overhead 2026-03-01 13:05:10 +00:00
Roland Tannous
986bef4f99 fix: support mmproj for local vision GGUF models + fix Windows pipe deadlock 2026-03-01 12:58:38 +00:00
Manan17
8cdeb006b6 code cleanup 2026-03-01 08:04:38 +00:00
Manan17
9e89f31bc7 revamping up the code and adding inference 2026-03-01 02:30:31 +00:00
Manan17
90332924de Changes with audio training 2026-03-01 02:27:45 +00:00
Manan17
2c5621dd8c merging with nightly 2026-03-01 02:27:45 +00:00
Manan17
b4311cca82 Aggregating sharded models, showing fit/oom for quantizations 2026-02-27 08:23:15 +00:00
Manan17
4bd5213c05 Passes metadata to get model size 2026-02-27 07:38:55 +00:00
Roland Tannous
c06adc3878 Flatten GGUF subdirs in export and fix metadata lookup in scanner 2026-02-26 11:35:04 +04:00
Roland Tannous
b91b979bf8 Add GGUF tag for exported models in chat page selector 2026-02-25 19:01:47 +04:00
Roland Tannous
efaa0bacfb Merge branch 'nightly' into feat/gguf-llama-cpp-inference 2026-02-25 16:06:03 +04:00
Manan17
5ce88f9aa1 fixing the chatml None error 2026-02-25 10:23:13 +00:00
Manan17
202dd082b7 My changes for dataset 2026-02-25 08:15:44 +00:00
Manan17
611febd2e3 adding custom mapping according to the chat templates 2026-02-25 07:56:30 +00:00
Roland Tannous
3ee4f1359a Use llama-server -hf mode, add GGUF variant selector, fix vision detection 
Replace Python-side GGUF download with llama-server's native -hf flag for
HuggingFace repos. Add frontend variant picker so users can choose
quantization (Q4_K_M, Q8_0, BF16, etc.) with file sizes. Fix vision
detection via mmproj files instead of hardcoding is_vision=False.
 2026-02-24 19:03:06 +04:00
Roland Tannous
4a82e704aa Preflight llama-server check before downloading remote GGUF files 2026-02-24 18:02:43 +04:00
Roland Tannous
c635d4f49c Fix GGUF detection for HuggingFace repo IDs (not just local paths) 2026-02-24 17:49:09 +04:00
Roland Tannous
2f985ccbb5 Add GGUF model inference via llama-server backend 2026-02-24 17:40:05 +04:00
Manan17
1071c137f4 Adding exported model for chat 2026-02-24 01:17:09 +00:00
Roland Tannous
834013aae5 Cap dataset.map num_proc on multi-GPU machines to prevent fork deadlocks 2026-02-23 14:25:31 +00:00
Roland Tannous
198433363a feat: clear unsloth_compiled_cache on startup, shutdown, and between model loads 2026-02-23 07:26:22 +00:00
Roland Tannous
62c260a109 Add GLM, Qwen3 MoE, TinyQwen3 MoE, and Ministral 3 VL model defaults and GLM train_on_responses_only mapping 2026-02-23 05:51:43 +00:00
Roland Tannous
11b3029dc6 Simplify dataset check to 2-tier, improve multimodal detection, auto-set trainOnCompletions, recheck dataset on reload 2026-02-19 11:25:54 +00:00
Manan17
2755cf922d Passing use_auth = True and also having different checks which is missed by the is_vision function 2026-02-19 02:55:46 +00:00
Roland Tannous
29b25169c0 Scale dataset num_proc dynamically to cpu_count//3 instead of hardcap 8 2026-02-18 08:38:53 +00:00
Manan17
949e57c334 fixing the hangup of training after multiple back to back training processes 2026-02-18 08:18:13 +00:00
Roland Tannous
c0f210bc2a fix: cap dataset.map() num_proc to 8 to prevent CUDA fork deadlocks 2026-02-17 23:12:45 +00:00
Manan17
8f1db03c15 Adding metadata for checkpoints 2026-02-16 23:46:17 +00:00
Roland Tannous
ed6d4b2fb6 feat: apply default chat template for base models without tokenizer chat_template 2026-02-16 15:56:06 +00:00
Roland Tannous
d49506b7b1 feat: add live GPU monitor with nvidia-smi polling during training 2026-02-16 11:47:43 +00:00
Roland Tannous
f3aa353540 feat: add GET /api/system/hardware endpoint for GPU info and package versions 2026-02-16 10:29:21 +00:00
Roland Tannous
1109839d2c feat: include training loss per checkpoint in /api/models/checkpoints response 2026-02-16 09:50:28 +00:00
Roland Tannous
8a239dc83e refactor: move checkpoint scanning to utils/models and /checkpoints endpoint to models router 2026-02-16 09:32:11 +00:00
Shine1i
78c7b6d7ba fix lora: outputs path local 2026-02-15 16:58:24 +01:00
sshah229
2483b98985 added the inference fetching from model mappers 2026-02-15 02:48:53 -07:00