ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
796 commits 391 branches 58 tags 304 MiB
Commit graph

67 commits

Author SHA1 Message Date
Roland Tannous
9696bd557a fix: exclude bitsandbytes from module purge to prevent duplicate operator registration 2026-03-05 16:40:20 +00:00
Roland Tannous
81b4928e99 Merge nightly into feature/transformers-v5-support 2026-03-05 06:49:44 +00:00
Roland Tannous
c171573a8f fix: check for http(s) prefix instead of bare string type for URL detection 2026-03-05 06:10:10 +00:00
Roland Tannous
9ca45826d4 feat: parallel URL image probe with time estimate and progress reporting 
- Add 200-sample parallel probe using ThreadPoolExecutor + safe_num_proc
  to estimate download speed and failure rate before full conversion
- Abort with clear error if >=30% of probe images fail to download
- Show estimated download time in the training overlay modal
- Parallel batch conversion for URL-based datasets (vs sequential for local)
- Add warning field to /check-format response for URL-based image datasets
- Display URL warning in dataset preview dialog (amber banner)
- Thread progress_callback from trainer through format_and_template_dataset
  to convert_to_vlm_format for real-time status updates
 2026-03-04 23:40:38 +00:00
Roland Tannous
f59eaad212 feat: add tqdm progress bar to VLM conversion and download benchmark test 2026-03-04 23:29:43 +00:00
Roland Tannous
50885a7aa3 fix: add early probe to fail fast on datasets with too many broken image URLs 2026-03-04 23:29:43 +00:00
Roland Tannous
fdc23f4a43 fix: use fsspec for URL image downloads with per-sample error handling 2026-03-04 23:29:43 +00:00
Roland Tannous
929c3e9e1e fix: cast URL image columns to HF Image() type in VLM conversion 2026-03-04 23:29:43 +00:00
Roland Tannous
91783c0fb2 Revert "Add index range dataset slicing to Studio training page" 2026-03-05 03:21:07 +04:00
Roland Tannous
e4ec16296e feat: add tqdm progress bar to VLM conversion and download benchmark test 2026-03-04 13:30:27 +00:00
Roland Tannous
5ee9479e37 fix: add early probe to fail fast on datasets with too many broken image URLs 2026-03-04 08:05:40 +00:00
Roland Tannous
722744cf04 fix: use fsspec for URL image downloads with per-sample error handling 2026-03-04 07:50:55 +00:00
Roland Tannous
34fb9ec973 fix: cast URL image columns to HF Image() type in VLM conversion 2026-03-04 06:42:37 +00:00
Roland Tannous
7e021886c8 Force num_proc=1 on Windows to avoid slow spawn overhead 2026-03-01 13:05:10 +00:00
Roland Tannous
ff93c97024 fix: support mmproj for local vision GGUF models + fix Windows pipe deadlock 2026-03-01 12:58:38 +00:00
Manan17
168957a87a Aggregating sharded models, showing fit/oom for quantizations 2026-02-27 08:23:15 +00:00
Manan17
2fea4cadd3 Passes metadata to get model size 2026-02-27 07:38:55 +00:00
Roland Tannous
ed18f9b9dd Flatten GGUF subdirs in export and fix metadata lookup in scanner 2026-02-26 11:35:04 +04:00
Roland Tannous
c21cf2ffcf Add GGUF tag for exported models in chat page selector 2026-02-25 19:01:47 +04:00
Roland Tannous
01082b84e5 Merge branch 'nightly' into feat/gguf-llama-cpp-inference 2026-02-25 16:06:03 +04:00
Manan17
6e8e70c987 fixing the chatml None error 2026-02-25 10:23:13 +00:00
Manan17
47fc79df6d My changes for dataset 2026-02-25 08:15:44 +00:00
Manan17
60912e45e6 adding custom mapping according to the chat templates 2026-02-25 07:56:30 +00:00
Roland Tannous
ef1cd3ac98 Use llama-server -hf mode, add GGUF variant selector, fix vision detection 
Replace Python-side GGUF download with llama-server's native -hf flag for
HuggingFace repos. Add frontend variant picker so users can choose
quantization (Q4_K_M, Q8_0, BF16, etc.) with file sizes. Fix vision
detection via mmproj files instead of hardcoding is_vision=False.
 2026-02-24 19:03:06 +04:00
Roland Tannous
4e88092452 Preflight llama-server check before downloading remote GGUF files 2026-02-24 18:02:43 +04:00
Roland Tannous
a900eb9ad7 Fix GGUF detection for HuggingFace repo IDs (not just local paths) 2026-02-24 17:49:09 +04:00
Roland Tannous
a40ebb1aab Add GGUF model inference via llama-server backend 2026-02-24 17:40:05 +04:00
Manan17
4be677e45d Adding exported model for chat 2026-02-24 01:17:09 +00:00
Roland Tannous
d74174f7f5 Cap dataset.map num_proc on multi-GPU machines to prevent fork deadlocks 2026-02-23 14:25:31 +00:00
Roland Tannous
f16a7f2d17 Merge nightly into feature/transformers-v5-support 2026-02-23 07:40:28 +00:00
Roland Tannous
dbbcdb4f09 feat: clear unsloth_compiled_cache on startup, shutdown, and between model loads 2026-02-23 07:26:22 +00:00
Roland Tannous
fb1c321ad3 Add GLM, Qwen3 MoE, TinyQwen3 MoE, and Ministral 3 VL model defaults and GLM train_on_responses_only mapping 2026-02-23 05:51:43 +00:00
Roland Tannous
5de6246142 Purge own utils/core modules and use lazy imports so is_vision_model picks up fresh AutoConfig after version switch 2026-02-22 20:04:35 +00:00
Roland Tannous
60997a75eb Install transformers into both site-packages and overlay to fix sub-package resolution during version switch 2026-02-22 19:44:12 +00:00
Roland Tannous
7cde520176 Move transformers overlay to local .venv_overlay/, add huggingface-hub to overlay install 2026-02-22 19:34:46 +00:00
Roland Tannous
15cb9b0f37 Use sys.path overlay to switch transformers versions in-process instead of modifying site-packages 2026-02-22 19:19:12 +00:00
Roland Tannous
0050e78aa3 Fix in-memory transformers version detection and aggressive module purge for 5.1.0/4.57.1 switching 2026-02-22 19:08:18 +00:00
Roland Tannous
1c2653fcc2 aggressive reload_transformers 2026-02-22 18:52:06 +00:00
Roland Tannous
4d06258e93 Auto-switch transformers version (5.1.0/4.57.1) for Ministral-3, GLM-4.7-Flash, Qwen3-30B-A3B models with LoRA adapter resolution 2026-02-22 18:29:40 +00:00
Roland Tannous
18c41c2b08 Simplify dataset check to 2-tier, improve multimodal detection, auto-set trainOnCompletions, recheck dataset on reload 2026-02-19 11:25:54 +00:00
Manan17
56869c63bd Passing use_auth = True and also having different checks which is missed by the is_vision function 2026-02-19 02:55:46 +00:00
Roland Tannous
d69431fa57 Scale dataset num_proc dynamically to cpu_count//3 instead of hardcap 8 2026-02-18 08:38:53 +00:00
Manan17
76cd1dc24c fixing the hangup of training after multiple back to back training processes 2026-02-18 08:18:13 +00:00
Roland Tannous
3d0d1c7020 fix: cap dataset.map() num_proc to 8 to prevent CUDA fork deadlocks 2026-02-17 23:12:45 +00:00
Manan17
c7b7ecab4f Adding metadata for checkpoints 2026-02-16 23:46:17 +00:00
Roland Tannous
b32ad350c5 feat: apply default chat template for base models without tokenizer chat_template 2026-02-16 15:56:06 +00:00
Roland Tannous
a0ebd9183a feat: add live GPU monitor with nvidia-smi polling during training 2026-02-16 11:47:43 +00:00
Roland Tannous
b20d50e8d0 feat: add GET /api/system/hardware endpoint for GPU info and package versions 2026-02-16 10:29:21 +00:00
Roland Tannous
fd49c56481 feat: include training loss per checkpoint in /api/models/checkpoints response 2026-02-16 09:50:28 +00:00
Roland Tannous
f0298edeb8 refactor: move checkpoint scanning to utils/models and /checkpoints endpoint to models router 2026-02-16 09:32:11 +00:00