ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
794 commits 391 branches 58 tags 304 MiB
Commit graph

67 commits

Author SHA1 Message Date
Roland Tannous
794b8fe866 fix: exclude bitsandbytes from module purge to prevent duplicate operator registration 2026-03-05 16:40:20 +00:00
Roland Tannous
f57664e268 Merge nightly into feature/transformers-v5-support 2026-03-05 06:49:44 +00:00
Roland Tannous
a1706c894f fix: check for http(s) prefix instead of bare string type for URL detection 2026-03-05 06:10:10 +00:00
Roland Tannous
e04b9d53d6 feat: parallel URL image probe with time estimate and progress reporting 
- Add 200-sample parallel probe using ThreadPoolExecutor + safe_num_proc
  to estimate download speed and failure rate before full conversion
- Abort with clear error if >=30% of probe images fail to download
- Show estimated download time in the training overlay modal
- Parallel batch conversion for URL-based datasets (vs sequential for local)
- Add warning field to /check-format response for URL-based image datasets
- Display URL warning in dataset preview dialog (amber banner)
- Thread progress_callback from trainer through format_and_template_dataset
  to convert_to_vlm_format for real-time status updates
 2026-03-04 23:40:38 +00:00
Roland Tannous
1f03754c95 feat: add tqdm progress bar to VLM conversion and download benchmark test 2026-03-04 23:29:43 +00:00
Roland Tannous
63f723cc36 fix: add early probe to fail fast on datasets with too many broken image URLs 2026-03-04 23:29:43 +00:00
Roland Tannous
9487d17b94 fix: use fsspec for URL image downloads with per-sample error handling 2026-03-04 23:29:43 +00:00
Roland Tannous
bc244aeb23 fix: cast URL image columns to HF Image() type in VLM conversion 2026-03-04 23:29:43 +00:00
Roland Tannous
9333f99dd3 Revert "Add index range dataset slicing to Studio training page" 2026-03-05 03:21:07 +04:00
Roland Tannous
cc11f066b1 feat: add tqdm progress bar to VLM conversion and download benchmark test 2026-03-04 13:30:27 +00:00
Roland Tannous
7804a4db2e fix: add early probe to fail fast on datasets with too many broken image URLs 2026-03-04 08:05:40 +00:00
Roland Tannous
9cbd3d44a7 fix: use fsspec for URL image downloads with per-sample error handling 2026-03-04 07:50:55 +00:00
Roland Tannous
3c4bf80cc2 fix: cast URL image columns to HF Image() type in VLM conversion 2026-03-04 06:42:37 +00:00
Roland Tannous
c297d7aa84 Force num_proc=1 on Windows to avoid slow spawn overhead 2026-03-01 13:05:10 +00:00
Roland Tannous
986bef4f99 fix: support mmproj for local vision GGUF models + fix Windows pipe deadlock 2026-03-01 12:58:38 +00:00
Manan17
b4311cca82 Aggregating sharded models, showing fit/oom for quantizations 2026-02-27 08:23:15 +00:00
Manan17
4bd5213c05 Passes metadata to get model size 2026-02-27 07:38:55 +00:00
Roland Tannous
c06adc3878 Flatten GGUF subdirs in export and fix metadata lookup in scanner 2026-02-26 11:35:04 +04:00
Roland Tannous
b91b979bf8 Add GGUF tag for exported models in chat page selector 2026-02-25 19:01:47 +04:00
Roland Tannous
efaa0bacfb Merge branch 'nightly' into feat/gguf-llama-cpp-inference 2026-02-25 16:06:03 +04:00
Manan17
5ce88f9aa1 fixing the chatml None error 2026-02-25 10:23:13 +00:00
Manan17
202dd082b7 My changes for dataset 2026-02-25 08:15:44 +00:00
Manan17
611febd2e3 adding custom mapping according to the chat templates 2026-02-25 07:56:30 +00:00
Roland Tannous
3ee4f1359a Use llama-server -hf mode, add GGUF variant selector, fix vision detection 
Replace Python-side GGUF download with llama-server's native -hf flag for
HuggingFace repos. Add frontend variant picker so users can choose
quantization (Q4_K_M, Q8_0, BF16, etc.) with file sizes. Fix vision
detection via mmproj files instead of hardcoding is_vision=False.
 2026-02-24 19:03:06 +04:00
Roland Tannous
4a82e704aa Preflight llama-server check before downloading remote GGUF files 2026-02-24 18:02:43 +04:00
Roland Tannous
c635d4f49c Fix GGUF detection for HuggingFace repo IDs (not just local paths) 2026-02-24 17:49:09 +04:00
Roland Tannous
2f985ccbb5 Add GGUF model inference via llama-server backend 2026-02-24 17:40:05 +04:00
Manan17
1071c137f4 Adding exported model for chat 2026-02-24 01:17:09 +00:00
Roland Tannous
834013aae5 Cap dataset.map num_proc on multi-GPU machines to prevent fork deadlocks 2026-02-23 14:25:31 +00:00
Roland Tannous
036d85c9e4 Merge nightly into feature/transformers-v5-support 2026-02-23 07:40:28 +00:00
Roland Tannous
198433363a feat: clear unsloth_compiled_cache on startup, shutdown, and between model loads 2026-02-23 07:26:22 +00:00
Roland Tannous
62c260a109 Add GLM, Qwen3 MoE, TinyQwen3 MoE, and Ministral 3 VL model defaults and GLM train_on_responses_only mapping 2026-02-23 05:51:43 +00:00
Roland Tannous
4d0f7c525b Purge own utils/core modules and use lazy imports so is_vision_model picks up fresh AutoConfig after version switch 2026-02-22 20:04:35 +00:00
Roland Tannous
332e071b6c Install transformers into both site-packages and overlay to fix sub-package resolution during version switch 2026-02-22 19:44:12 +00:00
Roland Tannous
7d2a8be0c0 Move transformers overlay to local .venv_overlay/, add huggingface-hub to overlay install 2026-02-22 19:34:46 +00:00
Roland Tannous
4d9ab493f7 Use sys.path overlay to switch transformers versions in-process instead of modifying site-packages 2026-02-22 19:19:12 +00:00
Roland Tannous
66aaaa9af7 Fix in-memory transformers version detection and aggressive module purge for 5.1.0/4.57.1 switching 2026-02-22 19:08:18 +00:00
Roland Tannous
2f212f95a2 aggressive reload_transformers 2026-02-22 18:52:06 +00:00
Roland Tannous
f5b30448e8 Auto-switch transformers version (5.1.0/4.57.1) for Ministral-3, GLM-4.7-Flash, Qwen3-30B-A3B models with LoRA adapter resolution 2026-02-22 18:29:40 +00:00
Roland Tannous
11b3029dc6 Simplify dataset check to 2-tier, improve multimodal detection, auto-set trainOnCompletions, recheck dataset on reload 2026-02-19 11:25:54 +00:00
Manan17
2755cf922d Passing use_auth = True and also having different checks which is missed by the is_vision function 2026-02-19 02:55:46 +00:00
Roland Tannous
29b25169c0 Scale dataset num_proc dynamically to cpu_count//3 instead of hardcap 8 2026-02-18 08:38:53 +00:00
Manan17
949e57c334 fixing the hangup of training after multiple back to back training processes 2026-02-18 08:18:13 +00:00
Roland Tannous
c0f210bc2a fix: cap dataset.map() num_proc to 8 to prevent CUDA fork deadlocks 2026-02-17 23:12:45 +00:00
Manan17
8f1db03c15 Adding metadata for checkpoints 2026-02-16 23:46:17 +00:00
Roland Tannous
ed6d4b2fb6 feat: apply default chat template for base models without tokenizer chat_template 2026-02-16 15:56:06 +00:00
Roland Tannous
d49506b7b1 feat: add live GPU monitor with nvidia-smi polling during training 2026-02-16 11:47:43 +00:00
Roland Tannous
f3aa353540 feat: add GET /api/system/hardware endpoint for GPU info and package versions 2026-02-16 10:29:21 +00:00
Roland Tannous
1109839d2c feat: include training loss per checkpoint in /api/models/checkpoints response 2026-02-16 09:50:28 +00:00
Roland Tannous
8a239dc83e refactor: move checkpoint scanning to utils/models and /checkpoints endpoint to models router 2026-02-16 09:32:11 +00:00