ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
838 commits 391 branches 58 tags 304 MiB
Commit graph

61 commits

Author SHA1 Message Date
Roland Tannous
9543fa0d1c fix: scope dataloader_num_workers=0 to Windows + transformers 5.x only 2026-03-07 17:55:59 +00:00
Roland Tannous
dcebfe718a fix: prevent training hang on Windows by adding triton-windows support 2026-03-07 17:53:36 +00:00
Roland Tannous
c882a3d2f7 fix: propagate PYTHONPATH to child subprocesses, revert tokenizer patching 2026-03-07 11:28:24 +00:00
Roland Tannous
42bd976a2f fix: patch TokenizersBackend by model name - Qwen3.5→Qwen2Tokenizer, GLM→PreTrainedTokenizer 2026-03-07 10:29:59 +00:00
Roland Tannous
44e9b838ae fix: patch Qwen3.5 broken tokenizer_class TokenizersBackend across all backends 2026-03-07 09:43:25 +00:00
Roland Tannous
f101befca7 fix: bump transformers 5.x pin from 5.1.0 to 5.2.0 for Qwen3.5 support 2026-03-07 09:10:09 +00:00
Roland Tannous
2c4c598832 fix: fail fast if runtime pip install of transformers 5.x fails 2026-03-07 08:40:25 +00:00
Roland Tannous
082a6f876e fix: join prior pump thread before starting new training job 2026-03-07 08:37:03 +00:00
Roland Tannous
7e59440029 fix: pin huggingface_hub==1.3.0 in .venv_t5 (satisfies transformers 5.x) 2026-03-06 06:19:28 +00:00
Roland Tannous
661ac4be96 feat: subprocess-based export, pin huggingface_hub==0.36.0 2026-03-06 06:03:09 +00:00
Roland Tannous
f90af41c5f feat: subprocess-based training for transformers version switching 2026-03-05 15:40:32 +00:00
Roland Tannous
f57664e268 Merge nightly into feature/transformers-v5-support 2026-03-05 06:49:44 +00:00
Roland Tannous
e04b9d53d6 feat: parallel URL image probe with time estimate and progress reporting 
- Add 200-sample parallel probe using ThreadPoolExecutor + safe_num_proc
  to estimate download speed and failure rate before full conversion
- Abort with clear error if >=30% of probe images fail to download
- Show estimated download time in the training overlay modal
- Parallel batch conversion for URL-based datasets (vs sequential for local)
- Add warning field to /check-format response for URL-based image datasets
- Display URL warning in dataset preview dialog (amber banner)
- Thread progress_callback from trainer through format_and_template_dataset
  to convert_to_vlm_format for real-time status updates
 2026-03-04 23:40:38 +00:00
Roland Tannous
b376c54c21 fix: abort training pipeline on dataset conversion failure 2026-03-04 23:29:43 +00:00
Roland Tannous
64889cd5fc feat: add index range dataset slicing to studio training page 
Add Start/End index inputs under Advanced in the dataset card,
allowing users to slice a dataset by row range before training.
Wired end-to-end: frontend store, API payload, backend Pydantic
model, and trainer dataset loading (inclusive on both ends).
 2026-03-04 23:24:09 +00:00
Roland Tannous
9333f99dd3 Revert "Add index range dataset slicing to Studio training page" 2026-03-05 03:21:07 +04:00
Roland Tannous
02b17ec6d9 feat: add index range dataset slicing to studio training page 
Add Start/End index inputs under Advanced in the dataset card,
allowing users to slice a dataset by row range before training.
Wired end-to-end: frontend store, API payload, backend Pydantic
model, and trainer dataset loading (inclusive on both ends).
 2026-03-04 21:48:40 +00:00
Roland Tannous
c6d82a6fad fix: abort training pipeline on dataset conversion failure 2026-03-04 06:42:48 +00:00
Roland Tannous
efaa0bacfb Merge branch 'nightly' into feat/gguf-llama-cpp-inference 2026-02-25 16:06:03 +04:00
Roland Tannous
d9434fee4a fix: use raw github URL for vision.py patch + add VLM processor diagnostic logging 2026-02-25 10:29:05 +00:00
Roland Tannous
a661784d41 Merge pull request #232 from unslothai/fix/disable-eval-by-default 
# fix/disable eval by default
 2026-02-24 13:35:11 +04:00
Roland Tannous
a15082acfe skip eval split and HF split detection when eval_steps is disabled 2026-02-24 09:26:54 +00:00
Leo Borcherding
86388a0242 fix: disable eval by default, set eval_steps to 0.0 
- Changed default eval_steps from 0.01 to 0.0 across backend and frontend
- Fixed UI to allow eval_steps=0 (removed min=0.001 constraint)
- Added conditional eval logic with helpful console messages
- Updated tooltip to explain how to disable evaluation
- Tested: confirmed eval disabled by default with eval_steps=0.0
 2026-02-23 13:07:47 -06:00
Roland Tannous
834013aae5 Cap dataset.map num_proc on multi-GPU machines to prevent fork deadlocks 2026-02-23 14:25:31 +00:00
Roland Tannous
fbc934c231 Merge branch 'nightly' into feature/transformers-v5-support 2026-02-23 13:32:51 +00:00
Roland Tannous
d94f842158 fix: error on >30% sample drop after train_on_responses_only instead of silent DataLoader crash 2026-02-23 12:21:06 +00:00
Roland Tannous
036d85c9e4 Merge nightly into feature/transformers-v5-support 2026-02-23 07:40:28 +00:00
Roland Tannous
198433363a feat: clear unsloth_compiled_cache on startup, shutdown, and between model loads 2026-02-23 07:26:22 +00:00
Roland Tannous
778762eb28 Patch adapter_config.json with unsloth_training_method and auto-detect load_in_4bit for LoRA inference 2026-02-22 20:27:52 +00:00
Roland Tannous
e666442b6e fix: pass full Processor as processing_class for VLM SFTTrainer 2026-02-22 14:11:12 +00:00
Manan17
bd0cee8c15 Setting it to total cpu_count // 4 2026-02-20 06:32:01 +00:00
Roland Tannous
c876b38780 reduce dataset_num_proc to 1/4 of cpu_count 2026-02-18 20:53:32 +00:00
Roland Tannous
23214c41c0 fix: unwrap ProcessorMixin to raw tokenizer for text-only SFTTrainer on VLM-architecture models 2026-02-18 19:13:20 +00:00
Roland Tannous
9840864662 Merge remote-tracking branch 'origin/nightly' into fix/dataset-mapping-vlm-text-datasets 2026-02-18 18:11:45 +04:00
Manan17
949e57c334 fixing the hangup of training after multiple back to back training processes 2026-02-18 08:18:13 +00:00
Manan17
db0fa1a270 Dividing the total cpu_count // 3 2026-02-18 07:59:57 +00:00
Manan17
c832c903b4 fix the linear path on backend 2026-02-18 07:08:32 +00:00
Roland Tannous
9d737559a5 debug statements 2026-02-18 00:37:09 +00:00
Roland Tannous
d94a1e0289 fix: normalize target_modules [all-linear] list to string for Unsloth/PEFT compatibility 2026-02-18 00:32:21 +00:00
Roland Tannous
f0613f5d07 fix: defensively rename VLM chat column to match model's forward() signature 2026-02-17 23:49:13 +00:00
Shine1i
b31461790f feat: enhance training stop and reset flow with detailed checks 2026-02-17 23:32:22 +01:00
Shine1i
f47c424be3 feat: integrate gradient norm tracking in training runtime and metrics 
- Enhanced chart logic to filter and visualize finite gradient norm values.
 2026-02-17 18:26:59 +01:00
Roland Tannous
108ec254cb Merge branch 'nightly' into feature/eval-split-auto-detection 2026-02-17 01:11:30 +04:00
Roland Tannous
18879a521b feat: auto-detect model+dataset compatibility to select VLM vs LLM training path 2026-02-16 19:18:49 +00:00
Roland Tannous
3b117189c5 feat: add eval_enabled flag and format-first-then-split for eval dataset 2026-02-16 14:13:55 +00:00
Roland Tannous
5962bec41a feat: add eval split auto-detection, eval_steps hyperparam, and eval_loss chart integration 2026-02-16 13:51:10 +00:00
Roland Tannous
90c3561adb feat: add eval split auto-detection, eval_steps hyperparam, and eval_loss chart integration 2026-02-16 13:38:54 +00:00
Roland Tannous
be584ccfa7 Merge pull request #97 from unslothai/fix/progress-metics 
Resolved the progress metrics
 2026-02-16 11:55:01 +04:00
sshah229
5bf2472af9 modified the num_tokens logic 2026-02-16 00:38:40 -07:00
Roland Tannous
38cb5c9496 feat: thread dataset subset/split params from API routes through to load_dataset calls 2026-02-16 03:56:22 +00:00