ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Merge pull request #159 from unslothai/fix/reduce_dataset_num_proc_25_pct

Browse source 
Fix/reduce dataset num proc 25 pct
This commit is contained in:
Roland Tannous 2026-02-19 00:55:02 +04:00 • committed by GitHub
commit 559ba976bc
1 changed files with 1 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

2
studio/backend/core/training/trainer.py
View file

@ -707,7 +707,7 @@ class UnslothTrainer:
"output_dir": output_dir,
"report_to": ["wandb"] if training_args.get('enable_wandb', False) else "none",
"include_num_input_tokens_seen": True, # Enable token counting
"dataset_num_proc": max(1, os.cpu_count() // 3),
"dataset_num_proc": max(1, os.cpu_count() // 4),
}
# Add warmup parameter - use warmup_ratio if provided, otherwise warmup_steps