ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
3,253 commits 391 branches 58 tags 304 MiB
Commit graph

3,253 commits

This branch
This branch
All branches
Author SHA1 Message Date
Kaitao Yang
dcd47ab763 reduce code duplication by _offload_frozen_module_for_training 2026-01-09 06:07:38 -08:00
Daniel Han
f3f9d14887
 		Merge pull request #3857 from Datta0/modelscope_stats 
[ModelScope] Disable stats when modelscope is being used
 2026-01-06 02:56:55 -08:00
pre-commit-ci[bot]
14204ea65a [pre-commit.ci] auto fixes from pre-commit.com hooks 
for more information, see https://pre-commit.ci
 2026-01-06 10:00:17 +00:00
Datta Nimmaturi
6b94be00f4
 		Check env var explicitly 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-01-06 15:30:06 +05:30
Datta Nimmaturi
8ce5813767 Disable stats when modelscope is being used 2026-01-06 09:53:20 +00:00
Daniel Han
e731f0b551 Versioning 2026-01-05 07:37:08 -08:00
Daniel Han
0b862a7a04
 		Merge pull request #3843 from unslothai/fix-grpo-version-compat 
Unify Version usage and fix TRL version handling
 2026-01-05 06:07:41 -08:00
Daniel Han
def6506c96
 		Merge pull request #3851 from unslothai/grpo-fix-on-pr3754 
GRPO: restore model mode after generate (stacked on #3754)
 2026-01-05 06:05:24 -08:00
danielhanchen
dafd25f13b Merge main into grpo-fix-on-pr3754 2026-01-05 14:02:18 +00:00
danielhanchen
ef533cddf7 Revert rl_replacements GRPO edits 2026-01-05 13:55:08 +00:00
danielhanchen
6918e2d31a Fix GRPO training state restoration 2026-01-05 13:50:48 +00:00
pre-commit-ci[bot]
242f0996b4 [pre-commit.ci] auto fixes from pre-commit.com hooks 
for more information, see https://pre-commit.ci
 2026-01-05 13:39:16 +00:00
danielhanchen
d83a5f4e7c Restore TRL version fallback in rl.py 2026-01-05 13:39:03 +00:00
Daniel Han
de43cf3430 Merge branch 'main' into fix-grpo-version-compat 2026-01-05 05:31:42 -08:00
danielhanchen
d74f5a562f Drop rl.py GRPO changes from this branch 2026-01-05 13:29:58 +00:00
Daniel Han
964fdf2563
 		Merge pull request #3849 from unslothai/fix-pdl-use-vllm-version-check 
Replace GitHub API check with vLLM version check for PDL fix
 2026-01-05 05:22:16 -08:00
pre-commit-ci[bot]
5951247b54 [pre-commit.ci] auto fixes from pre-commit.com hooks 
for more information, see https://pre-commit.ci
 2026-01-05 13:19:44 +00:00
Daniel Han
a85ad30f65 Address review feedback: add constant and debug logging 2026-01-05 13:19:37 +00:00
Daniel Han
65f95f579b Replace GitHub API check with vLLM version check for PDL fix 
The GitHub issue check had issues:
1. Network latency on import
2. Issue being closed does not mean the fix is in the installed vLLM version

Now skip the PDL workaround if vLLM version > 0.13.2, which is when
the upstream fix is expected to be included.
 2026-01-05 13:15:17 +00:00
Daniel Han
06c00b06b4
 		Merge pull request #3836 from ykaitao/remove_unused_variable_BlockDiagonalCausalMask 
remove unused variable BlockDiagonalCausalMask
 2026-01-05 04:42:25 -08:00
Daniel Han
3e193d08c3
 		Merge pull request #3842 from unslothai/fix-vllm-chat-template-sync 
Sync chat_template from tokenizer to vLLM
 2026-01-05 04:38:39 -08:00
Daniel Han
fcf43c6c29
 		Merge pull request #3841 from unslothai/fix-vllm-pdl-blackwell 
Fix vLLM PDL bug on Blackwell GPUs (B200/B100)
 2026-01-05 04:37:58 -08:00
Daniel Han
5e091e5ac5 Keep PDL module check but remove unnecessary env var setting 
The check skips the GitHub API call for old vLLM versions.
No need to set TRITON_DISABLE_PDL for versions without PDL support.
 2026-01-05 12:34:32 +00:00
Daniel Han
7090393ae2 Remove unnecessary PDL module existence check 
Old vLLM versions without PDL modules don't need the fix.
The patching code already handles missing modules gracefully.
 2026-01-05 12:32:16 +00:00
Daniel Han
0a07009eb5 Add None check for vLLM tokenizer 
- Check _vllm_tok is not None before accessing attributes
- Use getattr for safer chat_template access
 2026-01-05 10:02:11 +00:00
pre-commit-ci[bot]
9ce417b445 [pre-commit.ci] auto fixes from pre-commit.com hooks 
for more information, see https://pre-commit.ci
 2026-01-05 07:03:35 +00:00
danielhanchen
c81d9605e2 Improve TRL compatibility and GRPO state restore 2026-01-05 07:02:36 +00:00
Daniel Han
f469e76c65 Fix PDL patch: target utils.py source module and clear lru_cache 
- Patch vllm.lora.ops.triton_ops.utils directly where supports_pdl is defined
- Clear lru_cache before patching to prevent stale cached results
- Add fused_moe_lora_op to consumer modules list
- Use *args, **kwargs in fake function for compatibility
 2026-01-05 06:53:42 +00:00
Daniel Han
a520029636 Combine nested if statements for clarity 2026-01-05 05:25:53 +00:00
pre-commit-ci[bot]
c009a4b966 [pre-commit.ci] auto fixes from pre-commit.com hooks 
for more information, see https://pre-commit.ci
 2026-01-05 05:24:59 +00:00
Daniel Han
b0c3894f51 Address review feedback: refactor and scan all GPUs 
- Add _spec_exists helper function to reduce duplication
- Scan all GPUs for SM100 instead of just device 0
- Use loop for module patching to improve maintainability
 2026-01-05 05:24:52 +00:00
Daniel Han
abaee73d74 Add tokenizer fallback for chat_template sync 2026-01-05 05:10:24 +00:00
Daniel Han
3e1ceff307 Sync chat_template from tokenizer to vLLM 
When using base models with custom chat templates applied after loading,
vLLM's internal tokenizer may not have the chat_template set. This causes
issues during RL training with vLLM inference.

This fix syncs the chat_template from the processing_class (the tokenizer
you loaded and configured) to vLLM's internal tokenizer during trainer
initialization, but only if vLLM's tokenizer does not already have one set.
 2026-01-05 05:03:56 +00:00
pre-commit-ci[bot]
a031e7ec4f [pre-commit.ci] auto fixes from pre-commit.com hooks 
for more information, see https://pre-commit.ci
 2026-01-05 05:03:29 +00:00
Daniel Han
cf64ea1daf Fix vLLM PDL bug on Blackwell GPUs (B200/B100) 
vLLM's LoRA Triton kernels use tl.extra.cuda.gdc_wait() for PDL
optimization on SM90+ GPUs. This fails on SM100 (Blackwell) during
CUDA graph capture because Triton's pipeliner cannot handle gdc_wait
in complex kernels.

This fix:
- Detects SM100 GPUs and applies the workaround automatically
- Sets TRITON_DISABLE_PDL=1 environment variable
- Monkey-patches supports_pdl to return False in lora_expand_op and
  lora_shrink_op
- Checks GitHub issue #30872 status (with 3s timeout) to auto-disable
  the workaround once the upstream fix is merged
- Includes quick internet connectivity check (0.5s) to avoid delays
  when offline

Fixes the error:
'tt.elementwise_inline_asm' op pipeliner doesn't know how to predicate this op
LLVM ERROR: Fatal pipeliner error

See: https://github.com/vllm-project/vllm/issues/30872
 2026-01-05 05:02:53 +00:00
Kaitao Yang
f58696c5ad remove unused variable BlockDiagonalCausalMask 2026-01-04 09:21:44 -08:00
Daniel Han
85bfdaf7ab Versioning 2026-01-04 06:12:44 -08:00
Daniel Han
50dafa6794
 		Merge pull request #3835 from unslothai/quant-config-respect 
Respect user quantization_config
 2026-01-04 05:43:20 -08:00
Daniel Han
1d18839c8b
 		Merge pull request #3834 from unslothai/rl-fixes 
rl.py fixes: buffer reset, safer attribute access, typo fix
 2026-01-04 05:25:45 -08:00
danielhanchen
f01872b61d Keep 4bit flag for fast_inference 2026-01-04 13:18:15 +00:00
danielhanchen
f1b320ea34 Handle dict quantization_config flags 2026-01-04 13:14:03 +00:00
danielhanchen
eddc4a8754 Respect user quantization_config 2026-01-04 13:03:06 +00:00
danielhanchen
0dc55e9504 Fix psutil.cpu_count() potentially returning None in save.py 2026-01-04 12:58:45 +00:00
danielhanchen
eba90fe346 Handle older unsloth-zoo without reset_unsloth_gradient_checkpointing_buffers 2026-01-04 12:57:10 +00:00
danielhanchen
59edd7fa90 rl.py fixes: buffer reset, safer attribute access, typo fix 
1. Auto-reset gradient checkpointing buffers after trainer.train()
   - Import and call reset_unsloth_gradient_checkpointing_buffers() in
     prepare_for_training_mode wrapper to free memory after training
     while keeping buffers ready for subsequent runs

2. Replace eval/exec with safer getattr/setattr
   - eval(f"trl.trainer.{trainer}") -> getattr(trl.trainer, trainer)
   - exec(f"...{unwrap} = ...") -> setattr(current_trainer, unwrap, ...)
   - exec(f"Trainer.prediction_step=...") -> direct assignment

3. Fix psutil.cpu_count() potentially returning None
   - Change psutil.cpu_count()+4 to (psutil.cpu_count() or 1)+4
   - Prevents TypeError on systems where cpu_count() returns None

4. Fix typo: oriignal_is_vlm_text -> original_is_vlm_text
 2026-01-04 12:21:39 +00:00
Daniel Han
c6f8d00329
 		Merge pull request #3832 from ykaitao/ktyang_remove_redundant_code_has_block 
remove redundant code of has_block
 2026-01-03 23:18:27 -08:00
Kaitao Yang
31d1c0b928 remove redundant code of has_block 2026-01-03 22:38:37 -08:00
Daniel Han
fad15250b2
 		Merge pull request #3822 from Fizza-Mukhtar/fix/llama-build-curl 
Make llama.cpp CURL dependency optional when building from source
 2026-01-03 22:12:50 -08:00
pre-commit-ci[bot]
030339de39 [pre-commit.ci] auto fixes from pre-commit.com hooks 
for more information, see https://pre-commit.ci
 2026-01-02 16:58:04 +00:00
Fizza-Mukhtar
43fb35e061 Make llama.cpp CURL support optional during CMake builds 2026-01-02 08:55:58 -08:00