ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
3,248 commits 391 branches 58 tags 304 MiB
Commit graph

3,248 commits

This branch
This branch
All branches
Author SHA1 Message Date
Daniel Han
e7fe25ee43 Versioning 2026-01-05 07:37:08 -08:00
Daniel Han
a50f0e1a75 Merge pull request #3843 from unslothai/fix-grpo-version-compat 
Unify Version usage and fix TRL version handling
 2026-01-05 06:07:41 -08:00
Daniel Han
5f1a62bc0b Merge pull request #3851 from unslothai/grpo-fix-on-pr3754 
GRPO: restore model mode after generate (stacked on #3754)
 2026-01-05 06:05:24 -08:00
danielhanchen
4a45983bf6 Merge main into grpo-fix-on-pr3754 2026-01-05 14:02:18 +00:00
danielhanchen
77e7f73641 Revert rl_replacements GRPO edits 2026-01-05 13:55:08 +00:00
danielhanchen
506bcc48e5 Fix GRPO training state restoration 2026-01-05 13:50:48 +00:00
pre-commit-ci[bot]
27e9a672a2 [pre-commit.ci] auto fixes from pre-commit.com hooks 
for more information, see https://pre-commit.ci
 2026-01-05 13:39:16 +00:00
danielhanchen
7fd3a6c177 Restore TRL version fallback in rl.py 2026-01-05 13:39:03 +00:00
Daniel Han
02ef2c25ca Merge branch 'main' into fix-grpo-version-compat 2026-01-05 05:31:42 -08:00
danielhanchen
dc986cd7e2 Drop rl.py GRPO changes from this branch 2026-01-05 13:29:58 +00:00
Daniel Han
0812d6c1e4 Merge pull request #3849 from unslothai/fix-pdl-use-vllm-version-check 
Replace GitHub API check with vLLM version check for PDL fix
 2026-01-05 05:22:16 -08:00
pre-commit-ci[bot]
c612bfe3a3 [pre-commit.ci] auto fixes from pre-commit.com hooks 
for more information, see https://pre-commit.ci
 2026-01-05 13:19:44 +00:00
Daniel Han
cb42ce8dae Address review feedback: add constant and debug logging 2026-01-05 13:19:37 +00:00
Daniel Han
9ced3523aa Replace GitHub API check with vLLM version check for PDL fix 
The GitHub issue check had issues:
1. Network latency on import
2. Issue being closed does not mean the fix is in the installed vLLM version

Now skip the PDL workaround if vLLM version > 0.13.2, which is when
the upstream fix is expected to be included.
 2026-01-05 13:15:17 +00:00
Daniel Han
2de0915e6f Merge pull request #3836 from ykaitao/remove_unused_variable_BlockDiagonalCausalMask 
remove unused variable BlockDiagonalCausalMask
 2026-01-05 04:42:25 -08:00
Daniel Han
a68cd336b6 Merge pull request #3842 from unslothai/fix-vllm-chat-template-sync 
Sync chat_template from tokenizer to vLLM
 2026-01-05 04:38:39 -08:00
Daniel Han
d9d26699b5 Merge pull request #3841 from unslothai/fix-vllm-pdl-blackwell 
Fix vLLM PDL bug on Blackwell GPUs (B200/B100)
 2026-01-05 04:37:58 -08:00
Daniel Han
9b6d536e0e Keep PDL module check but remove unnecessary env var setting 
The check skips the GitHub API call for old vLLM versions.
No need to set TRITON_DISABLE_PDL for versions without PDL support.
 2026-01-05 12:34:32 +00:00
Daniel Han
6bf555a34c Remove unnecessary PDL module existence check 
Old vLLM versions without PDL modules don't need the fix.
The patching code already handles missing modules gracefully.
 2026-01-05 12:32:16 +00:00
Daniel Han
aff2dc9061 Add None check for vLLM tokenizer 
- Check _vllm_tok is not None before accessing attributes
- Use getattr for safer chat_template access
 2026-01-05 10:02:11 +00:00
pre-commit-ci[bot]
1a9543fadd [pre-commit.ci] auto fixes from pre-commit.com hooks 
for more information, see https://pre-commit.ci
 2026-01-05 07:03:35 +00:00
danielhanchen
b9bbf47710 Improve TRL compatibility and GRPO state restore 2026-01-05 07:02:36 +00:00
Daniel Han
35219633ab Fix PDL patch: target utils.py source module and clear lru_cache 
- Patch vllm.lora.ops.triton_ops.utils directly where supports_pdl is defined
- Clear lru_cache before patching to prevent stale cached results
- Add fused_moe_lora_op to consumer modules list
- Use *args, **kwargs in fake function for compatibility
 2026-01-05 06:53:42 +00:00
Daniel Han
ba548ff8c2 Combine nested if statements for clarity 2026-01-05 05:25:53 +00:00
pre-commit-ci[bot]
eac1f6b010 [pre-commit.ci] auto fixes from pre-commit.com hooks 
for more information, see https://pre-commit.ci
 2026-01-05 05:24:59 +00:00
Daniel Han
227c31f0ca Address review feedback: refactor and scan all GPUs 
- Add _spec_exists helper function to reduce duplication
- Scan all GPUs for SM100 instead of just device 0
- Use loop for module patching to improve maintainability
 2026-01-05 05:24:52 +00:00
Daniel Han
fbdb3b524e Add tokenizer fallback for chat_template sync 2026-01-05 05:10:24 +00:00
Daniel Han
36c9a841eb Sync chat_template from tokenizer to vLLM 
When using base models with custom chat templates applied after loading,
vLLM's internal tokenizer may not have the chat_template set. This causes
issues during RL training with vLLM inference.

This fix syncs the chat_template from the processing_class (the tokenizer
you loaded and configured) to vLLM's internal tokenizer during trainer
initialization, but only if vLLM's tokenizer does not already have one set.
 2026-01-05 05:03:56 +00:00
pre-commit-ci[bot]
efe949c941 [pre-commit.ci] auto fixes from pre-commit.com hooks 
for more information, see https://pre-commit.ci
 2026-01-05 05:03:29 +00:00
Daniel Han
6c6d0dfef1 Fix vLLM PDL bug on Blackwell GPUs (B200/B100) 
vLLM's LoRA Triton kernels use tl.extra.cuda.gdc_wait() for PDL
optimization on SM90+ GPUs. This fails on SM100 (Blackwell) during
CUDA graph capture because Triton's pipeliner cannot handle gdc_wait
in complex kernels.

This fix:
- Detects SM100 GPUs and applies the workaround automatically
- Sets TRITON_DISABLE_PDL=1 environment variable
- Monkey-patches supports_pdl to return False in lora_expand_op and
  lora_shrink_op
- Checks GitHub issue #30872 status (with 3s timeout) to auto-disable
  the workaround once the upstream fix is merged
- Includes quick internet connectivity check (0.5s) to avoid delays
  when offline

Fixes the error:
'tt.elementwise_inline_asm' op pipeliner doesn't know how to predicate this op
LLVM ERROR: Fatal pipeliner error

See: https://github.com/vllm-project/vllm/issues/30872
 2026-01-05 05:02:53 +00:00
Kaitao Yang
b5addbc936 remove unused variable BlockDiagonalCausalMask 2026-01-04 09:21:44 -08:00
Daniel Han
e63c2744ec Versioning 2026-01-04 06:12:44 -08:00
Daniel Han
741a24cd67 Merge pull request #3835 from unslothai/quant-config-respect 
Respect user quantization_config
 2026-01-04 05:43:20 -08:00
Daniel Han
5b2ebe13c9 Merge pull request #3834 from unslothai/rl-fixes 
rl.py fixes: buffer reset, safer attribute access, typo fix
 2026-01-04 05:25:45 -08:00
danielhanchen
e22ca346cf Keep 4bit flag for fast_inference 2026-01-04 13:18:15 +00:00
danielhanchen
bfa225b00c Handle dict quantization_config flags 2026-01-04 13:14:03 +00:00
danielhanchen
402e7d6285 Respect user quantization_config 2026-01-04 13:03:06 +00:00
danielhanchen
d31ec48a94 Fix psutil.cpu_count() potentially returning None in save.py 2026-01-04 12:58:45 +00:00
danielhanchen
08d619fca1 Handle older unsloth-zoo without reset_unsloth_gradient_checkpointing_buffers 2026-01-04 12:57:10 +00:00
danielhanchen
3d15865bbc rl.py fixes: buffer reset, safer attribute access, typo fix 
1. Auto-reset gradient checkpointing buffers after trainer.train()
   - Import and call reset_unsloth_gradient_checkpointing_buffers() in
     prepare_for_training_mode wrapper to free memory after training
     while keeping buffers ready for subsequent runs

2. Replace eval/exec with safer getattr/setattr
   - eval(f"trl.trainer.{trainer}") -> getattr(trl.trainer, trainer)
   - exec(f"...{unwrap} = ...") -> setattr(current_trainer, unwrap, ...)
   - exec(f"Trainer.prediction_step=...") -> direct assignment

3. Fix psutil.cpu_count() potentially returning None
   - Change psutil.cpu_count()+4 to (psutil.cpu_count() or 1)+4
   - Prevents TypeError on systems where cpu_count() returns None

4. Fix typo: oriignal_is_vlm_text -> original_is_vlm_text
 2026-01-04 12:21:39 +00:00
Daniel Han
34548a089c Merge pull request #3832 from ykaitao/ktyang_remove_redundant_code_has_block 
remove redundant code of has_block
 2026-01-03 23:18:27 -08:00
Kaitao Yang
1ea6585b0c remove redundant code of has_block 2026-01-03 22:38:37 -08:00
Daniel Han
021f0cbfb1 Merge pull request #3822 from Fizza-Mukhtar/fix/llama-build-curl 
Make llama.cpp CURL dependency optional when building from source
 2026-01-03 22:12:50 -08:00
pre-commit-ci[bot]
c50b7499ff [pre-commit.ci] auto fixes from pre-commit.com hooks 
for more information, see https://pre-commit.ci
 2026-01-02 16:58:04 +00:00
Fizza-Mukhtar
8fa3228590 Make llama.cpp CURL support optional during CMake builds 2026-01-02 08:55:58 -08:00
Fizza-Mukhtar
9a3908c552 Make llama.cpp CURL support optional during CMake builds 2026-01-02 08:42:59 -08:00
Daniel Han
a2833aafd3 Merge pull request #3821 from unslothai/nightly 
Bug fixes
 2026-01-02 06:22:08 -08:00
Daniel Han
d688d3f564 Bug fixes 2026-01-02 06:07:16 -08:00
Daniel Han
8b402b45c7 Merge branch 'main' into nightly 2026-01-02 06:06:11 -08:00
Daniel Han
5b66898c56 Merge pull request #3820 from unslothai/fix/fast-generate-wrapper-helpful-errors 
Add helpful error messages for fast_generate when fast_inference=False
 2026-01-02 06:02:52 -08:00