ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
3,250 commits 391 branches 58 tags 304 MiB
Commit graph

3,250 commits

This branch
This branch
All branches
Author SHA1 Message Date
Datta Nimmaturi
dc83a17239 Check env var explicitly 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
 2026-01-06 15:30:06 +05:30
Datta Nimmaturi
67caa21231 Disable stats when modelscope is being used 2026-01-06 09:53:20 +00:00
Daniel Han
52935bb00f Versioning 2026-01-05 07:37:08 -08:00
Daniel Han
3b4ac4aa1c Merge pull request #3843 from unslothai/fix-grpo-version-compat 
Unify Version usage and fix TRL version handling
 2026-01-05 06:07:41 -08:00
Daniel Han
7e13c424c9 Merge pull request #3851 from unslothai/grpo-fix-on-pr3754 
GRPO: restore model mode after generate (stacked on #3754)
 2026-01-05 06:05:24 -08:00
danielhanchen
3240ab3391 Merge main into grpo-fix-on-pr3754 2026-01-05 14:02:18 +00:00
danielhanchen
3efb799aad Revert rl_replacements GRPO edits 2026-01-05 13:55:08 +00:00
danielhanchen
6ede9a735d Fix GRPO training state restoration 2026-01-05 13:50:48 +00:00
pre-commit-ci[bot]
18f335cec5 [pre-commit.ci] auto fixes from pre-commit.com hooks 
for more information, see https://pre-commit.ci
 2026-01-05 13:39:16 +00:00
danielhanchen
d5fcc7ddde Restore TRL version fallback in rl.py 2026-01-05 13:39:03 +00:00
Daniel Han
7f93aa0a78 Merge branch 'main' into fix-grpo-version-compat 2026-01-05 05:31:42 -08:00
danielhanchen
9d4ccdbff5 Drop rl.py GRPO changes from this branch 2026-01-05 13:29:58 +00:00
Daniel Han
a1eaf90c7b Merge pull request #3849 from unslothai/fix-pdl-use-vllm-version-check 
Replace GitHub API check with vLLM version check for PDL fix
 2026-01-05 05:22:16 -08:00
pre-commit-ci[bot]
cd8c6d773d [pre-commit.ci] auto fixes from pre-commit.com hooks 
for more information, see https://pre-commit.ci
 2026-01-05 13:19:44 +00:00
Daniel Han
44f420db4f Address review feedback: add constant and debug logging 2026-01-05 13:19:37 +00:00
Daniel Han
e8def1194d Replace GitHub API check with vLLM version check for PDL fix 
The GitHub issue check had issues:
1. Network latency on import
2. Issue being closed does not mean the fix is in the installed vLLM version

Now skip the PDL workaround if vLLM version > 0.13.2, which is when
the upstream fix is expected to be included.
 2026-01-05 13:15:17 +00:00
Daniel Han
45e27a841d Merge pull request #3836 from ykaitao/remove_unused_variable_BlockDiagonalCausalMask 
remove unused variable BlockDiagonalCausalMask
 2026-01-05 04:42:25 -08:00
Daniel Han
6c79d84318 Merge pull request #3842 from unslothai/fix-vllm-chat-template-sync 
Sync chat_template from tokenizer to vLLM
 2026-01-05 04:38:39 -08:00
Daniel Han
10926b0e3a Merge pull request #3841 from unslothai/fix-vllm-pdl-blackwell 
Fix vLLM PDL bug on Blackwell GPUs (B200/B100)
 2026-01-05 04:37:58 -08:00
Daniel Han
c8a585a589 Keep PDL module check but remove unnecessary env var setting 
The check skips the GitHub API call for old vLLM versions.
No need to set TRITON_DISABLE_PDL for versions without PDL support.
 2026-01-05 12:34:32 +00:00
Daniel Han
bc0f1514f2 Remove unnecessary PDL module existence check 
Old vLLM versions without PDL modules don't need the fix.
The patching code already handles missing modules gracefully.
 2026-01-05 12:32:16 +00:00
Daniel Han
f22a35d903 Add None check for vLLM tokenizer 
- Check _vllm_tok is not None before accessing attributes
- Use getattr for safer chat_template access
 2026-01-05 10:02:11 +00:00
pre-commit-ci[bot]
defbf038b2 [pre-commit.ci] auto fixes from pre-commit.com hooks 
for more information, see https://pre-commit.ci
 2026-01-05 07:03:35 +00:00
danielhanchen
8e941a6422 Improve TRL compatibility and GRPO state restore 2026-01-05 07:02:36 +00:00
Daniel Han
9860d8859d Fix PDL patch: target utils.py source module and clear lru_cache 
- Patch vllm.lora.ops.triton_ops.utils directly where supports_pdl is defined
- Clear lru_cache before patching to prevent stale cached results
- Add fused_moe_lora_op to consumer modules list
- Use *args, **kwargs in fake function for compatibility
 2026-01-05 06:53:42 +00:00
Daniel Han
ff846db6f6 Combine nested if statements for clarity 2026-01-05 05:25:53 +00:00
pre-commit-ci[bot]
dabfa79a16 [pre-commit.ci] auto fixes from pre-commit.com hooks 
for more information, see https://pre-commit.ci
 2026-01-05 05:24:59 +00:00
Daniel Han
e627adb4d5 Address review feedback: refactor and scan all GPUs 
- Add _spec_exists helper function to reduce duplication
- Scan all GPUs for SM100 instead of just device 0
- Use loop for module patching to improve maintainability
 2026-01-05 05:24:52 +00:00
Daniel Han
5976c3f10f Add tokenizer fallback for chat_template sync 2026-01-05 05:10:24 +00:00
Daniel Han
e727c43685 Sync chat_template from tokenizer to vLLM 
When using base models with custom chat templates applied after loading,
vLLM's internal tokenizer may not have the chat_template set. This causes
issues during RL training with vLLM inference.

This fix syncs the chat_template from the processing_class (the tokenizer
you loaded and configured) to vLLM's internal tokenizer during trainer
initialization, but only if vLLM's tokenizer does not already have one set.
 2026-01-05 05:03:56 +00:00
pre-commit-ci[bot]
1a329b9e4f [pre-commit.ci] auto fixes from pre-commit.com hooks 
for more information, see https://pre-commit.ci
 2026-01-05 05:03:29 +00:00
Daniel Han
594d3baffe Fix vLLM PDL bug on Blackwell GPUs (B200/B100) 
vLLM's LoRA Triton kernels use tl.extra.cuda.gdc_wait() for PDL
optimization on SM90+ GPUs. This fails on SM100 (Blackwell) during
CUDA graph capture because Triton's pipeliner cannot handle gdc_wait
in complex kernels.

This fix:
- Detects SM100 GPUs and applies the workaround automatically
- Sets TRITON_DISABLE_PDL=1 environment variable
- Monkey-patches supports_pdl to return False in lora_expand_op and
  lora_shrink_op
- Checks GitHub issue #30872 status (with 3s timeout) to auto-disable
  the workaround once the upstream fix is merged
- Includes quick internet connectivity check (0.5s) to avoid delays
  when offline

Fixes the error:
'tt.elementwise_inline_asm' op pipeliner doesn't know how to predicate this op
LLVM ERROR: Fatal pipeliner error

See: https://github.com/vllm-project/vllm/issues/30872
 2026-01-05 05:02:53 +00:00
Kaitao Yang
d66548f904 remove unused variable BlockDiagonalCausalMask 2026-01-04 09:21:44 -08:00
Daniel Han
1dd67b372e Versioning 2026-01-04 06:12:44 -08:00
Daniel Han
eef05330ca Merge pull request #3835 from unslothai/quant-config-respect 
Respect user quantization_config
 2026-01-04 05:43:20 -08:00
Daniel Han
7bf648a882 Merge pull request #3834 from unslothai/rl-fixes 
rl.py fixes: buffer reset, safer attribute access, typo fix
 2026-01-04 05:25:45 -08:00
danielhanchen
15052dc8e7 Keep 4bit flag for fast_inference 2026-01-04 13:18:15 +00:00
danielhanchen
e72808553f Handle dict quantization_config flags 2026-01-04 13:14:03 +00:00
danielhanchen
3bfc927984 Respect user quantization_config 2026-01-04 13:03:06 +00:00
danielhanchen
0b1dbefacb Fix psutil.cpu_count() potentially returning None in save.py 2026-01-04 12:58:45 +00:00
danielhanchen
d1d9832e70 Handle older unsloth-zoo without reset_unsloth_gradient_checkpointing_buffers 2026-01-04 12:57:10 +00:00
danielhanchen
762ef9a20f rl.py fixes: buffer reset, safer attribute access, typo fix 
1. Auto-reset gradient checkpointing buffers after trainer.train()
   - Import and call reset_unsloth_gradient_checkpointing_buffers() in
     prepare_for_training_mode wrapper to free memory after training
     while keeping buffers ready for subsequent runs

2. Replace eval/exec with safer getattr/setattr
   - eval(f"trl.trainer.{trainer}") -> getattr(trl.trainer, trainer)
   - exec(f"...{unwrap} = ...") -> setattr(current_trainer, unwrap, ...)
   - exec(f"Trainer.prediction_step=...") -> direct assignment

3. Fix psutil.cpu_count() potentially returning None
   - Change psutil.cpu_count()+4 to (psutil.cpu_count() or 1)+4
   - Prevents TypeError on systems where cpu_count() returns None

4. Fix typo: oriignal_is_vlm_text -> original_is_vlm_text
 2026-01-04 12:21:39 +00:00
Daniel Han
dc538896a5 Merge pull request #3832 from ykaitao/ktyang_remove_redundant_code_has_block 
remove redundant code of has_block
 2026-01-03 23:18:27 -08:00
Kaitao Yang
d84602e549 remove redundant code of has_block 2026-01-03 22:38:37 -08:00
Daniel Han
6753691c92 Merge pull request #3822 from Fizza-Mukhtar/fix/llama-build-curl 
Make llama.cpp CURL dependency optional when building from source
 2026-01-03 22:12:50 -08:00
pre-commit-ci[bot]
975e36f888 [pre-commit.ci] auto fixes from pre-commit.com hooks 
for more information, see https://pre-commit.ci
 2026-01-02 16:58:04 +00:00
Fizza-Mukhtar
1b4abe7a4e Make llama.cpp CURL support optional during CMake builds 2026-01-02 08:55:58 -08:00
Fizza-Mukhtar
22210112c6 Make llama.cpp CURL support optional during CMake builds 2026-01-02 08:42:59 -08:00
Daniel Han
dc6df32f81 Merge pull request #3821 from unslothai/nightly 
Bug fixes
 2026-01-02 06:22:08 -08:00
Daniel Han
52aed3ad14 Bug fixes 2026-01-02 06:07:16 -08:00