Daniel Han
d45b63eb3d
Merge branch 'main' into nightly
2025-12-10 06:25:56 -08:00
Daniel Han
3bc349f2d8
Gemma issue
2025-12-10 06:25:49 -08:00
Daniel Han
22cb954f4d
Merge branch 'main' into nightly
2025-12-10 06:11:32 -08:00
Daniel Han
b16af7b0f5
Update _utils.py
2025-12-10 06:11:14 -08:00
Daniel Han
c010cc6421
Update trainer.py
2025-12-10 06:11:03 -08:00
Daniel Han
4761574752
Merge branch 'main' into nightly
2025-12-10 04:15:57 -08:00
Daniel Han
26a9b5b322
Nightly ( #3706 )
...
* Update _utils.py
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2025-12-10 04:13:13 -08:00
pre-commit-ci[bot]
d8564a05b4
[pre-commit.ci] auto fixes from pre-commit.com hooks
...
for more information, see https://pre-commit.ci
2025-12-10 12:12:49 +00:00
Daniel Han
0372af5ca4
Merge branch 'main' into nightly
2025-12-10 04:12:19 -08:00
Daniel Han
f0c8e21d59
Update import_fixes.py
2025-12-10 04:05:14 -08:00
Daniel Han
f9564cf84e
Update _utils.py
2025-12-10 03:48:01 -08:00
Datta Nimmaturi
62d19a12ff
[FIX] fbgemm version check ( #3704 )
...
* fbgemm version check
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* safer version check
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add check for torchvision-torch compatibility
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* refactor package check logic
* Remove logs and enforce torch
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2025-12-10 03:46:30 -08:00
Dan Saunders
75e0d7ce62
Auto-enable padding-free SFT ( #3672 )
...
* implement (sdpa, xformers, fa2) sample packing
* attention dispatching
* ddp working OOTB with CLI
* packed SWA and softcap support
* enable batch flattening
* LGPL license headers
* mask packed sequence boundaries
* auto-enable sample packing
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add explicit toggle for sample packing
* Add explicit toggle for sample packing
* Update __init__.py
* Update unsloth/kernels/rope_embedding.py
* Update unsloth/kernels/rope_embedding.py
* remove grad output clones; restore deleted FastLanguageModel arg
* fix
* restore rope embedding clones
* xformers mask cache
* implement (sdpa, xformers, fa2) sample packing
* attention dispatching
* ddp working OOTB with CLI
* packed SWA and softcap support
* enable batch flattening
* LGPL license headers
* mask packed sequence boundaries
* auto-enable sample packing
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add explicit toggle for sample packing
* Add explicit toggle for sample packing
* Update __init__.py
* Update unsloth/kernels/rope_embedding.py
* Update unsloth/kernels/rope_embedding.py
* remove grad output clones; restore deleted FastLanguageModel arg
* fix
* restore rope embedding clones
* xformers mask cache
* add back accidental deletion
* Update unsloth/kernels/rope_embedding.py
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix merge conflicts
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add **kwargs
* add back clobbered
* Update rope_embedding.py
* Update rope_embedding.py
* simplify trl warnings filter
* docstring
* nit
* bugfix
* add padding-free seqlen metadata
* auto-enable padding free
* gemma2 disable
* Apply suggestion from @danielhanchen
* Update trainer.py
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update trainer.py
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2025-12-10 03:07:29 -08:00
Daniel Han
c81025b24d
Merge branch 'main' into nightly
2025-12-09 17:37:18 -08:00
Dan Saunders
496f84ff6b
SFT sample packing ( #3566 )
...
* implement (sdpa, xformers, fa2) sample packing
* attention dispatching
* ddp working OOTB with CLI
* packed SWA and softcap support
* enable batch flattening
* LGPL license headers
* mask packed sequence boundaries
* auto-enable sample packing
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add explicit toggle for sample packing
* Add explicit toggle for sample packing
* Update __init__.py
* Update unsloth/kernels/rope_embedding.py
* Update unsloth/kernels/rope_embedding.py
* remove grad output clones; restore deleted FastLanguageModel arg
* fix
* restore rope embedding clones
* xformers mask cache
* implement (sdpa, xformers, fa2) sample packing
* attention dispatching
* ddp working OOTB with CLI
* packed SWA and softcap support
* enable batch flattening
* LGPL license headers
* mask packed sequence boundaries
* auto-enable sample packing
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add explicit toggle for sample packing
* Add explicit toggle for sample packing
* Update __init__.py
* Update unsloth/kernels/rope_embedding.py
* Update unsloth/kernels/rope_embedding.py
* remove grad output clones; restore deleted FastLanguageModel arg
* fix
* restore rope embedding clones
* xformers mask cache
* add back accidental deletion
* Update unsloth/kernels/rope_embedding.py
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* fix merge conflicts
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Add **kwargs
* add back clobbered
* Update rope_embedding.py
* Update rope_embedding.py
* simplify trl warnings filter
* docstring
* nit
* bugfix
* Apply suggestion from @danielhanchen
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update unsloth/trainer.py
* Update unsloth/trainer.py
* Update unsloth/trainer.py
* Update unsloth/trainer.py
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2025-12-09 17:36:45 -08:00
Daniel Han
e561bb3ef5
Merge branch 'main' into nightly
2025-12-09 03:31:30 -08:00
Daniel Han
2b3cb06925
Update _utils.py ( #3698 )
2025-12-09 03:31:20 -08:00
Daniel Han
cafa92a63b
Merge branch 'main' into nightly
2025-12-09 03:30:42 -08:00
Datta Nimmaturi
89787329d3
[Fix] [TRL] load_lora for multi line llm.chat/generate ( #3696 )
...
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Remove reload_weights rpc call from grpo trainer
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Use regex instead of static string
* patch openenv reload_weights call
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Better handle sleep and wakeup
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Reset indentation
* Handle multi line self.llm.chat better
* Use logger
* re-indent
* Stricter regex to replace wildcard
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2025-12-09 03:30:23 -08:00
Daniel Han
6264afbf87
Update _utils.py
2025-12-09 01:02:26 -08:00
Datta Nimmaturi
9e5b4052e5
Remove reload_weights rpc call from grpo trainer ( #3673 )
...
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Remove reload_weights rpc call from grpo trainer
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Use regex instead of static string
* patch openenv reload_weights call
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Better handle sleep and wakeup
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Reset indentation
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2025-12-08 23:36:22 -08:00
pre-commit-ci[bot]
c579cd7094
[pre-commit.ci] pre-commit autoupdate ( #3694 )
...
updates:
- [github.com/astral-sh/ruff-pre-commit: v0.14.7 → v0.14.8](https://github.com/astral-sh/ruff-pre-commit/compare/v0.14.7...v0.14.8 )
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2025-12-08 19:44:56 -08:00
Daniel Han
43ad66d37a
Versioning
2025-12-08 04:19:10 -08:00
Daniel Han
bebf042e0f
Update pyproject.toml
2025-12-08 04:13:45 -08:00
Daniel Han
e72e9d499d
Versioning
2025-12-08 04:06:01 -08:00
Noah Kirschmann
a80f1991c5
Update transformers version constraint in pyproject.toml ( #3689 )
...
* Update transformers version constraint in pyproject.toml
The latest transformers version just fixes the local training.
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Update transformers version constraint in pyproject.toml
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2025-12-08 03:27:18 -08:00
Daniel Han
034a35f6d4
Add **kwargs
2025-12-08 03:24:51 -08:00
Daniel Han
f4de7baea1
Update rl.py
2025-12-08 02:23:43 -08:00
Daniel Han
4408ddc081
Update vision.py
2025-12-07 23:09:13 -08:00
Daniel Han
d86ded6799
Update _utils.py
2025-12-07 16:52:59 -08:00
Daniel Han
cb4d8da5a2
Xformers fix
2025-12-07 16:40:51 -08:00
Michael Han
3d4f236155
Update README.md
2025-12-04 08:21:20 -08:00
Daniel Han
845e61d351
Update README.md
2025-12-02 04:08:54 -08:00
Daniel Han
14e8e3137d
Update README.md
2025-12-02 03:52:50 -08:00
pre-commit-ci[bot]
13f6491fe6
[pre-commit.ci] pre-commit autoupdate ( #3666 )
...
updates:
- [github.com/astral-sh/ruff-pre-commit: v0.14.6 → v0.14.7](https://github.com/astral-sh/ruff-pre-commit/compare/v0.14.6...v0.14.7 )
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2025-12-01 17:45:13 -08:00
Daniel Han
4fd865cc99
Update _utils.py
2025-12-01 08:01:05 -08:00
Daniel Han
d655c7434a
Update rl.py
2025-12-01 08:00:08 -08:00
Daniel Han
66649d18bd
Revert "[pre-commit.ci] auto fixes from pre-commit.com hooks"
...
This reverts commit cad158a56c
2025-12-01 07:24:58 -08:00
pre-commit-ci[bot]
cad158a56c
[pre-commit.ci] auto fixes from pre-commit.com hooks
...
for more information, see https://pre-commit.ci
2025-12-01 15:24:34 +00:00
Daniel Han
487a951914
Revert "[pre-commit.ci] auto fixes from pre-commit.com hooks"
...
This reverts commit 964c9fef95
2025-12-01 07:24:21 -08:00
pre-commit-ci[bot]
964c9fef95
[pre-commit.ci] auto fixes from pre-commit.com hooks
...
for more information, see https://pre-commit.ci
2025-12-01 15:23:44 +00:00
Daniel Han
5f27bc4db5
Revert "[pre-commit.ci] auto fixes from pre-commit.com hooks"
...
This reverts commit d34e0454ac
2025-12-01 07:23:31 -08:00
pre-commit-ci[bot]
d34e0454ac
[pre-commit.ci] auto fixes from pre-commit.com hooks
...
for more information, see https://pre-commit.ci
2025-12-01 15:20:22 +00:00
Daniel Han
d994280cdc
Update unsloth/models/rl.py
...
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2025-12-01 07:20:00 -08:00
Daniel Han
ebec564689
Update unsloth/models/rl.py
...
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
2025-12-01 07:19:50 -08:00
Daniel Han
b4f5a70878
Update qwen3_moe.py
2025-12-01 07:19:07 -08:00
Datta Nimmaturi
04cfc0d139
Vllm guided decoding ( #3663 )
...
* vllm sampling params fix
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* do not patch base_trainer
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* seperate vllm fixes
* [pre-commit.ci] auto fixes from pre-commit.com hooks
for more information, see https://pre-commit.ci
* Fixup deletion
* Fix indentation
* revert to old style
---------
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
2025-12-01 07:11:28 -08:00
Daniel Han
3d62c38ada
Verisoning
2025-12-01 07:09:17 -08:00
Daniel Han
3f4768ad1e
Update rl.py
2025-12-01 06:23:23 -08:00
Daniel Han
ba2897a318
Revert "[FIX] Vllm guided decoding params ( #3662 )"
...
This reverts commit fb4f0fdf56
2025-12-01 05:43:45 -08:00
