Daniel Han
|
c792fa4f20
|
Update tokenizer_utils.py
|
2025-02-11 03:21:59 -08:00 |
|
Daniel Han
|
9f1b839d34
|
Update rl.py
|
2025-02-11 03:20:41 -08:00 |
|
Daniel Han
|
6f6a544b5c
|
Update rl.py
|
2025-02-11 03:16:07 -08:00 |
|
Daniel Han
|
764d22e0bd
|
Update rl.py
|
2025-02-11 03:08:04 -08:00 |
|
Daniel Han
|
cd4778f78f
|
Update rl.py
|
2025-02-11 03:05:44 -08:00 |
|
Daniel Han
|
edb2a62bdb
|
Update rl.py
|
2025-02-11 03:04:05 -08:00 |
|
Daniel Han
|
cf4ccb543d
|
Update rl.py
|
2025-02-11 01:39:51 -08:00 |
|
Daniel Han
|
815ef563e4
|
Update rl.py
|
2025-02-11 01:36:57 -08:00 |
|
Daniel Han
|
9a64373b27
|
Update rl.py
|
2025-02-11 01:36:30 -08:00 |
|
Daniel Han
|
be18ce5db9
|
Update rl.py
|
2025-02-11 01:33:43 -08:00 |
|
Daniel Han
|
0b61f8e86e
|
Update tokenizer_utils.py
|
2025-02-11 01:30:02 -08:00 |
|
Daniel Han
|
835dab9903
|
Update tokenizer_utils.py
|
2025-02-11 01:28:28 -08:00 |
|
Daniel Han
|
d57f8a36ec
|
Update tokenizer_utils.py
|
2025-02-11 01:27:50 -08:00 |
|
Daniel Han
|
95029e4163
|
Update tokenizer_utils.py
|
2025-02-11 01:25:17 -08:00 |
|
Daniel Han
|
3261dffeed
|
Update tokenizer_utils.py
|
2025-02-11 01:22:58 -08:00 |
|
Daniel Han
|
3ded920491
|
Update tokenizer_utils.py
|
2025-02-11 01:17:13 -08:00 |
|
Daniel Han
|
afbad75e20
|
Update tokenizer_utils.py
|
2025-02-11 01:14:06 -08:00 |
|
Daniel Han
|
b3bbe3d3f9
|
Update tokenizer_utils.py
|
2025-02-11 00:42:47 -08:00 |
|
Daniel Han
|
12cccc52c9
|
Update rl.py
|
2025-02-11 00:37:08 -08:00 |
|
Daniel Han
|
0288ca825f
|
Update tokenizer_utils.py
|
2025-02-11 00:36:12 -08:00 |
|
Daniel Han
|
9d6ad6b400
|
Update rl.py
|
2025-02-11 00:24:31 -08:00 |
|
Daniel Han
|
07f39e0b05
|
Update tokenizer_utils.py
|
2025-02-11 00:23:24 -08:00 |
|
Daniel Han
|
543e6d5ab3
|
Update tokenizer_utils.py
|
2025-02-11 00:22:02 -08:00 |
|
Daniel Han
|
a4cde480a9
|
Update tokenizer_utils.py
|
2025-02-11 00:06:08 -08:00 |
|
Daniel Han
|
2aa87bd8f1
|
Auto patching
|
2025-02-10 23:33:15 -08:00 |
|
Daniel Han
|
0767a5eccb
|
Update tokenizer_utils.py
|
2025-02-10 23:30:08 -08:00 |
|
Daniel Han
|
d3fefc2095
|
Update rl.py
|
2025-02-10 23:25:37 -08:00 |
|
Daniel Han
|
56921915f5
|
Better TRL handling
|
2025-02-10 23:24:36 -08:00 |
|
Michael Han
|
a5d1391d12
|
Merge pull request #1654 from unslothai/shimmyshimmer-patch-4
Update README.md
|
2025-02-09 19:57:27 -08:00 |
|
Michael Han
|
9807456b29
|
Update README.md
|
2025-02-09 19:57:15 -08:00 |
|
Daniel Han
|
d07aa0e4d3
|
Update tokenizer_utils.py
|
2025-02-09 19:21:58 -08:00 |
|
Daniel Han
|
766f9e5d47
|
Update tokenizer_utils.py
|
2025-02-09 19:19:51 -08:00 |
|
Daniel Han
|
a80d468199
|
Merge branch 'main' into nightly
|
2025-02-09 19:06:32 -08:00 |
|
Diogo Neves
|
36c3d36e74
|
Fixed Triton url (#1607)
Triton's link was pointing to the old research url
|
2025-02-08 19:41:39 -08:00 |
|
Daniel Han
|
bc7897805b
|
Merge branch 'main' into nightly
|
2025-02-07 00:53:37 -08:00 |
|
Michael Han
|
74fce13683
|
Update README.md
|
2025-02-06 17:20:19 -08:00 |
|
Daniel Han
|
cd52ac2e16
|
GRPO Bug fixes (#1623)
* use exact model name
* Update save.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* print
* Update _utils.py
* Update _utils.py
* Update llama.py
* Update _utils.py
* Update vision.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update loader.py
* accurate_accumulation
* Update loader.py
* Update loader.py
* Update _utils.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update pyproject.toml
* Update __init__.py
* Update pyproject.toml
* Update __init__.py
* Update __init__.py
* Fix Triton heuristics
https://github.com/triton-lang/triton/issues/5224
* Update __init__.py
* Update __init__.py
* Update __init__.py
* Update __init__.py
* Xformers
* Update loader.py
* Update loader.py
* Rewind
* Update _utils.py
* Update _utils.py
* requires grad
* Update loader.py
* Update _utils.py
* Update loader.py
* changing model to base_model if peft model is already used
* Improve debugging experience (#1512)
* Create CONTRIBUTING.md (#1472)
Creating contributing guidelines
* Update CONTRIBUTING.md
improved sentence
* Improve logging control in `unsloth_compile_transformers` by conditionally redirecting stdout based on UNSLOTH_DISABLE_LOGGER environment variable
---------
Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
Co-authored-by: Nino Risteski <95188570+NinoRisteski@users.noreply.github.com>
* Update loader.py
* Update llama.py
* Update llama.py
* Revert "Update llama.py"
This reverts commit a8edd0931a.
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Auto change is_bfloat16_supported
* Update llama.py
* Force data-type
* Update llama.py
* All attention refactor fix (#1491)
* change initilization of n_heads, n_kv_heads, hidden_size in llama.py
* do the same for cohere, mistral, gemma2, granite
* do the same for flexattention,cohere, mistral, granite
* Update llama.py
* Update llama.py
* Update granite to work with latest post_patch methods (#1502)
* Update granite to work with latest post_patch methods
* Pass position_embeddings for granite even if transformers<4.47
* Update llama.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Minor fixes for granite models (#1503)
* Update granite.py
Grab residual multiplier directly from layer
* Update llama.py
Version should read >= 4.47.1 as that is the version requiring the changes
* Update granite.py
* Update llama.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* support modelscope models and datasets (#1481)
* support modelscope
* change modelscope args
* remove useless import
* remove useless import
* fix
* wip
* fix
* remove useless code
* add readme
* add some comments
* change print to raise error
* update comment
* Update loader.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Merge branch 'main' into nightly
* Phi 4
* Update llama.py
* Torch.Cuda Is Available Condition and Warning (#1545)
* check for torch.cuda and triton if available
on my machine(mac m3) the cuda were not available
* Update pyproject.toml
* Update __init__.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Update mistral.py
* Update mistral.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Fix
* Bug fixes
* Update mapper.py
* Add dropout to granite to match HF's implementation (#1557)
Signed-off-by: datta0 <venkatadattasainimmaturi@gmail.com>
* Update llama.py
* Update llama.py
* Bug fixes
* fix: flash_attn_detection_error (#1556)
* fix: flash_attn_detection_error
* Update _utils.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Update mapper.py
* Update gemma.py
* Update gemma.py
* Update gemma.py
* Update gemma.py
* dim fix
* Update _utils.py
* Torch 2.6 support
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Faster inference?
* Update llama.py
* Update llama.py
* Update utils.py
* Update llama.py
* Update llama.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update mapper.py
* Fast Inference via vLLM
* Update llama.py
* Update llama.py
* Update utils.py
* Create rl.py
* PatchRL
* Update rl.py
* Update rl.py
* Update rl.py
* PatchRLStatistics
* Update rl.py
* Update rl.py
* Update rl.py
* Update utils.py
* Update utils.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* RL metrics
* Update rl.py
* RL metrics
* Update __init__.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update chat_templates.py
* Update mapper.py
* Fp8 cache
* Update llama.py
* Update llama.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update __init__.py
* Update loader.py
* Update rl.py
* Update rl.py
* Update _utils.py
---------
Signed-off-by: datta0 <venkatadattasainimmaturi@gmail.com>
Co-authored-by: Itsuro Tajima <tajima@georepublic.de>
Co-authored-by: Muhammad Osama <muhammadosama1994@gmail.com>
Co-authored-by: Edd <68678137+Erland366@users.noreply.github.com>
Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
Co-authored-by: Nino Risteski <95188570+NinoRisteski@users.noreply.github.com>
Co-authored-by: Kareem <81531392+KareemMusleh@users.noreply.github.com>
Co-authored-by: Datta Nimmaturi <datta.nimmaturi@nutanix.com>
Co-authored-by: Z <coffeevampirebusiness@gmail.com>
Co-authored-by: tastelikefeet <58414341+tastelikefeet@users.noreply.github.com>
Co-authored-by: AminWhat <88392440+aminwhat@users.noreply.github.com>
Co-authored-by: Zhe Zhang <2631992879@qq.com>
|
2025-02-06 05:08:22 -08:00 |
|
Daniel Han
|
10b604431a
|
Update _utils.py
|
2025-02-06 05:07:37 -08:00 |
|
Daniel Han
|
d437235d15
|
Update rl.py
|
2025-02-06 04:29:04 -08:00 |
|
Daniel Han
|
0974094f02
|
Update rl.py
|
2025-02-06 04:23:20 -08:00 |
|
Daniel Han
|
7854bc2cb8
|
Merge branch 'main' into nightly
|
2025-02-06 03:49:16 -08:00 |
|
Daniel Han
|
c8e4d4b767
|
Update
|
2025-02-06 03:23:54 -08:00 |
|
Daniel Han
|
5d6275c957
|
Update _utils.py
|
2025-02-06 02:44:46 -08:00 |
|
Daniel Han
|
e288d96272
|
Update pyproject.toml
|
2025-02-06 02:44:23 -08:00 |
|
Daniel Han
|
144190bd06
|
GRPO, vLLM, Bug Fixes, Reinforcement Learning (#1620)
* use exact model name
* Update save.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* print
* Update _utils.py
* Update _utils.py
* Update llama.py
* Update _utils.py
* Update vision.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update loader.py
* accurate_accumulation
* Update loader.py
* Update loader.py
* Update _utils.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update loader.py
* Update pyproject.toml
* Update __init__.py
* Update pyproject.toml
* Update __init__.py
* Update __init__.py
* Fix Triton heuristics
https://github.com/triton-lang/triton/issues/5224
* Update __init__.py
* Update __init__.py
* Update __init__.py
* Update __init__.py
* Xformers
* Update loader.py
* Update loader.py
* Rewind
* Update _utils.py
* Update _utils.py
* requires grad
* Update loader.py
* Update _utils.py
* Update loader.py
* changing model to base_model if peft model is already used
* Improve debugging experience (#1512)
* Create CONTRIBUTING.md (#1472)
Creating contributing guidelines
* Update CONTRIBUTING.md
improved sentence
* Improve logging control in `unsloth_compile_transformers` by conditionally redirecting stdout based on UNSLOTH_DISABLE_LOGGER environment variable
---------
Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
Co-authored-by: Nino Risteski <95188570+NinoRisteski@users.noreply.github.com>
* Update loader.py
* Update llama.py
* Update llama.py
* Revert "Update llama.py"
This reverts commit a8edd0931a.
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Auto change is_bfloat16_supported
* Update llama.py
* Force data-type
* Update llama.py
* All attention refactor fix (#1491)
* change initilization of n_heads, n_kv_heads, hidden_size in llama.py
* do the same for cohere, mistral, gemma2, granite
* do the same for flexattention,cohere, mistral, granite
* Update llama.py
* Update llama.py
* Update granite to work with latest post_patch methods (#1502)
* Update granite to work with latest post_patch methods
* Pass position_embeddings for granite even if transformers<4.47
* Update llama.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Minor fixes for granite models (#1503)
* Update granite.py
Grab residual multiplier directly from layer
* Update llama.py
Version should read >= 4.47.1 as that is the version requiring the changes
* Update granite.py
* Update llama.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* support modelscope models and datasets (#1481)
* support modelscope
* change modelscope args
* remove useless import
* remove useless import
* fix
* wip
* fix
* remove useless code
* add readme
* add some comments
* change print to raise error
* update comment
* Update loader.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Merge branch 'main' into nightly
* Phi 4
* Update llama.py
* Torch.Cuda Is Available Condition and Warning (#1545)
* check for torch.cuda and triton if available
on my machine(mac m3) the cuda were not available
* Update pyproject.toml
* Update __init__.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Update mistral.py
* Update mistral.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Fix
* Bug fixes
* Update mapper.py
* Add dropout to granite to match HF's implementation (#1557)
Signed-off-by: datta0 <venkatadattasainimmaturi@gmail.com>
* Update llama.py
* Update llama.py
* Bug fixes
* fix: flash_attn_detection_error (#1556)
* fix: flash_attn_detection_error
* Update _utils.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Update mapper.py
* Update gemma.py
* Update gemma.py
* Update gemma.py
* Update gemma.py
* dim fix
* Update _utils.py
* Torch 2.6 support
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Update llama.py
* Faster inference?
* Update llama.py
* Update llama.py
* Update utils.py
* Update llama.py
* Update llama.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update utils.py
* Update mapper.py
* Fast Inference via vLLM
* Update llama.py
* Update llama.py
* Update utils.py
* Create rl.py
* PatchRL
* Update rl.py
* Update rl.py
* Update rl.py
* PatchRLStatistics
* Update rl.py
* Update rl.py
* Update rl.py
* Update utils.py
* Update utils.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* RL metrics
* Update rl.py
* RL metrics
* Update __init__.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update chat_templates.py
* Update mapper.py
* Fp8 cache
* Update llama.py
* Update llama.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update rl.py
* Update __init__.py
* Update loader.py
---------
Signed-off-by: datta0 <venkatadattasainimmaturi@gmail.com>
Co-authored-by: Itsuro Tajima <tajima@georepublic.de>
Co-authored-by: Muhammad Osama <muhammadosama1994@gmail.com>
Co-authored-by: Edd <68678137+Erland366@users.noreply.github.com>
Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
Co-authored-by: Nino Risteski <95188570+NinoRisteski@users.noreply.github.com>
Co-authored-by: Kareem <81531392+KareemMusleh@users.noreply.github.com>
Co-authored-by: Datta Nimmaturi <datta.nimmaturi@nutanix.com>
Co-authored-by: Z <coffeevampirebusiness@gmail.com>
Co-authored-by: tastelikefeet <58414341+tastelikefeet@users.noreply.github.com>
Co-authored-by: AminWhat <88392440+aminwhat@users.noreply.github.com>
Co-authored-by: Zhe Zhang <2631992879@qq.com>
|
2025-02-06 02:41:12 -08:00 |
|
Daniel Han
|
9dd1219358
|
Update loader.py
|
2025-02-06 02:36:42 -08:00 |
|
Daniel Han
|
7d9a7aaa3d
|
Update __init__.py
|
2025-02-06 02:32:34 -08:00 |
|
Daniel Han
|
79d2b0aea3
|
Update rl.py
|
2025-02-06 02:31:01 -08:00 |
|
Daniel Han
|
d6ff188cec
|
Update rl.py
|
2025-02-06 02:24:13 -08:00 |
|
Daniel Han
|
965dee5f5e
|
Update rl.py
|
2025-02-06 02:21:08 -08:00 |
|