ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
1,331 commits 391 branches 58 tags 304 MiB
Commit graph

1,331 commits

This branch
This branch
All branches
Author SHA1 Message Date
Daniel Han
4ecc4c3500 Update rl.py 2025-02-11 16:04:56 -08:00
Daniel Han
a5016481c5 Update rl.py 2025-02-11 16:03:33 -08:00
Daniel Han
1e78f1bb80 Update rl.py 2025-02-11 15:57:32 -08:00
Daniel Han
fb443fcfd6 Update rl.py 2025-02-11 15:53:46 -08:00
Daniel Han
eaed3eb015 Patching 2025-02-11 15:11:16 -08:00
Daniel Han
4be70adb57 Update rl.py 2025-02-11 15:00:44 -08:00
Daniel Han
65439f81c5 Update rl.py 2025-02-11 14:27:31 -08:00
Daniel Han
9d94c475ad max seq length 2025-02-11 14:22:19 -08:00
Daniel Han
7a6b9b8459 Update rl.py 2025-02-11 14:11:33 -08:00
Daniel Han
5fa88c3cb9 Update rl.py 2025-02-11 03:25:37 -08:00
Daniel Han
657ce5cab9 Update rl.py 2025-02-11 03:23:05 -08:00
Daniel Han
5c07343168 Update tokenizer_utils.py 2025-02-11 03:21:59 -08:00
Daniel Han
0d99380d54 Update rl.py 2025-02-11 03:20:41 -08:00
Daniel Han
af03bfc047 Update rl.py 2025-02-11 03:16:07 -08:00
Daniel Han
58c0823fc9 Update rl.py 2025-02-11 03:08:04 -08:00
Daniel Han
6697a6d95b Update rl.py 2025-02-11 03:05:44 -08:00
Daniel Han
71aff2d7f9 Update rl.py 2025-02-11 03:04:05 -08:00
Daniel Han
5cd1cbe05f Update rl.py 2025-02-11 01:39:51 -08:00
Daniel Han
ee31eb03e9 Update rl.py 2025-02-11 01:36:57 -08:00
Daniel Han
474822050a Update rl.py 2025-02-11 01:36:30 -08:00
Daniel Han
0d17681703 Update rl.py 2025-02-11 01:33:43 -08:00
Daniel Han
09e1fb7672 Update tokenizer_utils.py 2025-02-11 01:30:02 -08:00
Daniel Han
7bf6253395 Update tokenizer_utils.py 2025-02-11 01:28:28 -08:00
Daniel Han
058a2a7887 Update tokenizer_utils.py 2025-02-11 01:27:50 -08:00
Daniel Han
6195068e40 Update tokenizer_utils.py 2025-02-11 01:25:17 -08:00
Daniel Han
3dc5893e12 Update tokenizer_utils.py 2025-02-11 01:22:58 -08:00
Daniel Han
50f95ea65e Update tokenizer_utils.py 2025-02-11 01:17:13 -08:00
Daniel Han
55967af47d Update tokenizer_utils.py 2025-02-11 01:14:06 -08:00
Daniel Han
6ae8ef7fdf Update tokenizer_utils.py 2025-02-11 00:42:47 -08:00
Daniel Han
322225ce70 Update rl.py 2025-02-11 00:37:08 -08:00
Daniel Han
01b312cc5f Update tokenizer_utils.py 2025-02-11 00:36:12 -08:00
Daniel Han
fef409112c Update rl.py 2025-02-11 00:24:31 -08:00
Daniel Han
e58b548a95 Update tokenizer_utils.py 2025-02-11 00:23:24 -08:00
Daniel Han
5077ecea0e Update tokenizer_utils.py 2025-02-11 00:22:02 -08:00
Daniel Han
1fbd142c42 Update tokenizer_utils.py 2025-02-11 00:06:08 -08:00
Daniel Han
b8c47beb46 Auto patching 2025-02-10 23:33:15 -08:00
Daniel Han
d5792b3b02 Update tokenizer_utils.py 2025-02-10 23:30:08 -08:00
Daniel Han
7d2dd75939 Update rl.py 2025-02-10 23:25:37 -08:00
Daniel Han
ff4d021c72 Better TRL handling 2025-02-10 23:24:36 -08:00
Daniel Han
dbf46781c9 Update tokenizer_utils.py 2025-02-09 19:21:58 -08:00
Daniel Han
16ddec731a Update tokenizer_utils.py 2025-02-09 19:19:51 -08:00
Daniel Han
87034b1891 Merge branch 'main' into nightly 2025-02-09 19:06:32 -08:00
Diogo Neves
b674d2e484 Fixed Triton url (#1607) 
Triton's link was pointing to the old research url
 2025-02-08 19:41:39 -08:00
Daniel Han
8af27e9b67 Merge branch 'main' into nightly 2025-02-07 00:53:37 -08:00
Michael Han
aad2dc5cc8 Update README.md 2025-02-06 17:20:19 -08:00
Daniel Han
226a8ccefd GRPO Bug fixes (#1623) 
* use exact model name

* Update save.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* print

* Update _utils.py

* Update _utils.py

* Update llama.py

* Update _utils.py

* Update vision.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update loader.py

* accurate_accumulation

* Update loader.py

* Update loader.py

* Update _utils.py

* Update loader.py

* Update loader.py

* Update loader.py

* Update loader.py

* Update pyproject.toml

* Update __init__.py

* Update pyproject.toml

* Update __init__.py

* Update __init__.py

* Fix Triton heuristics

https://github.com/triton-lang/triton/issues/5224

* Update __init__.py

* Update __init__.py

* Update __init__.py

* Update __init__.py

* Xformers

* Update loader.py

* Update loader.py

* Rewind

* Update _utils.py

* Update _utils.py

* requires grad

* Update loader.py

* Update _utils.py

* Update loader.py

* changing model to base_model if peft model is already used

* Improve debugging experience (#1512)

* Create CONTRIBUTING.md (#1472)

Creating contributing guidelines

* Update CONTRIBUTING.md

improved sentence

* Improve logging control in `unsloth_compile_transformers` by conditionally redirecting stdout based on UNSLOTH_DISABLE_LOGGER environment variable

---------

Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
Co-authored-by: Nino Risteski <95188570+NinoRisteski@users.noreply.github.com>

* Update loader.py

* Update llama.py

* Update llama.py

* Revert "Update llama.py"

This reverts commit 67bb995878.

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Auto change is_bfloat16_supported

* Update llama.py

* Force data-type

* Update llama.py

* All attention refactor fix (#1491)

* change initilization of n_heads, n_kv_heads, hidden_size in llama.py

* do the same for cohere, mistral, gemma2, granite

* do the same for flexattention,cohere, mistral, granite

* Update llama.py

* Update llama.py

* Update granite to work with latest post_patch methods (#1502)

* Update granite to work with latest post_patch methods

* Pass position_embeddings for granite even if transformers<4.47

* Update llama.py

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* Minor fixes for granite models (#1503)

* Update granite.py

Grab residual multiplier directly from layer

* Update llama.py

Version should read >= 4.47.1 as that is the version requiring the changes

* Update granite.py

* Update llama.py

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* support modelscope models and datasets (#1481)

* support modelscope

* change modelscope args

* remove useless import

* remove useless import

* fix

* wip

* fix

* remove useless code

* add readme

* add some comments

* change print to raise error

* update comment

* Update loader.py

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* Merge branch 'main' into nightly

* Phi 4

* Update llama.py

* Torch.Cuda Is Available Condition and Warning (#1545)

* check for torch.cuda and triton if available
on my machine(mac m3) the cuda were not available

* Update pyproject.toml

* Update __init__.py

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* Update mistral.py

* Update mistral.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Fix

* Bug fixes

* Update mapper.py

* Add dropout to granite to match HF's implementation (#1557)

Signed-off-by: datta0 <venkatadattasainimmaturi@gmail.com>

* Update llama.py

* Update llama.py

* Bug fixes

* fix: flash_attn_detection_error (#1556)

* fix: flash_attn_detection_error

* Update _utils.py

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* Update mapper.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* dim fix

* Update _utils.py

* Torch 2.6 support

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Faster inference?

* Update llama.py

* Update llama.py

* Update utils.py

* Update llama.py

* Update llama.py

* Update utils.py

* Update utils.py

* Update utils.py

* Update utils.py

* Update utils.py

* Update utils.py

* Update utils.py

* Update utils.py

* Update utils.py

* Update utils.py

* Update utils.py

* Update utils.py

* Update utils.py

* Update mapper.py

* Fast Inference via vLLM

* Update llama.py

* Update llama.py

* Update utils.py

* Create rl.py

* PatchRL

* Update rl.py

* Update rl.py

* Update rl.py

* PatchRLStatistics

* Update rl.py

* Update rl.py

* Update rl.py

* Update utils.py

* Update utils.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* RL metrics

* Update rl.py

* RL metrics

* Update __init__.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update chat_templates.py

* Update mapper.py

* Fp8 cache

* Update llama.py

* Update llama.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update __init__.py

* Update loader.py

* Update rl.py

* Update rl.py

* Update _utils.py

---------

Signed-off-by: datta0 <venkatadattasainimmaturi@gmail.com>
Co-authored-by: Itsuro Tajima <tajima@georepublic.de>
Co-authored-by: Muhammad Osama <muhammadosama1994@gmail.com>
Co-authored-by: Edd <68678137+Erland366@users.noreply.github.com>
Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
Co-authored-by: Nino Risteski <95188570+NinoRisteski@users.noreply.github.com>
Co-authored-by: Kareem <81531392+KareemMusleh@users.noreply.github.com>
Co-authored-by: Datta Nimmaturi <datta.nimmaturi@nutanix.com>
Co-authored-by: Z <coffeevampirebusiness@gmail.com>
Co-authored-by: tastelikefeet <58414341+tastelikefeet@users.noreply.github.com>
Co-authored-by: AminWhat <88392440+aminwhat@users.noreply.github.com>
Co-authored-by: Zhe Zhang <2631992879@qq.com>
 2025-02-06 05:08:22 -08:00
Daniel Han
ee5aaa2e82 Update _utils.py 2025-02 2025-02-06 05:07:37 -08:00
Daniel Han
3cb045ba00 Update rl.py 2025-02-06 04:29:04 -08:00
Daniel Han
84b68d49df Update rl.py 2025-02-06 04:23:20 -08:00
Daniel Han
6dce39f1ef Merge branch 'main' into nightly 2025-02-06 03:49:16 -08:00