ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
1,335 commits 391 branches 58 tags 304 MiB
Commit graph

1,335 commits

This branch
This branch
All branches
Author SHA1 Message Date
Daniel Han
7b265dbe0c Update rl.py 2025-02-11 18:54:39 -08:00
Daniel Han
2af746b1bd Update rl.py 2025-02-11 18:49:09 -08:00
Daniel Han
7ea918d85b NEFTune 2025-02-11 18:19:16 -08:00
Daniel Han
800536774a Update rl.py 2025-02-11 16:20:14 -08:00
Daniel Han
af4cd27eb9 Update rl.py 2025-02-11 16:04:56 -08:00
Daniel Han
722c4ecca6 Update rl.py 2025-02-11 16:03:33 -08:00
Daniel Han
da96bd8374 Update rl.py 2025-02-11 15:57:32 -08:00
Daniel Han
7c3b51100f Update rl.py 2025-02-11 15:53:46 -08:00
Daniel Han
aae24f5ae3 Patching 2025-02-11 15:11:16 -08:00
Daniel Han
875181b6d2 Update rl.py 2025-02-11 15:00:44 -08:00
Daniel Han
da7e35fd35 Update rl.py 2025-02-11 14:27:31 -08:00
Daniel Han
9680d0f73a max seq length 2025-02-11 14:22:19 -08:00
Daniel Han
6fbba44ff0 Update rl.py 2025-02-11 14:11:33 -08:00
Daniel Han
766c71844f Update rl.py 2025-02-11 03:25:37 -08:00
Daniel Han
f1a924c31f Update rl.py 2025-02-11 03:23:05 -08:00
Daniel Han
c792fa4f20 Update tokenizer_utils.py 2025-02-11 03:21:59 -08:00
Daniel Han
9f1b839d34 Update rl.py 2025-02-11 03:20:41 -08:00
Daniel Han
6f6a544b5c Update rl.py 2025-02-11 03:16:07 -08:00
Daniel Han
764d22e0bd Update rl.py 2025-02-11 03:08:04 -08:00
Daniel Han
cd4778f78f Update rl.py 2025-02-11 03:05:44 -08:00
Daniel Han
edb2a62bdb Update rl.py 2025-02-11 03:04:05 -08:00
Daniel Han
cf4ccb543d Update rl.py 2025-02-11 01:39:51 -08:00
Daniel Han
815ef563e4 Update rl.py 2025-02-11 01:36:57 -08:00
Daniel Han
9a64373b27 Update rl.py 2025-02-11 01:36:30 -08:00
Daniel Han
be18ce5db9 Update rl.py 2025-02-11 01:33:43 -08:00
Daniel Han
0b61f8e86e Update tokenizer_utils.py 2025-02-11 01:30:02 -08:00
Daniel Han
835dab9903 Update tokenizer_utils.py 2025-02-11 01:28:28 -08:00
Daniel Han
d57f8a36ec Update tokenizer_utils.py 2025-02-11 01:27:50 -08:00
Daniel Han
95029e4163 Update tokenizer_utils.py 2025-02-11 01:25:17 -08:00
Daniel Han
3261dffeed Update tokenizer_utils.py 2025-02-11 01:22:58 -08:00
Daniel Han
3ded920491 Update tokenizer_utils.py 2025-02-11 01:17:13 -08:00
Daniel Han
afbad75e20 Update tokenizer_utils.py 2025-02-11 01:14:06 -08:00
Daniel Han
b3bbe3d3f9 Update tokenizer_utils.py 2025-02-11 00:42:47 -08:00
Daniel Han
12cccc52c9 Update rl.py 2025-02-11 00:37:08 -08:00
Daniel Han
0288ca825f Update tokenizer_utils.py 2025-02-11 00:36:12 -08:00
Daniel Han
9d6ad6b400 Update rl.py 2025-02-11 00:24:31 -08:00
Daniel Han
07f39e0b05 Update tokenizer_utils.py 2025-02-11 00:23:24 -08:00
Daniel Han
543e6d5ab3 Update tokenizer_utils.py 2025-02-11 00:22:02 -08:00
Daniel Han
a4cde480a9 Update tokenizer_utils.py 2025-02-11 00:06:08 -08:00
Daniel Han
2aa87bd8f1 Auto patching 2025-02-10 23:33:15 -08:00
Daniel Han
0767a5eccb Update tokenizer_utils.py 2025-02-10 23:30:08 -08:00
Daniel Han
d3fefc2095 Update rl.py 2025-02-10 23:25:37 -08:00
Daniel Han
56921915f5 Better TRL handling 2025-02-10 23:24:36 -08:00
Daniel Han
d07aa0e4d3 Update tokenizer_utils.py 2025-02-09 19:21:58 -08:00
Daniel Han
766f9e5d47 Update tokenizer_utils.py 2025-02-09 19:19:51 -08:00
Daniel Han
a80d468199 Merge branch 'main' into nightly 2025-02-09 19:06:32 -08:00
Diogo Neves
36c3d36e74 Fixed Triton url (#1607) 
Triton's link was pointing to the old research url
 2025-02-08 19:41:39 -08:00
Daniel Han
bc7897805b Merge branch 'main' into nightly 2025-02-07 00:53:37 -08:00
Michael Han
74fce13683 Update README.md 2025-02-06 17:20:19 -08:00
Daniel Han
cd52ac2e16 GRPO Bug fixes (#1623) 
* use exact model name

* Update save.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* print

* Update _utils.py

* Update _utils.py

* Update llama.py

* Update _utils.py

* Update vision.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update loader.py

* accurate_accumulation

* Update loader.py

* Update loader.py

* Update _utils.py

* Update loader.py

* Update loader.py

* Update loader.py

* Update loader.py

* Update pyproject.toml

* Update __init__.py

* Update pyproject.toml

* Update __init__.py

* Update __init__.py

* Fix Triton heuristics

https://github.com/triton-lang/triton/issues/5224

* Update __init__.py

* Update __init__.py

* Update __init__.py

* Update __init__.py

* Xformers

* Update loader.py

* Update loader.py

* Rewind

* Update _utils.py

* Update _utils.py

* requires grad

* Update loader.py

* Update _utils.py

* Update loader.py

* changing model to base_model if peft model is already used

* Improve debugging experience (#1512)

* Create CONTRIBUTING.md (#1472)

Creating contributing guidelines

* Update CONTRIBUTING.md

improved sentence

* Improve logging control in `unsloth_compile_transformers` by conditionally redirecting stdout based on UNSLOTH_DISABLE_LOGGER environment variable

---------

Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
Co-authored-by: Nino Risteski <95188570+NinoRisteski@users.noreply.github.com>

* Update loader.py

* Update llama.py

* Update llama.py

* Revert "Update llama.py"

This reverts commit a8edd0931a.

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Auto change is_bfloat16_supported

* Update llama.py

* Force data-type

* Update llama.py

* All attention refactor fix (#1491)

* change initilization of n_heads, n_kv_heads, hidden_size in llama.py

* do the same for cohere, mistral, gemma2, granite

* do the same for flexattention,cohere, mistral, granite

* Update llama.py

* Update llama.py

* Update granite to work with latest post_patch methods (#1502)

* Update granite to work with latest post_patch methods

* Pass position_embeddings for granite even if transformers<4.47

* Update llama.py

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* Minor fixes for granite models (#1503)

* Update granite.py

Grab residual multiplier directly from layer

* Update llama.py

Version should read >= 4.47.1 as that is the version requiring the changes

* Update granite.py

* Update llama.py

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* support modelscope models and datasets (#1481)

* support modelscope

* change modelscope args

* remove useless import

* remove useless import

* fix

* wip

* fix

* remove useless code

* add readme

* add some comments

* change print to raise error

* update comment

* Update loader.py

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* Merge branch 'main' into nightly

* Phi 4

* Update llama.py

* Torch.Cuda Is Available Condition and Warning (#1545)

* check for torch.cuda and triton if available
on my machine(mac m3) the cuda were not available

* Update pyproject.toml

* Update __init__.py

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* Update mistral.py

* Update mistral.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Fix

* Bug fixes

* Update mapper.py

* Add dropout to granite to match HF's implementation (#1557)

Signed-off-by: datta0 <venkatadattasainimmaturi@gmail.com>

* Update llama.py

* Update llama.py

* Bug fixes

* fix: flash_attn_detection_error (#1556)

* fix: flash_attn_detection_error

* Update _utils.py

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* Update mapper.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* Update gemma.py

* dim fix

* Update _utils.py

* Torch 2.6 support

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Faster inference?

* Update llama.py

* Update llama.py

* Update utils.py

* Update llama.py

* Update llama.py

* Update utils.py

* Update utils.py

* Update utils.py

* Update utils.py

* Update utils.py

* Update utils.py

* Update utils.py

* Update utils.py

* Update utils.py

* Update utils.py

* Update utils.py

* Update utils.py

* Update utils.py

* Update mapper.py

* Fast Inference via vLLM

* Update llama.py

* Update llama.py

* Update utils.py

* Create rl.py

* PatchRL

* Update rl.py

* Update rl.py

* Update rl.py

* PatchRLStatistics

* Update rl.py

* Update rl.py

* Update rl.py

* Update utils.py

* Update utils.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* RL metrics

* Update rl.py

* RL metrics

* Update __init__.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update chat_templates.py

* Update mapper.py

* Fp8 cache

* Update llama.py

* Update llama.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update rl.py

* Update __init__.py

* Update loader.py

* Update rl.py

* Update rl.py

* Update _utils.py

---------

Signed-off-by: datta0 <venkatadattasainimmaturi@gmail.com>
Co-authored-by: Itsuro Tajima <tajima@georepublic.de>
Co-authored-by: Muhammad Osama <muhammadosama1994@gmail.com>
Co-authored-by: Edd <68678137+Erland366@users.noreply.github.com>
Co-authored-by: Michael Han <107991372+shimmyshimmer@users.noreply.github.com>
Co-authored-by: Nino Risteski <95188570+NinoRisteski@users.noreply.github.com>
Co-authored-by: Kareem <81531392+KareemMusleh@users.noreply.github.com>
Co-authored-by: Datta Nimmaturi <datta.nimmaturi@nutanix.com>
Co-authored-by: Z <coffeevampirebusiness@gmail.com>
Co-authored-by: tastelikefeet <58414341+tastelikefeet@users.noreply.github.com>
Co-authored-by: AminWhat <88392440+aminwhat@users.noreply.github.com>
Co-authored-by: Zhe Zhang <2631992879@qq.com>
 2025-02-06 05:08:22 -08:00