ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
736 commits 391 branches 58 tags 304 MiB
Commit graph

736 commits

This branch
This branch
All branches
Author SHA1 Message Date
Daniel Han
5a7eaf8a60 Update _utils.py 2024-11-05 21:44:38 -08:00
Daniel Han
13d7412bbf Update _utils.py 2024-11-05 21:24:56 -08:00
Daniel Han
24ab0d2dea Merge branch 'main' into nightly 2024-11-05 21:24:47 -08:00
Daniel Han
3ea7044fc0
 		Bug fix (#1249) 
* Fix TRL

* Update mistral.py

* Patch processing_class

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Installation guide (#1165)

* chore: update chat_templates.py (#1166)

orginal -> original

* Disable Flex Attention

* Update tokenizer_utils.py

* Update _utils.py

* n_items

* Update cross_entropy_loss.py

* Fix DPO, ORPO

* Update _utils.py

* Update _utils.py

* fix/transformers-unpack (#1180)

* Fix DPO, ORPO (#1177)

* Fix TRL

* Update mistral.py

* Patch processing_class

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Installation guide (#1165)

* chore: update chat_templates.py (#1166)

orginal -> original

* Disable Flex Attention

* Update tokenizer_utils.py

* Update _utils.py

* n_items

* Update cross_entropy_loss.py

* Fix DPO, ORPO

* Update _utils.py

---------

Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>

* Add warning for missing Unpack and KwargsForCausalLM in older Transformers versions

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>

* Update cross_entropy_loss.py

* Update _utils.py

* Update _utils.py

* donot upcast lm_head and embeddings to float32 (#1186)

* Cleanup upcast logs (#1188)

* Fix/phi-longrope (#1193)

* Enhance rotary embedding handling in LlamaAttention and LongRopeRotaryEmbedding

* Typo

* Improve rotary embedding handling in LlamaAttention to prevent errors with short KV cache

* Update llama.py

* Update llama.py

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* Update transformers

* Unk token issues

* Update _utils.py

* Fix pad token

* Update llama.py

* Typo

* ignored labels

* Revert "ignored labels"

This reverts commit 9d07be077b.

* More patching

* Update _utils.py

* Update _utils.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Feat/all tmp (#1219)

* Update save.py

Check whether path is in /tmp dir for Kaggle environment

* Update save.py

Move temporary_location to /tmp in Kaggle

* Enhance Kaggle environment support in save and tokenizer utilities

---------

Co-authored-by: dendarrion <37800703+dendarrion@users.noreply.github.com>
Co-authored-by: Erland366 <erland.pg366@gmail.com>

* Bug fixes

* Update pyproject.toml

* Update _utils.py

* Update __init__.py

* Update __init__.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Tied weights

* Revert "Tied weights"

This reverts commit 8090b7c01a.

* Tied weights

* Utils

* CE Loss patching

* Update __init__.py

* Update __init__.py

* Patching

* Update cross_entropy_loss.py

* CE Loss

* Update _utils.py

* Update _utils.py

* CE Loss

* Update _utils.py

* Update _utils.py

* Layernorm

* Update _utils.py

* Update _utils.py

* Post patch

* Update _utils.py

* Update llama.py

* Update _utils.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* typing

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* int64

* Update _utils.py

* Update cross_entropy_loss.py

* constexpr

* constexpr

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* CE

* Update cross_entropy_loss.py

* Update _utils.py

* Update llama.py

* Update _utils.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update utils.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* typing

* Update rope_embedding.py

* types

* Disable compiling

* Update _utils.py

* Update _utils.py

* Forward hook

* Update _utils.py

* Update llama.py

* Update _utils.py

* Update llama.py

* Update llama.py

* Update _utils.py

* Update pyproject.toml

* Update _utils.py

* Update llama.py

* CE Loss

* Update cross_entropy_loss.py

* Update _utils.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update llama.py

---------

Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
Co-authored-by: Edd <68678137+Erland366@users.noreply.github.com>
Co-authored-by: Datta Nimmaturi <datta.nimmaturi@nutanix.com>
Co-authored-by: dendarrion <37800703+dendarrion@users.noreply.github.com>
Co-authored-by: Erland366 <erland.pg366@gmail.com>
 2024-11-05 21:08:11 -08:00
Daniel Han
ef2c56f3da Update llama.py 2024-11-05 21:07:02 -08:00
Daniel Han
0ed053256d Merge branch 'main' into nightly 2024-11-05 21:06:13 -08:00
Daniel Han
58e541bd91 Update cross_entropy_loss.py 2024-11-05 21:01:33 -08:00
Daniel Han
51fea97dec Update cross_entropy_loss.py 2024-11-05 20:58:15 -08:00
Daniel Han
cab1e722d6 Update cross_entropy_loss.py 2024-11-05 20:29:30 -08:00
Daniel Han
fb393fc9ba Update _utils.py 2024-11-05 14:54:07 -08:00
Daniel Han
aeec57e110 Update cross_entropy_loss.py 2024-11-05 14:43:49 -08:00
Daniel Han
70b65cf7a2 CE Loss 2024-11-05 14:40:57 -08:00
Daniel Han
da397f4f18 Update llama.py 2024-11-05 13:52:12 -08:00
Daniel Han
eb4da9d8cd Update _utils.py 2024-11-05 13:35:43 -08:00
Daniel Han
e2e406eba6
 		Bug fixes (#1245) 
* Fix TRL

* Update mistral.py

* Patch processing_class

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Installation guide (#1165)

* chore: update chat_templates.py (#1166)

orginal -> original

* Disable Flex Attention

* Update tokenizer_utils.py

* Update _utils.py

* n_items

* Update cross_entropy_loss.py

* Fix DPO, ORPO

* Update _utils.py

* Update _utils.py

* fix/transformers-unpack (#1180)

* Fix DPO, ORPO (#1177)

* Fix TRL

* Update mistral.py

* Patch processing_class

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Installation guide (#1165)

* chore: update chat_templates.py (#1166)

orginal -> original

* Disable Flex Attention

* Update tokenizer_utils.py

* Update _utils.py

* n_items

* Update cross_entropy_loss.py

* Fix DPO, ORPO

* Update _utils.py

---------

Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>

* Add warning for missing Unpack and KwargsForCausalLM in older Transformers versions

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>

* Update cross_entropy_loss.py

* Update _utils.py

* Update _utils.py

* donot upcast lm_head and embeddings to float32 (#1186)

* Cleanup upcast logs (#1188)

* Fix/phi-longrope (#1193)

* Enhance rotary embedding handling in LlamaAttention and LongRopeRotaryEmbedding

* Typo

* Improve rotary embedding handling in LlamaAttention to prevent errors with short KV cache

* Update llama.py

* Update llama.py

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* Update transformers

* Unk token issues

* Update _utils.py

* Fix pad token

* Update llama.py

* Typo

* ignored labels

* Revert "ignored labels"

This reverts commit 9d07be077b.

* More patching

* Update _utils.py

* Update _utils.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Feat/all tmp (#1219)

* Update save.py

Check whether path is in /tmp dir for Kaggle environment

* Update save.py

Move temporary_location to /tmp in Kaggle

* Enhance Kaggle environment support in save and tokenizer utilities

---------

Co-authored-by: dendarrion <37800703+dendarrion@users.noreply.github.com>
Co-authored-by: Erland366 <erland.pg366@gmail.com>

* Bug fixes

* Update pyproject.toml

* Update _utils.py

* Update __init__.py

* Update __init__.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Tied weights

* Revert "Tied weights"

This reverts commit 8090b7c01a.

* Tied weights

* Utils

* CE Loss patching

* Update __init__.py

* Update __init__.py

* Patching

* Update cross_entropy_loss.py

* CE Loss

* Update _utils.py

* Update _utils.py

* CE Loss

* Update _utils.py

* Update _utils.py

* Layernorm

* Update _utils.py

* Update _utils.py

* Post patch

* Update _utils.py

* Update llama.py

* Update _utils.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* typing

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* int64

* Update _utils.py

* Update cross_entropy_loss.py

* constexpr

* constexpr

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* CE

* Update cross_entropy_loss.py

* Update _utils.py

* Update llama.py

* Update _utils.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update utils.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* typing

* Update rope_embedding.py

* types

* Disable compiling

* Update _utils.py

* Update _utils.py

* Forward hook

* Update _utils.py

* Update llama.py

* Update _utils.py

* Update llama.py

* Update llama.py

* Update _utils.py

* Update pyproject.toml

---------

Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
Co-authored-by: Edd <68678137+Erland366@users.noreply.github.com>
Co-authored-by: Datta Nimmaturi <datta.nimmaturi@nutanix.com>
Co-authored-by: dendarrion <37800703+dendarrion@users.noreply.github.com>
Co-authored-by: Erland366 <erland.pg366@gmail.com>
 2024-11-05 13:29:37 -08:00
Daniel Han
486d0d6b33 Update pyproject.toml 2024-11-05 13:28:25 -08:00
Daniel Han
648ca59185 Update _utils.py 2024-11-05 12:05:17 -08:00
Daniel Han
536a1a6e52 Update llama.py 2024-11-05 02:06:27 -08:00
Daniel Han
daa7909994 Update llama.py 2024-11-05 01:55:17 -08:00
Daniel Han
ab9f8e1751 Update _utils.py 2024-11-05 01:52:27 -08:00
Daniel Han
745814c423 Update llama.py 2024-11-05 01:42:44 -08:00
Daniel Han
f84cf4be32 Update _utils.py 2024-11-05 01:41:04 -08:00
Daniel Han
34ce5d1dd9 Forward hook 2024-11-05 01:36:11 -08:00
Daniel Han
813cbdd220 Update _utils.py 2024-11-05 01:30:34 -08:00
Daniel Han
c30bd2a2ca Update _utils.py 2024-11-05 00:15:56 -08:00
Daniel Han
f3e2bd6e5e Disable compiling 2024-11-05 00:11:32 -08:00
Daniel Han
98927ee333 types 2024-11-05 00:09:11 -08:00
Daniel Han
74d89d1155 Update rope_embedding.py 2024-11-05 00:08:16 -08:00
Daniel Han
2df35d43e5 typing 2024-11-05 00:05:56 -08:00
Daniel Han
6f38731dfb Update rms_layernorm.py 2024-11-04 23:54:02 -08:00
Daniel Han
bd4ac7b218 Update rms_layernorm.py 2024-11-04 23:47:38 -08:00
Daniel Han
8f825eb284 Update rms_layernorm.py 2024-11-04 23:45:25 -08:00
Daniel Han
3e4c42f79c Update rms_layernorm.py 2024-11-04 23:39:03 -08:00
Daniel Han
ef684f8d23 Update rms_layernorm.py 2024-11-04 23:34:11 -08:00
Daniel Han
ef4b079b87 Update rms_layernorm.py 2024-11-04 23:33:08 -08:00
Daniel Han
14c3d2f900 Update rms_layernorm.py 2024-11-04 23:30:55 -08:00
Daniel Han
beb6854e76 Update rms_layernorm.py 2024-11-04 23:28:57 -08:00
Daniel Han
22da266219 Update rms_layernorm.py 2024-11-04 23:27:50 -08:00
Daniel Han
ae7cb78e91 Update rms_layernorm.py 2024-11-04 23:24:33 -08:00
Daniel Han
ecc1ad223c Update rms_layernorm.py 2024-11-04 23:20:32 -08:00
Daniel Han
2d8d1e1e2d Update rms_layernorm.py 2024-11-04 23:19:02 -08:00
Daniel Han
afc8af69f5 Update utils.py 2024-11-04 23:16:01 -08:00
Daniel Han
2909eaf08e Update rms_layernorm.py 2024-11-04 23:11:03 -08:00
Daniel Han
8496ff69e1 Update rms_layernorm.py 2024-11-04 23:08:05 -08:00
Daniel Han
f7d5c56584 Update rms_layernorm.py 2024-11-04 23:06:13 -08:00
Daniel Han
a3c84a3851 Update rms_layernorm.py 2024-11-04 23:04:34 -08:00
Daniel Han
56af302c45 Update rms_layernorm.py 2024-11-04 23:01:14 -08:00
Daniel Han
b9e1a49d2b Update rms_layernorm.py 2024-11-04 22:38:47 -08:00
Daniel Han
d8103e16b1 Update _utils.py 2024-11-04 22:10:52 -08:00
Daniel Han
f28d7f6f67 Update llama.py 2024-11-04 22:08:49 -08:00