ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
1,177 commits 391 branches 58 tags 304 MiB
Commit graph

1,177 commits

This branch
This branch
All branches
Author SHA1 Message Date
Daniel Han
ffc70682f4 Update _utils.py 2024-11-05 14:54:07 -08:00
Daniel Han
3616850a9f Update cross_entropy_loss.py 2024-11-05 14:43:49 -08:00
Daniel Han
e66c0eb3ab CE Loss 2024-11-05 14:40:57 -08:00
Daniel Han
e635bc4ea2 Update llama.py 2024-11-05 13:52:12 -08:00
Daniel Han
ed56a1daaa Update _utils.py 2024-11-05 13:35:43 -08:00
Daniel Han
4806f58aab Bug fixes (#1245) 
* Fix TRL

* Update mistral.py

* Patch processing_class

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Installation guide (#1165)

* chore: update chat_templates.py (#1166)

orginal -> original

* Disable Flex Attention

* Update tokenizer_utils.py

* Update _utils.py

* n_items

* Update cross_entropy_loss.py

* Fix DPO, ORPO

* Update _utils.py

* Update _utils.py

* fix/transformers-unpack (#1180)

* Fix DPO, ORPO (#1177)

* Fix TRL

* Update mistral.py

* Patch processing_class

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Installation guide (#1165)

* chore: update chat_templates.py (#1166)

orginal -> original

* Disable Flex Attention

* Update tokenizer_utils.py

* Update _utils.py

* n_items

* Update cross_entropy_loss.py

* Fix DPO, ORPO

* Update _utils.py

---------

Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>

* Add warning for missing Unpack and KwargsForCausalLM in older Transformers versions

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>

* Update cross_entropy_loss.py

* Update _utils.py

* Update _utils.py

* donot upcast lm_head and embeddings to float32 (#1186)

* Cleanup upcast logs (#1188)

* Fix/phi-longrope (#1193)

* Enhance rotary embedding handling in LlamaAttention and LongRopeRotaryEmbedding

* Typo

* Improve rotary embedding handling in LlamaAttention to prevent errors with short KV cache

* Update llama.py

* Update llama.py

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* Update transformers

* Unk token issues

* Update _utils.py

* Fix pad token

* Update llama.py

* Typo

* ignored labels

* Revert "ignored labels"

This reverts commit 4b25138ac7.

* More patching

* Update _utils.py

* Update _utils.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Feat/all tmp (#1219)

* Update save.py

Check whether path is in /tmp dir for Kaggle environment

* Update save.py

Move temporary_location to /tmp in Kaggle

* Enhance Kaggle environment support in save and tokenizer utilities

---------

Co-authored-by: dendarrion <37800703+dendarrion@users.noreply.github.com>
Co-authored-by: Erland366 <erland.pg366@gmail.com>

* Bug fixes

* Update pyproject.toml

* Update _utils.py

* Update __init__.py

* Update __init__.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Tied weights

* Revert "Tied weights"

This reverts commit 820cd4efef.

* Tied weights

* Utils

* CE Loss patching

* Update __init__.py

* Update __init__.py

* Patching

* Update cross_entropy_loss.py

* CE Loss

* Update _utils.py

* Update _utils.py

* CE Loss

* Update _utils.py

* Update _utils.py

* Layernorm

* Update _utils.py

* Update _utils.py

* Post patch

* Update _utils.py

* Update llama.py

* Update _utils.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* typing

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* int64

* Update _utils.py

* Update cross_entropy_loss.py

* constexpr

* constexpr

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* CE

* Update cross_entropy_loss.py

* Update _utils.py

* Update llama.py

* Update _utils.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update utils.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* typing

* Update rope_embedding.py

* types

* Disable compiling

* Update _utils.py

* Update _utils.py

* Forward hook

* Update _utils.py

* Update llama.py

* Update _utils.py

* Update llama.py

* Update llama.py

* Update _utils.py

* Update pyproject.toml

---------

Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
Co-authored-by: Edd <68678137+Erland366@users.noreply.github.com>
Co-authored-by: Datta Nimmaturi <datta.nimmaturi@nutanix.com>
Co-authored-by: dendarrion <37800703+dendarrion@users.noreply.github.com>
Co-authored-by: Erland366 <erland.pg366@gmail.com>
 2024-11-05 13:29:37 -08:00
Daniel Han
102d8cccbf Update pyproject.toml 2024-11-05 13:28:25 -08:00
Daniel Han
8d6d3b49c2 Update _utils.py 2024-11-05 12:05:17 -08:00
Daniel Han
cc0c4d824a Update llama.py 2024-11-05 02:06:27 -08:00
Daniel Han
e1ed9c85a0 Update llama.py 2024-11-05 01:55:17 -08:00
Daniel Han
302fe3e8fe Update _utils.py 2024-11-05 01:52:27 -08:00
Daniel Han
e61a58a7a6 Update llama.py 2024-11-05 01:42:44 -08:00
Daniel Han
7e596ea1be Update _utils.py 2024-11-05 01:41:04 -08:00
Daniel Han
44b3006678 Forward hook 2024-11-05 01:36:11 -08:00
Daniel Han
34209e9160 Update _utils.py 2024-11-05 01:30:34 -08:00
Daniel Han
f207f39d63 Update _utils.py 2024-11-05 00:15:56 -08:00
Daniel Han
d53599209d Disable compiling 2024-11-05 00:11:32 -08:00
Daniel Han
501e71f759 types 2024-11-05 00:09:11 -08:00
Daniel Han
62ed303345 Update rope_embedding.py 2024-11-05 00:08:16 -08:00
Daniel Han
15dbd771ec typing 2024-11-05 00:05:56 -08:00
Daniel Han
16eb66ae47 Update rms_layernorm.py 2024-11-04 23:54:02 -08:00
Daniel Han
db36694eb6 Update rms_layernorm.py 2024-11-04 23:47:38 -08:00
Daniel Han
938f657781 Update rms_layernorm.py 2024-11-04 23:45:25 -08:00
Daniel Han
e1fecc0b73 Update rms_layernorm.py 2024-11-04 23:39:03 -08:00
Daniel Han
f8d6966e2a Update rms_layernorm.py 2024-11-04 23:34:11 -08:00
Daniel Han
2de9091c04 Update rms_layernorm.py 2024-11-04 23:33:08 -08:00
Daniel Han
fa039eb57b Update rms_layernorm.py 2024-11-04 23:30:55 -08:00
Daniel Han
98630427b8 Update rms_layernorm.py 2024-11-04 23:28:57 -08:00
Daniel Han
e5f6ba3f27 Update rms_layernorm.py 2024-11-04 23:27:50 -08:00
Daniel Han
d966cbd5ea Update rms_layernorm.py 2024-11-04 23:24:33 -08:00
Daniel Han
71a3333c0f Update rms_layernorm.py 2024-11-04 23:20:32 -08:00
Daniel Han
aa5668049a Update rms_layernorm.py 2024-11-04 23:19:02 -08:00
Daniel Han
943cc1c3df Update utils.py 2024-11-04 23:16:01 -08:00
Daniel Han
2cd4b8debd Update rms_layernorm.py 2024-11-04 23:11:03 -08:00
Daniel Han
32fc5a59a0 Update rms_layernorm.py 2024-11-04 23:08:05 -08:00
Daniel Han
3fd823bc6c Update rms_layernorm.py 2024-11-04 23:06:13 -08:00
Daniel Han
8a0a732735 Update rms_layernorm.py 2024-11-04 23:04:34 -08:00
Daniel Han
da874b1292 Update rms_layernorm.py 2024-11-04 23:01:14 -08:00
Daniel Han
8df69d8e51 Update rms_layernorm.py 2024-11-04 22:38:47 -08:00
Daniel Han
980314a744 Update _utils.py 2024-11-04 22:10:52 -08:00
Daniel Han
f102728460 Update llama.py 2024-11-04 22:08:49 -08:00
Daniel Han
980e6cbb5e Update _utils.py 2024-11-04 21:47:48 -08:00
Daniel Han
1192c5d7b1 Update cross_entropy_loss.py 2024-11-04 19:48:09 -08:00
Daniel Han
e5e1027904 CE 2024-11-04 19:45:10 -08:00
Daniel Han
8ba738c75d Update _utils.py 2024-11-04 17:56:16 -08:00
Daniel Han
b1d9a2bd72 Update _utils.py 2024-11-04 01:23:24 -08:00
Daniel Han
3cb3c83023 Update _utils.py 2024-11-04 01:20:31 -08:00
Daniel Han
b2cdb409bd Update cross_entropy_loss.py 2024-11-04 00:32:35 -08:00
Daniel Han
e1b8dcf536 Update cross_entropy_loss.py 2024-11-04 00:07:52 -08:00
Daniel Han
e7be62a1ba constexpr 2024-11-04 00:05:51 -08:00