Daniel Han
0a6bdf93b5
Update _utils.py
2024-11-06 14:05:52 -08:00
Daniel Han
33874414b8
Update flex_attention.py
2024-11-06 13:09:29 -08:00
Edwin Fennell
08db916009
CLI now handles user input strings for dtype correctly ( #1235 )
...
Co-authored-by: root <root@ieeres.chu.cam.ac.uk>
2024-11-06 12:23:09 -08:00
Datta Nimmaturi
13ab93547e
Throw error when inferencing longer than max_popsition_embeddings ( #1236 )
...
* Throw error when inferencing longer than max_popsition_embeddings without rope scaling
* Update llama.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2024-11-06 12:22:08 -08:00
Edd
8a14fe3f1e
Fix: cast logits to float32 in cross_entropy_forward to prevent errors ( #1254 )
...
* Fix: cast logits to float32 in cross_entropy_forward to prevent errors
* Update cross_entropy_loss.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2024-11-06 12:16:02 -08:00
Daniel Han
ccce1f24e4
Merge branch 'main' into nightly
2024-11-06 12:13:56 -08:00
Daniel Han
4f8bf42442
Bug fixes ( #1255 )
...
* Fix TRL
* Update mistral.py
* Patch processing_class
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Installation guide (#1165 )
* chore: update chat_templates.py (#1166 )
orginal -> original
* Disable Flex Attention
* Update tokenizer_utils.py
* Update _utils.py
* n_items
* Update cross_entropy_loss.py
* Fix DPO, ORPO
* Update _utils.py
* Update _utils.py
* fix/transformers-unpack (#1180 )
* Fix DPO, ORPO (#1177 )
* Fix TRL
* Update mistral.py
* Patch processing_class
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Installation guide (#1165 )
* chore: update chat_templates.py (#1166 )
orginal -> original
* Disable Flex Attention
* Update tokenizer_utils.py
* Update _utils.py
* n_items
* Update cross_entropy_loss.py
* Fix DPO, ORPO
* Update _utils.py
---------
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
* Add warning for missing Unpack and KwargsForCausalLM in older Transformers versions
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
* Update cross_entropy_loss.py
* Update _utils.py
* Update _utils.py
* donot upcast lm_head and embeddings to float32 (#1186 )
* Cleanup upcast logs (#1188 )
* Fix/phi-longrope (#1193 )
* Enhance rotary embedding handling in LlamaAttention and LongRopeRotaryEmbedding
* Typo
* Improve rotary embedding handling in LlamaAttention to prevent errors with short KV cache
* Update llama.py
* Update llama.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Update transformers
* Unk token issues
* Update _utils.py
* Fix pad token
* Update llama.py
* Typo
* ignored labels
* Revert "ignored labels"
This reverts commit 9d07be077b#1219 )
* Update save.py
Check whether path is in /tmp dir for Kaggle environment
* Update save.py
Move temporary_location to /tmp in Kaggle
* Enhance Kaggle environment support in save and tokenizer utilities
---------
Co-authored-by: dendarrion <37800703+dendarrion@users.noreply.github.com>
Co-authored-by: Erland366 <erland.pg366@gmail.com>
* Bug fixes
* Update pyproject.toml
* Update _utils.py
* Update __init__.py
* Update __init__.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Tied weights
* Revert "Tied weights"
This reverts commit 8090b7c01a
2024-11-06 12:08:55 -08:00
Daniel Han
4df520f08e
Update _utils.py
2024-11-06 02:10:37 -08:00
Daniel Han
126b5a3ff7
Update _utils.py
2024-11-06 00:18:04 -08:00
Daniel Han
7cda12fd10
Update _utils.py
2024-11-05 22:48:36 -08:00
Daniel Han
362d187f53
Update _utils.py
2024-11-05 21:44:38 -08:00
Daniel Han
a12cc6b55e
Update _utils.py
2024-11-05 21:24:56 -08:00
Daniel Han
1fa35b1e69
Merge branch 'main' into nightly
2024-11-05 21:24:47 -08:00
Daniel Han
7c684fb793
Bug fix ( #1249 )
...
* Fix TRL
* Update mistral.py
* Patch processing_class
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Installation guide (#1165 )
* chore: update chat_templates.py (#1166 )
orginal -> original
* Disable Flex Attention
* Update tokenizer_utils.py
* Update _utils.py
* n_items
* Update cross_entropy_loss.py
* Fix DPO, ORPO
* Update _utils.py
* Update _utils.py
* fix/transformers-unpack (#1180 )
* Fix DPO, ORPO (#1177 )
* Fix TRL
* Update mistral.py
* Patch processing_class
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Installation guide (#1165 )
* chore: update chat_templates.py (#1166 )
orginal -> original
* Disable Flex Attention
* Update tokenizer_utils.py
* Update _utils.py
* n_items
* Update cross_entropy_loss.py
* Fix DPO, ORPO
* Update _utils.py
---------
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
* Add warning for missing Unpack and KwargsForCausalLM in older Transformers versions
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
* Update cross_entropy_loss.py
* Update _utils.py
* Update _utils.py
* donot upcast lm_head and embeddings to float32 (#1186 )
* Cleanup upcast logs (#1188 )
* Fix/phi-longrope (#1193 )
* Enhance rotary embedding handling in LlamaAttention and LongRopeRotaryEmbedding
* Typo
* Improve rotary embedding handling in LlamaAttention to prevent errors with short KV cache
* Update llama.py
* Update llama.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Update transformers
* Unk token issues
* Update _utils.py
* Fix pad token
* Update llama.py
* Typo
* ignored labels
* Revert "ignored labels"
This reverts commit 9d07be077b#1219 )
* Update save.py
Check whether path is in /tmp dir for Kaggle environment
* Update save.py
Move temporary_location to /tmp in Kaggle
* Enhance Kaggle environment support in save and tokenizer utilities
---------
Co-authored-by: dendarrion <37800703+dendarrion@users.noreply.github.com>
Co-authored-by: Erland366 <erland.pg366@gmail.com>
* Bug fixes
* Update pyproject.toml
* Update _utils.py
* Update __init__.py
* Update __init__.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Tied weights
* Revert "Tied weights"
This reverts commit 8090b7c01a
2024-11-05 21:08:11 -08:00
Daniel Han
1df89aec1e
Update llama.py
2024-11-05 21:07:02 -08:00
Daniel Han
6fed054c2e
Merge branch 'main' into nightly
2024-11-05 21:06:13 -08:00
Daniel Han
3f9b395d48
Update cross_entropy_loss.py
2024-11-05 21:01:33 -08:00
Daniel Han
84cf5e1585
Update cross_entropy_loss.py
2024-11-05 20:58:15 -08:00
Daniel Han
792c473b7d
Update cross_entropy_loss.py
2024-11-05 20:29:30 -08:00
Daniel Han
e9451a36d4
Update _utils.py
2024-11-05 14:54:07 -08:00
Daniel Han
cf2a714d74
Update cross_entropy_loss.py
2024-11-05 14:43:49 -08:00
Daniel Han
27cd2b3a0d
CE Loss
2024-11-05 14:40:57 -08:00
Daniel Han
fef9932bf0
Update llama.py
2024-11-05 13:52:12 -08:00
Daniel Han
92a044197d
Update _utils.py
2024-11-05 13:35:43 -08:00
Daniel Han
15268ba184
Bug fixes ( #1245 )
...
* Fix TRL
* Update mistral.py
* Patch processing_class
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Installation guide (#1165 )
* chore: update chat_templates.py (#1166 )
orginal -> original
* Disable Flex Attention
* Update tokenizer_utils.py
* Update _utils.py
* n_items
* Update cross_entropy_loss.py
* Fix DPO, ORPO
* Update _utils.py
* Update _utils.py
* fix/transformers-unpack (#1180 )
* Fix DPO, ORPO (#1177 )
* Fix TRL
* Update mistral.py
* Patch processing_class
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Installation guide (#1165 )
* chore: update chat_templates.py (#1166 )
orginal -> original
* Disable Flex Attention
* Update tokenizer_utils.py
* Update _utils.py
* n_items
* Update cross_entropy_loss.py
* Fix DPO, ORPO
* Update _utils.py
---------
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
* Add warning for missing Unpack and KwargsForCausalLM in older Transformers versions
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
* Update cross_entropy_loss.py
* Update _utils.py
* Update _utils.py
* donot upcast lm_head and embeddings to float32 (#1186 )
* Cleanup upcast logs (#1188 )
* Fix/phi-longrope (#1193 )
* Enhance rotary embedding handling in LlamaAttention and LongRopeRotaryEmbedding
* Typo
* Improve rotary embedding handling in LlamaAttention to prevent errors with short KV cache
* Update llama.py
* Update llama.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Update transformers
* Unk token issues
* Update _utils.py
* Fix pad token
* Update llama.py
* Typo
* ignored labels
* Revert "ignored labels"
This reverts commit 9d07be077b#1219 )
* Update save.py
Check whether path is in /tmp dir for Kaggle environment
* Update save.py
Move temporary_location to /tmp in Kaggle
* Enhance Kaggle environment support in save and tokenizer utilities
---------
Co-authored-by: dendarrion <37800703+dendarrion@users.noreply.github.com>
Co-authored-by: Erland366 <erland.pg366@gmail.com>
* Bug fixes
* Update pyproject.toml
* Update _utils.py
* Update __init__.py
* Update __init__.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Tied weights
* Revert "Tied weights"
This reverts commit 8090b7c01a
2024-11-05 13:29:37 -08:00
Daniel Han
5bb3e0d462
Update pyproject.toml
2024-11-05 13:28:25 -08:00
Daniel Han
09ad3757c9
Update _utils.py
2024-11-05 12:05:17 -08:00
Daniel Han
0f9b6390a6
Update llama.py
2024-11-05 02:06:27 -08:00
Daniel Han
250109be45
Update llama.py
2024-11-05 01:55:17 -08:00
Daniel Han
116f286d0c
Update _utils.py
2024-11-05 01:52:27 -08:00
Daniel Han
c18bac88be
Update llama.py
2024-11-05 01:42:44 -08:00
Daniel Han
ed61c33d2c
Update _utils.py
2024-11-05 01:41:04 -08:00
Daniel Han
a184f38f7a
Forward hook
2024-11-05 01:36:11 -08:00
Daniel Han
09c1f6d677
Update _utils.py
2024-11-05 01:30:34 -08:00
Daniel Han
b0f010259f
Update _utils.py
2024-11-05 00:15:56 -08:00
Daniel Han
f627c1bf35
Disable compiling
2024-11-05 00:11:32 -08:00
Daniel Han
81a87cc262
types
2024-11-05 00:09:11 -08:00
Daniel Han
4076dc49cb
Update rope_embedding.py
2024-11-05 00:08:16 -08:00
Daniel Han
0ab731d40b
typing
2024-11-05 00:05:56 -08:00
Daniel Han
9384cb9f72
Update rms_layernorm.py
2024-11-04 23:54:02 -08:00
Daniel Han
0b2363bacd
Update rms_layernorm.py
2024-11-04 23:47:38 -08:00
Daniel Han
6836e43f27
Update rms_layernorm.py
2024-11-04 23:45:25 -08:00
Daniel Han
eddf0c46f4
Update rms_layernorm.py
2024-11-04 23:39:03 -08:00
Daniel Han
bf97546a16
Update rms_layernorm.py
2024-11-04 23:34:11 -08:00
Daniel Han
4485a342e0
Update rms_layernorm.py
2024-11-04 23:33:08 -08:00
Daniel Han
34abdb309d
Update rms_layernorm.py
2024-11-04 23:30:55 -08:00
Daniel Han
fd07c2ee73
Update rms_layernorm.py
2024-11-04 23:28:57 -08:00
Daniel Han
c35f115537
Update rms_layernorm.py
2024-11-04 23:27:50 -08:00
Daniel Han
d80444fe3e
Update rms_layernorm.py
2024-11-04 23:24:33 -08:00
Daniel Han
40858e4f76
Update rms_layernorm.py
2024-11-04 23:20:32 -08:00
