Daniel Han
558efd0899
Merge branch 'main' into nightly
2024-10-26 01:22:21 -07:00
Daniel Han
4b02021f68
Bug fixes ( #1195 )
...
* Fix TRL
* Update mistral.py
* Patch processing_class
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Installation guide (#1165 )
* chore: update chat_templates.py (#1166 )
orginal -> original
* Disable Flex Attention
* Update tokenizer_utils.py
* Update _utils.py
* n_items
* Update cross_entropy_loss.py
* Fix DPO, ORPO
* Update _utils.py
* Update _utils.py
* fix/transformers-unpack (#1180 )
* Fix DPO, ORPO (#1177 )
* Fix TRL
* Update mistral.py
* Patch processing_class
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Installation guide (#1165 )
* chore: update chat_templates.py (#1166 )
orginal -> original
* Disable Flex Attention
* Update tokenizer_utils.py
* Update _utils.py
* n_items
* Update cross_entropy_loss.py
* Fix DPO, ORPO
* Update _utils.py
---------
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
* Add warning for missing Unpack and KwargsForCausalLM in older Transformers versions
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
* Update cross_entropy_loss.py
* Update _utils.py
* Update _utils.py
* donot upcast lm_head and embeddings to float32 (#1186 )
* Cleanup upcast logs (#1188 )
* Fix/phi-longrope (#1193 )
* Enhance rotary embedding handling in LlamaAttention and LongRopeRotaryEmbedding
* Typo
* Improve rotary embedding handling in LlamaAttention to prevent errors with short KV cache
* Update llama.py
* Update llama.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Update transformers
---------
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
Co-authored-by: Edd <68678137+Erland366@users.noreply.github.com>
Co-authored-by: Datta Nimmaturi <datta.nimmaturi@nutanix.com>
2024-10-26 01:21:24 -07:00
Daniel Han
49f427ddbe
Update transformers
2024-10-26 01:20:37 -07:00
Edd
e771f2a932
Fix/phi-longrope ( #1193 )
...
* Enhance rotary embedding handling in LlamaAttention and LongRopeRotaryEmbedding
* Typo
* Improve rotary embedding handling in LlamaAttention to prevent errors with short KV cache
* Update llama.py
* Update llama.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2024-10-25 15:44:10 -07:00
Datta Nimmaturi
7bb0fd96f6
Cleanup upcast logs ( #1188 )
2024-10-25 12:17:54 -07:00
Datta Nimmaturi
ffe4ba1a43
donot upcast lm_head and embeddings to float32 ( #1186 )
2024-10-25 01:28:12 -07:00
Daniel Han
2c1c1016c2
Merge branch 'main' into nightly
2024-10-24 12:17:57 -07:00
Daniel Han
828ebf815a
Update _utils.py
2024-10-24 12:17:48 -07:00
Daniel Han
dfdff91258
Fix 4.47 issue ( #1182 )
...
* Fix TRL
* Update mistral.py
* Patch processing_class
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Installation guide (#1165 )
* chore: update chat_templates.py (#1166 )
orginal -> original
* Disable Flex Attention
* Update tokenizer_utils.py
* Update _utils.py
* n_items
* Update cross_entropy_loss.py
* Fix DPO, ORPO
* Update _utils.py
* Update _utils.py
* fix/transformers-unpack (#1180 )
* Fix DPO, ORPO (#1177 )
* Fix TRL
* Update mistral.py
* Patch processing_class
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Installation guide (#1165 )
* chore: update chat_templates.py (#1166 )
orginal -> original
* Disable Flex Attention
* Update tokenizer_utils.py
* Update _utils.py
* n_items
* Update cross_entropy_loss.py
* Fix DPO, ORPO
* Update _utils.py
---------
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
* Add warning for missing Unpack and KwargsForCausalLM in older Transformers versions
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
* Update cross_entropy_loss.py
* Update _utils.py
* Update _utils.py
---------
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
Co-authored-by: Edd <68678137+Erland366@users.noreply.github.com>
2024-10-24 12:17:21 -07:00
Daniel Han
3ba142fd62
Update _utils.py
2024-10-24 12:17:09 -07:00
Daniel Han
10f3eedaf7
Update _utils.py
2024-10-24 12:14:14 -07:00
Daniel Han
6e0fa4ec2b
Update cross_entropy_loss.py
2024-10-24 12:11:38 -07:00
Edd
bd6ed7343a
fix/transformers-unpack ( #1180 )
...
* Fix DPO, ORPO (#1177 )
* Fix TRL
* Update mistral.py
* Patch processing_class
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Installation guide (#1165 )
* chore: update chat_templates.py (#1166 )
orginal -> original
* Disable Flex Attention
* Update tokenizer_utils.py
* Update _utils.py
* n_items
* Update cross_entropy_loss.py
* Fix DPO, ORPO
* Update _utils.py
---------
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
* Add warning for missing Unpack and KwargsForCausalLM in older Transformers versions
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
2024-10-24 12:10:52 -07:00
Daniel Han
ddde04459c
Update _utils.py
2024-10-24 01:11:20 -07:00
Daniel Han
6a90ba64b7
Fix DPO, ORPO ( #1177 )
...
* Fix TRL
* Update mistral.py
* Patch processing_class
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Installation guide (#1165 )
* chore: update chat_templates.py (#1166 )
orginal -> original
* Disable Flex Attention
* Update tokenizer_utils.py
* Update _utils.py
* n_items
* Update cross_entropy_loss.py
* Fix DPO, ORPO
* Update _utils.py
---------
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
2024-10-24 00:36:37 -07:00
Daniel Han
83c2c564b6
Update _utils.py
2024-10-24 00:25:28 -07:00
Daniel Han
7555e3979c
Merge branch 'main' into nightly
2024-10-24 00:24:27 -07:00
Daniel Han
cd1615dc64
Fix DPO, ORPO
2024-10-24 00:17:26 -07:00
Daniel Han
a048a791f4
Update cross_entropy_loss.py
2024-10-23 22:18:24 -07:00
Daniel Han
8390144201
n_items
2024-10-23 22:13:45 -07:00
Daniel Han
acef7d6a3d
Update _utils.py
2024-10-23 12:39:58 -07:00
Edd
f402d945ff
Fix/patch tokenizer ( #1171 )
...
* fix: correct tokenizer handling in patch_sft_trainer_tokenizer
* Revert "fix: correct tokenizer handling in patch_sft_trainer_tokenizer"
This reverts commit 7a98e465cbd4f980c8b364b0396d44f2d052090f.
* fix: correct condition for test_text assignment in patch_sft_trainer_tokenizer
2024-10-23 12:32:33 -07:00
Daniel Han
e5e67f83b4
Many bug fixes ( #1162 )
...
* Fix TRL
* Update mistral.py
* Patch processing_class
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Installation guide (#1165 )
* chore: update chat_templates.py (#1166 )
orginal -> original
* Disable Flex Attention
* Update tokenizer_utils.py
* Update _utils.py
---------
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
2024-10-23 03:14:57 -07:00
Daniel Han
93cee86d9c
Update _utils.py
2024-10-23 03:14:48 -07:00
Daniel Han
785a964f20
Update tokenizer_utils.py
2024-10-23 03:04:22 -07:00
Daniel Han
205b0437b3
Disable Flex Attention
2024-10-23 02:58:40 -07:00
Ikko Eltociear Ashimine
a8f79d46c6
chore: update chat_templates.py ( #1166 )
...
orginal -> original
2024-10-23 00:59:02 -07:00
timothelaborie
db0ce5d033
Installation guide ( #1165 )
2024-10-23 00:55:26 -07:00
Daniel Han
45b2689ad6
Update tokenizer_utils.py
2024-10-22 01:28:38 -07:00
Daniel Han
8f0a165a53
Update tokenizer_utils.py
2024-10-22 01:22:13 -07:00
Daniel Han
aaa151b15a
Update tokenizer_utils.py
2024-10-22 01:09:20 -07:00
Daniel Han
11e63948e5
Update tokenizer_utils.py
2024-10-22 01:05:41 -07:00
Daniel Han
a952e13d6a
Update tokenizer_utils.py
2024-10-22 00:57:36 -07:00
Daniel Han
010f24b3a7
Update tokenizer_utils.py
2024-10-22 00:55:47 -07:00
Daniel Han
e5e543c244
Patch processing_class
2024-10-22 00:53:38 -07:00
Daniel Han
d5becd080e
Update mistral.py
2024-10-22 00:29:30 -07:00
Daniel Han
1ca9a03ab7
Fix TRL
2024-10-21 01:02:53 -07:00
Daniel Han
c3c9e1604b
Update save.py
2024-10-20 01:52:21 -07:00
Daniel Han
251c3f9af2
Update _utils.py
2024-10-18 23:10:30 -07:00
Daniel Han
ac1c0fbaed
Fix get_token
2024-10-18 23:08:30 -07:00
vo1d-ai
5dcc0cd613
fix: compute_loss bug ( #1151 )
...
Currently, Unsloth doesn't pass additional parameters to Trainer.compute_loss such as return_outputs. This leads to errors when calling trainer.evaluate(). This change fixes the bug by properly passing parameters to Trainer.compute_loss.
2024-10-18 20:46:07 -07:00
Daniel Han
94087e83d9
Update _utils.py
2024-10-17 20:50:05 -07:00
Daniel Han
f14e49d10f
Update README.md
2024-10-17 20:46:11 -07:00
Daniel Han
a158a15965
Update README.md
2024-10-17 20:45:40 -07:00
Daniel Han
5e0a4bede8
Gradient Accumulation Fix ( #1146 )
...
* Unsloth Zoo
* Update trainer.py
* Update trainer.py
* Update cross_entropy_loss.py
* n_items
* Update llama.py
* kwargs
* Remove extraneous f prefixes (#1133 )
Co-authored-by: Emil Sadek <esadek@users.noreply.github.com>
* Update __init__.py
* kwargs
* Update trainer.py
* Update trainer.py
* Update trainer.py
* Fix GA
* Update _utils.py
* Update llama.py
* Update tokenizer_utils.py
* Warn on old versions
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
---------
Co-authored-by: Emil Sadek <esadek@hotmail.com>
Co-authored-by: Emil Sadek <esadek@users.noreply.github.com>
2024-10-17 20:43:07 -07:00
Daniel Han
98d68dae2b
Update mapper.py
2024-10-16 21:48:05 -07:00
Daniel Han
c05e87f18d
Gradient Accumulation Fix ( #1134 )
...
* Unsloth Zoo
* Update trainer.py
* Update trainer.py
* Update cross_entropy_loss.py
* n_items
* Update llama.py
* kwargs
* Remove extraneous f prefixes (#1133 )
Co-authored-by: Emil Sadek <esadek@users.noreply.github.com>
* Update __init__.py
---------
Co-authored-by: Emil Sadek <esadek@hotmail.com>
Co-authored-by: Emil Sadek <esadek@users.noreply.github.com>
2024-10-14 19:17:35 -07:00
Daniel Han
f75a92d14d
Update save.py
2024-10-11 00:00:06 -07:00
Daniel Han
71ac721cec
Update save.py
2024-10-10 23:22:17 -07:00
Giulia Baldini
4ed744ef16
Only remove folder in sentenpiece check if it was created ( #1121 )
2024-10-10 23:21:27 -07:00
