Daniel Han
cfc84d83d9
Update _utils.py
2024-10-30 13:51:10 -07:00
Daniel Han
6e6cb3c0da
Update __init__.py
2024-10-30 13:47:21 -07:00
Daniel Han
a919f17648
Update __init__.py
2024-10-30 13:44:05 -07:00
Daniel Han
bedce9ac2c
Update _utils.py
2024-10-30 13:35:01 -07:00
Daniel Han
f88122bd43
Update pyproject.toml
2024-10-30 13:16:00 -07:00
Daniel Han
2d86834813
Bug fixes
2024-10-30 13:11:43 -07:00
Daniel Han
09f667a533
Feat/all tmp ( #1219 )
...
* Update save.py
Check whether path is in /tmp dir for Kaggle environment
* Update save.py
Move temporary_location to /tmp in Kaggle
* Enhance Kaggle environment support in save and tokenizer utilities
---------
Co-authored-by: dendarrion <37800703+dendarrion@users.noreply.github.com>
Co-authored-by: Erland366 <erland.pg366@gmail.com>
2024-10-30 00:43:03 -07:00
Daniel Han
8e30e2e646
Update cross_entropy_loss.py
2024-10-28 15:01:04 -07:00
Daniel Han
d320355de4
Update cross_entropy_loss.py
2024-10-28 14:47:11 -07:00
Daniel Han
1726f04b97
Update cross_entropy_loss.py
2024-10-28 14:30:06 -07:00
Daniel Han
5c669defd5
Update _utils.py
2024-10-28 10:41:31 -07:00
Daniel Han
54ed0fa410
Update _utils.py
2024-10-28 01:12:33 -07:00
Daniel Han
cbbdff23fc
More patching
2024-10-28 01:10:23 -07:00
Daniel Han
38e5b23223
Revert "ignored labels"
...
This reverts commit 4b25138ac7
2024-10-27 22:18:05 -07:00
Daniel Han
4b25138ac7
ignored labels
2024-10-27 22:10:59 -07:00
Daniel Han
e4205ffad5
Typo
2024-10-27 19:09:27 -07:00
Daniel Han
65f754e0f4
Update llama.py
2024-10-27 19:08:14 -07:00
Daniel Han
ac8d5fc3cb
Fix pad token
2024-10-27 19:06:57 -07:00
Daniel Han
6f19b9aecd
Update _utils.py
2024-10-27 17:34:33 -07:00
Daniel Han
d818697900
Unk token issues
2024-10-27 17:32:26 -07:00
Daniel Han
b040e3407d
Merge branch 'main' into nightly
2024-10-27 16:24:20 -07:00
Daniel Han
9e9d6fe660
Merge branch 'main' of https://github.com/unslothai/unsloth
2024-10-27 15:09:42 -07:00
Daniel Han
55fd65a6ed
Update _utils.py
2024-10-27 15:09:35 -07:00
Edd
539fcea071
Fix/casting continue pretraining ( #1200 )
...
* Bring back float32 if float16 instead of bfloat16
* Refactor mixed precision handling for lm_head and embed_tokens to ensure correct dtype usage
* Fix dtype retrieval for embed_tokens and lm_head in mixed precision training
* Fix dtype retrieval for embed_tokens and lm_head to use weight dtype in mixed precision training
* Fix dtype handling for embed_tokens and lm_head to ensure correct float32 usage in mixed precision training
* Fix dtype assignment for lm_head modules to ensure correct weight dtype usage in mixed precision training
2024-10-27 15:06:45 -07:00
Daniel Han
9d5f58224d
Update pyproject.toml
2024-10-26 18:05:55 -07:00
Daniel Han
e7ede2f7db
Torch 2.5
2024-10-26 18:03:15 -07:00
Daniel Han
a4a724bab2
Merge branch 'main' into nightly
2024-10-26 01:22:21 -07:00
Daniel Han
c58dc701c8
Bug fixes ( #1195 )
...
* Fix TRL
* Update mistral.py
* Patch processing_class
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Installation guide (#1165 )
* chore: update chat_templates.py (#1166 )
orginal -> original
* Disable Flex Attention
* Update tokenizer_utils.py
* Update _utils.py
* n_items
* Update cross_entropy_loss.py
* Fix DPO, ORPO
* Update _utils.py
* Update _utils.py
* fix/transformers-unpack (#1180 )
* Fix DPO, ORPO (#1177 )
* Fix TRL
* Update mistral.py
* Patch processing_class
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Installation guide (#1165 )
* chore: update chat_templates.py (#1166 )
orginal -> original
* Disable Flex Attention
* Update tokenizer_utils.py
* Update _utils.py
* n_items
* Update cross_entropy_loss.py
* Fix DPO, ORPO
* Update _utils.py
---------
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
* Add warning for missing Unpack and KwargsForCausalLM in older Transformers versions
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
* Update cross_entropy_loss.py
* Update _utils.py
* Update _utils.py
* donot upcast lm_head and embeddings to float32 (#1186 )
* Cleanup upcast logs (#1188 )
* Fix/phi-longrope (#1193 )
* Enhance rotary embedding handling in LlamaAttention and LongRopeRotaryEmbedding
* Typo
* Improve rotary embedding handling in LlamaAttention to prevent errors with short KV cache
* Update llama.py
* Update llama.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Update transformers
---------
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
Co-authored-by: Edd <68678137+Erland366@users.noreply.github.com>
Co-authored-by: Datta Nimmaturi <datta.nimmaturi@nutanix.com>
2024-10-26 01:21:24 -07:00
Daniel Han
1e8980127c
Update transformers
2024-10-26 01:20:37 -07:00
Edd
c8c4cb3a6d
Fix/phi-longrope ( #1193 )
...
* Enhance rotary embedding handling in LlamaAttention and LongRopeRotaryEmbedding
* Typo
* Improve rotary embedding handling in LlamaAttention to prevent errors with short KV cache
* Update llama.py
* Update llama.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2024-10-25 15:44:10 -07:00
Datta Nimmaturi
1ba5a0161d
Cleanup upcast logs ( #1188 )
2024-10-25 12:17:54 -07:00
Datta Nimmaturi
06050f1802
donot upcast lm_head and embeddings to float32 ( #1186 )
2024-10-25 01:28:12 -07:00
Daniel Han
dcf27bcca7
Merge branch 'main' into nightly
2024-10-24 12:17:57 -07:00
Daniel Han
519c0df00c
Update _utils.py
2024-10-24 12:17:48 -07:00
Daniel Han
06a5c752e3
Fix 4.47 issue ( #1182 )
...
* Fix TRL
* Update mistral.py
* Patch processing_class
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Installation guide (#1165 )
* chore: update chat_templates.py (#1166 )
orginal -> original
* Disable Flex Attention
* Update tokenizer_utils.py
* Update _utils.py
* n_items
* Update cross_entropy_loss.py
* Fix DPO, ORPO
* Update _utils.py
* Update _utils.py
* fix/transformers-unpack (#1180 )
* Fix DPO, ORPO (#1177 )
* Fix TRL
* Update mistral.py
* Patch processing_class
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Installation guide (#1165 )
* chore: update chat_templates.py (#1166 )
orginal -> original
* Disable Flex Attention
* Update tokenizer_utils.py
* Update _utils.py
* n_items
* Update cross_entropy_loss.py
* Fix DPO, ORPO
* Update _utils.py
---------
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
* Add warning for missing Unpack and KwargsForCausalLM in older Transformers versions
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
* Update cross_entropy_loss.py
* Update _utils.py
* Update _utils.py
---------
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
Co-authored-by: Edd <68678137+Erland366@users.noreply.github.com>
2024-10-24 12:17:21 -07:00
Daniel Han
e24b2db194
Update _utils.py
2024-10-24 12:17:09 -07:00
Daniel Han
6f34885c29
Update _utils.py
2024-10-24 12:14:14 -07:00
Daniel Han
8603d08f3b
Update cross_entropy_loss.py
2024-10-24 12:11:38 -07:00
Edd
79effae03d
fix/transformers-unpack ( #1180 )
...
* Fix DPO, ORPO (#1177 )
* Fix TRL
* Update mistral.py
* Patch processing_class
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Installation guide (#1165 )
* chore: update chat_templates.py (#1166 )
orginal -> original
* Disable Flex Attention
* Update tokenizer_utils.py
* Update _utils.py
* n_items
* Update cross_entropy_loss.py
* Fix DPO, ORPO
* Update _utils.py
---------
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
* Add warning for missing Unpack and KwargsForCausalLM in older Transformers versions
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
2024-10-24 12:10:52 -07:00
Daniel Han
e3e4b7dfc3
Update _utils.py
2024-10-24 01:11:20 -07:00
Daniel Han
a6e4a8bf76
Fix DPO, ORPO ( #1177 )
...
* Fix TRL
* Update mistral.py
* Patch processing_class
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Installation guide (#1165 )
* chore: update chat_templates.py (#1166 )
orginal -> original
* Disable Flex Attention
* Update tokenizer_utils.py
* Update _utils.py
* n_items
* Update cross_entropy_loss.py
* Fix DPO, ORPO
* Update _utils.py
---------
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
2024-10-24 00:36:37 -07:00
Daniel Han
0bd8517b1e
Update _utils.py
2024-10-24 00:25:28 -07:00
Daniel Han
d6382ca656
Merge branch 'main' into nightly
2024-10-24 00:24:27 -07:00
Daniel Han
ccf1f946f3
Fix DPO, ORPO
2024-10-24 00:17:26 -07:00
Daniel Han
7fa3179e88
Update cross_entropy_loss.py
2024-10-23 22:18:24 -07:00
Daniel Han
dd8487a63e
n_items
2024-10-23 22:13:45 -07:00
Daniel Han
aa48184c41
Update _utils.py
2024-10-23 12:39:58 -07:00
Edd
da8e547678
Fix/patch tokenizer ( #1171 )
...
* fix: correct tokenizer handling in patch_sft_trainer_tokenizer
* Revert "fix: correct tokenizer handling in patch_sft_trainer_tokenizer"
This reverts commit 7a98e465cbd4f980c8b364b0396d44f2d052090f.
* fix: correct condition for test_text assignment in patch_sft_trainer_tokenizer
2024-10-23 12:32:33 -07:00
Daniel Han
4c85177719
Many bug fixes ( #1162 )
...
* Fix TRL
* Update mistral.py
* Patch processing_class
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Installation guide (#1165 )
* chore: update chat_templates.py (#1166 )
orginal -> original
* Disable Flex Attention
* Update tokenizer_utils.py
* Update _utils.py
---------
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
2024-10-23 03:14:57 -07:00
Daniel Han
8e4cd551e7
Update _utils.py
2024-10-23 03:14:48 -07:00
