Daniel Han
56ed3fbae1
Vision
2024-11-21 01:48:20 -08:00
Daniel Han
a00218b618
Update _utils.py
2024-11-20 19:40:08 -08:00
Daniel Han
877cada353
Update _utils.py
2024-11-20 19:15:05 -08:00
Daniel Han
93338ce246
Update fast_lora.py
2024-11-20 17:07:53 -08:00
Daniel Han
17bbba3dad
vision
2024-11-20 04:15:53 -08:00
Daniel Han
99e07e7c90
patch_fast_lora
2024-11-20 03:36:41 -08:00
Daniel Han
e8232f8a0c
Update _utils.py
2024-11-19 16:56:53 -08:00
Daniel Han
5bcc615d31
Update _utils.py
2024-11-19 15:49:01 -08:00
Daniel Han
7a45b84b6a
Update _utils.py
2024-11-19 13:01:38 -08:00
Daniel Han
d222c6485b
Update _utils.py
2024-11-19 03:54:35 -08:00
Daniel Han
f4ed54fbf6
Update _utils.py
2024-11-19 03:53:17 -08:00
Daniel Han
5e912ed110
Update _utils.py
2024-11-19 03:50:20 -08:00
Daniel Han
a1b7dcca50
Update _utils.py
2024-11-19 03:46:32 -08:00
Daniel Han
590b3067d3
Update _utils.py
2024-11-19 03:45:23 -08:00
Daniel Han
b3e5f72aea
Update _utils.py
2024-11-19 02:25:22 -08:00
Daniel Han
4a845151ef
Update mapper.py
2024-11-18 12:18:34 -08:00
Daniel Han
363ad32293
Update _utils.py
2024-11-17 22:01:20 -08:00
Daniel Han
9b735eb109
Update _utils.py
2024-11-17 22:01:07 -08:00
Daniel Han
8b8951d710
Update _utils.py
2024-11-17 21:31:17 -08:00
Daniel Han
0950053705
Update _utils.py
2024-11-17 20:24:10 -08:00
Daniel Han
fcea6dc7d9
Update _utils.py
2024-11-17 20:16:33 -08:00
Daniel Han
1c4545cdab
Update _utils.py
2024-11-17 17:12:34 -08:00
Daniel Han
aacbcedb6c
Update _utils.py
2024-11-17 16:56:26 -08:00
Daniel Han
50c94b7954
Update _utils.py
2024-11-17 16:54:44 -08:00
Daniel Han
4e98f6b16c
Update __init__.py
2024-11-17 16:13:13 -08:00
Daniel Han
673ca18e7b
Update __init__.py
2024-11-17 16:01:06 -08:00
Daniel Han
b592220e5c
Update llama.py
2024-11-17 16:00:49 -08:00
Daniel Han
bf7258824e
Update llama.py
2024-11-17 15:58:00 -08:00
Daniel Han
beba773ac1
Update llama.py
2024-11-17 15:54:12 -08:00
Daniel Han
a98dbd36f0
Cut Cross Entropy
2024-11-17 14:32:41 -08:00
Daniel Han
ee4d47c095
Update llama.py
2024-11-17 00:21:44 -08:00
Daniel Han
f940326484
Cut Cross Entropy
2024-11-16 23:53:46 -08:00
Daniel Han
6af0ece0c1
Update gemma2.py
2024-11-16 15:18:38 -08:00
Daniel Han
5f20a6cdee
Update rms_layernorm.py
2024-11-16 15:00:28 -08:00
Daniel Han
5adc847202
Gemma
2024-11-16 13:55:11 -08:00
Daniel Han
8bed15160d
Update rms_layernorm.py
2024-11-16 13:01:54 -08:00
Daniel Han
a8defc897b
Update rms_layernorm.py
2024-11-16 12:18:47 -08:00
Edd
125c5fc777
fix/sfttrainer-compatibility ( #1293 )
...
* Refactor trainer.py to import SFTConfig directly and update UnslothTrainingArguments class inheritance
* Update trainer.py
* Update trainer.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2024-11-14 17:07:29 -08:00
Daniel Han
323f1f7fc2
Fix #853
2024-11-14 01:26:13 -08:00
Daniel Han
0bb588bf8d
Update llama.py
2024-11-14 01:11:16 -08:00
Daniel Han
0045ee20b8
Merge branch 'main' into nightly
2024-11-13 19:07:33 -08:00
Daniel Han
d1dda6d396
Update _utils.py
2024-11-13 19:07:26 -08:00
Daniel Han
443878a3b0
Bug fixes ( #1288 )
...
* Fix TRL
* Update mistral.py
* Patch processing_class
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Installation guide (#1165 )
* chore: update chat_templates.py (#1166 )
orginal -> original
* Disable Flex Attention
* Update tokenizer_utils.py
* Update _utils.py
* n_items
* Update cross_entropy_loss.py
* Fix DPO, ORPO
* Update _utils.py
* Update _utils.py
* fix/transformers-unpack (#1180 )
* Fix DPO, ORPO (#1177 )
* Fix TRL
* Update mistral.py
* Patch processing_class
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Installation guide (#1165 )
* chore: update chat_templates.py (#1166 )
orginal -> original
* Disable Flex Attention
* Update tokenizer_utils.py
* Update _utils.py
* n_items
* Update cross_entropy_loss.py
* Fix DPO, ORPO
* Update _utils.py
---------
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
* Add warning for missing Unpack and KwargsForCausalLM in older Transformers versions
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
* Update cross_entropy_loss.py
* Update _utils.py
* Update _utils.py
* donot upcast lm_head and embeddings to float32 (#1186 )
* Cleanup upcast logs (#1188 )
* Fix/phi-longrope (#1193 )
* Enhance rotary embedding handling in LlamaAttention and LongRopeRotaryEmbedding
* Typo
* Improve rotary embedding handling in LlamaAttention to prevent errors with short KV cache
* Update llama.py
* Update llama.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Update transformers
* Unk token issues
* Update _utils.py
* Fix pad token
* Update llama.py
* Typo
* ignored labels
* Revert "ignored labels"
This reverts commit 110ee41971#1219 )
* Update save.py
Check whether path is in /tmp dir for Kaggle environment
* Update save.py
Move temporary_location to /tmp in Kaggle
* Enhance Kaggle environment support in save and tokenizer utilities
---------
Co-authored-by: dendarrion <37800703+dendarrion@users.noreply.github.com>
Co-authored-by: Erland366 <erland.pg366@gmail.com>
* Bug fixes
* Update pyproject.toml
* Update _utils.py
* Update __init__.py
* Update __init__.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Tied weights
* Revert "Tied weights"
This reverts commit 42bb212916#1254 )
* Fix: cast logits to float32 in cross_entropy_forward to prevent errors
* Update cross_entropy_loss.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Throw error when inferencing longer than max_popsition_embeddings (#1236 )
* Throw error when inferencing longer than max_popsition_embeddings without rope scaling
* Update llama.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* CLI now handles user input strings for dtype correctly (#1235 )
Co-authored-by: root <root@ieeres.chu.cam.ac.uk>
* Update flex_attention.py
* Update _utils.py
* Update _utils.py
* Update flex_attention.py
* Update flex_attention.py
* Update loader.py
* Update loader.py
* Update flex_attention.py
* Update flex_attention.py
* Update flex_attention.py
* Update flex_attention.py
* Update _utils.py
* Update cross_entropy_loss.py
* Update _utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* triton_cast
* Update utils.py
* Qwen 2.5 Coder
* Fix/export mistral (#1281 )
* Enhance install_python_non_blocking to handle protobuf installation and process management
* Revert "Enhance install_python_non_blocking to handle protobuf installation and process management"
This reverts commit a3b796a05841fb8d93c652c845591e12cf81ea93.
* Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION to 'python' to address issue #1266
* Revert "Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION to 'python' to address issue #1266 "
This reverts commit f00fbf5eac7ad4f5d48c70b98d770255d1a9ef58.
* Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION to 'python' to address issue #1266
* Update __init__.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* DOC Update - Update README.md with os.environ in example (#1269 )
* Update README.md with os.environ in example
Added OS Environ in example to avoid device conflicts , for a user at least in jupyter notebook this allows to select GPU in a multi GPU setup.
As currently the unsloth init checks all GPU's and takes the first in the order which can be a issue when some GPU's are in use and the list still shows them. So to manually avoid this, this os config is required.
Small change but a bit time saver for those who straight away copies the tutorials
* Update README.md
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* fix/get_chat_template (#1246 )
* Refactor `get_chat_template` to now support system message instead. It supposed to fix ollama tokenizer chattemplate to
* Remove type hinting
* Update chat_templates.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* fix/sft-trainer (#1276 )
* Add patch for SFTTrainer to maintain backward compatibility with TRL changes
* Update trainer.py
* Update trainer.py
* Refactor trainer patch to maintain backward compatibility with TRL changes
* Update trainer.py
* Refactor trainer.py to exclude non-convertible trainers from backward compatibility patch
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Update __init__.py
* Update trainer.py
* Update trainer.py
* Update trainer.py
* Update tokenizer_utils.py
---------
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
Co-authored-by: Edd <68678137+Erland366@users.noreply.github.com>
Co-authored-by: Datta Nimmaturi <datta.nimmaturi@nutanix.com>
Co-authored-by: dendarrion <37800703+dendarrion@users.noreply.github.com>
Co-authored-by: Erland366 <erland.pg366@gmail.com>
Co-authored-by: Edwin Fennell <edwinfennell1@gmail.com>
Co-authored-by: root <root@ieeres.chu.cam.ac.uk>
Co-authored-by: Uday Girish Maradana <einsteingirish@gmail.com>
2024-11-13 19:05:40 -08:00
Daniel Han
b3ac5569b1
Update tokenizer_utils.py
2024-11-13 19:05:15 -08:00
Daniel Han
7333bcb7ea
Update trainer.py
2024-11-13 18:53:40 -08:00
Daniel Han
9dae15349e
Update trainer.py
2024-11-13 18:48:59 -08:00
Daniel Han
617f7c0c2f
Update trainer.py
2024-11-13 18:44:54 -08:00
Daniel Han
a97ba4c6a4
Update __init__.py
2024-11-13 17:38:26 -08:00
Edd
c2e7ecf6f1
fix/sft-trainer ( #1276 )
...
* Add patch for SFTTrainer to maintain backward compatibility with TRL changes
* Update trainer.py
* Update trainer.py
* Refactor trainer patch to maintain backward compatibility with TRL changes
* Update trainer.py
* Refactor trainer.py to exclude non-convertible trainers from backward compatibility patch
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2024-11-13 17:33:30 -08:00
Edd
8545b34ba0
fix/get_chat_template ( #1246 )
...
* Refactor `get_chat_template` to now support system message instead. It supposed to fix ollama tokenizer chattemplate to
* Remove type hinting
* Update chat_templates.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2024-11-13 00:06:48 -08:00
