Daniel Han
d9f042d7e0
vision
2024-11-20 04:15:53 -08:00
Daniel Han
81cd49d2e7
patch_fast_lora
2024-11-20 03:36:41 -08:00
Daniel Han
6c3b7f0e32
Update _utils.py
2024-11-19 16:56:53 -08:00
Daniel Han
65a5049423
Update _utils.py
2024-11-19 15:49:01 -08:00
Daniel Han
e5b2f577de
Update _utils.py
2024-11-19 13:01:38 -08:00
Daniel Han
56a19d82de
Update _utils.py
2024-11-19 03:54:35 -08:00
Daniel Han
1f62b73677
Update _utils.py
2024-11-19 03:53:17 -08:00
Daniel Han
f8ccb5758a
Update _utils.py
2024-11-19 03:50:20 -08:00
Daniel Han
d67bd4cfb7
Update _utils.py
2024-11-19 03:46:32 -08:00
Daniel Han
699a9ff81e
Update _utils.py
2024-11-19 03:45:23 -08:00
Daniel Han
80f9f6a225
Update _utils.py
2024-11-19 02:25:22 -08:00
Daniel Han
ee98b75c06
Update mapper.py
2024-11-18 12:18:34 -08:00
Daniel Han
5f59faf526
Update _utils.py
2024-11-17 22:01:20 -08:00
Daniel Han
096a77d9e6
Update _utils.py
2024-11-17 22:01:07 -08:00
Daniel Han
0e77184c23
Update _utils.py
2024-11-17 21:31:17 -08:00
Daniel Han
c8082e46aa
Update _utils.py
2024-11-17 20:24:10 -08:00
Daniel Han
26546e68b7
Update _utils.py
2024-11-17 20:16:33 -08:00
Daniel Han
4b30c7a89b
Update _utils.py
2024-11-17 17:12:34 -08:00
Daniel Han
db1c5f414a
Update _utils.py
2024-11-17 16:56:26 -08:00
Daniel Han
d8c6c3e903
Update _utils.py
2024-11-17 16:54:44 -08:00
Daniel Han
ccf033893e
Update __init__.py
2024-11-17 16:13:13 -08:00
Daniel Han
3c2794ecee
Update __init__.py
2024-11-17 16:01:06 -08:00
Daniel Han
73bbd9e795
Update llama.py
2024-11-17 16:00:49 -08:00
Daniel Han
df62b6242d
Update llama.py
2024-11-17 15:58:00 -08:00
Daniel Han
05fb970edd
Update llama.py
2024-11-17 15:54:12 -08:00
Daniel Han
fa8e59eb1b
Cut Cross Entropy
2024-11-17 14:32:41 -08:00
Daniel Han
1dc066afda
Update llama.py
2024-11-17 00:21:44 -08:00
Daniel Han
c4eacf50da
Cut Cross Entropy
2024-11-16 23:53:46 -08:00
Daniel Han
e7ad484169
Update gemma2.py
2024-11-16 15:18:38 -08:00
Daniel Han
d47d838ee8
Update rms_layernorm.py
2024-11-16 15:00:28 -08:00
Daniel Han
263eaaa27f
Gemma
2024-11-16 13:55:11 -08:00
Daniel Han
e49a4d9277
Update rms_layernorm.py
2024-11-16 13:01:54 -08:00
Daniel Han
2cf0203166
Update rms_layernorm.py
2024-11-16 12:18:47 -08:00
Edd
b69fee4a36
fix/sfttrainer-compatibility ( #1293 )
...
* Refactor trainer.py to import SFTConfig directly and update UnslothTrainingArguments class inheritance
* Update trainer.py
* Update trainer.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2024-11-14 17:07:29 -08:00
Daniel Han
786aea6365
Fix #853
2024-11-14 01:26:13 -08:00
Daniel Han
8e899bf956
Update llama.py
2024-11-14 01:11:16 -08:00
Daniel Han
686a97d750
Merge branch 'main' into nightly
2024-11-13 19:07:33 -08:00
Daniel Han
892115606d
Update _utils.py
2024-11-13 19:07:26 -08:00
Daniel Han
2dca0cb94b
Bug fixes ( #1288 )
...
* Fix TRL
* Update mistral.py
* Patch processing_class
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Installation guide (#1165 )
* chore: update chat_templates.py (#1166 )
orginal -> original
* Disable Flex Attention
* Update tokenizer_utils.py
* Update _utils.py
* n_items
* Update cross_entropy_loss.py
* Fix DPO, ORPO
* Update _utils.py
* Update _utils.py
* fix/transformers-unpack (#1180 )
* Fix DPO, ORPO (#1177 )
* Fix TRL
* Update mistral.py
* Patch processing_class
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Installation guide (#1165 )
* chore: update chat_templates.py (#1166 )
orginal -> original
* Disable Flex Attention
* Update tokenizer_utils.py
* Update _utils.py
* n_items
* Update cross_entropy_loss.py
* Fix DPO, ORPO
* Update _utils.py
---------
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
* Add warning for missing Unpack and KwargsForCausalLM in older Transformers versions
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
* Update cross_entropy_loss.py
* Update _utils.py
* Update _utils.py
* donot upcast lm_head and embeddings to float32 (#1186 )
* Cleanup upcast logs (#1188 )
* Fix/phi-longrope (#1193 )
* Enhance rotary embedding handling in LlamaAttention and LongRopeRotaryEmbedding
* Typo
* Improve rotary embedding handling in LlamaAttention to prevent errors with short KV cache
* Update llama.py
* Update llama.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Update transformers
* Unk token issues
* Update _utils.py
* Fix pad token
* Update llama.py
* Typo
* ignored labels
* Revert "ignored labels"
This reverts commit 4b25138ac7#1219 )
* Update save.py
Check whether path is in /tmp dir for Kaggle environment
* Update save.py
Move temporary_location to /tmp in Kaggle
* Enhance Kaggle environment support in save and tokenizer utilities
---------
Co-authored-by: dendarrion <37800703+dendarrion@users.noreply.github.com>
Co-authored-by: Erland366 <erland.pg366@gmail.com>
* Bug fixes
* Update pyproject.toml
* Update _utils.py
* Update __init__.py
* Update __init__.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Tied weights
* Revert "Tied weights"
This reverts commit 820cd4efef#1254 )
* Fix: cast logits to float32 in cross_entropy_forward to prevent errors
* Update cross_entropy_loss.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Throw error when inferencing longer than max_popsition_embeddings (#1236 )
* Throw error when inferencing longer than max_popsition_embeddings without rope scaling
* Update llama.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* CLI now handles user input strings for dtype correctly (#1235 )
Co-authored-by: root <root@ieeres.chu.cam.ac.uk>
* Update flex_attention.py
* Update _utils.py
* Update _utils.py
* Update flex_attention.py
* Update flex_attention.py
* Update loader.py
* Update loader.py
* Update flex_attention.py
* Update flex_attention.py
* Update flex_attention.py
* Update flex_attention.py
* Update _utils.py
* Update cross_entropy_loss.py
* Update _utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* triton_cast
* Update utils.py
* Qwen 2.5 Coder
* Fix/export mistral (#1281 )
* Enhance install_python_non_blocking to handle protobuf installation and process management
* Revert "Enhance install_python_non_blocking to handle protobuf installation and process management"
This reverts commit a3b796a05841fb8d93c652c845591e12cf81ea93.
* Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION to 'python' to address issue #1266
* Revert "Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION to 'python' to address issue #1266 "
This reverts commit f00fbf5eac7ad4f5d48c70b98d770255d1a9ef58.
* Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION to 'python' to address issue #1266
* Update __init__.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* DOC Update - Update README.md with os.environ in example (#1269 )
* Update README.md with os.environ in example
Added OS Environ in example to avoid device conflicts , for a user at least in jupyter notebook this allows to select GPU in a multi GPU setup.
As currently the unsloth init checks all GPU's and takes the first in the order which can be a issue when some GPU's are in use and the list still shows them. So to manually avoid this, this os config is required.
Small change but a bit time saver for those who straight away copies the tutorials
* Update README.md
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* fix/get_chat_template (#1246 )
* Refactor `get_chat_template` to now support system message instead. It supposed to fix ollama tokenizer chattemplate to
* Remove type hinting
* Update chat_templates.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* fix/sft-trainer (#1276 )
* Add patch for SFTTrainer to maintain backward compatibility with TRL changes
* Update trainer.py
* Update trainer.py
* Refactor trainer patch to maintain backward compatibility with TRL changes
* Update trainer.py
* Refactor trainer.py to exclude non-convertible trainers from backward compatibility patch
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Update __init__.py
* Update trainer.py
* Update trainer.py
* Update trainer.py
* Update tokenizer_utils.py
---------
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
Co-authored-by: Edd <68678137+Erland366@users.noreply.github.com>
Co-authored-by: Datta Nimmaturi <datta.nimmaturi@nutanix.com>
Co-authored-by: dendarrion <37800703+dendarrion@users.noreply.github.com>
Co-authored-by: Erland366 <erland.pg366@gmail.com>
Co-authored-by: Edwin Fennell <edwinfennell1@gmail.com>
Co-authored-by: root <root@ieeres.chu.cam.ac.uk>
Co-authored-by: Uday Girish Maradana <einsteingirish@gmail.com>
2024-11-13 19:05:40 -08:00
Daniel Han
f554e663ec
Update tokenizer_utils.py
2024-11-13 19:05:15 -08:00
Daniel Han
2e4bca5cbf
Update trainer.py
2024-11-13 18:53:40 -08:00
Daniel Han
022d571835
Update trainer.py
2024-11-13 18:48:59 -08:00
Daniel Han
33c85a3bd0
Update trainer.py
2024-11-13 18:44:54 -08:00
Daniel Han
cec6e570a8
Update __init__.py
2024-11-13 17:38:26 -08:00
Edd
cad6df52c5
fix/sft-trainer ( #1276 )
...
* Add patch for SFTTrainer to maintain backward compatibility with TRL changes
* Update trainer.py
* Update trainer.py
* Refactor trainer patch to maintain backward compatibility with TRL changes
* Update trainer.py
* Refactor trainer.py to exclude non-convertible trainers from backward compatibility patch
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2024-11-13 17:33:30 -08:00
Edd
cb3608b72d
fix/get_chat_template ( #1246 )
...
* Refactor `get_chat_template` to now support system message instead. It supposed to fix ollama tokenizer chattemplate to
* Remove type hinting
* Update chat_templates.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2024-11-13 00:06:48 -08:00
Uday Girish Maradana
b230fa13eb
DOC Update - Update README.md with os.environ in example ( #1269 )
...
* Update README.md with os.environ in example
Added OS Environ in example to avoid device conflicts , for a user at least in jupyter notebook this allows to select GPU in a multi GPU setup.
As currently the unsloth init checks all GPU's and takes the first in the order which can be a issue when some GPU's are in use and the list still shows them. So to manually avoid this, this os config is required.
Small change but a bit time saver for those who straight away copies the tutorials
* Update README.md
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2024-11-12 23:55:28 -08:00
Edd
fb9a3ca1a1
Fix/export mistral ( #1281 )
...
* Enhance install_python_non_blocking to handle protobuf installation and process management
* Revert "Enhance install_python_non_blocking to handle protobuf installation and process management"
This reverts commit a3b796a05841fb8d93c652c845591e12cf81ea93.
* Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION to 'python' to address issue #1266
* Revert "Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION to 'python' to address issue #1266 "
This reverts commit f00fbf5eac7ad4f5d48c70b98d770255d1a9ef58.
* Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION to 'python' to address issue #1266
* Update __init__.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2024-11-12 23:53:50 -08:00
Daniel Han
c94fa058b0
Merge branch 'main' into nightly
2024-11-12 23:51:46 -08:00
Daniel Han
6007831cef
Update _utils.py
2024-11-12 10:54:58 -08:00
