Daniel Han
d6ccbfb27e
Vision
2024-11-21 01:48:20 -08:00
Daniel Han
1c94f04c9e
Update _utils.py
2024-11-20 19:40:08 -08:00
Daniel Han
4ddd1bb6d3
Update _utils.py
2024-11-20 19:15:05 -08:00
Daniel Han
7d5c9eddee
Update fast_lora.py
2024-11-20 17:07:53 -08:00
Daniel Han
cabf21f0dd
vision
2024-11-20 04:15:53 -08:00
Daniel Han
bd1a17538d
patch_fast_lora
2024-11-20 03:36:41 -08:00
Daniel Han
029f5d5c26
Update _utils.py
2024-11-19 16:56:53 -08:00
Daniel Han
81538c3046
Update _utils.py
2024-11-19 15:49:01 -08:00
Daniel Han
c7c984f725
Update _utils.py
2024-11-19 13:01:38 -08:00
Daniel Han
f92c16d9bc
Update _utils.py
2024-11-19 03:54:35 -08:00
Daniel Han
a0e709b214
Update _utils.py
2024-11-19 03:53:17 -08:00
Daniel Han
4cd14bb9ee
Update _utils.py
2024-11-19 03:50:20 -08:00
Daniel Han
caf4cd40ca
Update _utils.py
2024-11-19 03:46:32 -08:00
Daniel Han
d00dc52902
Update _utils.py
2024-11-19 03:45:23 -08:00
Daniel Han
9f93c49ee7
Update _utils.py
2024-11-19 02:25:22 -08:00
Daniel Han
b9b7a5bf66
Update mapper.py
2024-11-18 12:18:34 -08:00
Daniel Han
b144ff47bf
Update _utils.py
2024-11-17 22:01:20 -08:00
Daniel Han
4abf3deed4
Update _utils.py
2024-11-17 22:01:07 -08:00
Daniel Han
d5c1c171c7
Update _utils.py
2024-11-17 21:31:17 -08:00
Daniel Han
6ab1d3af1b
Update _utils.py
2024-11-17 20:24:10 -08:00
Daniel Han
8eae7f94b3
Update _utils.py
2024-11-17 20:16:33 -08:00
Daniel Han
07ee0da423
Update _utils.py
2024-11-17 17:12:34 -08:00
Daniel Han
2fade27029
Update _utils.py
2024-11-17 16:56:26 -08:00
Daniel Han
4ed6ae8726
Update _utils.py
2024-11-17 16:54:44 -08:00
Daniel Han
42a76f1673
Update __init__.py
2024-11-17 16:13:13 -08:00
Daniel Han
cc314c8d48
Update __init__.py
2024-11-17 16:01:06 -08:00
Daniel Han
49df51f11d
Update llama.py
2024-11-17 16:00:49 -08:00
Daniel Han
59caca96e9
Update llama.py
2024-11-17 15:58:00 -08:00
Daniel Han
0a5c519c8e
Update llama.py
2024-11-17 15:54:12 -08:00
Daniel Han
b18edb9733
Cut Cross Entropy
2024-11-17 14:32:41 -08:00
Daniel Han
4f51d87949
Update llama.py
2024-11-17 00:21:44 -08:00
Daniel Han
1a3d2d5859
Cut Cross Entropy
2024-11-16 23:53:46 -08:00
Daniel Han
9773fee4b8
Update gemma2.py
2024-11-16 15:18:38 -08:00
Daniel Han
50b0aba3d0
Update rms_layernorm.py
2024-11-16 15:00:28 -08:00
Daniel Han
82e44662d9
Gemma
2024-11-16 13:55:11 -08:00
Daniel Han
a6b8dda994
Update rms_layernorm.py
2024-11-16 13:01:54 -08:00
Daniel Han
74382dea47
Update rms_layernorm.py
2024-11-16 12:18:47 -08:00
Edd
a146521a50
fix/sfttrainer-compatibility ( #1293 )
...
* Refactor trainer.py to import SFTConfig directly and update UnslothTrainingArguments class inheritance
* Update trainer.py
* Update trainer.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2024-11-14 17:07:29 -08:00
Daniel Han
5eb971fa4e
Fix #853
2024-11-14 01:26:13 -08:00
Daniel Han
3b11ae7dd2
Update llama.py
2024-11-14 01:11:16 -08:00
Daniel Han
e4d1754c0b
Merge branch 'main' into nightly
2024-11-13 19:07:33 -08:00
Daniel Han
f26d4e739e
Update _utils.py
2024-11-13 19:07:26 -08:00
Daniel Han
0de5457252
Bug fixes ( #1288 )
...
* Fix TRL
* Update mistral.py
* Patch processing_class
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Installation guide (#1165 )
* chore: update chat_templates.py (#1166 )
orginal -> original
* Disable Flex Attention
* Update tokenizer_utils.py
* Update _utils.py
* n_items
* Update cross_entropy_loss.py
* Fix DPO, ORPO
* Update _utils.py
* Update _utils.py
* fix/transformers-unpack (#1180 )
* Fix DPO, ORPO (#1177 )
* Fix TRL
* Update mistral.py
* Patch processing_class
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Installation guide (#1165 )
* chore: update chat_templates.py (#1166 )
orginal -> original
* Disable Flex Attention
* Update tokenizer_utils.py
* Update _utils.py
* n_items
* Update cross_entropy_loss.py
* Fix DPO, ORPO
* Update _utils.py
---------
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
* Add warning for missing Unpack and KwargsForCausalLM in older Transformers versions
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
* Update cross_entropy_loss.py
* Update _utils.py
* Update _utils.py
* donot upcast lm_head and embeddings to float32 (#1186 )
* Cleanup upcast logs (#1188 )
* Fix/phi-longrope (#1193 )
* Enhance rotary embedding handling in LlamaAttention and LongRopeRotaryEmbedding
* Typo
* Improve rotary embedding handling in LlamaAttention to prevent errors with short KV cache
* Update llama.py
* Update llama.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Update transformers
* Unk token issues
* Update _utils.py
* Fix pad token
* Update llama.py
* Typo
* ignored labels
* Revert "ignored labels"
This reverts commit 9d07be077b#1219 )
* Update save.py
Check whether path is in /tmp dir for Kaggle environment
* Update save.py
Move temporary_location to /tmp in Kaggle
* Enhance Kaggle environment support in save and tokenizer utilities
---------
Co-authored-by: dendarrion <37800703+dendarrion@users.noreply.github.com>
Co-authored-by: Erland366 <erland.pg366@gmail.com>
* Bug fixes
* Update pyproject.toml
* Update _utils.py
* Update __init__.py
* Update __init__.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Tied weights
* Revert "Tied weights"
This reverts commit 8090b7c01a#1254 )
* Fix: cast logits to float32 in cross_entropy_forward to prevent errors
* Update cross_entropy_loss.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Throw error when inferencing longer than max_popsition_embeddings (#1236 )
* Throw error when inferencing longer than max_popsition_embeddings without rope scaling
* Update llama.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* CLI now handles user input strings for dtype correctly (#1235 )
Co-authored-by: root <root@ieeres.chu.cam.ac.uk>
* Update flex_attention.py
* Update _utils.py
* Update _utils.py
* Update flex_attention.py
* Update flex_attention.py
* Update loader.py
* Update loader.py
* Update flex_attention.py
* Update flex_attention.py
* Update flex_attention.py
* Update flex_attention.py
* Update _utils.py
* Update cross_entropy_loss.py
* Update _utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* triton_cast
* Update utils.py
* Qwen 2.5 Coder
* Fix/export mistral (#1281 )
* Enhance install_python_non_blocking to handle protobuf installation and process management
* Revert "Enhance install_python_non_blocking to handle protobuf installation and process management"
This reverts commit f09974b151#1266
* Revert "Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION to 'python' to address issue #1266 "
This reverts commit 9fc130785d#1266
* Update __init__.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* DOC Update - Update README.md with os.environ in example (#1269 )
* Update README.md with os.environ in example
Added OS Environ in example to avoid device conflicts , for a user at least in jupyter notebook this allows to select GPU in a multi GPU setup.
As currently the unsloth init checks all GPU's and takes the first in the order which can be a issue when some GPU's are in use and the list still shows them. So to manually avoid this, this os config is required.
Small change but a bit time saver for those who straight away copies the tutorials
* Update README.md
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* fix/get_chat_template (#1246 )
* Refactor `get_chat_template` to now support system message instead. It supposed to fix ollama tokenizer chattemplate to
* Remove type hinting
* Update chat_templates.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* fix/sft-trainer (#1276 )
* Add patch for SFTTrainer to maintain backward compatibility with TRL changes
* Update trainer.py
* Update trainer.py
* Refactor trainer patch to maintain backward compatibility with TRL changes
* Update trainer.py
* Refactor trainer.py to exclude non-convertible trainers from backward compatibility patch
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Update __init__.py
* Update trainer.py
* Update trainer.py
* Update trainer.py
* Update tokenizer_utils.py
---------
Co-authored-by: timothelaborie <97834767+timothelaborie@users.noreply.github.com>
Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com>
Co-authored-by: Edd <68678137+Erland366@users.noreply.github.com>
Co-authored-by: Datta Nimmaturi <datta.nimmaturi@nutanix.com>
Co-authored-by: dendarrion <37800703+dendarrion@users.noreply.github.com>
Co-authored-by: Erland366 <erland.pg366@gmail.com>
Co-authored-by: Edwin Fennell <edwinfennell1@gmail.com>
Co-authored-by: root <root@ieeres.chu.cam.ac.uk>
Co-authored-by: Uday Girish Maradana <einsteingirish@gmail.com>
2024-11-13 19:05:40 -08:00
Daniel Han
11b350f7c1
Update tokenizer_utils.py
2024-11-13 19:05:15 -08:00
Daniel Han
597169c14b
Update trainer.py
2024-11-13 18:53:40 -08:00
Daniel Han
035bccee75
Update trainer.py
2024-11-13 18:48:59 -08:00
Daniel Han
a31027c9e4
Update trainer.py
2024-11-13 18:44:54 -08:00
Daniel Han
84d6d36cde
Update __init__.py
2024-11-13 17:38:26 -08:00
Edd
dc0232c883
fix/sft-trainer ( #1276 )
...
* Add patch for SFTTrainer to maintain backward compatibility with TRL changes
* Update trainer.py
* Update trainer.py
* Refactor trainer patch to maintain backward compatibility with TRL changes
* Update trainer.py
* Refactor trainer.py to exclude non-convertible trainers from backward compatibility patch
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2024-11-13 17:33:30 -08:00
Edd
10565efe27
fix/get_chat_template ( #1246 )
...
* Refactor `get_chat_template` to now support system message instead. It supposed to fix ollama tokenizer chattemplate to
* Remove type hinting
* Update chat_templates.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
2024-11-13 00:06:48 -08:00
