Edd
|
8a6da33bd0
|
Feat/kto (#1316)
* Add PatchKTOTrainer and update model imports
* Update dpo.py
* Update __init__.py
* Delete unsloth/models/kto.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
|
2024-11-26 14:22:02 -08:00 |
|
Daniel Han
|
8bf04040dc
|
Update mapper.py
|
2024-11-26 03:20:21 -08:00 |
|
Daniel Han
|
4e55168525
|
Update _utils.py
|
2024-11-26 03:18:09 -08:00 |
|
Daniel Han
|
d3b147b491
|
Update _utils.py
|
2024-11-26 03:09:32 -08:00 |
|
Daniel Han
|
90f79d2863
|
Update loader.py
|
2024-11-26 03:07:50 -08:00 |
|
Daniel Han
|
207d047acb
|
Update loader.py
|
2024-11-26 02:58:16 -08:00 |
|
Daniel Han
|
6099293e25
|
recheck
|
2024-11-26 02:52:19 -08:00 |
|
Daniel Han
|
587223b4bf
|
prints
|
2024-11-26 00:47:27 -08:00 |
|
Daniel Han
|
ed1c7a9eda
|
Update loader.py
|
2024-11-26 00:35:05 -08:00 |
|
Daniel Han
|
49caeb244a
|
Update loader.py
|
2024-11-26 00:34:51 -08:00 |
|
Daniel Han
|
7321fe9b79
|
Old torch versions
|
2024-11-26 00:26:37 -08:00 |
|
Daniel Han
|
cfb769af32
|
Update loader.py
|
2024-11-26 00:18:00 -08:00 |
|
Daniel Han
|
4e245724fb
|
Update vision.py
|
2024-11-26 00:16:33 -08:00 |
|
Daniel Han
|
833f64df03
|
Update llama.py
|
2024-11-26 00:08:15 -08:00 |
|
Daniel Han
|
5d69df679e
|
Update loader.py
|
2024-11-26 00:01:13 -08:00 |
|
Daniel Han
|
0162d22508
|
Update _utils.py
|
2024-11-25 23:39:26 -08:00 |
|
Daniel Han
|
20e182b606
|
Update _utils.py
|
2024-11-25 23:35:36 -08:00 |
|
Daniel Han
|
8f4f2fe425
|
Update _utils.py
|
2024-11-25 23:33:19 -08:00 |
|
Daniel Han
|
6990917303
|
Update _utils.py
|
2024-11-25 23:03:35 -08:00 |
|
Daniel Han
|
7fef1a99b1
|
Update _utils.py
|
2024-11-25 22:56:24 -08:00 |
|
Daniel Han
|
2c38d3d9ea
|
Update _utils.py
|
2024-11-25 22:52:57 -08:00 |
|
Daniel Han
|
a6ec1e61b2
|
Update _utils.py
|
2024-11-25 22:37:02 -08:00 |
|
Daniel Han
|
b509681885
|
Update _utils.py
|
2024-11-25 22:35:55 -08:00 |
|
Daniel Han
|
071b29b7c4
|
Update _utils.py
|
2024-11-25 22:33:18 -08:00 |
|
Daniel Han
|
23716bfb80
|
Update _utils.py
|
2024-11-25 22:32:27 -08:00 |
|
Daniel Han
|
2febe57a51
|
Update _utils.py
|
2024-11-25 22:31:09 -08:00 |
|
Daniel Han
|
7949ddf050
|
Update _utils.py
|
2024-11-25 22:27:49 -08:00 |
|
Daniel Han
|
5bd41e3036
|
Update _utils.py
|
2024-11-25 22:20:16 -08:00 |
|
Daniel Han
|
7f3adad397
|
Update _utils.py
|
2024-11-25 22:18:33 -08:00 |
|
Daniel Han
|
bfc1c3e5d8
|
Update _utils.py
|
2024-11-25 22:17:20 -08:00 |
|
Daniel Han
|
6595603044
|
error
|
2024-11-25 22:12:22 -08:00 |
|
Daniel Han
|
608e31bd0e
|
Update _utils.py
|
2024-11-25 22:03:08 -08:00 |
|
Daniel Han
|
815576e6b1
|
Update _utils.py
|
2024-11-25 21:59:40 -08:00 |
|
Daniel Han
|
f7278b2624
|
Update _utils.py
|
2024-11-25 21:57:56 -08:00 |
|
Daniel Han
|
f7edb152e7
|
Update llama.py
|
2024-11-25 21:52:42 -08:00 |
|
Daniel Han
|
b93e9cda3b
|
Update llama.py
|
2024-11-25 21:46:36 -08:00 |
|
Daniel Han
|
e17dca4521
|
Update llama.py
|
2024-11-25 21:39:19 -08:00 |
|
Daniel Han
|
2bb066008b
|
logits
|
2024-11-25 21:31:43 -08:00 |
|
Daniel Han
|
80adcd609a
|
kwargs
|
2024-11-25 18:48:41 -08:00 |
|
Daniel Han
|
3782a5975c
|
Update loader.py
|
2024-11-22 16:04:53 -08:00 |
|
Daniel Han
|
482b1fc07d
|
Merge branch 'main' into nightly
|
2024-11-22 15:03:28 -08:00 |
|
Daniel Han
|
228b3cf46e
|
Update pyproject.toml
|
2024-11-21 17:46:22 -08:00 |
|
Daniel Han
|
def921c6bc
|
Delete docs github button.png
|
2024-11-21 11:25:12 -08:00 |
|
Daniel Han
|
d30c363d25
|
Vision (#1318)
* Add files via upload
* Add files via upload
* Add files via upload
* Add files via upload
* Update README.md
* Update README.md
* Update README.md
* Update README.md
---------
Co-authored-by: Michael <107991372+shimmyshimmer@users.noreply.github.com>
|
2024-11-21 11:24:12 -08:00 |
|
Daniel Han
|
c2b185e7db
|
Update _utils.py
|
2024-11-21 06:45:40 -08:00 |
|
Daniel Han
|
69cbd13a41
|
Update vision.py
|
2024-11-21 06:07:06 -08:00 |
|
Daniel Han
|
8b57e17c8d
|
Vision support (#1315)
* Fix pad token
* Update llama.py
* Typo
* ignored labels
* Revert "ignored labels"
This reverts commit 9d07be077b.
* More patching
* Update _utils.py
* Update _utils.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Feat/all tmp (#1219)
* Update save.py
Check whether path is in /tmp dir for Kaggle environment
* Update save.py
Move temporary_location to /tmp in Kaggle
* Enhance Kaggle environment support in save and tokenizer utilities
---------
Co-authored-by: dendarrion <37800703+dendarrion@users.noreply.github.com>
Co-authored-by: Erland366 <erland.pg366@gmail.com>
* Bug fixes
* Update pyproject.toml
* Update _utils.py
* Update __init__.py
* Update __init__.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Tied weights
* Revert "Tied weights"
This reverts commit 8090b7c01a.
* Tied weights
* Utils
* CE Loss patching
* Update __init__.py
* Update __init__.py
* Patching
* Update cross_entropy_loss.py
* CE Loss
* Update _utils.py
* Update _utils.py
* CE Loss
* Update _utils.py
* Update _utils.py
* Layernorm
* Update _utils.py
* Update _utils.py
* Post patch
* Update _utils.py
* Update llama.py
* Update _utils.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* typing
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* int64
* Update _utils.py
* Update cross_entropy_loss.py
* constexpr
* constexpr
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* CE
* Update cross_entropy_loss.py
* Update _utils.py
* Update llama.py
* Update _utils.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update utils.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* Update rms_layernorm.py
* typing
* Update rope_embedding.py
* types
* Disable compiling
* Update _utils.py
* Update _utils.py
* Forward hook
* Update _utils.py
* Update llama.py
* Update _utils.py
* Update llama.py
* Update llama.py
* Update _utils.py
* Update pyproject.toml
* Update _utils.py
* Update llama.py
* CE Loss
* Update cross_entropy_loss.py
* Update _utils.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update cross_entropy_loss.py
* Update llama.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Fix: cast logits to float32 in cross_entropy_forward to prevent errors (#1254)
* Fix: cast logits to float32 in cross_entropy_forward to prevent errors
* Update cross_entropy_loss.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Throw error when inferencing longer than max_popsition_embeddings (#1236)
* Throw error when inferencing longer than max_popsition_embeddings without rope scaling
* Update llama.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* CLI now handles user input strings for dtype correctly (#1235)
Co-authored-by: root <root@ieeres.chu.cam.ac.uk>
* Update flex_attention.py
* Update _utils.py
* Update _utils.py
* Update flex_attention.py
* Update flex_attention.py
* Update loader.py
* Update loader.py
* Update flex_attention.py
* Update flex_attention.py
* Update flex_attention.py
* Update flex_attention.py
* Update _utils.py
* Update cross_entropy_loss.py
* Update _utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* Update tokenizer_utils.py
* triton_cast
* Update utils.py
* Qwen 2.5 Coder
* Fix/export mistral (#1281)
* Enhance install_python_non_blocking to handle protobuf installation and process management
* Revert "Enhance install_python_non_blocking to handle protobuf installation and process management"
This reverts commit f09974b151.
* Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION to 'python' to address issue #1266
* Revert "Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION to 'python' to address issue #1266"
This reverts commit 9fc130785d.
* Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION to 'python' to address issue #1266
* Update __init__.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* DOC Update - Update README.md with os.environ in example (#1269)
* Update README.md with os.environ in example
Added OS Environ in example to avoid device conflicts , for a user at least in jupyter notebook this allows to select GPU in a multi GPU setup.
As currently the unsloth init checks all GPU's and takes the first in the order which can be a issue when some GPU's are in use and the list still shows them. So to manually avoid this, this os config is required.
Small change but a bit time saver for those who straight away copies the tutorials
* Update README.md
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* fix/get_chat_template (#1246)
* Refactor `get_chat_template` to now support system message instead. It supposed to fix ollama tokenizer chattemplate to
* Remove type hinting
* Update chat_templates.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* fix/sft-trainer (#1276)
* Add patch for SFTTrainer to maintain backward compatibility with TRL changes
* Update trainer.py
* Update trainer.py
* Refactor trainer patch to maintain backward compatibility with TRL changes
* Update trainer.py
* Refactor trainer.py to exclude non-convertible trainers from backward compatibility patch
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Update __init__.py
* Update trainer.py
* Update trainer.py
* Update trainer.py
* Update tokenizer_utils.py
* Update llama.py
* Fix #853
* fix/sfttrainer-compatibility (#1293)
* Refactor trainer.py to import SFTConfig directly and update UnslothTrainingArguments class inheritance
* Update trainer.py
* Update trainer.py
---------
Co-authored-by: Daniel Han <danielhanchen@gmail.com>
* Update rms_layernorm.py
* Update rms_layernorm.py
* Gemma
* Update rms_layernorm.py
* Update gemma2.py
* Cut Cross Entropy
* Update llama.py
* Cut Cross Entropy
* Update llama.py
* Update llama.py
* Update llama.py
* Update __init__.py
* Update __init__.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update mapper.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* Update _utils.py
* patch_fast_lora
* vision
* Update fast_lora.py
* Update _utils.py
* Update _utils.py
* Vision
* Update trainer.py
* Update save.py
* FastBaseVisionModel
* Update loader_utils.py
* Update vision.py
* Update loader.py
* Update vision.py
* Update loader.py
* Update vision.py
* Update _utils.py
* tokenizer_name
* Update loader.py
* Update vision.py
* Update save.py
* Update save.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update vision.py
* Update _utils.py
---------
Co-authored-by: dendarrion <37800703+dendarrion@users.noreply.github.com>
Co-authored-by: Erland366 <erland.pg366@gmail.com>
Co-authored-by: Edd <68678137+Erland366@users.noreply.github.com>
Co-authored-by: Datta Nimmaturi <datta.nimmaturi@nutanix.com>
Co-authored-by: Edwin Fennell <edwinfennell1@gmail.com>
Co-authored-by: root <root@ieeres.chu.cam.ac.uk>
Co-authored-by: Uday Girish Maradana <einsteingirish@gmail.com>
|
2024-11-21 05:01:44 -08:00 |
|
Daniel Han
|
535e89958e
|
Update _utils.py
|
2024-11-21 04:13:54 -08:00 |
|
Daniel Han
|
a98fc9c57c
|
Update vision.py
|
2024-11-21 04:10:43 -08:00 |
|
Daniel Han
|
5886ecb246
|
Update vision.py
|
2024-11-21 04:10:24 -08:00 |
|