ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
924 commits 391 branches 58 tags 304 MiB
Commit graph

924 commits

This branch
This branch
All branches
Author SHA1 Message Date
Daniel Han
4361fdeae6 Update save.py 2024-12-03 16:40:36 -08:00
Daniel Han
70893fcd9b Update save.py 2024-12-03 16:35:38 -08:00
Daniel Han
5041f9f124 Update save.py 2024-12-03 16:34:15 -08:00
Daniel Han
cf993d74b6 Update save.py 2024-12-03 16:25:40 -08:00
Daniel Han
1642ded035 Update vision.py 2024-12-03 16:25:13 -08:00
Daniel Han
41a045b220 Update save.py 2024-12-03 16:24:23 -08:00
Daniel Han
56fa57f567 Update save.py 2024-12-03 16:16:28 -08:00
Daniel Han
ae7afa2d34 Fix llama.cpp 2024-12-03 16:03:38 -08:00
Daniel Han
78160ab787 Update llama.py 2024-12-01 02:36:17 -08:00
Daniel Han
f2f6e1dc1f Update llama.py 2024-12-01 02:30:21 -08:00
Daniel Han
39c01fa4fa Update llama.py 2024-12-01 02:29:11 -08:00
Daniel Han
e6aa302e07 Update llama.py 2024-12-01 02:25:48 -08:00
Daniel Han
9dd59ae7ac Update llama.py 2024-12-01 02:22:21 -08:00
Daniel Han
a1b9d74367 Update llama.py 2024-12-01 02:20:54 -08:00
Daniel Han
370e460d8d Update llama.py 2024-12-01 02:19:07 -08:00
Daniel Han
a2b7a5ec45 Update llama.py 2024-12-01 02:15:21 -08:00
Daniel Han
6d468532e1 Update llama.py 2024-12-01 02:13:34 -08:00
Daniel Han
2515d19357 Update llama.py 2024-12-01 02:10:43 -08:00
Daniel Han
aad5b1fcee Update llama.py 2024-12-01 02:02:31 -08:00
Daniel Han
2e5393822b Update vision.py 2024-11-28 00:02:13 -08:00
Daniel Han
d4c06c0200 skip modules 2024-11-28 00:01:25 -08:00
Daniel Han
59946600cd Merge branch 'main' into nightly 2024-11-26 16:40:05 -08:00
cell-dame
98a78ddfe4
 		Fix orpo/dpo trainer (#1286) 
* change the colab notebook for dpo zephyr and orpo

* use original tokenizer

* Update README.md

* Update README.md

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>
 2024-11-26 14:32:06 -08:00
Edd
8a6da33bd0
 		Feat/kto (#1316) 
* Add PatchKTOTrainer and update model imports

* Update dpo.py

* Update __init__.py

* Delete unsloth/models/kto.py

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>
 2024-11-26 14:22:02 -08:00
Daniel Han
8558bc92b0 Update pyproject.toml 2024-11-26 03:29:59 -08:00
Daniel Han
51b384e905
 		Bug fixes for vision (#1340) 
* Update __init__.py

* Update __init__.py

* Patching

* Update cross_entropy_loss.py

* CE Loss

* Update _utils.py

* Update _utils.py

* CE Loss

* Update _utils.py

* Update _utils.py

* Layernorm

* Update _utils.py

* Update _utils.py

* Post patch

* Update _utils.py

* Update llama.py

* Update _utils.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* typing

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* int64

* Update _utils.py

* Update cross_entropy_loss.py

* constexpr

* constexpr

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* CE

* Update cross_entropy_loss.py

* Update _utils.py

* Update llama.py

* Update _utils.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update utils.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* typing

* Update rope_embedding.py

* types

* Disable compiling

* Update _utils.py

* Update _utils.py

* Forward hook

* Update _utils.py

* Update llama.py

* Update _utils.py

* Update llama.py

* Update llama.py

* Update _utils.py

* Update pyproject.toml

* Update _utils.py

* Update llama.py

* CE Loss

* Update cross_entropy_loss.py

* Update _utils.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update llama.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Fix: cast logits to float32 in cross_entropy_forward to prevent errors (#1254)

* Fix: cast logits to float32 in cross_entropy_forward to prevent errors

* Update cross_entropy_loss.py

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* Throw error when inferencing longer than max_popsition_embeddings (#1236)

* Throw error when inferencing longer than max_popsition_embeddings without rope scaling

* Update llama.py

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* CLI now handles user input strings for dtype correctly (#1235)

Co-authored-by: root <root@ieeres.chu.cam.ac.uk>

* Update flex_attention.py

* Update _utils.py

* Update _utils.py

* Update flex_attention.py

* Update flex_attention.py

* Update loader.py

* Update loader.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update _utils.py

* Update cross_entropy_loss.py

* Update _utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* triton_cast

* Update utils.py

* Qwen 2.5 Coder

* Fix/export mistral (#1281)

* Enhance install_python_non_blocking to handle protobuf installation and process management

* Revert "Enhance install_python_non_blocking to handle protobuf installation and process management"

This reverts commit f09974b151.

* Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION to 'python' to address issue #1266

* Revert "Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION to 'python' to address issue #1266"

This reverts commit 9fc130785d.

* Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION to 'python' to address issue #1266

* Update __init__.py

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* DOC Update - Update README.md with os.environ in example (#1269)

* Update README.md with os.environ in example

Added OS Environ in example to avoid device conflicts , for a user at least in jupyter notebook this allows to select GPU in a multi GPU setup. 
As currently the  unsloth init checks all GPU's and takes the first in the order which can be a issue when some GPU's are in use and the list still shows them. So to manually avoid this, this os config is required.
Small change but a bit time saver for those who straight away copies the tutorials

* Update README.md

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* fix/get_chat_template (#1246)

* Refactor `get_chat_template` to now support system message instead. It supposed to fix ollama tokenizer chattemplate to

* Remove type hinting

* Update chat_templates.py

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* fix/sft-trainer (#1276)

* Add patch for SFTTrainer to maintain backward compatibility with TRL changes

* Update trainer.py

* Update trainer.py

* Refactor trainer patch to maintain backward compatibility with TRL changes

* Update trainer.py

* Refactor trainer.py to exclude non-convertible trainers from backward compatibility patch

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* Update __init__.py

* Update trainer.py

* Update trainer.py

* Update trainer.py

* Update tokenizer_utils.py

* Update llama.py

* Fix #853

* fix/sfttrainer-compatibility (#1293)

* Refactor trainer.py to import SFTConfig directly and update UnslothTrainingArguments class inheritance

* Update trainer.py

* Update trainer.py

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* Update rms_layernorm.py

* Update rms_layernorm.py

* Gemma

* Update rms_layernorm.py

* Update gemma2.py

* Cut Cross Entropy

* Update llama.py

* Cut Cross Entropy

* Update llama.py

* Update llama.py

* Update llama.py

* Update __init__.py

* Update __init__.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update mapper.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* patch_fast_lora

* vision

* Update fast_lora.py

* Update _utils.py

* Update _utils.py

* Vision

* Update trainer.py

* Update save.py

* FastBaseVisionModel

* Update loader_utils.py

* Update vision.py

* Update loader.py

* Update vision.py

* Update loader.py

* Update vision.py

* Update _utils.py

* tokenizer_name

* Update loader.py

* Update vision.py

* Update save.py

* Update save.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update _utils.py

* Update loader.py

* kwargs

* logits

* Update llama.py

* Update llama.py

* Update llama.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* error

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update loader.py

* Update llama.py

* Update vision.py

* Update loader.py

* Old torch versions

* Update loader.py

* Update loader.py

* prints

* recheck

* Update loader.py

* Update loader.py

* Update _utils.py

* Update _utils.py

* Update mapper.py

---------

Co-authored-by: Edd <68678137+Erland366@users.noreply.github.com>
Co-authored-by: Datta Nimmaturi <datta.nimmaturi@nutanix.com>
Co-authored-by: Edwin Fennell <edwinfennell1@gmail.com>
Co-authored-by: root <root@ieeres.chu.cam.ac.uk>
Co-authored-by: Uday Girish Maradana <einsteingirish@gmail.com>
 2024-11-26 03:25:55 -08:00
Daniel Han
8bf04040dc Update mapper.py 2024-11-26 03:20:21 -08:00
Daniel Han
4e55168525 Update _utils.py 2024-11-26 03:18:09 -08:00
Daniel Han
d3b147b491 Update _utils.py 2024-11-26 03:09:32 -08:00
Daniel Han
90f79d2863 Update loader.py 2024-11-26 03:07:50 -08:00
Daniel Han
207d047acb Update loader.py 2024-11-26 02:58:16 -08:00
Daniel Han
6099293e25 recheck 2024-11-26 02:52:19 -08:00
Daniel Han
587223b4bf prints 2024-11-26 00:47:27 -08:00
Daniel Han
ed1c7a9eda Update loader.py 2024-11-26 00:35:05 -08:00
Daniel Han
49caeb244a Update loader.py 2024-11-26 00:34:51 -08:00
Daniel Han
7321fe9b79 Old torch versions 2024-11-26 00:26:37 -08:00
Daniel Han
cfb769af32 Update loader.py 2024-11-26 00:18:00 -08:00
Daniel Han
4e245724fb Update vision.py 2024-11-26 00:16:33 -08:00
Daniel Han
833f64df03 Update llama.py 2024-11-26 00:08:15 -08:00
Daniel Han
5d69df679e Update loader.py 2024-11-26 00:01:13 -08:00
Daniel Han
0162d22508 Update _utils.py 2024-11-25 23:39:26 -08:00
Daniel Han
20e182b606 Update _utils.py 2024-11-25 23:35:36 -08:00
Daniel Han
8f4f2fe425 Update _utils.py 2024-11-25 23:33:19 -08:00
Daniel Han
6990917303 Update _utils.py 2024-11-25 23:03:35 -08:00
Daniel Han
7fef1a99b1 Update _utils.py 2024-11-25 22:56:24 -08:00
Daniel Han
2c38d3d9ea Update _utils.py 2024-11-25 22:52:57 -08:00
Daniel Han
a6ec1e61b2 Update _utils.py 2024-11-25 22:37:02 -08:00
Daniel Han
b509681885 Update _utils.py 2024-11-25 22:35:55 -08:00
Daniel Han
071b29b7c4 Update _utils.py 2024-11-25 22:33:18 -08:00
Daniel Han
23716bfb80 Update _utils.py 2024-11-25 22:32:27 -08:00