ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
1,177 commits 391 branches 58 tags 304 MiB
Commit graph

1,177 commits

This branch
This branch
All branches
Author SHA1 Message Date
Michael Han
717dc4e0e2 Merge pull request #1374 from unslothai/shimmyshimmer-patch-1 
Update README.md
 2024-12-03 16:52:21 -08:00
Michael Han
f0390236e9 Update README.md 
Fixing Qwen links
 2024-12-03 16:50:52 -08:00
Daniel Han
c4687c8ebe Update save.py 2024-12-03 16:40:43 -08:00
Daniel Han
59e127546c Update save.py 2024-12-03 16:40:36 -08:00
Daniel Han
b3b9d6c486 Update save.py 2024-12-03 16:35:38 -08:00
Daniel Han
767d08422a Update save.py 2024-12-03 16:34:15 -08:00
Daniel Han
9b583fc157 Update save.py 2024-12-03 16:25:40 -08:00
Daniel Han
684a0774cf Update vision.py 2024-12-03 16:25:13 -08:00
Daniel Han
feaf087a71 Update save.py 2024-12-03 16:24:23 -08:00
Daniel Han
90db16c3d2 Update save.py 2024-12-03 16:16:28 -08:00
Daniel Han
1dbb5fab5c Fix llama.cpp 2024-12-03 16:03:38 -08:00
Daniel Han
cd60ce1638 Update llama.py 2024-12-01 02:36:17 -08:00
Daniel Han
d7965b0446 Update llama.py 2024-12-01 02:30:21 -08:00
Daniel Han
cba8ffaeaa Update llama.py 2024-12-01 02:29:11 -08:00
Daniel Han
32396a1435 Update llama.py 2024-12-01 02:25:48 -08:00
Daniel Han
11e7bf14b8 Update llama.py 2024-12-01 02:22:21 -08:00
Daniel Han
e327044acc Update llama.py 2024-12-01 02:20:54 -08:00
Daniel Han
95a46a6ff5 Update llama.py 2024-12-01 02:19:07 -08:00
Daniel Han
43028257df Update llama.py 2024-12-01 02:15:21 -08:00
Daniel Han
82aa368151 Update llama.py 2024-12-01 02:13:34 -08:00
Daniel Han
4b512bac6c Update llama.py 2024-12-01 02:10:43 -08:00
Daniel Han
b51c1ab908 Update llama.py 2024-12-01 02:02:31 -08:00
Daniel Han
10c6c2f5fe Update vision.py 2024-11-28 00:02:13 -08:00
Daniel Han
5246a649bb skip modules 2024-11-28 00:01:25 -08:00
Daniel Han
b4d1f443a1 Merge branch 'main' into nightly 2024-11-26 16:40:05 -08:00
cell-dame
d8b3e50668 Fix orpo/dpo trainer (#1286) 
* change the colab notebook for dpo zephyr and orpo

* use original tokenizer

* Update README.md

* Update README.md

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>
 2024-11-26 14:32:06 -08:00
Edd
407d707ced Feat/kto (#1316) 
* Add PatchKTOTrainer and update model imports

* Update dpo.py

* Update __init__.py

* Delete unsloth/models/kto.py

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>
 2024-11-26 14:22:02 -08:00
Daniel Han
a3287090af Update pyproject.toml 2024-11-26 03:29:59 -08:00
Daniel Han
e1349b6de6 Bug fixes for vision (#1340) 
* Update __init__.py

* Update __init__.py

* Patching

* Update cross_entropy_loss.py

* CE Loss

* Update _utils.py

* Update _utils.py

* CE Loss

* Update _utils.py

* Update _utils.py

* Layernorm

* Update _utils.py

* Update _utils.py

* Post patch

* Update _utils.py

* Update llama.py

* Update _utils.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* typing

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* int64

* Update _utils.py

* Update cross_entropy_loss.py

* constexpr

* constexpr

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* CE

* Update cross_entropy_loss.py

* Update _utils.py

* Update llama.py

* Update _utils.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update utils.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* typing

* Update rope_embedding.py

* types

* Disable compiling

* Update _utils.py

* Update _utils.py

* Forward hook

* Update _utils.py

* Update llama.py

* Update _utils.py

* Update llama.py

* Update llama.py

* Update _utils.py

* Update pyproject.toml

* Update _utils.py

* Update llama.py

* CE Loss

* Update cross_entropy_loss.py

* Update _utils.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update llama.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Fix: cast logits to float32 in cross_entropy_forward to prevent errors (#1254)

* Fix: cast logits to float32 in cross_entropy_forward to prevent errors

* Update cross_entropy_loss.py

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* Throw error when inferencing longer than max_popsition_embeddings (#1236)

* Throw error when inferencing longer than max_popsition_embeddings without rope scaling

* Update llama.py

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* CLI now handles user input strings for dtype correctly (#1235)

Co-authored-by: root <root@ieeres.chu.cam.ac.uk>

* Update flex_attention.py

* Update _utils.py

* Update _utils.py

* Update flex_attention.py

* Update flex_attention.py

* Update loader.py

* Update loader.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update _utils.py

* Update cross_entropy_loss.py

* Update _utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* triton_cast

* Update utils.py

* Qwen 2.5 Coder

* Fix/export mistral (#1281)

* Enhance install_python_non_blocking to handle protobuf installation and process management

* Revert "Enhance install_python_non_blocking to handle protobuf installation and process management"

This reverts commit a3b796a05841fb8d93c652c845591e12cf81ea93.

* Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION to 'python' to address issue #1266

* Revert "Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION to 'python' to address issue #1266"

This reverts commit f00fbf5eac7ad4f5d48c70b98d770255d1a9ef58.

* Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION to 'python' to address issue #1266

* Update __init__.py

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* DOC Update - Update README.md with os.environ in example (#1269)

* Update README.md with os.environ in example

Added OS Environ in example to avoid device conflicts , for a user at least in jupyter notebook this allows to select GPU in a multi GPU setup. 
As currently the  unsloth init checks all GPU's and takes the first in the order which can be a issue when some GPU's are in use and the list still shows them. So to manually avoid this, this os config is required.
Small change but a bit time saver for those who straight away copies the tutorials

* Update README.md

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* fix/get_chat_template (#1246)

* Refactor `get_chat_template` to now support system message instead. It supposed to fix ollama tokenizer chattemplate to

* Remove type hinting

* Update chat_templates.py

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* fix/sft-trainer (#1276)

* Add patch for SFTTrainer to maintain backward compatibility with TRL changes

* Update trainer.py

* Update trainer.py

* Refactor trainer patch to maintain backward compatibility with TRL changes

* Update trainer.py

* Refactor trainer.py to exclude non-convertible trainers from backward compatibility patch

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* Update __init__.py

* Update trainer.py

* Update trainer.py

* Update trainer.py

* Update tokenizer_utils.py

* Update llama.py

* Fix #853

* fix/sfttrainer-compatibility (#1293)

* Refactor trainer.py to import SFTConfig directly and update UnslothTrainingArguments class inheritance

* Update trainer.py

* Update trainer.py

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* Update rms_layernorm.py

* Update rms_layernorm.py

* Gemma

* Update rms_layernorm.py

* Update gemma2.py

* Cut Cross Entropy

* Update llama.py

* Cut Cross Entropy

* Update llama.py

* Update llama.py

* Update llama.py

* Update __init__.py

* Update __init__.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update mapper.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* patch_fast_lora

* vision

* Update fast_lora.py

* Update _utils.py

* Update _utils.py

* Vision

* Update trainer.py

* Update save.py

* FastBaseVisionModel

* Update loader_utils.py

* Update vision.py

* Update loader.py

* Update vision.py

* Update loader.py

* Update vision.py

* Update _utils.py

* tokenizer_name

* Update loader.py

* Update vision.py

* Update save.py

* Update save.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update vision.py

* Update _utils.py

* Update loader.py

* kwargs

* logits

* Update llama.py

* Update llama.py

* Update llama.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* error

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update loader.py

* Update llama.py

* Update vision.py

* Update loader.py

* Old torch versions

* Update loader.py

* Update loader.py

* prints

* recheck

* Update loader.py

* Update loader.py

* Update _utils.py

* Update _utils.py

* Update mapper.py

---------

Co-authored-by: Edd <68678137+Erland366@users.noreply.github.com>
Co-authored-by: Datta Nimmaturi <datta.nimmaturi@nutanix.com>
Co-authored-by: Edwin Fennell <edwinfennell1@gmail.com>
Co-authored-by: root <root@ieeres.chu.cam.ac.uk>
Co-authored-by: Uday Girish Maradana <einsteingirish@gmail.com>
 2024-11-26 03:25:55 -08:00
Daniel Han
441e463c2f Update mapper.py 2024-11-26 03:20:21 -08:00
Daniel Han
7785763b39 Update _utils.py 2024-11-26 03:18:09 -08:00
Daniel Han
4e7470fff6 Update _utils.py 2024-11-26 03:09:32 -08:00
Daniel Han
43875ad4be Update loader.py 2024-11-26 03:07:50 -08:00
Daniel Han
212879720d Update loader.py 2024-11-26 02:58:16 -08:00
Daniel Han
b3f7d54686 recheck 2024-11-26 02:52:19 -08:00
Daniel Han
7a22da6348 prints 2024-11-26 00:47:27 -08:00
Daniel Han
442472f8f1 Update loader.py 2024-11-26 00:35:05 -08:00
Daniel Han
2a9fd4c4c2 Update loader.py 2024-11-26 00:34:51 -08:00
Daniel Han
99c738055a Old torch versions 2024-11-26 00:26:37 -08:00
Daniel Han
77fc0c96c3 Update loader.py 2024-11-26 00:18:00 -08:00
Daniel Han
34ea5bc89f Update vision.py 2024-11-26 00:16:33 -08:00
Daniel Han
41f3d0845e Update llama.py 2024-11-26 00:08:15 -08:00
Daniel Han
3db56503ff Update loader.py 2024-11-26 00:01:13 -08:00
Daniel Han
54566530cf Update _utils.py 2024-11-25 23:39:26 -08:00
Daniel Han
1a9d39a778 Update _utils.py 2024-11-25 23:35:36 -08:00
Daniel Han
6e22722076 Update _utils.py 2024-11-25 23:33:19 -08:00
Daniel Han
fde2079deb Update _utils.py 2024-11-25 23:03:35 -08:00
Daniel Han
b3339806c6 Update _utils.py 2024-11-25 22:56:24 -08:00
Daniel Han
1657a364ab Update _utils.py 2024-11-25 22:52:57 -08:00
Daniel Han
9362b96845 Update _utils.py 2024-11-25 22:37:02 -08:00