ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
470 commits 391 branches 58 tags 304 MiB
Commit graph

470 commits

This branch
This branch
All branches
Author SHA1 Message Date
Daniel Han
b417295e95 Update README.md 2024-09-26 00:05:38 -07:00
Daniel Han
b6b640a67f Update README.md 2024-09-26 00:02:15 -07:00
Daniel Han
412963a078 Update pyproject.toml 2024-09-25 23:47:15 -07:00
Daniel Han
fb0ee5fc0c Update pyproject.toml 2024-09-25 23:13:49 -07:00
Daniel Han
703699134e Remove version checks 2024-09-25 23:00:09 -07:00
Daniel Han
83782024a0 Update _utils.py 2024-09-25 22:56:41 -07:00
Daniel Han
0472fba0a6 Update llama.py 2024-09-25 22:12:21 -07:00
Daniel Han
5c7cb05923 Update llama.py 2024-09-25 12:42:24 -07:00
Daniel Han
96e92c9282 Fix version 2024-09-25 12:35:38 -07:00
Daniel Han
063a2b42b0 Llama 3.2 2024-09-25 12:18:43 -07:00
Daniel Han
379d8101a5 Llama 3.2 (#1058) 
* Layernorm

* Update layernorm.py

* Update layernorm.py

* Update layernorm.py

* Update layernorm.py

* Update layernorm.py

* Update layernorm.py

* Patch layernorm

* Update layernorm.py

* RMS Layernorm

* Update rms_layernorm.py

* Causal LM

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update layernorm.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update _utils.py

* Update _utils.py

* Llama 3.2
 2024-09-25 11:48:24 -07:00
Daniel Han
45a7984b94 Update _utils.py September-2024 2024-09-23 10:56:55 -07:00
Daniel Han
6ed2461bd9 Update README.md 2024-09-23 01:36:50 -07:00
Daniel Han
f48338601c Qwen 2.5 2024-09-23 01:27:12 -07:00
Daniel Han
f6505a1a48 Update chat_templates.py 2024-09-23 01:07:06 -07:00
Daniel Han
53e7776acf Upgrade Ollama presets 2024-09-23 01:02:24 -07:00
Daniel Han
05004997ec Update chat_templates.py 2024-09-23 00:29:21 -07:00
Daniel Han
05561c52f4 Update tokenizer_utils.py 2024-09-23 00:04:33 -07:00
Daniel Han
2a0727fc8f Update tokenizer_utils.py 2024-09-22 23:00:24 -07:00
Daniel Han
4795231ed3 Update tokenizer_utils.py 2024-09-22 22:48:35 -07:00
Daniel Han
f565b8e556 Update mapper.py 2024-09-22 22:13:59 -07:00
Daniel Han
92d71cd292 Update _utils.py 2024-09-22 02:38:28 -07:00
Nazim Ali
80fbb1a3fe fix: chat_templates.py bug (#1048) 
* fix: chat_template bug

* fix: check trainer attribute values are not None
 2024-09-22 01:18:37 -07:00
Daniel Han
4e9c09556c Update chat_templates.py 2024-09-21 01:56:17 -07:00
Daniel Han
b5f534274f Merge branch 'nightly' 2024-09-18 14:30:33 -07:00
Daniel Han
e40a622085 Update mapper.py 2024-09-18 14:23:22 -07:00
Daniel Han
e675a8ae10 Update README.md (#1036) 2024-09-18 13:23:45 -07:00
Daniel Han
8ba439928a Update llama.py 2024-09-17 17:38:12 -07:00
Daniel Han
0cb5ec84de Update mapper.py 2024-09-17 10:50:57 -07:00
Daniel Han
cecc6caaf3 Update mapper.py 2024-09-15 21:50:00 -07:00
Daniel Han
3d8635cba6 Update _utils.py 2024-09-15 18:04:18 -07:00
Daniel Han
ae566032d1 Update README.md (#1033) 2024-09-15 17:42:09 -07:00
Daniel Han
6f5f82647d Update utils.py 2024-09-08 19:47:21 -07:00
Daniel Han
0879404c63 Update __init__.py 2024-09-08 15:51:27 -07:00
Daniel Han
df132d13e6 Update README.md 2024-09-08 14:30:54 -07:00
Daniel Han
f02315e50e Update README.md 2024-09-08 12:29:31 -07:00
Daniel Han
1aaa624b4e Bug fixes (#1004) 
* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* update token retrieval logic (#952)

* Fix DPO (#947)

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* update hf token retrieval logic

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* Update llama.py

* get_token

* Update README.md

* Update gemma2.py

* Update rms_layernorm.py

* synchronize

* Update gemma2.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* layernorm

* Update rms_layernorm.py

* Update gemma2.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* revert

* Gemma

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update gemma2.py

* Change UnslothTrainingArguments base class to SFTConfig (#979)

* Cohere

* Update trainer.py

* Cohere

* Cohere

* New models

* Update llama.py

* Update llama.py

* Update cohere.py

* Update llama.py

* Update cohere.py

* retry

* Update fast_lora.py

* Update llama.py

* Update fast_lora.py

* Update llama.py

* Update llama.py

* Update cross_entropy_loss.py

* _apply_lora_mlp

* Update _utils.py

* Gemma fixes

* Update llama.py

* Update flex_attention.py

* Update llama.py

* layernorm

* Update llama.py

* Update llama.py

* Flex Attention

* Update gemma2.py

* Update __init__.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update chat_templates.py (#999)

fix all misspelled "unsued" to "unused"

* Update key from "from" to "user" (#1000)

When use [tokenizer.apply_chat_template](https://huggingface.co/docs/transformers/main/en/chat_templating), the key should be "role" rather than "from", this is liknk to [this issue](https://github.com/unslothai/unsloth/issues/994)

I don't know it is suitable for all situation, I also can add a dedicated parameter of the key if you think it is better.

* Update chat_templates.py

* Also patch the KTO trainer (#1001)

* flex attention

* Update llama.py

* Update flex_attention.py

* Update flex_attention.py

* Update _utils.py

* Update _utils.py

* Update flex_attention.py

* Update gemma2.py

* Update gemma2.py

---------

Co-authored-by: Hafedh <70411813+not-lain@users.noreply.github.com>
Co-authored-by: Tuan Pham <82665400+vTuanpham@users.noreply.github.com>
Co-authored-by: Yihao Wang <42559837+AgainstEntropy@users.noreply.github.com>
Co-authored-by: Peng <zphu1024@gmail.com>
Co-authored-by: Kyle Corbitt <kyle@openpipe.ai>
 2024-09-08 03:16:09 -07:00
Daniel Han
e4ab7a7378 Bug fixes 2024-09-04 00:28:53 -07:00
Daniel Han
be310cac1e Fix bug 2024-09-03 17:30:40 -07:00
Daniel Han
7a531b7c65 Gemma faster inference (#987) 
* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* update token retrieval logic (#952)

* Fix DPO (#947)

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* update hf token retrieval logic

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* Update llama.py

* get_token

* Update README.md

* Update gemma2.py

* Update rms_layernorm.py

* synchronize

* Update gemma2.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* layernorm

* Update rms_layernorm.py

* Update gemma2.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* revert

* Gemma

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update gemma2.py

* Change UnslothTrainingArguments base class to SFTConfig (#979)

* Cohere

* Update trainer.py

* Cohere

* Cohere

* New models

* Update llama.py

* Update llama.py

* Update cohere.py

* Update llama.py

* Update cohere.py

* retry

* Update fast_lora.py

* Update llama.py

* Update fast_lora.py

* Update llama.py

* Update llama.py

* Update cross_entropy_loss.py

* _apply_lora_mlp

* Update _utils.py

* Gemma fixes

* Update llama.py

* Update flex_attention.py

---------

Co-authored-by: Hafedh <70411813+not-lain@users.noreply.github.com>
Co-authored-by: Tuan Pham <82665400+vTuanpham@users.noreply.github.com>
 2024-09-03 13:52:12 -07:00
Daniel Han
492f466c29 Cohere, Bug fixes (#984) 
* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* update token retrieval logic (#952)

* Fix DPO (#947)

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* update hf token retrieval logic

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* Update llama.py

* get_token

* Update README.md

* Update gemma2.py

* Update rms_layernorm.py

* synchronize

* Update gemma2.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* layernorm

* Update rms_layernorm.py

* Update gemma2.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* revert

* Gemma

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update gemma2.py

* Change UnslothTrainingArguments base class to SFTConfig (#979)

* Cohere

* Update trainer.py

* Cohere

* Cohere

* New models

* Update llama.py

* Update llama.py

* Update cohere.py

* Update llama.py

* Update cohere.py

* retry

* Update fast_lora.py

* Update llama.py

* Update fast_lora.py

* Update llama.py

* Update llama.py

* Update cross_entropy_loss.py

* _apply_lora_mlp

* Update _utils.py

---------

Co-authored-by: Hafedh <70411813+not-lain@users.noreply.github.com>
Co-authored-by: Tuan Pham <82665400+vTuanpham@users.noreply.github.com>
 2024-09-03 01:52:32 -07:00
Daniel Han
f43eb7608b Update save.py 2024-08-27 00:08:39 -07:00
Daniel Han
e16fcd7d94 Update gemma2.py 2024-08-23 23:43:57 -07:00
Daniel Han
bc3690a755 Phi 3.5 bug fix (#955) 
* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* update token retrieval logic (#952)

* Fix DPO (#947)

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* update hf token retrieval logic

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* Update llama.py

* get_token

* Update README.md

---------

Co-authored-by: Hafedh <70411813+not-lain@users.noreply.github.com>
 2024-08-23 17:38:24 -07:00
Daniel Han
d2e363d222 Fix DPO (#947) 
* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py
 2024-08-22 02:18:03 -07:00
Daniel Han
8665d45518 Update README.md (#941) 
Co-authored-by: Michael <107991372+shimmyshimmer@users.noreply.github.com>
 2024-08-20 17:59:50 -07:00
Daniel Han
1c64e2645f Update chat_templates.py 2024-08-20 16:54:11 -07:00
Daniel Han
a93e3dea2a Phi 3.5 (#940) 
* LongRoPE

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update mapper.py

* Phi 3.5
 2024-08-20 16:51:39 -07:00
Daniel Han
0ea6e6e4fd Update README.md (#938) 2024-08-19 17:18:30 -07:00
Daniel Han
1fc020b936 Merge branch 'main' into nightly 2024-08-19 17:13:12 -07:00