ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
477 commits 391 branches 58 tags 304 MiB
Commit graph

477 commits

This branch
This branch
All branches
Author SHA1 Message Date
Daniel Han
507a490355 Update cross_entropy_loss.py 2024-09-25 00:00:51 -07:00
Daniel Han
e56ce736ff Update layernorm.py 2024-09-24 23:59:09 -07:00
Daniel Han
5bb6b3bbd2 Update cross_entropy_loss.py 2024-09-24 23:57:22 -07:00
Daniel Han
45554fb66d Update cross_entropy_loss.py 2024-09-24 23:55:25 -07:00
Daniel Han
6f39ef47c0 Update cross_entropy_loss.py 2024-09-24 23:54:14 -07:00
Daniel Han
7f74268e01 Update cross_entropy_loss.py 2024-09-24 23:52:54 -07:00
Daniel Han
85556bc385 Causal LM 2024-09-24 23:49:57 -07:00
Daniel Han
634489002d Update rms_layernorm.py 2024-09-24 23:13:51 -07:00
Daniel Han
fc4ca43ee1 RMS Layernorm 2024-09-24 22:50:33 -07:00
Daniel Han
d23fd17447 Update layernorm.py 2024-09-24 17:24:39 -07:00
Daniel Han
5103781c3f Patch layernorm 2024-09-24 17:22:20 -07:00
Daniel Han
9079e3c6e4 Update layernorm.py 2024-09-24 17:03:19 -07:00
Daniel Han
6801979ef3 Update layernorm.py 2024-09-24 17:00:23 -07:00
Daniel Han
4f7e68a3fe Update layernorm.py 2024-09-24 16:54:52 -07:00
Daniel Han
9b2c5d4814 Update layernorm.py 2024-09-24 16:45:50 -07:00
Daniel Han
55d41d72a0 Update layernorm.py 2024-09-24 16:44:33 -07:00
Daniel Han
873f245009 Update layernorm.py 2024-09-24 16:29:54 -07:00
Daniel Han
88269a26a8 Layernorm 2024-09-24 02:49:48 -07:00
Daniel Han
b41c182296 Update _utils.py 2024-09-23 10:56:55 -07:00
Daniel Han
9c26f9d3bb Update README.md 2024-09-23 01:36:50 -07:00
Daniel Han
45ca9501a4 Qwen 2.5 2024-09-23 01:27:12 -07:00
Daniel Han
6e387d8ff8 Update chat_templates.py 2024-09-23 01:07:06 -07:00
Daniel Han
388d5149a9 Upgrade Ollama presets 2024-09-23 01:02:24 -07:00
Daniel Han
a08812cc52 Update chat_templates.py 2024-09-23 00:29:21 -07:00
Daniel Han
1b8ef43c14 Update tokenizer_utils.py 2024-09-23 00:04:33 -07:00
Daniel Han
96fd381293 Update tokenizer_utils.py 2024-09-22 23:00:24 -07:00
Daniel Han
c1c37c49a6 Update tokenizer_utils.py 2024-09-22 22:48:35 -07:00
Daniel Han
7e2654ab7a Update mapper.py 2024-09-22 22:13:59 -07:00
Daniel Han
f216cdc289 Update _utils.py 2024-09-22 02:38:28 -07:00
Nazim Ali
0904f7395d fix: chat_templates.py bug (#1048) 
* fix: chat_template bug

* fix: check trainer attribute values are not None
 2024-09-22 01:18:37 -07:00
Daniel Han
cf3e072867 Update chat_templates.py 2024-09-21 01:56:17 -07:00
Daniel Han
c7c674472f Merge branch 'nightly' 2024-09-18 14:30:33 -07:00
Daniel Han
9fd82c95aa Update mapper.py 2024-09-18 14:23:22 -07:00
Daniel Han
1d4ae059c5 Update README.md (#1036) 2024-09-18 13:23:45 -07:00
Daniel Han
6d8b4c53a5 Update llama.py 2024-09-17 17:38:12 -07:00
Daniel Han
3289975025 Update mapper.py 2024-09-17 10:50:57 -07:00
Daniel Han
563432635a Update mapper.py 2024-09-15 21:50:00 -07:00
Daniel Han
89ada9ef0b Update _utils.py 2024-09-15 18:04:18 -07:00
Daniel Han
c5d7bb591d Update README.md (#1033) 2024-09-15 17:42:09 -07:00
Daniel Han
d27d992f58 Update utils.py 2024-09-08 19:47:21 -07:00
Daniel Han
0ad16b7c1f Update __init__.py 2024-09-08 15:51:27 -07:00
Daniel Han
ffb6aa905f Update README.md 2024-09-08 14:30:54 -07:00
Daniel Han
1bba6954f1 Update README.md 2024-09-08 12:29:31 -07:00
Daniel Han
74c2141bb3 Bug fixes (#1004) 
* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* update token retrieval logic (#952)

* Fix DPO (#947)

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* update hf token retrieval logic

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* Update llama.py

* get_token

* Update README.md

* Update gemma2.py

* Update rms_layernorm.py

* synchronize

* Update gemma2.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* layernorm

* Update rms_layernorm.py

* Update gemma2.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* revert

* Gemma

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update gemma2.py

* Change UnslothTrainingArguments base class to SFTConfig (#979)

* Cohere

* Update trainer.py

* Cohere

* Cohere

* New models

* Update llama.py

* Update llama.py

* Update cohere.py

* Update llama.py

* Update cohere.py

* retry

* Update fast_lora.py

* Update llama.py

* Update fast_lora.py

* Update llama.py

* Update llama.py

* Update cross_entropy_loss.py

* _apply_lora_mlp

* Update _utils.py

* Gemma fixes

* Update llama.py

* Update flex_attention.py

* Update llama.py

* layernorm

* Update llama.py

* Update llama.py

* Flex Attention

* Update gemma2.py

* Update __init__.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update chat_templates.py (#999)

fix all misspelled "unsued" to "unused"

* Update key from "from" to "user" (#1000)

When use [tokenizer.apply_chat_template](https://huggingface.co/docs/transformers/main/en/chat_templating), the key should be "role" rather than "from", this is liknk to [this issue](https://github.com/unslothai/unsloth/issues/994)

I don't know it is suitable for all situation, I also can add a dedicated parameter of the key if you think it is better.

* Update chat_templates.py

* Also patch the KTO trainer (#1001)

* flex attention

* Update llama.py

* Update flex_attention.py

* Update flex_attention.py

* Update _utils.py

* Update _utils.py

* Update flex_attention.py

* Update gemma2.py

* Update gemma2.py

---------

Co-authored-by: Hafedh <70411813+not-lain@users.noreply.github.com>
Co-authored-by: Tuan Pham <82665400+vTuanpham@users.noreply.github.com>
Co-authored-by: Yihao Wang <42559837+AgainstEntropy@users.noreply.github.com>
Co-authored-by: Peng <zphu1024@gmail.com>
Co-authored-by: Kyle Corbitt <kyle@openpipe.ai>
 2024-09-08 03:16:09 -07:00
Daniel Han
658e162032 Bug fixes 2024-09-04 00:28:53 -07:00
Daniel Han
a8490a2a8a Fix bug 2024-09-03 17:30:40 -07:00
Daniel Han
1b81cf1859 Gemma faster inference (#987) 
* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* update token retrieval logic (#952)

* Fix DPO (#947)

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* update hf token retrieval logic

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* Update llama.py

* get_token

* Update README.md

* Update gemma2.py

* Update rms_layernorm.py

* synchronize

* Update gemma2.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* layernorm

* Update rms_layernorm.py

* Update gemma2.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* revert

* Gemma

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update gemma2.py

* Change UnslothTrainingArguments base class to SFTConfig (#979)

* Cohere

* Update trainer.py

* Cohere

* Cohere

* New models

* Update llama.py

* Update llama.py

* Update cohere.py

* Update llama.py

* Update cohere.py

* retry

* Update fast_lora.py

* Update llama.py

* Update fast_lora.py

* Update llama.py

* Update llama.py

* Update cross_entropy_loss.py

* _apply_lora_mlp

* Update _utils.py

* Gemma fixes

* Update llama.py

* Update flex_attention.py

---------

Co-authored-by: Hafedh <70411813+not-lain@users.noreply.github.com>
Co-authored-by: Tuan Pham <82665400+vTuanpham@users.noreply.github.com>
 2024-09-03 13:52:12 -07:00
Daniel Han
7c3d1091ba Cohere, Bug fixes (#984) 
* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* update token retrieval logic (#952)

* Fix DPO (#947)

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* update hf token retrieval logic

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* Update llama.py

* get_token

* Update README.md

* Update gemma2.py

* Update rms_layernorm.py

* synchronize

* Update gemma2.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* layernorm

* Update rms_layernorm.py

* Update gemma2.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* revert

* Gemma

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update gemma2.py

* Change UnslothTrainingArguments base class to SFTConfig (#979)

* Cohere

* Update trainer.py

* Cohere

* Cohere

* New models

* Update llama.py

* Update llama.py

* Update cohere.py

* Update llama.py

* Update cohere.py

* retry

* Update fast_lora.py

* Update llama.py

* Update fast_lora.py

* Update llama.py

* Update llama.py

* Update cross_entropy_loss.py

* _apply_lora_mlp

* Update _utils.py

---------

Co-authored-by: Hafedh <70411813+not-lain@users.noreply.github.com>
Co-authored-by: Tuan Pham <82665400+vTuanpham@users.noreply.github.com>
 2024-09-03 01:52:32 -07:00
Daniel Han
b140367a87 Update save.py 2024-08-27 00:08:39 -07:00
Daniel Han
9f36b83db0 Update gemma2.py 2024-08-23 23:43:57 -07:00