ceos/unsloth
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
462 commits 391 branches 58 tags 304 MiB
Commit graph

462 commits

This branch
This branch
All branches
Author SHA1 Message Date
Daniel Han
0c07b760de Fix version 2024-09-25 12:35:38 -07:00
Daniel Han
f22a14801d Llama 3.2 2024-09-25 12:18:43 -07:00
Daniel Han
0a05da55f0 Llama 3.2 (#1058) 
* Layernorm

* Update layernorm.py

* Update layernorm.py

* Update layernorm.py

* Update layernorm.py

* Update layernorm.py

* Update layernorm.py

* Patch layernorm

* Update layernorm.py

* RMS Layernorm

* Update rms_layernorm.py

* Causal LM

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update layernorm.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update cross_entropy_loss.py

* Update _utils.py

* Update _utils.py

* Llama 3.2
 2024-09-25 11:48:24 -07:00
Daniel Han
b41c182296 Update _utils.py 2024-09-23 10:56:55 -07:00
Daniel Han
9c26f9d3bb Update README.md 2024-09-23 01:36:50 -07:00
Daniel Han
45ca9501a4 Qwen 2.5 2024-09-23 01:27:12 -07:00
Daniel Han
6e387d8ff8 Update chat_templates.py 2024-09-23 01:07:06 -07:00
Daniel Han
388d5149a9 Upgrade Ollama presets 2024-09-23 01:02:24 -07:00
Daniel Han
a08812cc52 Update chat_templates.py 2024-09-23 00:29:21 -07:00
Daniel Han
1b8ef43c14 Update tokenizer_utils.py 2024-09-23 00:04:33 -07:00
Daniel Han
96fd381293 Update tokenizer_utils.py 2024-09-22 23:00:24 -07:00
Daniel Han
c1c37c49a6 Update tokenizer_utils.py 2024-09-22 22:48:35 -07:00
Daniel Han
7e2654ab7a Update mapper.py 2024-09-22 22:13:59 -07:00
Daniel Han
f216cdc289 Update _utils.py 2024-09-22 02:38:28 -07:00
Nazim Ali
0904f7395d fix: chat_templates.py bug (#1048) 
* fix: chat_template bug

* fix: check trainer attribute values are not None
 2024-09-22 01:18:37 -07:00
Daniel Han
cf3e072867 Update chat_templates.py 2024-09-21 01:56:17 -07:00
Daniel Han
c7c674472f Merge branch 'nightly' 2024-09-18 14:30:33 -07:00
Daniel Han
9fd82c95aa Update mapper.py 2024-09-18 14:23:22 -07:00
Daniel Han
1d4ae059c5 Update README.md (#1036) 2024-09-18 13:23:45 -07:00
Daniel Han
6d8b4c53a5 Update llama.py 2024-09-17 17:38:12 -07:00
Daniel Han
3289975025 Update mapper.py 2024-09-17 10:50:57 -07:00
Daniel Han
563432635a Update mapper.py 2024-09-15 21:50:00 -07:00
Daniel Han
89ada9ef0b Update _utils.py 2024-09-15 18:04:18 -07:00
Daniel Han
c5d7bb591d Update README.md (#1033) 2024-09-15 17:42:09 -07:00
Daniel Han
d27d992f58 Update utils.py 2024-09-08 19:47:21 -07:00
Daniel Han
0ad16b7c1f Update __init__.py 2024-09-08 15:51:27 -07:00
Daniel Han
ffb6aa905f Update README.md 2024-09-08 14:30:54 -07:00
Daniel Han
1bba6954f1 Update README.md 2024-09-08 12:29:31 -07:00
Daniel Han
74c2141bb3 Bug fixes (#1004) 
* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* update token retrieval logic (#952)

* Fix DPO (#947)

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* update hf token retrieval logic

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* Update llama.py

* get_token

* Update README.md

* Update gemma2.py

* Update rms_layernorm.py

* synchronize

* Update gemma2.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* layernorm

* Update rms_layernorm.py

* Update gemma2.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* revert

* Gemma

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update gemma2.py

* Change UnslothTrainingArguments base class to SFTConfig (#979)

* Cohere

* Update trainer.py

* Cohere

* Cohere

* New models

* Update llama.py

* Update llama.py

* Update cohere.py

* Update llama.py

* Update cohere.py

* retry

* Update fast_lora.py

* Update llama.py

* Update fast_lora.py

* Update llama.py

* Update llama.py

* Update cross_entropy_loss.py

* _apply_lora_mlp

* Update _utils.py

* Gemma fixes

* Update llama.py

* Update flex_attention.py

* Update llama.py

* layernorm

* Update llama.py

* Update llama.py

* Flex Attention

* Update gemma2.py

* Update __init__.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update flex_attention.py

* Update chat_templates.py (#999)

fix all misspelled "unsued" to "unused"

* Update key from "from" to "user" (#1000)

When use [tokenizer.apply_chat_template](https://huggingface.co/docs/transformers/main/en/chat_templating), the key should be "role" rather than "from", this is liknk to [this issue](https://github.com/unslothai/unsloth/issues/994)

I don't know it is suitable for all situation, I also can add a dedicated parameter of the key if you think it is better.

* Update chat_templates.py

* Also patch the KTO trainer (#1001)

* flex attention

* Update llama.py

* Update flex_attention.py

* Update flex_attention.py

* Update _utils.py

* Update _utils.py

* Update flex_attention.py

* Update gemma2.py

* Update gemma2.py

---------

Co-authored-by: Hafedh <70411813+not-lain@users.noreply.github.com>
Co-authored-by: Tuan Pham <82665400+vTuanpham@users.noreply.github.com>
Co-authored-by: Yihao Wang <42559837+AgainstEntropy@users.noreply.github.com>
Co-authored-by: Peng <zphu1024@gmail.com>
Co-authored-by: Kyle Corbitt <kyle@openpipe.ai>
 2024-09-08 03:16:09 -07:00
Daniel Han
658e162032 Bug fixes 2024-09-04 00:28:53 -07:00
Daniel Han
a8490a2a8a Fix bug 2024-09-03 17:30:40 -07:00
Daniel Han
1b81cf1859 Gemma faster inference (#987) 
* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* update token retrieval logic (#952)

* Fix DPO (#947)

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* update hf token retrieval logic

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* Update llama.py

* get_token

* Update README.md

* Update gemma2.py

* Update rms_layernorm.py

* synchronize

* Update gemma2.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* layernorm

* Update rms_layernorm.py

* Update gemma2.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* revert

* Gemma

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update gemma2.py

* Change UnslothTrainingArguments base class to SFTConfig (#979)

* Cohere

* Update trainer.py

* Cohere

* Cohere

* New models

* Update llama.py

* Update llama.py

* Update cohere.py

* Update llama.py

* Update cohere.py

* retry

* Update fast_lora.py

* Update llama.py

* Update fast_lora.py

* Update llama.py

* Update llama.py

* Update cross_entropy_loss.py

* _apply_lora_mlp

* Update _utils.py

* Gemma fixes

* Update llama.py

* Update flex_attention.py

---------

Co-authored-by: Hafedh <70411813+not-lain@users.noreply.github.com>
Co-authored-by: Tuan Pham <82665400+vTuanpham@users.noreply.github.com>
 2024-09-03 13:52:12 -07:00
Daniel Han
7c3d1091ba Cohere, Bug fixes (#984) 
* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* update token retrieval logic (#952)

* Fix DPO (#947)

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* update hf token retrieval logic

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* Update llama.py

* get_token

* Update README.md

* Update gemma2.py

* Update rms_layernorm.py

* synchronize

* Update gemma2.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* layernorm

* Update rms_layernorm.py

* Update gemma2.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* revert

* Gemma

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update rms_layernorm.py

* Update gemma2.py

* Change UnslothTrainingArguments base class to SFTConfig (#979)

* Cohere

* Update trainer.py

* Cohere

* Cohere

* New models

* Update llama.py

* Update llama.py

* Update cohere.py

* Update llama.py

* Update cohere.py

* retry

* Update fast_lora.py

* Update llama.py

* Update fast_lora.py

* Update llama.py

* Update llama.py

* Update cross_entropy_loss.py

* _apply_lora_mlp

* Update _utils.py

---------

Co-authored-by: Hafedh <70411813+not-lain@users.noreply.github.com>
Co-authored-by: Tuan Pham <82665400+vTuanpham@users.noreply.github.com>
 2024-09-03 01:52:32 -07:00
Daniel Han
b140367a87 Update save.py 2024-08-27 00:08:39 -07:00
Daniel Han
9f36b83db0 Update gemma2.py 2024-08-23 23:43:57 -07:00
Daniel Han
353991f14a Phi 3.5 bug fix (#955) 
* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* update token retrieval logic (#952)

* Fix DPO (#947)

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* update hf token retrieval logic

---------

Co-authored-by: Daniel Han <danielhanchen@gmail.com>

* Update llama.py

* get_token

* Update README.md

---------

Co-authored-by: Hafedh <70411813+not-lain@users.noreply.github.com>
 2024-08-23 17:38:24 -07:00
Daniel Han
199766c644 Fix DPO (#947) 
* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py
 2024-08-22 02:18:03 -07:00
Daniel Han
cadff4f883 Update README.md (#941) 
Co-authored-by: Michael <107991372+shimmyshimmer@users.noreply.github.com>
 2024-08-20 17:59:50 -07:00
Daniel Han
8e61906e6f Update chat_templates.py 2024-08-20 16:54:11 -07:00
Daniel Han
fb60340a90 Phi 3.5 (#940) 
* LongRoPE

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update _utils.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update llama.py

* Update mapper.py

* Phi 3.5
 2024-08-20 16:51:39 -07:00
Daniel Han
0927c34392 Update README.md (#938) 2024-08-19 17:18:30 -07:00
Daniel Han
861f232047 Merge branch 'main' into nightly 2024-08-19 17:13:12 -07:00
Daniel Han
92a3aec9e9 Update _auto_install.py 2024-08-19 17:12:46 -07:00
Daniel Han
4450110756 Create _auto_install.py 2024-08-19 17:12:32 -07:00
Daniel Han
bb9539cc82 Fix NEFTune (#937) 
* untrained tokens llama 3.1 base

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Bug fixes

* Update llama.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update tokenizer_utils.py

* Update llama.py

* Update llama.py

* Update llama.py
 2024-08-19 16:17:52 -07:00
Daniel Han
3b4ce17bc9 Merge branch 'main' into nightly 2024-08-19 16:17:00 -07:00
Daniel Han
91bdf27729 Update llama.py 2024-08-19 16:14:01 -07:00
Daniel Han
ce1863d91f Update llama.py 2024-08-19 16:11:09 -07:00
Daniel Han
dcf7e6e952 Update llama.py 2024-08-19 16:08:14 -07:00
Daniel Han
4f51fe0a8c Update tokenizer_utils.py 2024-08-19 16:03:24 -07:00